A great way to get a bunch of components all at once is to get an starter kit. I recommend the Arduino Starter Kit by Arduino.
In the UK, Maplin is increasing stocking Arduino products and continues to sell electronic components, both online and on the high street.
Online you find parts from a large number of suppliers. Here are just a few: