There could be a lot of things, affecting your internet speed.
1. Telephone line
2. Modem (update the firmware or buy a new modem)
3. Faulty Filter Line
4. The computer itself, (try optimizing your TCP Stack)
5. Defrag your compute, if you are using Windows
6. Check if you have sufficient ram
7. Get your ISP to run a test on the line, to determine if there are any issues
8. Try upgrading your internet plan to see if that makes a difference
9. Do you share your internet connection with anyone else (might want to check P2P software is running)
Watch this Hak5 episode, Snubs suggests a tool that can pinpoint root cause of your slow internet speed.
http://www.hak5.org/episodes/episode-726