If you are hosting your website from a web hosting company, you might want to check out with them to see if there are any issues at their end, or you may want to upgrade to a better plan or service.
Now if you are hosting from your own computer at home, then the slowness is your internet connection. Generally your upload speed is a lot slower than your download speed.
I run virtual servers at home and I can tell you, the speeds aren't that great. I get an average of 60Kbps to 100Kbps upwards which is OK for what I do. But for hosting a website like Youtube forget it.
The users will get very frustrated because its taking a very long time for the videos to load.
Hope this helps!
Regards,
Infiltrator