multiple connections to download 1 file

[tim.vangehugten]

I was wondering how download accelerators,torrent files, or any other thing split up a file into pieces to download it.

Does anyone know how to split a file like that the you can send the "little parts" to another pc but in such way that the whole file remains. Is there a way to program it? And which language would be the easiest way?

[digip]

I was wondering how download accelerators,torrent files, or any other thing split up a file into pieces to download it.

Does anyone know how to split a file like that the you can send the "little parts" to another pc but in such way that the whole file remains. Is there a way to program it? And which language would be the easiest way?

Download accelerators and torrent files are two different things. Most browsers these days can mimic a download accelerator anyway, just change how many connections to a server, and it speeds up the download process, make too many connections, and you will probably time out, or get banned for too many requests at once and may look lilke a DoS attack. Download accelerators basically send multiple requests for a file at the same time, and get different parts of the files all at the same time. Some of them have connections to web proxies that query several of them at once to pull mirrored images of a file from several locations, and this also speeds up the process. This is similar to a Torrent.

Torrents use one central tracker everyone shares that has a list of all the other users with the same file. Once you download the torrent, your torrent client(Like uTorrent, etc) queries the tracker for updates on who has the file you want. Once it has the list, it reads the hashes and makes requests for different parts of the file from multiple file sharers, and then proceeds to merge these all into one file. It can be better explained on the wiki: http://en.wikipedia.org/wiki/.torrent

http://en.wikipedia.org/wiki/Internet_Download_Accelerator

DAP: http://en.wikipedia.org/wiki/Download_Accelerator_Plus

http://en.wikipedia.org/wiki/List_of_download_managers

To split a file for uploading is something different, but if you want to break a file up, you could use something like winrar or other zip programs to slit a large file into multiple smaller parts and then distribute peices where you want. You would then only be able to unzip it if you had all the smaller parts together again in one location.

Please explain more on what you mean by "Does anyone know how to split a file like that the you can send the "little parts" to another pc but in such way that the whole file remains."

[tim.vangehugten]

Your explanation helped, thanks for that:) But I indeed ment how you can program a program so that it download another part of the file. Does it search for a certain string to jump to and start downloading from that point or how does it works? Cause that's not explained in the URLs you posted.

[digip]

Your explanation helped, thanks for that:) But I indeed ment how you can program a program so that it download another part of the file. Does it search for a certain string to jump to and start downloading from that point or how does it works? Cause that's not explained in the URLs you posted.

I beleive the look at the whol file and break it into hashed parts, keeping track of which hashed segments it has and which ones it needs to download. Say a file fails to download, and you want to get it again without redownloding the whole thing, it checks to see which hashed parts it has and then goes after the other parts it still needs. How they do this, I really don't know, but a little google-fu will probably turn up all kinds of answers.

I never tried to code a file downloader before, so not sure how that works, but I imagine they do several things. They probably query the server holding the file for the file size, type, creation date, etc. Then it keeps those statistics until it finishes the download. If it is incomplete, it rechecks to see if the file is any newer, in which case it would redownload the entire file, if not and it passes something like an md5 check or some other verification mechinism(not sure how though, since it doesn't have the entire file to look at), then it looks for the hashed peices of the file it still needs and resumes the download.