Dаrren Kitchen Posted February 16, 2007 Share Posted February 16, 2007 Anyone know any system tweaks to speed up slow USB Thumbdrives? I've noticed that it's faster to copy one 300MB file than three-hundred 1mb files, is this because of extra edits that have to be made to the file allocation table? Quote Link to comment Share on other sites More sharing options...
xxAtimiskxx Posted February 17, 2007 Share Posted February 17, 2007 USB 2.0 Quote Link to comment Share on other sites More sharing options...
Guest Twilight Zone Posted February 17, 2007 Share Posted February 17, 2007 Wow...Now you open his eyes. Quote Link to comment Share on other sites More sharing options...
Dаrren Kitchen Posted February 17, 2007 Author Share Posted February 17, 2007 Maybe I should have stated this in my first post, it is already USB 2 and so is the port. Quote Link to comment Share on other sites More sharing options...
Guest TehHacks Posted February 17, 2007 Share Posted February 17, 2007 I've noticed that it's faster to copy one 300MB file than three-hundred 1mb files, is this because of extra edits that have to be made to the file allocation table? I have also noticed this, i usually compress the files to .tar.gz or .rar to get passed this problem. it usually works a lot quicker when all the files are nice and neatly bundled in to an archive. Quote Link to comment Share on other sites More sharing options...
Dаrren Kitchen Posted February 18, 2007 Author Share Posted February 18, 2007 A few more findings: 1. I thought that my Symantec AV may be the problem, but after removal it made no difference. 2. Read and writes are much faster under Linux than under Vista/XP. 3. It's faster in Vista to mount a Trucrypt volume off the thumbdrive, then run your apps off of it then it is to run the app straight off of the thumbdrive. 4. Changing the device options in control panel from the default "Optimize for removal" to "Optimize for performance". Quote Link to comment Share on other sites More sharing options...
theSmiler Posted February 18, 2007 Share Posted February 18, 2007 The same issue is on all computers... Quote Link to comment Share on other sites More sharing options...
Deveant Posted February 18, 2007 Share Posted February 18, 2007 A few more findings:4. Changing the device options in control panel from the default "Optimize for removal" to "Optimize for performance". in this case always use safty remove, otherwise corruption will occur. But yea this happens on all machiens, even with plan HDD's try copying 1000 files (ie counter-strike: Source) from one HDD to another, then zip it and move it back, theres about a 5min difference. Quote Link to comment Share on other sites More sharing options...
W4RP3D Posted February 18, 2007 Share Posted February 18, 2007 just out of interest. why is this?[/code] Quote Link to comment Share on other sites More sharing options...
Deveant Posted February 18, 2007 Share Posted February 18, 2007 y is it slow? ive been told its todo with file indexing and such, XP needs to close, and then re-start the file coping procedure every time u copy a new file. When there is a few u dont notice it but when there is alot, XP needs to make sure for each file that: the path exists, its writable, there isnt already another file with the same name, the path its coming from exists, the file is allowed to be copied (not so much in the order). Then when finished copying just that file, it checks to see: of the file is the copied, if its the same size, if its the same file (checks the first and last few lines of binary ive been told, but not sure on this), and then checks to see if all the file rights are the same. for a normal file this takes about .3 seconds, though when working with say 200 files, there u have already added 1minute to ur overall copy time. Were as if u used an archive, it would only use this process once, therefore speeding the proccess up a shit load. Quote Link to comment Share on other sites More sharing options...
Dаrren Kitchen Posted February 19, 2007 Author Share Posted February 19, 2007 Ok, even more interesting test results. I used a tool from http://oette.info/ for benchmarking different files systems on a 512MB Thumbdrive. I set it to write five 1 MB files. Here are the results: Fat16 16kb allocation unit size Result: Write time: 3.4228 Sec. 0.292158466752366 MB/s Read time: 0.0862 Sec. 11.6009280742459 MB/s Fat16 64kb allocation unit size Result: Write time: 3.3486 Sec. 0.298632264229827 MB/s Read time: 0.0842 Sec. 11.8764845605701 MB/s Fat16 default allocation unit size Result: Write time: 3.3108 Sec. 0.302041802585478 MB/s Read time: 0.0862 Sec. 11.6009280742459 MB/s Result: Write time: 3.3208 Sec. 0.301132257287401 MB/s Read time: 0.0762 Sec. 13.1233595800525 MB/s Fat32 default allocation unit size Result: Write time: 23.81 Sec. 4.19991600167997E-02 MB/s Read time: 0.08 Sec. 12.5 MB/s Result: Write time: 23.7444 Sec. 4.21151934771988E-02 MB/s Read time: 0.1202 Sec. 8.31946755407654 MB/s NTFS default allocation unit size Result: Write time: 0.8712 Sec. 1.14784205693297 MB/s Read time: 0.0682 Sec. 14.6627565982405 MB/s Result: Write time: 0.9174 Sec. 1.09003706126008 MB/s Read time: 0.0882 Sec. 11.3378684807256 MB/s To cut to the chase, NTFS seems to be the fastest, FAT16 (just called FAT in the dialog) next and FAT32 was dog ass slow. I used the HP UFD format tool to do the formating. Any good reason not to use NTFS on the thumbdrive besides compatibility with other OSes? Quote Link to comment Share on other sites More sharing options...
Sparda Posted February 19, 2007 Share Posted February 19, 2007 I'd be very suspicious of those NTFS results. When formatting a removable drive to NTFS you have to enable 'performance' mode for that particular device. This means windows caches stuff you write to it on the HD before you actually write it. Flash write speeds are limited primarily by the hardware. With a single chip of flash you will have a average write speed of about 900KB/s (or some thing like that), slow right? Well, to improve performance what USB flash manufacturers do is they have multiple flash chips on the same board and (effectively) RAID 0 them together to improve performance. Using this same technique they can produce bigger (both physically and storage space) devices using the same hardware they currently have. Quote Link to comment Share on other sites More sharing options...
Dаrren Kitchen Posted February 19, 2007 Author Share Posted February 19, 2007 The idea that it could be cached came to mind, but I'd still like some other folks to do some testing as well. Anybody else get results like this? Quote Link to comment Share on other sites More sharing options...
Sparda Posted February 19, 2007 Share Posted February 19, 2007 The idea that it could be cached came to mind, but I'd still like some other folks to do some testing as well. Anybody else get results like this? I think the ultimate test of performance is to: dd if=/dev/urandom of=/dev/sda Quote Link to comment Share on other sites More sharing options...
Dаrren Kitchen Posted February 19, 2007 Author Share Posted February 19, 2007 Damn, the $GB UFD is much slower than the 512MB: FAT16 on the 4GB Result: Write time: 24.2908 Sec. 4.11678495562106E-02 MB/s Read time: 0.1502 Sec. 6.6577896138482 MB/s Quote Link to comment Share on other sites More sharing options...
Dаrren Kitchen Posted February 19, 2007 Author Share Posted February 19, 2007 I did some more test, this time with my 1GB Apacer UFD. I copied Torpark 1.5.0.7 (27 MB in 275 files, 83 directories) to the UFD as a more organic test: Fat32 1:05 fat16 0:55 NTFS 1:35 1:23 These new, longer results gave me pause, thats why I ran it twice. I also test NTFS under the following conditions: No AV live protection 1:11 AV on, using XCOPY 1:38 Compression on 1:17 Seems like turning compression on (less total data to write to the disk) and AV off seems to help. Over all, seems like FAT 16 is the best choice for faster thumbdrive file systems. Edit: After testing some of my other drives: 1GB XP FAT16 Result: Write time: 11.0596 Sec. 9.04191833339361E-02 MB/s Read time: 0.1406 Sec. 7.11237553342816 MB/s Result: Write time: 10.5154 Sec. 0.095098617266105 MB/s Read time: 0.1376 Sec. 7.26744186046512 MB/s 2GB FAT16 Result: Write time: 3.528 Sec. 0.283446712018141 MB/s Read time: 0.1282 Sec. 7.8003120124805 MB/s 4GB FAT16 Result: Write time: 27.4376 Sec. 3.64463364142636E-02 MB/s Read time: 0.1 Sec. 10 MB/s Result: Write time: 23.7406 Sec. 4.21219345762112E-02 MB/s Read time: 0.1094 Sec. 9.14076782449726 MB/s 1GB XP FAT32 Result: Write time: 11.4752 Sec. 8.71444506413832E-02 MB/s Read time: 0.1124 Sec. 8.89679715302491 MB/s Result: Write time: 11.6156 Sec. 8.60911188401804E-02 MB/s Read time: 0.122 Sec. 8.19672131147541 MB/s 1GB XP NTFS Result: Write time: 5.4874 Sec. 0.182235667164777 MB/s Read time: 0.122 Sec. 8.19672131147541 MB/s Result: Write time: 5.6312 Sec. 0.177582042903822 MB/s Read time: 0.1156 Sec. 8.65051903114187 MB/s I did the Tor test again and with the 1GB and it took 1:24, so something must be wrong with my synthetic tests. Quote Link to comment Share on other sites More sharing options...
Dаrren Kitchen Posted February 19, 2007 Author Share Posted February 19, 2007 Looks like these guys did some tests, and over all FAT16 is the winner: http://www.anandtech.com/printarticle.aspx?i=2549 Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.