I've been running some tests on files created by rsync and noticing big fragmentation issues. I started the testing because our 5TB array started performing very slowly and it appears fragmentation was the culprit. The test I conducted was straighforward:
1. Copy over a 49GB file. Analyzed with contig (from sysinternals), no fragments.
2. Ran rsync and the file was recreated normally (rsync options were -ruvi --times --no-blocking-io --compress-level=9 --bwlimit=0 --stats --delete --modify-window=2)
3. Resulting file had almost 12000 fragments.
It doesn't really surprise me since rsync grabs blocks as it needs them for the new file. I was wondering why rsync doesn't preallocate the space like copy or other utilites do. This would insure you have the space to create the new file.
Just wondering...any suggestions? By the way, your site has been a huge help, thanks for the work!
An interesting observation! I am sure that sending this post to Rsync users mailing list is a good idea, since that behaviour is probably design-related.
I posted this to the rsync mailing list. Thanks!
This subject is also being discussed in the cygwin users' list.
Based on discussions on the rsync board I'm building a new version of the cygwin1.dll with support for the posix_fallocate function. I've also received a patch for rsync which will preallocate the space for the new file on the drive. Once I complete all the builds I'll report the results back via the rsync message boards. It appears that preallocation is not in the current version. Hopefully we'll get it in!
Rob
I was able to build a cygwin release from the 083006 snapshot and successfully applied the posix_fallocate function patch to the receiver.c code. The resulting build of rsync did not have any fragmentation. The feature would not work with the --inplace option but if that option is excluded the file fragmentation can be eliminated with rsync and cygwin.
Thanks for your contributions, Rob !