Sometimes, on directories containing a mix of big and small files, rsync can suddenly slows down on big files. It can slow down up to 15kB/s, even on a LAN! It generally happens when --ignore-times is used. Thank you.
Release news
- 2023-05-30 Nagwin 5.2.0
- 2023-05-23 Copssh server 7.13.2
- 2023-05-23 Copssh server 8.1.2
- 2023-03-29 Rsync Client Helper GUI 1.0.3.7
- 2023-03-29 Nagwin 5.1.2
Actually this may happen as rsync's quick check mechanism is turned off by --ignore-times option:
By specifying that option, you tell rsync to use rsync algorithm all the time whenever rsync daemon is involved. That may result in extra work as each updated file must be checked by block by block.
I agree with you, however I'm using "--ignore-times" on purpose. I have to check all files block by block. It's a transfer of SQL Server database files, so the timestamps are not updated everytime... The problem is the following: how do you explain a speed of 15kB/s on a 10GB file, over a 8Mbit/s WAN and even over a Gigabit LAN?
Well, that explains the problem: An SQL Server manages contents of the database files itself. It may move chunks of data from one position to another as a result of garbage collection or disk optimization for example. That may result in a completely different image in comparison to a previous one, causing rsync to use excessive amount of time for block-to-block sync.
I don't think this is the problem, because this generally happens on the SQL LOG file, which is very sequential. Moreover, when this slow down occurs, the speed is very stable (between 15kB/s and 20kB/s), until the end of the file... So this is a big problem. I'll try to make this problem reproductible, making a copy of both versions of files (local and remote) when this will occur again. Thank you.
When you mention a 10GB file, I was thinking of database files themselves :-) I have informed you about the possible scenarios. I am not sure what else I can contribute here. If those slowdowns happens randomly, it is even more difficult for me to pinpoint a potential problem. It could be anything. You can try to run rsync with three -v options to see what happens.