hang while syncing folders with many files

11 posts / 0 new
Last post
itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
hang while syncing folders with many files

we have starting backup up some directories to a remote server using rsync and the -e ssh option. it is mostly working fine.
but there appears to be an issue when it tries to sync a directory with lots of folders or files. when the source folder has a lot of files, it will hang somewhere on a line like this:
recv_file_name(archive/CDW3NEW/_vti_cnf)

what is it doing at this point that it hangs? is there a size-limit on the queue it can build? what is going wrong? it will hang there indefinitely, I have left it for several hours and nothing happens.

I did just install cwRsync 2.0.3 with rsync 2.6.6, and this did not resolve the problem.

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

Can you try to establish external secure channels instead of using internal ssh support in rsync ? Cygwin has some problems with it.

The last version (2.0.4) should be stable enough for external tunneling.

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

I think I've got the same problem. Is the internal SSH going to work in the future? Why is it not stable today? Is there any variable I can pass throu in order to make it work. Like a speedlimit or something...

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

I think I've got the same problem. Is the internal SSH going to work in the future? Why is it not stable today? Is there any variable I can pass throu in order to make it work. Like a speedlimit or something...

AFAIK, The problem is due to cygwin local pipe implementation between ssh and rsync. No solution yet.

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

i am told lowering the version of rsync 5-6 versions helps not tried it though..

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

I think I've got the same problem. Is the internal SSH going to work in the future? Why is it not stable today? Is there any variable I can pass throu in order to make it work. Like a speedlimit or something...

AFAIK, The problem is due to cygwin local pipe implementation between ssh and rsync. No solution yet.

Anyone know if this issue has been resolved? Would it work with a newer cygwin implementation?

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

Unfortunately, still no achievements at this front. Use of external secure tunnels is recommended.

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

There are similar posts to this forum witht the same issue(s)In particular find the post refering to 'rsync is "Flaky"'

http://www.itefix.no/phpws/index.php?module=phpwsbb&PHPWSBB_MAN_OP=view&...

May or may not be part of your solution..

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

When you refer to a external secured channel. what exactly are you talking about?
I am also very interested in making secured RSync connectios over the internet.

thanks,

DM

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

When you refer to a external secured channel. what exactly are you talking about? I am also very interested in making secured RSync connectios over the internet. 

In a external secure channel scenario, you initiate rsync operations directly to an rsync daemon and wrap the whole communication into an explicitly established secure channel.

Check FAQs below for examples:

How can I secure connections between windows rsync clients and cwRsync servers ?

How can I secure connections between linux/unix rsync clients and cwRsync servers ?

itefix
Offline
Last seen: 2 weeks 10 hours ago
Joined: 01.05.2008 - 21:33
Re: hang while syncing folders with many files

I have had exactly the same problem. I have a client who has a Windows 2003 Server (with Terminal Services), and on mass transfers the machine would *completely and utterly* hang! Nothing - no network access, task manager kicked up a stink.

 Fortunately, after much stuffing around, I found an answer (at least it worked for me anway).

These guys have a multi-processor machine (Quad Xeon). I read a post about random hangs with cygwin on multi-processor machines. So, I downloaded imagecfg (Microsoft tool, but you can get it from http://www.robpol86.com/Pages/imagecfg.php if you don't want to download the whole resource kit), ran it against rsync.exe (imagecfg -u rsync.exe), and voila! No more hanging...

Hope that you guys all have multi-processor machines too!

Regards,
Lenny

Topic locked

Release announcements