rsync+ssh hanging during transfers?

11 posts / 0 new
Last post
itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
rsync+ssh hanging during transfers?

Hi guys,
My server is running cwRsync 2.0.4. on a Windows 2003 Server machine. My client machine is running Red Hat Linux 8.0. I'm using the SSH package from the cwRsync installer, and using the default "svcwrsync" user to connect to my Windows server.

The command I'm using from my Linux box is:

rsync --progress -ave "ssh -i cwrsync.key" svcwrsync@10.0.0.50:/cygdrive/f/Noddy .

It will connect to the server, and find the directory that it wants. I have some files (previously transferred via a different method) that already exist on the client, so it doesn't need to download them again. But when it finds a new file, it either downloads the first chunks of the file, or doesn't download any of the file. :( For example:

# rsync --progress -ave "ssh -i cwrsync.key" svcwrsync@10.0.0.50:/cygdrive/f/Noddy .
receiving file list ...
214 files to consider
Noddy/
Noddy/Noddy_data.mdb
32768 0% 390.24kB/s 0:04:35

My rsyncd.conf on the server looks like this:

use chroot = false
strict modes = true
hosts allow = *
log file = rsyncd.log
pid file = rsyncd.pid

path = /cygdrive/f/Maintenance
read only = true
transfer logging = yes

I am trying to do this across a ~1Mbit/second 802.11b wireless connection. I've used rsync under Linux (client AND server) before so I know it's not a connection speed problem... thanks!

Chris

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

I forgot to add, if I do not use SSH then the transfers work fine. I get about 370Kbit/s transfer speeds.

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

Your problem appears due to improper socket handling at cygwin side (cwRsyncServer), affecting rsync thru ssh. Setting external tunnels are recommended. FAQ How can I secure connections between linux/unix rsync clients and cwRsync servers ? might be helpful in your case.

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

Thanks!
NB! You must forward ssh port 22 to CWRSYNC_SERVER at the edge of your secure network (your router/firewall). I recommend strongly to tighten security further by using options available in rsync and openssh (host limitations, secrets file ...).

I am confused about how to do this, exactly. Can you please tell me the exact command I need to run on my Linux client to do this? I do not have a router/firewall, only a wireless bridge.

10.0.0.57 (Linux client) goes through 10.0.0.18 (one side of the wireless brige), and then 10.0.0.14 (the other side of the wireless bridge) and so can connect to 10.0.0.50 (Windows server). Does that help?

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

It seems to me that you have both servers in the same network. You can then simply run rsync in daemon mode without ssh transport:

rsync --progress -ave 10.0.0.50::maintenance .

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

Yes, they are in the same network but the connection is a 2km wireless link through a busy city. I use WEP but the wireless bridges are old, and I do not trust the encryption -- so I prefer to use SSH. Can you still help? :-|

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

Yes, they are in the same network but the connection is a 2km wireless link through a busy city. I use WEP but the wireless bridges are old, and I do not trust the encryption -- so I prefer to use SSH. Can you still help? :-|

You can still use the FAQ I mentioned before. You can ignore forwarding of port 22, since that does not apply to your situation.

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

Thankyou for your reply. I am still having some trouble :(
I can use this command to set up the tunnel:

ssh -i cwrsync.key svcwrsync@10.0.0.50 -T -N -f -v

And it is successful. But then, how do I tunnel my rsync data through it? I am confused at that part. :( If I use the command:

rsync -uav --progress --delete svcwrsync@10.0.0.50::Maintenance .

Then the rsync works fine, but obviously not over the SSH tunnel.

Any ideas?

Chris

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

You have to setup ssh local port forwarding to make it work (ssh option -L. Try to customize the shell script mentioned in the FAQ. It will set up ssh tunnell with local port forwarding.

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

I understand, but I am confused -- how can I use local port forwarding without remote port forwarding of port 22?

itefix
Offline
Last seen: 8 hours 23 min ago
Joined: 01.05.2008 - 21:33
Re: rsync+ssh hanging during transfers?

echo Establishing secure channel ...
ssh -i cwrsync.key -L 9119:127.0.0.1:873 SvcwRsync@10.0.0.50 -T -N -f

The ssh command above connects to 10.0.0.50 on port 22 (default) and establishes a secure tunnel after having authenticated by your private key. It also configures itself to listen to local port 9119 on your linux box. All traffic directed to 9119 will then be forwarded to port 873 (rsync server) on remote machine (10.0.0.50).

# your rsync command
rsync -uav --progress --delete rsync://SvcwRsync@localhost:9119/Maintenance .

The rsync command above will connect to port 9119 on localhost. All traffic will then be forwarded to port 873 on 10.0.0.50 by ssh tunnel.

#!/bin/bash
identity='cwrsync.key'

# localport: local port for forwarding
localport=9119

# remoteport: termination port (this should be the port rsync daemon listens to)
remoteport=873

# remotehost: cwRsync Server name/ip-adress
remotehost=10.0.0.50

# your rsync module at server side
rsyncmodule=Maintenance

# Function to terminate secure tunnel processes
TerminateTunnel ()
{
ps ax | grep "ssh -i $identity -L $localport" | awk '{print $1}' | xargs -i kill {} 2&>/dev/null
}

# Clean up ... terminate zombie tunnels
TerminateTunnel

echo Establishing secure channel ...
ssh -i $identity -L $localport:127.0.0.1:$remoteport SvcwRsync@$remotehost -T -N -f

# your rsync command, you can edit:
rsync -uav --progress --delete rsync://SvcwRsync@localhost:9119/Maintenance .

echo Terminating secure channel ...
TerminateTunnel

The customized script above makes it all for you. It also terminates ssh tunnel when rsync finished its operations.

Hope it helps.

Topic locked

Release announcements