cwRsync FAQs

cwRsync client is too slow!

Try to remove /etc/fstab and use the option --no-perms  option instead. The problem can also be related to real-time anti-virus scanning. Consider file/folder exclusion if possible.

I can't use long path names

FAQ

From Windows Developer network:

“In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters. A local path is structured in the following order: drive letter, colon, backslash, name components separated by backslashes, and a terminating null character. For example, the maximum path on drive D is "D:\some 256-character path string<NUL>" where "<NUL>" represents the invisible terminating null character for the current system codepage. (The characters < > are used here for visual clarity and cannot be part of a valid path string.)

The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the "\\?\" prefix. For example, "\\?\D:\very long path".”

 

Even if the underlying APIs may have support for really long paths, the problem occurs due to user interface tools like Windows Explorer, Cmd prompt and PowerShell which don't support Unicode variants of the Windows APIs. The result is then path names up to 256 chars. As of version 1.7, Cygwin uses unicode Windows APIs, you can use tools like bash shell or nano editor for management of files/paths with long names.

Log example for cwRsync transfer with long path names:

...
2014/03/25 22:08:42 [1784] >f+++++++++ backup/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/testfile
...

How can I pull files from windows to linux/unix within a secure network ?

Assuming that you want to pull directory c:\backup on Windows machine to /var/backup on Linux machine:

On Windows machine :

  • Install Cwrsync Server
  • Start RsyncServer service
  • Add a new module to the configuration file rsyncd.conf :

[backup]
path = /cygdrive/c/backup
read only = true
transfer logging = yes


On Linux machine :

  • Use the command below to initiate rsync:

rsync -vrt windows_machine::backup /var/backup


NB! Keep in mind that the recipe above is suitable for operations within a secure network.

How can I make pre-exec and post-exec work ?

As of version 2.6.7, rsync server can kick scripts before and after transfers. You can use the recipe below to implement this very handy functionality in cwrsync server:

  1. Install Copssh and Cwrsync Server.
  2. Start a windows command shell from cwrsync start menu and make a copy of bash.exe as sh.exe in the bin directory.

    copy bin\bash.exe bin\sh.exe
  3. Create two script files in the bin directory for pre-exec operations:

    c:\program files\ICW\bin\pre-exec.sh



    # Shell script for your pre-exec operations
    cmd /c "c:\program files\ICW\bin\pre-exec.cmd"
    exit 0




    c:\program files\ICW\bin\pre-exec.cmd




    @ECHO OFF
    .....
  4. Create two script files for post-exec operations as described above.
  5. Update your rsyncd.conf file:




    # Module definitions
    [test]
    path = .....
    ......
    pre-xfer exec = /bin/pre-exec.sh
    post-xfer exec = /bin/post-exec.sh

Why is it called cwRsync?

Use of vocals would be inappropriate (!) as the word rsync has no vocals in it, and this package is also dependent on cygwin:

 

CYGWIN + RSYNC

How can I secure connections between windows rsync clients and cwRsync servers ?

Assuming that you want to synchronize c:\documents and settings\user on WINCLIENT to c:\backup on WINSERVER :

On WINSERVER:
 Install cwRsync Server and Copssh into the same directory. 

  • Start RsyncServer and OpenSSHD services
  • Activate a user (test1 in our example below) via Copssh control panel and specify c:\backup as home directory.
  • Add a new module to rsyncd.conf :

[backup]
path = /cygdrive/c/backup
read only = false
transfer logging = yes

 

on WINCLIENT :

  • Download cwRsync client
  • Open a command prompt and initiate an SSH tunnel (Port 5000 will be used for local port forwarding to Rsync daemon port 873 at WINSERVER)

bin\ssh -L 5000:WINSERVER:873 test1@WINSERVER

  • Open a new command prompt and access Rsync daemon via port 5000:

bin\rsync -vrt "/cygdrive/c/documents and settings/user/" rsync://localhost:5000/backup

  • Close the SSH tunnel upon the completion of Rsync transfers.

Can rsync transfer security/ownership information between two windows machines ?

cwRsync itself has no support for it. However, after an rsync operation, you can use Robocopy (standard in more recent Windows versions), to transfer all windows specific file information (NTFS security, timestamps, attributes, ownership, auditing info):


ROBOCOPY source destination /XO /XN /XC /E /COPY:ATSOU

 

options /XO (exclude older), /XN (exclude newer) and /XC (exclude changed) assure that only existing files are targeted. Option /E is for recursive directory operation including empty ones. Option /COPY:ATSOU instructs Robocopy to copy only attributes (A), timestamps (T), NTFS security information (S), Ownership O) and auditing (U).

How can I use cwRsync between two windows machines within a secure network ?

Assuming that you want to synchronize from directory c:\srcdir on machine WINCLIENT to directory c:\destdir on machine WINSERVER :

On WINSERVER :

  • Install cwRsync Server
  • Start RsyncServer service
  • Make sure that the service account has write access to c:\destdir. You can use Prep a Dir for Upload wizard from start menu.
  • Add a new module to rsyncd.conf :

[wintest]
path = /cygdrive/c/destdir
read only = false
transfer logging = yes

 

On WINCLIENT :

  • Download cwRsync client.
  • Use the command below to initiate rsync communication:

".\bin\rsync" -av /cygdrive/c/srcdir/ WINSERVER::wintest


NB! Keep in mind that the recipe above is suitable for operations within a secure network.

How can I secure connections between linux/unix rsync clients and cwRsync servers ?

Assuming that you want to synchronize from directory /home/user on *nix machine to directory c:\backup on Windows machine :

On Windows:
 

  • Install Copssh and cwRsync server. Use the same service account.
  • Start RsyncServer and OpenSSHD services
  • Activate a user
  • Import your public key via Copssh Control Panel:

Copssh Control Panel - Users - Keys - Import

  •  Make sure that the user and the service account has write access to c:\backup
  • Add a new module to rsyncd.conf :

[backup]
path = /cygdrive/c/backup
read only = false
transfer logging = yes

On Linux:

  • Use shell script below after having it updated according to your needs:
#!/bin/bash
# Secure Channel Wrapper for connection to cwRsync servers from Linux/Unix clients
# v1.0 - Initial version, Sep 2005, Tevfik K., http://itefix.no

# Customize variables below according to your needs

# identity: private key file for user winuser
identity='winuser.key'

# localport: local port for forwarding
localport=9119

# remoteport: termination port (this should be the port rsync daemon listens to)
remoteport=873

# remotehost: cwRsync Server name/ip-adress
remotehost=192.168.2.26

# your rsync module at server side
rsyncmodule=backup

# Function to terminate secure tunnel processes
TerminateTunnel ()
{
ps ax | grep "ssh -i $identity -L $localport" | awk '{print $1}' | xargs -i kill {} 2&>/dev/null
}

# Clean up ... terminate zombie tunnels
TerminateTunnel

echo Establishing secure channel ...
ssh -i $identity -L $localport:127.0.0.1:$remoteport winuser@$remotehost -T -N -f

# your rsync command, you can edit:
# - rsync options (-av)
# - source files (temp/)
# - exchange source and destination
# - and more :-) Try and tell me!
rsync -vrt /home/user/ rsync://winuser@localhost:$localport/$rsyncmodule

echo Terminating secure channel ...
TerminateTunnel


NB! You must forward ssh port 22 to Windows machine at the edge of your secure network (your router/firewall). I recommend strongly to tighten security further by using options available in rsync and openssh (host limitations, secrets file ...).

How can I change port number used by rsync server ?

Rsync server uses port 873 by default. You can add port directive to the configuration file rsyncd.conf to specify an alternative port.

 

port new-port-number

 

Don't forget to restart the service.

I want to set up ssh communication without passwords !!

Here is a simple recipe ( NB! This is not a very secure solution, since we use null passphrase during generation of public key pairs, use it at your own risk):

  • cwRsync client. Generate key pairs :

ssh-keygen -q -t rsa -f cwrsync -N ""


Files cwrsync and cwrsync.pub will be generated.

  • Transfer cwrsync.pub to your *nix machine.
  • *nix machine. Run following commands:

mkdir -p ~$user/.ssh
cat cwrsync.pub > ~$user/.ssh/authorized_keys
chmod 755 ~$user ~$user/.ssh
chmod 644 ~$user/.ssh/authorized_keys
chown $user ~$user ~$user/.ssh ~$user/.ssh/authorized_keys

  • cwRsync client. Update your rsync commands in your batch file by specifying option -e "ssh -i cwrsync".

How can I secure connections between cwrsync clients and unix/linux servers ?

cwRsync has all ssh binaries you need to establish ssh communication.


Use batch file example in the cwRsync client package to set up communication correctly.

Why does my cwrsync try to load ssh?

It's trying to fire up ssh because your target has a single colon in it.

server:/path/to/stuff means "fire up ssh or rsh and make me a tunnel to
server"

server::module/path/to/stuff means "try to access an rsync daemon on
server and access path/to/stuff on module"

/path/to/stuff means "access /path/to/stuff on the local machine"

Hope that helps.

by Jim Salter

Rsync does not recognize windows paths in a correct manner !

Windows paths may contain a colon (:) as a part of drive designation and backslashes (example c:\, g:\). However, in rsync syntax, a colon in a path means searching for a remote host.

Solution: use absolute path 'a la unix', replace backslashes (\) with slashes (/) and put -/cygdrive/- in front of the drive letter.

Example :

 

C:\WORK\*

    • -->

/cygdrive/c/work/*

Problem with secrets file

If you're setting up cwrsync to run as a Service on a Windows machine ensure the .conf file has the line:

strict modes = false

otherwise rsync passwords will not work in Windows.

Can cwRsync co-exist if cygwin is already installed ?

YES: Cygwin 1.7 binaries used by Cwrsync allow co-existence of multiple Cygwin implementations.

Permissions on files/directories are cluttered/mixed up !

If you run into permissions problems or your directories' security ACLs are populated by some unwanted groups/users, you need to make sure that:

  • File ../etc/fstab  exists with at least the content below (with Unix line endings even if the file has only one line!):

none /cygdrive cygdrive binary,posix=0,user,noacl 0 0
  

That will instruct Cygwin not to touch permissions.

 

NB! You may experience very slow transfer rates and/or building the file list may take long time. You can remove /etc/fstab and use the option --no-perms  option instead. The problem can also be related to real-time anti-virus scanning. Consider file/folder exclusion if possible.

 

More information:

The Cygwin Mount Table

The cygdrive path prefix

How can I use rsync to pull files from windows to linux/unix within a secure network ?

Assuming that you want to pull directory c:\backup on machine WIN_SOURCE to directory /var/backup on machine LINUX_DEST :

On WIN_SOURCE :

- Install cwRsync Server (without openssh component)
- Start RsyncServer service
- Add a new module to rsyncd.conf :

[backup]

 path = /cygdrive/c/backup

 read only = true

 transfer logging = yes


On LINUX_DEST :

- Use the command below to initiate rsync:

  • rsync -av WIN_SOURCE::backup /var/backup


NB! Keep in mind that the recipe above is suitable for operations within a secure network.