YES: Cygwin 1.7 binaries used by Cwrsync allow co-existence of multiple Cygwin implementations.
cwRsync itself has no support for it. However, after an rsync operation, you can use Robocopy (standard in more recent Windows versions), to transfer all windows specific file information (NTFS security, timestamps, attributes, ownership, auditing info):
ROBOCOPY source destination /XO /XN /XC /E /COPY:ATSOU
options /XO (exclude older), /XN (exclude newer) and /XC (exclude changed) assure that only existing files are targeted. Option /E is for recursive directory operation including empty ones. Option /COPY:ATSOU instructs Robocopy to copy only attributes (A), timestamps (T), NTFS security information (S), Ownership O) and auditing (U).
Try to remove /etc/fstab and use the option --no-perms option instead. The problem can also be related to real-time anti-virus scanning. Consider file/folder exclusion if possible.
Rsync server uses port 873 by default. You can add port directive to the configuration file rsyncd.conf to specify an alternative port.
port new-port-number
Restart the service.
As of version 2.6.7, rsync server can kick scripts before and after transfers. You can use the recipe below to implement this very handy functionality in cwrsync server:
- Install Copssh and Cwrsync Server.
- Start a windows command shell from cwrsync start menu and make a copy of bash.exe as sh.exe in the bin directory.
copy bin\bash.exe bin\sh.exe - Create two script files in the bin directory for pre-exec operations:
c:\program files\ICW\bin\pre-exec.sh
# Shell script for your pre-exec operations
cmd /c "c:\program files\ICW\bin\pre-exec.cmd"
exit 0
c:\program files\ICW\bin\pre-exec.cmd
@ECHO OFF
..... - Create two script files for post-exec operations as described above.
- Update your rsyncd.conf file:
# Module definitions
path = .....
......
pre-xfer exec = /bin/pre-exec.sh
post-xfer exec = /bin/post-exec.sh
Assuming that you want to pull directory c:\backup on Windows machine to /var/backup on Linux machine:
On Windows machine :
- Install Cwrsync Server
- Start RsyncServer service
- Add a new module to the configuration file rsyncd.conf :
path = /cygdrive/c/backup
read only = true
transfer logging = yes
On Linux machine :
- Use the command below to initiate rsync:
rsync -vrt windows_machine::backup /var/backup
NB! Keep in mind that the recipe above is suitable for operations within a secure network.
cwRsync has all ssh binaries you need to establish ssh communication.
Use batch file example in the cwRsync client package to set up communication correctly.
On Windows:
- Install Copssh and cwRsync server. Use the same service account.
- Start RsyncServer and OpenSSHD services
- Activate a user
- Import your public key via Copssh Control Panel:
- Make sure that the user and the service account has write access to c:\backup
- Add a new module to rsyncd.conf :
[backup] path = /cygdrive/c/backup read only = false transfer logging = yes
On Linux:
- Use shell script below after having it updated according to your needs:
#!/bin/bash localport=9119 remoteport=873 remotehost=192.168.100.21 rsyncmodule=backup remoteuser=lab1user sourcedir=/bin/ echo Establishing secure channel ... ssh -L $localport:127.0.0.1:$remoteport $remoteuser@$remotehost -T -N -f rsync -vrt $sourcedir rsync://localhost:$localport/$rsyncmodule echo Terminating secure channel ... ps ax | grep "ssh -L $localport" | awk '{print $1}' | xargs -i kill {} 2&>/dev/null
On WINSERVER:
Install cwRsync Server and Copssh into the same directory.
- Start RsyncServer and OpenSSHD services
- Activate a user (test1 in our example below) via Copssh control panel and specify c:\backup as home directory.
- Add a new module to rsyncd.conf :
[backup] path = /cygdrive/c/backup read only = false transfer logging = yes
on WINCLIENT :
- Download cwRsync client
- Create a batch file for automation (the example below copies contents of c:\windows\fonts\s* to the directory behind the rsync module test at the server side via a secure ssh tunnel:
@ECHO OFF SET CWRSYNC_DIR=C:\cwrsync_client_x64 SET CHANNEL_ID=Secure_Channel_ABC SET LOCALPORT=9119 SET REMOTEPORT=873 SET REMOTEHOST=192.168.100.21 SET REMOTEUSER=lab1user SET RSYNCMODULE=test SET SOURCEDIR=/cygdrive/c/windows/fonts/s* echo Establish secure channel ... start "%CHANNEL_ID%" /MIN %CWRSYNC_DIR%\bin\ssh -L %LOCALPORT%:127.0.0.1:%REMOTEPORT% %REMOTEUSER%@%REMOTEHOST% -T -N -f "%CWRSYNC_DIR%\bin\rsync" -vrt %SOURCEDIR% "rsync://localhost:%LOCALPORT%/%RSYNCMODULE%" echo Terminate secure channel ... taskkill /FI "WINDOWTITLE eq %CHANNEL_ID%"
Assuming that you want to synchronize from directory c:\srcdir on machine WINCLIENT to directory c:\destdir on machine WINSERVER :
On WINSERVER :
- Install cwRsync Server
- Start RsyncServer service
- Make sure that the service account has write access to c:\destdir. You can use Prep a Dir for Upload wizard from start menu.
- Add a new module to rsyncd.conf :
path = /cygdrive/c/destdir
read only = false
transfer logging = yes
On WINCLIENT :
- Download cwRsync client.
- Use the command below to initiate rsync communication:
".\bin\rsync" -av /cygdrive/c/srcdir/ WINSERVER::wintest
NB! Keep in mind that the recipe above is suitable for operations within a secure network.
Assuming that you want to pull directory c:\backup on machine WIN_SOURCE to directory /var/backup on machine LINUX_DEST :
On WIN_SOURCE :
- Install cwRsync Server (without openssh component)
- Start RsyncServer service
- Add a new module to rsyncd.conf :
On LINUX_DEST :
path = /cygdrive/c/backup
read only = true
transfer logging = yes
- Use the command below to initiate rsync:
rsync -av WIN_SOURCE::backup /var/backup
NB! Keep in mind that the recipe above is suitable for operations within a secure network.
From Windows Developer network:
“In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters. A local path is structured in the following order: drive letter, colon, backslash, name components separated by backslashes, and a terminating null character. For example, the maximum path on drive D is "D:\some 256-character path string<NUL>" where "<NUL>" represents the invisible terminating null character for the current system codepage. (The characters < > are used here for visual clarity and cannot be part of a valid path string.)
The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the "\\?\" prefix. For example, "\\?\D:\very long path".”
Even if the underlying APIs may have support for really long paths, the problem occurs due to user interface tools like Windows Explorer, Cmd prompt and PowerShell which don't support Unicode variants of the Windows APIs. The result is then path names up to 256 chars. As of version 1.7, Cygwin uses unicode Windows APIs, you can use tools like bash shell or nano editor for management of files/paths with long names.
Log example for cwRsync transfer with long path names:
...
2014/03/25 22:08:42 >f+++++++++ backup/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789
/0123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789/testfile
...
Here is a simple recipe ( NB! This is not a very secure solution, since we use null passphrase during generation of public key pairs, use it at your own risk):
- cwRsync client. Generate key pairs :
ssh-keygen -q -t rsa -f cwrsync -N ""
Files cwrsync and cwrsync.pub will be generated.
- Transfer cwrsync.pub to your *nix machine.
- *nix machine. Run following commands:
mkdir -p ~$user/.ssh
cat cwrsync.pub > ~$user/.ssh/authorized_keys
chmod 755 ~$user ~$user/.ssh
chmod 644 ~$user/.ssh/authorized_keys
chown $user ~$user ~$user/.ssh ~$user/.ssh/authorized_keys
- cwRsync client. Update your rsync commands in your batch file by specifying option -e "ssh -i cwrsync".
If you run into permissions problems or your directories' security ACLs are populated by some unwanted groups/users, you need to make sure that:
- File ../etc/fstab exists with at least the content below (with Unix line endings even if the file has only one line!):
none /cygdrive cygdrive binary,posix=0,user,noacl 0 0
That will instruct Cygwin not to touch permissions.
NB! You may experience very slow transfer rates and/or building the file list may take long time. You can remove /etc/fstab and use the option --no-perms option instead. The problem can also be related to real-time anti-virus scanning. Consider file/folder exclusion if possible.
More information:
If you're setting up cwrsync to run as a Service on a Windows machine ensure the .conf file has the line:
strict modes = false
otherwise rsync passwords will not work in Windows.
Windows paths may contain a colon (:) as a part of drive designation and backslashes (example c:\, g:\). However, in rsync syntax, a colon in a path means searching for a remote host.
Solution: use absolute path 'a la unix', replace backslashes (\) with slashes (/) and put -/cygdrive/- in front of the drive letter.
Example :
C:\WORK\* --> /cygdrive/c/work/*
It's trying to fire up ssh because your target has a single colon in it.
server:/path/to/stuff means "fire up ssh or rsh and make me a tunnel to
server"
server::module/path/to/stuff means "try to access an rsync daemon on
server and access path/to/stuff on module"
/path/to/stuff means "access /path/to/stuff on the local machine"
Hope that helps.
by Jim Salter
Use of vocals would be inappropriate (!) as the word rsync has no vocals in it, and this package is also dependent on Cygwin:
CYGWIN + RSYNC