Special Mac characters

12 posts / 0 new
Last post
Philippe Ferreira
Offline
Last seen: 8 years 1 month ago
Joined: 26.11.2008 - 16:51
Special Mac characters

Hello,

We are transfering files from Windows 2008 Server, that hosts files coming from Mac computers.

However, sometimes special characters prevent cwrsync from transfering files. We have to manually rename each of these files to get rid of special characters...

The log file says :

2012/01/13 20:25:17

file has vanished: "/d/Documents/my file .eml"

(I can upload a zip file containing such a file)

Thank you

itefix
Offline
Last seen: 26 min 18 sec ago
Joined: 01.05.2008 - 21:33
Rsync executable packaged in

Rsync executable packaged in the cwRsync package is compiled with the transliterate patch to address problems like yours. Try to use --tr option:

--tr=BAD/GOOD

Transliterates filenames on the receiver, after the iconv conversion (if any). This can be used to remove characters illegal on the destination filesystem. If you use this option, consider saving a dqfind . -lsdq listing of the source in the destination to help you determine the original filenames in case of need.
The argument consists of a string of characters to remove, optionally followed by a slash and a string of corresponding characters with which to replace them. The second string may be shorter, in which case any leftover characters in the first string are simply deleted. For example, --tr=':\/!' replaces colons with exclamation marks and deletes backslashes. Slashes cannot be transliterated because it would cause havoc.
If the receiver is invoked over a remote shell, use --protect-args to stop the shell from interpreting any nasty characters in the argument.

Philippe Ferreira
Offline
Last seen: 8 years 1 month ago
Joined: 26.11.2008 - 16:51
Hello,I tested with --tr,

Hello,

I tested with --tr, however I've got 2 problems :

1) Error "rsync: on remote machine: --tr= /.: unknown option"
I think I have to recompile rsync on the remote machine with the transliterate patch ? (Solaris) But before, I decided to test first local to local (cwrsync to cwrsync)...

2) Local to local: I still get "file has vanished"... I can't manage to copy/paste the special character from the filename to my cwrync command line.

I'll try to send you the file.

Thank you

 

itefix
Offline
Last seen: 26 min 18 sec ago
Joined: 01.05.2008 - 21:33
File name you sent me contain

File name you sent me contain non-english chars. Have you tried to use iconv option in rsync ?

 

--iconv=CONVERT_SPEC
Rsync can convert filenames between character sets using this option. Using a CONVERT_SPEC of "." tells rsync to look up the default character-set via the locale setting. Alternately, you can fully specify what conversion to do by giving a local and a remote charset separated by a comma in the order --iconv=LOCAL,REMOTE, e.g. --iconv=utf8,iso88591. This order ensures that the option will stay the same whether you're pushing or pulling files. Finally, you can specify either --no-iconv or a CONVERT_SPEC of "-" to turn off any conversion. The default setting of this option is site-specific, and can also be affected via the RSYNC_ICONV environment variable.
Philippe Ferreira
Offline
Last seen: 8 years 1 month ago
Joined: 26.11.2008 - 16:51
I tried several --iconv

I tried several --iconv combinations, but with no luck. Have you been able to check with the folder with exotic characters ? (included in the zip file)

Philippe Ferreira
Offline
Last seen: 8 years 1 month ago
Joined: 26.11.2008 - 16:51
It seems to me that cygwin

It seems to me that cygwin (on Windows) cannot handle those special characters coming from Mac. Can you confirm?

itefix
Offline
Last seen: 26 min 18 sec ago
Joined: 01.05.2008 - 21:33
I took a test on an XP

I took a test on an XP machine with both Cwrsync and Cwrsync server installed:

  • copied the directory you sent me to c:\temp
  • Defined the following rsync command via Cwrsync GUI

 "C:\Program Files\cwRsync\bin\rsync.exe" --recursive --stats --progress "/cygdrive/C/temp/AMERICAN´Çá" "localhost::tevdoc"

  • Rsync module tevdoc points to the localdirectory c:\temp\dir22.
  • Run the command via GUI with successful results, log file:

2012/02/02 22:23:54 building file list
2012/02/02 22:23:54 cd+++++++++ AMERICAN´Çá/
2012/02/02 22:23:54 Number of files: 1
2012/02/02 22:23:54 Number of files transferred: 0
2012/02/02 22:23:54 Total file size: 0 bytes
2012/02/02 22:23:54 Total transferred file size: 0 bytes
2012/02/02 22:23:54 Literal data: 0 bytes
2012/02/02 22:23:54 Matched data: 0 bytes
2012/02/02 22:23:54 File list size: 31
2012/02/02 22:23:54 File list generation time: 0.001 seconds
2012/02/02 22:23:54 File list transfer time: 0.000 seconds
2012/02/02 22:23:54 Total bytes sent: 44
2012/02/02 22:23:54 Total bytes received: 12
2012/02/02 22:23:54 sent 44 bytes  received 12 bytes  37.33 bytes/sec
2012/02/02 22:23:54 total size is 0  speedup is 0.00

  • Content of the c:\temp\dir22 is as follows:

C:\temp>dir dir22
 Volume in drive C has no label.
 Volume Serial Number is A027-3EF6

 Directory of C:\temp\dir22

02.02.2012  22:23    <DIR>          .
02.02.2012  22:23    <DIR>          ..
02.02.2012  22:23    <DIR>          AMERICAN'Çá
               0 File(s)              0 bytes
               3 Dir(s)   4 560 736 256 bytes free

 

It seems that this works in Windows. I am wondering if all stuff is related to iconv support. Can you run rsync with --version switch at the source side to check if it supports iconv ?

Philippe Ferreira
Offline
Last seen: 8 years 1 month ago
Joined: 26.11.2008 - 16:51
Can you please unzip the

Can you please unzip the folder with "7-zip", instead of the default Windows program ? 7-Zip keeps the real filename (with the Mac special characters). Thanks.

Philippe Ferreira
Offline
Last seen: 8 years 1 month ago
Joined: 26.11.2008 - 16:51
Hello,In addition to

Hello,

In addition to unzipping with 7-zip, can you please transfer the parent "/cygdrive/C/temp/" (instead of "/cygdrive/C/temp/AMERICAN´Çá"). Generally those special chars are encountered in subfolders.

Thank you. 

itefix
Offline
Last seen: 26 min 18 sec ago
Joined: 01.05.2008 - 21:33
I can confirm that the

I can confirm that the behaviour was as you suggested when I unzipped the directory by using 7-zip. Which charset do those special chars belong to ? It seems to me that they need to be converted by using transliteration patch before sending to cwrsync side.

Philippe Ferreira
Offline
Last seen: 8 years 1 month ago
Joined: 26.11.2008 - 16:51
Hello, these characters are

Hello, these characters are coming from Mac computers (they are dropping files to a Windows share). Thank you.

itefix
Offline
Last seen: 26 min 18 sec ago
Joined: 01.05.2008 - 21:33
I don't have any other

I don't have any other solution than establishing a copy environment in where rsync with transliteration patch is in use at both ends. It seems that rsync as it is used will not solve this problem. Closing the case ..