I have recently needed to upload several hundred GB of files from a Macbook Pro Laptop to S3, then download the S3 Files into 4GB t2.medium Windows 2016 Server EC2 instance so I could use Google Backup and Sync to upload the pictures and docs to a GSuite account. This is required because Australian home internet in Melbourne is incredibly poor perhaps you can get .6 to 1Mbps upload speed so it takes weeks. Also trying to run google backup and sync means that the upload fails regularly.

1. Backup laptop files to a USB drive rsync -av /Users/username /Volumes/USBSHARE/backup
2. On a network with a link better than the normal australian internet upload the files to S3 aws s3 sync /Volumes/USBSHARE/backup/Pictures s3://myS3Bucket/Pictures
3. Create an EC2 instance, install the aws cli and then download from the S3 bucket aws s3 sync s3://myS3Bucket/Pictures ./Pictures
4. Run backup and sync and perform your file upload from the cloud so avoiding time outs and delays caused by the poor Aussie internet.

The trouble I ran into during the above process was getting hundreds of the following type errors:

download failed: s3://yours3Bucket/Path/To/Folder with/Some dots../Fieldwire_ Setup a project for free.html to Path\To\Folder with\Some dots..\Fieldwire_ Setup a project for free.html [Errno 2] No such file or directory: ‘C:\\Users\\Administrator\\Desktop\\BackupAndSync\\Path\\To\\Folder with\\Some dots..\\Fieldwire_ Setup a project for free.html.65FFf5B6′

download failed: s3://yours3Bucket/Documents/Tax Invoices/21 Nowhere Else Rd Nowhere/Tax Invoice 9:21 AVR .docx to Documents\Tax Invoices\21 Nowhere Else Rd Nowhere\Tax Invoice 9:21 AVR .docx [WinError 87] The parameter is incorrect: ‘C:\\Users\\Administrator\\Desktop\\BackupAndSyncDocs\\Documents\\Tax Invoices\\21 Nowhere Else Rd Nowhere\\Tax Invoice 9:21 AVR .docx.2016DBaE’ -> ‘C:\\Users\\Administrator\\Desktop\\BackupAndSyncDocs\\Documents\\Tax Invoices\\21 Nowhere Else Rd Nowhere\\Tax Invoice 9:21 AVR .docx’

After investigating the problem relates to this windows GUI image. Incorrect file characters in the files that came from the Macbook.

So to search for these characters you need to run the following from the source macbook to find the offending characters in the folder you are trying to use on Windows

find . -name ‘*[<>:”/\\|?*]*’

You need some extra software on the mac to be able to easily rename the offending files

Use brew to install the Perl utility rename

brew install rename

Then you just have to change to the parent directory of the files you need to mac2winize and run some commands.

find . -name '*[<>:"/\\|?*]*'  # search for the filename characters Not allow in windows
find . -name '*\**' -exec rename 's/\*//' {} \; # replace * with 
find . -name '*[<>:"/\\|?*]*'
find . -name '*\**' -exec rename 's/\*//g' {} \; # replace all global
find . -name '*[<>:"/\\|?*]*'
find . -name '*\?*' -exec rename 's/\?//g' {} \; # replace question marks
find . -name '*[<>:"/\\|?*]*'
find . -name '*"*' -exec rename s/\"/\'/g {} \; # replace double quotes with windows allowed single quotes
find . -name '*[<>:"/\\|?*]*'
find . -name '*|*' -exec rename 's/\|/-/g' {} \; # replace the pipe character (notice it's escaped \|)
find . -name '*[<>:"/\\|?*]*'
find . -name '*:*' -exec rename 's/:/_/g' {} \; # replace colon with underscore
find . -name '*[<>:"/\\|?*]*'
find . -name '*\>*' -exec rename 's/\>//g' {} \; # replace greater than (notice it's escaped \>)
find . -name '*[<>:"/\\|?*]*'
find . -name '*\**' -exec rename 's/\* //g' {} \; # this one removing a star with a space
find . -name '*[<>:"/\\|?*]*'
find . -name '*\?*' -exec rename 's/\?/a/g' {} \; # replace question mark with an 'a' (useful if the target filename already exists)
find . -name '*[<>:"/\\|?*]*'
find . -name '*\<*' -exec rename 's/\<//g' {} \; # replace less than symbol
find . -name '*[<>:"/\\|?*]*' # finally this should return nothing as all files and folders have had non-windows characters removed

Doing the above will change the local Mac compatible files to Windows compatible and then you need to upload to S3. But the problem is this will not fix already up loaded files so to do that I created an AWS Mac2Win File / Path Fixer