In this post I’m going to explain the challenges of syncing an S3 bucket in the US with an Aliyun OSS bucket hosted in China.

The goal was simple: Let people reliably upload large files (e.g. 500MB marketing videos) using their Content Management System, from the US, Europe and within China. Since OSS is mostly API-compatible with S3, I configured our CMS to upload files directly to OSS and gave it a test.

Surprise, it didn’t work! The Great Firewall was blocking me.

Failed Attempt #1: A Naive Request

The US-hosted CMS backend cannot upload large files to OSS because the request will reliably fail. I thought perhaps certain IP ranges are blocked, so I changed the upload mechanism to upload directly from the browser to OSS: Not only was this way more complicated, it solved nothing. It would work on Monday but, thanks to constantly changing blocking heuristics, it would break by Friday.

So let’s introduce a queue.

Failed Attempt #2: Queue Won’t Believe What Happened Next

I set up an hourly job to copy files from AWS to OSS.

Again, the sync seems to work at first, but the bandwidth quickly stagnates until the connection is dropped. I can only guess the GFW doesn't like connections originating from outside pushing large amounts of data in. The problem is so bad, even if you chop your file transfer into smaller chunks it will take multiple days to upload just 1 gigabyte.

You need something on the inside.

Success: The Arbiter Of Files

Luckily for me, someone set me up a computer on the Chinese internet. I installed a Github Actions runner on it that syncs all files from S3 to its disk and then syncs from disk to OSS. It worked! Having a local copy of everything on Chinese internet, behind the GFW made all the difference.