Dear reader, I hope you are doing great. In today’s post, I will demonstrate “How to Enable Cross Region Replication in S3” in a beginner friendly manner.
If you are new to S3, I highly recommended you to have a look on my previous post How to Create an S3 Bucket on AWS. This will provide you a good insight on getting started with Amazon S3.
However, if you already know the nitty gritty of S3, that’s good and you can continue with next steps.
By the end of this post, we will also talk about replicating existing objects into another bucket in another region. Therefore, stay hooked. 🙂
What say- shall we start?
Alright..
Let’s know CRR a bit –
What is S3 Cross Region Replication?
S3 cross region replication or CRR is the type of S3 replication where an object uploaded to an S3 bucket is automatically copied into another bucket in a different region.
- Objects are replicated asynchronously and hence eventually copied
- Usually objects are replicated within 15 minutes but sometimes they can take hours
- You can use CRR for disaster recovery , proving lower latency to your users in different region or other use cases depending on requirement
Things to Keep In Mind While Setting up CRR(Cross Region Replication)
- By default only new objects are replicated.
- Both source and destination bucket needs to have versioning enabled.
- A role needs to be created that permits the source bucket to write into the destination bucket.
- Objects can be replicated to same account or in a bucket in different account
- There are charges associated with CRR such as destination storage cost, inter region data transfer cost etc. Please check official pricing page for updated details
- If you need to replicated existing object as well, at the time of replication rule setup, you can configure it as add on.
Use case
I want to replicate all the objects uploaded to my primary bucket into a replica bucket which is in different region.
To demonstrate that, we will create two S3 buckets in different region.
- cloudkatha-crr-demo-primary (ap-south-1)
- cloudkatha-crr-demo-replica (us-west-1)
And, in this tutorial, we will setup replication on source bucket that is cloudkatha-crr-demo-primary to automatically replicate all data uploaded in it into the destination bucket cloudkatha-crr-demo-replica.
Steps to Setup S3 Cross Region Replication
- Login to AWS Management Console and Open S3
- Create Two S3 Buckets In Primary and Replica Region
- Enable Cross Region Replication on Primary/Origin Bucket
- Verify Object Replication
1. Login to AWS Management Console and Open S3
Login to AWS Management Console . Search s3 in the search bar and once it appears, click on it to open S3.
Once you click on s3, s3 dashboard is shown with all the buckets listing.
2. Create Two S3 Bucket In Primary and Replica Regions
1. Create Primary Bucket:
Click on Create bucket
In general configuration, provide the name of the primary bucket and region you want to keep it into.
For this tutorial, I am keeping them like below-
Scroll down to Bucket Versioning section and enable versioning.
All you need to do is to click on Enable as shown in below screenshot. For detail, you can check my previous post on How to Enable Versioning In an S3 Bucket.
Leave everything default, scroll down and click Create bucket.
Primary bucket is successfully created and you get success message like below.
2. Create Replica Bucket
Again click on Create bucket in S3 dashboard.
Choose name and region you want your replica bucket to reside in.
For this tutorial- I have kept them like below-
Scroll down and enable versioning on the bucket from Bucket Versioning section
Leave everything else as it is and click on Create bucket.
Replica bucket is created successfully and you get to see message like below-
We have created two buckets – one in primary and one in replica region. Let’s go ahead and setup cross region replication on primary bucket which will be the origin bucket.
3. Enable Cross Region Replication on Primary/Origin Bucket
From the S3 dashboard, click on the primary bucket name to navigate inside the bucket.
Navigate to Management tab, scroll down to Replication rules section and click Create replication rule
Fill up details in replication rule configuration. You can use any name you like. Leave the status as enabled– If enabled, as soon as rule is created, s3 will start replicating newly uploaded objects in this bucket into the replica bucket.
However, if you don’t want that to happen. You can choose to make it disabled and later enable it when you want.
In Source bucket section, you can choose rule scope. You might want to replicate only a specific prefix or any other filter like object tags. You can define those filters/tags in this section.
To keep things simple, I have chosen the other option to apply to all objects in the bucket. So any object uploaded into it will be replicated to the replica bucket.
Next, choose destination bucket by clicking on Browse S3 button like shown below
Select your destination/replica bucket and click on Choose path.
Once you click on choose path, Destination section looks like below-
I have chosen a bucket in the same account. However you can decide to choose a bucket in different account as well like you see below-
Once destination is selected, you need to choose a role that has permission to write into the destination bucket. If you want s3 to create a new role for you, click on the dropdown and choose Create new role.
For simplicity, I am leaving encryption and destination storage class as it is.
By this time we have done general configuration for replication rule. So we can go ahead and create this rule. However, just for your info there are few additional options like delete marker replication which doesn’t happen by default. You can choose these options based on your need.
Also as I said, by simply creating replication rule, only new objects are replicated. Old ones are not, so in case you are setting replication rule on an existing bucket and need s3 to replication your existing objects, choose “Yes, replicate existing objects“. On choosing that option you will be take on a page where you will configure details for existing object replication.
As I am doing it on a new bucket, so I have chosen other option. Once you are done, click on submit and your replication rule will get created.
You can see them in them in the Replication rules section as expected.
4. Verify Object Replication
We have setup a replication rule on our primary bucket to replicate all newly uploaded objects into the replica bucket.
Let’s verify that by uploading an object. Click on the bucket name to go inside the bucket. By default you are in Objects tab. Click upload as shown below.
Choose the file from your system that you want to upload and upload it. Once uploaded, you will see the object as below-
Now it’s time to verify the replica bucket. Go inside replica bucket and you will see the object there as you can see below.
Congratulations, You have successfully enabled cross region replication on your bucket.
Note: As you noticed, the object was copied just in seconds. But as I said earlier, this replication is asynchronous and usually can take up to 15 minutes. But in rare scenarios, it might even take hours. If you don’t see the object in destination bucket, try checking replication status by checking object details. I have given an screenshot below for your reference.
Conclusion
In this post, we learnt to enable Cross Region Replication on Amazon S3 Bucket in a step by step manner.
We created two buckets – one in source and one in destination region. We enabled versioning on both the bucket while creating.
Then we setup replication rule on source bucket. Uploaded a file in source bucket and validated the CRR by checking the destination bucket for upload object.
I hope it was clear enough for you. In case you have any doubt, please feel free to drop in comment. I would be happy to help.
Enjoyed the content?
Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.
Don’t forget to motivate me by-
- Adding a comment below on what you liked and what can be improved.
- Follow us on
- Share this post with your friends