Resolved: Number of distinct destination bucket ARNs cannot exceed 1
Error Message :
Number of distinct destination bucket ARNs cannot exceed 1
Problem:
I was trying to set up Cross Region Replication on my S3 bucket using CloudFormation. I had a use case where replication needed to happen from one source bucket to multiple destination buckets.
As far as I knew, AWS supported multiple destinations for replication as of Dec 1, 2020, as per this announcement. However, when I tried it, my stack creation failed.
For reference, below is the portion of my CloudFormation template, where I am setting two rules, one for each replica bucket-
PrimaryBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref SourceBucketName
VersioningConfiguration:
Status: Enabled
ReplicationConfiguration:
Role: !GetAtt S3ReplicationRole.Arn
Rules:
- Id: Rule1
Status: Enabled
Priority: 1
Destination:
Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket1}'
- Id: Rule2
Status: Enabled
Priority: 2
Destination:
Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket2}'My Stack Failed With Below Error !!!
Number of distinct destination bucket ARNs cannot exceed 1
What is the problem with the above template?
Well, on doing a little bit of research I found, out why it was not working and how to make it work.
Let’s start with understanding what’s wrong.
AWS S3 stores Replication configuration as XML. And the latest version of the schema is V2. Although, AWS keeps supporting V1 for backward compatibility.
I found that, in order for multiple destinations to work, you must force ReplicationRules to use the V2 schema.
Now, you might be wondering how I do that –
Well, AWS has an answer to that-
The use of the filter field indicates this is a V2 replication configuration. V1 does not have this field.
Under the hood if you see these are the minimum required fields for a schema to be V2.
...
<Rule>
<ID>Rule-1</ID>
<Status>Enabled-or-Disabled</Status>
<Filter>
<Prefix></Prefix>
</Filter>
<Priority>integer</Priority>
<DeleteMarkerReplication>
<Status>Enabled-or-Disabled</Status>
</DeleteMarkerReplication>
<Destination>
<Bucket>arn:aws:s3:::bucket-name</Bucket>
</Destination>
</Rule>
<Rule>
...
</Rule>
...
...Therefore, ideally, the problem with the template is we are using V1 of the ReplicationRule schema and we must use V2 of the schema to make things work.
Solution:
The solution is exactly what are thinking of.
We will force Replication Rule to make use of the Filter element on each of the rules to ensure that it’s using V2 schema.
Now as per the syntax, “If you specify Filter elements, you must also include Priority and DeleteMarkerReplication elements.
Based on these assumptions, I edited my template like below and gave it another try. And, as expected, this worked perfectly fine.
AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template to Setup S3 CRR
Parameters:
SourceBucketName:
Type: String
Description: Primary Bucket Name
Default: cloudkatha-web-bucket-primary
ReplicaBucket1:
Type: String
Description: Replica Bucket 1
Default: cloudkatha-web-bucket-replica1
ReplicaBucket2:
Type: String
Description: Replica Bucket 2
Default: cloudkatha-web-bucket-replica2
Resources:
PrimaryBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref SourceBucketName
VersioningConfiguration:
Status: Enabled
ReplicationConfiguration:
Role: !GetAtt S3ReplicationRole.Arn
Rules:
- Id: Rule1
Status: Enabled
DeleteMarkerReplication:
Status: Enabled
Priority: 1
Filter:
Prefix: ''
Destination:
Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket1}'
- Id: Rule2
Status: Enabled
DeleteMarkerReplication:
Status: Enabled
Priority: 2
Filter:
Prefix: ''
Destination:
Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket2}'
S3ReplicationRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- 'sts:AssumeRole'
Effect: Allow
Principal:
Service:
- s3.amazonaws.com
S3ReplicationPolicy:
Type: 'AWS::IAM::Policy'
Properties:
PolicyName: S3BucketReplicationPolicy
Roles:
- !Ref S3ReplicationRole
PolicyDocument:
Statement:
- Action:
- 's3:GetReplicationConfiguration'
- 's3:ListBucket'
Effect: Allow
Resource: !Sub 'arn:aws:s3:::${SourceBucketName}'
- Action:
- 's3:GetObjectVersion'
- 's3:GetObjectVersionAcl'
Effect: Allow
Resource: !Sub 'arn:aws:s3:::${SourceBucketName}'
- Action:
- 's3:ReplicateObject'
- 's3:ReplicateDelete'
Effect: Allow
Resource:
- !Sub 'arn:aws:s3:::${ReplicaBucket1}'
- !Sub 'arn:aws:s3:::${ReplicaBucket2}'Once replication rules were created on the primary bucket, this is how it looked like a console.
Conclusion
In this quick fix post, we learnt that when you get Number of distinct destination bucket ARNs cannot exceed 1 error while setting up CRR using CloudFormation, it’s the schema version which is the culprit.
You are required to use V2 of schema and you can do that by adding a Filter property on the rule.
We also saw that even if you want the rule to apply to the whole bucket, still you add a Filter with an empty prefix which is equivalent to applying to the whole bucket.
I hope you were able to solve this issue. Please let me know in the comment section if you are still facing an issue.
Enjoyed the content?
Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.
Don’t forget to motivate me by-
- Adding a comment below on what you liked and what can be improved.
- Follow us on
- Share this post with your friends
2 thoughts on “Resolved: Number of distinct destination bucket ARNs cannot exceed 1”
Thank you so much! This did the trick!
Thank you 🙂 Glad it helped …