Resolved: Number of distinct destination bucket ARNs cannot exceed 1

Error Message :

Number of distinct destination bucket ARNs cannot exceed 1

Problem:

I was trying to set up Cross Region Replication on my S3 bucket using CloudFormation. I had a use case where replication needed to happen from one source bucket to multiple destination buckets.

As far as I knew, AWS supported multiple destinations for replication as of Dec 1, 2020, as per this announcement. However, when I tried it, my stack creation failed.

For reference, below is the portion of my CloudFormation template, where I am setting two rules, one for each replica bucket-

  PrimaryBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref SourceBucketName
      VersioningConfiguration:
        Status: Enabled
      ReplicationConfiguration:
        Role: !GetAtt S3ReplicationRole.Arn
        Rules:
          - Id: Rule1
            Status: Enabled
            Priority: 1
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket1}'
          - Id: Rule2
            Status: Enabled
            Priority: 2
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket2}'

My Stack Failed With Below Error !!!

Number of distinct destination bucket ARNs cannot exceed 1

What is the problem with the above template?

Well, on doing a little bit of research I found, out why it was not working and how to make it work.

Let’s start with understanding what’s wrong.

AWS S3 stores Replication configuration as XML. And the latest version of the schema is V2. Although, AWS keeps supporting V1 for backward compatibility.

I found that, in order for multiple destinations to work, you must force ReplicationRules to use the V2 schema.

Now, you might be wondering how I do that –

Well, AWS has an answer to that-

The use of the filter field indicates this is a V2 replication configuration. V1 does not have this field.

Under the hood if you see these are the minimum required fields for a schema to be V2.

...
    <Rule>
        <ID>Rule-1</ID>
        <Status>Enabled-or-Disabled</Status>
        <Filter>
            <Prefix></Prefix>   
        </Filter>
        <Priority>integer</Priority>
        <DeleteMarkerReplication>
           <Status>Enabled-or-Disabled</Status>
        </DeleteMarkerReplication>
        <Destination>        
           <Bucket>arn:aws:s3:::bucket-name</Bucket> 
        </Destination>    
    </Rule>
    <Rule>
         ...
    </Rule>
     ...
...

Therefore, ideally, the problem with the template is we are using V1 of the ReplicationRule schema and we must use V2 of the schema to make things work.

Solution:

The solution is exactly what are thinking of.

We will force Replication Rule to make use of the Filter element on each of the rules to ensure that it’s using V2 schema.

Now as per the syntax, “If you specify Filter elements, you must also include Priority and DeleteMarkerReplication elements.

Based on these assumptions, I edited my template like below and gave it another try. And, as expected, this worked perfectly fine.

AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template to Setup S3 CRR

Parameters:
  SourceBucketName:
    Type: String
    Description: Primary Bucket Name
    Default: cloudkatha-web-bucket-primary
  ReplicaBucket1:
    Type: String
    Description: Replica Bucket 1
    Default: cloudkatha-web-bucket-replica1
  ReplicaBucket2:
    Type: String
    Description: Replica Bucket 2
    Default: cloudkatha-web-bucket-replica2
Resources:
  PrimaryBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref SourceBucketName
      VersioningConfiguration:
        Status: Enabled
      ReplicationConfiguration:
        Role: !GetAtt S3ReplicationRole.Arn
        Rules:
          - Id: Rule1
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 1
            Filter:
              Prefix: ''
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket1}'
          - Id: Rule2
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 2
            Filter:
              Prefix: ''
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket2}'
  S3ReplicationRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action:
              - 'sts:AssumeRole'
            Effect: Allow
            Principal:
              Service:
                - s3.amazonaws.com

  S3ReplicationPolicy:
    Type: 'AWS::IAM::Policy'
    Properties:
      PolicyName: S3BucketReplicationPolicy
      Roles:
        - !Ref S3ReplicationRole
      PolicyDocument:
        Statement:
          - Action:
              - 's3:GetReplicationConfiguration'
              - 's3:ListBucket'
            Effect: Allow
            Resource: !Sub 'arn:aws:s3:::${SourceBucketName}'
          - Action:
              - 's3:GetObjectVersion'
              - 's3:GetObjectVersionAcl'
            Effect: Allow
            Resource: !Sub 'arn:aws:s3:::${SourceBucketName}'
          - Action:
              - 's3:ReplicateObject'
              - 's3:ReplicateDelete'
            Effect: Allow
            Resource:
              - !Sub 'arn:aws:s3:::${ReplicaBucket1}'
              - !Sub 'arn:aws:s3:::${ReplicaBucket2}'

Once replication rules were created on the primary bucket, this is how it looked like a console.

Conclusion

In this quick fix post, we learnt that when you get Number of distinct destination bucket ARNs cannot exceed 1 error while setting up CRR using CloudFormation, it’s the schema version which is the culprit.

You are required to use V2 of schema and you can do that by adding a Filter property on the rule.

We also saw that even if you want the rule to apply to the whole bucket, still you add a Filter with an empty prefix which is equivalent to applying to the whole bucket.

I hope you were able to solve this issue. Please let me know in the comment section if you are still facing an issue.

Enjoyed the content?

Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.

Don’t forget to motivate me by-

• Adding a comment below on what you liked and what can be improved.
• Follow us on
  • Facebook
  • Twitter
  • LinkedIn
  • Instagram
• Share this post with your friends

Suggested Read:

• Create a DynamoDB table using CloudFormation
• 5 Ways to Create and Manage your AWS Resources
• AWS S3 Storage Classes: Everything you need to know