Error Message :

Number of distinct destination bucket ARNs cannot exceed 1

Problem:

I was trying to setup Cross Region Replication on my S3 bucket using CloudFormation. I had a use-case where, replication needed to happen from one source bucket to multiple destination buckets.

As far as I knew, AWS supported multiple destination for replication as of Dec 1, 2020 as per this announcement. However, when I tried it, my stack creation failed.

For reference, below is the portion of my CloudFormation template, where I am setting two rules, one for each replica bucket-

  PrimaryBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref SourceBucketName
      VersioningConfiguration:
        Status: Enabled
      ReplicationConfiguration:
        Role: !GetAtt S3ReplicationRole.Arn
        Rules:
          - Id: Rule1
            Status: Enabled
            Priority: 1
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket1}'
          - Id: Rule2
            Status: Enabled
            Priority: 2
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket2}'

My Stack Failed With Below Error !!!

Number of distinct destination bucket ARNs cannot exceed 1

What is the problem with above template?

Well, on doing little bit of research I found, why it was not working and how to make it work.

Let’s start with understanding what’s wrong.

AWS S3 stores Replication configuration as XML. And the latest version of schema is V2. Although, AWS keeps supporting V1 for backward compatibility.

I found that, in order for multiple destinations to work, you must force ReplicationRules to use V2 schema.

Now, you might be wondering how do I do that –

Well, AWS has answer to that-

The use of the filter field indicates this is a V2 replication configuration. V1 does not have this field.

Under the hood if you see these are the minimum required field for a schema to be V2.

...
    <Rule>
        <ID>Rule-1</ID>
        <Status>Enabled-or-Disabled</Status>
        <Filter>
            <Prefix></Prefix>   
        </Filter>
        <Priority>integer</Priority>
        <DeleteMarkerReplication>
           <Status>Enabled-or-Disabled</Status>
        </DeleteMarkerReplication>
        <Destination>        
           <Bucket>arn:aws:s3:::bucket-name</Bucket> 
        </Destination>    
    </Rule>
    <Rule>
         ...
    </Rule>
     ...
...

Therefore, ideally the problem with template is we are using V1 of ReplicationRule schema and we must use V2 of schema to make things work.

Solution:

Solution is exactly what are thinking of.

We will force Replication Rule to make use of Filter element on each of the rule to ensure that it’s using V2 schema.

Now as per the syntax, “If you specify Filter elements , you must also include Priority and DeleteMarkerReplication elements.

Based on these assumption, I edited my template like below and gave it another try. And, as expected, this worked perfectly fine.

AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template to Setup S3 CRR

Parameters:
  SourceBucketName:
    Type: String
    Description: Primary Bucket Name
    Default: cloudkatha-web-bucket-primary
  ReplicaBucket1:
    Type: String
    Description: Replica Bucket 1
    Default: cloudkatha-web-bucket-replica1
  ReplicaBucket2:
    Type: String
    Description: Replica Bucket 2
    Default: cloudkatha-web-bucket-replica2
Resources:
  PrimaryBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref SourceBucketName
      VersioningConfiguration:
        Status: Enabled
      ReplicationConfiguration:
        Role: !GetAtt S3ReplicationRole.Arn
        Rules:
          - Id: Rule1
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 1
            Filter:
              Prefix: ''
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket1}'
          - Id: Rule2
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 2
            Filter:
              Prefix: ''
            Destination:
              Bucket: !Sub 'arn:aws:s3:::${ReplicaBucket2}'
  S3ReplicationRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Action:
              - 'sts:AssumeRole'
            Effect: Allow
            Principal:
              Service:
                - s3.amazonaws.com

  S3ReplicationPolicy:
    Type: 'AWS::IAM::Policy'
    Properties:
      PolicyName: S3BucketReplicationPolicy
      Roles:
        - !Ref S3ReplicationRole
      PolicyDocument:
        Statement:
          - Action:
              - 's3:GetReplicationConfiguration'
              - 's3:ListBucket'
            Effect: Allow
            Resource: !Sub 'arn:aws:s3:::${SourceBucketName}'
          - Action:
              - 's3:GetObjectVersion'
              - 's3:GetObjectVersionAcl'
            Effect: Allow
            Resource: !Sub 'arn:aws:s3:::${SourceBucketName}'
          - Action:
              - 's3:ReplicateObject'
              - 's3:ReplicateDelete'
            Effect: Allow
            Resource:
              - !Sub 'arn:aws:s3:::${ReplicaBucket1}'
              - !Sub 'arn:aws:s3:::${ReplicaBucket2}'

Once replication rules was created on the primary bucket, this is how it looked like console.

Conclusion

In this quick fix post, we learnt that when you get Number of distinct destination bucket ARNs cannot exceed 1 error while setting up CRR using CloudFormation, it’s the schema version which is culprit.

You are required to use V2 of schema and you can do that adding a Filter property on the rule.

We also saw that even if you want the rule to apply on whole bucket, still you add a Filter with empty prefix which is equivalent to applying on whole bucket.

I hope you were able to solve this issue. Please let me know in comment section if you are still facing an issue.

Enjoyed the content?

Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.

Don’t forget to motivate me by-

• Adding a comment below on what you liked and what can be improved.
• Follow us on
  • Facebook
  • Twitter
  • LinkedIn
  • Instagram
• Share this post with your friends

Suggested Read:

• Create a DynamoDB table using CloudFormation
• 5 Ways to Create and Manage your AWS Resources
• AWS S3 Storage Classes: Everything you need to know