How to Create DynamoDB Global Table using CloudFormation
In my previous post, we learnt How to Create DynamoDB Global Table using AWS Console. We also learnt that DynamoDB global tables are helpful when we have our application users spread worldwide. It helps us provide data access at lower latency and protects against regional failure.
In this post, we’ll learn to create DynamoDB global table using CloudFormation. This post is going to be a little long so, grab a cup of coffee and stay tuned with me till the end.
Related Read:
- 5 Ways to Create and Manage AWS Resources
- Create a DynamoDB table using CloudFormation
- Create a DynamoDB table using Terraform
A Bit of Background on Amazon DynamoDB Global Table
Amazon DynamoDB global table was released in Nov 2017 itself. But the CloudFormation support was added in May of 2021 only when the new version of the DynamoDB global table was released.
To our surprise, CloudFormation only supports the new version i.e.(Version 2019.11.21). This means you can’t use CloudFormation to create or manage a Version 2017.11.29 global table. Nevertheless, the CloudFormation support gave us a good chance to manage our global table using CloudFormation. So why wait?
Let’s learn to create DynamoDB global table using CloudFormation.
But hey, before we create our global table, let’s try to understand DynamoDB global table first.
What is DynamoDB Global Table?
DynamoDB global tables allow you to seamlessly replicate your table data across regions.
In the simple terms if I say, a DynamoDB global table is a collection of one or more replica tables, all owned by a single AWS account.
You can create a DynamoDB global table by creating one or more replicas in different regions. When your application writes data into one of the replica tables, DynamoDB replicates your data automatically across all other replicas. Therefore If you have created a DynamoDB global table with three replica regions, at any point in time, these contain identical data.
DynamoDB Global Table Versions
As we saw at the beginning of this post, DynamoDB global table has two versions –
If you are creating global table now, AWS recommends to start with Version 2019.11.21 as it is better and provides more features.
AWS CloudFormation at the moment allows you to create and manage a Version 2019.11.21 global table only.
That means, when today we are going to create a DynamoDB global table using CloudFormation, it is going to be a global table Version 2019.11.21.
Things you Should Know Before Creating a DynamoDB Global Table using CloudFormation –
- The table must have DynamoDB Streams enabled, with the stream containing both the new and the old images of the item.
- All replicas in your global table will have the same billing mode.
- If you use
PROVISIONEDbilling mode, you must provide an auto-scaling configuration via theWriteProvisionedThroughputSettingsproperty. - In AWS CloudFormation, each global table is controlled by a single stack, in a single region, regardless of the number of replicas. That means no matter how many replicas you have in different regions but you only deploy your Cloudformation stack in one region.
- You cannot convert a resource of type AWS::DynamoDB::Table into a resource of type AWS::DynamoDB::GlobalTable by changing its type in your template. Doing so might result in the deletion of your DynamoDB table.
- Having said that, you can use the GlobalTable resource to create a new table in a single Region.
- If you try to create a global table with the same name as an existing table in the same region, your existing table might be deleted
- At least one replica must exist in the same region in which you deploy the CloudFormation stack. For example, if you are deploying your stack from eu-west-1, at least one of the replicas must be in eu-west-1 as well.
Example Use case:
Now that we have got some idea about DynamoDB global tables, let’s see the use-case that I’ll implement today.
I am implementing a DynamoDB global table with two replica regions.
- ap-south-1(Mumbai)
- us-east-1(North Virginia)
And I will be deploying my stack in the Mumbai region.
Steps to DynamoDB Global Table using CloudFormation
- Step 1: Provide Proper Permission
- Step 2: Prepare Your Template
- Step 3: Create a Stack using Prepared Template
- Step 4: Validate Global Table Creation
- Step 5: Cleanup
Step 1: Provide Proper Permission
Before a user/role can create a CloudFormation stack containing various resources like a global table, you need to explicitly provide permission to it, to be able to perform global table creation.
If we look at the documentation for global table creation, the policies sum up to these statements roughly-
dynamodb:CreateTable
dynamodb:UpdateTable
dynamodb:DeleteTable
dynamodb:DescribeContinuousBackups
dynamodb:DescribeContributorInsights
dynamodb:DescribeTable
dynamodb:DescribeTableReplicaAutoScaling
dynamodb:DescribeTimeToLive
dynamodb:ListTables
dynamodb:UpdateTimeToLive
dynamodb:UpdateContributorInsights
dynamodb:UpdateContinuousBackups
dynamodb:ListTagsOfResource
dynamodb:TableClass
dynamodb:TagResource
dynamodb:UntagResource
dynamodb:BatchWriteItem
dynamodb:CreateTableReplica
dynamodb:DeleteItem
dynamodb:DeleteTableReplica
dynamodb:DisableKinesisStreamingDestination
dynamodb:EnableKinesisStreamingDestination
dynamodb:GetItem
dynamodb:PutItem
dynamodb:Query
dynamodb:Scan
dynamodb:UpdateItem
dynamodb:DescribeTableReplicaAutoScaling
dynamodb:UpdateTableReplicaAutoScaling
iam:CreateServiceLinkedRole
kms:CreateGrant
kms:DescribeKey
application-autoscaling:DeleteScalingPolicy
application-autoscaling:DeleteScheduledAction
application-autoscaling:DeregisterScalableTarget
application-autoscaling:DescribeScalingPolicies
application-autoscaling:DescribeScalableTargets
application-autoscaling:PutScalingPolicy
application-autoscaling:PutScheduledAction
application-autoscaling:RegisterScalableTargetOn top of this, you will need cloudformation:* as well to be able to create/update/delete a stack.
Step 2: Prepare Your Template
You can use YAML or JSON for your template. I prefer YAML for writing my templates. But don’t worry, If you want it in JSON, I will provide a JSON template as well.
To create a DynamoDB global table, all you need is an AWS::DynamoDB::GlobalTable resource like the below section-
DemoDynamoDBGlobalTable:
Type: AWS::DynamoDB::GlobalTable
Properties:
.............
.............
Replicas:
- Region: !Ref ReplicaRegion1
- Region: !Ref ReplicaRegion2
.............
.............As you can see above, we are using Replicas parameter to indicate regions in which we want our replicas. This resource is quite similar to the DynamoDB resource apart from a few fields like Tags that go with the replica itself.
The Replicas list must contain at least one element, the region where this stack is going to be deployed. For example, as I will deploy the stack in ap-south-1(Mumbai), I must have an entry in Replicas with the region ap-south-1. You cannot remove the replica in the stack region. We will see later while executing this stack what happens if we remove the region.
Ideally, a simple DynamoDB global table resource looks like below–
DemoDynamoDBGlobalTable:
Type: AWS::DynamoDB::GlobalTable
Properties:
TableName: Employee
BillingMode: PAY_PER_REQUEST
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
AttributeDefinitions:
- AttributeName: EmployeeId
AttributeType: S
KeySchema:
- AttributeName: EmployeeId
KeyType: HASH
Replicas:
- Region: !Ref ReplicaRegion1
- Region: !Ref ReplicaRegion2Please note that I am using just mandatory parameters here and there are quite a few things you can add as well for example encryption, TTL, point-in-time recovery etc.
Template to Create DynamoDB Global Table using CloudFormation: YAML
In this template, we are declaring three parameters. One for table name and two for region names.
AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template to Create DynamoDB Global Table
Parameters:
TableName:
Type: String
Description: DynamoDb Global Table Name
Default: Employee
ReplicaRegion1:
Type: String
Description: Replica Region 1
Default: ap-south-1
ReplicaRegion2:
Type: String
Description: Replica Region 2
Default: us-east-1
Resources:
DemoDynamoDBGlobalTable:
Type: AWS::DynamoDB::GlobalTable
Properties:
TableName: !Ref TableName
BillingMode: PAY_PER_REQUEST
StreamSpecification:
StreamViewType: NEW_AND_OLD_IMAGES
AttributeDefinitions:
- AttributeName: EmployeeId
AttributeType: S
KeySchema:
- AttributeName: EmployeeId
KeyType: HASH
Replicas:
- Region: !Ref ReplicaRegion1
- Region: !Ref ReplicaRegion2
Outputs:
GlobalTableName:
Description: Global table Name
Value: !Ref DemoDynamoDBGlobalTableTemplate to Create DynamoDB Global Table using CloudFormation: JSON
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS CloudFormation Template to Create DynamoDB Global Table",
"Parameters": {
"TableName": {
"Type": "String",
"Description": "DynamoDb Global Table Name",
"Default": "Employee"
},
"ReplicaRegion1": {
"Type": "String",
"Description": "Replica Region 1",
"Default": "ap-south-1"
},
"ReplicaRegion2": {
"Type": "String",
"Description": "Replica Region 2",
"Default": "us-east-1"
}
},
"Resources": {
"DemoDynamoDBGlobalTable": {
"Type": "AWS::DynamoDB::GlobalTable",
"Properties": {
"TableName": "Employee",
"BillingMode": "PAY_PER_REQUEST",
"StreamSpecification": {
"StreamViewType": "NEW_AND_OLD_IMAGES"
},
"AttributeDefinitions": [
{
"AttributeName": "EmployeeId",
"AttributeType": "S"
}
],
"KeySchema": [
{
"AttributeName": "EmployeeId",
"KeyType": "HASH"
}
],
"Replicas": [
{
"Region": {
"Ref": "ReplicaRegion1"
}
},
{
"Region": {
"Ref": "ReplicaRegion2"
}
}
]
}
}
},
"Outputs": {
"GlobalTableName": {
"Description": "Global table Name",
"Value": {
"Ref": "DemoDynamoDBGlobalTable"
}
}
}
}Now we have the template, let’s go and deploy the template.
Step3: Create a Stack using Prepared Template
Now, we know the basics and we have the template so let’s go and create the stack.
- Grab the YAML or JSON template from above as per your convenience.
- Save the template with .yml or .json as per the choice of template and follow the below steps.
- Login to AWS Management Console, navigate to CloudFormation and click on Create stack
- Click on “Upload a template file”, upload your saved .yml or .json file and click Next
- Enter the stack name and click on Next. In configuration, keep everything as default and click on Next.
- In the events tab of the stack, you can view the status.
- Once the stack is successfully created, you can go to DynamoDB service console and verify your global table.
When I first started, I deployed my stack in a different region than both of my replicas and got the below error.
Invalid request provided: The Replicas section must contain an entry for the current region
The error was expected and the region in which you create your global table stack must contain one replica.
After that, another error was when I missed enabling stream.
Error was like – you must specify stream specification-
Resource handler returned message: “Invalid request provided: You must specify StreamSpecification for a global table.” (RequestToken: 83d7818b-7267-7e74-cf72-8ef57862190d, HandlerErrorCode: InvalidRequest)
Once the template had the correct replica region and stream enabled, my global table was successfully created.
Step 4: Validate Global Table Creation
Now that our stack is successfully created, you can open DyanmoDB table and validate global table creation.
Click on table name and go to Global tables tab and there you will find your replica regions. Please note that it is showing one value US East only because Mumbai is quite evident because we are in Mumbai region only right now. If you go to US East, there it will show you Mumbai as the replica.
Congratulation !!! You have successfully created your DynamoDB Global tables using DynamoDB global table for learning purposes. Don’t forget to delete your CloudFormation stack so that your DynamoDB global table is deleted and you don’t bear any cost.
Happy Learning !!!
Conclusion:
In this post, we learnt, How to Create DynamoDB Global Table using CloudFormation. We also learned that in order to create a DynamoDB global table, stream must be enabled on your table.
By having DynamoDB global table, you can improve the user experience of a widely spread global application.
You can use CloudFormation to create and manage Version 2019.11.21 global table only. Your stack must be deployed in one of the replica regions only.
I hope you found this post helpful.
Enjoyed the content?
Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.
Don’t forget to motivate me by-
- Add a comment below on what you liked and what can be improved.
- Follow us on
- Share this post with your friends