How to Create DynamoDB table with Global Secondary Index using CloudFormation?
A few days ago, I wrote a post on Creating a DynamoDB table using CloudFormation. Back then, I didn’t cover Global Secondary Indexes or GSI.
You might ask, why?
Well, I wanted to give GSI special attention as there is a lot to talk about them.
So, here I am with this post. Technically my aim for the post is to help you create a DynamoDB table with Global Secondary Index using CloudFormation.
Apart from that, I will let you know a few issues that you may encounter while adding GSI using CloudFormation and their solutions.
Before we prepare our template, Let’s start with understanding GSI.
Related: How to Create a DynamoDB Global Table using CloudFormation
What is Global Secondary Index?
In DynamoDB, If you want to perform a query on a non-key attribute, you need to create a secondary index.
DynamoDB supports two types of indexes.
- Local Secondary Index
- Global Secondary Index
You can read about secondary indexes in more detail here.
Let’s cover a bit of the Global Secondary Index(GSI) here.
Global Secondary Index
GSI is an index with a partition key and sort key(optional) that can be different from the base table. It can help you speed up queries on non-key attributes thus providing you better flexibility while querying your data. Ideally, your GSI can span across partitions in the base table and thus justifies the name global.
GSI = Partition Key + optional sort key
- You can create a GSI at the time of table creation or later
- You can create maximum of 20 global secondary indexes on a table
Things you need to specify while creating DynamoDB table with GSI using CloudFormation?
- IndexName: The name of your index, must be unique across indexes on this table
- KeySchema: Key schema for your GSI
- Projection: The attributes that you would like to project on your GSI. Can have three values – ALL | INCLUDE | KEYS_ONLY.
- ALL- means, all the attributes from the base table will be projected into your GSI and this is the most expensive.
- KEYS_ONLY- Only the index and primary keys are projected into the index.
- INCLUDE- Keys are always projected but using INCLUDE you can specify list of other non-key attributes that you would like to have in the projection
- ProvisionedThroughput: If you have set up base table with BillingMode: PAY_PER_REQUEST then ProvisionedThroughput can’t be specified otherwise you must specify RCU and WCU using ProvisionedThroughput
Note: When the Projection attribute is set to INCLUDE, this is how you need to write it down in the template.
Projection:
ProjectionType: INCLUDE
NonKeyAttributes:
- Attribute1
- Attribute2
- Attribute3How does a GSI look in a CloudFormation template?
Resources:
EmployeeTable:
Type: AWS::DynamoDB::Table
Properties:
# Various parameters
# .....
GlobalSecondaryIndexes:
-
IndexName: Location-index
KeySchema:
-
AttributeName: LocationId
KeyType: HASH
Projection:
ProjectionType: ALL
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5Note: Please note that when BillingMode is set to PAY_PER_REQUEST, you can not specify ProvisionedThroughput. Either you provide ProvisionedThroughput RCU, WCU values on the base table and GSI both or make BillingMode on the base table to be PAY_PER_REQUEST and don’t specify ProvisionedThroughput anywhere.
Template to Create DynamoDB table with Global Secondary Index using CloudFormation JSON
In this template, we are specifying a table Employee with the primary key as EmployeeId. We are also adding two Global Secondary Indexes named Location-index and Department-index.
AWSTemplateFormatVersion: 2010-09-09
Description: AWS CloudFormation Template To Create a DynamoDB With GSI
Resources:
EmployeeTable:
Type: AWS::DynamoDB::Table
Properties:
TableName: Employee
AttributeDefinitions:
-
AttributeName: EmployeeId
AttributeType: S
-
AttributeName: LocationId
AttributeType: S
-
AttributeName: DepartmentId
AttributeType: S
KeySchema:
-
AttributeName: EmployeeId
KeyType: HASH
GlobalSecondaryIndexes:
-
IndexName: Location-index
KeySchema:
-
AttributeName: LocationId
KeyType: HASH
Projection:
ProjectionType: ALL
-
IndexName: Department-index
KeySchema:
-
AttributeName: DepartmentId
KeyType: HASH
Projection:
ProjectionType: ALL
BillingMode: PAY_PER_REQUEST
Outputs:
Employee:
Description: Table Created using this template.
Value: !Ref EmployeeTableTemplate to Create DynamoDB table with Global Secondary Index using CloudFormation JSON
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description": "AWS CloudFormation Template To Create a DynamoDB With GSI",
"Resources": {
"EmployeeTable": {
"Type": "AWS::DynamoDB::Table",
"Properties": {
"TableName": "Employee",
"AttributeDefinitions": [
{
"AttributeName": "EmployeeId",
"AttributeType": "S"
},
{
"AttributeName": "LocationId",
"AttributeType": "S"
},
{
"AttributeName": "DepartmentId",
"AttributeType": "S"
}
],
"KeySchema": [
{
"AttributeName": "EmployeeId",
"KeyType": "HASH"
}
],
"GlobalSecondaryIndexes": [
{
"IndexName": "Location-index",
"KeySchema": [
{
"AttributeName": "LocationId",
"KeyType": "HASH"
}
],
"Projection": {
"ProjectionType": "ALL"
}
},
{
"IndexName": "Department-index",
"KeySchema": [
{
"AttributeName": "DepartmentId",
"KeyType": "HASH"
}
],
"Projection": {
"ProjectionType": "ALL"
}
}
],
"BillingMode": "PAY_PER_REQUEST"
}
}
},
"Outputs": {
"Employee": {
"Description": "Table Created using this template.",
"Value": {
"Ref": "EmployeeTable"
}
}
}
}Steps to Create DynamoDB table with Global Secondary Index using CloudFormation
Now, we know the basics and we have the template so let’s go and create the stack.
- Grab the YAML or JSON template from above.
- Save the template with .yml or .json as per the choice of template and follow the below steps.
- Login to AWS Management Console, navigate to CloudFormation and click on Create stack
- Click on “Upload a template file”, upload your saved .yml or .json file and click Next
- Enter the stack name and click on Next. In the configuration, keep everything as default and click on Next.
- In the events tab of the stack, you can view the status.
- Once the stack is successfully created, you can go to the DynamoDB service and verify your table and all its GSI.
Important Notes:
- You can create more than one GSI at the time of table creation
- You can not create or delete more than one GSI in a single CloudFormation stack update
- So ideally once your table is created, you can add indexes one by one. If you try to add more than one GSI you will get an error like: cannot perform more than one gsi creation or deletion in a single update
- All the attributes present in the KeySchema(Primary key or Index) must be present in the AttributeDefinitions
- Do not add any extra attribute in AttributeDefinitions
- Do not use BillingMode: PAY_PER_REQUEST and ProvisionedThroughput at a time. Please mind you can only specify one out of two otherwise your stack creation will fail with Property ProvisionedThroughput can’t be used with PAY_PER_REQUEST BillingMode.
- If you are creating indexes on more than one table at a time, add depends on attribute so that they are created sequentially or your stack will fail
I hope you are able to create your table successfully. Even after following all the above, If you face problems, Please feel free to drop in a comment. We will help you at the earliest.
Clean Up
If you are creating this DynamoDB table with GSI for learning purposes. Don’t forget to delete your CloudFormation stack so that your table is deleted and you don’t bear any cost.
Happy Learning !!!
Conclusion
In this post, you learnt to create DynamoDB table with Global Secondary Index using CloudFormation. You also learnt a few issues that might occur during the stack creation and also how to solve them.
I hope you found the post useful.
Enjoyed the content?
Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.
Don’t forget to motivate me by-
- Adding a comment below on what you liked and what can be improved.
- Follow us on
- Share this post with your friends
Suggested Read:
- 5 Ways to Create and Manage your AWS Resources
- AWS S3 Storage Classes: Everything you need to know
- How to Create an EC2 instance in an existing VPC using CloudFormation
- Attach an IAM role to an EC2 instance using CloudFormation
- How to create an S3 bucket using CloudFormation
- Understand IAM PassRole to Secure Your AWS Infrastructure
- Understand IAM PassRole to Secure your AWS Infrastructure
- Serverless Services on AWS with Explanation