Amazon S3 or Simple Storage Service is a widely used object storage service. When you are storing your data in S3, by default they are private for security purposes. S3 bucket policy is one of the frequently used ways to provide access to your bucket and objects.

In one of my previous posts, I shared with you “How to Create an S3 Bucket using Terraform” .

In this post, I will show you how to create S3 bucket policy using one of the most popular IaC tools called Terraform. You will also see how the policy is created and attached to the bucket.

What is an S3 Bucket Policy?

An S3 bucket policy is a resource-based IAM policy that you can use to provide access to your s3 bucket and the objects in it.

The meaning of resource-based policy is, that instead of applying the policy to a principal like user/group/role, you apply the policy to the s3 bucket itself on which access is required. While doing so, you specify which principal(user/group/role) is allowed to access the bucket.

• You can add a bucket policy to an S3 bucket to permit other IAM users or accounts to be able to access the bucket and objects in it.
• An S3 bucket can only have a single bucket policy at any point in time.
• Bucket policies are handy to help you set up public access for all users, limited access for an IAM user/role for your account or even cross-account access permissions.

You Might be interested in :

• This is Why S3 Bucket Name is unique Globally
• Is S3 Region specific or Global? What do you Think
• How to Create IAM Role using CloudFormation
• Understand IAM PassRole to Secure Your AWS Infrastructure

An example s3 bucket policy to allow read-only access to everyone

One of the famous labs for beginners in AWS is to setup a static website using S3 bucket. While doing so, you are required to provide public access to everyone so that they can see your website. But mind it, you want to give them read-only access and not write access. You don’t want them to go and change your website, do you?

Of course not !!

Let’s see how we can define a bucket policy on a bucket named cloudkatha-bucket. If you look closely at the actions list, two permissions are present.

• s3:GetObject: To get/read objects in the bucket
• s3:ListBucket: To list objects in the bucket

{ 
  "Statement": [
    {
      "Sid": "AllowEveryoneReadOnlyAccess",
      "Effect": "Allow",
      "Principal": "*",
      "Action": [ "s3:GetObject", "s3:ListBucket" ],
      "Resource": ["arn:aws:s3:::cloudkatha-bucket",
                   "arn:aws:s3:::cloudkatha-bucket/*"]
    }
  ]
}

The above policy says that Principle ‘*’ , means everyone can do actions list bucket and get an object on the resource bucket cloudkatha-bucket and all objects in this bucket.

Now that we know a bit about bucket policy, let’s know about terraform bit and how you can use terraform an s3 bucket policy. Alright?

Let’s go…

What is Terraform?

Terraform is a very popular and open-source Infrastructure as Code(IaC) tool by HashiCorp.

• You can use it to provision, update and version your infrastructure in an efficient manner
• You declare your required infrastructure in a configuration file and terraform creates it in the correct order.
• Configuration files are in a human-readable format using HashiCorp Configuration Language(HCL) or even JSON is supported.
• Terraform is Cloud Agnostic and supports numerous cloud providers like AWS, Azure, GCP etc.

How Do You Create a Resource Using Terraform on AWS?

Unlike CloudFormation, you need to install terraform in your system before you can use it to create a resource like an S3 bucket policy on your cloud provider(In our case AWS).

Once installed, you create your configuration file(file-name.tf – they have .tf extension), and use the below set of commands to deploy your resources.

$ terraform init
$ terraform plan
$ terraform apply
$ terraform destroy

I highly recommend you check my step-by-step guide to help you get started with terraform on AWS in the right way. Here is the link to the post-Getting Started With Terraform on AWS In Right Way

By this time, I assume you already know how to deploy a resource on AWS using Terraform.

Alright, let’s get started with S3 bucket policy.

Prerequisite:

• An AWS Account: Setup Free Tier Account on AWS In Right Way
• Required Permission to Create S3 Bucket and Policy
• AWS CLI
• An Editor Like Notepad or VS Code

How to Create S3 Bucket Policy using Terraform

When it comes to creating an S3 bucket policy using terraform, there are two ways in which you can do it-

• Using policy parameter in the resource aws_s3_bucket
• Creating a aws_s3_bucket_policy resource(recommended)

As of now, specifying policy in the aws_s3_bucket resource is the old way of doing it and is already deprecated. So it’s recommended to use the stand-alone aws_s3_bucket_policy resource to create an s3 bucket policy and attach it to a bucket.

Here is what a simple s3 bucket policy resource looks like-

resource "aws_s3_bucket_policy" "public_read_access" {
  bucket = aws_s3_bucket.demo-bucket.id
  policy = <Your JSON Policy>
}

Now you can specify your policy like –

  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
	  "Principal": "*",
      "Action": [ "s3:*" ],
      "Resource": [
        "${aws_s3_bucket.demo-bucket.arn}",
        "${aws_s3_bucket.demo-bucket.arn}/*"
      ]
    }
  ]
}
EOF

or

this-

data.aws_iam_policy_document.public_read_access.json

Important Note

Please note that in the policy you can specify your policy as multiple strings using heredoc string format. But when your policy becomes complicated or you want to reuse your policy then it becomes difficult. In those cases, it is recommended to use aws_iam_policy_document data source which can generate a JSON policy document for you.

aws_iam_policy_document generates an IAM policy document in JSON format for use with resources that expect policy documents such as aws_iam_policy.

Steps to Create an S3 Bucket using Terraform

1. Create a Working Directory/Folder
2. Create your Bucket Policy Configuration File
3. Initialize Your Directory to Download AWS Plugins
4. Plan and Deploy

Step 1: Create a Working Directory/Folder

Create a folder in which you’ll keep your s3 bucket policy terraform configuration file.

Step 2: Create your Bucket Policy Configuration File

Navigate inside the folder and create your configuration file. You can name it as per your wish, but to keep things simple, I will name it main.tf

I have started with just provider declaration which specifies that we are using AWS provider. Also, it specifies the credential profile that will be used to authenticate to AWS and the region in which resources are to be created by default.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }

  required_version = ">= 0.14.9"
}

#Provider profile and region in which all the resources will create
provider "aws" {
  profile = "default"
  region  = "ap-south-1"
}

Now let’s add an s3 bucket and an s3 bucket policy resource.

S3 Bucket Policy using heredoc string format:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }
}

provider "aws" {
  profile = "default"
  region  = "ap-south-1"
}


#S3 Bucket on Which we will add policy
resource "aws_s3_bucket" "demo-bucket"{
  bucket = "ck-demo-bucket-04th"
}

#Resource to attach a bucket policy to a bucket 
resource "aws_s3_bucket_policy" "public_read_access" {
  bucket = aws_s3_bucket.demo-bucket.id
  policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
	  "Principal": "*",
      "Action": [ "s3:*" ],
      "Resource": [
        "${aws_s3_bucket.demo-bucket.arn}",
        "${aws_s3_bucket.demo-bucket.arn}/*"
      ]
    }
  ]
}
EOF
}

S3 Bucket Policy using reusable aws_iam_policy_document

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }
}

provider "aws" {
  profile = "default"
  region  = "ap-south-1"
}


#S3 Bucket on Which we will add policy
resource "aws_s3_bucket" "demo-bucket"{
  bucket = "ck-demo-bucket-04th"
}

#Resource to add bucket policy to a bucket 
resource "aws_s3_bucket_policy" "public_read_access" {
  bucket = aws_s3_bucket.demo-bucket.id
  policy = data.aws_iam_policy_document.public_read_access.json
}

#DataSource to generate a policy document
data "aws_iam_policy_document" "public_read_access" {
  statement {
    principals {
	  type = "*"
	  identifiers = ["*"]
	}

    actions = [
      "s3:GetObject",
      "s3:ListBucket",
    ]

    resources = [
      aws_s3_bucket.demo-bucket.arn,
      "${aws_s3_bucket.demo-bucket.arn}/*",
    ]
  }
}

You can use any of the above to create a bucket policy. However, keep in mind what I told above about reusability and complexity. In which case, aws_iam_policy_document to generate JSON policy document is always a better choice. That’s the reason, I will go with aws_iam_policy_document way.

Step 3: Initialize Your Directory to Download AWS Plugins

You only do this step once per folder/directory. This basically means you are downloading relevant codes/plugins for your mentioned provider which in our case is AWS.

terraform init

once you hit enter,

Your wording directory gets initialized with the provider-related code and is ready to deploy a resource.

Step 4: Plan and Deploy

The configuration file is created and the directory is initialized. That means we are all ready to deploy our s3 bucket policy.

Now, if you want, you can run the command terraform plan to see what’s actually being created.

terraform plan

Using terraform plan shows what you are going to create-

However, to keep things simple, I just run terraform apply. Ideally, terraform runs terraform plan every time you hit command terraform apply. Once you review the plan and confirm yes then only resources will be created.

Terraform will look for .tf file and show you what’s being created.

Review the output and if all is fine say yes to the bucket policy creation.

Once I typed terraform apply and hit enter, within a few seconds only my policy got attached to my bucket.

As you can see bucket is created and the policy is attached.

As I said, I used aws_iam_policy_document to generate the JSON policy document., However, if you used the here doc multi-line string format- it will still work fine and this is how it looks like during the plan stage.

Validate S3 Bucket Policy Creation

Login to AWS Management Console and navigate to S3 service. Click on your bucket name and click on the Permissions tab as shown below screenshot-

Scroll down to the Bucket policy section and you will see our public read-only policy got attached to our bucket.

Final Configuration File to Create S3 Bucket Policy using Terraform

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.27"
    }
  }
}

provider "aws" {
  profile = "default"
  region  = "ap-south-1"
}


#S3 Bucket on Which we will add policy
resource "aws_s3_bucket" "demo-bucket"{
  bucket = "ck-demo-bucket-04th"
}

#Resource to add bucket policy to a bucket 
resource "aws_s3_bucket_policy" "public_read_access" {
  bucket = aws_s3_bucket.demo-bucket.id
  policy = data.aws_iam_policy_document.public_read_access.json
}

#DataSource to generate a policy document
data "aws_iam_policy_document" "public_read_access" {
  statement {
    principals {
	  type = "*"
	  identifiers = ["*"]
	}

    actions = [
      "s3:GetObject",
      "s3:ListBucket",
    ]

    resources = [
      aws_s3_bucket.demo-bucket.arn,
      "${aws_s3_bucket.demo-bucket.arn}/*",
    ]
  }
}

Clean Up

Finally, if you are doing this exercise for learning purposes, you can clean up by destroying the created resource.

terraform destroy

Type yes, and hit enter

Once you hit enter, your resources get destroyed. Once done, this is how you see the destruction complete message.

Conclusion

In this article, we learnt How to Create S3 Bucket Policy using Terraform. You can use the standalone resource aws_s3_bucket_policy to create a policy or use policy parameter in the resource aws_s3_bucket .

However, using policy parameter on bucket resource is deprecated and it is recommended to use aws_s3_bucket_policy resource to attach an s3 bucket policy to your bucket.

I hope you were able to work up with me and able to create the s3 bucket policy. If you get stuck at any time feel free to add a comment. I will reply to your query asap.

Well, That was my take on “How to Create S3 Bucket Policy using Terraform“. Please feel free to share your feedback.

Enjoyed the content?

Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.

If you liked reading my post, you can motivate me by-

• Adding a comment below on what you liked and what can be improved.
• Follow us on Facebook, Twitter, LinkedIn, Instagram
• Share this post with your friends and colleagues.

Suggested Read:

• AWS S3 Encryption: All You Need to Know
• Amazon S3 Storage Classes: Everything You need to Know