How to Check If a File Exists in S3 Bucket Using Nodejs

Dear Reader, I hope you are doing well today. A few days ago, I wrote a post on how to check if a key exists in an S3 bucket using Boto3 library. In this post, I am here to talk about how to check If a file exists in S3 Bucket using Nodejs.

Amazon S3: Amazon S3 is one of the widely used AWS services. As the name says, it stores everything as an object. You can find your object uniquely using the key.

If you want to check if a file/object is there in the S3 bucket. You can do so by checking if the key exists in the bucket instead of retrieving it. We’ll learn to do it using Node.js today which will use AWS SDK for Javascript under the hood to make the API call.

Node.js: Node.js is a cross-platform, open-source server environment. It simply lets you run javascript code on the server.

In this example tutorial using Node.js, you’ll learn to find out if a file exists in s3 bucket or not.

Don’t want to miss any posts from us? join us on our Facebook group, and follow us on Facebook, Twitter, LinkedIn, and Instagram. You can also subscribe to our newsletter below to not miss any updates from us.

Prerequisite

• An AWS Account: Set up AWS Free Tier Account in Right Way
• Node.js
• AWS Credentials
• An IDE like VS Code

How to Check If a File Exists in S3 Bucket Using Nodejs

While working with Node.js, you can use AWS SDK for Javascript to check If a file exists in the S3 bucket. The AWS SDK for Javascript makes it pretty straightforward for you to work with the AWS service of your choice for example S3.

Now coming to the ways in which you can find out if an object or file is present inside an s3 bucket. There are many as you would guess, however not all solutions are fit for all the use cases.

Here I am sharing a few ways that work and might fit your use case.

Note: Before using any of the below methods, after creating your node.js project install AWS SDK by using npm install aws-sdk command.

Method 1: Using AWS.S3.headObject Method

This is by far the simplest and most efficient way to check if a file exists in an S3 bucket.

In the below section, we are using the AWS.S3.headObject method to find out if a key exists in a bucket.

const AWS = require('aws-sdk');
const s3 = new AWS.S3();

const demoBucketName = 'ck-demo-bucket-18th';
const demokey = 'terraform.drawio.pngg';

const params = {
  Bucket: demoBucketName,
  Key: demokey,
};

s3.headObject(params, (err, data) => {
  if (err) {
    if (err.code === 'NotFound') {
      console.log('File does not exist in S3 Bucket');
    } else {
      console.log('Error Occured While headObject():', err);
    }
  } else {
    console.log('File exists');
  }
});

Code Explanation:

• Firstly we are importing AWS SDK and creating a client of S3.
• Then we are defining two variables to pass the bucket name and the key we are trying to find out.
• Next, we have prepared the params using bucketName and Key to pass to headObject method.
• Our method call errors out with an error code NotFound in case the key is not found. If the key is there, the call will succeed and then print File exists.

Success Output: File exists

Success Response:

{
  AcceptRanges: 'bytes',
  LastModified: 2023-05-02T12:23:22.000Z,
  ContentLength: 39878,
  ETag: '"178164a990002f87e297319f6cebb5c1"',
  VersionId: '0Eu9OSWya9O1PYfAiqhXyWXbIIz.s4nd',
  ContentType: 'image/png',
  ServerSideEncryption: 'AES256',
  Metadata: {}
}

Error Output: File does not exist in S3 Bucket

Error Response:

If you try to print the error response as –

{
  "message": null,
  "code": "NotFound",
  "region": null,
  "time": "2023-08-02T11:30:16.274Z",
  "requestId": "2JD73ZHHY9NDS2G8",
  "extendedRequestId": "W74eIZYM/S1bqsojQ+iBKR7Fr7O0OP9A/7n9xeQiXsqBYLVt7k9jO6xeOaq/yj0uhgdxyj2NTdQ=",
  "statusCode": 404,
  "retryable": false,
  "retryDelay": 130.29666849225978
}

This is what it looks like-

Using the above error, you can catch a code NotFound and know if the file doesn’t exist. On the other hand, If you get 403 you know its access denied. and the same applies to any other 4XX error.

Method 2: Using list_objects_v2 Method

Usually, you are interested in one single object. And to know if one key/file exists in a bucket headObject is almost always the first choice. However, if there is a requirement to see if a folder exists in s3 or not, how would you do that?

Well, you can use list_objects_v2 and specify the folder name as a prefix. For example, I have a logs folder in my s3 bucket, If I want to make sure that logs folder exists in my bucket using Node.js, I can do it using the below code.

const AWS = require('aws-sdk');
const s3 = new AWS.S3();

const demoBucketName = 'ck-demo-bucket-18th';
const demokey = 'logs';

const params = {
  Bucket: demoBucketName,
  Prefix: demokey,
};

s3.listObjectsV2(params, function (err, data) {
    if (err) {
        console.log(err, err.stack);
    } else {
        console.log('response',data);

        if (data.Contents && data.Contents.length) {
            console.log('Prefix exists');
          } else {
            console.log('Prefix does not exist');
          }

    }
});

Output:

Prefix exists

Explanation:

• We are creating a new instance of the AWS.S3 client so that we can call listObjectsV2 a method on it.
• We are passing the bucket name and prefix/folder that we want to check in the parameter
• If the folder/prefix exists, then the listObjectsV2 method will return an object with the key logs/ in the Contents array. If the folder/prefix does not exist, then the listObjectsV2 method will return an empty Contents array.
• We are checking the length of Contents array to determine if the prefix exits in the bucket. If the Contents array is empty that means the prefix doesn’t exist.

Here is an example of the output of the code:

Success Output – Prefix Exists

Success Response:

{
  IsTruncated: false,
  Contents: [
    {
      Key: 'logs/',
      LastModified: 2023-05-02T12:56:59.000Z,
      ETag: '"d41d8cd98f00b204e9800998ecf8427e"',
      ChecksumAlgorithm: [],
      Size: 0,
      StorageClass: 'STANDARD'
    }
  ],
  Name: 'ck-demo-bucket-18th',
  Prefix: 'logs',
  MaxKeys: 1000,
  CommonPrefixes: [],
  KeyCount: 1
}

As you can see the Contents fiend has the details about the key/prefix.

Error Output – Prefix Doesn’t Exist

Error Response:

{
  IsTruncated: false,
  Contents: [],
  Name: 'ck-demo-bucket-18th',
  Prefix: 'logsgggg',
  MaxKeys: 1000,
  CommonPrefixes: [],
  KeyCount: 0
}

If you look at the above response snippet, the Contents array is empty in the response.

And, this is the exact logic we have used in our code. If for a key/prefix, Contents is returned that means it exists in the s3 bucket otherwise it doesn’t.

Method 3: Using the AWS.S3.getObject method:

There are times when we want to know if an object exists in s3 and then we want to get it or retrieve it. Instead of making two subsequent calls why not combine both of them and just call s3.getObject.

If the file exists, you will get the content as a response. If it doesn’t, an error with err code NoSuchKey is thrown and you can catch the same to know it doesn’t exist.

In the below code, we are doing the same to find if a file exists in the s3 bucket from our Node.js program.

const AWS = require('aws-sdk');
const s3 = new AWS.S3();

const demoBucketName = 'ck-demo-bucket-18th';
const demokey = 'terraform.drawio.pngg';

const params = {
  Bucket: demoBucketName,
  Key: demokey,
};

s3.getObject(params, (err, data) => {
  if (err) {
    //console.log('Error response', err);
    if (err.code === 'NoSuchKey') {
      console.log('File does not exist in S3 Bucket');
    } else {
      console.log('Error Occured While getObject():');
    }
  } else {
    //console.log('Suceess response',data)
    console.log('File exists');
  }
});

Success Output: File exists

Success Response:

{
  AcceptRanges: 'bytes',
  LastModified: 2023-05-02T12:23:22.000Z,
  ContentLength: 39878,
  ETag: '"178164a990002f87e297319f6cebb5c1"',
  VersionId: '0Eu9OSWya9O1PYfAiqhXyWXbIIz.s4nd',
  ContentType: 'image/png',
  ServerSideEncryption: 'AES256',
  Metadata: {},
  Body: <Buffer 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 00 00 02 63 00 00 01 60 08 06 00 00 00 f7 b4 7e 4e 00 00 00 01 73 52 47 42 00 ae ce 1c e9 00 00 06 68 ... 39828 more bytes>
}

Error Output: File does not exist in S3 Bucket

Error Response:

{
  code: 'NoSuchKey',
  region: null,
  time: 2023-08-02T17:29:29.414Z,
  requestId: '9S22YMANJ7CGF1QR',
  extendedRequestId: 'SqN9UdJI+DM7ZPNqB7KKrVHuCizt8Ip0kVT0b7mY8ujONVWn2bLXQ3dROF1qPCB+XNdS6dBh3O8=',
  cfId: undefined,
  statusCode: 404,
  retryable: false,
  retryDelay: 28.99117439179415
}

Explanation

• We are passing the name of the bucket and the key of the file to the getObject method. The getObject method will return the contents of the file if the file exists, or an error if the file does not exist.
• If an error occurs with the error code NoSuchKey, we know the key doesn’t exist in the bucket. If the call passes without any error, the file exists in the bucket.
• You might get any other error such as 403 in case sufficient permission to getObject is not there.

Best Way to Check If a File Exists in S3 Bucket Using Nodejs?

We saw three different ways/methods to check If a file exists in the S3 bucket. All three ways have their own pros and cons. Finding the best method to use depends on your use case to be honest.

Here’s a very simple comparison of the three methods to make it easier for you to decide:

• listObjectsV2: This method is useful if you want to retrieve a list of all the objects in a bucket with a certain prefix. However, if the bucket has a large number of objects, this method may be slower and consume more resources than the other two methods. Also can be expensive if you have a large bucket.
• headObject: headObject is preferable when all you need is to check if a file exists in a bucket and don’t need to retrieve the contents of the key. It’s faster and consumes lesser resources compared to listObjectsV2 because it only retrieves the metadata for the specified key.
• getObject: getObject is useful if you need to retrieve the contents of a file as well as check if it exists. Obviously, it’s slower and consumes more resources than the other two methods because it retrieves the contents of the key. If the key doesn’t exist, this method throws NoSuchKey error, which you’ll need to catch and handle appropriately.

In general, if you only need to check if a key exists in a bucket and don’t need to retrieve its contents, using headObject is often the best choice because it’s typically faster and consumes fewer resources than the other two methods. However, if you need to retrieve the contents of a key or a list of all the objects in a bucket, you should use getObject or listObjectsV2, respectively.

Conclusion:

In this post, you learnt how to check If a file exists in S3 bucket using Nodejs. We also learnt some important points such as-

• headObject is the recommended as first choice if you just need to check if the file exists and is most efficient.

I hope you found this post helpful. Feel free to drop your questions in the comment section.

Enjoyed the content?

Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.

Don’t forget to motivate me by-

• Adding a comment below on what you liked and what can be improved.
• Follow us on
  • Facebook, Twitter, LinkedIn and Instagram
• Share this post with your friends

Suggested Read:

• Attach an IAM Role to an EC2 Instance using Terraform