SQS Dead Letter Queue

Imagine a situation !!

Your application is consuming messages from the SQS queue and processing them. After the successful processing, you delete the message from the queue.

All good so far 🙂

Now, suppose one of the messages is corrupted and despite multiple tries, your application is not able to process it. This will result in an infinite loop of trying and re-trying and re-trying …

What should you do?

Well, to keep the flow of the queue normal, it’s a good idea to store those messages somewhere for later debugging.

Okay, that’s good !!!

But, where should we store them and how?

The SQS dead letter queue comes to the rescue exactly in this case. AWS allows us to configure a dead letter queue to handle these failed messages.

Later you can check DLQ to debug your application and find the reason for failure.

Now that’s cool right !!!

Don’t want to miss any posts from us? join us on our Facebook group, and follow us on Facebook, Twitter, LinkedIn, and Instagram. You can also subscribe to our newsletter below to not miss any updates from us.

Overall in this post, we will learn-

• What is a dead letter queue?
• What is SQS Dead Letter Queue?
• Features of SQS Dead Letter Queue?
• Benefits of Using SQS Dead Letter Queue
• How does Amazon SQS Dead Letter Queue Work?
• What an SQS DLQ can’t do?

What is a Dead Letter Queue?

A dead letter queue or DLQ is an amazing feature of message queuing systems.

It allows for to storage of messages which meets certain criteria.

for example-

• Incorrect Message Payload
• The maximum receive count reached
• The queue doesn’t exist
• Queue or message length exceeded etc.

What is SQS Dead Letter Queue?

Now just to reiterate what I said earlier, SQS DLQ is a queue which stores failed messages. In other words, SQS DLQ allows us to store messages that couldn’t be processed by a consumer due to whatsoever reasons like corrupted messages or coding errors etc.

To be more specific, SQS DLQ stores the messages which were received successfully by a consumer but could not be processed even after multiple tries.

We have the ability to set the maximum number of times, the consumer needs to try before a message will be moved to DLQ to continue the normal flow.

It can be used by the many source queue to store their failure messages.

However, please note that SQS doesn’t create DLQ for you. If you want to make use of DLQ, you must first create one before trying to send a message to it.

Related: How to Configure SQS Dead Letter Queue using CloudFormation

Features of SQS Dead Letter Queue?

Multiple queues can send messages to a single dead letter queue but, all the queues must be of the same type. In other words, the type of source queue and dead letter queue must be the same.

for example- The standard queue can send messages to standard DLQ and the FIFO queue can send messages to FIFO DLQ.

Must Read: SQS Standard Queue vs FIFO Queue: All you need to know

Both the Source queue and DLQ should be created in the same AWS account and region

Benefits of Using SQS Dead Letter Queue

As DLQ contains messages that could not be processed successfully. So it allows us to-

• Debug our application to find the reason behind message processing failure.
• Handle failed messages in an elegant way instead of piling them up in the queue for an indefinite period.
• Cross-check the message to find a potential reason for failure.
• Maintain queue flow and cope with data loss
• Reduce the number of unprocessed messages in the queue

How does Amazon SQS Dead Letter Queue Work?

SQS DLQ works by a concept known as redrive policy. All the queues which want to send failed messages to the DLQ must set a redrive policy.

Redrive policy contains below things-

• Source Queue
• Dead letter queue for the source queue
• Maximum receive count after which SQS will move a message from source to dead letter queue.

Steps to configure a dead letter queue

1. Go to SQS Service
2. Choose the queue for which you want to enable DLQ
3. click on Edit and scroll down to the dead letter queue section
4. In the DLQ section, enable it, in the chosen queue, select the queue you want to use as DLQ and provide a maximum number of retries and click save to finish. Tada, you are done !!!

What an SQS DLQ can’t do?

The dead letter queue works on receive count and if that doesn’t change, the message will not be moved to the dead letter queue.

Note:

• Do not use a DLQ in case you want to try indefinitely to process a message.
• It’s best to design your application in a way that minimizes the number of messages sent to DLQ or to avoid it altogether.

Conclusion

In this post, we learnt that in order to maintain the normal flow of SQS message processing, the SQS Dead letter queue comes very handy. It lets us store failed messages so that we can debug later the cause of failure.

We also had a look at the features of SQS DLQ, its benefit and how to create one in the AWS console. To sum up, consider using them when you want to segregate failed messages for later processing/debugging.

Enjoyed the content?

Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.

Well, that was my take on Amazon SQS DLQ. If you liked reading my post, you can motivate me by-

• Follow us on
  • Facebook
  • Twitter
  • LinkedIn
  • Instagram
• Share this post with your friends

Suggested Read:

• How to Create an S3 Bucket Using CloudFormation
• Send Message to SQS Queue using Lambda Function and Node JS
• How to Create DynamoDB table using CloudFormation
• Basics Of Cloud Computing for Beginners
• Basics Of Cyber Security for Beginners
• Create a “Hello World” Lambda in AWS in 5 minutes.
• How to create an AWS Lambda in JAVA
• Create an EC2 instance in an existing VPC using CloudFormation
• 5 Ways to create and manage resources in AWS