SQS Dead Letter Queue
Imagine a situation !!
Your application is consuming messages from SQS queue and processing them. After the successful processing, you delete the message from the queue.
All good so far 🙂
Now, suppose one of the message is corrupted and despite multiple tries your application is not able to process it. This will result in an infinite loop of trying and re-trying and re-trying …
What should you do?
Well, to keep the flow of queue normal, it’s a good idea to store those messages somewhere for later debugging.
Okay,that’s good !!!
But, where should we store them and how?
Dead letter queue comes to rescue exactly in this case. AWS allows us to configure a dead letter queue to handle these failed messages.
Later you can check DLQ to debug your application and find the reason of failure.
Now that’s cool right !!!
That was the background of this article. Overall in this post, we will learn-
- What is a dead letter queue?
- What is SQS Dead Letter Queue?
- Features of SQS Dead Letter Queue?
- Benefits of Using SQS Dead Letter Queue
- How does Amazon SQS Dead Letter Queue Work?
- What an SQS DLQ can’t do?
What is a Dead Letter Queue?
Dead letter queue or DLQ is an amazing feature of message queuing systems.
It allows to store messages which meets certain criteria.
- Incorrect Message Payload
- Maximum receive count reached
- Queue doesn’t exist
- Queue or message length exceeded etc.
What is SQS Dead Letter Queue?
Now just to re iterate on what I said earlier, SQS DLQ is a queue which stores failed message. In other words, SQS DLQ allows us to store message that couldn’t be processed by a consumer due to whatsoever reasons like corrupted message or coding errors etc.
To be more specific, SQS DLQ stores the messages which was received successfully by a consumer but could not be processed even after multiple tries.
We have the ability to set that maximum number of times, consumer needs to try before a message will be moved to DLQ to continue the normal flow.
It can be used by the many source queue to store their failure messages.
However, please note that, SQS doesn’t create DLQ for you. If you want to make use of DLQ, you must first create one before trying to send a message to it.
Features of SQS Dead Letter Queue?
Multiple queues can send message to a single dead letter queue but, all the queues must be of same type. In other words, type of source queue and dead letter queue must be same.
for example- Standerd queue can send message to standerd DLQ and FIFO queue can send message to FIFO DLQ.
Both the Source queue and DLQ should be created in the same AWS account and region
Benefits of Using SQS Dead Letter Queue
As DLQ contains the messages that could not be processed successfully. So it allows us to-
- Debug our application to find the reason behind message processing failure.
- Handle failed messages in an elegant way instead of piling them up in the queue for indefinite period.
- Cross check the message to find a potential reason of failure.
- Maintain queue flow and cope with data loss
- Reduce the number of unprocessed message in the queue
How does Amazon SQS Dead Letter Queue Work?
SQS DLQ works by a concept known as redrive policy. All the queue which want to send failed messages to the DLQ must set a redrive policy.
Redrive policy contains below things-
- Source Queue
- Dead letter queue for the source queue
- Maximum receive count after which SQS will move message from source to dead letter queue.
Steps to configure a dead letter queue
- Go to SQS Service
- Choose the queue for which you want to enable DLQ
- click on Edit and scroll down to dead letter queue section
- In the DLQ section, enable it, in the choose queue, select the queue you want to use as DLQ and provide maximum number of retries and click save to finish. Tada,you are done !!!
What an SQS DLQ can’t do?
Dead letter queue works on receive count and if that doesn’t change, the message will not be moved to dead letter queue.
- Do not use a DLQ in case you want to try indefinitely to process a message.
- It’s best to design your application in a way that minimizes the number of message sent to DLQ or to avoid it altogether.
In this post we learnt that in order to maintain the normal flow of SQS message processing, SQS Dead letter queue comes very handy. It let’s us store failed messages so that we can debug later the cause of failure.
We also had a look on feature of sqs DLQ, it’s benefit and how to create one in AWS console. To sum up, consider using them when you want to segregate failed message for later processing/debugging.
Enjoyed the content?
Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.
Well, that was my take Amazon SQS DLQ. If you liked reading my post , you can motivate me by-
- Adding a comment below on what you liked and what can be improved.
- Follow us on
- Share this post with your friends
- How to Create an S3 Bucket Using CloudFormation
- How to Create DynamoDB table using CloudFormation
- Basics Of Cloud Computing for Beginners
- Basics Of Cyber Security for Beginners
- Create a “Hello World” Lambda in AWS in 5 minutes.
- How to create an AWS Lambda in JAVA
- Create an EC2 instance in an existing VPC using CloudFormation
- 5 Ways to create and manage resources in AWS