AWS SQS Dead Letter Queue

sqs dead letter queue
Sharing is Caring:

SQS Dead Letter Queue

Imagine a situation !!

Your application is consuming messages from SQS queue and processing them. After the successful processing, you delete the message from the queue.

Now, suppose one of the message is corrupted and despite multiple tries your application is not able to process it. This will result in an infinite loop of trying and re-trying…

What should you do?

Well, to keep the flow of queue normal, it’s a good idea to store those messages somewhere for later debugging.

Okay,that’s good !!!

But where should we store them and how?

Dead letter queue comes to rescue exactly in this case. AWS allows us to configure a dead letter queue to handle these failed messages.

Later you can check DLQ to debug your application and find the reason of failure.

Now that’s cool right !!!

That was the background of this article.Overall in this post we will learn-

What is a Dead Letter Queue?

Dead letter queue or DLQ is an amazing feature of message queuing systems.

It allows to store messages which meets certain criteria.

for example-

  • Incorrect Message Payload
  • Maximum receive count reached
  • Queue doesn’t exist
  • Queue or message length exceeded etc.

What is SQS Dead Letter Queue?

Now just to re iterate on what I said earlier, SQS DLQ is a queue which stores failed message. In other words, SQS DLQ allows us to store message that couldn’t be processed by a consumer due to whatsoever reasons like corrupted message or coding errors etc.

To be more specific, SQS DLQ stores the messages which was received successfully by a consumer but could not be processed even after multiple tries.

We have the ability to set that maximum number of times, consumer needs to try before a message will be moved to DLQ to continue the normal flow.

It can be used by the many source queue to store their failure messages.

However, please note that, SQS doesn’t create DLQ for you. If you want to make use of DLQ, you must first create one before trying to send a message to it.

Features of SQS Dead Letter Queue?

Multiple queues can send message to a single dead letter queue but, all the queues must be of same type. In other words, type of source queue and dead letter queue must be same.

for example- Standerd queue can send message to standerd DLQ and FIFO queue can send message to FIFO DLQ.

Must Read: SQS Standard Queue vs FIFO Queue: All you need to know

Both the Source queue and DLQ should be created in the same AWS account and region

Benefits of Using SQS Dead Letter Queue

As DLQ contains the messages that could not be processed successfully. So it allows us to-

  • Debug our application to find the reason behind message processing failure.
  • Handle failed messages in an elegant way instead of piling them up in the queue for indefinite period.
  • Cross check the message to find a potential reason of failure.
  • Maintain queue flow and cope with data loss
  • Reduce the number of unprocessed message in the queue

How does Amazon SQS Dead Letter Queue Work?

SQS DLQ works by a concept known as redrive policy. All the queue which want to send failed messages to the DLQ must set a redrive policy.

Redrive policy contains below things-

  • Source Queue
  • Dead letter queue for the source queue
  • Maximum receive count after which SQS will move message from source to dead letter queue.

Steps to configure a dead letter queue

  1. Go to SQS Service
  2. Choose the queue for which you want to enable DLQ
  3. click on Edit and scroll down to dead letter queue section
  4. In the DLQ section, enable it, in the choose queue, select the queue you want to use as DLQ and provide maximum number of retries and click save to finish. Tada,you are done !!!

What an SQS DLQ can’t do?

Dead letter queue works on receive count and if that doesn’t change, the message will not be moved to dead letter queue.

Note:

  • Do not use a DLQ in case you want to try indefinitely to process a message.
  • It’s best to design your application in a way that minimizes the number of message sent to DLQ or to avoid it altogether.

Conclusion

In this post we learnt that in order to maintain the normal flow of SQS message processing, SQS Dead letter queue comes very handy. It let’s us store failed messages so that we can debug later the cause of failure.

We also had a look on feature of sqs DLQ, it’s benefit and how to create one in AWS console. To sum up, consider using them when you want to segregate failed message for later processing/debugging.

Enjoyed the content?

Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.

Subscribe to our newsletter below to get awesome AWS learning materials delivered straight to your inbox.

Well, that was my take Amazon SQS DLQ. If you liked reading my post , you can motivate me by-

Suggested Read:

Sharing is Caring:

7 thoughts on “AWS SQS Dead Letter Queue

  1. Thanks a lot bro. DLQ things are hard when I work on it. But I got clear idea about DLQ stuffs. If you can do some articles on ELK stack stuffs. It’s important these days.

  2. Preeti nice efforts!!!1 great….
    very much likedthis blog name….cloud katha….imagine…cloud with pandit g….hahaha

        1. Sure Vikas,We have that in our bucket list and we will be coming with real scenario architecture discussion in future.

Leave a Reply

Your email address will not be published. Required fields are marked *