Amazon SQS

What it is

Amazon Simple Queue Service (Amazon SQS) is a fully managed message queue. A producer sends messages to a queue, and consumers poll the queue, process each message, and then delete it. The queue sits between the two, so the producer never waits for the consumer: if consumers are slow, busy, or down, messages wait safely in the queue until a consumer is ready.

That is why SQS is the usual answer when a question asks you to decouple two tiers, absorb a traffic spike, or keep work from being lost when a component fails.

Standard and FIFO queues

	Standard queue	FIFO queue
Throughput	Nearly unlimited API (Application programming interface)A defined way for one program to ask another for data or actions, such as a set of HTTPS requests and responses. Every AWS service is controlled through its API, which the console, the CLI, and the SDKs call for you.Glossary entry calls per second	300 API calls per second per action, or 3,000 messages per second with batches of 10; high throughput mode goes much higher
Delivery	At least once: a message is occasionally delivered more than once	Exactly-once processingEach message is delivered and processed once, with no duplicates. Amazon SQS FIFO queues remove duplicate messages sent within a 5-minute deduplication interval.Glossary entry: duplicates sent within 5 minutes are removed
Order	Best effort	FIFO (First in, first out)Processing items in the order they arrived. Amazon SQS FIFO queues keep messages in order within a message group and remove duplicates.Glossary entry within each message group
Name	Any name	Must end in `.fifo`

Design consumers of a standard queue to be IdempotencyAn operation is idempotent when repeating it has the same effect as doing it once, such as setting a value rather than adding to it. It makes retries safe; many AWS APIs accept a client token for this.Glossary entry, so processing a message twice does no harm. Choose FIFO when the order of operations matters, such as price updates or commands that must run in sequence, or when duplicates cannot be tolerated.

On a FIFO queue:

The message group ID (required) sets the unit of ordering. Messages in one group are delivered one at a time, in order; different groups are processed in parallel, so more groups mean more throughput.
The message deduplication ID, or content-based deduplication (a hash of the body), removes duplicates sent within the 5-minute deduplication interval.

You cannot convert a standard queue into a FIFO queue, or the other way round; create a new queue.

Priority: SQS has no message priority. A receive does not sort by message attributes, and FIFO message groups do not give one group precedence over another. To process urgent work first, use two queues: consumers poll the high-priority queue first and read the normal queue only when the high-priority one is empty.

How a message moves through the queue

Visibility timeout: when a consumer receives a message, the message stays in the queue but is hidden from other consumers for the visibility timeout: 30 seconds by default, up to 12 hours. If the consumer deletes the message in time, it is gone. If not, it becomes visible again and another consumer can retry it. Set the timeout longer than your processing time; a consumer can extend it with ChangeMessageVisibility. Messages processed twice are often a sign that the timeout is too short.
DLQ (Dead-letter queue)A queue that receives messages that could not be processed after a set number of attempts, so they stop blocking other messages and can be inspected later. Amazon SQS, Amazon SNS, AWS Lambda, and Amazon EventBridge can send failed messages to one.Glossary entry (DLQ): a redrive policy moves a message to a DLQ after it has been received maxReceiveCount times without being deleted. This keeps a "poison" message from blocking or repeating forever, and isolates it for analysis. The DLQ must be the same type (standard or FIFO), account, and AWS RegionA separate geographic area where AWS runs its services, such as us-east-1 in Northern Virginia, made up of several Availability Zones. Most resources live in one Region; you choose Regions for latency, data residency, and the services you need.Glossary entry as its source queue. After fixing the problem, DLQ redrive moves the messages back to the source queue.
Retention: messages are kept for 4 days by default, from 1 minute up to 14 days. Give the DLQ a longer retention than the source queue, because a standard queue's message keeps its original timestamp when it is moved.
Delay queues and message timers: a delay hides new messages for up to 15 minutes, for the whole queue or per message. For longer or recurring delays, use EventBridge Scheduler.
Message size: up to 1 MiB. For bigger payloads, the Extended Client Library stores the payload in Amazon S3 and sends a reference, up to 2 GB.

An SQS message's life. A producer sends a message to an SQS queue. A consumer receives it, and the message stays hidden from other consumers for the visibility timeout; the consumer deletes it when processing succeeds. A message received maxReceiveCount times without being deleted moves to a dead-letter queue, and after a fix, redrive moves it back to the source queue.

Short and long polling

With short polling (the default), ReceiveMessage checks a subset of servers and returns immediately, often with nothing. With long polling, the request waits up to 20 seconds for a message to arrive. Long polling reduces empty responses, which lowers cost and gets messages to consumers as soon as they arrive. Turn it on by setting the receive wait time above 0.

Consumers: Lambda and Auto Scaling groups

AWS Lambda can poll a queue for you through an event source mapping and invoke your function with batches of messages. Set the queue's visibility timeout to at least six times the function timeout, and a maxReceiveCount of at least 5 before messages go to the DLQ. With partial batch responses, only the failed messages in a batch return to the queue.
EC2 Auto Scaling groups of workers scale best on backlog per instance: the number of visible messages divided by the number of running instances. A policy based only on queue length ignores how many instances are already working on it.

Security, resilience, performance, and cost

Security

IAM policies control who can send, receive, and manage queues. A queue access policy (resource-based) grants access to other accounts and to AWS services, for example letting an SNS topic, an S3 event notification, or an EventBridge rule send messages to the queue.
Cross-account access: the queue policy in the queue's account must allow the other account's PrincipalA person or workload that can make requests to AWS, such as an IAM user, a role session, the root user, a federated user, or an AWS service. Resource-based policies name the principals they allow.Glossary entry, and that principal's own identity-based policy must also allow the action on the queue's ARN (Amazon Resource Name)The unique identifier of an AWS resource, in the form arn:partition:service:region:account-id:resource. IAM policies use ARNs to name the resources they allow or deny.Glossary entry; both must allow it. Use an IAM role, not shared access keys. SQS queues are not shared through AWS RAM.
Encryption at restEncrypting data where it is stored, such as on disks, in databases, or in backups, so that it cannot be read without the key. On AWS it usually uses keys in AWS KMS.Glossary entry: SSE (Server-side encryption)Encryption that a service applies when it stores your data and removes when you read it back. Amazon S3 encrypts every new object with SSE-S3 by default; SSE-KMS uses keys in AWS KMS instead.Glossary entry protects message bodies, either with SQS-managed keys (SSE-SQS, the default for new queues, at no extra cost) or with AWS KMS keys (SSE-KMS). SSE does not encrypt queue or message metadata, such as attributes. Encrypted queues require HTTPS (HTTP Secure)HTTP sent over a TLS-encrypted connection, so that requests and responses cannot be read or changed on the way, and the server proves its identity with a certificate.Glossary entry requests.
With SSE-KMS, AWS services that send to the queue, such as SNS or S3, need a Customer managed keyA KMS key that you create, own, and manage: you control its key policy, rotation, and deletion. AWS managed keys, in contrast, are created and managed for you by an AWS service.Glossary entry whose key policy allows them to use it, and consumers need kms:Decrypt.
Private access: an interface VPC endpointA private connection from a VPC to an AWS service or another endpoint service without crossing the internet. Gateway endpoints, for Amazon S3 and DynamoDB, are route table targets; interface endpoints are network interfaces that use AWS PrivateLink.Glossary entry (AWS PrivateLink) lets instances in SubnetA range of IP addresses in a VPC, inside one Availability Zone. A subnet is public when its route table sends internet traffic to an internet gateway, and private when it does not.Glossary entry use SQS without an internet gateway or NAT (Network address translation)Rewriting the IP addresses in packets as they pass through a device, most often so that many private addresses can share one public address to reach the internet.Glossary entry device. A queue policy can deny requests that do not come through the endpoint. SQS has no gateway endpoint; gateway endpoints exist only for Amazon S3 and DynamoDB.

Resilience

Standard queues store multiple copies of each message across multiple AZ (Availability Zone)An isolated location within an AWS Region, made up of one or more data centers with their own power, cooling, and networking. The AZs in a Region are linked by fast, low-latency connections, so spreading resources across them protects against the failure of one site.Glossary entry.
The queue itself is the resilience pattern: when a consumer fails, messages wait and become visible again after the visibility timeout. DLQs catch messages that keep failing.
On FIFO queues, a message that keeps failing blocks its message group until it is deleted, expires, or moves to a DLQ. A DLQ breaks the strict order, so leave it out only when order matters more than progress.

Performance

Standard queues scale automatically; add consumers to process faster.
FIFO throughput depends on the number of message groups and on batching; turn on high throughput mode for higher limits.
Batch sends, receives, and deletes (up to 10 messages per call) and use long polling to reduce calls.
A standard queue allows about 120,000 in-flight messages (received but not yet deleted); delete messages promptly.

Cost

You pay per request, and each 64 KB chunk of payload counts as one request, so a 1 MiB payload is 16 requests. FIFO requests cost more than standard ones.
The first 1 million requests each month are free.
Long polling and batching cut the number of requests. Data transfer within a Region is free; KMS calls and S3 storage for large payloads are charged separately.

Monitoring

Amazon CloudWatch receives queue metrics every minute at no charge. The key ones are:
- ApproximateNumberOfMessagesVisible: the backlog. Alarm on it, and on the DLQ, to catch failures.
- ApproximateAgeOfOldestMessage: how far behind consumers are.
- NumberOfEmptyReceives: a sign that you should use long polling.
- ApproximateNumberOfMessagesNotVisible: messages in flight.
AWS CloudTrail records queue API calls.

SQS or another service?

If the requirement is	Choose
Buffer work between two tiers; consumers pull and delete	Amazon SQS
Send the same message to several consumers	SNS topic fanning out to several SQS queues
Several applications read the same ordered records, or replay them	Amazon Kinesis Data Streams
Route events by content from AWS services or SaaS applications	Amazon EventBridge
Coordinate several steps with branching and retries	AWS Step Functions
Move existing ActiveMQ or RabbitMQ applications without rewriting them	Amazon MQ

Reverse mindmap: when the question points here

Is it Amazon SQS?

Yes, when the question says
- Decouple the web tier from the processing tier leads to Amazon SQS
- Absorb spikes so no orders are lost leads to SQS queue between the tiers
- Process in the exact order sent, no duplicates leads to FIFO queue
- Order per customer, but in parallel leads to Message group ID per customer
Tune the queue
- Messages processed twice; processing takes longer than the timeout leads to Increase the visibility timeout
- A bad message fails again and again leads to Dead-letter queue with maxReceiveCount
- Too many empty responses, high request cost leads to Long polling
- Payload larger than 1 MiB leads to Extended Client Library with S3
- Hold each message for 10 minutes before processing leads to Delay queue
- Urgent messages must be processed before normal ones leads to Separate high-priority queue, polled first
Scale and secure it
- Scale EC2 workers with the queue leads to Target tracking on backlog per instance
- Private subnets reach SQS without NAT leads to Interface VPC endpoint
- Encrypt messages with your own key leads to SSE-KMS with a customer managed key
- A role in another account sends to the queue leads to Queue policy plus the role's identity-based policy
No, when the question says
- Every subscriber gets a copy leads to Amazon SNS
- Replay ordered records with several readers leads to Kinesis Data Streams
- Existing JMS or AMQP application leads to Amazon MQ

Key takeaways

SQS decouples components: producers send, consumers poll, process, and delete.
Standard queues are nearly unlimited, at least once, and best-effort order; FIFO queues give order per message group and exactly-once processing at lower throughput.
The visibility timeout hides a message while it is processed; DLQs catch messages that keep failing.
Long polling (up to 20 seconds) and batching reduce cost; scale workers on backlog per instance.
Encrypt with SSE-SQS or SSE-KMS, grant services access with a queue policy, and use an interface VPC endpoint for private access.