| Concept | Description |
|---|---|
| SQS | Fully managed message queuing service for decoupling application components |
| Message Retention | Default 4 days, configurable from 1 minute to 14 days |
| Message Size | 1 byte to 256 KB per message |
| Payload Extension | Use S3 for messages >256 KB with SQS Extended Client Library |
| Queue Limit | Unlimited number of messages in queue |
| Delivery | At-least-once delivery (Standard), exactly-once processing (FIFO) |
| Feature | Details |
|---|---|
| Throughput | Nearly unlimited transactions per second |
| Ordering | Best-effort ordering, no guarantee |
| Delivery | At-least-once delivery, occasional duplicates possible |
| Use Case | High throughput applications where order is not critical |
| Feature | Details |
|---|---|
| Throughput | 300 messages/sec (3000 with batching) |
| High Throughput Mode | 3000 messages/sec (30,000 with batching) |
| Ordering | Strict ordering preserved, first-in-first-out |
| Delivery | Exactly-once processing, no duplicates |
| Naming | Queue name must end with .fifo suffix |
| Message Group ID | Required tag to ensure ordering within group |
| Deduplication ID | Token for 5-minute deduplication interval |
| Content-Based Deduplication | SHA-256 hash of message body for automatic deduplication |
| Parameter | Details |
|---|---|
| Definition | Period when message is invisible to other consumers after being received |
| Default | 30 seconds |
| Range | 0 seconds to 12 hours |
| ChangeMessageVisibility API | Extend timeout if more processing time needed |
| Timeout Expiration | Message becomes visible again if not deleted |
| Concept | Details |
|---|---|
| Purpose | Reduce empty responses and API calls by waiting for messages |
| Wait Time | 1 to 20 seconds (ReceiveMessageWaitTimeSeconds) |
| Benefit | Decreases costs, reduces empty receives |
| Configuration | Queue-level or API-level (ReceiveMessage call) |
| Preferred Over | Short polling (WaitTimeSeconds = 0) |
| Aspect | Details |
|---|---|
| Purpose | Store messages that fail processing after maximum receive count |
| MaximumReceives | Threshold after which message moves to DLQ |
| Queue Type Match | DLQ must be same type as source (Standard or FIFO) |
| Retention | Set DLQ retention longer than source queue for debugging |
| Redrive Policy | JSON configuration specifying DLQ ARN and MaximumReceives |
| Feature | Details |
|---|---|
| Definition | Postpone delivery of messages to consumers |
| Range | 0 to 15 minutes |
| Default | 0 seconds (immediate delivery) |
| Queue-Level | DelaySeconds parameter affects all messages |
| Message-Level | Override queue default with message timer (Standard only) |
| API Call | Function |
|---|---|
| SendMessage | Add single message to queue |
| SendMessageBatch | Add up to 10 messages in single request |
| ReceiveMessage | Poll messages (up to 10) from queue |
| DeleteMessage | Remove message after successful processing |
| DeleteMessageBatch | Remove up to 10 messages in single request |
| ChangeMessageVisibility | Modify visibility timeout for received message |
| PurgeQueue | Delete all messages in queue |
| Concept | Description |
|---|---|
| SNS | Fully managed pub/sub messaging and mobile notifications service |
| Publisher | Application or service that sends messages to topic |
| Topic | Communication channel with access control and delivery policies |
| Subscriber | Endpoint that receives messages from topic |
| Message Size | Maximum 256 KB per notification |
| Fan-Out Pattern | One message to multiple subscribers simultaneously |
| Type | Details |
|---|---|
| Standard | Best-effort ordering, at-least-once delivery, high throughput |
| FIFO | Strict ordering, exactly-once delivery, 300 msg/sec (3000 with batching) |
| FIFO Naming | Topic name must end with .fifo suffix |
| FIFO Subscribers | Only SQS FIFO queues can subscribe to SNS FIFO topics |
| Protocol | Details |
|---|---|
| HTTP/HTTPS | Deliver JSON-encoded message to web server endpoint |
| Send message to registered email address | |
| Email-JSON | Send JSON-formatted message to email |
| SQS | Deliver message to SQS queue |
| Lambda | Invoke Lambda function with message payload |
| SMS | Send text message to phone number |
| Platform Application Endpoint | Mobile push notifications (APNs, FCM, etc.) |
| Feature | Details |
|---|---|
| Filter Policy | JSON policy applied to subscription to filter messages |
| Attributes | Filter based on message attributes (not body) |
| Default Behavior | Without filter, subscription receives all topic messages |
| Operators | Exact match, prefix, numeric ranges, exists check |
| Phase | Details |
|---|---|
| Immediate Retry | No delay between retries, 3 attempts |
| Pre-Backoff | 1 second delay, 10 attempts |
| Backoff | Exponential backoff from 1 sec to 20 sec |
| Post-Backoff | 20 second delay, 100,000+ attempts |
| Total Duration | HTTP/S endpoints: retries over several hours |
| Aspect | Details |
|---|---|
| Pattern | SNS topic delivers message to multiple SQS queues |
| Benefit | Fully decoupled, no data loss, parallel async processing |
| Use Case | Multiple downstream systems need same data |
| Permissions | SQS queue policy must allow SNS SendMessage |
| Cross-Region | SNS can deliver to SQS queues in different regions |
| Pattern | Implementation |
|---|---|
| Load Leveling | SQS queue smooths variable load to steady consumption rate |
| Priority Queue | Use separate queues for different priority levels |
| Request-Response | Use message attributes and correlation ID with two queues |
| Batch Processing | Accumulate messages in queue then process in batches |
| Scenario | Solution |
|---|---|
| No Order Required | Use Standard SQS for maximum throughput |
| Strict Order Required | Use FIFO SQS with Message Group ID |
| Partial Order | FIFO queue with multiple Message Group IDs for parallel processing |
| Order with SNS | SNS FIFO to SQS FIFO preserves order |
| Type | Details |
|---|---|
| In-Transit | HTTPS API endpoints, TLS encryption |
| At-Rest (SQS) | SSE using KMS keys (AWS managed or customer managed) |
| At-Rest (SNS) | SSE using KMS keys for messages stored temporarily |
| Key Types | AWS managed key (aws/sqs or aws/sns) or customer managed CMK |
| Policy Type | Use Case |
|---|---|
| IAM Policy | Control user/role access to SQS/SNS APIs |
| SQS Queue Policy | Grant cross-account access, allow SNS to send messages |
| SNS Topic Policy | Control who can publish or subscribe to topic |
| VPC Endpoint | Private connection from VPC without internet gateway |
| Metric | Description |
|---|---|
| ApproximateNumberOfMessages | Messages available for retrieval |
| ApproximateNumberOfMessagesVisible | Messages in queue not in-flight |
| ApproximateNumberOfMessagesNotVisible | Messages in-flight (sent but not deleted) |
| NumberOfMessagesSent | Messages added to queue |
| NumberOfMessagesReceived | ReceiveMessage API calls returning messages |
| NumberOfMessagesDeleted | Messages deleted from queue |
| ApproximateAgeOfOldestMessage | Age of oldest non-deleted message (seconds) |
| Metric | Description |
|---|---|
| NumberOfMessagesPublished | Messages published to topic |
| NumberOfNotificationsDelivered | Messages successfully delivered |
| NumberOfNotificationsFailed | Messages failed to deliver |
| PublishSize | Size of published messages (bytes) |
| Problem | Solution |
|---|---|
| Duplicate Messages | Use FIFO queue or implement idempotent processing |
| Message Loss | Check DLQ, verify DeleteMessage only after successful processing |
| Slow Processing | Increase visibility timeout, scale consumers horizontally |
| SNS Delivery Failure | Check endpoint availability, subscription filter policy, IAM permissions |
| High Latency | Enable long polling, use batching APIs, check queue depth |
| Strategy | Implementation |
|---|---|
| Message Batching | SendMessageBatch and DeleteMessageBatch APIs |
| Long Polling | Reduce empty receives, set ReceiveMessageWaitTimeSeconds to 20 |
| Message Size | Use S3 for large payloads, reference in message |
| Filter Subscriptions | SNS message filtering reduces unnecessary deliveries |
| Pattern | Implementation |
|---|---|
| Decouple Components | SQS between microservices for loose coupling |
| Handle Failures | DLQ for poison messages, retry logic in consumers |
| Scale Independently | Producers and consumers scale based on different metrics |
| Multi-Region | SNS to SQS queues in multiple regions for DR |
| Event-Driven | SNS triggers Lambda for serverless event processing |
| Use Standard When | Use FIFO When |
|---|---|
| Throughput is priority (unlimited) | Order must be preserved |
| Order is not critical | Exactly-once processing required |
| Can handle duplicates | Duplicates are unacceptable |
| Best-effort ordering sufficient | Sequential operations needed |