Amazon SQS

Amazon Simple Queue Service (Amazon SQS) is a an AWS message queuing service that enables customers to decouple and scale microservices, distributed systems, and serverless applications. Amazon SQS offers common constructs such as dead-letter queues and cost allocation tags. It provides a generic web services API and it can be accessed by any programming language that the AWS SDK supports.

• Using SQS, AWS customers can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.
• Amazon SQS uses FIFO (first-in-first-out) model, which preserves queues in the exact order in which messages are sent and received. For those customers, who use a FIFO queue don’t have to place sequencing information in their messages.
  • FIFO queues provide exactly-once processing, that means that each message is delivered once and remains available until a consumer processes it and deletes it. Duplicates are not introduced into the queue.
• Standard queues provide at-least-once delivery, because of that each message is delivered at least once.

Amazon SQS FEATURES

AWS customers can control who can send messages to and receive messages from an Amazon SQS queue.

• Server-side encryption (SSE) enables customers to transmit sensitive data by protecting the contents of messages in queues using keys managed in AWS Key Management Service (AWS KMS).
• There is no limit to the number of messages per queue, and standard queues provide nearly unlimited throughput. 
• Costs are based on usage which provides significant cost saving versus the “always-on” model of self-managed messaging middleware. 

Customers can store the contents of messages larger than 256 KB using Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB, with Amazon SQS holding a pointer to the Amazon S3 object, or you can split a large message into smaller messages

• Use Amazon SQS to transmit any volume of data, at any level of throughput, without losing messages or requiring other services to be available. 
• SQS enables customers to decouple application components so that they run and fail independently, increasing the overall fault tolerance of the system. 

Amazon SQS locks customers messages during processing, so that multiple producers can send and multiple consumers can receive messages at the same time.

• It has multiple copies of every message that are stored redundantly across multiple availability zones so that they are available whenever needed.

To ensure the safety of customers’ messages, Amazon SQS stores them on multiple servers. Standard queues support at-least-once message delivery, and FIFO queues support exactly-once message processing.

• Amazon SQS uses redundant infrastructure to provide highly-concurrent access to messages and high availability for producing and consuming messages. 

Amazon SQS can process each buffered request independently, scaling transparently to handle any load increases or spikes without any provisioning instructions.

• Using Amazon SQS, AWS customers can exchange sensitive data between applications using server-side encryption (SSE) to encrypt each message body.
•  Amazon SQS SSE integration with AWS Key Management Service (KMS) allows customers to centrally manage the keys that protect SQS messages along with keys that protect them from other AWS resources. 
• AWS KMS logs every use of customers encryption keys to AWS CloudTrail to help meet your regulatory and compliance needs.

SQS Functionality

• Create unlimited Amazon SQS queues with an unlimited number of message in any region
• Message payloads can contain up to 256KB of text in any format. Each 64KB ‘chunk’ of payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests. To send messages larger than 256KB, you can use the Amazon SQS Extended Client Library for Java, which uses Amazon S3 to store the message payload. A reference to the message payload is sent using SQS.
• Using Batches AWS customers can send, receive, or delete messages in batches of up to 10 messages or 256KB. Batches cost the same amount as single messages, meaning SQS can be even more cost effective for customers that use batching.
• Long polling is used to reduce extraneous polling, and minimize cost while receiving new messages as quickly as possible. When your queue is empty, long-poll requests wait up to 20 seconds for the next message to arrive. Long poll requests cost the same amount as regular requests.
• Retain messages in queues for up to 14 days. Send and read messages simultaneously.
• When a message is received, it becomes “locked” while being processed. This keeps other computers from processing the message simultaneously. If the message processing fails, the lock will expire and the message will be available again

Types Queues

Amazon SQS offers two queue types for different application requirements:

FIFO Queues

FIFO queues are designed to enhance messaging between applications when the order of operations and events is critical, or where duplicates can’t be tolerated. It ensure that user-entered commands are executed in the right order, it displays  the correct product price by sending price modifications in the right order, and prevents a student from enrolling in a course before registering for an account.

• High Throughput: By default, FIFO queues support up to 300 messages per second (300 send, receive, or delete operations per second). When you batch 10 messages per operation (maximum), FIFO queues can support up to 3,000 messages per second. To request a quota increase, file a support request.

• Exactly-Once Processing: A message is delivered once and remains available until a consumer processes and deletes it. Duplicates aren’t introduced into the queue.
• First-In-First-Out Delivery: The order in which messages are sent and received is strictly preserved (i.e. First-In-First-Out).

Standard Queues

AWS customers can use standard message queues in many scenarios, as long as their application can process messages that arrive more than once and out of order, for example: 

It enables customers’ upload media while resizing or encoding it, it processes a high number of credit card validation requests and schedules multiple entries to be added to a database.

• Unlimited Throughput: Standard queues support a nearly unlimited number of transactions per second (TPS) per API action.
• At-Least-Once Delivery: A message is delivered at least once, but occasionally more than one copy of a message is delivered.
• Best-Effort Ordering: Occasionally, messages might be delivered in an order different from which they were sent

Amazon SQS message queuing can be used with other AWS Services such as Redshift, DynamoDB, RDS, EC2, ECS, Lambda, and S3, to make distributed applications more scalable and reliable. Below are some common design patterns:

• Work Queues decouple components of a distributed application that may not all process the same amount of work simultaneously.
• Buffer and Batch Operations adds scalability and reliability to your architecture, and smooth out temporary volume spikes without losing messages or increasing latency.
• Request Offloading enables customers move slow operations off of interactive request paths by queueing the request.
• Fanout combines SQS with Simple Notification Service (SNS) to send identical copies of a message to multiple queues in parallel.
• Priority is used to separate queues prioritization of work.
• Since message queues decouple customers’ processes, it’s easy to scale up the send or receive rate of messages by adding another process.