Skip to main content

Amazon SQS - Interview Preparation

Total Questions: 100 Levels: Core Fundamentals (100), Intermediate (120), Advanced (80)


Table of Contents​

  1. Core Fundamentals
  2. Intermediate Topics
  3. Advanced Topics

1. Core Fundamentals​

Q1: What is Amazon SQS and what are its core features?​

Technical Answer: Amazon SQS is a fully managed message queue service for decoupling and scaling microservices.

Core Features:

  • Standard Queues: Unlimited throughput, at-least-once delivery, best-effort ordering
  • FIFO Queues: Exactly-once processing, strict ordering, 3000 msg/s limit
  • Message Retention: 1 minute to 14 days (default 4 days)
  • Message Size: Up to 256KB per message
  • Visibility Timeout: Period when message is hidden after being received

Basic Operations:

import boto3
import json

sqs = boto3.client('sqs')

# Get queue URL
queue_url = sqs.get_queue_url(QueueName='my-queue')['QueueUrl']

# Send message
response = sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps({'order_id': '123', 'action': 'process'}),
MessageAttributes={
'Priority': {'StringValue': 'high', 'DataType': 'String'}
}
)
print(f"Message ID: {response['MessageId']}")

# Receive message
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20, # Long polling
MessageAttributeNames=['All']
)

messages = response.get('Messages', [])
for msg in messages:
body = json.loads(msg['Body'])
print(f"Received: {body}")

# Delete after processing
sqs.delete_message(
QueueUrl=queue_url,
ReceiptHandle=msg['ReceiptHandle']
)

FIFO Queue Example:

# FIFO queues require MessageGroupId and MessageDeduplicationId
response = sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody=json.dumps({'user_id': '456', 'action': 'update'}),
MessageGroupId='user-456',
MessageDeduplicationId='order-123-created'
)

🔍 Interviewer Notes: Testing message queue fundamentals and SQS specifics.

❌ Common Mistakes:

  • Using Standard queues when ordering required
  • Not setting MessageDeduplicationId for FIFO
  • Not handling visibility timeout correctly

Q2: How does SQS visibility timeout work?​

Technical Answer: Visibility timeout is the period when a received message is hidden from other consumers. If not deleted within this time, it becomes visible again.

Visibility Timeout Flow:

import boto3

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789/my-queue'

# Receive message (starts visibility timeout)
response = sqs.receive_message(
QueueUrl=queue_url,
VisibilityTimeout=30
)

if 'Messages' in response:
message = response['Messages'][0]
receipt_handle = message['ReceiptHandle']

try:
result = process_long_task(message['Body'])
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=receipt_handle)
except Exception as e:
# Don't delete - message will reappear after visibility timeout
print(f"Processing failed: {e}")

Change Visibility Timeout:

# Extend visibility timeout for long-running tasks
sqs.change_message_visibility(
QueueUrl=queue_url,
ReceiptHandle=receipt_handle,
VisibilityTimeout=300 # Extend to 5 minutes
)

# Can extend up to 12 hours from original receive

Dead Letter Queue:

# Configure main queue with DLQ
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={
'RedrivePolicy': json.dumps({
'deadLetterTargetArn': dlq_arn,
'maxReceiveCount': 3 # Move to DLQ after 3 failures
}),
'VisibilityTimeout': '60'
}
)

🔍 Interviewer Notes: Testing message lifecycle and failure handling.

❌ Common Mistakes:

  • Visibility timeout shorter than processing time
  • Not configuring DLQ for failed messages
  • Not handling message redrive from DLQ

Q3: What is the difference between short polling and long polling?​

Technical Answer:

  • Short Polling: Returns immediately, even if no messages (wastes requests)
  • Long Polling: Waits up to specified time for messages (reduces empty responses)

Comparison:

import boto3

sqs = boto3.client('sqs')
queue_url = 'https://sqs.us-east-1.amazonaws.com/123456789/my-queue'

# ❌ SHORT POLLING - Don't do this
response = sqs.receive_message(
QueueUrl=queue_url,
WaitTimeSeconds=0
)

# ✅ LONG POLLING - Recommended
response = sqs.receive_message(
QueueUrl=queue_url,
WaitTimeSeconds=20, # Max 20 seconds
MaxNumberOfMessages=10
)

# Set at queue level
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'ReceiveMessageWaitTimeSeconds': '20'}
)

Benefits:

  • Reduces empty responses (cost savings)
  • Lower latency when messages arrive
  • Fewer API calls

🔍 Interviewer Notes: Testing cost optimization and best practices.

❌ Common Mistakes:

  • Using short polling in production
  • Setting WaitTimeSeconds too low
  • Making frequent receive calls

Q4: How do you implement SQS with Lambda for event-driven processing?​

Technical Answer: SQS triggers Lambda functions to process messages asynchronously.

Event Source Mapping:

import boto3

lambda_client = boto3.client('lambda')

# Create event source mapping
response = lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:us-east-1:123456789:my-queue',
FunctionName='my-processor-function',
BatchSize=10,
MaximumBatchingWindowInSeconds=5,
FunctionResponseTypes=['ReportBatchItemFailures']
)

Lambda Handler for SQS:

import json

def lambda_handler(event, context):
batch_item_failures = []

for record in event['Records']:
try:
message = json.loads(record['body'])
process_order(message['order_id'])
except Exception as e:
batch_item_failures.append({'itemIdentifier': record['messageId']})

# Only failed messages go back to queue
return {'batchItemFailures': batch_item_failures}

FIFO Queue Trigger:

lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:us-east-1:123456789:my-fifo-queue.fifo',
FunctionName='my-fifo-processor',
BatchSize=10
# Lambda automatically handles ordering per MessageGroupId
)

🔍 Interviewer Notes: Testing event-driven architecture and SQS-Lambda integration.

❌ Common Mistakes:

  • Not enabling ReportBatchItemFailures
  • Deleting messages manually in Lambda
  • Not handling partial batch failures

Q5: How do you implement priority queues with SQS?​

Technical Answer: SQS doesn't natively support priorities. Use multiple queues or message attributes.

Multiple Queue Approach:

import boto3
import json

sqs = boto3.client('sqs')

queues = {
'high': 'https://sqs.us-east-1.amazonaws.com/123456789/high-priority',
'medium': 'https://sqs.us-east-1.amazonaws.com/123456789/medium-priority',
'low': 'https://sqs.us-east-1.amazonaws.com/123456789/low-priority'
}

def send_priority_message(message, priority='medium'):
queue_url = queues.get(priority, queues['medium'])

response = sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps(message)
)
return response['MessageId']

# Consumer polls high priority first
def process_priority_messages():
for priority in ['high', 'medium', 'low']:
queue_url = queues[priority]

response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=1
)

if response.get('Messages'):
return priority, response['Messages']

return None, []

Lambda Priority Processing:

# Separate event source mappings with different concurrency
lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:...:high-priority',
FunctionName='priority-processor',
BatchSize=10,
ScalingConfig={'MaximumConcurrency': 100}
)

lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:...:low-priority',
FunctionName='priority-processor',
BatchSize=10,
ScalingConfig={'MaximumConcurrency': 10}
)

🔍 Interviewer Notes: Testing creative problem solving and architectural patterns.

❌ Common Mistakes:

  • Trying to use single queue with priorities
  • Not having separate scaling for priorities
  • Starving low priority messages

Q6: How do you handle message ordering and exactly-once processing?​

Technical Answer: Use FIFO queues for ordering and exactly-once semantics.

FIFO Queue Configuration:

import boto3
import json

sqs = boto3.client('sqs')

# Create FIFO queue
response = sqs.create_queue(
QueueName='orders.fifo',
Attributes={
'FifoQueue': 'true',
'ContentBasedDeduplication': 'false',
'VisibilityTimeout': '300'
}
)

# Send to FIFO queue
response = sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps({'order_id': '123', 'status': 'created'}),
MessageGroupId='customer-456',
MessageDeduplicationId='order-123-created'
)

Exactly-Once Processing:

import boto3

dynamodb = boto3.resource('dynamodb')
processed_table = dynamodb.Table('processed-messages')

def lambda_handler(event, context):
for record in event['Records']:
message_id = record['messageId']
dedup_id = record.get('attributes', {}).get('MessageDeduplicationId', message_id)

# Check if already processed
try:
processed_table.put_item(
Item={'message_id': dedup_id, 'processed_at': datetime.utcnow().isoformat()},
ConditionExpression='attribute_not_exists(message_id)'
)
except dynamodb.meta.client.exceptions.ConditionalCheckFailedException:
print(f"Message {dedup_id} already processed, skipping")
continue

process_message(record['body'])

🔍 Interviewer Notes: Testing distributed systems consistency understanding.

❌ Common Mistakes:

  • Using Standard queues for ordering
  • Not using idempotent consumers
  • Not handling deduplication

Q7: What is the maximum throughput for SQS?​

Technical Answer:

Queue TypeThroughput
StandardNearly unlimited
FIFO300 msg/s (or 3000 with batching)

High-Throughput FIFO:

# Distribute messages across multiple MessageGroupIds
message_groups = [f'group-{i}' for i in range(100)]

def send_high_throughput(messages):
import hashlib

for msg in messages:
# Hash to consistently assign to group
group = message_groups[int(hashlib.md5(msg['id'].encode()).hexdigest(), 16) % 100]

sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody=json.dumps(msg),
MessageGroupId=group
)

🔍 Interviewer Notes: Testing performance optimization.


Q8: How do you implement delayed messages in SQS?​

Technical Answer: Use DelaySeconds parameter (0-900 seconds / 15 minutes).

# Send with delay
sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps({'task': 'process_payment'}),
DelaySeconds=300 # Deliver after 5 minutes
)

# For longer delays, use multiple queues with Lambda scheduling
# or Step Functions with Wait state

🔍 Interviewer Notes: Testing workarounds for service limitations.


Q9: How do you use SQS message attributes?​

Technical Answer: Message attributes provide metadata without parsing the message body. Types: String, Number, Binary, String.Array.

sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps({'order': '123'}),
MessageAttributes={
'order_id': {'StringValue': 'ORD-12345', 'DataType': 'String'},
'priority': {'StringValue': 'high', 'DataType': 'String'},
'retry_count': {'StringValue': '3', 'DataType': 'Number'}
}
)

# Receive with attributes
response = sqs.receive_message(
QueueUrl=queue_url,
MessageAttributeNames=['All']
)

🔍 Interviewer Notes: Testing metadata handling patterns.

❌ Common Mistakes: Putting data in attributes instead of body; not using attributes for routing; exceeding 256KB total message size.


Q10: How do you implement SQS message deduplication?​

Technical Answer: FIFO queues provide automatic deduplication (5-minute window); standard queues need custom handling with DynamoDB or external stores.

# FIFO - automatic deduplication
sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody=message_body,
MessageGroupId='order-group',
MessageDeduplicationId=sha256(message_id.encode()).hexdigest()[:128]
)

# Standard - custom deduplication
def process_with_dedup(message):
msg_id = message['MessageId']
try:
processed_table.put_item(
Item={'message_id': msg_id, 'processed_at': int(time.time())},
ConditionExpression='attribute_not_exists(message_id)'
)
except:
print(f"Duplicate: {msg_id} - skipping")
return
process_message(message)

🔍 Interviewer Notes: Testing idempotency patterns.

❌ Common Mistakes: Assuming standard queues deduplicate; not handling duplicates in consumers; wrong deduplication ID generation.


Q11: How do you optimize SQS costs?​

Technical Answer: Optimize costs by using batch operations (up to 10 messages per API call), enabling long polling (WaitTimeSeconds), and right-sizing message retention.

# Batch operations (10x cheaper per message)
sqs.send_message_batch(
QueueUrl=queue_url,
Entries=[
{'Id': '1', 'MessageBody': 'msg1'},
{'Id': '2', 'MessageBody': 'msg2'},
{'Id': '3', 'MessageBody': 'msg3'},
]
)

# Long polling reduces empty responses
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20 # Long polling
)

# Delete batch
sqs.delete_message_batch(
QueueUrl=queue_url,
Entries=[{'Id': str(i), 'ReceiptHandle': msg['ReceiptHandle']}]
)

🔍 Interviewer Notes: Testing cost awareness.

❌ Common Mistakes: Single-message operations only; short polling with high frequency; wrong queue type for use case (FIFO costs more).


Q12: How do you monitor SQS queues?​

Technical Answer: Monitor with CloudWatch metrics: ApproximateNumberOfMessagesVisible (backlog), ApproximateNumberOfMessagesNotVisible (in-flight), ApproximateAgeOfOldestMessage (delay).

# Create CloudWatch alarm for queue depth
cloudwatch.put_metric_alarm(
AlarmName='SQS-Queue-Backlog',
MetricName='ApproximateNumberOfMessagesVisible',
Namespace='AWS/SQS',
Statistic='Average',
Period=300,
EvaluationPeriods=2,
Threshold=1000,
ComparisonOperator='GreaterThanThreshold',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-queue'}],
AlarmActions=['arn:aws:sns:us-east-1:123456789:alerts']
)

# Monitor DLQ
cloudwatch.put_metric_alarm(
AlarmName='DLQ-Messages',
MetricName='ApproximateNumberOfMessagesVisible',
Namespace='AWS/SQS',
Statistic='Sum',
Threshold=1,
Dimensions=[{'Name': 'QueueName', 'Value': 'my-dlq'}]
)

🔍 Interviewer Notes: Testing observability setup.

❌ Common Mistakes: Not monitoring DLQ; no alarm on queue depth; missing message age monitoring.


Q13: How do you implement SQS with Lambda?​

Technical Answer: Lambda event source mapping triggers functions from SQS messages with batch processing and partial batch failure support.

# Create event source mapping
mapping = lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:us-east-1:123456789:my-queue',
FunctionName='arn:aws:lambda:us-east-1:123456789:function:processor',
BatchSize=10,
MaximumBatchingWindowInSeconds=5,
FunctionResponseTypes=['ReportBatchItemFailures']
)

# Lambda handler with batch item failures
def lambda_handler(event, context):
batch_item_failures = []
for record in event['Records']:
try:
process_message(record)
except Exception as e:
batch_item_failures.append({'itemIdentifier': record['messageId']})
return {'batchItemFailures': batch_item_failures}

🔍 Interviewer Notes: Testing serverless integration.

❌ Common Mistakes: Not using batch item failures; batch size too small (inefficient); no dead letter queue configured.


Q14: How do you secure SQS queues?​

Technical Answer: Secure with IAM policies, server-side encryption (SSE-SQS or SSE-KMS), VPC endpoints for private access, and queue policies for fine-grained access.

policy = {
"Version": "2012-10-17",
"Statement": [{
"Sid": "AllowOnlySpecificRole",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::123456789:role/app-role"},
"Action": ["sqs:SendMessage", "sqs:ReceiveMessage", "sqs:DeleteMessage"],
"Resource": "arn:aws:sqs:us-east-1:123456789:my-queue",
"Condition": {
"StringEquals": {"aws:SourceVpce": "vpce-12345"}
}
}]
}

sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={
'Policy': json.dumps(policy),
'KmsMasterKeyId': 'alias/aws/sqs',
'KmsDataKeyReusePeriodSeconds': '86400'
}
)

🔍 Interviewer Notes: Testing security implementation.

❌ Common Mistakes: No encryption at rest; overly permissive queue policies; public queue access; hardcoded credentials.


Q15: How do you handle SQS throttling?​

Technical Answer: Handle throttling with exponential backoff, batch operations, and request optimization. FIFO: 300 msg/sec; Standard: unlimited but may throttle.

import time
import random
from botocore.exceptions import ClientError

def sqs_with_backoff(operation, max_retries=5):
for attempt in range(max_retries):
try:
return operation()
except ClientError as e:
if e.response['Error']['Code'] == 'Throttling':
wait = (2 ** attempt) + random.uniform(0, 1)
time.sleep(wait)
else:
raise
raise Exception("Max retries exceeded")

# Reduce throttling: use batch operations, cache queue URLs, distribute across queues
sqs_with_backoff(lambda: sqs.send_message_batch(QueueUrl=queue_url, Entries=[...]))

🔍 Interviewer Notes: Testing rate limiting handling.

❌ Common Mistakes: No backoff strategy; single-threaded consumers; not using batch operations; caching ReceiptHandles too long.


Q16: How do you use SQS with SNS?​

Technical Answer: SNS fan-out pattern sends one message to multiple SQS queues. Use for pub/sub, filtering with message attributes, and cross-account messaging.

# Create SNS topic
topic = sns.create_topic(Name='orders-topic')
topic_arn = topic['TopicArn']

# Subscribe queues to topic
for queue_url in [email_queue, inventory_queue, shipping_queue]:
queue_arn = get_queue_arn(queue_url)

# Allow SNS to send messages
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'Policy': json.dumps({
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": "sns.amazonaws.com"},
"Action": "sqs:SendMessage",
"Resource": queue_arn,
"Condition": {"ArnEquals": {"aws:SourceArn": topic_arn}}
}]
})}
)

sns.subscribe(TopicArn=topic_arn, Protocol='sqs', Endpoint=queue_arn)

# Publish with filter attributes
sns.publish(
TopicArn=topic_arn,
Message=json.dumps({'order_id': 'ORD-123'}),
MessageAttributes={'priority': {'DataType': 'String', 'StringValue': 'high'}}
)

🔍 Interviewer Notes: Testing pub/sub architecture.

❌ Common Mistakes: Not setting queue permissions for SNS; no message filtering; processing all messages in all queues; not using fan-out for decoupling.


Q17: How do you implement SQS cross-region?​

Technical Answer: SQS is regional. Use replication via Lambda or EventBridge for cross-region setups. Patterns: active-active, active-passive, DR replication.

# Cross-region replication Lambda
sqs_primary = boto3.client('sqs', region_name='us-east-1')
sqs_dr = boto3.client('sqs', region_name='us-west-2')

def lambda_handler(event, context):
for record in event['Records']:
sqs_dr.send_message(
QueueUrl=dr_queue_url,
MessageBody=record['body'],
MessageAttributes=record.get('messageAttributes', {})
)

# EventBridge cross-region routing
events.put_targets(
Rule='cross-region-replication',
Targets=[{
'Id': '1',
'Arn': 'arn:aws:events:us-west-2:123456789:event-bus/default'
}]
)

🔍 Interviewer Notes: Testing multi-region architecture.

❌ Common Mistakes: Assuming SQS is global; no replication for critical messages; wrong failover strategy; not handling replication lag.


Q18: How do you handle large SQS messages?​

Technical Answer: SQS limit is 256KB. Use S3 for large payloads - Extended Client Library or manual S3 pointer pattern.

# Extended client for large messages
extended_client = SQSExtendedClient(
sqs_client=sqs,
s3_client=s3,
bucket_name='sqs-payload-bucket',
always_through_s3=True
)

# Send large message (stores in S3, sends pointer in SQS)
extended_client.send_message(
queue_url=queue_url,
message_body=large_payload, # Can exceed 256KB
message_attributes={...}
)

# Manual implementation: upload to S3, send URI in message
s3.put_object(Bucket='payloads', Key=msg_id, Body=large_data)
sqs.send_message(QueueUrl=queue_url, MessageBody=json.dumps({'s3_uri': f's3://payloads/{msg_id}'}))

🔍 Interviewer Notes: Testing message size handling.

❌ Common Mistakes: Trying to send > 256KB directly; not cleaning up S3 objects; no payload compression; no pointer validation.


Q19: How do you implement SQS request-response?​

Technical Answer: Implement request-response using correlation IDs with response queues, temporary queues per request, or WebSocket callbacks.

# Correlation ID pattern
def send_request(payload, timeout=30):
correlation_id = str(uuid.uuid4())

sqs.send_message(
QueueUrl=request_queue,
MessageBody=json.dumps({'payload': payload, 'correlation_id': correlation_id}),
MessageAttributes={'correlation_id': {'StringValue': correlation_id, 'DataType': 'String'}}
)

# Wait for response with matching correlation ID
start = time.time()
while time.time() - start < timeout:
response = sqs.receive_message(QueueUrl=response_queue, MessageAttributeNames=['correlation_id'])
for msg in response.get('Messages', []):
if msg['MessageAttributes'].get('correlation_id', {}).get('StringValue') == correlation_id:
sqs.delete_message(QueueUrl=response_queue, ReceiptHandle=msg['ReceiptHandle'])
return json.loads(msg['Body'])
raise TimeoutError("Request timed out")

🔍 Interviewer Notes: Testing async request-response patterns.

❌ Common Mistakes: Synchronous blocking on response; no timeout handling; memory leaks from temp queues; not handling correlation ID mismatches.


Q20: How do you use SQS for rate limiting?​

Technical Answer: Use SQS as a buffer to implement rate limiting with token bucket pattern or delayed message delivery.

# Token bucket consumer
def rate_limited_consumer(queue_url, max_rate=10, window=1):
tokens = max_rate
last_check = time.time()

while True:
now = time.time()
tokens = min(max_rate, tokens + (now - last_check) * max_rate / window)
last_check = now

if tokens >= 1:
response = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=min(10, int(tokens)))
for msg in response.get('Messages', []):
if tokens < 1:
time.sleep((1 - tokens) * window / max_rate)
tokens = 1
process_message(msg)
tokens -= 1
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])

# Delayed delivery for pacing
if time.time() - last_send < min_interval:
sqs.send_message(QueueUrl=queue_url, MessageBody=msg, DelaySeconds=delay)

🔍 Interviewer Notes: Testing traffic shaping.

❌ Common Mistakes: No backoff on downstream limits; consumers too fast (overwhelm downstream); ignoring queue depth signals; no circuit breaker pattern.


Q21: How do you migrate SQS queues?​

Technical Answer: Migrate using dual-write pattern: create new queue, write to both, update consumers, drain old queue.

def migrate_queue(old_queue_url, new_queue_url):
# Dual-write decorator
def dual_write(func):
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
sqs.send_message(
QueueUrl=new_queue_url,
MessageBody=kwargs.get('MessageBody', ''),
MessageAttributes=kwargs.get('MessageAttributes', {})
)
return result
return wrapper

# Drain old queue
drained = 0
while True:
messages = sqs.receive_message(QueueUrl=old_queue_url, MaxNumberOfMessages=10).get('Messages', [])
if not messages:
break
for msg in messages:
sqs.send_message(QueueUrl=new_queue_url, MessageBody=msg['Body'])
sqs.delete_message(QueueUrl=old_queue_url, ReceiptHandle=msg['ReceiptHandle'])
drained += 1

🔍 Interviewer Notes: Testing operational migration skills.

❌ Common Mistakes: Not draining old queue; message loss during cutover; no rollback plan; no validation of message counts.


Q22: How do you test SQS applications?​

Technical Answer: Test with moto mock for unit tests, integration tests with test queues, and failure injection for resilience testing.

# Unit test with moto
@mock_sqs
def test_message_processing():
sqs = boto3.client('sqs', region_name='us-east-1')
queue = sqs.create_queue(QueueName='test-queue')

sqs.send_message(QueueUrl=queue['QueueUrl'], MessageBody='test')
response = sqs.receive_message(QueueUrl=queue['QueueUrl'])

assert len(response['Messages']) == 1
assert response['Messages'][0]['Body'] == 'test'

# Failure testing - visibility timeout retry
sqs.send_message(QueueUrl=queue_url, MessageBody='test')
response = sqs.receive_message(QueueUrl=queue_url, VisibilityTimeout=1)
msg = response['Messages'][0]
time.sleep(2) # Wait for visibility timeout
response = sqs.receive_message(QueueUrl=queue_url)
assert response['Messages'][0]['MessageId'] == msg['MessageId'] # Should be visible again

🔍 Interviewer Notes: Testing testing strategies.

❌ Common Mistakes: Only testing happy path; no visibility timeout tests; no DLQ testing; not cleaning up test queues.


Q23: How do you troubleshoot SQS issues?​

Technical Answer: Troubleshoot with CloudWatch metrics, queue inspection scripts, and common issue pattern matching.

def diagnose_queue(queue_url):
attrs = sqs.get_queue_attributes(QueueUrl=queue_url, AttributeNames=['All'])['Attributes']

visible = int(attrs['ApproximateNumberOfMessages'])
in_flight = int(attrs['ApproximateNumberOfMessagesNotVisible'])

print(f"Visible: {visible}, In-flight: {in_flight}")

if visible > 1000:
print("⚠️ Large backlog - scale up consumers")
if in_flight > 1000:
print("⚠️ Many in-flight - check consumer failures")

# Sample messages for inspection
response = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=5, VisibilityTimeout=0)
for msg in response.get('Messages', []):
print(f"ID: {msg['MessageId']}, Body: {msg['Body'][:200]}")

# Common issues
ISSUES = {
"Messages not processing": ["Check IAM permissions", "Verify queue URL", "Check CloudWatch Logs"],
"Duplicate processing": ["Implement idempotency", "Check visibility timeout > processing time"],
"High latency": ["Use long polling", "Increase batch size", "Check consumer scaling"]
}

🔍 Interviewer Notes: Testing debugging skills.

❌ Common Mistakes: Not checking CloudWatch first; deleting messages before confirmation; no visibility into queue state; not using structured logging.


Q24: How do you use SQS FIFO queues?​

Technical Answer: FIFO queues guarantee ordering and exactly-once processing. Required: MessageGroupId (for grouping), optional MessageDeduplicationId.

# FIFO queue - strict ordering within group
sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody=order_data,
MessageGroupId='customer-123', # Orders by customer processed in order
MessageDeduplicationId=order_id # Prevent duplicates (5-min window)
)

# Content-based deduplication (enable on queue)
# Queue setting: ContentBasedDeduplication=true
# Then omit MessageDeduplicationId

# Receive with multiple groups
response = sqs.receive_message(
QueueUrl=fifo_queue_url,
MaxNumberOfMessages=10, # Can get messages from different groups
AttributeNames=['MessageGroupId']
)

# Delete all messages from same group before moving to next group

FIFO vs Standard:

FeatureStandardFIFO
ThroughputUnlimited300 msg/sec (default), 3000 with batch
OrderingBest-effortGuaranteed per group
DeduplicationNo5-minute window
Price$0.40/million$0.50/million

🔍 Interviewer Notes: Testing ordering requirements.

❌ Common Mistakes: Using FIFO when ordering not required (higher cost); not using MessageGroupId (required); expecting cross-group ordering.


Q25: How do you implement SQS queue policies?​

Technical Answer: Queue policies define access permissions at the resource level, complementing IAM policies.

# Restrict to specific VPC endpoint
vpc_policy = {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Deny",
"Principal": "*",
"Action": "sqs:*",
"Resource": queue_arn,
"Condition": {
"StringNotEquals": {
"aws:SourceVpce": "vpce-12345"
}
}
}]
}

# Cross-account access
cross_account_policy = {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::987654321:root"
},
"Action": ["sqs:SendMessage", "sqs:ReceiveMessage"],
"Resource": queue_arn,
"Condition": {
"StringEquals": {
"aws:PrincipalTag/Environment": "Production"
}
}
}]
}

sqs.set_queue_attributes(QueueUrl=queue_url, Attributes={'Policy': json.dumps(policy)})

🔍 Interviewer Notes: Testing resource-level security.

❌ Common Mistakes: Overly permissive policies; no condition keys; not using least privilege; no VPC endpoint restrictions.



2. Intermediate Topics​

Q26: How do you implement SQS for high-throughput scenarios?​

Q27: How do you implement SQS for high-throughput scenarios?​

Technical Answer:

Batching:

# Send up to 10 messages per request
messages = [
{'Id': '1', 'MessageBody': json.dumps({'data': 'msg1'})},
{'Id': '2', 'MessageBody': json.dumps({'data': 'msg2'})},
]

response = sqs.send_message_batch(
QueueUrl=queue_url,
Entries=messages
)

# Handle failures
if response.get('Failed'):
for failure in response['Failed']:
print(f"Failed: {failure['Id']}: {failure['Message']}")
# Retry with backoff

Receive Batch:

# Receive up to 10 messages at once
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20
)

# Process and delete batch
entries = [{'Id': str(i), 'ReceiptHandle': msg['ReceiptHandle']}
for i, msg in enumerate(response['Messages'])]

sqs.delete_message_batch(
QueueUrl=queue_url,
Entries=entries
)

🔍 Interviewer Notes: Testing batching strategies.


Q28: How do you monitor SQS queues?​

Technical Answer:

CloudWatch Metrics:

import boto3

cloudwatch = boto3.client('cloudwatch')

# Key SQS metrics:
# - ApproximateNumberOfMessagesVisible
# - ApproximateNumberOfMessagesNotVisible
# - ApproximateNumberOfMessagesDelayed
# - ApproximateAgeOfOldestMessage
# - NumberOfMessagesSent
# - NumberOfMessagesReceived
# - NumberOfMessagesDeleted

response = cloudwatch.get_metric_statistics(
Namespace='AWS/SQS',
MetricName='ApproximateNumberOfMessagesVisible',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-queue'}],
StartTime=datetime.utcnow() - timedelta(hours=1),
EndTime=datetime.utcnow(),
Period=300,
Statistics=['Average', 'Maximum']
)

Set CloudWatch Alarms:

cloudwatch.put_metric_alarm(
AlarmName='SQS-Queue-Depth-High',
AlarmDescription='Alarm when queue depth > 1000',
ActionsEnabled=True,
AlarmActions=['arn:aws:sns:...:alerts'],
MetricName='ApproximateNumberOfMessagesVisible',
Namespace='AWS/SQS',
Statistic='Average',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-queue'}],
Period=300,
EvaluationPeriods=2,
Threshold=1000,
ComparisonOperator='GreaterThanThreshold'
)

🔍 Interviewer Notes: Testing monitoring and alerting practices.


Q29: How do you implement queue-based decoupling?​

Technical Answer:

Producer Service:

import boto3

sqs = boto3.client('sqs')

def create_order(order_data):
# Validate order
validate(order_data)

# Save to database
order_id = save_to_db(order_data)

# Queue for processing
sqs.send_message(
QueueUrl=process_queue_url,
MessageBody=json.dumps({
'order_id': order_id,
'action': 'process_order'
})
)

return {'order_id': order_id, 'status': 'accepted'}

Consumer Service:

import boto3

sqs = boto3.client('sqs')

def process_messages():
while True:
response = sqs.receive_message(
QueueUrl=process_queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20
)

messages = response.get('Messages', [])

for message in messages:
try:
data = json.loads(message['Body'])
process_order(data['order_id'])

sqs.delete_message(
QueueUrl=process_queue_url,
ReceiptHandle=message['ReceiptHandle']
)
except Exception as e:
print(f"Error processing: {e}")
# Let visibility timeout expire for retry

Benefits:

  • Services can scale independently
  • Producer not blocked by consumer processing
  • Buffer against traffic spikes
  • Consumer can be down temporarily

🔍 Interviewer Notes: Testing microservices architecture understanding.


3. Advanced Topics​

Q30: How do you implement cross-region SQS replication?​

Technical Answer: Use Lambda to replicate messages between regions.

import boto3

# Primary region
sqs_primary = boto3.client('sqs', region_name='us-east-1')
# DR region
sqs_dr = boto3.client('sqs', region_name='us-west-2')

dr_queue_url = 'https://sqs.us-west-2.amazonaws.com/.../my-queue'

def lambda_handler(event, context):
# Triggered by primary queue
for record in event['Records']:
# Replicate to DR region
sqs_dr.send_message(
QueueUrl=dr_queue_url,
MessageBody=record['body'],
MessageAttributes=record.get('messageAttributes', {})
)

🔍 Interviewer Notes: Testing disaster recovery patterns.


Q31: How do you implement SQS for fan-out patterns?​

Technical Answer: Use SNS to fan out to multiple SQS queues.

import boto3

sns = boto3.client('sns')
sqs = boto3.client('sqs')

# Create topic
topic = sns.create_topic(Name='order-events')
topic_arn = topic['TopicArn']

# Create queues
email_queue = sqs.create_queue(QueueName='email-notifications')
analytics_queue = sqs.create_queue(QueueName='analytics-events')

# Subscribe queues to topic
sns.subscribe(
TopicArn=topic_arn,
Protocol='sqs',
Endpoint=email_queue.attributes['QueueArn']
)

sns.subscribe(
TopicArn=topic_arn,
Protocol='sqs',
Endpoint=analytics_queue.attributes['QueueArn']
)

# Publish message
sns.publish(
TopicArn=topic_arn,
Message=json.dumps({'order_id': '123', 'status': 'completed'})
)
# Both queues receive the message

🔍 Interviewer Notes: Testing pub/sub architecture understanding.


Q32: How do you implement message filtering?​

Technical Answer:

SNS Message Filtering:

# Subscribe with filter policy
sns.subscribe(
TopicArn=topic_arn,
Protocol='sqs',
Endpoint=high_priority_queue_arn,
Attributes={
'FilterPolicy': json.dumps({
'priority': ['high', 'critical']
})
}
)

# Only high/critical priority messages go to this queue
sns.publish(
TopicArn=topic_arn,
Message='Urgent!',
MessageAttributes={
'priority': {'DataType': 'String', 'StringValue': 'high'}
}
)

🔍 Interviewer Notes: Testing message routing patterns.


Section 4: Extended Questions (70 more)​

Q33: How do you implement DLQ redrive in SQS?​

Technical Answer: DLQ redrive moves messages from Dead Letter Queue back to the source queue for reprocessing.

import boto3

sqs = boto3.client('sqs')

# Redrive messages from DLQ back to source
def redrive_messages(dlq_url, source_queue_url, max_messages=100):
redriven_count = 0

while redriven_count < max_messages:
# Receive messages from DLQ
response = sqs.receive_message(
QueueUrl=dlq_url,
MaxNumberOfMessages=10,
VisibilityTimeout=300
)

messages = response.get('Messages', [])
if not messages:
break

for msg in messages:
try:
# Send back to source queue
sqs.send_message(
QueueUrl=source_queue_url,
MessageBody=msg['Body'],
MessageAttributes=msg.get('MessageAttributes', {})
)

# Delete from DLQ
sqs.delete_message(
QueueUrl=dlq_url,
ReceiptHandle=msg['ReceiptHandle']
)

redriven_count += 1
except Exception as e:
print(f"Failed to redrive message {msg['MessageId']}: {e}")

return redriven_count

# Or use AWS Console/API for redrive
sqs.start_message_move_task(
SourceArn='arn:aws:sqs:us-east-1:123456789:my-dlq',
DestinationArn='arn:aws:sqs:us-east-1:123456789:my-queue',
MaxNumberOfMessagesPerSecond=100
)

🔍 Interviewer Notes: Testing DLQ management and message recovery patterns.

❌ Common Mistakes:

  • Not checking why messages failed before redriving
  • Redriving without fixing the underlying issue
  • Not using the built-in redrive feature (AWS Console/API)

Q34: How do you implement queue encryption in SQS?​

Technical Answer: SQS supports server-side encryption (SSE) using KMS keys.

import boto3

sqs = boto3.client('sqs')

# Create encrypted queue
response = sqs.create_queue(
QueueName='encrypted-queue',
Attributes={
'KmsMasterKeyId': 'arn:aws:kms:us-east-1:123456789:key/12345678-1234-1234-1234-123456789012',
'KmsDataKeyReusePeriodSeconds': '300', # 60-86400 seconds
'SqsManagedSseEnabled': 'false' # Use customer-managed key
}
)

# Or use SQS-managed encryption (free)
response = sqs.create_queue(
QueueName='sqs-managed-encryption',
Attributes={
'SqsManagedSseEnabled': 'true'
}
)

# Send encrypted message (automatic)
sqs.send_message(
QueueUrl=queue_url,
MessageBody='Sensitive data',
MessageAttributes={
'sensitivity': {'StringValue': 'high', 'DataType': 'String'}
}
)

🔍 Interviewer Notes: Testing security and compliance knowledge.

❌ Common Mistakes:

  • Not enabling encryption for sensitive data
  • Using wrong KMS key permissions
  • Not understanding SQS-managed vs customer-managed

Q35: How do you optimize SQS costs?​

Technical Answer: SQS pricing is based on API requests and data transfer.

Cost Optimization Strategies:

# 1. Use batch operations
messages = [{'Id': str(i), 'MessageBody': f'Message {i}'} for i in range(10)]
response = sqs.send_message_batch(
QueueUrl=queue_url,
Entries=messages
)
# Cost: 1 request for 10 messages vs 10 individual requests

# 2. Long polling (reduces empty receives)
response = sqs.receive_message(
QueueUrl=queue_url,
WaitTimeSeconds=20, # Max 20 seconds
MaxNumberOfMessages=10
)
# Cost: Fewer requests, less CPU on client

# 3. Use FIFO only when needed
# FIFO costs more per request

# 4. Process multiple messages per Lambda invocation
lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:...:my-queue',
FunctionName='my-function',
BatchSize=10, # Process up to 10 per invocation
MaximumBatchingWindowInSeconds=5 # Wait to fill batch
)

Pricing Calculation:

  • Standard: $0.40 per million requests (first 1B/month)
  • FIFO: $0.50 per million requests
  • Data transfer: $0.09/GB out to internet

🔍 Interviewer Notes: Testing cost optimization strategies.

❌ Common Mistakes:

  • Not using batch operations
  • Using short polling (more requests)
  • Not considering FIFO cost premium

Q36: How do you implement message deduplication in SQS?​

Technical Answer: FIFO queues provide built-in deduplication. For Standard queues, implement client-side deduplication.

FIFO Queue Deduplication:

# Content-based deduplication (enabled on queue)
sqs.create_queue(
QueueName='my-fifo-queue.fifo',
Attributes={
'FifoQueue': 'true',
'ContentBasedDeduplication': 'true'
}
)

# Or provide deduplication ID
sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody='Order #12345',
MessageGroupId='customer-456',
MessageDeduplicationId='order-12345-v1' # 5 min dedup window
)

Client-Side Deduplication for Standard Queues:

import boto3
import hashlib
from datetime import datetime, timedelta

dynamodb = boto3.resource('dynamodb')
dedup_table = dynamodb.Table('message-dedup')

def send_deduplicated_message(queue_url, message_body, dedup_window_minutes=5):
# Generate message fingerprint
fingerprint = hashlib.sha256(message_body.encode()).hexdigest()

# Check if recently sent
try:
dedup_table.put_item(
Item={
'fingerprint': fingerprint,
'ttl': int((datetime.utcnow() + timedelta(minutes=dedup_window_minutes)).timestamp())
},
ConditionExpression='attribute_not_exists(fingerprint)'
)
except dynamodb.meta.client.exceptions.ConditionalCheckFailedException:
print("Duplicate message, skipping")
return None

# Send to SQS
response = sqs.send_message(
QueueUrl=queue_url,
MessageBody=message_body
)
return response['MessageId']

🔍 Interviewer Notes: Testing exactly-once semantics and deduplication patterns.

❌ Common Mistakes:

  • Not using FIFO when ordering+dedup needed
  • Not handling deduplication window (5 min for FIFO)
  • Not implementing client-side dedup for Standard queues

Q37: How do you monitor SQS queue depth and set alarms?​

Technical Answer: Monitor queue depth to detect processing issues.

import boto3

cloudwatch = boto3.client('cloudwatch')
sqs = boto3.client('sqs')

# Get queue attributes
queue_attrs = sqs.get_queue_attributes(
QueueUrl=queue_url,
AttributeNames=['ApproximateNumberOfMessages',
'ApproximateNumberOfMessagesNotVisible',
'ApproximateNumberOfMessagesDelayed']
)

visible = int(queue_attrs['Attributes']['ApproximateNumberOfMessages'])
in_flight = int(queue_attrs['Attributes']['ApproximateNumberOfMessagesNotVisible'])
total = visible + in_flight

# Create CloudWatch alarm
cloudwatch.put_metric_alarm(
AlarmName='SQS-Queue-Depth-High',
AlarmDescription='Alert when queue depth > 1000',
MetricName='ApproximateNumberOfMessagesVisible',
Namespace='AWS/SQS',
Statistic='Average',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-queue'}],
Period=300,
EvaluationPeriods=2,
Threshold=1000,
ComparisonOperator='GreaterThanThreshold',
AlarmActions=['arn:aws:sns:us-east-1:123456789:alerts']
)

# Also alarm on age of oldest message
cloudwatch.put_metric_alarm(
AlarmName='SQS-Message-Age-High',
AlarmDescription='Alert when oldest message > 5 min',
MetricName='ApproximateAgeOfOldestMessage',
Namespace='AWS/SQS',
Statistic='Maximum',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-queue'}],
Period=60,
EvaluationPeriods=3,
Threshold=300, # 5 minutes
ComparisonOperator='GreaterThanThreshold'
)

🔍 Interviewer Notes: Testing monitoring and alerting practices.

❌ Common Mistakes:

  • Not monitoring in-flight messages
  • Not setting age-based alarms (processing lag)
  • Not alerting on DLQ growth

Q38: How do you implement SQS message scheduling?​

Technical Answer: SQS supports delayed messages up to 15 minutes. For longer delays, use EventBridge or Step Functions.

SQS Delay (up to 15 minutes):

# Delay single message
sqs.send_message(
QueueUrl=queue_url,
MessageBody='Process this later',
DelaySeconds=900 # 15 minutes max
)

# Delay queue (all messages)
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'DelaySeconds': '300'} # 5 min default delay
)

EventBridge for Longer Scheduling:

import boto3

events = boto3.client('events')

# Schedule message for 1 hour later
events.put_rule(
Name='scheduled-task',
ScheduleExpression='rate(1 hour)',
State='ENABLED'
)

# Or one-time schedule
events.put_rule(
Name='one-time-task',
ScheduleExpression='at(2024-01-01T12:00:00)', # ISO 8601
State='ENABLED'
)

# Target SQS
events.put_targets(
Rule='scheduled-task',
Targets=[{
'Id': '1',
'Arn': 'arn:aws:sqs:us-east-1:123456789:my-queue',
'SqsParameters': {
'MessageGroupId': 'scheduled-messages' # For FIFO
}
}]
)

Step Functions for Complex Scheduling:

# Wait state in Step Functions
{
"StartAt": "Wait",
"States": {
"Wait": {
"Type": "Wait",
"Seconds": 3600, # 1 hour
"Next": "SendToSQS"
},
"SendToSQS": {
"Type": "Task",
"Resource": "arn:aws:states:::aws-sdk:sqs:sendMessage",
"Parameters": {
"QueueUrl": "https://sqs...",
"MessageBody.$": "$.message"
},
"End": true
}
}
}

🔍 Interviewer Notes: Testing scheduling patterns and workarounds for limits.

❌ Common Mistakes:

  • Trying to delay >15 minutes with SQS
  • Not considering EventBridge for scheduling
  • Not handling timezone issues

Q39: How does SQS Lambda event source mapping scale with queue depth?​

Technical Answer: SQS Lambda triggers scale automatically based on queue depth, up to 1000 concurrent batches per queue. Each batch is processed by a separate Lambda instance, and scaling stops when concurrency limits are reached or the queue is drained.

Code/Config Example:

import boto3

lambda_client = boto3.client('lambda')

# Event source mapping with scaling configuration
response = lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:us-east-1:123456789:my-queue',
FunctionName='arn:aws:lambda:us-east-1:123456789:function:processor',
BatchSize=10,
MaximumBatchingWindowInSeconds=5,
ScalingConfig={
'MaximumConcurrency': 500 # Limit concurrent batches
},
FunctionResponseTypes=['ReportBatchItemFailures']
)

🔍 Interviewer Notes: Tests understanding of serverless auto-scaling with SQS triggers.

❌ Common Mistakes:

  • Not setting MaximumConcurrency leading to Lambda throttling
  • Batch size too small causing excessive Lambda invocations
  • Not monitoring ConcurrentBatches metric

Q40: What is the difference between SQS Standard and FIFO queue Lambda triggers?​

Technical Answer: Standard queue triggers process batches in any order with at-least-once delivery, while FIFO triggers process messages in strict order per MessageGroupId with exactly-once processing guarantees. FIFO triggers have a maximum batch size of 10 and support ordering per group.

Code/Config Example:

# Standard queue trigger (no ordering)
std_mapping = lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:us-east-1:123456789:std-queue',
FunctionName='std-processor',
BatchSize=10
)

# FIFO queue trigger (ordering per MessageGroupId)
fifo_mapping = lambda_client.create_event_source_mapping(
EventSourceArn='arn:aws:sqs:us-east-1:123456789:fifo-queue.fifo',
FunctionName='fifo-processor',
BatchSize=10 # Max for FIFO
# Lambda automatically processes one MessageGroupId at a time
)

🔍 Interviewer Notes: Tests FIFO vs Standard queue Lambda integration differences.

❌ Common Mistakes:

  • Expecting cross-group ordering in FIFO triggers
  • Using FIFO for unordered workloads (higher cost)
  • Not handling MessageGroupId in FIFO consumers

Q41: What IAM permissions are required for SQS producers and consumers?​

Technical Answer: Producers need sqs:SendMessage and sqs:SendMessageBatch permissions, while consumers need sqs:ReceiveMessage, sqs:DeleteMessage, and sqs:ChangeMessageVisibility. Fine-grained access can be enforced via IAM policies or resource-based queue policies.

Code/Config Example:

# IAM policy for SQS producer
producer_policy = {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["sqs:SendMessage", "sqs:SendMessageBatch"],
"Resource": "arn:aws:sqs:us-east-1:123456789:my-queue"
}]
}

# IAM policy for SQS consumer
consumer_policy = {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:ChangeMessageVisibility"],
"Resource": "arn:aws:sqs:us-east-1:123456789:my-queue"
}]
}

🔍 Interviewer Notes: Tests IAM permission scoping for SQS.

❌ Common Mistakes:

  • Granting full sqs:* permissions
  • Not restricting resources to specific queue ARNs
  • Forgetting sqs:ChangeMessageVisibility for long-running tasks

Q42: How to configure SSE-KMS encryption for SQS queues?​

Technical Answer: SQS supports server-side encryption with KMS using customer-managed keys (CMK) or AWS managed keys. Set KmsMasterKeyId on queue creation or update, with optional KmsDataKeyReusePeriodSeconds (60-86400 seconds).

Code/Config Example:

import boto3

sqs = boto3.client('sqs')

# Create queue with CMK encryption
response = sqs.create_queue(
QueueName='encrypted-queue',
Attributes={
'KmsMasterKeyId': 'arn:aws:kms:us-east-1:123456789:key/1234abcd-12ab-34cd-56ef-1234567890ab',
'KmsDataKeyReusePeriodSeconds': '3600', # Reuse data key for 1 hour
'SqsManagedSseEnabled': 'false'
}
)

# Update existing queue to use AWS managed SQS key
sqs.set_queue_attributes(
QueueUrl='https://sqs.us-east-1.amazonaws.com/123456789/encrypted-queue',
Attributes={'KmsMasterKeyId': 'alias/aws/sqs'}
)

🔍 Interviewer Notes: Tests encryption configuration and KMS integration.

❌ Common Mistakes:

  • Using SQS-managed encryption when custom key rotation is needed
  • Not granting SQS service principal access to KMS key
  • Setting KmsDataKeyReusePeriodSeconds outside allowed range

Q43: What is the difference between SSE-SQS and SSE-KMS encryption?​

Technical Answer: SSE-SQS uses an SQS-managed service key (free, no KMS charges) with 30-day data key reuse, while SSE-KMS uses customer-managed or AWS managed KMS keys (incurs KMS API and storage costs) with configurable data key reuse periods. SSE-KMS provides more auditability via CloudTrail.

Code/Config Example:

# SSE-SQS (free, SQS-managed)
sqs.create_queue(
QueueName='sqs-managed-queue',
Attributes={'SqsManagedSseEnabled': 'true'}
)

# SSE-KMS (AWS managed key)
sqs.create_queue(
QueueName='kms-managed-queue',
Attributes={'KmsMasterKeyId': 'alias/aws/sqs'}
)

# Check encryption type
attrs = sqs.get_queue_attributes(
QueueUrl=queue_url,
AttributeNames=['KmsMasterKeyId', 'SqsManagedSseEnabled']
)['Attributes']
print(f"Encryption: {attrs.get('KmsMasterKeyId', 'SQS-Managed')}")

🔍 Interviewer Notes: Tests encryption option tradeoffs.

❌ Common Mistakes:

  • Assuming SSE-KMS has no free tier (AWS managed key has no monthly fee, only API costs)
  • Not enabling CloudTrail for SSE-KMS audit logs
  • Using SSE-KMS for non-sensitive workloads (unnecessary cost)

Q44: How to restrict SQS queue access to specific VPC endpoints?​

Technical Answer: Use queue policies with the aws:SourceVpce condition key to deny all access not originating from specified VPC endpoints. This ensures queues are only accessible from approved VPCs over private network connections.

Code/Config Example:

import json

vpce_policy = {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyNonVpcAccess",
"Effect": "Deny",
"Principal": "*",
"Action": "sqs:*",
"Resource": "arn:aws:sqs:us-east-1:123456789:my-queue",
"Condition": {
"StringNotEquals": {
"aws:SourceVpce": ["vpce-12345", "vpce-67890"]
}
}
},
{
"Sid": "AllowQueueOwner",
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::123456789:root"},
"Action": "sqs:*",
"Resource": "arn:aws:sqs:us-east-1:123456789:my-queue"
}
]
}

sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'Policy': json.dumps(vpce_policy)}
)

🔍 Interviewer Notes: Tests VPC endpoint security controls.

❌ Common Mistakes:

  • Not allowing queue owner access (locks out admin)
  • Forgetting to add all required VPC endpoints
  • Using StringEquals instead of StringNotEquals for deny rules

Q45: How to implement cross-account SQS queue access?​

Technical Answer: Use resource-based queue policies to grant specific AWS accounts access to SQS actions, or use identity-based IAM roles with cross-account assume role. Queue policies are attached to the queue, while IAM roles are assumed by the external account.

Code/Config Example:

# Queue policy granting cross-account access
cross_account_policy = {
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::987654321:root"},
"Action": ["sqs:SendMessage", "sqs:ReceiveMessage"],
"Resource": "arn:aws:sqs:us-east-1:123456789:my-queue",
"Condition": {
"StringEquals": {"aws:PrincipalTag/Environment": "Production"}
}
}]
}

sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'Policy': json.dumps(cross_account_policy)}
)

# Cross-account role assumption
sts = boto3.client('sts')
assumed_role = sts.assume_role(
RoleArn='arn:aws:iam::987654321:role/SQSAccessRole',
RoleSessionName='CrossAccountSession'
)
sqs_cross_account = boto3.client(
'sqs',
aws_access_key_id=assumed_role['Credentials']['AccessKeyId'],
aws_secret_access_key=assumed_role['Credentials']['SecretAccessKey'],
aws_session_token=assumed_role['Credentials']['SessionToken']
)

🔍 Interviewer Notes: Tests cross-account access patterns.

❌ Common Mistakes:

  • Granting access to * instead of specific account root
  • Not using condition keys for additional security
  • Forgetting to enable IAM role assumption permissions

Q46: How to implement batch operations for SQS?​

Technical Answer: SQS supports batched send (up to 10 messages), receive (up to 10), and delete (up to 10) operations, which count as a single API call to reduce costs and improve throughput.

Code/Config Example:

# Batch send messages
sqs.send_message_batch(
QueueUrl=queue_url,
Entries=[
{'Id': '1', 'MessageBody': json.dumps({'task': 'a'})},
{'Id': '2', 'MessageBody': json.dumps({'task': 'b'})}
]
)

# Batch delete messages
sqs.delete_message_batch(
QueueUrl=queue_url,
Entries=[{'Id': str(i), 'ReceiptHandle': msg['ReceiptHandle']}
for i, msg in enumerate(messages)]
)

🔍 Interviewer Notes: Tests cost optimization and throughput improvement strategies.

❌ Common Mistakes:

  • Not handling partial batch failures
  • Exceeding 10 message limit per batch
  • Not checking batch response for failed entries

Q47: What is long polling and how to configure it?​

Technical Answer: Long polling waits up to 20 seconds for messages to arrive, reducing empty responses and API costs. It can be configured per-request via WaitTimeSeconds or at queue level via ReceiveMessageWaitTimeSeconds.

Code/Config Example:

# Request-level long polling
sqs.receive_message(
QueueUrl=queue_url,
WaitTimeSeconds=20,
MaxNumberOfMessages=10
)

# Queue-level long polling
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'ReceiveMessageWaitTimeSeconds': '20'}
)

🔍 Interviewer Notes: Tests cost optimization best practices.

❌ Common Mistakes:

  • Using WaitTimeSeconds=0 (short polling)
  • Not setting queue-level long polling
  • High frequency of empty receive calls

Q48: How to optimize SQS throughput for high-volume workloads?​

Technical Answer: Use batch operations, long polling, multiple queues (sharding), and FIFO high-throughput mode (up to 3000 msg/s with batching). Distribute FIFO messages across multiple MessageGroupIds to parallelize processing.

Code/Config Example:

# FIFO high throughput with multiple groups
import hashlib

for msg in large_batch:
# Shard across 100 groups for parallel processing
group_id = int(hashlib.md5(msg['id'].encode()).hexdigest(), 16) % 100
sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody=json.dumps(msg),
MessageGroupId=f'group-{group_id}'
)

🔍 Interviewer Notes: Tests high-throughput SQS design patterns.

❌ Common Mistakes:

  • Using single MessageGroupId for FIFO (limits throughput)
  • Not using batch operations
  • Under-provisioning consumers

Q49: What is SQS visibility timeout and how to adjust it?​

Technical Answer: Visibility timeout is the period a message is hidden from other consumers after being received. Default is 30 seconds, max 12 hours. Adjust via queue attributes, per-message in receive_message, or extend via ChangeMessageVisibility.

Code/Config Example:

# Queue-level visibility timeout
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'VisibilityTimeout': '300'} # 5 minutes
)

# Per-message visibility timeout
response = sqs.receive_message(
QueueUrl=queue_url,
VisibilityTimeout=60 # 1 minute for this request
)

# Extend visibility timeout for long-running tasks
sqs.change_message_visibility(
QueueUrl=queue_url,
ReceiptHandle=receipt_handle,
VisibilityTimeout=300 # Extend to 5 minutes
)

🔍 Interviewer Notes: Tests message lifecycle understanding.

❌ Common Mistakes:

  • Visibility timeout shorter than processing time
  • Not extending for long-running tasks
  • Not handling visibility timeout expiration

Q50: How to handle SQS message retries and redrive?​

Technical Answer: Configure maxReceiveCount in the queue redrive policy to move messages to a Dead Letter Queue (DLQ) after N failed receives. Lambda triggers automatically retry on failure if ReportBatchItemFailures is enabled.

Code/Config Example:

# Set redrive policy to move failed messages to DLQ
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'RedrivePolicy': json.dumps({
'deadLetterTargetArn': dlq_arn,
'maxReceiveCount': 3 # Move to DLQ after 3 failed receives
})}
)

🔍 Interviewer Notes: Tests failure handling and DLQ configuration.

❌ Common Mistakes:

  • Not configuring a DLQ
  • Setting maxReceiveCount too high/low
  • Not handling retries in consumers

Q51: How to integrate SQS with EC2 Auto Scaling groups?​

Technical Answer: Deploy SQS consumers on EC2 instances in an Auto Scaling group (ASG) that scales based on queue depth (ApproximateNumberOfMessagesVisible metric). Use lifecycle hooks to drain messages before terminating instances.

Code/Config Example:

# Auto Scaling scaling policy based on SQS queue depth
autoscaling.put_scaling_policy(
AutoScalingGroupName='sqs-consumer-asg',
PolicyName='ScaleBasedOnQueueDepth',
PolicyType='TargetTrackingScaling',
TargetTrackingConfiguration={
'CustomizedMetricSpecification': {
'MetricName': 'ApproximateNumberOfMessagesVisible',
'Namespace': 'AWS/SQS',
'Dimensions': [{'Name': 'QueueName', 'Value': 'my-queue'}],
'Statistic': 'Average'
},
'TargetValue': 100 # Aim for 100 messages per instance
}
)

🔍 Interviewer Notes: Tests auto-scaling integration with SQS.

❌ Common Mistakes:

  • Not using queue depth as scaling metric
  • Not handling instance termination (message loss)
  • Scaling too aggressively (thrashing)

Q52: How to implement SQS consumers on EC2 instances?​

Technical Answer: Run long-lived consumer processes on EC2 that poll SQS in a loop, process messages, and delete them after success. Use systemd to manage consumer processes and restart on failure.

Code/Config Example:

# EC2 consumer loop
while True:
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20
)
messages = response.get('Messages', [])
for msg in messages:
try:
process_message(msg['Body'])
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])
except Exception as e:
print(f"Processing failed: {e}")
# Let visibility timeout expire for retry

🔍 Interviewer Notes: Tests EC2 consumer implementation.

❌ Common Mistakes:

  • Not using long polling
  • Exiting on empty receive (wastes resources)
  • Not handling exceptions (message loss)

Q53: How to drain SQS messages from EC2 instances before termination?​

Technical Answer: Use Auto Scaling lifecycle hooks to trigger a drain process that stops polling, processes remaining messages, then completes the lifecycle action. Prevent new messages from being received during drain.

Code/Config Example:

# Lifecycle hook handler (runs on EC2)
def handle_termination():
# Signal ASG that instance is draining
autoscaling.complete_lifecycle_action(
LifecycleHookName='sqs-drain-hook',
AutoScalingGroupName='sqs-consumer-asg',
LifecycleActionResult='CONTINUE',
InstanceId=instance_id
)
# Process remaining messages
while True:
response = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=10)
messages = response.get('Messages', [])
if not messages:
break
for msg in messages:
process_message(msg['Body'])
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])

🔍 Interviewer Notes: Tests graceful shutdown patterns.

❌ Common Mistakes:

  • Not handling lifecycle hooks
  • Terminating before draining messages
  • Not stopping new message polling during drain

Q54: How to monitor EC2 SQS consumer health?​

Technical Answer: Monitor consumer health via CloudWatch metrics (CPU, memory), custom metrics for processing rate, and a heartbeat message sent to CloudWatch. Use ELB health checks if behind a load balancer.

Code/Config Example:

# Send custom metric for processing rate
cloudwatch.put_metric_data(
Namespace='EC2/SQS/Consumer',
MetricData=[{
'MetricName': 'ProcessingRate',
'Value': processed_count,
'Unit': 'Count',
'Dimensions': [{'Name': 'InstanceId', 'Value': instance_id}]
}]
)

🔍 Interviewer Notes: Tests observability for EC2 consumers.

❌ Common Mistakes:

  • Not monitoring processing rate
  • Missing heartbeat/health checks
  • Not alerting on consumer failures

Q55: How to optimize EC2 SQS consumer performance?​

Technical Answer: Use multi-threading/multi-processing to process messages in parallel, batch operations, long polling, and prefetch messages. Adjust visibility timeout to match processing time.

Code/Config Example:

from concurrent.futures import ThreadPoolExecutor

def consumer_loop():
with ThreadPoolExecutor(max_workers=10) as executor:
while True:
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20
)
messages = response.get('Messages', [])
futures = [executor.submit(process_message, msg) for msg in messages]
for msg, future in zip(messages, futures):
try:
future.result()
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])
except Exception as e:
print(f"Processing failed: {e}")

🔍 Interviewer Notes: Tests performance optimization for EC2 consumers.

❌ Common Mistakes:

  • Single-threaded processing (underutilized resources)
  • Not matching threads to instance vCPUs
  • Not handling thread exceptions

Q56: How to use SQS message attributes?​

Technical Answer: Message attributes provide metadata (String, Number, Binary, String.Array) without parsing the message body. They can be used for routing, filtering, and adding context to messages, with a combined limit of 256KB per message.

Code/Config Example:

sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps({'order_id': '123'}),
MessageAttributes={
'order_type': {'StringValue': 'priority', 'DataType': 'String'},
'retry_count': {'StringValue': '0', 'DataType': 'Number'}
}
)

# Receive with attributes
response = sqs.receive_message(
QueueUrl=queue_url,
MessageAttributeNames=['All']
)

🔍 Interviewer Notes: Tests message metadata usage and routing patterns.

❌ Common Mistakes:

  • Putting core data in attributes instead of body
  • Exceeding 256KB total message size
  • Not requesting attributes in receive_message

Q57: How to implement delay queues in SQS?​

Technical Answer: Delay queues add a delay (0-15 minutes) before messages are visible to consumers. Set DelaySeconds at queue level for all messages, or per-message in send_message.

Code/Config Example:

# Queue-level delay (all messages delayed)
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'DelaySeconds': '300'} # 5 minutes
)

# Per-message delay
sqs.send_message(
QueueUrl=queue_url,
MessageBody='delayed',
DelaySeconds=900 # 15 minutes max
)

🔍 Interviewer Notes: Tests delay queue configuration and use cases.

❌ Common Mistakes:

  • Setting delay >15 minutes (not supported)
  • Using per-message delay when queue-level is more efficient
  • Not handling delayed messages in consumers

Q58: How to use SQS message timers?​

Technical Answer: Message timers are per-message delays set via DelaySeconds (0-900 seconds). They are equivalent to per-message delay queue settings and only apply to individual messages.

Code/Config Example:

# Delay message delivery by 5 minutes
sqs.send_message(
QueueUrl=queue_url,
MessageBody='process later',
DelaySeconds=300
)

# For delays >15 minutes, use EventBridge or Step Functions

🔍 Interviewer Notes: Tests delay mechanisms and service limitations.

❌ Common Mistakes:

  • Expecting timer >15 minutes
  • Confusing with queue-level delay
  • Not using alternative services for long delays

Q59: How to filter SQS messages with SNS?​

Technical Answer: Use SNS message filtering to route messages to different SQS queues based on message attributes. Set FilterPolicy on SNS subscriptions to match specific attribute values.

Code/Config Example:

sns.subscribe(
TopicArn=topic_arn,
Protocol='sqs',
Endpoint=queue_arn,
Attributes={
'FilterPolicy': json.dumps({
'priority': ['high', 'critical']
})
}
)

🔍 Interviewer Notes: Tests pub/sub filtering patterns and SNS-SQS integration.

❌ Common Mistakes:

  • Not setting filter policy on subscription
  • Using wrong attribute names in filter
  • Not testing filter logic before deployment

Q60: How to handle large SQS message attributes?​

Technical Answer: Message attributes have a combined size limit of 256KB (including body). Use S3 pointers for large metadata, or compress attributes before adding them to the message.

Code/Config Example:

import zlib
import json

# Compress large attribute
large_attr = json.dumps({'large': 'data'})
compressed = zlib.compress(large_attr.encode())

sqs.send_message(
QueueUrl=queue_url,
MessageBody='body',
MessageAttributes={
'compressed_data': {'BinaryValue': compressed, 'DataType': 'Binary'}
}
)

🔍 Interviewer Notes: Tests large attribute handling and size optimization.

❌ Common Mistakes:

  • Exceeding 256KB limit
  • Not compressing binary attributes
  • Storing large data directly in attributes

Q61: What are key SQS CloudWatch metrics to monitor?​

Technical Answer: Key metrics include ApproximateNumberOfMessagesVisible (queue depth), ApproximateNumberOfMessagesNotVisible (in-flight messages), ApproximateAgeOfOldestMessage (processing lag), and NumberOfMessagesSent/Received.

Code/Config Example:

# Get queue depth metric
cloudwatch.get_metric_statistics(
Namespace='AWS/SQS',
MetricName='ApproximateNumberOfMessagesVisible',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-queue'}],
StartTime=datetime.utcnow() - timedelta(hours=1),
EndTime=datetime.utcnow(),
Period=300,
Statistics=['Average', 'Maximum']
)

🔍 Interviewer Notes: Tests SQS monitoring and observability knowledge.

❌ Common Mistakes:

  • Not monitoring queue depth
  • Ignoring message age metrics
  • Missing in-flight message tracking

Q62: How to set CloudWatch alarms for SQS?​

Technical Answer: Set alarms on queue depth (ApproximateNumberOfMessagesVisible), message age (ApproximateAgeOfOldestMessage), and DLQ depth. Trigger SNS notifications or Auto Scaling actions on threshold breaches.

Code/Config Example:

cloudwatch.put_metric_alarm(
AlarmName='SQS-Queue-Depth-High',
MetricName='ApproximateNumberOfMessagesVisible',
Namespace='AWS/SQS',
Statistic='Average',
Threshold=1000,
ComparisonOperator='GreaterThanThreshold',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-queue'}],
AlarmActions=['arn:aws:sns:us-east-1:123456789:alerts']
)

🔍 Interviewer Notes: Tests alerting setup and operational readiness.

❌ Common Mistakes:

  • Not setting alarm on DLQ
  • Threshold too high/low
  • No notification actions configured

Q63: How to monitor SQS DLQ activity?​

Technical Answer: Monitor DLQ depth (ApproximateNumberOfMessagesVisible) and set alarms for any messages in DLQ. Investigate root causes and redrive messages to source queue after fixing issues.

Code/Config Example:

cloudwatch.put_metric_alarm(
AlarmName='DLQ-Non-Empty',
MetricName='ApproximateNumberOfMessagesVisible',
Namespace='AWS/SQS',
Statistic='Sum',
Threshold=1,
ComparisonOperator='GreaterThanThreshold',
Dimensions=[{'Name': 'QueueName', 'Value': 'my-dlq'}]
)

🔍 Interviewer Notes: Tests DLQ monitoring and incident response.

❌ Common Mistakes:

  • Not monitoring DLQ
  • Not investigating DLQ messages
  • Redriving without fixing root cause

Q64: How to track SQS message processing latency?​

Technical Answer: Track latency via ApproximateAgeOfOldestMessage (time since oldest message entered queue) and custom metrics for processing time (time from receive to delete).

Code/Config Example:

# Custom processing time metric
start = time.time()
process_message(msg)
duration = time.time() - start

cloudwatch.put_metric_data(
Namespace='SQS/Consumer',
MetricData=[{
'MetricName': 'ProcessingTime',
'Value': duration,
'Unit': 'Seconds'
}]
)

🔍 Interviewer Notes: Tests latency monitoring and performance tuning.

❌ Common Mistakes:

  • Not tracking processing time
  • Ignoring oldest message age
  • No alerting on latency spikes

Q65: How to audit SQS API calls?​

Technical Answer: Enable CloudTrail logging to audit all SQS API calls (SendMessage, ReceiveMessage, DeleteMessage). Use CloudTrail Insights to detect unusual activity patterns.

Code/Config Example:

cloudtrail.create_trail(
Name='sqs-audit-trail',
S3BucketName='audit-logs-bucket',
IsMultiRegionTrail=True,
IncludeGlobalServiceEvents=True
)

🔍 Interviewer Notes: Tests audit and compliance practices.

❌ Common Mistakes:

  • Not enabling CloudTrail
  • Not monitoring audit logs
  • Not detecting unusual API patterns

Q66: How to implement SQS priority queues?​

Technical Answer: SQS does not support native message priorities. Use multiple queues (high, medium, low priority) and poll higher priority queues first, or use message attributes with SNS filtering for routing.

Code/Config Example:

queues = {
'high': 'https://sqs.us-east-1.amazonaws.com/123456789/high-priority',
'low': 'https://sqs.us-east-1.amazonaws.com/123456789/low-priority'
}

def send_priority(msg, priority='low'):
sqs.send_message(
QueueUrl=queues[priority],
MessageBody=json.dumps(msg)
)

# Consumer polls high priority first
def consume():
for priority in ['high', 'medium', 'low']:
resp = sqs.receive_message(QueueUrl=queues[priority], MaxNumberOfMessages=10)
if resp.get('Messages'):
process_messages(resp['Messages'])

🔍 Interviewer Notes: Tests creative problem solving for missing native features.

❌ Common Mistakes:

  • Trying to set priority on a single queue
  • Not polling higher priority queues first
  • Starving low priority queues

Q67: How to implement SQS request-response pattern?​

Technical Answer: SQS is asynchronous, so request-response requires a correlation ID in the message, a temporary response queue, or WebSocket callback to return results.

Code/Config Example:

def request_response(req_body):
corr_id = str(uuid.uuid4())
# Send to request queue with reply queue
sqs.send_message(
QueueUrl=req_queue,
MessageBody=json.dumps(req_body),
MessageAttributes={
'correlation_id': {'StringValue': corr_id, 'DataType': 'String'},
'reply_queue': {'StringValue': resp_queue_url, 'DataType': 'String'}
}
)
# Poll response queue for correlation ID
start = time.time()
while time.time() - start < 30:
resp = sqs.receive_message(
QueueUrl=resp_queue_url,
MessageAttributeNames=['correlation_id']
)
for msg in resp.get('Messages', []):
if msg['MessageAttributes']['correlation_id']['StringValue'] == corr_id:
return json.loads(msg['Body'])

🔍 Interviewer Notes: Tests async request-response patterns.

❌ Common Mistakes:

  • Blocking forever on response
  • Not handling timeouts
  • Not cleaning up response messages

Q68: How to implement SQS message deduplication for Standard queues?​

Technical Answer: Standard queues do not deduplicate. Use DynamoDB conditional writes, Redis, or S3 to track processed message IDs with TTL to avoid duplicates.

Code/Config Example:

dedup_table = dynamodb.Table('sqs-dedup')

def process_with_dedup(msg):
msg_id = msg['MessageId']
try:
dedup_table.put_item(
Item={'message_id': msg_id, 'ttl': int(time.time()) + 300},
ConditionExpression='attribute_not_exists(message_id)'
)
except:
print(f"Duplicate: {msg_id}")
return
process(msg)

🔍 Interviewer Notes: Tests deduplication patterns for Standard queues.

❌ Common Mistakes:

  • Assuming Standard queues deduplicate
  • Not setting TTL on dedup records
  • Not handling conditional write failures

Q69: How to implement SQS partial batch failure handling?​

Technical Answer: In Lambda, return batchItemFailures with the message IDs of failed messages. Only failed messages are retried, reducing processing overhead.

Code/Config Example:

def lambda_handler(event, context):
failures = []
for record in event['Records']:
try:
process(record['body'])
except:
failures.append({'itemIdentifier': record['messageId']})
return {'batchItemFailures': failures}

🔍 Interviewer Notes: Tests Lambda SQS integration best practices.

❌ Common Mistakes:

  • Deleting all messages regardless of failure
  • Not returning batchItemFailures
  • Retrying all messages on partial failure

Q70: How to handle SQS message storms?​

Technical Answer: Use Auto Scaling to scale consumers, throttle producers, implement rate limiting on consumers, and use SNS fan-out to distribute load.

Code/Config Example:

# Consumer rate limiting
tokens = 10
last_refill = time.time()

def consume():
nonlocal tokens, last_refill
now = time.time()
tokens = min(10, tokens + (now - last_refill) * 10)
last_refill = now
if tokens < 1:
time.sleep(0.1)
return
# Process message
tokens -= 1

🔍 Interviewer Notes: Tests load handling and throttling strategies.

❌ Common Mistakes:

  • Not scaling consumers
  • Overwhelming downstream systems
  • Not throttling producers

Q71: How to use SQS message attributes for routing?​

Technical Answer: Use message attributes to route messages to different consumers or queues. Set attributes like routing_key and filter in consumers or SNS subscriptions.

Code/Config Example:

sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps(msg),
MessageAttributes={
'routing_key': {'StringValue': 'inventory', 'DataType': 'String'}
}
)

# Consumer filters by attribute
resp = sqs.receive_message(
QueueUrl=queue_url,
MessageAttributeNames=['routing_key']
)
for msg in resp.get('Messages', []):
if msg['MessageAttributes']['routing_key']['StringValue'] == 'inventory':
process_inventory(msg)

🔍 Interviewer Notes: Tests attribute-based routing patterns.

❌ Common Mistakes:

  • Not requesting attributes in receive
  • Putting routing data in message body
  • Not standardizing attribute names

Q72: How to validate SQS message attributes?​

Technical Answer: Validate attribute types (String, Number, Binary) and required attributes before processing. Use JSON schema or custom validation logic to ensure data integrity.

Code/Config Example:

def validate_attributes(msg):
attrs = msg.get('MessageAttributes', {})
if 'priority' not in attrs:
raise ValueError("Missing priority attribute")
if attrs['priority']['DataType'] != 'String':
raise ValueError("priority must be String type")

🔍 Interviewer Notes: Tests input validation practices.

❌ Common Mistakes:

  • Not validating attribute types
  • Assuming attributes always exist
  • Not handling missing attributes

Q73: How to use SQS delay queues for scheduled tasks?​

Technical Answer: Delay queues (0-15 minutes) or per-message DelaySeconds to schedule tasks. For longer delays, use EventBridge or Step Functions.

Code/Config Example:

# Delay task by 10 minutes
sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps({'task': 'cleanup'}),
DelaySeconds=600
)

🔍 Interviewer Notes: Tests scheduled task patterns with SQS.

❌ Common Mistakes:

  • Setting delay >15 minutes
  • Not using EventBridge for long delays
  • Not handling delayed messages in consumers

Q74: How to implement temporary SQS queues?​

Technical Answer: Create temporary queues with unique names, set short message retention periods, or delete after processing. Use for request-response patterns or short-lived workloads.

Code/Config Example:

temp_queue = sqs.create_queue(
QueueName=f'temp-{uuid.uuid4()}',
Attributes={'MessageRetentionPeriod': '3600'} # 1 hour retention
)
# Process and delete queue after use
sqs.delete_queue(QueueUrl=temp_queue['QueueUrl'])

🔍 Interviewer Notes: Tests temporary resource management.

❌ Common Mistakes:

  • Not deleting temporary queues
  • Forgetting retention period settings
  • Queue name collisions

Q75: How to use SQS with SNS fan-out?​

Technical Answer: SNS topics fan out messages to multiple SQS queues, where each queue receives a copy of the message for parallel processing across different consumers.

Code/Config Example:

# Subscribe multiple queues to SNS topic
for queue_arn in [inventory_arn, email_arn, analytics_arn]:
sns.subscribe(
TopicArn=topic_arn,
Protocol='sqs',
Endpoint=queue_arn
)

🔍 Interviewer Notes: Tests pub/sub architecture patterns.

❌ Common Mistakes:

  • Not setting queue permissions for SNS
  • Not filtering messages per queue
  • Processing all messages in all queues

Q76: How to implement SQS DLQ redrive?​

Technical Answer: DLQ redrive moves messages from a Dead Letter Queue back to the source queue for reprocessing. Use the start_message_move_task API or manual copy with batch operations.

Code/Config Example:

# AWS API redrive (recommended)
sqs.start_message_move_task(
SourceArn=dlq_arn,
DestinationArn=source_queue_arn,
MaxNumberOfMessagesPerSecond=100
)

# Manual redrive with batch operations
def manual_redrive(dlq_url, source_url):
while True:
resp = sqs.receive_message(QueueUrl=dlq_url, MaxNumberOfMessages=10)
messages = resp.get('Messages', [])
if not messages:
break
for msg in messages:
sqs.send_message(
QueueUrl=source_url,
MessageBody=msg['Body'],
MessageAttributes=msg.get('MessageAttributes', {})
)
sqs.delete_message(QueueUrl=dlq_url, ReceiptHandle=msg['ReceiptHandle'])

🔍 Interviewer Notes: Tests DLQ management and recovery patterns.

❌ Common Mistakes:

  • Redriving without fixing root cause of failures
  • Using manual copy instead of AWS API
  • Not monitoring redrive progress

Q77: How to implement SQS queue tags?​

Technical Answer: Tags are key-value metadata for cost allocation, access control, and organization. Use TagQueue API to add up to 50 tags per queue.

Code/Config Example:

sqs.tag_queue(
QueueUrl=queue_url,
Tags={
'Environment': 'Production',
'Team': 'Payments',
'CostCenter': '12345'
}
)

# Tag-based IAM access control
tag_policy = {
"Statement": [{
"Effect": "Allow",
"Action": "sqs:*",
"Resource": "*",
"Condition": {"StringEquals": {"aws:ResourceTag/Environment": "Production"}}
}]
}

🔍 Interviewer Notes: Tests tagging and tag-based access patterns.

❌ Common Mistakes:

  • Not using tags for cost allocation
  • Exceeding 50 tags per queue
  • Not using tag-based conditions

Q78: How to monitor SQS queue health holistically?​

Technical Answer: Monitor via composite CloudWatch alarms combining queue depth, DLQ depth, message age, and consumer health metrics for end-to-end visibility.

Code/Config Example:

# Composite alarm for queue health
cloudwatch.put_composite_alarm(
AlarmName='SQS-Queue-Unhealthy',
AlarmRule='(ALARM("SQS-Queue-Depth-High") OR ALARM("SQS-DLQ-Non-Empty"))',
AlarmActions=['arn:aws:sns:us-east-1:123456789:alerts']
)

🔍 Interviewer Notes: Tests holistic monitoring strategies.

❌ Common Mistakes:

  • Monitoring only queue depth
  • Not using composite alarms
  • Missing consumer health metrics

Q79: How to handle SQS message expiration?​

Technical Answer: Messages expire after the queue's MessageRetentionPeriod (1 minute to 14 days). Expired messages are automatically deleted and cannot be recovered.

Code/Config Example:

# Set 7-day retention
sqs.set_queue_attributes(
QueueUrl=queue_url,
Attributes={'MessageRetentionPeriod': '604800'} # 7 days
)

🔍 Interviewer Notes: Tests message lifecycle and retention policies.

❌ Common Mistakes:

  • Not adjusting retention period for long-lived messages
  • Expecting expired messages to persist
  • Setting retention too short for processing needs

Q80: How to implement SQS cost allocation tags?​

Technical Answer: Use tags like CostCenter, Environment, and Project to allocate SQS costs in AWS Cost Explorer. Activate cost allocation tags in billing settings first.

Code/Config Example:

# Add cost allocation tags
sqs.tag_queue(
QueueUrl=queue_url,
Tags={'CostCenter': 'IT-123', 'Project': 'OrderProcessing'}
)

# Activate in AWS Billing Dashboard: Billing > Cost Allocation Tags

🔍 Interviewer Notes: Tests cost management and tagging best practices.

❌ Common Mistakes:

  • Not activating tags in billing settings
  • Not standardizing tag names across resources
  • Missing cost allocation tags

Q81: How to implement SQS with Step Functions?​

Technical Answer: Use Step Functions SQS integration to send messages as Task states. Supports standard and FIFO queues, with built-in error handling and retry logic.

Code/Config Example:

{
"StartAt": "SendToSQS",
"States": {
"SendToSQS": {
"Type": "Task",
"Resource": "arn:aws:states:::aws-sdk:sqs:sendMessage",
"Parameters": {
"QueueUrl": "https://sqs.us-east-1.amazonaws.com/123456789/my-queue",
"MessageBody.$": "$.order_data"
},
"Next": "Processed"
},
"Processed": { "Type": "Succeed" }
}
}

🔍 Interviewer Notes: Tests Step Functions and SQS integration.

❌ Common Mistakes:

  • Not configuring IAM role for Step Functions
  • Using wrong SQS API action
  • Not handling task state errors

Q82: How to implement SQS with EventBridge Pipes?​

Technical Answer: EventBridge Pipes can poll SQS queues and deliver to targets (Lambda, Step Functions, etc.) with filtering and enrichment. Supports batch processing and error handling.

Code/Config Example:

events.put_pipe(
Name='sqs-to-lambda-pipe',
Source={'SqsQueueSource': {'QueueUrl': queue_url}},
Target={'LambdaFunctionTarget': {'Arn': lambda_arn}}
)

🔍 Interviewer Notes: Tests EventBridge integration patterns.

❌ Common Mistakes:

  • Not setting IAM permissions for Pipes
  • Not filtering messages at the source
  • Wrong source configuration

Q83: How to implement SQS for event-driven architecture?​

Technical Answer: Use SQS as an event buffer between producers and consumers, with Lambda triggers or EC2 consumers. Decouples services and absorbs traffic spikes.

Code/Config Example:

# Producer sends to SQS
def create_order(order):
sqs.send_message(QueueUrl=queue_url, MessageBody=json.dumps(order))
return {"status": "accepted"}

🔍 Interviewer Notes: Tests event-driven design principles.

❌ Common Mistakes:

  • Tight coupling between producer/consumer
  • No error handling for send failures
  • Not using DLQ

Q84: How to implement SQS for microservices decoupling?​

Technical Answer: Each microservice sends/receives from dedicated queues, with SNS fan-out for broadcast. Enables independent scaling and failure isolation.

Code/Config Example:

# Order service sends to multiple queues via SNS
sns.publish(
TopicArn=order_topic_arn,
Message=json.dumps(order)
)

🔍 Interviewer Notes: Tests microservices architecture patterns.

❌ Common Mistakes:

  • Shared queues between services
  • No service isolation
  • Not handling cross-service failures

Q85: How to implement SQS with AWS Lambda destinations?​

Technical Answer: Lambda destinations send async invocation results (success/failure) to SQS queues. Configure per function or alias for observability.

Code/Config Example:

lambda_client.put_function_event_invoke_config(
FunctionName='my-func',
DestinationConfig={
'OnSuccess': {'Destination': sqs_queue_arn},
'OnFailure': {'Destination': dlq_arn}
}
)

🔍 Interviewer Notes: Tests Lambda destinations and async patterns.

❌ Common Mistakes:

  • Not configuring destinations
  • Wrong destination ARN format
  • Not handling async failures

Q86: How to implement SQS with AWS Batch?​

Technical Answer: AWS Batch can use SQS as a job queue. Poll SQS for jobs, submit to Batch, and delete messages after successful job submission.

Code/Config Example:

# Batch job submission from SQS
while True:
resp = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=10)
for msg in resp.get('Messages', []):
batch.submit_job(
jobName='sqs-job',
jobQueue='sqs-job-queue',
jobDefinition='sqs-job-def',
parameters={'message': msg['Body']}
)
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])

🔍 Interviewer Notes: Tests Batch integration and job processing patterns.

❌ Common Mistakes:

  • Not deleting messages after job submission
  • Not handling Batch job failures
  • Not using long polling for Batch consumers

Q87: How to implement SQS with ECS/EKS consumers?​

Technical Answer: Deploy SQS consumers as ECS tasks or EKS pods, scale based on queue depth. Use ECS auto scaling or K8s HPA with custom SQS metrics.

Code/Config Example:

# K8s HPA for SQS queue depth
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: sqs-consumer-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: sqs-consumer
minReplicas: 1
maxReplicas: 10
metrics:
- type: External
external:
metric:
name: sqs_queue_depth
selector:
matchLabels:
queue: my-queue
target:
type: AverageValue
averageValue: "100"

🔍 Interviewer Notes: Tests containerized consumer patterns and auto scaling.

❌ Common Mistakes:

  • Not using queue depth for scaling
  • Not draining pods before termination
  • Not configuring IAM roles for tasks/pods

Q88: How to implement SQS with AWS Fargate?​

Technical Answer: Run SQS consumers as Fargate tasks, with auto scaling based on SQS metrics. Use Fargate Spot for cost savings on fault-tolerant workloads.

Code/Config Example:

# Fargate task definition for SQS consumer
task_def = {
"family": "sqs-consumer",
"containerDefinitions": [{
"name": "consumer",
"image": "my-consumer:latest",
"environment": [
{"name": "QUEUE_URL", "value": queue_url}
]
}],
"runtimePlatform": {"operatingSystemFamily": "LINUX"},
"requiresCompatibilities": ["FARGATE"]
}

🔍 Interviewer Notes: Tests serverless container patterns with SQS.

❌ Common Mistakes:

  • Not using auto scaling
  • Not handling task termination (drain messages)
  • Not using Spot instances for cost savings

Q89: How to implement SQS for backup and restore?​

Technical Answer: Backup SQS messages by consuming and storing in S3, restore by re-sending from S3 to queue. Use for disaster recovery scenarios.

Code/Config Example:

# Backup messages to S3
while True:
resp = sqs.receive_message(QueueUrl=queue_url, MaxNumberOfMessages=10)
messages = resp.get('Messages', [])
if not messages:
break
s3.put_object(
Bucket='sqs-backup',
Key=f'backup/{datetime.utcnow().isoformat()}',
Body=json.dumps(messages)
)
for msg in messages:
sqs.delete_message(QueueUrl=queue_url, ReceiptHandle=msg['ReceiptHandle'])

🔍 Interviewer Notes: Tests disaster recovery patterns for SQS.

❌ Common Mistakes:

  • Not deleting messages after backup
  • Not restoring in order (for FIFO)
  • Not testing restore process regularly

Q90: How to implement SQS with DynamoDB Streams?​

Technical Answer: DynamoDB Streams trigger Lambda to send changes to SQS for async processing. Decouples DB changes from downstream processing.

Code/Config Example:

# Lambda trigger from DynamoDB Streams
def lambda_handler(event, context):
for record in event['Records']:
if record['eventName'] == 'INSERT':
sqs.send_message(
QueueUrl=queue_url,
MessageBody=json.dumps(record['dynamodb']['NewImage'])
)

🔍 Interviewer Notes: Tests DynamoDB-SQS integration patterns.

❌ Common Mistakes:

  • Not handling all event types
  • Not batching SQS sends
  • Not handling DLQ for failed sends

Additional Core Questions Summary​

Key SQS Concepts Checklist:​

  • Standard vs FIFO queues (ordering, deduplication)
  • Visibility timeout and handling
  • Long polling vs short polling
  • Dead letter queues (DLQ)
  • Lambda integration and batch processing
  • Priority queues (multiple queues pattern)
  • Message attributes and filtering
  • Queue encryption
  • Cross-region replication

Interview Quick Reference:​

ConceptKey Point
Standard QueueAt-least-once, best-effort ordering, unlimited throughput
FIFO QueueExactly-once, strict ordering, 300 msg/s (3000 with batching)
Visibility Timeout30s default, max 12 hours
Long PollingWaitTimeSeconds=20, reduces cost
DLQmaxReceiveCount determines retry before DLQ

Senior / Principal — Amazon SQS Real World Scenarios​

Q91: SQS visibility timeout too short — duplicate processing​

Level: Senior

Scenario: A worker processes an SQS message that takes 45 seconds (video transcoding). The visibility timeout is 30 seconds (default). The message becomes visible again while still being processed. A second worker picks it up. The video is transcoded twice. Customer charged twice.

Impact: Duplicate processing. Increased cost. Possible data corruption if processing is not idempotent.

Root Cause: Visibility timeout is shorter than task processing time. The message reappears in the queue before the worker finishes, so another worker processes it.

Simple explanation: You check out a library book for 30 minutes. You need 45 minutes. The book goes back on the shelf after 30 minutes. Someone else checks it out. Two people read the same book.

Technical Answer:

  1. Set visibility timeout to exceed maximum expected processing time (e.g., 300 seconds for 5-minute tasks).
  2. Use ExtendMessageVisibility (heartbeat) to reset the timeout while processing long tasks.
  3. Make processing idempotent so duplicates don't cause harm.
  4. Monitor ApproximateAgeOfOldestMessage and NumberOfMessagesReceived vs NumberOfMessagesDeleted.

Code/Syntax Justification:

import boto3

sqs = boto3.client('sqs')

# Set visibility timeout longer than processing time
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=1,
WaitTimeSeconds=20,
VisibilityTimeout=300 # 5 minutes
)

# Heartbeat: extend visibility while processing
sqs.change_message_visibility(
QueueUrl=queue_url,
ReceiptHandle=receipt_handle,
VisibilityTimeout=300 # reset to 5 minutes again
)

Workflow Diagram:

Message received
|
v
Visibility timeout starts (300s)
|
+-- worker finishes before timeout --> delete message -> success
|
+-- worker still processing --> call ChangeMessageVisibility (heartbeat)
|
v
timeout reset, continue processing
|
+-- timeout expires before heartbeat --> message visible again
|
v
second worker picks it up
-> duplicate processing (if not idempotent)

🔍 Behind the Question: Visibility timeout, message heartbeat, idempotency, duplicate processing prevention.

❌ The Red Flags: Using default 30s timeout for long tasks; no heartbeat; non-idempotent processing; no monitoring of oldest message age.


Q92: SQS message not deleted after processing — poison pill redelivery loop​

Level: Senior

Scenario: A worker receives an SQS message, processes it, but fails to delete it (network error, exception). The message remains in the queue. After visibility timeout, it's redelivered. The worker processes it again and again. Infinite redelivery loop.

Impact: Infinite duplicate processing. Queue never drains. Wasted compute. DLQ never gets the message because maxReceiveCount isn't reached cleanly.

Root Cause: The delete call fails or isn't called. The message isn't removed from SQS after successful processing.

Simple explanation: You take a letter from your mailbox, read it, but don't throw it away. The mail carrier keeps delivering the same letter every day because it's still in your box.

Technical Answer:

  1. Always call DeleteMessage after successful processing, in a finally block or after confirming success.
  2. Wrap processing + delete in try/except; only delete on success.
  3. Use maxReceiveCount on the redrive policy so poison pills go to DLQ after N attempts.
  4. Monitor NumberOfMessagesReceived vs NumberOfMessagesDeleted — they should match.

Code/Syntax Justification:

import boto3
from botocore.exceptions import ClientError

sqs = boto3.client('sqs')

def process_message(message):
try:
# Process the message
do_work(message['Body'])
# Delete only on success
sqs.delete_message(
QueueUrl=queue_url,
ReceiptHandle=message['ReceiptHandle']
)
except Exception as exc:
# Log error, do NOT delete -> message will be redelivered
# After maxReceiveCount attempts, goes to DLQ
logger.error(f"Failed to process message: {exc}")
raise

Workflow Diagram:

Receive message
|
v
Process message
|
+-- success --> DeleteMessage --> message removed from queue
|
+-- failure --> do not delete
|
v
Visibility timeout expires
|
v
Message redelivered
|
+-- success on retry --> delete
|
+-- fail again --> repeat
|
+-- maxReceiveCount reached --> DLQ

🔍 Behind the Question: Message deletion, redelivery loops, DLQ, redrive policy, maxReceiveCount.

❌ The Red Flags: Not deleting after success; swallowing exceptions without delete; no DLQ configured; maxReceiveCount too high.


Q93: SQS FIFO queue throughput limit — 300 messages per second​

Level: Senior

Scenario: A FIFO queue is used for order processing. During peak traffic, the system needs to process 5000 orders per second. SQS FIFO queues are limited to 300 messages per second (without batching). Orders back up.

Impact: Order processing latency spikes. Orders delayed. Revenue impact during peak.

Root Cause: SQS FIFO queues have a hard limit of 300 messages/second (3000 with batching of 10). Standard queues have nearly unlimited throughput.

Simple explanation: A FIFO queue is a single checkout lane with one cashier. A standard queue is 50 lanes. If you need to serve 5000 customers, one lane isn't enough.

Technical Answer:

  1. Use batching: send up to 10 messages per SendMessageBatch call (3000 msg/s effective for FIFO).
  2. Use multiple FIFO queues with different MessageGroupId values to parallelize (each group is ordered independently).
  3. For high throughput without strict ordering, use Standard queue.
  4. Shard by MessageGroupId so different groups can be processed in parallel by different workers.
  5. Consider Kinesis Data Streams or EventBridge for very high throughput ordered events.

Code/Syntax Justification:

# Batch send (up to 10 messages)
entries = []
for i, order in enumerate(orders):
entries.append({
'Id': str(i),
'MessageBody': json.dumps(order),
'MessageGroupId': f'order-group-{order["customer_id"] % 10}',
'MessageDeduplicationId': order['order_id']
})
sqs.send_message_batch(QueueUrl=fifo_queue_url, Entries=entries)

Workflow Diagram:

Single FIFO queue (one MessageGroupId): Multiple FIFO queues / groups:
300 msg/s limit Group A -> Worker 1 (300 msg/s)
| Group B -> Worker 2 (300 msg/s)
v Group C -> Worker 3 (300 msg/s)
One worker processes sequentially Total: 900 msg/s (parallel)
|
v
Backlog at 5000 msg/s peak

🔍 Behind the Question: FIFO throughput limits, MessageGroupId sharding, batching, Standard vs FIFO trade-offs.

❌ The Red Flags: One MessageGroupId for all messages (no parallelism); no batching; expecting >300 msg/s from single FIFO group.


Q94: SQS long polling vs short polling — cost and latency​

Level: Senior

Scenario: A worker polls SQS every 1 second with WaitTimeSeconds=0 (short polling). 90% of polls return empty. Each empty poll costs money and adds latency. AWS bill is high.

Impact: High SQS API costs. High latency (worker waits for next poll). Inefficient.

Root Cause: Short polling (WaitTimeSeconds=0) returns immediately even if queue is empty. Most polls are wasted.

Simple explanation: Checking the mailbox every 10 seconds to see if mail arrived. Most of the time it's empty. You waste time walking to the mailbox.

Technical Answer:

  1. Use long polling: WaitTimeSeconds=20 (max). SQS holds the connection open until a message arrives or timeout.
  2. Reduces empty responses and API costs significantly.
  3. Slightly higher latency for first message (up to 20s wait), but better throughput and cost.
  4. Combine with MaxNumberOfMessages=10 for batch processing.

Code/Syntax Justification:

# Short polling (expensive, many empty responses)
sqs.receive_message(QueueUrl=queue_url, WaitTimeSeconds=0)

# Long polling (cost-effective, fewer empty responses)
sqs.receive_message(
QueueUrl=queue_url,
WaitTimeSeconds=20, # hold connection up to 20s
MaxNumberOfMessages=10
)

Workflow Diagram:

Short polling (WaitTimeSeconds=0): Long polling (WaitTimeSeconds=20):
Poll -> empty -> cost Poll -> wait up to 20s for message
Poll -> empty -> cost | |
Poll -> empty -> cost v v
| message arrives or timeout
v
Wasted API calls, high cost

🔍 Behind the Question: Long polling vs short polling, SQS cost optimization, API efficiency.

❌ The Red Flags: WaitTimeSeconds=0 in production; polling too frequently; not batching receives.


Q95: SQS dead letter queue — poison pill handling​

Level: Principal

Scenario: A queue processes image uploads. One message contains a corrupted image file that always crashes the processor. The message is received, fails, becomes visible again, received again, fails again. After maxReceiveCount (e.g., 5), it should go to DLQ. But the DLQ is not configured, so it loops forever.

Impact: Poison pill blocks queue processing. Other messages delayed. Worker resources wasted on the same bad message.

Root Cause: No DLQ configured, or redrive policy maxReceiveCount not set. Poison messages are never isolated.

Simple explanation: A mail carrier keeps trying to deliver a letter to an address that doesn't exist. Instead of returning it to sender, they keep trying every day forever.

Technical Answer:

  1. Configure a DLQ (separate SQS queue) for the source queue.
  2. Set RedrivePolicy with maxReceiveCount (e.g., 5) on the source queue.
  3. After maxReceiveCount failed deliveries, SQS automatically moves the message to DLQ.
  4. Monitor DLQ depth; alert when messages appear in DLQ.
  5. Investigate DLQ messages: fix the data, fix the processor, or manually discard.

Code/Syntax Justification:

import boto3

sqs = boto3.client('sqs')

# Create DLQ
dlq = sqs.create_queue(QueueName='image-processing-dlq')
dlq_url = dlq['QueueUrl']
dlq_arn = sqs.get_queue_attributes(QueueUrl=dlq_url, AttributeNames=['QueueArn'])['Attributes']['QueueArn']

# Set redrive policy on source queue
sqs.set_queue_attributes(
QueueUrl=source_queue_url,
Attributes={
'RedrivePolicy': json.dumps({
'deadLetterTargetArn': dlq_arn,
'maxReceiveCount': '5'
})
}
)

Workflow Diagram:

Message received
|
v
Processing fails
|
v
Message becomes visible again (after visibility timeout)
|
v
Received again (attempt 2 of 5)
|
+-- fail again --> repeat attempts
|
v
Attempt 5 (maxReceiveCount reached)
|
v
Message moved to DLQ automatically
|
v
Alert on DLQ depth; investigate poison pill

🔍 Behind the Question: DLQ, redrive policy, maxReceiveCount, poison pill isolation, DLQ monitoring.

❌ The Red Flags: No DLQ configured; maxReceiveCount too high (too many retries); not monitoring DLQ; DLQ messages never inspected.


Q96: SQS message deduplication — FIFO deduplication window​

Level: Senior

Scenario: A FIFO queue processes payments. The same payment request is sent twice (client retry). Both messages have different MessageDeduplicationId. SQS processes both. Customer charged twice.

Impact: Double charging. Financial loss. Customer trust damage.

Root Cause: Deduplication IDs are different for the same logical payment. SQS deduplication only works within the deduplication window (5 minutes) and only if the same ID is used.

Simple explanation: You send the same payment twice with two different reference numbers. The bank treats them as two different payments because the reference numbers differ.

Technical Answer:

  1. Use deterministic MessageDeduplicationId based on business key (e.g., order ID + idempotency key).
  2. Deduplication window is 5 minutes; duplicates within 5 minutes with same ID are dropped.
  3. For deduplication beyond 5 minutes, use application-level idempotency (DynamoDB/Redis table of processed IDs).
  4. Combine SQS FIFO deduplication with application idempotency for exactly-once processing.

Code/Syntax Justification:

# Deterministic deduplication ID based on business key
dedup_id = f"payment-{order_id}-{idempotency_key}"

sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody=json.dumps(payment),
MessageGroupId=f'customer-{customer_id}',
MessageDeduplicationId=dedup_id
)

Workflow Diagram:

Payment request sent (idempotency_key: abc-123)
|
v
SQS FIFO: MessageDeduplicationId = "payment-456-abc-123"
|
+-- same ID sent again within 5 minutes --> SQS drops duplicate
|
+-- different ID --> SQS processes as new message
|
v
Double charge (if not idempotent at app level)
|
v
Fix: deterministic dedup ID + application-level idempotency table

🔍 Behind the Question: FIFO deduplication, MessageDeduplicationId, 5-minute window, application idempotency, exactly-once processing.

❌ The Red Flags: Random/UUID deduplication IDs; relying only on SQS for >5 min dedup; no application idempotency layer.


Q97: SQS + Lambda batch processing — partial batch failure​

Level: Senior

Scenario: A Lambda function is triggered by SQS with a batch of 10 messages. The Lambda processes messages 1-9 successfully but fails on message 10. The entire batch is returned to the queue. All 10 messages are reprocessed. Messages 1-9 are processed again (duplicate work).

Impact: Duplicate processing. Increased Lambda invocations. Higher cost. Possible side effects from reprocessing.

Root Cause: Lambda SQS trigger treats the batch as atomic. If any message fails, the whole batch is retried. No partial failure handling.

Simple explanation: You grade 10 exams. The last one is unreadable. You throw away all 10 and make the students retake all of them.

Technical Answer:

  1. Use ReportBatchItemFailures to return only the failed message IDs. SQS retries only those.
  2. Process messages individually within the Lambda; track successes/failures.
  3. Make Lambda idempotent so reprocessing is safe.
  4. Reduce batch size if partial failures are frequent.
  5. Use maxReceiveCount + DLQ for messages that keep failing.

Code/Syntax Justification:

import json

def lambda_handler(event, context):
batch_item_failures = []

for record in event['Records']:
try:
process_message(record['body'])
except Exception as exc:
# Mark this message for retry only
batch_item_failures.append({"itemIdentifier": record['messageId']})

return {"batchItemFailures": batch_item_failures}

Workflow Diagram:

Lambda receives batch of 10 messages
|
v
Process messages 1-9: success
Process message 10: failure
|
v
Without batchItemFailures: With batchItemFailures:
All 10 returned to queue Only message 10 returned to queue
| |
v v
Messages 1-9 reprocessed Messages 1-9 not reprocessed
(duplicate work) Message 10 retried

🔍 Behind the Question: Lambda SQS batch processing, partial failure, batchItemFailures, idempotency.

❌ The Red Flags: Returning entire batch on single failure; not using batchItemFailures; non-idempotent Lambda.


Q98: SQS cross-account access — producer and consumer in different AWS accounts​

Level: Principal

Scenario: Account A produces messages to an SQS queue. Account B consumes them. The consumer in Account B gets AccessDenied when calling ReceiveMessage.

Impact: Cross-account messaging broken. Pipeline stalled.

Root Cause: SQS queue policy doesn't allow Account B to consume. No cross-account permissions configured.

Simple explanation: You send a package to a friend's address. The security guard won't let your friend pick it up because they're not on the guest list.

Technical Answer:

  1. Add a queue policy on Account A's queue allowing Account B's ARN to ReceiveMessage, DeleteMessage, GetQueueAttributes.
  2. Account B's IAM user/role needs permissions to call SQS on Account A's queue.
  3. Use aws:SourceAccount condition to restrict to specific account.
  4. Encryption: if using KMS, both accounts need KMS key permissions.

Code/Syntax Justification:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {"AWS": "arn:aws:iam::ACCOUNT_B:root"},
"Action": [
"sqs:SendMessage",
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
],
"Resource": "arn:aws:sqs:region:ACCOUNT_A:my-queue",
"Condition": {
"StringEquals": {"aws:SourceAccount": "ACCOUNT_B"}
}
}
]
}

Workflow Diagram:

Account A (producer) Account B (consumer)
| |
v v
Sends message to queue ReceiveMessage -> AccessDenied
| |
v v
Queue policy allows Account B Fix: add queue policy +
IAM permissions in Account B
| |
v v
Cross-account messaging works Consumer can process messages

🔍 Behind the Question: Cross-account SQS, queue policies, IAM permissions, KMS cross-account.

❌ The Red Flags: No queue policy for cross-account; missing IAM permissions in consumer account; KMS key not shared.


Q99: SQS FIFO message group ordering — strict ordering per group​

Level: Senior

Scenario: A FIFO queue processes orders for multiple customers. All orders use the same MessageGroupId. Orders for Customer A and Customer B are strictly ordered globally. Customer B's order waits for Customer A's order even though they're independent.

Impact: Unnecessary blocking. Head-of-line blocking. One slow customer delays all others.

Root Cause: All messages share one MessageGroupId. FIFO guarantees strict ordering within a group, but blocks across groups if one consumer is busy.

Simple explanation: One checkout lane for all customers. Customer A's complex return delays Customer B's simple purchase, even though they're unrelated.

Technical Answer:

  1. Use different MessageGroupId per customer (or per independent entity).
  2. Each group is ordered independently; groups can be processed in parallel by different workers.
  3. SQS FIFO supports unlimited MessageGroupId values.
  4. Within a group, messages are strictly ordered; across groups, no ordering guarantee.

Code/Syntax Justification:

# Group by customer ID for independent ordering
message_group_id = f"customer-{customer_id}"

sqs.send_message(
QueueUrl=fifo_queue_url,
MessageBody=json.dumps(order),
MessageGroupId=message_group_id,
MessageDeduplicationId=order_id
)

Workflow Diagram:

One MessageGroupId (all customers): Per-customer MessageGroupId:
Order A1 -> Order A2 -> Order B1 Group A: Order A1 -> Order A2
| Group B: Order B1 -> Order B2
v (processed in parallel)
Customer B waits for Customer A |
| v
Head-of-line blocking No blocking across customers

🔍 Behind the Question: MessageGroupId, FIFO ordering groups, head-of-line blocking, parallel processing.

❌ The Red Flags: One MessageGroupId for all messages; not leveraging group parallelism; expecting cross-group ordering.


Q100: SQS cost optimization — batch operations and long polling​

Level: Principal

Scenario: A high-throughput system sends and receives millions of SQS messages monthly. The SQS bill is dominated by NumberOfMessagesSent and NumberOfMessagesReceived API calls. Each API call is charged individually.

Impact: High AWS SQS costs. Cost doesn't scale linearly with message volume due to per-call pricing.

Root Cause: Sending/receiving messages one at a time. Not using batch operations. Not using long polling to reduce empty receives.

Simple explanation: Buying stamps one at a time at the post office counter instead of buying a book of 100. You pay the counter fee every time.

Technical Answer:

  1. Use SendMessageBatch (up to 10 messages per call) — charged as 1 batch request but sends up to 10 messages.
  2. Use ReceiveMessage with MaxNumberOfMessages=10 — receives up to 10 messages per call.
  3. Use long polling (WaitTimeSeconds=20) to reduce empty receive calls.
  4. Combine all three for maximum cost efficiency (up to 10x fewer API calls).
  5. Monitor ApproximateNumberOfMessagesVisible and CloudWatch SQS metrics.

Code/Syntax Justification:

# Batch send (10 messages, 1 API call)
entries = [
{'Id': str(i), 'MessageBody': json.dumps(msg)}
for i, msg in enumerate(messages[:10])
]
sqs.send_message_batch(QueueUrl=queue_url, Entries=entries)

# Batch receive (up to 10 messages, 1 API call, long polling)
response = sqs.receive_message(
QueueUrl=queue_url,
MaxNumberOfMessages=10,
WaitTimeSeconds=20
)

Workflow Diagram:

Individual sends/receives: Batched + long polling:
10 messages = 10 API calls 10 messages = 1 batch send + 1 batch receive
| |
v v
High API cost ~90% fewer API calls
Per-message pricing Lower monthly bill

🔍 Behind the Question: SQS pricing model, batch operations, long polling, cost optimization, API call reduction.

❌ The Red Flags: Sending one message per API call; no batching; short polling with empty responses; not monitoring SQS costs.