aws-sam-python-starter

A starting point for Python services on AWS SAM that take a webhook, answer immediately, and do the slow part on a queue.

POST /webhook  ->  WebhookFunction  ->  SQS  ->  WorkerFunction  ->  DynamoDB
                    (5s timeout)          |       (300s timeout)
                                          v
                                         DLQ  ->  CloudWatch alarm

It is deliberately small: two handlers, one queue, one table. The value is in the parts that are tedious to get right and easy to leave out — batch item failures, a dead letter queue with an alarm on it, log groups that expire, deployment without static AWS keys, and tests that run without an AWS account.

Why it is shaped this way

The HTTP function does almost nothing. API Gateway gives up at 29 seconds and most callers give up sooner. Telegram, Stripe and GitHub all retry a webhook they think failed, so a slow handler turns into duplicate work. The edge function authenticates, enqueues, and returns 202.

The worker reports partial batch failures. Without ReportBatchItemFailures, one bad message in a batch of ten sends all ten back to the queue, and the nine good ones get processed again. The handler catches per record and returns only the identifiers that failed.

Queue visibility timeout is longer than the worker timeout. 360s against 300s. If it were shorter, SQS would hand the same message to a second invocation while the first was still working on it.

Nothing is named by hand. Every resource takes its name from ${AWS::StackName} and ${Environment}, so dev, staging and prod coexist in one account. There are no hardcoded account IDs, queue ARNs or table names anywhere in the template — those are what make a template impossible to reuse and awkward to publish.

The secret lives in SSM, not in a CloudFormation parameter. Parameters passed at deploy time end up in the stack's parameter list and in whatever CI log printed the command. The webhook function reads the parameter at cold start instead, and its IAM policy grants ssm:GetParameter on that one path.

Deployment uses OIDC. deploy.yml assumes a role via GitHub's identity token. There is no AWS_ACCESS_KEY_ID in repository secrets to leak or rotate.

The table is retained. DeletionPolicy: Retain on the DynamoDB table, so sam delete does not take the data with it.

Layout

src/
  handlers/
    webhook.py    # authenticate, enqueue, 202
    worker.py     # SQS consumer, partial batch failures
  lib/
    aws.py        # lazily built, cached boto3 clients
    config.py     # environment variables, read once per cold start
tests/            # pytest + moto, no AWS account required
template.yaml     # the whole stack

Clients are built lazily rather than at import time. A module that creates a boto3 client on import cannot be imported in a test without credentials, and it slows down every cold start whether or not the client is used.

Use it

make setup
make test          # pytest against moto
make lint          # ruff, black, cfn-lint

To deploy into your own account:

# 1. Store the shared secret the webhook will check
make put-secret SECRET=$(openssl rand -hex 32) ENVIRONMENT=dev

# 2. Deploy
make deploy STACK_NAME=my-service ENVIRONMENT=dev REGION=eu-central-1

# 3. Watch the worker
make logs

make deploy prints the webhook URL. Call it with the secret in the X-Webhook-Secret header:

curl -X POST "$WEBHOOK_URL" \
  -H "X-Webhook-Secret: $SECRET" \
  -H "Content-Type: application/json" \
  -d '{"hello": "world"}'
# {"id": "...", "status": "accepted"}

For CI deploys, set the repository variables AWS_ROLE_ARN and AWS_REGION, and trust the GitHub OIDC provider in that role's trust policy.

Making it yours

1. Replace the body of process_message in src/handlers/worker.py. That function is the only place the example does anything specific.
2. Adjust the key schema in template.yaml if pk/sk does not suit your access pattern.
3. Change the authentication in src/handlers/webhook.py if your caller signs requests rather than sending a shared secret. Stripe and GitHub both send an HMAC of the body — verify it against the raw body, before parsing.

Tests

make test

moto stands in for SQS, DynamoDB and SSM, so the suite runs offline and in CI without credentials. Coverage gate is 85%.

License

MIT — see LICENSE.