Most teams I've worked with start serverless for cost reasons and stay for the scaling story. You write a function, wire it to an event source, and the platform handles the rest—no EC2 instances to patch, no autoscaling groups to tune. But the real win isn't avoiding infrastructure; it's picking the right patterns for your workload so you don't paint yourself into a corner six months later.
Below are seven serverless patterns I've seen work in production hosting and SaaS environments, from straightforward API endpoints to complex multi-stage workflows. Each has tradeoffs you should know before committing.
Event-driven processing
This is the canonical serverless pattern: something happens—a file lands in S3, a message hits a queue, a database row changes—and your function wakes up to handle it. No polling, no cron jobs checking every minute. The event source invokes your Lambda directly.
Common event sources include S3 object creation, SQS messages, DynamoDB Streams, SNS topics, and EventBridge rules. The hosting world uses this pattern for log ingestion (CloudWatch Logs trigger a function that ships to an external SIEM), image processing (upload triggers a resize pipeline), and webhook handling (Stripe or GitHub POST hits API Gateway, which invokes a function).
The beauty is automatic concurrency. If a hundred files arrive in the same second, the platform spins up a hundred function instances. You don't provision capacity; you set a concurrency limit if you need to protect a downstream database.
When it breaks down
Event-driven works until your function takes longer than the event source's visibility timeout. SQS standard queues default to 30 seconds; if your function runs for two minutes, the message becomes visible again and you process it twice. Use SQS FIFO with a longer timeout, or break the work into smaller chunks.
Another gotcha: cold starts on spiky workloads. The first invocation after idle time can take seconds if you're loading heavy dependencies. Keep your deployment package small, use provisioned concurrency for latency-sensitive paths, or accept the tradeoff.
API Gateway with Lambda backends
REST and HTTP APIs in front of Lambda functions are how most teams expose serverless logic to the world. API Gateway handles SSL termination, request validation, throttling, and API keys; Lambda handles business logic. You get a fully managed API with per-request pricing and automatic scaling.
I've deployed dozens of these for hosting dashboards, webhook receivers, and lightweight admin tools. The pattern is simple: define routes in API Gateway, map each route to a Lambda function (or use a single function with internal routing), and return JSON. For WordPress sites I've worked on, this often means a Lambda function that queries RDS, checks Redis, and returns account stats without touching the main web server.
# Example serverless.yml snippet
functions:
getAccount:
handler: handlers/account.get
events:
- httpApi:
path: /account/{id}
method: GET
API Gateway has two flavors: REST APIs (older, more features like request validation and API keys) and HTTP APIs (cheaper, lower latency, simpler). For new projects, start with HTTP APIs unless you need the extra features.
Authentication and authorization
API Gateway supports Lambda authorizers, which let you run custom auth logic before the main function executes. This is where you verify JWTs, check API keys against DynamoDB, or integrate with an external identity provider. The authorizer returns an IAM policy that allows or denies the request.
For internal tools, I often use IAM authentication with SigV4 signed requests. No custom auth code, and AWS handles everything.
Scheduled jobs (cron replacement)
EventBridge rules replace traditional cron jobs in serverless environments. You define a schedule expression—rate(5 minutes) or cron(0 2 * * ? *)—and EventBridge invokes your Lambda on that cadence. No server sitting idle waiting for the next tick.
In hosting support, this pattern handles certificate expiration checks, DNS record audits, backup verification, and cleanup jobs. One function I maintain runs every six hours, queries an RDS database for stale sessions, and deletes them. The entire stack costs pennies per month because the function only runs when needed.
# Example handler for a cleanup job
import boto3
from datetime import datetime, timedelta
def handler(event, context):
cutoff = datetime.utcnow() - timedelta(days=7)
# Query and delete stale records
# Return summary for CloudWatch Logs
return {"deleted": count}
EventBridge also supports event patterns, so you can trigger functions when specific AWS API calls happen (like an IAM policy change) or when custom events match a filter. This blurs the line between scheduled jobs and event-driven processing.
Overlapping executions
If your job takes longer than the interval, you'll have concurrent executions. Set reserved concurrency to 1 if you need mutual exclusion, or use DynamoDB conditional writes to claim a distributed lock before starting work.
Stream processing with Kinesis or DynamoDB Streams
When you need to process ordered records in near real-time—think clickstream data, database change logs, or IoT telemetry—Lambda's stream integrations shine. The function polls the stream (Kinesis or DynamoDB Streams), processes batches of records, and automatically checkpoints progress.
I've used this pattern for audit logging: every write to a DynamoDB table emits a stream record, and a Lambda function writes it to S3 for long-term compliance storage. The function sees every change in order, retries on failure, and DLQs records it can't process after max attempts.
Batch size and batch window are the tuning knobs. A small batch size (10-100 records) gives you low latency; a large one (up to 10,000) amortizes overhead. The batch window lets you wait up to five minutes to fill a batch, which is useful for reducing invocations when traffic is light.
// Example Node.js handler for DynamoDB Streams
exports.handler = async (event) => {
for (const record of event.Records) {
if (record.eventName === 'INSERT') {
const item = record.dynamodb.NewImage;
// Process new item
}
}
};
Streams guarantee at-least-once delivery, so your function must be idempotent. Use a unique request ID to deduplicate, or design your logic so processing the same record twice doesn't break anything.
Fan-out workflows with SNS and SQS
Fan-out means one event triggers multiple independent actions. SNS topic sends the same message to multiple SQS queues, each consumed by a different Lambda function. This pattern decouples producers from consumers and lets you add new subscribers without changing the publisher.
A real-world example from my work: when a customer uploads a backup file, an S3 event publishes to an SNS topic. Three queues subscribe—one for virus scanning, one for indexing metadata, and one for updating billing usage. Each Lambda function handles its piece in parallel. If the billing function fails, the other two still succeed.
S3 → SNS Topic → SQS (scan) → Lambda (ClamAV)
→ SQS (index) → Lambda (metadata)
→ SQS (billing) → Lambda (usage)
SQS buffers messages when your function can't keep up, and dead-letter queues catch poison messages that fail repeatedly. The standard pattern is to set a visibility timeout longer than your function's max runtime and configure a DLQ after a few retries.
SNS also supports filtering, so subscribers only receive messages matching their filter policy. This keeps queue depth low and reduces wasted invocations.
Choreography vs orchestration
So how do you coordinate multi-step workflows?
Choreography is event-driven: each service listens for events, does its work, and publishes new events. There's no central coordinator. Step Functions call this "event-driven orchestration," but it's really choreography—services react to what happened before them. This works for simple flows where each step is independent.
Orchestration uses a central state machine (AWS Step Functions) that explicitly calls each Lambda function in sequence or parallel, handles retries, and manages state. You define the workflow in JSON (Amazon States Language), and Step Functions executes it. This gives you visibility, built-in error handling, and retry logic without writing that code yourself.
{
"StartAt": "ValidateInput",
"States": {
"ValidateInput": {
"Type": "Task",
"Resource": "arn:aws:lambda:...:function:validate",
"Next": "ProcessData"
},
"ProcessData": {
"Type": "Task",
"Resource": "arn:aws:lambda:...:function:process",
"End": true
}
}
}
I default to orchestration for anything with more than three steps or branching logic, because debugging a choreographed workflow is hard—you're grepping logs across multiple functions to reconstruct what happened. Step Functions gives you an execution history and visual diagram.
The tradeoff is cost: Step Functions charges per state transition, so tight loops get expensive fast. For high-throughput, low-latency paths, choreography with SQS and EventBridge is cheaper.
Hybrid serverless-container designs
Not everything fits Lambda's limits: 15-minute max execution, 10 GB memory, cold start latency. When you hit those walls, run containers on ECS Fargate or App Runner and trigger them from Lambda or EventBridge.
A pattern I've used for batch processing: an EventBridge schedule triggers a Lambda function that starts a Fargate task. The task runs for an hour processing millions of records, writes results to S3, and exits. The Lambda function costs a few cents; the Fargate task costs dollars but only runs when needed. No idle servers.
Another hybrid: API Gateway invokes Lambda for light requests (user lookups, session checks) but Lambda calls ECS for heavy lifting (PDF generation, video transcoding). The API stays responsive because Lambda returns immediately with a job ID, and the container sends a webhook when done.
# Lambda triggers Fargate task
import boto3
ecs = boto3.client('ecs')
def handler(event, context):
response = ecs.run_task(
cluster='processing-cluster',
taskDefinition='batch-job',
launchType='FARGATE',
networkConfiguration={...}
)
return {"taskArn": response['tasks'][0]['taskArn']}
App Runner is simpler if you just need a container that auto-scales from zero. You push a Docker image, and App Runner handles the rest—TLS, auto-scaling, health checks. It's more expensive than Lambda per invocation but easier than managing ECS.
Picking the right pattern
Start with the simplest pattern that meets your requirements. Most projects begin with API Gateway + Lambda or event-driven S3 triggers and only add complexity when needed.
If you have a clear sequence of steps with conditional logic, use Step Functions. If steps are independent and order doesn't matter, use SNS fan-out. If you're processing high-volume streams in order, use Kinesis or DynamoDB Streams. If you need long-running jobs, go hybrid with Fargate.
Don't prematurely optimize. Lambda's per-invocation cost is negligible until you're processing millions of requests per month, and the operational simplicity is worth the premium.
What about vendor lock-in?
It's a real concern. These patterns are AWS-native and don't port cleanly to other clouds. If multi-cloud is a hard requirement, consider Kubernetes-based serverless (Knative) or framework-agnostic tools like the Serverless Framework or Terraform. The tradeoff is more complexity and less integration with managed services.
In my experience supporting hundreds of hosting customers, vendor lock-in matters less than execution speed and operational burden. Pick the platform that lets you ship, and abstract the scary parts (database clients, queue wrappers) so you can swap them later if needed.
FAQ
How do I handle secrets in Lambda functions?
Use AWS Secrets Manager or Systems Manager Parameter Store and fetch secrets at runtime (cache them in global scope to reuse across invocations). Never hardcode credentials or check them into version control.
What's the cold start penalty in practice?
For Node.js or Python with small dependencies, typically 200-800 ms. Java and .NET are worse (1-3 seconds). Keep your deployment package under 10 MB, use Lambda layers for shared dependencies, and enable provisioned concurrency for latency-sensitive APIs.
Can I run stateful workloads in Lambda?
No, not directly. Lambda is stateless; each invocation starts fresh. Use DynamoDB, RDS, ElastiCache, or EFS to persist state between invocations. For session stickiness, route requests through a load balancer with session affinity, but that defeats Lambda's auto-scaling model.
How do I test serverless functions locally?
Use the AWS SAM CLI or Serverless Framework's invoke local command. They simulate the Lambda runtime and pass test events to your handler. For integration tests, deploy to a dev stack and invoke the real functions—local emulation only gets you so far.
Where to start tomorrow
If you're new to serverless, deploy one event-driven function: S3 upload triggers a Lambda that logs the filename. See how fast it runs, check the CloudWatch metrics, and iterate. If you're replacing an existing cron job, move one scheduled task to EventBridge and measure the cost difference.
Serverless isn't a silver bullet—it trades infrastructure complexity for distributed systems complexity—but these seven patterns handle the majority of workloads I've supported in hosting and SaaS environments. Pick the one that fits your problem, test it under load, and ship.