Serverless architecture promises zero server management and pay-per-execution billing. In practice, cold starts kill your API response time, vendor-specific APIs trap you in one cloud, and debugging a function crash at 3 AM without SSH feels like flying blind.
I've supported teams migrating to serverless and teams ripping it out six months later. The pattern repeats: a developer reads the marketing, deploys a Lambda function in ten minutes, celebrates infinite scale, then hits the first 800ms cold start or discovers the bill scales faster than traffic. This post walks through the real tradeoffs—latency, lock-in, observability, and cost—so you can decide when serverless makes sense and when a container or VM is the better bet.
What serverless actually means
Serverless doesn't mean no servers. It means you write functions and the cloud provider runs them on-demand. AWS Lambda, Azure Functions, Google Cloud Functions, and Cloudflare Workers all follow the same model: upload your code, configure a trigger (HTTP request, queue message, file upload), and the platform spins up an execution environment when an event arrives.
You pay per invocation and per compute time, usually rounded to 100ms increments. No charge when the function sits idle. No OS patches, no load balancer config, no capacity planning. The provider handles scaling from zero to thousands of concurrent executions automatically.
That's the promise. The reality is more textured.
Cold-start latency: the tax on intermittent traffic
Cold starts happen when the platform has no warm execution environment ready. The provider must allocate a container or VM slice, inject your code, initialize the runtime, and finally run your handler. This takes time.
How much time depends on runtime and package size. A minimal Node.js function might cold-start in under 200ms on AWS Lambda. A Python function importing heavy ML libraries can take multiple seconds. Java and .NET are notorious for second-plus cold starts because the JVM or CLR must initialize.
Warm starts—when a recent execution environment is reused—run in single-digit milliseconds. The problem is predicting when you'll hit a cold start. Traffic spikes, deployments, and idle periods all trigger cold initialization. Even under steady load, providers recycle execution environments after minutes of inactivity or for internal maintenance.
For background jobs and async processing, cold starts are tolerable. For user-facing APIs, a 1-2 second delay kills the experience. E-commerce checkouts, auth endpoints, and real-time dashboards can't absorb that latency.
Containers and VMs don't have this problem because you keep instances running. An always-on Nginx container responds in milliseconds, every time. You pay for idle capacity, but you buy predictable performance.
Vendor lock-in: the hidden cost of proprietary APIs
Every major cloud has its own function signature, deployment format, and auxiliary service APIs. AWS Lambda uses the lambda_handler(event, context) interface and integrates tightly with API Gateway, DynamoDB, and S3 via IAM roles. Azure Functions use different bindings and integrate with Cosmos DB and Blob Storage. Google Cloud Functions expect yet another signature and tie into Firestore and Pub/Sub.
Your code ends up littered with vendor-specific SDK calls. Need a secret? AWS Lambda reads from Secrets Manager; Azure Functions use Key Vault; GCP uses Secret Manager. Each has different API semantics and error handling. Switching clouds means rewriting integration points, not just deploying the same artifact somewhere else.
Frameworks like Serverless Framework and AWS SAM try to abstract deployment, but they don't abstract runtime APIs. Once you call s3.getObject() or blobService.downloadToStream(), you're locked in. Open-source alternatives like Knative and OpenFaaS let you run functions on Kubernetes with a vendor-neutral interface, but then you're managing Kubernetes—which defeats the serverless premise.
Containers offer more portability. A Docker image runs the same on AWS ECS, Azure Container Instances, Google Cloud Run, DigitalOcean, or a VPS. You control the base image, dependencies, and runtime. Moving clouds means updating a registry URL and cluster endpoint, not rewriting code.
VMs are even more portable. An Ubuntu 24.04 image boots anywhere. You manage more, but you own the stack.
Debugging and observability: flying without SSH
Serverless functions are ephemeral and opaque. You can't SSH in. You can't attach a debugger mid-execution. When a function crashes, you get a stack trace in CloudWatch Logs or Application Insights if you're lucky, but recreating the exact state that triggered the failure is hard.
Local development is awkward. AWS SAM and the Serverless Framework provide local emulators, but they don't perfectly replicate cloud behavior—especially around IAM permissions, VPC networking, and service integrations. A function that works locally can still fail in production because the Lambda execution role lacks a policy, or the VPC subnet has no route to the internet, or a dependency timeout is enforced differently.
Distributed tracing helps. AWS X-Ray, Azure Monitor, and third-party tools like Datadog and Honeycomb can stitch together function invocations across services. But setting up tracing means instrumenting every function and paying for trace storage. In a container or VM, you can run strace, tcpdump, or gdb. That level of introspection doesn't exist in serverless.
Log aggregation is mandatory. Functions spin up and down constantly, so logs scatter across ephemeral instances. Centralized logging (CloudWatch, Stackdriver, Splunk) is the only way to search errors across invocations. You'll spend time tuning log retention and sampling to control costs.
Cost structure: predictable vs. spiky
Serverless pricing is simple on paper: pay per request and per GB-second of compute. Run a function for 100ms with 512MB of memory, pay a fraction of a cent. No requests, no charges. This makes serverless cheap for low-traffic projects and event-driven workflows that run sporadically.
But cost scales unpredictably with traffic. Double your requests, double your bill—exactly. There's no volume discount until you hit reserved capacity tiers (which reintroduce the fixed-cost problem you were avoiding). A sudden traffic spike from a bot, a misconfigured retry loop, or a viral social post can inflate your bill by 10x overnight.
I've seen support tickets where a developer deployed a Lambda function behind an API Gateway, forgot to set a rate limit, and racked up thousands in charges from a scanning bot hitting the endpoint non-stop. With a VM, the worst case is saturating the instance's CPU or bandwidth—the monthly cost stays the same.
Containers offer a middle ground. Services like AWS Fargate, Google Cloud Run, and Azure Container Apps bill per vCPU-second but let you set min/max instance counts. You can scale to zero for true serverless billing or keep a minimum pool warm to avoid cold starts, depending on the workload.
For steady, predictable load, VMs are cheaper. A reserved instance or dedicated server gives you fixed monthly cost and better price-per-compute than serverless at scale. For bursty or unpredictable load, serverless can be cheaper—if you watch for runaway invocations.
So when does serverless actually make sense?
Serverless shines in a few specific patterns:
- Event-driven background jobs: processing S3 uploads, resizing images, sending emails after a database insert. Latency doesn't matter, and the workload is naturally asynchronous.
- Infrequent scheduled tasks: a cron job that runs once an hour to clean up old records or sync data between systems. Paying per execution beats running a VM 24/7.
- Rapid prototyping and MVPs: deploy a REST API in minutes without provisioning infrastructure. Good for demos and early-stage products where simplicity trumps performance.
- Traffic with extreme variance: a webhook receiver that gets ten requests per day normally but 10,000 during a campaign. Serverless scales instantly without pre-provisioning capacity.
Serverless is a poor fit for:
- Latency-sensitive APIs: user-facing endpoints that need sub-100ms responses can't tolerate cold starts.
- Long-running processes: functions have strict timeout limits (15 minutes on Lambda). Video transcoding, model training, and large ETL jobs exceed these limits.
- High-throughput sustained workloads: a service handling thousands of requests per second continuously costs less on dedicated instances than on per-invocation billing.
- Stateful applications: serverless functions are stateless by design. Session state, file handles, and database connections don't persist across invocations. You can work around this with external storage (Redis, DynamoDB), but it's clunky.
How containers and VMs compare
Containers give you the packaging benefits of serverless—immutable images, fast deployments—without the runtime constraints. You control startup time by keeping instances warm. You choose the base image, runtime version, and system libraries. Orchestrators like Kubernetes or Docker Swarm handle scaling, health checks, and zero-downtime deployments.
The tradeoff is operational overhead. You manage the cluster, configure autoscaling policies, monitor resource usage, and handle OS patches. Managed services like AWS ECS, Google Cloud Run, and Azure Container Apps reduce the burden but don't eliminate it.
VMs offer maximum control and compatibility. Any software that runs on Linux runs in a VM. No runtime restrictions, no artificial timeouts, no vendor-specific APIs. You can SSH in, install debugging tools, and inspect live processes. VMs are also the most portable option: an AMI or qcow2 image boots on any hypervisor.
The downside is slower deployment and coarser scaling. Spinning up a VM takes seconds to minutes, not milliseconds. Scaling means launching new instances, not just adding function invocations. You pay for idle capacity unless you aggressively autoscale.
Debugging strategies without shell access
When a serverless function fails in production, start with structured logs. Log every function entry, key variable values, and external API calls with correlation IDs. JSON-formatted logs make parsing easier downstream.
Use environment-specific config to reproduce issues. If a function fails in production but works in dev, compare environment variables, IAM permissions, VPC settings, and dependency versions. Provisioned concurrency can help isolate cold-start issues—it keeps a pool of warm execution environments ready.
Integrate synthetic monitoring. Trigger your function on a schedule with known test input and assert expected output. This catches regressions faster than waiting for real traffic to hit broken code.
For complex failures, reproduce the issue locally with event payloads captured from production logs. Most cloud CLIs let you invoke functions with a JSON payload file, and local emulators can replay events with matching context.
Managing costs before they spike
Set billing alerts and budget limits in your cloud console. Configure alerts at 50%, 75%, and 90% of your monthly budget so you catch runaway costs before month-end.
Implement rate limiting at the API gateway layer. Throttle requests per IP or API key to prevent abuse and accidental retry storms. Most gateways support burst limits and quota enforcement out of the box.
Use reserved concurrency to cap maximum parallel executions. If your function can only handle 100 simultaneous invocations without overloading a backend database, set the concurrency limit to 100. Requests beyond that get throttled instead of executing and failing.
Monitor invocation counts, duration, and error rates in your observability platform. Sudden changes in any metric often signal a problem—traffic patterns shifting, a new bug introduced, or a downstream dependency timing out.
What to evaluate before choosing serverless
Ask these questions:
- What's the 99th percentile latency budget for this workload? If it's under 500ms, cold starts will hurt.
- How often will the function run? Sporadic invocations favor serverless; continuous load favors dedicated compute.
- Does the code depend on vendor-specific APIs? Heavy integration with one cloud's services increases lock-in risk.
- How complex is the debugging and testing workflow? Functions that call multiple external services are harder to test locally.
- What's the worst-case cost if traffic spikes 10x overnight? Run the math on per-invocation pricing vs. fixed instance costs.
- Can the workload tolerate stateless execution and cold starts? Some patterns (like WebSockets or long-polling) don't fit the function model.
If most answers point toward low latency requirements, steady traffic, and complex dependencies, a container or VM is safer. If the workload is bursty, asynchronous, and event-driven, serverless can save time and money.
Frequently asked questions
Can I run serverless functions on my own infrastructure?
Yes, using Knative, OpenFaaS, or Fission on Kubernetes. You lose the operational simplicity of managed services but gain control and avoid vendor lock-in.
How do I keep a serverless function warm to avoid cold starts?
Use provisioned concurrency (AWS Lambda), minimum instances (Google Cloud Run), or schedule a synthetic request every few minutes to keep one instance alive.
Is serverless cheaper than a VPS for a low-traffic site?
It depends. A basic VPS costs around $5-10/month. If your serverless invocations stay under a few thousand per month, the free tier might cover it. Beyond that, compare the math.
What's the maximum execution time for serverless functions?
AWS Lambda caps at 15 minutes. Azure Functions allow up to 10 minutes on the consumption plan, longer on premium plans. Google Cloud Functions go up to 60 minutes for second-gen functions.
Can I use serverless for real-time applications like chat or gaming?
Not easily. Serverless functions are stateless and designed for short-lived executions. Real-time apps need persistent connections (WebSockets), which don't fit the function model. Consider containers or VMs instead.
When to use what
Serverless works best for event-driven background tasks, infrequent scheduled jobs, and bursty workloads with relaxed latency requirements. Cold starts, vendor lock-in, and debugging constraints make it a poor choice for latency-sensitive APIs, long-running processes, and high-throughput sustained loads.
Containers split the difference: you keep the packaging and deployment benefits of serverless while controlling startup time and avoiding runtime restrictions. VMs give maximum portability and introspection at the cost of managing more infrastructure.
Pick the tool that fits the job. Don't force serverless into patterns it wasn't designed for, and don't dismiss it for workloads where it genuinely simplifies operations. The right answer depends on your latency budget, traffic patterns, and willingness to trade operational overhead for flexibility.