Lambda Cold Starts: When the Latency Math Says No

Cold starts are the thing AWS hoped you wouldn't think too hard about. For high-traffic endpoints, they're a rounding error. For the low-traffic internal tools and healthcare portals I build, they can be the entire user experience.

I spent a chunk of last year migrating a client's internal ops tool off a small EC2 instance onto a Lambda-backed API Gateway setup because the infrastructure team wanted to "go serverless." The Lambda functions worked. The cold start latency was embarrassing. We moved it back. Here's what I learned and how I now think about the tradeoff before committing.

What Lambda Cold Starts Actually Are

When a Lambda function hasn't been invoked recently, AWS has torn down the execution environment. The next request has to spin one back up — download your deployment package, initialize the runtime, run your initialization code — before it can actually handle the request. That's the cold start.

Behind API Gateway, this happens synchronously. The client is sitting there waiting. There's no queue, no background warmup. The user clicked a button and now they're watching a spinner.

For a Python or Node function with a 50MB package, cold starts are annoying — maybe 500ms to 1.5s depending on the runtime and region. For anything running on a JVM, or for a PHP runtime via Brachtel/Bref with a fat Composer vendor directory, you can be looking at 3-6 seconds on a cold start. I've seen worse.

The Latency Math for Low-Traffic Endpoints

Here's the part nobody puts in the serverless pitch deck.

If your endpoint gets 10,000 requests per day, distributed reasonably evenly, you're hitting it roughly once every 8.6 seconds. Lambda's execution environment stays warm for somewhere between 5 and 15 minutes of inactivity — AWS doesn't publish an exact number, and it varies. In practice, on a busy endpoint, you'll almost never see a cold start. The function stays warm because there's always another request coming.

Now flip the scenario. An internal tool used by 12 people. A client portal that gets hit a few times a day. A webhook endpoint that fires when a HL7 message comes in from a lab system — which might happen 40 times a day, but not on a predictable schedule.

On those, every request is potentially a cold start. Or close to it. I've had endpoints where I watched CloudWatch logs and 80% of invocations were cold. The function was sleeping almost constantly because no one was hitting it.

The math: if your p50 warm latency is 120ms and your cold start is 2,800ms, and 80% of your invocations are cold, your real-world p50 is closer to 2,300ms. That's not a web application. That's a loading screen.

What I Actually Tried

Before pulling the plug on Lambda for that ops tool, I went through the standard mitigation playbook.

Provisioned Concurrency. AWS lets you pay to keep a specified number of execution environments initialized and ready. It works. Cold starts disappear. The cost is the problem — you're paying for idle compute 24/7, which eliminates the main cost argument for Lambda on low-traffic workloads. For a function invoked 40 times a day, provisioned concurrency is more expensive per-request than just running a t4g.small.

Scheduled warmup pings. A CloudWatch Events rule hitting the function every 4 minutes to keep it warm. This is a hack and I've shipped it anyway. It works until it doesn't — Lambda can have multiple concurrent execution environments, and your ping only warms one of them. If you get a small burst, you're still cold on the others. Also you're now maintaining a cron job whose only job is to prevent your infrastructure from working as designed.

Reducing package size. Real gains here. For a Bref/PHP setup, being ruthless about what goes in the vendor directory — stripping test files, unused locale data, development dependencies — can cut cold start time meaningfully. A 40MB package cold starts noticeably faster than a 120MB one. But you're still working around the problem, not solving it.

Here's a simple PHP example of what a Bref handler looks like, so you can see what I'm talking about in terms of initialization overhead:

<?php

require __DIR__ . '/vendor/autoload.php';

use Bref\Context\Context;
use Bref\Event\Http\HttpHandler;
use Bref\Event\Http\HttpRequest;
use Bref\Event\Http\HttpResponse;

// Everything here runs on EVERY cold start before the first request.
// Boot Laravel, resolve the container, load config, connect to DB if you're eager-loading.
$app = require_once __DIR__ . '/bootstrap/app.php';
$kernel = $app->make(Illuminate\Contracts\Http\Kernel::class);

return new class($kernel) extends HttpHandler {
    public function __construct(private $kernel) {}

    public function handleRequest(HttpRequest $request, Context $context): HttpResponse
    {
        $illuminateRequest = IlluminateRequest::createFromBase(
            SymfonyRequest::create(
                $request->getUri(),
                $request->getMethod(),
                $request->getQueryParameters(),
                [],
                [],
                [],
                $request->getBody()
            )
        );

        $response = $this->kernel->handle($illuminateRequest);

        return new HttpResponse(
            $response->getContent(),
            $response->headers->all(),
            $response->getStatusCode()
        );
    }
};

That bootstrap call — the require_once for the Laravel app — runs every cold start. It's not catastrophic, but it's not free either, especially once you factor in service providers, config loading, and any eager-loaded singletons. Keep your bootstrap lean or you're paying the price in cold start time on every sleeping function.

The Actual Decision Framework I Use Now

Before recommending Lambda behind API Gateway, I run through this honestly:

What's the expected request rate? If the endpoint is getting fewer than one request per minute on average, I assume most invocations will be cold. Plan accordingly.

Who's waiting? An async webhook processor where the upstream system is retrying? Cold starts are tolerable — maybe even irrelevant. An end user clicking a button on a portal expecting a snappy response? Different conversation entirely.

What's the runtime? Node or Python with a small package: cold starts are survivable, often under a second. PHP via Bref with a full Laravel app: budget 2-4 seconds on cold, more if your vendor directory is fat. Java: I avoid it for latency-sensitive Lambda work entirely.

What does the cost comparison actually look like? Do the math. A t4g.small with Nginx and PHP-FPM runs about $12/month reserved, handles concurrent requests natively, zero cold starts. If provisioned concurrency would cost more than that — and for low-traffic workloads it often does — the serverless cost argument is gone.

Is the team operationally comfortable with EC2/containers? Sometimes Lambda wins on ops simplicity even if the latency math is marginal. But if you're already running other EC2 instances or an ECS cluster, adding another small instance isn't the burden it's sometimes made out to be.

When I'd Still Reach for Lambda

Lambda behind API Gateway is genuinely the right call in several situations I hit regularly.

Event-driven async processing — S3 triggers, SQS consumers, SNS handlers. The user isn't waiting. Cold starts don't matter.

High-traffic, predictable endpoints. Concurrent execution environments stay warm. The 0.01% cold start rate is a stat, not an experience.

Burst workloads that would otherwise require over-provisioned servers. A print job processing queue that spikes to 500 concurrent jobs for 20 minutes a day, then goes quiet. Lambda handles that burst elastically in a way a fixed fleet doesn't.

When I genuinely don't know the traffic pattern yet and want to defer the infrastructure decision. Lambda is a fine placeholder if you're honest with yourself about revisiting it.

When I Walk Away

I won't put Lambda behind API Gateway for user-facing, synchronous, low-traffic endpoints anymore without an explicit conversation about cold starts first. That healthcare portal I mentioned — 8 users, used intermittently throughout the day — Lambda would have been a disaster. It runs on a t4g.small, responds in 90ms, costs $11 a month. Nobody's complaining.

The same goes for internal ops tools, client dashboards with a small user base, and any endpoint where the person waiting is a human who will interpret a 3-second delay as "this thing is broken."

Serverless is a deployment model, not a universal architecture. The cold start problem is real, well-documented, and still somehow undersold when teams are evaluating the move. Do the latency math before you commit — not after your client calls wondering why their portal is slow.