The Poll That Ate the Fleet: Designing Check-In Intervals That Scale Past a Million Devices

The first line of code in nearly every OTA system looks the same. A device boots, connects to the network, and asks a server a simple question: is there anything new for me?

It's a humble request. A few hundred bytes, a TLS handshake, a JSON response that usually says no. At a hundred devices, it's invisible. At ten thousand, it's a rounding error on your cloud bill. At a million devices checking every hour, you're looking at roughly 24 million requests a day — 8.7 billion a year — for the privilege of telling devices there's nothing to do.

The check-in interval is the most consequential configuration decision in an OTA system, and it's almost always the least deliberate one. Someone picks "every hour" during a prototype, the constant ships to production, and five years later it's load-bearing infrastructure that nobody remembers choosing.

The Arithmetic Nobody Runs

Start with the basics. A fleet of N devices polling every I seconds generates N × 86,400 / I requests per day. The problem is that this number is a floor, not a ceiling, because real fleets don't poll politely.

Devices reboot after power outages, and they reboot in correlated waves — the same ice storm that knocks out power in Dallas brings every smart thermostat back online within the same twenty-minute window. Without countermeasures, your fleet's reconnection behavior is a thundering herd with a geographic coordinate. We've reviewed post-incident reports from teams whose update servers went down not because of a firmware push, but because a regional utility restored power.

The second problem is that a fixed interval is almost always wrong in both directions simultaneously. It's too frequent for a device that will go months without an update, and it's too infrequent when you're trying to push a security patch to a vulnerable fleet. You cannot tune a constant to fix a distribution problem.

Jitter: The Cheapest Reliability You'll Ever Buy

If you take one thing from this article, take this: never let devices check in on a synchronized schedule. If every device polls at the top of the hour, your traffic graph develops cliffs, and your load balancer's autoscaling — which reacts to load after it happens — will always be a few minutes behind the spike.

The standard remedy is jitter. Instead of polling at exactly 3,600 seconds, a device polls at 3,600 seconds plus or minus a random offset. Full jitter — choosing a random delay anywhere in the interval — flattens traffic almost completely, at the cost of some predictability. Equal jitter, which randomizes only a fraction of the delay, preserves approximate timing while still spreading the load. Either beats the synchronized cliff.

This isn't exotic knowledge. AWS SDKs have shipped default jitter on retry logic for years precisely because unjittered traffic patterns take down systems that were sized for their average load. The same principle applies to check-ins, and yet we still see production fleets where every device phones home at minute zero.

Backoff and the Long-Lived Fleet

Exponential backoff is usually discussed in the context of failures — retry after 1 second, then 2, then 4. But the same mechanism solves a different problem on the success path: devices that have checked in and found nothing many times in a row are statistically unlikely to find something soon.

A well-designed client increases its interval multiplicatively with each empty response — 15 minutes, 30 minutes, an hour, two hours, four — capped at some maximum. The moment the server signals that an update is available (or the device receives a push notification, more on that below), the backoff resets.

The economics are compelling. A device that checks hourly generates 8,760 requests a year. A device that backs off to a six-hour ceiling after a week of nothing generates around 1,400. Across a million-device fleet, that's the difference between a modest service and a genuinely expensive one — and remember, most of those requests carry zero information in either direction.

Let the Server Set the Clock

Hardcoding the interval in firmware is a trap, because firmware is the one thing you can't easily change after deployment. The better pattern is server-driven configuration: the device's check-in response includes a next_check_in value, and the client treats it as authoritative.

This turns your polling cadence from a firmware constant into an operational lever. During a security incident, you can drop the fleet's interval to five minutes and get a patch deployed in hours. During quiet periods, you can stretch intervals to reduce load and cost. During a server-side incident, you can push intervals out to shed traffic while you recover. None of this requires an OTA update to change, which is the entire point — the system that delivers your updates should not itself require an update to be tunable.

The protocol design matters here. Define a sane default the client falls back to if the response is malformed or the field is missing, and clamp server-provided values to firmware-defined bounds so a misconfigured backend can't accidentally command a million devices to poll every second. We've seen that failure mode too. It looks exactly like you'd expect.

Push Beats Poll, When You Can Get It

Where the network allows it, a persistent connection or push channel changes the calculus entirely. A device that can be woken on demand can afford to check rarely on its own, treating the poll as a heartbeat and the push as the actual update signal. Cellular LPWA networks, Thread, and modern mobile OS push services all offer variations on this.

The catch is that push channels fail quietly. NAT timeouts, carrier quirks, and firewall behavior on consumer routers all silently break persistent connections, which is why even push-first architectures need a polling fallback. Treat the push as the fast path and the backoff poll as the correctness guarantee.

A Decision Framework

Match your strategy to three variables: fleet size, connectivity profile, and update urgency.

Small fleets (under 10,000 devices), always-on connectivity. Fixed intervals with jitter are fine. Your load is trivial; optimize for simplicity and debuggability instead.

Mid-size fleets (10,000 to 500,000), mixed connectivity. Server-driven intervals with exponential backoff on empty responses, plus jitter on reconnect. This is the sweet spot where the arithmetic starts to matter but the architecture is still manageable.

Large fleets (500,000+), or high update urgency. Push-first where available, aggressive backoff ceilings, server-driven cadence with firmware clamps, and — critically — load testing that simulates correlated reconnection events, because your region's next ice storm is a when, not an if.

The two-cent question — when should a device check? — turns out to have a two-million-dollar answer if you never ask it. Ask it early, put the answer under server control, and let the math, not the prototype, set the clock.