Ten Questions That Separate an OTA Platform From a Demo: A 2026 Buyer's Guide

The Demo Is Not the Product

An OTA platform demo is a carefully lit stage. A firmware update appears, a progress bar fills, a device reboots, and everyone nods. The useful questions begin after the demo, when the sales engineer has to describe what happens at 2 AM when a canary cohort starts failing, a signing key needs rotation, or a customer asks for per-device egress costs across a million devices.

The ten questions below are written for US engineering leaders evaluating an OTA toolchain in 2026. They are not exhaustive, but each one exposes a place where a weak platform hides behind a polished interface.

Question 1: What exactly happens during a rollback?

Ask for the sequence, not the feature name. Does the platform support atomic A/B rollback, or does it depend on the device bootloader? What triggers a rollback automatically, and what requires an operator? How long does a rollback take across a large cohort? What happens to data written by the new firmware if the device returns to the old build?

Walk away if rollback is described only as a button in the console. A credible answer includes rollback semantics, state migration, timeout behavior, and evidence from a live fleet.

Question 2: Who holds the signing keys, and how are they rotated?

Firmware signing is the root of trust for the entire pipeline. Ask whether keys are stored in an HSM or a managed signing service, who can use them, and whether they can be exported. Ask for the rotation procedure and the audit trail for every signing operation.

Walk away if the vendor says keys are stored in the platform database or that rotation requires a support ticket with no documented ceremony.

Question 3: How does the system handle devices that are offline for weeks?

Many connected products are not continuously connected. Ask how updates are queued, how long an update remains valid, whether downloads resume after a network drop, and how the platform prevents a returning device from receiving a stale or withdrawn release.

Walk away if the answer assumes persistent connectivity or a device that checks in every few minutes.

Question 4: What is the real per-device egress cost?

Pricing pages rarely tell the whole story. Ask for cost at 100,000 devices, 1 million devices, and 5 million devices, including CDN egress, storage, and API calls. Ask whether delta updates are supported and how they affect bandwidth. Ask what happens when a rollout is paused and resumed.

Walk away if the vendor quotes a per-device price without modeling update size, retry behavior, or regional delivery.

Question 5: How are cohorts and blast radius controlled?

Progressive deployment is more than a percentage slider. Ask how cohorts are defined, whether they can be based on hardware revision, geography, battery state, or prior update history. Ask how the platform prevents a bad build from reaching a high-value customer segment.

Walk away if cohorts are limited to random percentages and manual lists.

Question 6: What post-update telemetry feeds back into the controller?

An OTA platform that only reports download success is a delivery tool, not a deployment system. Ask which device metrics are ingested, how they are tied to release IDs, and whether the platform can pause a rollout automatically based on crash rate, check-in success, or power telemetry.

Walk away if the answer is a dashboard that a human must watch manually during the rollout.

Question 7: How are SBOMs and artifact provenance handled?

US procurement and security reviews increasingly ask for a software bill of materials with each release. Ask whether the platform stores signed SBOMs, binds them to firmware digests, and supports querying releases by component or CVE. Ask whether the platform verifies signatures before serving an artifact.

Walk away if SBOMs are treated as an attachment for the sales team rather than a release artifact.

Question 8: What happens when ten thousand devices retry at once?

Firmware delivery at scale is a distributed systems problem. Ask about idempotent update attempts, exponential backoff, jitter, and server-side rate limiting. Ask what the platform does when a CDN node fails or an update server returns errors during a peak window.

Walk away if the answer is that the cloud provider handles it. The cloud provider handles capacity; the OTA platform handles update semantics.

Question 9: What audit evidence can we export?

Ask for the retention period for deployment logs, signing events, device check-ins, and rollout decisions. Ask whether logs can be exported in a machine-readable format and whether access to them is itself audited. Ask how the platform proves that a given device received a given release.

Walk away if audit logs are only visible in the console and cannot be exported with retention guarantees.

Question 10: How do we leave?

Vendor lock-in is an engineering risk. Ask how firmware artifacts, release metadata, device records, and telemetry are exported. Ask whether the platform supports open formats and whether the device-side agent is proprietary in a way that makes migration impractical.

Walk away if the exit plan is a professional services engagement with no data export specification.

The Pattern Behind the Questions

Every question above tests the same thing: whether the platform is designed for the messy middle of an OTA program. Demos show the happy path. Procurement should test rollback, key custody, offline behavior, cost, observability, and exit. An OTA platform that answers these ten questions with specifics is worth a second meeting. One that deflects is a demo, not infrastructure.