SageMaker Bidirectional Streaming: Usage-Based AWS Marketplace Metering

On November 11, 2026, AWS added usage-based metering for SageMaker bidirectional streaming sold through AWS Marketplace — so model providers can bill streaming speech and voice agents for units consumed mid-session, not just for hours the endpoint stayed warm. Infrastructure charges for the SageMaker real-time endpoint are unchanged; this is the software line item Marketplace buyers see consolidated on their AWS bill.

Deepgram is the launch partner, extending a chain explainx.ai already covered when Deepgram acquired Daily and kept Pipecat open and when we walked through vLLM bidirectional KV transfer on AWS P5en.

TL;DR

Why streaming broke the old header model

Request-response SageMaker Marketplace metering could stash a unit count in an HTTP response header when the response completed. Bidirectional streaming keeps a session open — 30 seconds or 30 minutes in AWS's example — and usage accrues continuously. Headers fire once; streaming needs incremental records.

Hourly software pricing also bills idle capacity. Usage-based reporting lets providers align Marketplace software with seconds of audio processed (or whatever unit the offer defines) while buyers still pay endpoint infra for provisioned GPUs or CPUs.

That matches the mental model in Sam Altman's metered-utility metaphor — but here the meter is explicit JSON on a second socket, not a post-hoc invoice guess.

Companion metadata WebSocket

When a session starts, SageMaker may open two connections:

1. Data connection — existing inference stream.
2. Metadata connection — metering-only JSON text frames.

Both carry the same X-Amzn-SageMaker-Request-Id so the model server correlates usage with the active session. Providers define what counts as a unit; AWS maps it to Marketplace inference.count with a single per-unit price (up to three decimal places).

AWS recommends incremental ConsumedUnits as usage accrues so an abrupt disconnect loses less unreported time. Transport delivery alone does not prove billing — reconcile against Marketplace reports and buyer charges.

Optional X-Amzn-SageMaker-Metadata-Stream-Required: true stops inference if metadata fails; default false keeps audio flowing in data-only mode without software metering — important for builders choosing availability vs revenue capture.

Builder checklist for model providers

1. Return opt-in headers on the data WebSocket upgrade response.
2. Accept the metadata socket on the default or custom path.
3. Emit one JSON metering record per frame, max once per second.
4. Use ClientToken UUIDs so duplicate frames dedupe silently.
5. Run Limited listing tests with a known quantity (AWS's 720 seconds = 12 minutes example).
6. Tear down test endpoints to avoid infra bleed.

AWS points providers to a SageMaker Marketplace Onboarding Agent Skill with starter templates — the same "skills for shipping" pattern as AWS Quick Skills, but for billing integration.

What changes for voice agent teams

If you deploy contact-center STT, captioning, or Fish Audio-style duplex agents on SageMaker:

• Marketplace software can track metered audio through consolidated AWS billing and private offers.
• Endpoint infra is still a separate line — capacity planning stays tied to concurrent streams and model size.
• Client code for Deepgram's launch path needs no client-side metering config per AWS; the container emits records server-side.

Pair this with Confucius stable-prefix streaming ASR research on latency, but remember finance teams will now read per-second indexes on the same dashboards as technical latency metrics.

Header reference for implementers

AWS documents three response headers on the data WebSocket upgrade that control metadata behavior:

The metadata socket shares X-Amzn-SageMaker-Request-Id with the data socket so your server attributes bursts of audio to the correct billing session even when clients reconnect.

JSON schema constraints

Each metering frame must be a single JSON object in a text WebSocket frame, not an array batch. Fields:

• Metering.Dimension — must be the literal inference.count for this capability.
• Metering.ConsumedUnits — non-negative number; zero is accepted but not billed.
• Metering.ClientToken — optional UUID for idempotency within the connection (64 char max).

Exceeding one message per second or 512 bytes risks silent drops — design aggregators accordingly. For audio priced per second, AWS's worked example uses 720 units after twelve minutes of counted audio, not twelve minutes of wall clock if silence is excluded by provider rules.

Deepgram customer path on Marketplace

AWS lists a concrete onboarding chain:

1. Subscribe to a Deepgram Voice AI streaming listing such as Nova-3 monolingual STT on Marketplace.
2. Deploy a SageMaker real-time endpoint from that model package.
3. Connect with the SageMaker bidirectional streaming API (HTTP/2 event stream client side, WebSocket server side).

Deepgram's container handles metering frames; application clients do not parse billing. That keeps mobile and browser voice clients thin — important for teams already standardizing on Pipecat transports but selling through AWS contracts.

Reconciliation discipline

AWS repeats that transport delivery is not billing. Model owners should:

• Compare server-side metering logs with Marketplace usage reports on a delay schedule AWS documents per region (USE1-SMMP:inference.count.m.i.c style usage types).
• Test normal completion, client disconnect, metadata socket failure, and malformed JSON before public listing.
• Run Limited visibility with an allowlisted buyer account before broad release.

FinOps teams should model two curves: provisioned endpoint hours (capacity) and ConsumedUnits (software). A long idle WebSocket might burn infra without software units if your counting rules exclude silence — clarify that in private offer documents.

How this fits the bidirectional inference stack

explainx.ai has tracked SageMaker streaming in several layers. vLLM push-based KV transfer covers model-server physics on P5en; this metering post covers Marketplace economics on top. Disaggregated inference papers optimize tokens per second; metering JSON optimizes dollars per counted second — teams need both spreadsheets.

Voice-agent builders comparing Fish Audio duplex APIs against Marketplace-hosted Deepgram should compare all-in cost: endpoint instance family, autoscaling policy, and ConsumedUnits definition (does silence count?).

Marketplace listing checklist (condensed)

AWS's post walks sellers through seller registration, model package publishing, Partner Central machine-learning product creation, and choosing per-inference pricing (not interchangeable with hourly listings post-release). Private offers can still negotiate unit rates for enterprise buyers — the metering transport does not change; only the price card does.

Sellers should delete test endpoints after validation. AWS warns idle endpoints continue infra billing even when no ConsumedUnits flow.

Buyer FAQ

Will my bill show two line items? Expect SageMaker hosting charges plus Marketplace software usage keyed on inference.count.

Can I cap spend? AWS billing alarms still apply to both components; software caps are offer-specific unless you negotiate contract spend limits.

Does the client SDK change? Deepgram's launch path claims no client metering configuration — only server-side frames change.

Error-handling matrix (provider view)

AWS publishes explicit behavior when metadata misbehaves:

Providers choosing Required=true trade revenue certainty against user-visible dropouts — common in regulated billing, rare in consumer voice UX.

Historical thread on AWS streaming posts

This metering release is the third beat in AWS's Deepgram partnership story documented on the AWS ML blog: bidirectional streaming APIs, IAM temporary delegation and metrics, and now usage-based Marketplace billing. Each beat removes friction for enterprises that want one AWS invoice and one security review instead of a separate vendor procurement path.

If you previously self-hosted Deepgram containers without Marketplace, compare private offer per-unit rates against direct contracts — Marketplace wins when buyers need consolidated spend visibility and negotiated enterprise pricing on top of the public unit card.

Voice-agent architecture note

Metering sits below the application orchestration layer. Pipecat or custom WebSocket clients still manage turn-taking, barge-in, and tool calls; the metadata socket is invisible to them. Plan load tests accordingly: synthetic audio generators should mimic real duty cycles (speech + silence) so you validate both ConsumedUnits logic and infra autoscaling.

For procurement reviewers, attach this capability to existing AWS Enterprise Discount Program conversations — software usage lines may grow faster than infra once audio traffic scales, even if GPUs stay right-sized. Document counting rules in the private offer appendix so finance does not dispute silence handling after go-live.

Validation before you ship a listing

AWS's pre-release checklist is unusually concrete: interrupted sessions, metadata loss, malformed JSON, and reconciliation delay on usage type tokens like USE1-SMMP:inference.count.m.i.c. Treat malformed-message drops as unbilled risk even when Required=true.

Related reading

• Deepgram Acquires Daily, Pipecat Stays Open
• vLLM Bidirectional KV Transfer on AWS P5en
• Fish Audio Live 3 Duplex Voice Agents
• Sam Altman Metered AI Utility Vision
• AWS Quick Skills Agents Catalog
• Confucius4 Stable-Prefix Streaming ASR