PersonaPlex 7B:
A Full-Duplex Conversation
Below 10 Cents an Hour

TL;DR

Our PersonaPlex-7B implementation runs 80 concurrent sessions on a single GPU - at full utilization, this translates to below 10 cents per hour for each user. We believe the low cost of natural conversation between human and AI will open new possibilities for interfaces and consumer applications.

What Real-time Means for a Full-duplex Agent

Unlike TTS, where faster-than-real-time generation is beneficial, we cannot generate far ahead in duplex model serving. Audio is received as it happens, and we must react to interruptions even when the agent is talking.

In the case of PersonaPlex, the model is trained to process a single 80 ms frame at a time.

Fun fact: the paper’s original experimental setup used our Dia TTS model to synthesize its two-speaker customer-service conversation data.

Thus, the serving goal is not maximum tokens per second. Rather, we need to achieve the largest number of simultaneous conversations while keeping the forward pass under 80 ms.

The OSS Baseline

As of early September 2026, vLLM-Omni^△ was the only open-source serving framework with multi-session PersonaPlex support. We measured both its two-stage Unified path and single-process Standalone path on 1x H100 SXM.

How We Reached 80 Sessions

Reaching 80 concurrent sessions on a single GPU required a variety of techniques. Here’s a quick overview of how we did it.

Predictable GPU execution

Omni-models such as PersonaPlex are made up of several components that run every frame. In eager mode, these small GPU operations add launch overhead, making large batch sizes impossible to serve reliably.

We first reduced the Depformer’s autoregressive loop from 16 steps to the 8 needed to generate agent audio. We then captured the frame in a CUDA Graph. Replaying the graph avoids setup work and keeps latency consistent as the batch size increases.

Compressing the Temporal KV cache

After the compute path fit within 80 ms, memory became the next limit, mostly due to the Temporal Transformer KV cache.

So we keep the model weights and activations in BF16, but store the Temporal KV cache in FP8. This cuts the cache size in half while having negligible impact on audio quality and turn-taking.

A server built around the conversation clock

We make the server advance every session on the same 80 ms clock. This means all sessions run in fixed-shape batches, and new sessions can easily fill slots without changing GPU execution.

We also make sure Opus encoding, decoding, and WebSocket I/O run outside this loop. This way, a slow client cannot block another call from hitting its 80 ms tick.

Below is a short conversation example between a human and PersonaPlex 7B.

We acknowledge that there is a long way to go before OSS models reach the intelligence and naturalness of models such as GPT-Live. But PersonaPlex is a step in the right direction, and more models such as Nemotron-VoiceChat are being released. We are bullish about the future of open duplex models and will continue to work on building ultrafast inference around it.

We built this system for PersonaPlex, but the same problems appear in other omni-models as well. If you are deploying one and need help, talk to our engineers.