The left context of the buffered chunked decode is no longer a caller
knob: it derives from the codec's own sliding window (2x144 frames),
placing the default decode at the residual floor of the split.
codec_chunk_sec moves from qt_tts_params to qt_init_params, resolved
once to frames at load. The mid-struct removal bumps the ABI to a
closed range [QT_ABI_MIN_VERSION, QT_ABI_VERSION] = [4, 4]; the probe
asserts both bounds reject through the range check.
The 15 predictor flavors build and allocate once at load, positions,
kv rows, and mask baked as never freed graph outputs, and replay
directly on the backend. The prefill slices the last position before
lm_head so every flavor reads one logits row at offset zero. Replaces
the per step graph rebuild, sched allocation, and debug prints.
The speech body accepts seed, max_new_tokens, temperature, top_k,
top_p, and repetition_penalty. Unset fields keep the engine defaults,
a temperature of zero selects greedy decoding, and the subtalker
mirrors the talker knobs. A fixed seed makes a request reproducible.
seed_reference hashes the ICL reference codes and restores the conv
contexts, KV ring, and position from a per reference snapshot slot on
a repeat, saving the primed state device to device after a fresh
prime. The reference priming cost amortizes across repeated cloned
voice requests.
POST /v1/voices registers a voice from a WAV extracted server side
through qt_extract_voice_ref or from pre extracted .spk and .rvq
latents taken verbatim, DELETE drops it and GET lists it alongside the
model speakers. A registered voice wins over a speaker of the same
name and injects the reference latents into qt_tts_params, ref_text
present selects ICL clone mode. The registry lives in process RAM
under the synthesis mutex, so registration and lookups never race a
running synthesis. The audio and rvq readers gain buffer variants
factored from the file paths. The README and the architecture
document catch up on the streaming decode, the hidden bridge, and the
server endpoints.
With -o '-', stdin is read line by line and every line synthesises
immediately as its own utterance, model and speaker staying resident
across lines. Each utterance after the first opens with a fresh RIFF
header, armed at end of line and consumed lazily at the next audio,
so a client can split the stream into standalone WAV clips on the
RIFF magic. Port of the feature contributed to omnivoice.cpp in
ServeurpersoCom/omnivoice.cpp#11.
Co-authored-by: Jeffrey van Binsbergen <comgenie@comgenie.com>
Pascal
karlandPascal
