Prevenzione allucinazioni/loop infiniti (Qwen3-TTS autoregressivo):
- block_repeated_ngrams: maschera i token che ricreerebbero un n-gram gia visto (n=4)
- has_repeating_cycle: ferma la generazione su cicli periodici (periodo 1-16, 4 ripetizioni)
- stuck detector: token dominante nella finestra recente (4 occorrenze in 8 token)
- fallback EOS quando tutti i logits sono mascherati (evita NaN)
- KV cache talker configurabile (--kv-cache, default 8192): 4096 overflowava con
reference lunghe + testi lunghi ("decode would overflow cache")
- parametri esposti via API (no_repeat_ngram_size, loop_max_period, loop_repeats,
loop_window) e CLI (--no-repeat-ngram, --loop-period, --loop-repeats, --loop-window)
- Docker: TTS_KV_CACHE env (default 8192)
Validato: testo 2788 char che prima degenerava in loop ora sintetizza pulito
(141s via API, nessuna ripetizione).
The fused mode appended the codec stream tail to the predictor frame
graph, so one compute produced both a frame's codes and its 80 ms of
audio with no host round trip. The experiment applied to max_batch 1
with a streaming synthesis only, it cost throughput against the
buffered flush that stays the default, and it kept a second frame
graph, its ring inputs and an init flag alive for that single case. It
is not worth keeping.
Remove the tail helpers, the fused graph of CodePredGraphSet, the
codec_fused field of qt_init_params, the --codec-fused flag of both
tools and the harness switch that exercised it. The predictor frame
unroll and the in graph sampler are untouched.
One static frame graph per batch width replaces the per step chain:
prefill and the 15 acoustic steps run in a single backend compute.
This is the target architecture for the llama.cpp Qwen3-TTS port and
serves as its working GGML reference while under test.
Sampling is a plain op chain batched over slots: temperature, argsort
top_k (descending order is guaranteed on every backend, unlike top_k),
softmax, cumsum, cdf crossing against a per step philox uniform.
Greedy draws with u = 0 and lands on the argmax. Faster than the
fused sampling op under CUDA graph capture, greedy codes stay exact
against the Python reference on CPU, CUDA and Vulkan.
Opt in single slot latency mode (--codec-fused on qwen-tts and
tts-server, codec_fused in qt_init_params): the codec stream tail
joins the frame graph at T=1, codes read through a device view, one
80 ms chunk per compute with no host round trip.
Predictor 3.34 -> 3.11 ms/frame on CUDA, end to end -4%.
The left context of the buffered chunked decode is no longer a caller
knob: it derives from the codec's own sliding window (2x144 frames),
placing the default decode at the residual floor of the split.
codec_chunk_sec moves from qt_tts_params to qt_init_params, resolved
once to frames at load. The mid-struct removal bumps the ABI to a
closed range [QT_ABI_MIN_VERSION, QT_ABI_VERSION] = [4, 4]; the probe
asserts both bounds reject through the range check.
qt_extract_voice_ref now pays the speaker encoder weight load on its
first call, mirroring the qt_synthesize ref_audio path. The server
extraction endpoint works without a prior ref wav synthesis.
qwen-codec --talker extracts the speaker embedding (.spk, raw f32)
and the ICL codes (.rvq) in one pass, encode truncated to the hop
boundary conforming to the --ref-wav path. qwen-tts loads them via
--ref-spk / --ref-rvq and skips the speaker encoder and codec encode
on every synthesis: TTFA 205 ms -> 89 ms. Extends qt_tts_params with
ABI v2 latent fields, adds qt_num_codebooks(), ships freeman.spk +
freeman.rvq and switches clone scripts to the latent path. Output is
bit-identical to the raw path at fixed seed.
enne2
Pascal
Andres Marafioti