enne2/qwentts.cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

api: derived codec left context, chunk width hoisted to qt_init

Browse Source 
The left context of the buffered chunked decode is no longer a caller
knob: it derives from the codec's own sliding window (2x144 frames),
placing the default decode at the residual floor of the split.
codec_chunk_sec moves from qt_tts_params to qt_init_params, resolved
once to frames at load. The mid-struct removal bumps the ABI to a
closed range [QT_ABI_MIN_VERSION, QT_ABI_VERSION] = [4, 4]; the probe
asserts both bounds reject through the range check.
This commit is contained in:
Pascal
2026-07-25 18:56:28 +02:00
parent 710a52af75
commit d03ffb97f9
8 changed files with 240 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+24 -14
View File
@@ -322,12 +322,22 @@ share one layout.
A standalone codec decode of an isolated window shows edge artefacts at A standalone codec decode of an isolated window shows edge artefacts at
the chunk boundary, because the causal conv kernels and the sliding the chunk boundary, because the causal conv kernels and the sliding
window attention have no left context. `codec_chunked_decode` prepends window attention have no left context. `codec_chunked_decode` prepends
`codec_left_context_sec` worth of previously decoded frames, decodes, `left_ctx_frames` worth of previously decoded frames, decodes, then
then strips the samples that belong to the left context. Defaults match strips the samples that belong to the left context. The chunk width
the upstream tokenizer : `codec_chunk_sec` 24.0 (300 frames at 12.5 Hz) comes from `qt_init_params.codec_chunk_sec` (default 24.0, 300 frames at
and `codec_left_context_sec` 2.0 (25 frames). The first chunk collapses 12.5 Hz) and resolves to a frame count once at `qt_init`. The left
its left context to whatever is available. This routine serves the context is not a caller knob : it derives from the decoder's own sliding
buffered one-shot decode only. window, 2 x sliding_window (144 frames on this codec), which is where
the chunk output reaches the residual floor of the split. A shorter context leaves the decode
audibly off, a longer one redecodes frames for nothing. The first chunk
collapses its left context to whatever is available. This routine serves
the buffered one-shot decode only.
A chunk covering the whole utterance decodes in a single pass and is
bit exact against `pipeline_codec_decode`. Any split leaves a residual
around -50 dB that no amount of left context removes, growing slowly
with the pass count, so the chunk is a memory knob and not a quality
one.
### Streaming decode (stateful path) ### Streaming decode (stateful path)
@@ -490,14 +500,15 @@ QT_STATUS_CANCELLED -5
decode step, ~83 ms granularity) and `on_chunk`. With `on_chunk` set, decode step, ~83 ms granularity) and `on_chunk`. With `on_chunk` set,
synthesis runs in streaming mode : every generated frame emits its synthesis runs in streaming mode : every generated frame emits its
1920 samples immediately through the stateful codec and `out` stays 1920 samples immediately through the stateful codec and `out` stays
empty on success. `codec_chunk_sec` / `codec_left_context_sec` drive empty on success. `qt_init_params.codec_chunk_sec` drives the chunk framing of the
the chunk framing of the buffered path only; the streaming path buffered path only; the streaming path ignores it.
ignores both.
`QT_ABI_VERSION` guards struct growth : callers set `abi_version` (or `QT_ABI_VERSION` and `QT_ABI_MIN_VERSION` bound the struct layouts this
let the default-params helpers do it) and the lib rejects a struct laid build addresses : callers set `abi_version` (or let the default-params
out for a newer header. `qt_version()` returns the git short hash and helpers do it) and the lib rejects anything outside that closed range,
commit date. a struct laid out for a newer header as well as one whose fields sit at
offsets this build no longer maps. `qt_version()` returns the git short
hash and commit date.
### Low-level API : src/pipeline-tts.h, src/pipeline-codec.h ### Low-level API : src/pipeline-tts.h, src/pipeline-codec.h
@@ -554,7 +565,6 @@ Optional:
--ref-text <path> Transcript file for the reference (enables ICL clone mode) --ref-text <path> Transcript file for the reference (enables ICL clone mode)
--max-new <n> Max new audio frames (default: 2048) --max-new <n> Max new audio frames (default: 2048)
--codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0) --codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0)
--codec-left-dur <f> Codec decode left context duration in seconds (default: 2.0)
--stream-by-line Flush synthesis at each newline, one WAV header per line (-o '-') --stream-by-line Flush synthesis at each newline, one WAV header per line (-o '-')
Sampling: Sampling:
src/pipeline-tts.cpp
+22 -15
View File
@@ -139,7 +139,8 @@ bool pipeline_tts_load(PipelineTTS * pt,
BackendPair bp, BackendPair bp,
bool use_fa, bool use_fa,
bool clamp_fp16, bool clamp_fp16,
int max_batch) { int max_batch,
float codec_chunk_sec) {
pt->bp = bp; pt->bp = bp;
pt->backend = bp.backend; pt->backend = bp.backend;
pt->sched = NULL; pt->sched = NULL;
@@ -149,6 +150,10 @@ bool pipeline_tts_load(PipelineTTS * pt,
pt->hidden_bridge = NULL; pt->hidden_bridge = NULL;
pt->max_batch = max_batch > 1 ? max_batch : 1; pt->max_batch = max_batch > 1 ? max_batch : 1;
// Chunk width of the buffered decode. The conversion is a fixed
// 12.5 Hz ratio, so it lands here once instead of per synthesis.
pt->codec_chunk_frames = pipeline_tts_duration_sec_to_tokens(pt, codec_chunk_sec);
// Fused flash attention needs a GPU kernel; CPU only backends fall // Fused flash attention needs a GPU kernel; CPU only backends fall
// back to the F32 manual chain automatically. clamp_fp16 is forwarded // back to the F32 manual chain automatically. clamp_fp16 is forwarded
// verbatim: a no op on backends that already accumulate in F32, an // verbatim: a no op on backends that already accumulate in F32, an
@@ -209,6 +214,13 @@ bool pipeline_tts_load(PipelineTTS * pt,
// allocates the [t, c, S] state tensors from it. // allocates the [t, c, S] state tensors from it.
pt->codec.stream_sets = pt->max_batch + 1; pt->codec.stream_sets = pt->max_batch + 1;
// Left context of the buffered chunked decode. Two decoder windows
// warm the transformer attention and the causal conv stack deep
// enough for the chunk output to sit at the residual floor of the
// split; a shorter context leaves the decode audibly off, a longer
// one buys nothing and redecodes frames for nothing.
pt->codec_left_ctx_frames = 2 * pt->codec.transformer.sliding_window;
// Scheduler shared by talker_forward_* and code_predictor_step. // Scheduler shared by talker_forward_* and code_predictor_step.
// Routes ops the GPU backend cannot run (typical case: K-quant // Routes ops the GPU backend cannot run (typical case: K-quant
// get_rows on CUDA) to the CPU backend. 4096 nodes covers the 28L // get_rows on CUDA) to the CPU backend. 4096 nodes covers the 28L
@@ -729,13 +741,10 @@ bool tts_engine_admit(TtsEngine * e, TtsJob * job) {
const std::string speaker = params->speaker ? params->speaker : ""; const std::string speaker = params->speaker ? params->speaker : "";
const std::string ref_text = params->ref_text ? params->ref_text : ""; const std::string ref_text = params->ref_text ? params->ref_text : "";
// ABI v2 latent reference fields. Callers compiled against ABI 1 const float * lat_spk_emb = params->ref_spk_emb;
// never set them; the abi_version gate keeps their uninitialised const int lat_spk_dim = params->ref_spk_dim;
// tail bytes out of the read path. const int32_t * lat_codes = params->ref_codes;
const float * lat_spk_emb = (params->abi_version >= 2) ? params->ref_spk_emb : NULL; const int lat_T = params->ref_T;
const int lat_spk_dim = (params->abi_version >= 2) ? params->ref_spk_dim : 0;
const int32_t * lat_codes = (params->abi_version >= 2) ? params->ref_codes : NULL;
const int lat_T = (params->abi_version >= 2) ? params->ref_T : 0;
const bool has_ref_audio = (params->ref_audio_24k != NULL) && (params->ref_n_samples > 0); const bool has_ref_audio = (params->ref_audio_24k != NULL) && (params->ref_n_samples > 0);
const bool has_lat_spk = (lat_spk_emb != NULL) && (lat_spk_dim > 0); const bool has_lat_spk = (lat_spk_emb != NULL) && (lat_spk_dim > 0);
@@ -1005,13 +1014,11 @@ static void tts_slot_complete(TtsEngine * e, TtsSlot & s) {
// reference codes prepends the buffer so the onset is voiced with // reference codes prepends the buffer so the onset is voiced with
// the reference's causal state, mirroring the upstream pipeline // the reference's causal state, mirroring the upstream pipeline
// which decodes reference plus generated then trims; the seeded // which decodes reference plus generated then trims; the seeded
// samples strip from the front afterwards. Raising // samples strip from the front afterwards. The seed caps at the
// codec_left_context_sec past the reference duration reproduces the // derived left context, so a reference longer than that window
// upstream full reference decode exactly. // contributes only its tail.
const float chunk_sec = params->codec_chunk_sec > 0.0f ? params->codec_chunk_sec : 24.0f; const int chunk_frames = pt->codec_chunk_frames;
const float left_ctx_sec = params->codec_left_context_sec >= 0.0f ? params->codec_left_context_sec : 2.0f; const int left_ctx_frames = pt->codec_left_ctx_frames;
const int chunk_frames = pipeline_tts_duration_sec_to_tokens(pt, chunk_sec);
const int left_ctx_frames = pipeline_tts_duration_sec_to_tokens(pt, left_ctx_sec);
const int T_frames = (int) s.all_codes.size(); const int T_frames = (int) s.all_codes.size();
int seed = 0; int seed = 0;
src/pipeline-tts.h
+13 -3
View File
@@ -120,6 +120,13 @@ struct PipelineTTS {
// single sequence layout and behavior. // single sequence layout and behavior.
int max_batch; int max_batch;
// Codec decode framing of the buffered path, in 12.5 Hz frames.
// chunk sizes the decode window and comes from the caller; left_ctx
// is the warmup the decoder needs and derives from its own sliding
// window at load.
int codec_chunk_frames;
int codec_left_ctx_frames;
CodecSpecials codec_specials; CodecSpecials codec_specials;
TextSpecials text_specials; TextSpecials text_specials;
std::vector<LanguageEntry> languages; std::vector<LanguageEntry> languages;
@@ -172,15 +179,18 @@ struct PipelineTTS {
// invalid metadata. use_fa is gated on bp.has_gpu inside the load: // invalid metadata. use_fa is gated on bp.has_gpu inside the load:
// CPU only runs always use the manual F32 attention chain. clamp_fp16 // CPU only runs always use the manual F32 attention chain. clamp_fp16
// is forwarded as is. max_batch sizes the KV sets, the bridge columns // is forwarded as is. max_batch sizes the KV sets, the bridge columns
// and the maximum concurrent slots (minimum 1). Caller frees with // and the maximum concurrent slots (minimum 1). codec_chunk_sec
// pipeline_tts_free. // converts to the chunk width every buffered decode on this handle
// runs with; the left context that pairs with it derives from the
// codec's own sliding window. Caller frees with pipeline_tts_free.
bool pipeline_tts_load(PipelineTTS * pt, bool pipeline_tts_load(PipelineTTS * pt,
const char * talker_gguf_path, const char * talker_gguf_path,
const char * codec_gguf_path, const char * codec_gguf_path,
BackendPair bp, BackendPair bp,
bool use_fa, bool use_fa,
bool clamp_fp16, bool clamp_fp16,
int max_batch); int max_batch,
float codec_chunk_sec);
void pipeline_tts_free(PipelineTTS * pt); void pipeline_tts_free(PipelineTTS * pt);
src/qwen.cpp
+50 -45
View File
@@ -212,6 +212,10 @@ void qt_log_set(qt_log_cb cb, void * user_data) {
g_log_cb.store(cb, std::memory_order_release); g_log_cb.store(cb, std::memory_order_release);
} }
// Codec chunk default, shared by qt_init_default_params and the
// qt_init resolution of an unset value.
static const float QT_CODEC_CHUNK_SEC_DEFAULT = 24.0f;
void qt_init_default_params(struct qt_init_params * p) { void qt_init_default_params(struct qt_init_params * p) {
p->abi_version = QT_ABI_VERSION; p->abi_version = QT_ABI_VERSION;
p->talker_path = nullptr; p->talker_path = nullptr;
@@ -219,39 +223,39 @@ void qt_init_default_params(struct qt_init_params * p) {
p->use_fa = true; p->use_fa = true;
p->clamp_fp16 = false; p->clamp_fp16 = false;
p->max_batch = 1; p->max_batch = 1;
p->codec_chunk_sec = QT_CODEC_CHUNK_SEC_DEFAULT;
} }
void qt_tts_default_params(struct qt_tts_params * p) { void qt_tts_default_params(struct qt_tts_params * p) {
p->abi_version = QT_ABI_VERSION; p->abi_version = QT_ABI_VERSION;
p->text = nullptr; p->text = nullptr;
p->lang = nullptr; p->lang = nullptr;
p->instruct = nullptr; p->instruct = nullptr;
p->speaker = nullptr; p->speaker = nullptr;
p->ref_audio_24k = nullptr; p->ref_audio_24k = nullptr;
p->ref_n_samples = 0; p->ref_n_samples = 0;
p->ref_text = nullptr; p->ref_text = nullptr;
p->seed = -1; p->seed = -1;
p->max_new_tokens = 2048; p->max_new_tokens = 2048;
p->do_sample = true; p->do_sample = true;
p->temperature = 0.9f; p->temperature = 0.9f;
p->top_k = 50; p->top_k = 50;
p->top_p = 1.0f; p->top_p = 1.0f;
p->repetition_penalty = 1.05f; p->repetition_penalty = 1.05f;
p->subtalker_do_sample = true; p->subtalker_do_sample = true;
p->subtalker_temperature = 0.9f; p->subtalker_temperature = 0.9f;
p->subtalker_top_k = 50; p->subtalker_top_k = 50;
p->subtalker_top_p = 1.0f; p->subtalker_top_p = 1.0f;
p->dump_dir = nullptr; p->dump_dir = nullptr;
p->cancel = nullptr; p->cancel = nullptr;
p->cancel_user_data = nullptr; p->cancel_user_data = nullptr;
p->on_chunk = nullptr; p->on_chunk = nullptr;
p->on_chunk_user_data = nullptr; p->on_chunk_user_data = nullptr;
p->codec_chunk_sec = 24.0f; p->ref_spk_emb = nullptr;
p->codec_left_context_sec = 2.0f; p->ref_spk_dim = 0;
p->ref_spk_emb = nullptr; p->ref_codes = nullptr;
p->ref_spk_dim = 0; p->ref_T = 0;
p->ref_codes = nullptr;
p->ref_T = 0;
} }
int qt_num_codebooks(const struct qt_context * q) { int qt_num_codebooks(const struct qt_context * q) {
@@ -324,18 +328,21 @@ struct qt_context * qt_init(const struct qt_init_params * params) {
qt_log(QT_LOG_ERROR, "[Qwen] qt_init requires talker_path and codec_path"); qt_log(QT_LOG_ERROR, "[Qwen] qt_init requires talker_path and codec_path");
return nullptr; return nullptr;
} }
if (params->abi_version > QT_ABI_VERSION) { if (params->abi_version > QT_ABI_VERSION || params->abi_version < QT_ABI_MIN_VERSION) {
qt_set_error("qt_init: params->abi_version %d > QT_ABI_VERSION %d (binding compiled against a newer header)", qt_set_error("qt_init: params->abi_version %d outside the supported range [%d, %d]", params->abi_version,
params->abi_version, QT_ABI_VERSION); QT_ABI_MIN_VERSION, QT_ABI_VERSION);
qt_log(QT_LOG_ERROR, "[Qwen] qt_init params struct is from a newer ABI (%d > %d)", params->abi_version, qt_log(QT_LOG_ERROR, "[Qwen] qt_init params struct carries an unsupported ABI (%d, supported [%d, %d])",
QT_ABI_VERSION); params->abi_version, QT_ABI_MIN_VERSION, QT_ABI_VERSION);
return nullptr; return nullptr;
} }
qt_log(QT_LOG_INFO, "[Qwen] qwentts.cpp %s", qt_version()); qt_log(QT_LOG_INFO, "[Qwen] qwentts.cpp %s", qt_version());
// ABI v3 tail field: zero init from older callers means 1. const int max_batch = params->max_batch > 1 ? params->max_batch : 1;
const int max_batch = (params->abi_version >= 3 && params->max_batch > 1) ? params->max_batch : 1;
// The chunk width resolves once here: it is a property of the
// handle, read by every buffered decode it runs.
const float chunk_sec = params->codec_chunk_sec > 0.0f ? params->codec_chunk_sec : QT_CODEC_CHUNK_SEC_DEFAULT;
// new qt_context() value-initialises every field: POD aggregates // new qt_context() value-initialises every field: POD aggregates
// (BackendPair, PipelineTTS) are zero-init, std containers in // (BackendPair, PipelineTTS) are zero-init, std containers in
@@ -355,7 +362,7 @@ struct qt_context * qt_init(const struct qt_init_params * params) {
} }
if (!pipeline_tts_load(&q->pt, params->talker_path, params->codec_path, q->bp, params->use_fa, if (!pipeline_tts_load(&q->pt, params->talker_path, params->codec_path, q->bp, params->use_fa,
params->clamp_fp16, max_batch)) { params->clamp_fp16, max_batch, chunk_sec)) {
qt_throw("qt_init: pipeline_tts_load failed for '%s' / '%s'", params->talker_path, params->codec_path); qt_throw("qt_init: pipeline_tts_load failed for '%s' / '%s'", params->talker_path, params->codec_path);
} }
@@ -553,10 +560,9 @@ enum qt_status qt_synthesize(struct qt_context * q, const struct qt_tts_params *
qt_set_error("qt_synthesize: out is NULL in buffered mode"); qt_set_error("qt_synthesize: out is NULL in buffered mode");
return QT_STATUS_INVALID_PARAMS; return QT_STATUS_INVALID_PARAMS;
} }
if (params->abi_version > QT_ABI_VERSION) { if (params->abi_version > QT_ABI_VERSION || params->abi_version < QT_ABI_MIN_VERSION) {
qt_set_error( qt_set_error("qt_synthesize: params->abi_version %d outside the supported range [%d, %d]", params->abi_version,
"qt_synthesize: params->abi_version %d > QT_ABI_VERSION %d (binding compiled against a newer header)", QT_ABI_MIN_VERSION, QT_ABI_VERSION);
params->abi_version, QT_ABI_VERSION);
if (out) { if (out) {
qt_audio_free(out); qt_audio_free(out);
} }
@@ -606,9 +612,8 @@ enum qt_status qt_synthesize(struct qt_context * q, const struct qt_tts_params *
} }
return QT_STATUS_MODE_INVALID; return QT_STATUS_MODE_INVALID;
} }
// ABI v2 latent reference fields, same gate as the pipeline. const bool has_lat_spk = params->ref_spk_emb && params->ref_spk_dim > 0;
const bool has_lat_spk = params->abi_version >= 2 && params->ref_spk_emb && params->ref_spk_dim > 0; const bool has_lat_codes = params->ref_codes && params->ref_T > 0;
const bool has_lat_codes = params->abi_version >= 2 && params->ref_codes && params->ref_T > 0;
if ((params->ref_audio_24k || has_lat_spk) && mt != "base") { if ((params->ref_audio_24k || has_lat_spk) && mt != "base") {
qt_set_error("--ref-wav / --ref-spk is only valid for base models (loaded: %s)", mt.c_str()); qt_set_error("--ref-wav / --ref-spk is only valid for base models (loaded: %s)", mt.c_str());
src/qwen.h
+41 -31
View File
@@ -44,20 +44,28 @@ extern "C" {
# define QT_API # define QT_API
#endif #endif
// Struct ABI version. Incremented every time a public POD struct grows a // Struct ABI version. Incremented every time a public POD struct
// new field at the end. Callers fill `.abi_version = QT_ABI_VERSION` // changes layout. Callers fill `.abi_version = QT_ABI_VERSION` (or let
// (or let qwen_*_default_params set it). Entries that consume those // qwen_*_default_params set it). Entries that consume those structs
// structs reject inputs whose abi_version exceeds the build-time // accept the closed range [QT_ABI_MIN_VERSION, QT_ABI_VERSION] and
// constant: this guards a binary built against vN from receiving a // reject anything outside it with a diagnostic rather than reading
// struct laid out for vN+1 by a freshly compiled binding. Adding fields // fields at offsets the caller never wrote: above the ceiling the
// stays backward compat because the new tail is zero init in older // struct comes from a newer header, below the floor it carries a
// callers and the lib reads only what its abi_version permits. // layout this build no longer addresses. Fields appended at the tail
// keep older callers valid down to the floor, since their unwritten
// tail is zero init and the lib gates on abi_version before reading it.
// //
// There is no separate semver triple. The runtime build identity is the // There is no separate semver triple. The runtime build identity is the
// git short hash + commit date string returned by qt_version(); for // git short hash + commit date string returned by qt_version(); for
// binding compat checks, QT_ABI_VERSION is the only number that // binding compat checks, QT_ABI_VERSION is the only number that
// matters. // matters.
#define QT_ABI_VERSION 3 #define QT_ABI_VERSION 4
// Oldest struct layout this build addresses. A v3 or older
// qt_tts_params places its trailing fields at offsets this build does
// not map, so such a struct is unreadable here and its caller rebuilds
// against this header.
#define QT_ABI_MIN_VERSION 4
// Returns a static string of the form "<git-hash> (<date>)" identifying // Returns a static string of the form "<git-hash> (<date>)" identifying
// the exact commit this binary was built from. Safe to call from any // the exact commit this binary was built from. Safe to call from any
@@ -134,10 +142,27 @@ struct qt_init_params {
// back into the qwen_* API. qt_synthesize itself stays blocking // back into the qwen_* API. qt_synthesize itself stays blocking
// and thread safe in both modes. // and thread safe in both modes.
int max_batch; int max_batch;
// ABI v4. Chunk width of the buffered codec decode, in seconds of
// audio, resolved to an integer frame count at the codec frame rate
// by qt_init and applied to every synthesis on the handle. A chunk
// shorter than the utterance bounds the peak decode memory; the
// decode window is that chunk plus the left context the decoder
// needs to start warm, which qt_init derives from the codec's own
// sliding window rather than taking from the caller. Peak memory
// therefore floors at that warmup, and driving the chunk below it
// buys no memory while costing one redecode of the context per
// chunk. A chunk covering the whole utterance decodes in a single
// pass and is the exact reference; any split leaves a residual on
// the order of -50 dB. 0 selects the upstream default, 24.0 (300
// frames at 12.5 Hz). The streaming path frames its own chunks
// through the persistent codec stream state and reads none of this.
float codec_chunk_sec;
}; };
// Initialise to the standard defaults: both paths NULL (caller must set // Initialise to the standard defaults: both paths NULL (caller must set
// them before calling qt_init), use_fa true, clamp_fp16 false. // them before calling qt_init), use_fa true, clamp_fp16 false,
// max_batch 1, codec_chunk_sec 24.0.
QT_API void qt_init_default_params(struct qt_init_params * p); QT_API void qt_init_default_params(struct qt_init_params * p);
// Allocate every module described by params. Returns NULL on any // Allocate every module described by params. Returns NULL on any
@@ -203,10 +228,11 @@ typedef bool (*qt_cancel_cb)(void * user_data);
// 24 kHz; valid only for the duration of the call. // 24 kHz; valid only for the duration of the call.
// user_data is forwarded verbatim from on_chunk_user_data. // user_data is forwarded verbatim from on_chunk_user_data.
// //
// The chunk granularity is driven by chunk_duration_sec in qt_tts_params: // The chunk granularity is a ramp over the persistent codec stream
// once the AR loop has produced enough frames to cover that duration, // state: the first flush covers a single 12.5 Hz frame for the lowest
// the codec decodes that bundle and emits it. The last chunk on EOS / // time to first audio, then the target width doubles up to 8 frames as
// max_new flushes whatever frames remain. // the stream settles. The last chunk on EOS / max_new flushes whatever
// frames remain.
typedef bool (*qt_audio_chunk_cb)(const float * samples, int n_samples, void * user_data); typedef bool (*qt_audio_chunk_cb)(const float * samples, int n_samples, void * user_data);
// Log severity. Numerically ordered so a callback can filter with a // Log severity. Numerically ordered so a callback can filter with a
@@ -311,21 +337,6 @@ struct qt_tts_params {
qt_audio_chunk_cb on_chunk; qt_audio_chunk_cb on_chunk;
void * on_chunk_user_data; void * on_chunk_user_data;
// Codec decode framing. Applied to both the streaming path (chunk
// by chunk emission) and the buffered path (one shot decode at the
// end) : the chunked decode rolls a left context window across the
// codec frames to avoid edge artefacts at chunk boundaries. The
// first chunk has its left context collapsed to whatever is
// available, matching the upstream Qwen3-TTS 12 Hz tokenizer
// chunked_decode rule. Defaults match the upstream reference :
// codec_chunk_sec 24.0 (300 frames at 12.5 Hz) and
// codec_left_context_sec 2.0 (25 frames at 12.5 Hz). Values are
// converted internally to integer frame counts via the codec frame
// rate ; codec_chunk_sec clamps to >= 1 frame, codec_left_context_sec
// clamps to >= 0 frames.
float codec_chunk_sec;
float codec_left_context_sec;
// ABI v2. Pre-encoded voice reference, the latent counterpart of // ABI v2. Pre-encoded voice reference, the latent counterpart of
// ref_audio_24k. ref_spk_emb is the speaker embedding produced by // ref_audio_24k. ref_spk_emb is the speaker embedding produced by
// the speaker encoder (ref_spk_dim f32 values, must equal the // the speaker encoder (ref_spk_dim f32 values, must equal the
@@ -343,8 +354,7 @@ struct qt_tts_params {
// Initialise to the standard defaults. Strings NULL, seed -1, // Initialise to the standard defaults. Strings NULL, seed -1,
// max_new_tokens 2048, do_sample true, temperature 0.9, top_k 50, // max_new_tokens 2048, do_sample true, temperature 0.9, top_k 50,
// top_p 1.0, repetition_penalty 1.05, subtalker mirrors talker, // top_p 1.0, repetition_penalty 1.05, subtalker mirrors talker,
// dump_dir NULL, cancel NULL, on_chunk NULL, codec_chunk_sec 24.0, // dump_dir NULL, cancel NULL, on_chunk NULL.
// codec_left_context_sec 2.0.
QT_API void qt_tts_default_params(struct qt_tts_params * p); QT_API void qt_tts_default_params(struct qt_tts_params * p);
// Number of RVQ codebooks (K) of the loaded codec. Pre-encoded ICL // Number of RVQ codebooks (K) of the loaded codec. Pre-encoded ICL
tests/abi-c.c
+29 -3
View File
@@ -68,12 +68,15 @@ int main(void) {
qt_tts_default_params(&params); qt_tts_default_params(&params);
/* Sanity-check a few default values, including the abi_version and /* Sanity-check a few default values, including the abi_version and
* the new use_fa / clamp_fp16 / on_chunk / codec_chunk_sec / * the use_fa / clamp_fp16 / on_chunk / codec framing slots. */
* codec_left_context_sec slots. */ if (params.max_new_tokens != 2048) {
if (params.max_new_tokens != 2048 || params.codec_chunk_sec <= 0.0f || params.codec_left_context_sec < 0.0f) {
fprintf(stderr, "[Probe] default values do not match\n"); fprintf(stderr, "[Probe] default values do not match\n");
return 1; return 1;
} }
if (iparams.codec_chunk_sec <= 0.0f || iparams.max_batch != 1) {
fprintf(stderr, "[Probe] init_params chunk / max_batch defaults do not match\n");
return 1;
}
if (iparams.abi_version != QT_ABI_VERSION || params.abi_version != QT_ABI_VERSION) { if (iparams.abi_version != QT_ABI_VERSION || params.abi_version != QT_ABI_VERSION) {
fprintf(stderr, "[Probe] abi_version not set by qt_*_default_params\n"); fprintf(stderr, "[Probe] abi_version not set by qt_*_default_params\n");
return 1; return 1;
@@ -148,6 +151,29 @@ int main(void) {
qt_free(rejected); qt_free(rejected);
return 8; return 8;
} }
/* The paths point at a file that does not exist, so a NULL return
* alone does not prove the ABI gate fired : the error string must
* name the range check, not a failed GGUF open. */
if (strstr(qt_last_error(), "supported range") == NULL) {
fprintf(stderr, "[Probe] future abi_version rejection did not come from the range check: '%s'\n",
qt_last_error());
return 8;
}
/* The floor is the other half of the range check : a struct laid out
* by a pre-QT_ABI_MIN_VERSION header must be rejected just as hard. */
future_iparams.abi_version = QT_ABI_MIN_VERSION - 1;
rejected = qt_init(&future_iparams);
if (rejected != NULL) {
fprintf(stderr, "[Probe] qt_init accepted an abi_version below the floor\n");
qt_free(rejected);
return 8;
}
if (strstr(qt_last_error(), "supported range") == NULL) {
fprintf(stderr, "[Probe] floor abi_version rejection did not come from the range check: '%s'\n",
qt_last_error());
return 8;
}
enum qt_status rc = qt_synthesize(NULL, &params, &audio); enum qt_status rc = qt_synthesize(NULL, &params, &audio);
if (rc != QT_STATUS_INVALID_PARAMS) { if (rc != QT_STATUS_INVALID_PARAMS) {
tools/qwen-tts.cpp
+48 -52
View File
@@ -54,7 +54,6 @@ static void print_usage(const char * prog) {
" --ref-text <path> Transcript file for the reference (enables ICL clone mode)\n" " --ref-text <path> Transcript file for the reference (enables ICL clone mode)\n"
" --max-new <n> Max new audio frames (default: 2048)\n" " --max-new <n> Max new audio frames (default: 2048)\n"
" --codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0)\n" " --codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0)\n"
" --codec-left-dur <f> Codec decode left context duration in seconds (default: 2.0)\n"
" --stream-by-line Flush synthesis at each newline, one WAV header per line (-o '-')\n\n" " --stream-by-line Flush synthesis at each newline, one WAV header per line (-o '-')\n\n"
"Sampling:\n" "Sampling:\n"
" --seed <int> Sampling seed (default: -1 for random)\n" " --seed <int> Sampling seed (default: -1 for random)\n"
@@ -101,7 +100,6 @@ struct Args {
bool clamp_fp16; bool clamp_fp16;
bool stream_by_line; bool stream_by_line;
float codec_chunk_sec; float codec_chunk_sec;
float codec_left_context_sec;
}; };
// Read all of stdin into a string. Binary mode on Windows so UTF-16 input // Read all of stdin into a string. Binary mode on Windows so UTF-16 input
@@ -180,25 +178,26 @@ static bool read_text_file(const char * path, std::string & out) {
} }
static bool parse_args(int argc, char ** argv, Args & a) { static bool parse_args(int argc, char ** argv, Args & a) {
a = {}; a = {};
a.lang = "auto"; a.lang = "auto";
a.format = "wav16"; a.format = "wav16";
a.max_new_tokens = 2048; a.max_new_tokens = 2048;
a.seed = -1; a.seed = -1;
a.do_sample = true; a.do_sample = true;
a.temperature = 0.9f; a.temperature = 0.9f;
a.top_k = 50; a.top_k = 50;
a.top_p = 1.0f; a.top_p = 1.0f;
a.repetition_penalty = 1.05f; a.repetition_penalty = 1.05f;
a.subtalker_do_sample = true; a.subtalker_do_sample = true;
a.subtalker_top_k = 50; a.subtalker_top_k = 50;
a.subtalker_top_p = 1.0f; a.subtalker_top_p = 1.0f;
a.subtalker_temperature = 0.9f; a.subtalker_temperature = 0.9f;
a.use_fa = true; a.use_fa = true;
a.clamp_fp16 = false; a.clamp_fp16 = false;
a.stream_by_line = false; a.stream_by_line = false;
a.codec_chunk_sec = 24.0f; // Chunk sentinel : qt_init resolves a non positive value to the
a.codec_left_context_sec = 2.0f; // library default.
a.codec_chunk_sec = 0.0f;
for (int i = 1; i < argc; i++) { for (int i = 1; i < argc; i++) {
const char * arg = argv[i]; const char * arg = argv[i];
if (std::strcmp(arg, "-h") == 0 || std::strcmp(arg, "--help") == 0) { if (std::strcmp(arg, "-h") == 0 || std::strcmp(arg, "--help") == 0) {
@@ -260,8 +259,6 @@ static bool parse_args(int argc, char ** argv, Args & a) {
a.stream_by_line = true; a.stream_by_line = true;
} else if (std::strcmp(arg, "--codec-chunk-dur") == 0 && i + 1 < argc) { } else if (std::strcmp(arg, "--codec-chunk-dur") == 0 && i + 1 < argc) {
a.codec_chunk_sec = (float) std::atof(argv[++i]); a.codec_chunk_sec = (float) std::atof(argv[++i]);
} else if (std::strcmp(arg, "--codec-left-dur") == 0 && i + 1 < argc) {
a.codec_left_context_sec = (float) std::atof(argv[++i]);
} else if (std::strcmp(arg, "-o") == 0 && i + 1 < argc) { } else if (std::strcmp(arg, "-o") == 0 && i + 1 < argc) {
a.out_wav = argv[++i]; a.out_wav = argv[++i];
} else { } else {
@@ -279,10 +276,11 @@ static int run(const Args & a) {
// off the two GGUF paths and reports qt_last_error on failure. // off the two GGUF paths and reports qt_last_error on failure.
qt_init_params iparams; qt_init_params iparams;
qt_init_default_params(&iparams); qt_init_default_params(&iparams);
iparams.talker_path = a.model; iparams.talker_path = a.model;
iparams.codec_path = a.codec; iparams.codec_path = a.codec;
iparams.use_fa = a.use_fa; iparams.use_fa = a.use_fa;
iparams.clamp_fp16 = a.clamp_fp16; iparams.clamp_fp16 = a.clamp_fp16;
iparams.codec_chunk_sec = a.codec_chunk_sec;
qt_context * q = qt_init(&iparams); qt_context * q = qt_init(&iparams);
if (!q) { if (!q) {
@@ -391,31 +389,29 @@ static int run(const Args & a) {
// verbatim and resolved by qt_synthesize via std::random_device. // verbatim and resolved by qt_synthesize via std::random_device.
qt_tts_params params; qt_tts_params params;
qt_tts_default_params(&params); qt_tts_default_params(&params);
params.text = text; params.text = text;
params.lang = a.lang; params.lang = a.lang;
params.instruct = a.instruct; params.instruct = a.instruct;
params.speaker = a.speaker; params.speaker = a.speaker;
params.ref_audio_24k = ref_audio_24k; params.ref_audio_24k = ref_audio_24k;
params.ref_n_samples = ref_n_samples; params.ref_n_samples = ref_n_samples;
params.ref_text = ref_text; params.ref_text = ref_text;
params.ref_spk_emb = ref_spk_emb.empty() ? NULL : ref_spk_emb.data(); params.ref_spk_emb = ref_spk_emb.empty() ? NULL : ref_spk_emb.data();
params.ref_spk_dim = (int) ref_spk_emb.size(); params.ref_spk_dim = (int) ref_spk_emb.size();
params.ref_codes = ref_codes.empty() ? NULL : ref_codes.data(); params.ref_codes = ref_codes.empty() ? NULL : ref_codes.data();
params.ref_T = ref_T; params.ref_T = ref_T;
params.seed = a.seed; params.seed = a.seed;
params.max_new_tokens = a.max_new_tokens; params.max_new_tokens = a.max_new_tokens;
params.do_sample = a.do_sample; params.do_sample = a.do_sample;
params.temperature = a.temperature; params.temperature = a.temperature;
params.top_k = a.top_k; params.top_k = a.top_k;
params.top_p = a.top_p; params.top_p = a.top_p;
params.repetition_penalty = a.repetition_penalty; params.repetition_penalty = a.repetition_penalty;
params.subtalker_do_sample = a.subtalker_do_sample; params.subtalker_do_sample = a.subtalker_do_sample;
params.subtalker_temperature = a.subtalker_temperature; params.subtalker_temperature = a.subtalker_temperature;
params.subtalker_top_k = a.subtalker_top_k; params.subtalker_top_k = a.subtalker_top_k;
params.subtalker_top_p = a.subtalker_top_p; params.subtalker_top_p = a.subtalker_top_p;
params.dump_dir = a.dump_dir; params.dump_dir = a.dump_dir;
params.codec_chunk_sec = a.codec_chunk_sec;
params.codec_left_context_sec = a.codec_left_context_sec;
if (stream_to_stdout) { if (stream_to_stdout) {
wav_stream ws = {}; wav_stream ws = {};
tools/tts-server.cpp
+13 -17
View File
@@ -52,8 +52,7 @@ static void print_usage(const char * prog) {
" --max-batch <n> Concurrent requests batched on the GPU (default: 1)\n" " --max-batch <n> Concurrent requests batched on the GPU (default: 1)\n"
" --no-fa Disable flash attention\n" " --no-fa Disable flash attention\n"
" --clamp-fp16 Clamp hidden states to FP16 range\n" " --clamp-fp16 Clamp hidden states to FP16 range\n"
" --codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0)\n" " --codec-chunk-dur <f> Codec decode chunk duration in seconds, wav responses (default: 24.0)\n",
" --codec-left-dur <f> Codec decode left context duration in seconds (default: 2.0)\n",
prog); prog);
} }
@@ -73,8 +72,9 @@ int main(int argc, char ** argv) {
bool use_fa = true; bool use_fa = true;
bool clamp_fp16 = false; bool clamp_fp16 = false;
int max_batch = 1; int max_batch = 1;
float codec_chunk_dur = 24.0f; // Chunk sentinel : qt_init resolves a non positive value to the
float codec_left_dur = 2.0f; // library default.
float codec_chunk_dur = 0.0f;
for (int i = 1; i < argc; i++) { for (int i = 1; i < argc; i++) {
const char * arg = argv[i]; const char * arg = argv[i];
@@ -98,8 +98,6 @@ int main(int argc, char ** argv) {
max_batch = std::atoi(argv[++i]); max_batch = std::atoi(argv[++i]);
} else if (!std::strcmp(arg, "--codec-chunk-dur") && i + 1 < argc) { } else if (!std::strcmp(arg, "--codec-chunk-dur") && i + 1 < argc) {
codec_chunk_dur = (float) std::atof(argv[++i]); codec_chunk_dur = (float) std::atof(argv[++i]);
} else if (!std::strcmp(arg, "--codec-left-dur") && i + 1 < argc) {
codec_left_dur = (float) std::atof(argv[++i]);
} else if (!std::strcmp(arg, "--help") || !std::strcmp(arg, "-h")) { } else if (!std::strcmp(arg, "--help") || !std::strcmp(arg, "-h")) {
print_usage(argv[0]); print_usage(argv[0]);
return 0; return 0;
@@ -117,11 +115,12 @@ int main(int argc, char ** argv) {
struct qt_init_params iparams; struct qt_init_params iparams;
qt_init_default_params(&iparams); qt_init_default_params(&iparams);
iparams.talker_path = talker_path; iparams.talker_path = talker_path;
iparams.codec_path = codec_path; iparams.codec_path = codec_path;
iparams.use_fa = use_fa; iparams.use_fa = use_fa;
iparams.clamp_fp16 = clamp_fp16; iparams.clamp_fp16 = clamp_fp16;
iparams.max_batch = max_batch; iparams.max_batch = max_batch;
iparams.codec_chunk_sec = codec_chunk_dur;
struct qt_context * q = qt_init(&iparams); struct qt_context * q = qt_init(&iparams);
if (!q) { if (!q) {
@@ -220,14 +219,11 @@ int main(int argc, char ** argv) {
// the same name and injects the pre-extracted reference latents. A // the same name and injects the pre-extracted reference latents. A
// name matching neither is rejected instead of silently generating // name matching neither is rejected instead of silently generating
// voiceless. // voiceless.
be.synthesize = [q, &lang, codec_chunk_dur, codec_left_dur](const tts_request & req, const tts_sink & sink, be.synthesize = [q, &lang](const tts_request & req, const tts_sink & sink, std::string & err) -> int {
std::string & err) -> int {
struct qt_tts_params p; struct qt_tts_params p;
qt_tts_default_params(&p); qt_tts_default_params(&p);
p.text = req.input.c_str(); p.text = req.input.c_str();
p.lang = lang.c_str(); p.lang = lang.c_str();
p.codec_chunk_sec = codec_chunk_dur;
p.codec_left_context_sec = codec_left_dur;
// Copy the registered voice latents out under the lock: the // Copy the registered voice latents out under the lock: the
// synthesis may run for seconds while another connection // synthesis may run for seconds while another connection