enne2/qwentts.cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

api: derived codec left context, chunk width hoisted to qt_init

Browse Source 
The left context of the buffered chunked decode is no longer a caller
knob: it derives from the codec's own sliding window (2x144 frames),
placing the default decode at the residual floor of the split.
codec_chunk_sec moves from qt_tts_params to qt_init_params, resolved
once to frames at load. The mid-struct removal bumps the ABI to a
closed range [QT_ABI_MIN_VERSION, QT_ABI_VERSION] = [4, 4]; the probe
asserts both bounds reject through the range check.
This commit is contained in:
Pascal
2026-07-25 18:56:28 +02:00
parent 710a52af75
commit d03ffb97f9
8 changed files with 240 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+24 -14
View File
@@ -322,12 +322,22 @@ share one layout.
A standalone codec decode of an isolated window shows edge artefacts at
the chunk boundary, because the causal conv kernels and the sliding
window attention have no left context. `codec_chunked_decode` prepends
`codec_left_context_sec` worth of previously decoded frames, decodes,
then strips the samples that belong to the left context. Defaults match
the upstream tokenizer : `codec_chunk_sec` 24.0 (300 frames at 12.5 Hz)
and `codec_left_context_sec` 2.0 (25 frames). The first chunk collapses
its left context to whatever is available. This routine serves the
buffered one-shot decode only.
`left_ctx_frames` worth of previously decoded frames, decodes, then
strips the samples that belong to the left context. The chunk width
comes from `qt_init_params.codec_chunk_sec` (default 24.0, 300 frames at
12.5 Hz) and resolves to a frame count once at `qt_init`. The left
context is not a caller knob : it derives from the decoder's own sliding
window, 2 x sliding_window (144 frames on this codec), which is where
the chunk output reaches the residual floor of the split. A shorter context leaves the decode
audibly off, a longer one redecodes frames for nothing. The first chunk
collapses its left context to whatever is available. This routine serves
the buffered one-shot decode only.
A chunk covering the whole utterance decodes in a single pass and is
bit exact against `pipeline_codec_decode`. Any split leaves a residual
around -50 dB that no amount of left context removes, growing slowly
with the pass count, so the chunk is a memory knob and not a quality
one.
### Streaming decode (stateful path)
@@ -490,14 +500,15 @@ QT_STATUS_CANCELLED -5
decode step, ~83 ms granularity) and `on_chunk`. With `on_chunk` set,
synthesis runs in streaming mode : every generated frame emits its
1920 samples immediately through the stateful codec and `out` stays
empty on success. `codec_chunk_sec` / `codec_left_context_sec` drive
the chunk framing of the buffered path only; the streaming path
ignores both.
empty on success. `qt_init_params.codec_chunk_sec` drives the chunk framing of the
buffered path only; the streaming path ignores it.
`QT_ABI_VERSION` guards struct growth : callers set `abi_version` (or
let the default-params helpers do it) and the lib rejects a struct laid
out for a newer header. `qt_version()` returns the git short hash and
commit date.
`QT_ABI_VERSION` and `QT_ABI_MIN_VERSION` bound the struct layouts this
build addresses : callers set `abi_version` (or let the default-params
helpers do it) and the lib rejects anything outside that closed range,
a struct laid out for a newer header as well as one whose fields sit at
offsets this build no longer maps. `qt_version()` returns the git short
hash and commit date.
### Low-level API : src/pipeline-tts.h, src/pipeline-codec.h
@@ -554,7 +565,6 @@ Optional:
--ref-text <path> Transcript file for the reference (enables ICL clone mode)
--max-new <n> Max new audio frames (default: 2048)
--codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0)
--codec-left-dur <f> Codec decode left context duration in seconds (default: 2.0)
--stream-by-line Flush synthesis at each newline, one WAV header per line (-o '-')
Sampling:
src/pipeline-tts.cpp
+22 -15
View File
@@ -139,7 +139,8 @@ bool pipeline_tts_load(PipelineTTS * pt,
BackendPair bp,
bool use_fa,
bool clamp_fp16,
int max_batch) {
int max_batch,
float codec_chunk_sec) {
pt->bp = bp;
pt->backend = bp.backend;
pt->sched = NULL;
@@ -149,6 +150,10 @@ bool pipeline_tts_load(PipelineTTS * pt,
pt->hidden_bridge = NULL;
pt->max_batch = max_batch > 1 ? max_batch : 1;
// Chunk width of the buffered decode. The conversion is a fixed
// 12.5 Hz ratio, so it lands here once instead of per synthesis.
pt->codec_chunk_frames = pipeline_tts_duration_sec_to_tokens(pt, codec_chunk_sec);
// Fused flash attention needs a GPU kernel; CPU only backends fall
// back to the F32 manual chain automatically. clamp_fp16 is forwarded
// verbatim: a no op on backends that already accumulate in F32, an
@@ -209,6 +214,13 @@ bool pipeline_tts_load(PipelineTTS * pt,
// allocates the [t, c, S] state tensors from it.
pt->codec.stream_sets = pt->max_batch + 1;
// Left context of the buffered chunked decode. Two decoder windows
// warm the transformer attention and the causal conv stack deep
// enough for the chunk output to sit at the residual floor of the
// split; a shorter context leaves the decode audibly off, a longer
// one buys nothing and redecodes frames for nothing.
pt->codec_left_ctx_frames = 2 * pt->codec.transformer.sliding_window;
// Scheduler shared by talker_forward_* and code_predictor_step.
// Routes ops the GPU backend cannot run (typical case: K-quant
// get_rows on CUDA) to the CPU backend. 4096 nodes covers the 28L
@@ -729,13 +741,10 @@ bool tts_engine_admit(TtsEngine * e, TtsJob * job) {
const std::string speaker = params->speaker ? params->speaker : "";
const std::string ref_text = params->ref_text ? params->ref_text : "";
// ABI v2 latent reference fields. Callers compiled against ABI 1
// never set them; the abi_version gate keeps their uninitialised
// tail bytes out of the read path.
const float * lat_spk_emb = (params->abi_version >= 2) ? params->ref_spk_emb : NULL;
const int lat_spk_dim = (params->abi_version >= 2) ? params->ref_spk_dim : 0;
const int32_t * lat_codes = (params->abi_version >= 2) ? params->ref_codes : NULL;
const int lat_T = (params->abi_version >= 2) ? params->ref_T : 0;
const float * lat_spk_emb = params->ref_spk_emb;
const int lat_spk_dim = params->ref_spk_dim;
const int32_t * lat_codes = params->ref_codes;
const int lat_T = params->ref_T;
const bool has_ref_audio = (params->ref_audio_24k != NULL) && (params->ref_n_samples > 0);
const bool has_lat_spk = (lat_spk_emb != NULL) && (lat_spk_dim > 0);
@@ -1005,13 +1014,11 @@ static void tts_slot_complete(TtsEngine * e, TtsSlot & s) {
// reference codes prepends the buffer so the onset is voiced with
// the reference's causal state, mirroring the upstream pipeline
// which decodes reference plus generated then trims; the seeded
// samples strip from the front afterwards. Raising
// codec_left_context_sec past the reference duration reproduces the
// upstream full reference decode exactly.
const float chunk_sec = params->codec_chunk_sec > 0.0f ? params->codec_chunk_sec : 24.0f;
const float left_ctx_sec = params->codec_left_context_sec >= 0.0f ? params->codec_left_context_sec : 2.0f;
const int chunk_frames = pipeline_tts_duration_sec_to_tokens(pt, chunk_sec);
const int left_ctx_frames = pipeline_tts_duration_sec_to_tokens(pt, left_ctx_sec);
// samples strip from the front afterwards. The seed caps at the
// derived left context, so a reference longer than that window
// contributes only its tail.
const int chunk_frames = pt->codec_chunk_frames;
const int left_ctx_frames = pt->codec_left_ctx_frames;
const int T_frames = (int) s.all_codes.size();
int seed = 0;
src/pipeline-tts.h
+13 -3
View File
@@ -120,6 +120,13 @@ struct PipelineTTS {
// single sequence layout and behavior.
int max_batch;
// Codec decode framing of the buffered path, in 12.5 Hz frames.
// chunk sizes the decode window and comes from the caller; left_ctx
// is the warmup the decoder needs and derives from its own sliding
// window at load.
int codec_chunk_frames;
int codec_left_ctx_frames;
CodecSpecials codec_specials;
TextSpecials text_specials;
std::vector<LanguageEntry> languages;
@@ -172,15 +179,18 @@ struct PipelineTTS {
// invalid metadata. use_fa is gated on bp.has_gpu inside the load:
// CPU only runs always use the manual F32 attention chain. clamp_fp16
// is forwarded as is. max_batch sizes the KV sets, the bridge columns
// and the maximum concurrent slots (minimum 1). Caller frees with
// pipeline_tts_free.
// and the maximum concurrent slots (minimum 1). codec_chunk_sec
// converts to the chunk width every buffered decode on this handle
// runs with; the left context that pairs with it derives from the
// codec's own sliding window. Caller frees with pipeline_tts_free.
bool pipeline_tts_load(PipelineTTS * pt,
const char * talker_gguf_path,
const char * codec_gguf_path,
BackendPair bp,
bool use_fa,
bool clamp_fp16,
int max_batch);
int max_batch,
float codec_chunk_sec);
void pipeline_tts_free(PipelineTTS * pt);
src/qwen.cpp
+50 -45
View File
@@ -212,6 +212,10 @@ void qt_log_set(qt_log_cb cb, void * user_data) {
g_log_cb.store(cb, std::memory_order_release);
}
// Codec chunk default, shared by qt_init_default_params and the
// qt_init resolution of an unset value.
static const float QT_CODEC_CHUNK_SEC_DEFAULT = 24.0f;
void qt_init_default_params(struct qt_init_params * p) {
p->abi_version = QT_ABI_VERSION;
p->talker_path = nullptr;
@@ -219,39 +223,39 @@ void qt_init_default_params(struct qt_init_params * p) {
p->use_fa = true;
p->clamp_fp16 = false;
p->max_batch = 1;
p->codec_chunk_sec = QT_CODEC_CHUNK_SEC_DEFAULT;
}
void qt_tts_default_params(struct qt_tts_params * p) {
p->abi_version = QT_ABI_VERSION;
p->text = nullptr;
p->lang = nullptr;
p->instruct = nullptr;
p->speaker = nullptr;
p->ref_audio_24k = nullptr;
p->ref_n_samples = 0;
p->ref_text = nullptr;
p->seed = -1;
p->max_new_tokens = 2048;
p->do_sample = true;
p->temperature = 0.9f;
p->top_k = 50;
p->top_p = 1.0f;
p->repetition_penalty = 1.05f;
p->subtalker_do_sample = true;
p->subtalker_temperature = 0.9f;
p->subtalker_top_k = 50;
p->subtalker_top_p = 1.0f;
p->dump_dir = nullptr;
p->cancel = nullptr;
p->cancel_user_data = nullptr;
p->on_chunk = nullptr;
p->on_chunk_user_data = nullptr;
p->codec_chunk_sec = 24.0f;
p->codec_left_context_sec = 2.0f;
p->ref_spk_emb = nullptr;
p->ref_spk_dim = 0;
p->ref_codes = nullptr;
p->ref_T = 0;
p->abi_version = QT_ABI_VERSION;
p->text = nullptr;
p->lang = nullptr;
p->instruct = nullptr;
p->speaker = nullptr;
p->ref_audio_24k = nullptr;
p->ref_n_samples = 0;
p->ref_text = nullptr;
p->seed = -1;
p->max_new_tokens = 2048;
p->do_sample = true;
p->temperature = 0.9f;
p->top_k = 50;
p->top_p = 1.0f;
p->repetition_penalty = 1.05f;
p->subtalker_do_sample = true;
p->subtalker_temperature = 0.9f;
p->subtalker_top_k = 50;
p->subtalker_top_p = 1.0f;
p->dump_dir = nullptr;
p->cancel = nullptr;
p->cancel_user_data = nullptr;
p->on_chunk = nullptr;
p->on_chunk_user_data = nullptr;
p->ref_spk_emb = nullptr;
p->ref_spk_dim = 0;
p->ref_codes = nullptr;
p->ref_T = 0;
}
int qt_num_codebooks(const struct qt_context * q) {
@@ -324,18 +328,21 @@ struct qt_context * qt_init(const struct qt_init_params * params) {
qt_log(QT_LOG_ERROR, "[Qwen] qt_init requires talker_path and codec_path");
return nullptr;
}
if (params->abi_version > QT_ABI_VERSION) {
qt_set_error("qt_init: params->abi_version %d > QT_ABI_VERSION %d (binding compiled against a newer header)",
params->abi_version, QT_ABI_VERSION);
qt_log(QT_LOG_ERROR, "[Qwen] qt_init params struct is from a newer ABI (%d > %d)", params->abi_version,
QT_ABI_VERSION);
if (params->abi_version > QT_ABI_VERSION || params->abi_version < QT_ABI_MIN_VERSION) {
qt_set_error("qt_init: params->abi_version %d outside the supported range [%d, %d]", params->abi_version,
QT_ABI_MIN_VERSION, QT_ABI_VERSION);
qt_log(QT_LOG_ERROR, "[Qwen] qt_init params struct carries an unsupported ABI (%d, supported [%d, %d])",
params->abi_version, QT_ABI_MIN_VERSION, QT_ABI_VERSION);
return nullptr;
}
qt_log(QT_LOG_INFO, "[Qwen] qwentts.cpp %s", qt_version());
// ABI v3 tail field: zero init from older callers means 1.
const int max_batch = (params->abi_version >= 3 && params->max_batch > 1) ? params->max_batch : 1;
const int max_batch = params->max_batch > 1 ? params->max_batch : 1;
// The chunk width resolves once here: it is a property of the
// handle, read by every buffered decode it runs.
const float chunk_sec = params->codec_chunk_sec > 0.0f ? params->codec_chunk_sec : QT_CODEC_CHUNK_SEC_DEFAULT;
// new qt_context() value-initialises every field: POD aggregates
// (BackendPair, PipelineTTS) are zero-init, std containers in
@@ -355,7 +362,7 @@ struct qt_context * qt_init(const struct qt_init_params * params) {
}
if (!pipeline_tts_load(&q->pt, params->talker_path, params->codec_path, q->bp, params->use_fa,
params->clamp_fp16, max_batch)) {
params->clamp_fp16, max_batch, chunk_sec)) {
qt_throw("qt_init: pipeline_tts_load failed for '%s' / '%s'", params->talker_path, params->codec_path);
}
@@ -553,10 +560,9 @@ enum qt_status qt_synthesize(struct qt_context * q, const struct qt_tts_params *
qt_set_error("qt_synthesize: out is NULL in buffered mode");
return QT_STATUS_INVALID_PARAMS;
}
if (params->abi_version > QT_ABI_VERSION) {
qt_set_error(
"qt_synthesize: params->abi_version %d > QT_ABI_VERSION %d (binding compiled against a newer header)",
params->abi_version, QT_ABI_VERSION);
if (params->abi_version > QT_ABI_VERSION || params->abi_version < QT_ABI_MIN_VERSION) {
qt_set_error("qt_synthesize: params->abi_version %d outside the supported range [%d, %d]", params->abi_version,
QT_ABI_MIN_VERSION, QT_ABI_VERSION);
if (out) {
qt_audio_free(out);
}
@@ -606,9 +612,8 @@ enum qt_status qt_synthesize(struct qt_context * q, const struct qt_tts_params *
}
return QT_STATUS_MODE_INVALID;
}
// ABI v2 latent reference fields, same gate as the pipeline.
const bool has_lat_spk = params->abi_version >= 2 && params->ref_spk_emb && params->ref_spk_dim > 0;
const bool has_lat_codes = params->abi_version >= 2 && params->ref_codes && params->ref_T > 0;
const bool has_lat_spk = params->ref_spk_emb && params->ref_spk_dim > 0;
const bool has_lat_codes = params->ref_codes && params->ref_T > 0;
if ((params->ref_audio_24k || has_lat_spk) && mt != "base") {
qt_set_error("--ref-wav / --ref-spk is only valid for base models (loaded: %s)", mt.c_str());
src/qwen.h
+41 -31
View File
@@ -44,20 +44,28 @@ extern "C" {
# define QT_API
#endif
// Struct ABI version. Incremented every time a public POD struct grows a
// new field at the end. Callers fill `.abi_version = QT_ABI_VERSION`
// (or let qwen_*_default_params set it). Entries that consume those
// structs reject inputs whose abi_version exceeds the build-time
// constant: this guards a binary built against vN from receiving a
// struct laid out for vN+1 by a freshly compiled binding. Adding fields
// stays backward compat because the new tail is zero init in older
// callers and the lib reads only what its abi_version permits.
// Struct ABI version. Incremented every time a public POD struct
// changes layout. Callers fill `.abi_version = QT_ABI_VERSION` (or let
// qwen_*_default_params set it). Entries that consume those structs
// accept the closed range [QT_ABI_MIN_VERSION, QT_ABI_VERSION] and
// reject anything outside it with a diagnostic rather than reading
// fields at offsets the caller never wrote: above the ceiling the
// struct comes from a newer header, below the floor it carries a
// layout this build no longer addresses. Fields appended at the tail
// keep older callers valid down to the floor, since their unwritten
// tail is zero init and the lib gates on abi_version before reading it.
//
// There is no separate semver triple. The runtime build identity is the
// git short hash + commit date string returned by qt_version(); for
// binding compat checks, QT_ABI_VERSION is the only number that
// matters.
#define QT_ABI_VERSION 3
#define QT_ABI_VERSION 4
// Oldest struct layout this build addresses. A v3 or older
// qt_tts_params places its trailing fields at offsets this build does
// not map, so such a struct is unreadable here and its caller rebuilds
// against this header.
#define QT_ABI_MIN_VERSION 4
// Returns a static string of the form "<git-hash> (<date>)" identifying
// the exact commit this binary was built from. Safe to call from any
@@ -134,10 +142,27 @@ struct qt_init_params {
// back into the qwen_* API. qt_synthesize itself stays blocking
// and thread safe in both modes.
int max_batch;
// ABI v4. Chunk width of the buffered codec decode, in seconds of
// audio, resolved to an integer frame count at the codec frame rate
// by qt_init and applied to every synthesis on the handle. A chunk
// shorter than the utterance bounds the peak decode memory; the
// decode window is that chunk plus the left context the decoder
// needs to start warm, which qt_init derives from the codec's own
// sliding window rather than taking from the caller. Peak memory
// therefore floors at that warmup, and driving the chunk below it
// buys no memory while costing one redecode of the context per
// chunk. A chunk covering the whole utterance decodes in a single
// pass and is the exact reference; any split leaves a residual on
// the order of -50 dB. 0 selects the upstream default, 24.0 (300
// frames at 12.5 Hz). The streaming path frames its own chunks
// through the persistent codec stream state and reads none of this.
float codec_chunk_sec;
};
// Initialise to the standard defaults: both paths NULL (caller must set
// them before calling qt_init), use_fa true, clamp_fp16 false.
// them before calling qt_init), use_fa true, clamp_fp16 false,
// max_batch 1, codec_chunk_sec 24.0.
QT_API void qt_init_default_params(struct qt_init_params * p);
// Allocate every module described by params. Returns NULL on any
@@ -203,10 +228,11 @@ typedef bool (*qt_cancel_cb)(void * user_data);
// 24 kHz; valid only for the duration of the call.
// user_data is forwarded verbatim from on_chunk_user_data.
//
// The chunk granularity is driven by chunk_duration_sec in qt_tts_params:
// once the AR loop has produced enough frames to cover that duration,
// the codec decodes that bundle and emits it. The last chunk on EOS /
// max_new flushes whatever frames remain.
// The chunk granularity is a ramp over the persistent codec stream
// state: the first flush covers a single 12.5 Hz frame for the lowest
// time to first audio, then the target width doubles up to 8 frames as
// the stream settles. The last chunk on EOS / max_new flushes whatever
// frames remain.
typedef bool (*qt_audio_chunk_cb)(const float * samples, int n_samples, void * user_data);
// Log severity. Numerically ordered so a callback can filter with a
@@ -311,21 +337,6 @@ struct qt_tts_params {
qt_audio_chunk_cb on_chunk;
void * on_chunk_user_data;
// Codec decode framing. Applied to both the streaming path (chunk
// by chunk emission) and the buffered path (one shot decode at the
// end) : the chunked decode rolls a left context window across the
// codec frames to avoid edge artefacts at chunk boundaries. The
// first chunk has its left context collapsed to whatever is
// available, matching the upstream Qwen3-TTS 12 Hz tokenizer
// chunked_decode rule. Defaults match the upstream reference :
// codec_chunk_sec 24.0 (300 frames at 12.5 Hz) and
// codec_left_context_sec 2.0 (25 frames at 12.5 Hz). Values are
// converted internally to integer frame counts via the codec frame
// rate ; codec_chunk_sec clamps to >= 1 frame, codec_left_context_sec
// clamps to >= 0 frames.
float codec_chunk_sec;
float codec_left_context_sec;
// ABI v2. Pre-encoded voice reference, the latent counterpart of
// ref_audio_24k. ref_spk_emb is the speaker embedding produced by
// the speaker encoder (ref_spk_dim f32 values, must equal the
@@ -343,8 +354,7 @@ struct qt_tts_params {
// Initialise to the standard defaults. Strings NULL, seed -1,
// max_new_tokens 2048, do_sample true, temperature 0.9, top_k 50,
// top_p 1.0, repetition_penalty 1.05, subtalker mirrors talker,
// dump_dir NULL, cancel NULL, on_chunk NULL, codec_chunk_sec 24.0,
// codec_left_context_sec 2.0.
// dump_dir NULL, cancel NULL, on_chunk NULL.
QT_API void qt_tts_default_params(struct qt_tts_params * p);
// Number of RVQ codebooks (K) of the loaded codec. Pre-encoded ICL
tests/abi-c.c
+29 -3
View File
@@ -68,12 +68,15 @@ int main(void) {
qt_tts_default_params(&params);
/* Sanity-check a few default values, including the abi_version and
* the new use_fa / clamp_fp16 / on_chunk / codec_chunk_sec /
* codec_left_context_sec slots. */
if (params.max_new_tokens != 2048 || params.codec_chunk_sec <= 0.0f || params.codec_left_context_sec < 0.0f) {
* the use_fa / clamp_fp16 / on_chunk / codec framing slots. */
if (params.max_new_tokens != 2048) {
fprintf(stderr, "[Probe] default values do not match\n");
return 1;
}
if (iparams.codec_chunk_sec <= 0.0f || iparams.max_batch != 1) {
fprintf(stderr, "[Probe] init_params chunk / max_batch defaults do not match\n");
return 1;
}
if (iparams.abi_version != QT_ABI_VERSION || params.abi_version != QT_ABI_VERSION) {
fprintf(stderr, "[Probe] abi_version not set by qt_*_default_params\n");
return 1;
@@ -148,6 +151,29 @@ int main(void) {
qt_free(rejected);
return 8;
}
/* The paths point at a file that does not exist, so a NULL return
* alone does not prove the ABI gate fired : the error string must
* name the range check, not a failed GGUF open. */
if (strstr(qt_last_error(), "supported range") == NULL) {
fprintf(stderr, "[Probe] future abi_version rejection did not come from the range check: '%s'\n",
qt_last_error());
return 8;
}
/* The floor is the other half of the range check : a struct laid out
* by a pre-QT_ABI_MIN_VERSION header must be rejected just as hard. */
future_iparams.abi_version = QT_ABI_MIN_VERSION - 1;
rejected = qt_init(&future_iparams);
if (rejected != NULL) {
fprintf(stderr, "[Probe] qt_init accepted an abi_version below the floor\n");
qt_free(rejected);
return 8;
}
if (strstr(qt_last_error(), "supported range") == NULL) {
fprintf(stderr, "[Probe] floor abi_version rejection did not come from the range check: '%s'\n",
qt_last_error());
return 8;
}
enum qt_status rc = qt_synthesize(NULL, &params, &audio);
if (rc != QT_STATUS_INVALID_PARAMS) {
tools/qwen-tts.cpp
+48 -52
View File
@@ -54,7 +54,6 @@ static void print_usage(const char * prog) {
" --ref-text <path> Transcript file for the reference (enables ICL clone mode)\n"
" --max-new <n> Max new audio frames (default: 2048)\n"
" --codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0)\n"
" --codec-left-dur <f> Codec decode left context duration in seconds (default: 2.0)\n"
" --stream-by-line Flush synthesis at each newline, one WAV header per line (-o '-')\n\n"
"Sampling:\n"
" --seed <int> Sampling seed (default: -1 for random)\n"
@@ -101,7 +100,6 @@ struct Args {
bool clamp_fp16;
bool stream_by_line;
float codec_chunk_sec;
float codec_left_context_sec;
};
// Read all of stdin into a string. Binary mode on Windows so UTF-16 input
@@ -180,25 +178,26 @@ static bool read_text_file(const char * path, std::string & out) {
}
static bool parse_args(int argc, char ** argv, Args & a) {
a = {};
a.lang = "auto";
a.format = "wav16";
a.max_new_tokens = 2048;
a.seed = -1;
a.do_sample = true;
a.temperature = 0.9f;
a.top_k = 50;
a.top_p = 1.0f;
a.repetition_penalty = 1.05f;
a.subtalker_do_sample = true;
a.subtalker_top_k = 50;
a.subtalker_top_p = 1.0f;
a.subtalker_temperature = 0.9f;
a.use_fa = true;
a.clamp_fp16 = false;
a.stream_by_line = false;
a.codec_chunk_sec = 24.0f;
a.codec_left_context_sec = 2.0f;
a = {};
a.lang = "auto";
a.format = "wav16";
a.max_new_tokens = 2048;
a.seed = -1;
a.do_sample = true;
a.temperature = 0.9f;
a.top_k = 50;
a.top_p = 1.0f;
a.repetition_penalty = 1.05f;
a.subtalker_do_sample = true;
a.subtalker_top_k = 50;
a.subtalker_top_p = 1.0f;
a.subtalker_temperature = 0.9f;
a.use_fa = true;
a.clamp_fp16 = false;
a.stream_by_line = false;
// Chunk sentinel : qt_init resolves a non positive value to the
// library default.
a.codec_chunk_sec = 0.0f;
for (int i = 1; i < argc; i++) {
const char * arg = argv[i];
if (std::strcmp(arg, "-h") == 0 || std::strcmp(arg, "--help") == 0) {
@@ -260,8 +259,6 @@ static bool parse_args(int argc, char ** argv, Args & a) {
a.stream_by_line = true;
} else if (std::strcmp(arg, "--codec-chunk-dur") == 0 && i + 1 < argc) {
a.codec_chunk_sec = (float) std::atof(argv[++i]);
} else if (std::strcmp(arg, "--codec-left-dur") == 0 && i + 1 < argc) {
a.codec_left_context_sec = (float) std::atof(argv[++i]);
} else if (std::strcmp(arg, "-o") == 0 && i + 1 < argc) {
a.out_wav = argv[++i];
} else {
@@ -279,10 +276,11 @@ static int run(const Args & a) {
// off the two GGUF paths and reports qt_last_error on failure.
qt_init_params iparams;
qt_init_default_params(&iparams);
iparams.talker_path = a.model;
iparams.codec_path = a.codec;
iparams.use_fa = a.use_fa;
iparams.clamp_fp16 = a.clamp_fp16;
iparams.talker_path = a.model;
iparams.codec_path = a.codec;
iparams.use_fa = a.use_fa;
iparams.clamp_fp16 = a.clamp_fp16;
iparams.codec_chunk_sec = a.codec_chunk_sec;
qt_context * q = qt_init(&iparams);
if (!q) {
@@ -391,31 +389,29 @@ static int run(const Args & a) {
// verbatim and resolved by qt_synthesize via std::random_device.
qt_tts_params params;
qt_tts_default_params(&params);
params.text = text;
params.lang = a.lang;
params.instruct = a.instruct;
params.speaker = a.speaker;
params.ref_audio_24k = ref_audio_24k;
params.ref_n_samples = ref_n_samples;
params.ref_text = ref_text;
params.ref_spk_emb = ref_spk_emb.empty() ? NULL : ref_spk_emb.data();
params.ref_spk_dim = (int) ref_spk_emb.size();
params.ref_codes = ref_codes.empty() ? NULL : ref_codes.data();
params.ref_T = ref_T;
params.seed = a.seed;
params.max_new_tokens = a.max_new_tokens;
params.do_sample = a.do_sample;
params.temperature = a.temperature;
params.top_k = a.top_k;
params.top_p = a.top_p;
params.repetition_penalty = a.repetition_penalty;
params.subtalker_do_sample = a.subtalker_do_sample;
params.subtalker_temperature = a.subtalker_temperature;
params.subtalker_top_k = a.subtalker_top_k;
params.subtalker_top_p = a.subtalker_top_p;
params.dump_dir = a.dump_dir;
params.codec_chunk_sec = a.codec_chunk_sec;
params.codec_left_context_sec = a.codec_left_context_sec;
params.text = text;
params.lang = a.lang;
params.instruct = a.instruct;
params.speaker = a.speaker;
params.ref_audio_24k = ref_audio_24k;
params.ref_n_samples = ref_n_samples;
params.ref_text = ref_text;
params.ref_spk_emb = ref_spk_emb.empty() ? NULL : ref_spk_emb.data();
params.ref_spk_dim = (int) ref_spk_emb.size();
params.ref_codes = ref_codes.empty() ? NULL : ref_codes.data();
params.ref_T = ref_T;
params.seed = a.seed;
params.max_new_tokens = a.max_new_tokens;
params.do_sample = a.do_sample;
params.temperature = a.temperature;
params.top_k = a.top_k;
params.top_p = a.top_p;
params.repetition_penalty = a.repetition_penalty;
params.subtalker_do_sample = a.subtalker_do_sample;
params.subtalker_temperature = a.subtalker_temperature;
params.subtalker_top_k = a.subtalker_top_k;
params.subtalker_top_p = a.subtalker_top_p;
params.dump_dir = a.dump_dir;
if (stream_to_stdout) {
wav_stream ws = {};
tools/tts-server.cpp
+13 -17
View File
@@ -52,8 +52,7 @@ static void print_usage(const char * prog) {
" --max-batch <n> Concurrent requests batched on the GPU (default: 1)\n"
" --no-fa Disable flash attention\n"
" --clamp-fp16 Clamp hidden states to FP16 range\n"
" --codec-chunk-dur <f> Codec decode chunk duration in seconds (default: 24.0)\n"
" --codec-left-dur <f> Codec decode left context duration in seconds (default: 2.0)\n",
" --codec-chunk-dur <f> Codec decode chunk duration in seconds, wav responses (default: 24.0)\n",
prog);
}
@@ -73,8 +72,9 @@ int main(int argc, char ** argv) {
bool use_fa = true;
bool clamp_fp16 = false;
int max_batch = 1;
float codec_chunk_dur = 24.0f;
float codec_left_dur = 2.0f;
// Chunk sentinel : qt_init resolves a non positive value to the
// library default.
float codec_chunk_dur = 0.0f;
for (int i = 1; i < argc; i++) {
const char * arg = argv[i];
@@ -98,8 +98,6 @@ int main(int argc, char ** argv) {
max_batch = std::atoi(argv[++i]);
} else if (!std::strcmp(arg, "--codec-chunk-dur") && i + 1 < argc) {
codec_chunk_dur = (float) std::atof(argv[++i]);
} else if (!std::strcmp(arg, "--codec-left-dur") && i + 1 < argc) {
codec_left_dur = (float) std::atof(argv[++i]);
} else if (!std::strcmp(arg, "--help") || !std::strcmp(arg, "-h")) {
print_usage(argv[0]);
return 0;
@@ -117,11 +115,12 @@ int main(int argc, char ** argv) {
struct qt_init_params iparams;
qt_init_default_params(&iparams);
iparams.talker_path = talker_path;
iparams.codec_path = codec_path;
iparams.use_fa = use_fa;
iparams.clamp_fp16 = clamp_fp16;
iparams.max_batch = max_batch;
iparams.talker_path = talker_path;
iparams.codec_path = codec_path;
iparams.use_fa = use_fa;
iparams.clamp_fp16 = clamp_fp16;
iparams.max_batch = max_batch;
iparams.codec_chunk_sec = codec_chunk_dur;
struct qt_context * q = qt_init(&iparams);
if (!q) {
@@ -220,14 +219,11 @@ int main(int argc, char ** argv) {
// the same name and injects the pre-extracted reference latents. A
// name matching neither is rejected instead of silently generating
// voiceless.
be.synthesize = [q, &lang, codec_chunk_dur, codec_left_dur](const tts_request & req, const tts_sink & sink,
std::string & err) -> int {
be.synthesize = [q, &lang](const tts_request & req, const tts_sink & sink, std::string & err) -> int {
struct qt_tts_params p;
qt_tts_default_params(&p);
p.text = req.input.c_str();
p.lang = lang.c_str();
p.codec_chunk_sec = codec_chunk_dur;
p.codec_left_context_sec = codec_left_dur;
p.text = req.input.c_str();
p.lang = lang.c_str();
// Copy the registered voice latents out under the lock: the
// synthesis may run for seconds while another connection