enne2/qwentts.cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

server: cloned voice registry over the OpenAI surface

Browse Source 
POST /v1/voices registers a voice from a WAV extracted server side
through qt_extract_voice_ref or from pre extracted .spk and .rvq
latents taken verbatim, DELETE drops it and GET lists it alongside the
model speakers. A registered voice wins over a speaker of the same
name and injects the reference latents into qt_tts_params, ref_text
present selects ICL clone mode. The registry lives in process RAM
under the synthesis mutex, so registration and lookups never race a
running synthesis. The audio and rvq readers gain buffer variants
factored from the file paths. The README and the architecture
document catch up on the streaming decode, the hidden bridge, and the
server endpoints.
This commit is contained in:
Pascal
2026-07-05 12:28:55 +02:00
parent a37ff074ff
commit 62dec12580
6 changed files with 464 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+25 -4
View File
@@ -13,8 +13,9 @@ runs on CPU, CUDA, Metal, Vulkan.
in-context with a matching transcript
- Voice design from a free text attribute instruction (gender, age,
pitch, style)
- Streaming synthesis : autoregressive frame loop with chunked codec
decode over a rolling left context, low latency chunk callback API
- Streaming synthesis : stateful frame-by-frame codec decode, the first
audio callback fires one frame after the first Talker step and the
output matches the offline full decode exactly
- Two stage generation : the Talker LM emits the semantic codebook, a
code predictor MTP head emits the 15 acoustic codes per frame, both
KV cached
@@ -22,8 +23,9 @@ runs on CPU, CUDA, Metal, Vulkan.
(repetition penalty -> temperature -> top-k -> top-p -> multinomial)
- Q8_0 and Q4_K_M quantisation of the Qwen3 talker backbone (0.6B and
1.7B), the RVQ codec paths kept at F32
- Two CLI tools : `qwen-tts` (text -> WAV) and `qwen-codec`
(WAV <-> RVQ codes)
- Three tools : `qwen-tts` (text -> WAV), `qwen-codec`
(WAV <-> RVQ codes) and `tts-server` (OpenAI-compatible HTTP server
with a cloned voice registry)
## Build
@@ -121,6 +123,25 @@ Voice design (`tts.sh`, VoiceDesign, attribute instruction) :
--lang English -o out.wav < prompt.txt
```
OpenAI-compatible server (`tts-server`) : `response_format` "pcm"
streams s16le as it is generated, "wav" returns a one-shot file. Cloned
voices register once over HTTP (a WAV extracted server side, or the
`.spk` / `.rvq` latents from `qwen-codec`), then any OAI client selects
them by name :
```
./build/tts-server \
--model models/qwen-talker-1.7b-base-Q8_0.gguf \
--codec models/qwen-tokenizer-12hz-Q8_0.gguf --port 8080
curl -X POST localhost:8080/v1/voices -H "Content-Type: application/json" \
-d "{\"name\":\"freeman\",\"ref_text\":\"$(cat ref.txt)\",
\"spk_b64\":\"$(base64 -w0 ref.spk)\",\"rvq_b64\":\"$(base64 -w0 ref.rvq)\"}"
curl -X POST localhost:8080/v1/audio/speech -H "Content-Type: application/json" \
-d '{"input":"Hello world.","voice":"freeman","response_format":"wav"}' -o out.wav
```
## Embedding the library
The CLI tools are thin wrappers over a public ABI. Single-header,
docs/ARCHITECTURE.md
+101 -17
View File
@@ -317,7 +317,7 @@ graph multiplies plain F32 buffers. The whole DAC pipeline runs T-first
(`ne[0] = T`, `ne[1] = C`) so the fused SNAKE op and `ggml_conv_1d`
share one layout.
### Chunked decode
### Chunked decode (buffered path)
A standalone codec decode of an isolated window shows edge artefacts at
the chunk boundary, because the causal conv kernels and the sliding
@@ -326,8 +326,33 @@ window attention have no left context. `codec_chunked_decode` prepends
then strips the samples that belong to the left context. Defaults match
the upstream tokenizer : `codec_chunk_sec` 24.0 (300 frames at 12.5 Hz)
and `codec_left_context_sec` 2.0 (25 frames). The first chunk collapses
its left context to whatever is available. The same routine serves both
the buffered one-shot decode and the streaming chunk-by-chunk emission.
its left context to whatever is available. This routine serves the
buffered one-shot decode only.
### Streaming decode (stateful path)
Every module in the decoder is causal, so a stateful decode of one
frame at a time reproduces the offline full decode exactly, with no
re-decoded context and no chunk seams. Each stride-1 causal conv keeps
its left context ((k-1)*d input rows) in a persistent backend tensor
that the graph concats ahead of the fresh rows and refreshes in graph;
each DAC transposed conv carries its col2im overlap tail (kernel -
stride rows, bias free) into the next frame; the decoder transformer
attends over a 128-slot sliding window KV ring written through
set_rows, with the ring slot and the absolute RoPE position carried as
input data. All state clears to zero at reset, which matches the
offline zero left pads bit for bit.
The whole T=1 frame graph builds and allocates once
(`pipeline_codec_stream_ensure`, lazy) and computes directly on the
backend without the scheduler : a frame decode is four input uploads
(16 codes, position, ring slot, window mask), one graph compute, and a
1920-sample readback. The constant topology and tensor addresses keep
the CUDA graph cache in pure replay. ICL cloning primes the state by
running the full reference codes through the same path with the
readback skipped, matching the upstream reference-plus-generated
decode; the transformer receptive field (8 layers x window 72) exceeds
any reference length, so the full prime is the exact one.
## Inference pipeline
@@ -386,14 +411,20 @@ the model_type, returning `QT_STATUS_INVALID_PARAMS` :
prefill the Talker on the prompt prefix writes T_ctx into talker_kv
for frame in 0..max_new_tokens-1 :
poll cancel
c0 = sample(codec_head(talker_hidden_last)) codebook 0, top-k/top-p
codes[1..15] = code_predictor_step(talker_hidden_last, c0)
c0 = sample(codec_head(last logits)) codebook 0, top-k/top-p
codes[1..15] = code_predictor_step(hidden_bridge, c0) reads the device bridge
if c0 == codec_eos : break
next_emb = codec_embd(codes) summed over 16 groups
talker_forward_decode(next_emb) appends one position
emit / accumulate codec decode of the gathered frames
streaming : decode the frame through the stateful codec, emit 1920 samples
talker_forward_decode(codes, overlay) gathers next_emb in graph
buffered : chunked codec decode of the gathered frames
```
The talker's last-position hidden never round-trips through the host on
the hot path : the talker graph copies it into a persistent device
tensor (the hidden bridge) that the code predictor prefill reads as a
graph leaf. The only per-frame host traffic is the code ids and overlay
row up, and the logits down for sampling.
Sampling matches the HuggingFace `generate()` chain in F32 :
`repetition_penalty -> temperature -> top_k -> top_p -> softmax ->
multinomial`, the uniform draw coming from `philox_uniform_fill` so a
@@ -448,9 +479,11 @@ QT_STATUS_CANCELLED -5
`qt_tts_params` exposes `cancel` (polled at the top of every Talker
decode step, ~83 ms granularity) and `on_chunk`. With `on_chunk` set,
synthesis runs in streaming mode : audio emits chunk by chunk and `out`
stays empty on success. `codec_chunk_sec` / `codec_left_context_sec`
drive the chunk framing in both buffered and streaming paths.
synthesis runs in streaming mode : every generated frame emits its
1920 samples immediately through the stateful codec and `out` stays
empty on success. `codec_chunk_sec` / `codec_left_context_sec` drive
the chunk framing of the buffered path only; the streaming path
ignores both.
`QT_ABI_VERSION` guards struct growth : callers set `abi_version` (or
let the default-params helpers do it) and the lib rejects a struct laid
@@ -461,6 +494,7 @@ commit date.
Direct access to `pipeline_tts_load` / `pipeline_tts_synthesize`,
`pipeline_codec_encode` / `pipeline_codec_decode`,
`pipeline_codec_stream_reset` / `pipeline_codec_decode_stream`,
`codec_chunked_decode`, and the talker / predictor forwards. Used by the
`qwen-codec` round-trip and the Python cossim harness through dump
files. C++ types in the signatures, not part of the public ABI.
@@ -555,6 +589,53 @@ When -i is omitted, runs a load self-test of the codec GGUF.
The `.rvq` container packs the 16 codes per frame at 11 bits LSB-first.
### tts-server
OpenAI-compatible HTTP server over the public ABI, one GPU-resident
context, synthesis serialized FIFO across connections. The shared HTTP
core lives in `src/tts-server.h` (also consumed by the sibling *.cpp
ports); `tools/tts-server.cpp` wires the `qt_*` ABI into it. Verbatim
`--help` :
```
Usage: ./build/tts-server --model <gguf> --codec <gguf> [options]
Required:
--model <gguf> Talker LM GGUF (qwen-talker-*.gguf)
--codec <gguf> Codec GGUF (qwen-tokenizer-*.gguf)
Optional:
--host <ip> Listen address (default: 127.0.0.1)
--port <n> Listen port (default: 8080)
--lang <n> Language label (default: auto)
--no-fa Disable flash attention
--clamp-fp16 Clamp hidden states to FP16 range
```
Endpoints :
```
POST /v1/audio/speech OAI text-to-speech; response_format "pcm"
streams s16le 24 kHz mono chunked as it is
generated, "wav" returns a one-shot RIFF file
GET /v1/models single loaded model
GET /v1/voices model speakers plus registered cloned voices
POST /v1/voices register a cloned voice: {name, ref_text,
wav_b64} extracts server side through
qt_extract_voice_ref, {name, ref_text,
spk_b64, rvq_b64} takes the pre-extracted
latents verbatim
DELETE /v1/voices/{name} drop a registered voice
GET /health liveness probe
```
A registered voice wins over a model speaker of the same name and
injects the reference latents into `qt_tts_params` : `ref_text` present
selects ICL clone mode, absent selects the x-vector-only mode. The
registry lives in process RAM and every access shares the synthesis
mutex, so registration (which runs the extraction on the GPU) and
lookups never race a running synthesis.
## Module map
```
@@ -584,20 +665,23 @@ src/
encoder-downsample.h 25 Hz -> 12.5 Hz downsample conv
quantizer-encode.h RVQ encode (16 codebooks, split semantic/acoustic)
quantizer-decode.h RVQ decode, per-split output_proj
tokenizer-transformer.h 8-layer local-causal decoder transformer (sw 72)
convnext-block.h ConvNeXt upsample stage (2 blocks, 4x)
causal-trans-conv.h Causal ConvTranspose1d via col2im_1d
dac-decoder-v2.h DAC decoder (Descript Audio Codec; strides 8/5/4/3, SnakeBeta)
codec-chunked-decode.h Bounded-VRAM decode with rolling left context
tokenizer-transformer.h 8-layer local-causal decoder transformer (sw 72), KV ring stream variant
convnext-block.h ConvNeXt upsample stage (2 blocks, 4x), depthwise stream states
causal-trans-conv.h Causal Conv1d / ConvTranspose1d, offline and stateful stream variants
dac-decoder-v2.h DAC decoder (Descript Audio Codec; strides 8/5/4/3, SnakeBeta), stream states
codec-chunked-decode.h Buffered chunked decode plus the stateful frame-by-frame stream decoder
rvq-file.h Packed .rvq code stream IO, file and buffer readers
prompt-builder.h Talker prefix assembly, modes, ICL geometry
pipeline-codec.{h,cpp} Audio tokenizer end-to-end
pipeline-codec.{h,cpp} Audio tokenizer end-to-end, persistent stream state, static frame graph
pipeline-tts.{h,cpp} Full TTS orchestration, prefill, frame loop, decode
tts-server.h Shared OAI HTTP core : routes, parsing, voice registry hooks
qwen.{h,cpp} Public ABI : opaque qt_context, plain C99 header
tools/
qwen-tts.cpp CLI : text to WAV
qwen-codec.cpp CLI : codes <-> WAV
tts-server.cpp OAI HTTP server : qt_* adapter, cloned voice registry
quantize.cpp GGUF requantizer with the codec-aware policy
version.cmake Embeds the git short hash into the binary
src/audio-io.h
+29 -11
View File
@@ -96,17 +96,10 @@ static float * audio_read(const char * path, int * T_out, int * sr_out) {
return result;
}
// Read WAV, resample to target_sr, downmix to mono.
// Returns a flat buffer of T floats at target_sr mono. Caller frees.
static float * audio_read_mono(const char * path, int target_sr, int * T_out) {
int T = 0;
int sr = 0;
float * raw = audio_read(path, &T, &sr);
if (!raw) {
*T_out = 0;
return NULL;
}
// Resample a planar stereo buffer to target_sr and downmix to mono.
// Consumes raw (freed on every path). Returns a flat buffer of T floats,
// caller frees.
static float * audio_mono_from_planar(float * raw, int T, int sr, int target_sr, int * T_out) {
// Resample planar stereo to target_sr first to keep both channels
// coherent when the source rate differs.
float * stereo_rs = raw;
@@ -144,6 +137,31 @@ static float * audio_read_mono(const char * path, int target_sr, int * T_out) {
return mono;
}
// Read WAV, resample to target_sr, downmix to mono.
// Returns a flat buffer of T floats at target_sr mono. Caller frees.
static float * audio_read_mono(const char * path, int target_sr, int * T_out) {
int T = 0;
int sr = 0;
float * raw = audio_read(path, &T, &sr);
if (!raw) {
*T_out = 0;
return NULL;
}
return audio_mono_from_planar(raw, T, sr, target_sr, T_out);
}
// Same conversion from an in-memory WAV byte buffer.
static float * audio_read_mono_buf(const uint8_t * data, size_t size, int target_sr, int * T_out) {
int T = 0;
int sr = 0;
float * raw = audio_io_read_wav_buf(data, size, &T, &sr);
if (!raw) {
*T_out = 0;
return NULL;
}
return audio_mono_from_planar(raw, T, sr, target_sr, T_out);
}
// WAV output format
enum WavFormat {
WAV_S16, // 16-bit signed integer PCM (classic RIFF, default)
src/rvq-file.h
+25 -10
View File
@@ -60,6 +60,30 @@ static std::vector<int32_t> rvq_unpack_codes(const std::vector<uint8_t> & in, si
// Read a .rvq file and unpack it into K*T codes. T is inferred from the
// file size.
// Unpack a raw .rvq byte stream: K codebooks at code_bits per code,
// LSB-first, [K, T] row-major. T derives from the byte count.
static bool rvq_read_buf(const uint8_t * data,
size_t size,
int K,
int code_bits,
std::vector<int32_t> & codes,
int * n_frames) {
if (size == 0) {
fprintf(stderr, "[RVQ] FATAL: empty code stream\n");
return false;
}
const size_t total_bits = size * 8;
const size_t n_codes = total_bits / (size_t) code_bits;
if (n_codes == 0 || (n_codes % (size_t) K) != 0) {
fprintf(stderr, "[RVQ] FATAL: stream yields %zu codes, not a multiple of K=%d\n", n_codes, K);
return false;
}
std::vector<uint8_t> buf(data, data + size);
codes = rvq_unpack_codes(buf, n_codes, code_bits);
*n_frames = (int) (n_codes / (size_t) K);
return true;
}
static bool rvq_read_file(const char * path, int K, int code_bits, std::vector<int32_t> & codes, int * n_frames) {
FILE * f = utf8_fopen(path, "rb");
if (!f) {
@@ -81,16 +105,7 @@ static bool rvq_read_file(const char * path, int K, int code_bits, std::vector<i
return false;
}
fclose(f);
const size_t total_bits = (size_t) sz * 8;
const size_t n_codes = total_bits / (size_t) code_bits;
if (n_codes == 0 || (n_codes % (size_t) K) != 0) {
fprintf(stderr, "[RVQ] FATAL: %s yields %zu codes, not a multiple of K=%d\n", path, n_codes, K);
return false;
}
codes = rvq_unpack_codes(buf, n_codes, code_bits);
*n_frames = (int) (n_codes / (size_t) K);
return true;
return rvq_read_buf(buf.data(), buf.size(), K, code_bits, codes, n_frames);
}
// Pack and write a .rvq file.
src/tts-server.h
+168 -5
View File
@@ -8,10 +8,15 @@
// are identical across projects ; only the adapter differs.
//
// Endpoints:
// POST /v1/audio/speech OAI text-to-speech
// GET /v1/models single loaded model
// GET /v1/voices named speakers (empty when the model has none)
// GET /health liveness probe
// POST /v1/audio/speech OAI text-to-speech
// GET /v1/models single loaded model
// GET /v1/voices model speakers plus registered cloned voices
// POST /v1/voices register a cloned voice: {name, ref_text,
// wav_b64} extracts server side, {name,
// ref_text, spk_b64, rvq_b64} takes
// pre-extracted latents verbatim
// DELETE /v1/voices/{name} drop a registered voice
// GET /health liveness probe
//
// Audio out: response_format "pcm" streams s16le 24 kHz mono chunked as it
// is generated (real time), "wav" returns a one-shot RIFF file. pcm is the
@@ -39,6 +44,19 @@ struct tts_request {
float speed; // OAI speed, parsed then ignored (no time stretch in the ABI)
};
// One voice registration parsed from the POST /v1/voices JSON body.
// Exactly one payload form is present: wav holds decoded base64 WAV
// bytes for server side extraction, or spk plus rvq hold the raw
// contents of pre-extracted .spk and .rvq files. ref_text carries the
// reference transcript enabling ICL clone mode when present.
struct tts_voice_upload {
std::string name;
std::string ref_text;
std::string wav; // WAV file bytes
std::string spk; // .spk file bytes, raw f32 values
std::string rvq; // .rvq file bytes, packed codes
};
// The adapter pushes mono f32 24 kHz audio here. Returns false to abort the
// synthesis (client gone or cancellation), which propagates into the ABI
// on_chunk and stops generation.
@@ -53,6 +71,13 @@ struct tts_backend {
// the ABI status (0 on success), and fills err with the ABI message on
// failure. The shared layer maps the status to an HTTP code.
std::function<int(const tts_request & req, const tts_sink & sink, std::string & err)> synthesize;
// Voice registry hooks, all optional: a null hook answers 501 on the
// matching route. register_voice stores or replaces a cloned voice,
// remove_voice drops one (false when absent), registered_voices lists
// the current names for GET /v1/voices alongside the model speakers.
std::function<bool(const tts_voice_upload & up, std::string & err)> register_voice;
std::function<bool(const std::string & name)> remove_voice;
std::function<std::vector<std::string>()> registered_voices;
};
struct server_config {
@@ -215,6 +240,131 @@ static void tts_handle_speech(const tts_backend & be, const httplib::Request & h
});
}
// Decode standard base64 (with optional padding) into out. Returns
// false on any character outside the alphabet.
static bool tts_b64_decode(const std::string & in, std::string & out) {
static int8_t table[256];
static bool init = false;
if (!init) {
for (int i = 0; i < 256; i++) {
table[i] = -1;
}
const char * alpha = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
for (int i = 0; i < 64; i++) {
table[(uint8_t) alpha[i]] = (int8_t) i;
}
init = true;
}
out.clear();
out.reserve(in.size() / 4 * 3);
uint32_t acc = 0;
int bits = 0;
for (char c : in) {
if (c == '=' || c == '\n' || c == '\r') {
continue;
}
int8_t v = table[(uint8_t) c];
if (v < 0) {
return false;
}
acc = (acc << 6) | (uint32_t) v;
bits += 6;
if (bits >= 8) {
bits -= 8;
out.push_back((char) ((acc >> bits) & 0xff));
}
}
return true;
}
// Parse the POST /v1/voices body: name plus either wav_b64 or the
// spk_b64 / rvq_b64 pair, ref_text optional (enables ICL clone mode).
static bool tts_parse_voice_upload(const std::string & body, tts_voice_upload & up, std::string & err) {
yyjson_doc * doc = yyjson_read(body.c_str(), body.size(), 0);
if (!doc) {
err = "request body is not valid JSON";
return false;
}
yyjson_val * root = yyjson_doc_get_root(doc);
if (!yyjson_is_obj(root)) {
err = "request body must be a JSON object";
yyjson_doc_free(doc);
return false;
}
yyjson_val * name = yyjson_obj_get(root, "name");
if (!yyjson_is_str(name) || yyjson_get_len(name) == 0) {
err = "'name' must be a non-empty string";
yyjson_doc_free(doc);
return false;
}
up.name = yyjson_get_str(name);
yyjson_val * ref_text = yyjson_obj_get(root, "ref_text");
up.ref_text = yyjson_is_str(ref_text) ? yyjson_get_str(ref_text) : "";
yyjson_val * wav = yyjson_obj_get(root, "wav_b64");
yyjson_val * spk = yyjson_obj_get(root, "spk_b64");
yyjson_val * rvq = yyjson_obj_get(root, "rvq_b64");
const bool has_wav = yyjson_is_str(wav) && yyjson_get_len(wav) > 0;
const bool has_spk = yyjson_is_str(spk) && yyjson_get_len(spk) > 0;
const bool has_rvq = yyjson_is_str(rvq) && yyjson_get_len(rvq) > 0;
const bool has_latents = has_spk && has_rvq;
if (has_spk != has_rvq || has_wav == has_latents) {
err = "provide either 'wav_b64' or both 'spk_b64' and 'rvq_b64'";
yyjson_doc_free(doc);
return false;
}
if ((has_wav && !tts_b64_decode(yyjson_get_str(wav), up.wav)) ||
(has_spk && !tts_b64_decode(yyjson_get_str(spk), up.spk)) ||
(has_rvq && !tts_b64_decode(yyjson_get_str(rvq), up.rvq))) {
err = "invalid base64 payload";
yyjson_doc_free(doc);
return false;
}
yyjson_doc_free(doc);
return true;
}
static void tts_handle_voice_register(const tts_backend & be,
const httplib::Request & http_req,
httplib::Response & res) {
if (!be.register_voice) {
tts_json_error(res, 501, "not_implemented", "this backend has no voice registry");
return;
}
tts_voice_upload up;
std::string err;
if (!tts_parse_voice_upload(http_req.body, up, err)) {
tts_json_error(res, 400, "invalid_request_error", err.c_str());
return;
}
if (!be.register_voice(up, err)) {
tts_json_error(res, 400, "invalid_request_error", err.empty() ? "voice registration failed" : err.c_str());
return;
}
std::string body = "{\"name\":\"" + up.name + "\",\"status\":\"registered\"}";
res.set_content(body, "application/json");
}
static void tts_handle_voice_delete(const tts_backend & be,
const httplib::Request & http_req,
httplib::Response & res) {
if (!be.remove_voice) {
tts_json_error(res, 501, "not_implemented", "this backend has no voice registry");
return;
}
const std::string name = http_req.matches[1];
if (!be.remove_voice(name)) {
tts_json_error(res, 404, "not_found_error", "no registered voice with this name");
return;
}
res.set_content("{\"status\":\"deleted\"}", "application/json");
}
static void tts_handle_models(const tts_backend & be, const httplib::Request &, httplib::Response & res) {
yyjson_mut_doc * doc = yyjson_mut_doc_new(NULL);
yyjson_mut_val * root = yyjson_mut_obj(doc);
@@ -243,8 +393,17 @@ static void tts_handle_voices(const tts_backend & be, const httplib::Request &,
for (const std::string & v : be.voices) {
yyjson_mut_val * one = yyjson_mut_obj(doc);
yyjson_mut_obj_add_str(doc, one, "name", v.c_str());
yyjson_mut_obj_add_str(doc, one, "kind", "speaker");
yyjson_mut_arr_add_val(arr, one);
}
if (be.registered_voices) {
for (const std::string & v : be.registered_voices()) {
yyjson_mut_val * one = yyjson_mut_obj(doc);
yyjson_mut_obj_add_val(doc, one, "name", yyjson_mut_strcpy(doc, v.c_str()));
yyjson_mut_obj_add_str(doc, one, "kind", "registered");
yyjson_mut_arr_add_val(arr, one);
}
}
yyjson_mut_obj_add_val(doc, root, "voices", arr);
char * json = yyjson_mut_write(doc, 0, NULL);
res.set_content(json ? json : "{}", "application/json");
@@ -294,7 +453,7 @@ static int tts_server_run(const tts_backend & be, const server_config & cfg) {
{ "Access-Control-Allow-Origin", "*" }
});
svr.Options("/.*", [](const httplib::Request &, httplib::Response & res) {
res.set_header("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
res.set_header("Access-Control-Allow-Methods", "GET, POST, DELETE, OPTIONS");
res.set_header("Access-Control-Allow-Headers", "Content-Type");
});
@@ -304,6 +463,10 @@ static int tts_server_run(const tts_backend & be, const server_config & cfg) {
[&be](const httplib::Request & req, httplib::Response & res) { tts_handle_models(be, req, res); });
svr.Get("/v1/voices",
[&be](const httplib::Request & req, httplib::Response & res) { tts_handle_voices(be, req, res); });
svr.Post("/v1/voices",
[&be](const httplib::Request & req, httplib::Response & res) { tts_handle_voice_register(be, req, res); });
svr.Delete(R"(/v1/voices/(.+))",
[&be](const httplib::Request & req, httplib::Response & res) { tts_handle_voice_delete(be, req, res); });
svr.Get("/health", tts_handle_health);
signal(SIGINT, tts_on_signal);
tools/tts-server.cpp
+116 -2
View File
@@ -6,11 +6,32 @@
#include "tts-server.h"
#include "qwen.h"
#include "rvq-file.h"
#include "version.h"
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <string>
#include <unordered_map>
#include <vector>
// Packed .rvq code width, fixed by the Qwen3-TTS 12 Hz codec (2048
// entries per codebook).
static const int RVQ_CODE_BITS = 11;
// One registered cloned voice: the extraction latents in the ABI
// ownership contract (malloc-owned, released by qt_voice_ref_free) plus
// the reference transcript that enables ICL clone mode when present.
struct voice_entry {
struct qt_voice_ref ref;
std::string ref_text;
};
// Registered voices, name keyed. Every access happens under
// g_synth_mutex: registration touches the GPU through the extraction
// path and lookups run inside the already serialized synthesize.
static std::unordered_map<std::string, voice_entry> g_voices;
static void print_usage(const char * prog) {
fprintf(stderr, "qwentts.cpp %s\n\n", QWEN_VERSION);
@@ -94,15 +115,108 @@ int main(int argc, char ** argv) {
be.voices.push_back(qt_speaker_name(q, i));
}
// Voice registry: POST /v1/voices stores a cloned voice either from a
// WAV (server side extraction through qt_extract_voice_ref) or from
// pre-extracted .spk / .rvq payloads. Re-registering a name replaces
// the previous entry.
be.register_voice = [q](const tts_voice_upload & up, std::string & err) -> bool {
voice_entry entry;
entry.ref = {};
entry.ref_text = up.ref_text;
if (!up.wav.empty()) {
int T = 0;
float * pcm = audio_read_mono_buf((const uint8_t *) up.wav.data(), up.wav.size(), 24000, &T);
if (!pcm) {
err = "cannot decode the WAV payload";
return false;
}
enum qt_status rc;
{
std::lock_guard<std::mutex> lock(g_synth_mutex);
rc = qt_extract_voice_ref(q, pcm, T, &entry.ref);
}
free(pcm);
if (rc != QT_STATUS_OK) {
err = qt_last_error();
return false;
}
} else {
if (up.spk.size() % sizeof(float) != 0 || up.spk.empty()) {
err = "'spk_b64' must decode to a positive multiple of 4 bytes";
return false;
}
std::vector<int32_t> codes;
int ref_T = 0;
const int K = qt_num_codebooks(q);
if (!rvq_read_buf((const uint8_t *) up.rvq.data(), up.rvq.size(), K, RVQ_CODE_BITS, codes, &ref_T)) {
err = "'rvq_b64' does not decode to a valid packed code stream";
return false;
}
entry.ref.ref_spk_dim = (int) (up.spk.size() / sizeof(float));
entry.ref.ref_spk_emb = (float *) malloc(up.spk.size());
std::memcpy(entry.ref.ref_spk_emb, up.spk.data(), up.spk.size());
entry.ref.ref_T = ref_T;
entry.ref.num_codebooks = K;
entry.ref.ref_codes = (int32_t *) malloc(codes.size() * sizeof(int32_t));
std::memcpy(entry.ref.ref_codes, codes.data(), codes.size() * sizeof(int32_t));
}
std::lock_guard<std::mutex> lock(g_synth_mutex);
auto it = g_voices.find(up.name);
if (it != g_voices.end()) {
qt_voice_ref_free(&it->second.ref);
g_voices.erase(it);
}
fprintf(stderr, "[Server] voice '%s' registered (T=%d, ref_text=%s)\n", up.name.c_str(), entry.ref.ref_T,
entry.ref_text.empty() ? "no" : "yes");
g_voices.emplace(up.name, std::move(entry));
return true;
};
be.remove_voice = [](const std::string & name) -> bool {
std::lock_guard<std::mutex> lock(g_synth_mutex);
auto it = g_voices.find(name);
if (it == g_voices.end()) {
return false;
}
qt_voice_ref_free(&it->second.ref);
g_voices.erase(it);
return true;
};
be.registered_voices = []() -> std::vector<std::string> {
std::lock_guard<std::mutex> lock(g_synth_mutex);
std::vector<std::string> names;
names.reserve(g_voices.size());
for (const auto & kv : g_voices) {
names.push_back(kv.first);
}
return names;
};
// The adapter always drives the streaming pipeline : on_chunk routes to
// the shared sink, which either streams to the socket (pcm) or fills a
// one-shot buffer (wav). Either way the audio path is identical.
// one-shot buffer (wav). Either way the audio path is identical. A
// registered voice wins over a model speaker of the same name and
// injects the pre-extracted reference latents.
be.synthesize = [q, &lang](const tts_request & req, const tts_sink & sink, std::string & err) -> int {
struct qt_tts_params p;
qt_tts_default_params(&p);
p.text = req.input.c_str();
p.lang = lang.c_str();
if (!req.voice.empty() && qt_n_speakers(q) > 0) {
auto vit = req.voice.empty() ? g_voices.end() : g_voices.find(req.voice);
if (vit != g_voices.end()) {
const voice_entry & v = vit->second;
p.ref_spk_emb = v.ref.ref_spk_emb;
p.ref_spk_dim = v.ref.ref_spk_dim;
if (!v.ref_text.empty() && v.ref.ref_codes) {
p.ref_codes = v.ref.ref_codes;
p.ref_T = v.ref.ref_T;
p.ref_text = v.ref_text.c_str();
}
} else if (!req.voice.empty() && qt_n_speakers(q) > 0) {
p.speaker = req.voice.c_str();
}
if (!req.instructions.empty()) {