server: cloned voice registry over the OpenAI surface
POST /v1/voices registers a voice from a WAV extracted server side through qt_extract_voice_ref or from pre extracted .spk and .rvq latents taken verbatim, DELETE drops it and GET lists it alongside the model speakers. A registered voice wins over a speaker of the same name and injects the reference latents into qt_tts_params, ref_text present selects ICL clone mode. The registry lives in process RAM under the synthesis mutex, so registration and lookups never race a running synthesis. The audio and rvq readers gain buffer variants factored from the file paths. The README and the architecture document catch up on the streaming decode, the hidden bridge, and the server endpoints.
This commit is contained in:
@@ -13,8 +13,9 @@ runs on CPU, CUDA, Metal, Vulkan.
|
||||
in-context with a matching transcript
|
||||
- Voice design from a free text attribute instruction (gender, age,
|
||||
pitch, style)
|
||||
- Streaming synthesis : autoregressive frame loop with chunked codec
|
||||
decode over a rolling left context, low latency chunk callback API
|
||||
- Streaming synthesis : stateful frame-by-frame codec decode, the first
|
||||
audio callback fires one frame after the first Talker step and the
|
||||
output matches the offline full decode exactly
|
||||
- Two stage generation : the Talker LM emits the semantic codebook, a
|
||||
code predictor MTP head emits the 15 acoustic codes per frame, both
|
||||
KV cached
|
||||
@@ -22,8 +23,9 @@ runs on CPU, CUDA, Metal, Vulkan.
|
||||
(repetition penalty -> temperature -> top-k -> top-p -> multinomial)
|
||||
- Q8_0 and Q4_K_M quantisation of the Qwen3 talker backbone (0.6B and
|
||||
1.7B), the RVQ codec paths kept at F32
|
||||
- Two CLI tools : `qwen-tts` (text -> WAV) and `qwen-codec`
|
||||
(WAV <-> RVQ codes)
|
||||
- Three tools : `qwen-tts` (text -> WAV), `qwen-codec`
|
||||
(WAV <-> RVQ codes) and `tts-server` (OpenAI-compatible HTTP server
|
||||
with a cloned voice registry)
|
||||
|
||||
## Build
|
||||
|
||||
@@ -121,6 +123,25 @@ Voice design (`tts.sh`, VoiceDesign, attribute instruction) :
|
||||
--lang English -o out.wav < prompt.txt
|
||||
```
|
||||
|
||||
OpenAI-compatible server (`tts-server`) : `response_format` "pcm"
|
||||
streams s16le as it is generated, "wav" returns a one-shot file. Cloned
|
||||
voices register once over HTTP (a WAV extracted server side, or the
|
||||
`.spk` / `.rvq` latents from `qwen-codec`), then any OAI client selects
|
||||
them by name :
|
||||
|
||||
```
|
||||
./build/tts-server \
|
||||
--model models/qwen-talker-1.7b-base-Q8_0.gguf \
|
||||
--codec models/qwen-tokenizer-12hz-Q8_0.gguf --port 8080
|
||||
|
||||
curl -X POST localhost:8080/v1/voices -H "Content-Type: application/json" \
|
||||
-d "{\"name\":\"freeman\",\"ref_text\":\"$(cat ref.txt)\",
|
||||
\"spk_b64\":\"$(base64 -w0 ref.spk)\",\"rvq_b64\":\"$(base64 -w0 ref.rvq)\"}"
|
||||
|
||||
curl -X POST localhost:8080/v1/audio/speech -H "Content-Type: application/json" \
|
||||
-d '{"input":"Hello world.","voice":"freeman","response_format":"wav"}' -o out.wav
|
||||
```
|
||||
|
||||
## Embedding the library
|
||||
|
||||
The CLI tools are thin wrappers over a public ABI. Single-header,
|
||||
|
||||
Reference in New Issue
Block a user