codec: add pre-encoded voice reference (--ref-spk / --ref-rvq)

qwen-codec --talker extracts the speaker embedding (.spk, raw f32)
and the ICL codes (.rvq) in one pass, encode truncated to the hop
boundary conforming to the --ref-wav path. qwen-tts loads them via
--ref-spk / --ref-rvq and skips the speaker encoder and codec encode
on every synthesis: TTFA 205 ms -> 89 ms. Extends qt_tts_params with
ABI v2 latent fields, adds qt_num_codebooks(), ships freeman.spk +
freeman.rvq and switches clone scripts to the latent path. Output is
bit-identical to the raw path at fixed seed.
This commit is contained in:
Pascal
2026-06-11 21:45:13 +02:00
parent e8e33629c1
commit 0bf4a18b22
11 changed files with 399 additions and 144 deletions
+16
View File
@@ -82,6 +82,22 @@ Voice cloning (`clone.sh`, Base, reference WAV plus its transcript) :
--lang English -o out.wav < prompt.txt
```
Pre-encoded reference (`clone.sh`): `qwen-codec --talker` encodes a reference
WAV into two compact latents in one pass, the `.spk` speaker embedding and
the `.rvq` ICL codes, bit-identical to what the `--ref-wav` path computes
internally. Passing them via `--ref-spk` / `--ref-rvq` skips the speaker
encoder and the codec encode on every synthesis:
```
build/qwen-codec --model models/qwen-tokenizer-12hz-Q8_0.gguf \
--talker models/qwen-talker-1.7b-base-Q8_0.gguf -i ref.wav
build/qwen-tts \
--model models/qwen-talker-1.7b-base-Q8_0.gguf \
--codec models/qwen-tokenizer-12hz-Q8_0.gguf \
--ref-spk ref.spk --ref-rvq ref.rvq --ref-text ref.txt \
--lang English -o out.wav < prompt.txt
```
Named speaker (`customvoice.sh`, CustomVoice) :
```