enne2/qwentts.cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

codec: primed state snapshot LRU keyed by the reference content

Browse Source 
seed_reference hashes the ICL reference codes and restores the conv
contexts, KV ring, and position from a per reference snapshot slot on
a repeat, saving the primed state device to device after a fresh
prime. The reference priming cost amortizes across repeated cloned
voice requests.
This commit is contained in:
Pascal
2026-07-05 14:34:27 +02:00
parent 7a4c799f9c
commit 1a680e8816
4 changed files with 172 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ARCHITECTURE.md
+9
View File
@@ -354,6 +354,15 @@ readback skipped, matching the upstream reference-plus-generated
decode; the transformer receptive field (8 layers x window 72) exceeds
any reference length, so the full prime is the exact one.
The primed state feeds a per-reference snapshot LRU
(`CODEC_SNAP_SLOTS`): after a fresh prime the conv contexts, KV ring,
and position copy device to device into a slot keyed by the FNV-1a
hash of the reference codes, and a later request with the same
reference restores the exact state in one pass of tensor copies
instead of re-decoding every reference frame. Slots allocate lazily
and evict least recently used, so the priming cost amortizes across
repeated cloned-voice requests without any registry coupling.
## Inference pipeline
### Prompt assembly