fix speaker encoder ECAPA forward cossim 0.86 -> 0.996

mel-spk and mel-mag dumps in speaker-encoder-extract.h applied an extra
ggml_transpose plus cont before write. Raw ggml ne=(C, T) already
streams as numpy [T, C], so the transpose was inverting axes vs the
python upstream. Removed it.
MelMag 0.04 -> 0.999, MelSpk 0.92 -> 0.998

spk_conv1d_same passed ggml_im2col a kernel ne=(K, 1, IC, 1) and an
input ne=(T_pad, 1, IC, 1) with IC in ne[2]. But the im2col impl reads
IC = b->ne[1] when is_2D=false, so it saw IC=1, wrote OW*K floats into
a buffer declared for OW*IC*K floats, and mul_mat consumed 99% garbage.
Moved IC into ne[1] for both kernel and input, which makes the impl
read the real IC and writes a buffer coherent with the declared ne. The
permute and the retranspose after pad become unnecessary, dropped both.
SpkFrontend 0.74 -> 0.994, SpeakerEmb 0.86 -> 0.996

Adds ECAPA bisection infrastructure : 4 stage out params in
speaker_encoder_forward (frontend, block3, mfa, asp), codec encoder
intermediate dumps in pipeline-codec.cpp (seanet-out, enc-transformer
out, codec-pre-fsq), matching pytorch hooks in debug-clone-cossim.py.
This commit is contained in:
Pascal
2026-05-10 20:56:14 +02:00
parent acb75fca36
commit 7e89929a70
7 changed files with 431 additions and 52 deletions
+9 -4
View File
@@ -86,12 +86,17 @@ bool pipeline_codec_load(PipelineCodec * pc, const char * gguf_path, BackendPair
std::vector<float> pipeline_codec_decode(PipelineCodec * pc, const int32_t * codes, int K, int T);
// Encode a 24 kHz mono waveform into RVQ codes.
// audio : [n_samples] f32 mono 24 kHz. Must be a multiple of
// QWEN_TOKENIZER_HOP_LENGTH (1920); the caller is expected
// to pad with zeros if needed.
// audio : [n_samples] f32 mono 24 kHz. Must be a multiple of
// QWEN_TOKENIZER_HOP_LENGTH (1920); the caller is expected
// to pad with zeros if needed.
// dump_dir : optional path. When non NULL, dumps the SEANet, encoder
// transformer and post-downsample (pre-FSQ latents) buffers
// into seanet-out.bin, enc-transformer-out.bin and
// codec-pre-fsq.bin under that directory. Quiet otherwise.
// Returns codes flat as [K, T] row-major, K = QWEN_TOKENIZER_NUM_CODEBOOKS,
// T = n_samples / 1920. Empty on failure.
std::vector<int32_t> pipeline_codec_encode(PipelineCodec * pc, const float * audio, int n_samples);
std::vector<int32_t> pipeline_codec_encode(PipelineCodec * pc, const float * audio, int n_samples,
const char * dump_dir = NULL);
// Free every backend buffer and ggml context. Safe to call on a zeroed struct.
void pipeline_codec_free(PipelineCodec * pc);