fix speaker encoder ECAPA forward cossim 0.86 -> 0.996
mel-spk and mel-mag dumps in speaker-encoder-extract.h applied an extra ggml_transpose plus cont before write. Raw ggml ne=(C, T) already streams as numpy [T, C], so the transpose was inverting axes vs the python upstream. Removed it. MelMag 0.04 -> 0.999, MelSpk 0.92 -> 0.998 spk_conv1d_same passed ggml_im2col a kernel ne=(K, 1, IC, 1) and an input ne=(T_pad, 1, IC, 1) with IC in ne[2]. But the im2col impl reads IC = b->ne[1] when is_2D=false, so it saw IC=1, wrote OW*K floats into a buffer declared for OW*IC*K floats, and mul_mat consumed 99% garbage. Moved IC into ne[1] for both kernel and input, which makes the impl read the real IC and writes a buffer coherent with the declared ne. The permute and the retranspose after pad become unnecessary, dropped both. SpkFrontend 0.74 -> 0.994, SpeakerEmb 0.86 -> 0.996 Adds ECAPA bisection infrastructure : 4 stage out params in speaker_encoder_forward (frontend, block3, mfa, asp), codec encoder intermediate dumps in pipeline-codec.cpp (seanet-out, enc-transformer out, codec-pre-fsq), matching pytorch hooks in debug-clone-cossim.py.
This commit is contained in:
@@ -86,12 +86,17 @@ bool pipeline_codec_load(PipelineCodec * pc, const char * gguf_path, BackendPair
|
||||
std::vector<float> pipeline_codec_decode(PipelineCodec * pc, const int32_t * codes, int K, int T);
|
||||
|
||||
// Encode a 24 kHz mono waveform into RVQ codes.
|
||||
// audio : [n_samples] f32 mono 24 kHz. Must be a multiple of
|
||||
// QWEN_TOKENIZER_HOP_LENGTH (1920); the caller is expected
|
||||
// to pad with zeros if needed.
|
||||
// audio : [n_samples] f32 mono 24 kHz. Must be a multiple of
|
||||
// QWEN_TOKENIZER_HOP_LENGTH (1920); the caller is expected
|
||||
// to pad with zeros if needed.
|
||||
// dump_dir : optional path. When non NULL, dumps the SEANet, encoder
|
||||
// transformer and post-downsample (pre-FSQ latents) buffers
|
||||
// into seanet-out.bin, enc-transformer-out.bin and
|
||||
// codec-pre-fsq.bin under that directory. Quiet otherwise.
|
||||
// Returns codes flat as [K, T] row-major, K = QWEN_TOKENIZER_NUM_CODEBOOKS,
|
||||
// T = n_samples / 1920. Empty on failure.
|
||||
std::vector<int32_t> pipeline_codec_encode(PipelineCodec * pc, const float * audio, int n_samples);
|
||||
std::vector<int32_t> pipeline_codec_encode(PipelineCodec * pc, const float * audio, int n_samples,
|
||||
const char * dump_dir = NULL);
|
||||
|
||||
// Free every backend buffer and ggml context. Safe to call on a zeroed struct.
|
||||
void pipeline_codec_free(PipelineCodec * pc);
|
||||
|
||||
Reference in New Issue
Block a user