enne2/qwentts.cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

loader, quantize: align GGUF on llama.cpp, conv kernels widened to F16 at load

Browse Source 
GGUF norm matches llama.cpp policy: F32 master stays F32, BF16
variant keeps source BF16, K-quants fall back to F16 when kernel
rows do not align. No conv override in pick_type.

Conv kernels widen to F16 at load through gf_load_conv (12 sites).
qwen_load_ctw_f32 accepts BF16 source.

TODO upstream GGML: ggml_conv_1d and ggml_conv_1d_dw force F16 on
their im2col output, while conv_2d picks the kernel dtype. This
crashes F32 and BF16 kernels on CPU (im2col only handles F16) and
BF16 on Vulkan (mul_mat refuses BF16 on the operand the kernel
ends up on). Aligning conv_1d on conv_2d removes the workaround.
This commit is contained in:
Pascal
2026-05-11 12:42:54 +02:00
parent 59fda26827
commit 530eed6ac5
8 changed files with 113 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/pipeline-codec.cpp
+1 -1
View File
@@ -57,7 +57,7 @@ bool pipeline_codec_load(PipelineCodec * pc, const char * gguf_path, BackendPair
{
WeightCtx wctx;
wctx_init(&wctx, 4);
pc->pre_conv_w = gf_load_tensor(&wctx, pc->gguf, "tok_dec.pre_conv.weight");
pc->pre_conv_w = gf_load_conv(&wctx, pc->gguf, "tok_dec.pre_conv.weight");
pc->pre_conv_b = gf_load_tensor(&wctx, pc->gguf, "tok_dec.pre_conv.bias");
if (!wctx_alloc(&wctx, pc->backend)) {
qt_log(QT_LOG_ERROR, "[Pipeline] pre_conv backend allocation failed");