Pascal
0725d2e53b
codec: adaptive chunk width on the streaming decode
...
The stream graphs come in width classes T in {1, 2, 4, 8} sharing the
state and KV ring tensors, built lazily per class. The decoder ramps
1 -> 2 -> 4 -> 8 so the first frame keeps its latency while the steady
state interleaves the codec 8x less often and the batch amortizes the
kernel count; drain flushes the tail at EOS and the reference priming
runs in max width chunks.
2026-07-06 22:43:52 +02:00
..
2026-07-05 12:28:55 +02:00
2026-05-14 16:47:45 +02:00
2026-05-10 15:57:15 +02:00
2026-05-14 18:09:07 +02:00
2026-05-14 16:47:45 +02:00
2026-07-05 10:25:21 +02:00
2026-07-06 21:46:08 +02:00
2026-07-06 21:46:08 +02:00
2026-06-21 17:30:21 +02:00
2026-07-06 22:43:52 +02:00
2026-07-05 10:25:21 +02:00
2026-07-05 10:25:21 +02:00
2026-05-14 23:27:09 +02:00
2026-05-14 18:09:07 +02:00
2026-05-14 18:09:07 +02:00
2026-05-14 23:27:09 +02:00
2026-07-04 14:14:59 +02:00
2026-05-14 16:47:45 +02:00
2026-05-14 23:27:09 +02:00
2026-07-06 22:43:52 +02:00
2026-07-06 22:43:52 +02:00
2026-07-06 22:43:52 +02:00
2026-07-06 22:26:19 +02:00
2026-06-09 19:53:56 +02:00
2026-05-14 18:09:07 +02:00
2026-07-05 10:25:21 +02:00
2026-05-14 18:09:07 +02:00
2026-07-05 13:45:38 +02:00
2026-06-24 17:18:32 +02:00
2026-07-05 12:28:55 +02:00
2026-05-14 16:47:45 +02:00
2026-05-14 18:09:07 +02:00
2026-05-14 16:47:45 +02:00
2026-05-14 16:47:45 +02:00
2026-05-14 16:47:45 +02:00
2026-07-06 22:26:19 +02:00
2026-07-06 22:26:19 +02:00
2026-06-21 17:30:21 +02:00
2026-05-31 15:05:48 +02:00
2026-07-05 10:25:21 +02:00
2026-07-05 15:01:37 +02:00
2026-07-04 01:14:00 +02:00
2026-05-10 15:57:15 +02:00
2026-05-10 15:57:15 +02:00