The Qwen3.8 weight quantization ladder

This is a continuation from Measuring impact of KV quantization on Qwen3.8.

There I compared two weight quantizations (Q4/Q3) for Qwen3.8 27B, looked at their KLDs vs Q8 baseline. Those two weight quants were so far apart that I was curious to understand if the Q4 quant I am running was similarly far away from Q5, and whether it was worth upping the weight quant and dropping KV quantization further to compensate.

Here, I am testing a few variations of Q3 through Q5 weight quants from Unsloth, using the same methodology from the previous post. The caveat, again, is that the corpus I am testing with is prose/narrative heavy (not coding/STEM/tool calling), and limited to 20K context size.

Summary of results

• Holding KV dtype static, as expected, higher quant weights beat lower weight quants (see ladder below)
• Unsloth Q4_K_XL weight quantization is the inflection point. Moving up or down the weight quantization means accepting a less favorable KLD mean/VRAM improvement/regression ratio.
• There is no practical KV cache quantization that makes it worth moving the weight quant. For example, Q5_K_S weight quantization, even at Q4 KV cache dtype requires more VRAM and has worse KLD mean than running Q4_K_XL at F16.
• At 24GB VRAM, Q4_K_XL at Q8 dtype KV seems to be the sweet spot (in my opinion), given the saving on VRAM (allowing for bigger context window limits) and reasonable trade-off on the distance from the baseline model’s probability distribution (with all the caveats: narrative corpus, 20K context window test, measured against Q8 baseline).

Weight quantization comparison

Weight-only comparison (same F16/F16 cache dtype for the runs in the table below), ranked by ascending mean KLD against the Q8_K_XL reference:

Marginal cost per GB freed, moving down the ladder:

Based on this, the answer to my original question is: no, not worth going from Q4_K_XL -> Q5_K_S for the extra 1GB VRAM cost. I’d rather deploy that into a larger context window. Going lower to Q4_K_M or below is also not enticing given the acceleration of the KLD mean movement.

Weight + KV quantization visualization

Claude generated these nice visualizations (thanks, Claude!).

Static image.

Bands are ordered top to bottom by fidelity; within each band, rows are the same nine K/V cache combinations in the same fixed order as the previous post’s tables, so a given cache setting sits at the same offset in every band. The x-axis is zoomed to the data range (it doesn’t start at zero) — this is a comparative point-range chart, not a bar chart, so a truncated axis doesn’t distort area the way it would on a bar. Hover (or tap) any point: it draws a guide line through it and colors every other point green (better) or red (worse) relative to it, and the tooltip includes an estimated total VRAM (weights + KV cache) for that exact config at 20K context.

Static image.

A long line between the two dots on a row means that config is fine most of the time but has a sharp catastrophic tail; a short line means the damage is spread evenly. Across the whole ladder the lines get longer going down (P99 grows faster than median), so the degradation from more aggressive weight quantization isn’t uniform blurring, it’s an increasingly heavy tail of near-total failures on a small fraction of tokens.

Full results

Q5_K_M vs Q8_K_XL (19.77 GB weights):

Q5_K_S vs Q8_K_XL (18.67 GB weights):

Q4_K_XL vs Q8_K_XL (17.56 GB weights, see previous post for K/V cache table):

Q4_K_M vs Q8_K_XL (16.46 GB weights):

Q4_K_S vs Q8_K_XL (15.36 GB weights):

Q3_K_XL vs Q8_K_XL (13.15 GB weights, see previous post for K/V cache table):

Limitations

Same as the previous post (check for details): prose/narrative corpus (results on STEM/code may differ), 20K context only, no qualitative assessment, not benchmarked against known answers.

What next

• Repeat at 90K context (the close to max context window I can run on 24GB VRAM with Q4_K_XL)