Quantization for vLLM: FP8, INT4 and NVFP4 without losing the plot

You didn't quantize your 30B FP8 checkpoint: you downloaded it from Hugging Face with the -FP8 suffix. That's the right call 90% of the time. The interesting quantization starts when the model you want has no quantized checkpoint, or when it barely fits your GPU and you need headroom for the KV cache. This chapter covers which format to use, when to grab one that's made, and when to build your own with LLM Compressor.

The real menu, by hardware

Four formats dominate vLLM serving in 2026, and your GPU card decides for you:

NVFP4 is the format Blackwell GPUs run natively, and every large model this year (DeepSeek V4, Kimi K3, Qwen3.8) shipped NVFP4 checkpoints on day one. If your GPU isn't Blackwell, that suffix in the checkpoint name is just letters. Source: vLLM announcements, October 2026.

Search first, quantize later

Before building a checkpoint, check whether Red Hat AI already published yours. Their LLM Compressor pipeline ships production-ready checkpoints for vLLM constantly: Qwen3.8-27B exists in INT4, NVFP4 and MXFP4; GLM-5.3 ships in NVFP4 and in a mixed MXFP4xFP8 format that takes the checkpoint from ~1507 GB down to ~415 GB, 72% smaller, keeping the router and embeddings at original precision. Source: LLM Compressor README.

The rule: an official quantized checkpoint with published evaluation > quantizing it yourself with RTN at 2 AM. Do it yourself when the model is your fine-tune, or it's too new to have a quantized version.

Build your own: LLM Compressor in 15 lines

# quantize.py
from transformers import AutoModelForCausalLM, AutoTokenizer
from llmcompressor import oneshot
from llmcompressor.modifiers.quantization import QuantizationModifier

MODEL_ID = "Qwen/Qwen3-30B-A3B"

model = AutoModelForCausalLM.from_pretrained(MODEL_ID)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

recipe = QuantizationModifier(
    targets="Linear",
    scheme="FP8_BLOCK",
    ignore=["lm_head", "re:.*mlp.gate$"],
)

oneshot(model=model, recipe=recipe)
model.save_pretrained("Qwen3-30B-A3B-FP8-BLOCK")
tokenizer.save_pretrained("Qwen3-30B-A3B-FP8-BLOCK")

Two lines of that recipe matter more than the rest. ignore excludes lm_head and the MoE gate: quantizing the output layer or the router is the fastest way to make your model talk nonsense. And scheme="FP8_BLOCK" is block-wise FP8 with block size 128, the format vLLM loads directly. Note the example uses the simplest algorithm (round-to-nearest); to recover more quality there's GPTQ with calibration, at the cost of a slower oneshot. Source: LLM Compressor guide.

For models that don't fit your GPU even compressed, LLM Compressor supports disk offloading and layer-by-layer processing: a 400 GB checkpoint quantizes on a machine with 64 GB of RAM, just slower.

The KV cache: the other half of the bill

Quantizing weights is half the VRAM pie; the other half you already handle with --kv-cache-dtype fp8 from chapter 5: 96 KB per token in FP16 drops to 48 KB in FP8, double the context on the same card.

What about going lower, to 4 bits? vLLM published a TurboQuant study in May 2026 comparing 4-bit KV cache against FP8 on long workloads: cases where 4 bits help exist, but the conclusion was that FP8 remains the default — the extra memory savings don't pay for the quality risk in most workloads. Source: vLLM blog. Same advice for FP8 attention: vLLM validated on Hopper and Blackwell that some layers should be skipped, and the flag does that for you.

Verify you didn't lose the plot

The honest verification runs on your prompts: 20 real inputs from your workload against the BF16 model and the quantized one, compared side by side. In the LLM Compressor studies and RedHatAI model cards, FP8 W8A8 typically lands under a single point of loss on standard benchmarks; INT4 can fall harder on code and math tasks. If your 20-prompt suite passes, the checkpoint goes to production. If not, step up to FP8 or try GPTQ with calibration before blaming quantization.

Checkpoint and the typical error

Checkpoint: you know which format your GPU demands before searching for a checkpoint, and you have a 20-prompt suite to validate any quantization before serving.

The typical error: quantizing without an ignore list. lm_head and MoE gates at 4 bits break output deterministically: the model answers fluently for 200 tokens and then rambles. Always exclude those layers, and if you download a community checkpoint, check its quantization config excludes them too.

Next chapter

Your model fits and speaks JSON. Now we speed it up: speculative decoding, the technique that cuts your TPOT without touching the hardware.