Structured outputs and tool calling: your vLLM stops returning prose
Chapter 01 // 4 min
Guide5/5 chapters
vLLM: from hello world to production
Five chapters to take your vLLM to production: structured outputs and tool calling, quantization, speculative decoding, KV offloading and multi-GPU.
Guide: a closed list on a single topic, start to finish.
Updated:
Progress100%
Part two of vLLM from scratch. Your server already works: now it speaks JSON with guarantees, fits the GPU you can pay for, answers faster and survives your agents in production. Five chapters, measured numbers in each one.
Quantization for vLLM: FP8, INT4 and NVFP4 without losing the plot
Chapter 02 // 4 min
Speculative decoding: cut your TPOT with MTP and EAGLE-3
Chapter 03 // 3 min
KV cache that doesn't fit: offloading in vLLM, from file to object
Chapter 04 // 3 min
When one GPU isn't enough: multi-GPU and P/D disaggregation
Chapter 05 // 4 min