Skip to content
AI•5 min read

Surviving the vLLM release train, a migration guide for 2026

vLLM ships a minor version every two weeks. September brought v0.29 with Model Runner V2 as default, October brought v0.31 and v0.32 with Python 3.10 dropped and cascade attention removed, and November arrived with v0.33 and images built for CUDA 13 only. If your inference server lives on a rented pod, every hop can break your stack without warning.

Nobody writes vLLM migration guides. Everyone cites the release notes and moves on. This guide fills that gap with what actually broke in the last three versions and the process to upgrade without taking your service down. - Source vLLM releases

What broke this quarter

Each version ships a list of changes the release notes document well and almost nobody reads until the pod won't boot.

v0.33.0, November 2026. Images are CUDA 13.0 only, the -cu129 variants stopped publishing. Medusa and the MLP speculator were removed from the core. Requests during paused generation now return HTTP 503. The NIXL connector requires version 1.5+. LM Format Enforcer is deprecated. - Source v0.33.0 release

v0.32.0, October 2026. Python 3.10 lost support, the range is now 3.11 to 3.14. CORS no longer allows every origin by default, which breaks frontends calling your API straight from the browser. Cascade attention was removed. max_tokens no longer reserves from the model's context window. Responses API reasoning stream events got renamed. - Source v0.32.0 release

v0.31.0, October 2026. tokenizer_mode="slow" removed, the online quantization="fp8" parser was replaced by fp8_per_tensor, and --enforce-eager now also disables JIT kernel warmup. - Source v0.31.0 release

The structural change is Model Runner V2. v0.29 made it default for all models and the V1 deprecation is explicit in the release notes, with removal scheduled. If your flags and scripts from the V1 era are still alive, they're in the excavator's path. - Source v0.29.0 release

The upgrade process that doesn't take you down

The golden rule of the release train is simple. Never upgrade and serve at the same time on the same pod.

Step 1, freeze the current state. Before touching anything, document the full pin. Container tag, model version, exact startup flags and the output of a baseline benchmark.

# snapshot of current state, keep it in the repo
docker inspect vllm-openai --format '{{.Config.Image}}' > deploy-pin.txt
ps aux | grep vllm >> deploy-pin.txt
vllm bench serve --model $MODEL --num-prompts 50 --percentile-metrics ttft,tpot \
  --save-result baseline-v032.json

Step 2, upgrade in a new container, not the live pod. Pull the new image, boot the same model with the same flags next to the current service, and compare /health plus a short bench before touching production. On RunPod this is cheap, one extra hour on the same GPU.

docker run --gpus all --rm -p 8001:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm:v0.33.0 --model $MODEL [your-flags]

# smoke test against the new port
vllm bench serve --base-url http://localhost:8001 --num-prompts 30 \
  --percentile-metrics ttft,tpot,itl --save-result canary-v033.json

Step 3, check the three places vLLM warns you. The release's breaking changes section, deprecation warnings in the startup log, and the output of vllm serve with unknown flags. Since v0.33, unsupported flags fail at startup instead of being silently ignored, a small change that prevents long nights. - Source v0.33.0 release

Step 4, cut traffic with a canary. Route a percentage of traffic to the new pod through LiteLLM or your router of choice, compare TTFT p90 and error rate per hour, and complete the cutover when the canary holds the load. If anything smells wrong, rollback is pointing the router at the old pod, which stays alive.

The three changes that will find you

CUDA 13 and the base images. If your pod runs CUDA 12.9 drivers, v0.33 won't boot. Update host drivers or stay on v0.32 with the -cu129 image while your GPU provider moves its stack. Providers are already rotating, but RunPod has whole pools on 12.x. - Source PR #62575

The death of Medusa. If your speculative decoding used Medusa heads, v0.33 removed them. The modern path is EAGLE 3.1, DFlash or DSpark with Speculators checkpoints, and the draft-slot budget fix pays back more than the drafter you lost. DeepSeek-V4.1-Flash with DSpark went from 1373 to 3390 output tokens/s on GB300 with that change. - Source PR #61701, PR #59468

The Rust frontend and the usual 500s. Long conversations that returned HTTP 500 due to Python frontend limits no longer exist there, and unsupported requests fail at startup. If you kept workarounds for those bugs, it's time to try without them. - Source PR #62374

What's coming, worth anticipating

The Q4 2026 roadmap is explicit. Full Model Runner V1 removal, the Rust frontend heading to default, FP4 KV cache support for more models with disk-backed offloading, and the declared aspiration of enabling speculative decoding by default. Every item on that list is a future change to your stack. - Source Roadmap Q4 2026

Two novelties soften the train. vllm preload keeps post-quantized weights GPU-resident between restarts with a cache daemon, and the experimental vllm snapshot create/restore restores a fully initialized TP1 engine via CRIU. On an ephemeral pod, restarting without re-loading 30 GB of weights changes the economics of upgrading. - Source v0.31.0 release, v0.30.0 release

Final tips

  1. One version at a time, two at most. Jumping from v0.29 to v0.33 stacks three sets of breaking changes and multiplies the debugging.
  2. Automate the canary bench in CI. A script that boots the new image, runs 30 prompts and compares p90 against baseline turns upgrades into routine.
  3. Read the deprecations section before the features one. Features sell the release, deprecations break your night.
  4. Subscribe to the GitHub releases feed. With a two-week cadence, finding out 15 days late means always migrating two versions at once.
  5. Keep the old image locally. docker save the container that works. When the registry moves tags, your rollback doesn't depend on the internet.

The release train doesn't stop, and it shouldn't. The passenger adapts.

How often do you upgrade your vLLM server, or do you wait until something breaks? Tell me in the comments.


> More posts