What Serving an AI Model Actually Means (and Why vLLM Exists)
Chapter 01 // 5 min
Guide8/8 chapters
vLLM from zero: your first inference server
From zero to your own vLLM server on a rented GPU: 8 hands-on chapters, for under $5, with measured numbers at every step.
Guide: a closed list on a single topic, start to finish.
Updated:
Progress100%
A hands-on guide to go from zero inference knowledge to your own vLLM server on a rented GPU: for under $5, with measured numbers at every step.
What you'll build
- A local model running on your machine with Ollama
- A vLLM server with a 30B model in FP8 on a 48 GB GPU
- A gateway with per-person keys, budgets, and logs
- Your own TTFT, tokens/s, and concurrency benchmarks
8 chapters, each with commands that ran on a real pod and a time/cost checkpoint. Start at chapter 1 and don't skip steps: each one builds on the previous.
Your First Local Model in 10 Minutes (Ollama)
Chapter 02 // 4 min
Rent Your First GPU Without Fear (~$1 per hour)
Chapter 03 // 3 min
Your First vLLM Server (hello world)
Chapter 04 // 3 min
The 5 vLLM Flags That Actually Matter
Chapter 05 // 4 min
Measure Your vLLM: TTFT, tokens/s and Concurrency Without Fooling Yourself
Chapter 06 // 4 min
LiteLLM: Keys, Budgets and Your vLLM's Bill
Chapter 07 // 4 min
When It Explodes: The 7 Errors of Serving vLLM on a Rented GPU
Chapter 08 // 4 min