Structured outputs and tool calling: your vLLM stops returning prose

Your vLLM server from chapter 5 returns text. An agent needs executable JSON: function names, typed arguments, fields a client can parse without a duct-taped regex. In vLLM that comes in two layers: structured outputs for guaranteed JSON and tool calling for functions. This chapter wires up both, with the flags that changed this year and one that no longer exists.

How this differs from the 5 essential flags: there you flipped --enable-auto-tool-choice and moved on. Here we go inside each piece, because when your agent fails, knowing whether it's "the parser" or "the grammar" saves you hours.

Guaranteed JSON: structured outputs

vLLM constrains the model's output through a grammar compiled from your schema. Two backends: xgrammar and guidance (llguidance). If you don't pick one, the auto backend decides per request based on the schema. Compilation costs: the first request with a new schema takes several seconds, then it's cached. If your service receives 50 different schemas per minute, that cache is your silent bottleneck.

The client-side code, with a Pydantic model defining the schema:

# structured_output.py
from pydantic import BaseModel
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
model = client.models.list().data[0].id

class ExtractedIssue(BaseModel):
    file: str
    line: int
    severity: str
    summary: str

completion = client.chat.completions.create(
    model=model,
    messages=[{"role": "user", "content": "Extract the issue from this log: ..."}],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "issue", "schema": ExtractedIssue.model_json_schema()},
    },
)
print(completion.choices[0].message.content)

Watch out for the old field: if you copied tutorial code with extra_body={"guided_json": schema}, that parameter was removed in vLLM v0.12.0. The replacement is {"structured_outputs": {"json": schema}}. Source: structured outputs docs, updated October 24, 2026. Half of the 400 errors you'll hit migrating old examples come from this.

If your model reasons (a thinking Qwen3, a DeepSeek-R1), the JSON comes out of the reasoning and the content field arrives empty. You fix it at server startup with --reasoning-parser and, for Qwen3 Coder, --structured-outputs-config.enable_in_reasoning=True. Without that flag, structured outputs and reasoning don't play well together.

Tool calling: the parser belongs to the model, not the server

The startup flags are still two:

vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml

What changed is the parser catalog, because every model family emits calls in its own format. The ones you'll cross in 2026:

If your previous guide used qwen3_coder and vLLM rejects it today, it's not your server: the parser for Qwen3-Coder is now called qwen3_xml. The catalog moves between versions; vllm serve --help | grep -A20 tool-call-parser beats any tutorial.

By default, tool_choice="auto" enforces nothing. The model can emit malformed markup and vLLM extracts the calls from raw text, with arguments that occasionally violate your schema. This year vLLM gained strict mode: mark the tool with strict: true in the request and vLLM pins the argument schema with a grammar. If you own the server and don't trust the clients, there's --tool-strict-level parameter, which enforces argument schemas for every tool on every request.

The full loop, minimal

An agent is nothing but a while loop that executes tool_calls until the model answers without them:

# agent_loop.py
import json
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")

def read_file(path: str) -> str:
    return open(path).read()[:2000]

tools = [{
    "type": "function",
    "function": {
        "name": "read_file",
        "description": "Reads a file from the repo",
        "parameters": {
            "type": "object",
            "properties": {"path": {"type": "string"}},
            "required": ["path"],
            "additionalProperties": False,
        },
        "strict": True,
    },
}]

messages = [{"role": "user", "content": "Summarize the repo's README.md"}]
for _ in range(5):
    res = client.chat.completions.create(
        model=client.models.list().data[0].id,
        messages=messages, tools=tools, tool_choice="auto",
    )
    msg = res.choices[0].message
    if not msg.tool_calls:
        print(msg.content)
        break
    messages.append(msg)
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = read_file(**args)
        messages.append({"role": "tool", "tool_call_id": call.id, "content": result})

Two decisions worth noting. The schema follows OpenAI's strict style: additionalProperties: false and every field in required — that's what lets vLLM's grammar pin the arguments without ambiguity. And the loop caps at 5 iterations: without a cap, a model that insists on calling tools costs you infinite money. The full guardrails live in my agents guide. Your server already speaks the language; the rest of the work lives in your client.

Checkpoint and the typical error

Checkpoint: your server emits structured tool_calls (not prose with JSON inside) and schema requests always return valid JSON. The test: send a request with a schema and validate the output with your Pydantic — if it never throws, you're ready.

The typical error: mixing reasoning and structured outputs without the --structured-outputs-config.enable_in_reasoning=True flag. Symptom: empty content, or the JSON trapped inside the thinking. Runner-up: a parser from the tutorial you copied — the catalog changes between versions.

Next chapter

Your agent works. Now we make it fit the GPU you can actually pay for: FP8, INT4 and NVFP4 quantization with quality numbers, not marketing ones.