ai-agents-guided-generation-wo-vllm-guided-json-or-response-format-with-json-sche-3b796942

THEN When using vLLM with Pydantic schemas that include Enum types or $defs/$ref, override the guided decoding backend to 'outlines' by adding 'guided_decoding_backend': 'outlines' to extra_body. Alternatively, flatten the schema by inlining Enum values (e.g., use Literal) and avoid $ref. For response_format with strict=True, ensure the schema has additionalProperties: false and all properties are simple types without $ref. If the issue persists, downgrade to v0.7.3 where xgrammar backend may accept the schema.

completion = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct",
    messages=[{"role": "user", "content": "Generate a JSON with brand, model, car_type of iconic 90's car"}],
    extra_body={
        "guided_json": json_schema,
        "guided_decoding_backend": "outlines"
    }
)