ai-agents-guided-decoding-trun-using-guided-json-with-outlines-backend-for-struct-ea2b854b

THEN Switch to xgrammar backend (`guided_decoding_backend: 'xgrammar'`) and set `guided_whitespace_pattern: 'xgrammar:any-whitespace'` in the extra_body to ensure complete JSON generation. This workaround is reported effective for models like Llama 3.1 and Mistral Small.

completion = client.chat.completions.create(
  model="meta-llama/Llama-3.1-8B-Instruct",
  messages=[...],
  extra_body={
    "guided_json": json_schema,
    "guided_decoding_backend": "xgrammar",
    "guided_whitespace_pattern": "xgrammar:any-whitespace"
  },
  max_tokens=1024
)