ai-agents-llama-model-rope-con-llama-3-models-produce-nonsensical-output-when-con-40ac5a1c

THEN When using LangChain's LlamaCpp with Llama 3 models, explicitly set the `rope_freq_base` parameter in the LlamaCpp constructor to 500000 (for standard context). Do not use `model_kwargs` to pass it, as LangChain's constructor overrides them with a hardcoded default (10000) that is incompatible with Llama 3's RoPE base frequency. Without this explicit setting, the model misbehaves for longer contexts.

llm = LlamaCpp(
    model_path="./models/Meta-Llama-3-70B-Instruct.Q4_K_M.gguf",
    n_ctx=8192,
    rope_freq_base=500000,  # explicitly set, not via model_kwargs
    ...
)