ai-agents-configuration-doc-mi-documentation-for-qwen3-model-states-bottom-layers-ffcea4e8

THEN Verify the default attention layer assignment by inspecting the source code of configuration_qwen3.py. The code sets layers with index >= max_window_layers to 'sliding_attention', meaning top layers use SWA. Either update your local documentation or adjust your configuration (e.g., manually set layer_types) if you need SWA on bottom layers. Trust the code behavior as the maintainers confirmed it is correct.

if self.layer_types is None:
    self.layer_types = [
        "sliding_attention" if self.sliding_window is not None and i >= self.max_window_layers else "full_attention"
        for i in range(self.num_hidden_layers)
    ]