We don't publish
your competitive advantage.

IFLoading a Gemma 3 checkpoint with AutoModelForCausalLM on transformers < v4.50 raises 'Transformers does not recognize this architecture' because the model type 'gemma3' is not yet in the release version.

THENInstall the compatible branch from GitHub: `pip install git+https://github.com/huggingface/transformers@v4.49.0-Gemma-3`, or use `AutoModelForImageTextToText` for multimodal usage. For text-only, use `AutoModelForCausalLM` from the main branch (available in v4.50+).

IFWhen loading a Gemma3 checkpoint with AutoModelForCausalLM.from_pretrained in transformers < v4.50, a ValueError is raised indicating the 'gemma3' architecture is not recognized.

THENInstall transformers from the specific git branch that supports Gemma3: `pip install git+https://github.com/huggingface/transformers@v4.49.0-Gemma-3`. For multimodal use cases, use `AutoModelForImageTextToText.from_pretrained`. For text-only use, install transformers from `main` (Gemma3 support will be included in v4.50).

IFUsing device_map='auto' in AutoModelForCausalLM.from_pretrained on a system without GPU or with GPU not detected leads to IndexError: list index out of range in accelerate/big_modeling.py.

THENDo not specify device_map='auto' for CPU-only inference. Either omit device_map (the model will run on CPU by default) or explicitly set device_map='cpu'. If GPU is intended, verify that torch.cuda.is_available() returns True before using device_map='auto'.

IFLoading a model with tied weights (e.g., DeepSeekR1, DeepSeek V3) fails when CPU offloading is enabled using --cpu-offload-gb.

THENTemporarily disable CPU offloading by omitting the --cpu-offload-gb flag when serving models known to use weight tying. Monitor the vLLM issue tracker for a permanent fix in a future release.

IFLoading a model (e.g., Qwen3-235B) via vllm with transformers>=4.57.2 fails with 'AttributeError: 'dict' object has no attribute 'model_type'' in tokenization_utils_base.py.

THENDowngrade transformers to version 4.57.1 by running `pip install transformers==4.57.1`, or apply a monkey patch to change `_config.model_type` to `_config.get("model_type")` in the tokenizer loading code. This avoids a regression where the config is passed as a dict instead of an object.

IFLoading a local Hugging Face model with AutoProcessor.from_pretrained fails with AttributeError: 'dict' object has no attribute 'model_type'

THENIn tokenization_utils_base.py, replace `_config.model_type` with `_config.get('model_type')` to handle cases where the config is a dictionary instead of an object. Alternatively, if you cannot modify the library, set `transformers_version` in `config.json` to `"4.57.2"` as a temporary workaround.

IFError 'AttributeError: dict object has no attribute model_type' when loading a local model with AutoProcessor.from_pretrained using transformers v4.57.2.

THENApply the code fix: replace `_config.model_type` with `_config['model_type']` in `tokenization_utils_base.py` line 2419. Alternatively, set the `transformers_version` key in the model's `config.json` to `"4.57.2"` as a temporary workaround (may have risks). Ensure you are using a version that includes the fix (v4.57.3+).

IFGemma3 is a multimodal model that may not be loadable with AutoModelForCausalLM in older Transformers versions.

THENUse AutoModelForImageTextToText instead of AutoModelForCausalLM to load Gemma3, as it supports multimodal models.

IFKeyError when loading a model with bitsandbytes quantization if the model repository contains both Hugging Face (safetensors) and Mistral format weights.

THENEnsure that only one format of weights is loaded by filtering out duplicate parameter names or by using a single weight format (e.g., --load-format bitsandbytes with explicit file patterns). For vLLM, avoid mixing weight formats in the same model repo when using quantization like bitsandbytes.

IFAutoModel.from_pretrained fails for model names containing invalid Python identifier characters (e.g., dots) when custom code uses relative imports.

THENSanitize the module name by replacing invalid characters such as '.' with a sentinel like '_dot_' when creating the cached module directory and module name in dynamic_module_utils.py to ensure valid Python module identifiers.

IFWhen loading a fine-tuned model that includes a BNB (bitsandbytes) quantization configuration into vLLM, the error 'Cannot find any of ["adapter_name_or_path"] in the model's quantization config' occurs.

THENRemove the BNB quantization configuration from the model's config before loading with vLLM. In the model's config.json, set the 'quantization_config' field to null or delete the key. Alternatively, load the model without the quantization config using transformers and then pass the resulting model to vLLM.

IFphi-3 model fails to load with AssertionError: 'factor' in rope_scaling when using vLLM 0.4.0.post1 or earlier.

THENEdit the model's config.json to add a 'factor' key to the rope_scaling dictionary (e.g., 'factor': 1.0), or upgrade to a vLLM version that includes the fix from PR #4298.