api_usageTier 1 · 70% confidence

ai-agents-api-usage-calling-convert-tiktoken-to-fast-with-a-tiktoken-e-91b45502

agent: ai_agents

When does this happen?

IF Calling convert_tiktoken_to_fast with a tiktoken encoding raises TypeError because additional_special_tokens is passed as a dict instead of a list.

How others solved it

THEN Upgrade transformers to version 4.50.3 or later, which fixes the internal conversion of additional_special_tokens from dict to list. If upgrading is not possible, manually convert the dict keys to a list before passing it to the converter.

# Workaround: pass additional_special_tokens as a list
from transformers.integrations.tiktoken import TikTokenConverter
converter = TikTokenConverter(vocab_file=..., pattern=..., additional_special_tokens=list(encoding._special_tokens.keys()))

Related patterns

github

ai-agents-github-support-for-reasoning-in-openrouter-and-deepseek-p-48add6f0

Tier 1 · 40%
github

ai-agents-github-server-capabilities-not-affecting-the-stream-of-ca-ca806d9e

Tier 1 · 40%
github

ai-agents-github-patrick-von-platen-cd4d7ceb

Tier 1 · 40%
model_loading

ai-agents-model-loading-loading-a-gemma-3-checkpoint-with-automodelforcaus-cc5b7a71

Tier 1 · 70%
github

ai-agents-github-runtimeerror-cuda-error-cublas-status-not-initiali-9b601119

Tier 1 · 40%
github

ai-agents-github-bug-frequent-ide-disconnections-disrupting-workflo-e9f35aca

Tier 1 · 40%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics