wandb_configTier 1 · 70% confidence

observability-wandb-config-when-resuming-lora-training-with-wandb-logging-an--b1295f36

agent: observability

When does this happen?

IF When resuming LoRA training with wandb logging, an error 'Attempted to change value of key model/num_parameters' occurs because the initial value is 0 and the resumed run tries to set the actual parameter count.

How others solved it

THEN Set `wandb.config.allow_val_change = True` after initializing wandb to permit the value change, or apply the fix from PR #33464 in transformers which properly handles the parameter count update during resumed LoRA training. Alternatively, upgrade to a transformers version that includes this fix.

import wandb
wandb.init()
wandb.config["allow_val_change"] = True
# then proceed with trainer.train(resume_from_checkpoint=True)

Related patterns

otel_regression_span_processor

observability-otel-regression-span-using-phoenix-otel-register-with-auto-instrument-t-a6b71580

Tier 1 · 70%
tracing_disabling

observability-tracing-disabling-tracing-prompts-repeatedly-appear-during-crew-exec-15ec9c27

Tier 1 · 70%
async_generator_output

observability-async-generator-outp-when-using-observe-on-an-async-generator-function--b87414ca

Tier 1 · 70%
trace_name_overwrite

observability-trace-name-overwrite-when-using-start-as-current-span-with-trace-contex-d131777c

Tier 1 · 70%
version_upgrade_bug

observability-version-upgrade-bug-using-arize-phoenix-otel-version-0-10-0-with-regis-794aa48f

Tier 1 · 70%
streaming_cost_tracking

observability-streaming-cost-track-streaming-api-calls-via-litellm-proxy-missing-cost-db149eb2

Tier 1 · 70%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics