Fallback Models: Mid-Run Failure

Trigger a fallback when the primary model fails mid-run, after a tool call has already executed.

Tests what happens when the primary model fails AFTER a tool call (and or within a run).

mid_run_fallback.py
"""
Fallback Models — Mid-Run Failure
====================================

Tests what happens when the primary model fails AFTER a tool call (and or within a run).

Flow:
  1. gpt-5.6-luna receives the request and makes a tool call
  2. The tool mutates the model instance's id to something invalid
  3. The next API call inside Model.response()'s while-loop fails
  4. The error bubbles up to call_model_with_fallback
  5. Fallback (Claude) is tried
"""

from agno.agent import Agent
from agno.models.anthropic import Claude
from agno.models.openai import OpenAIChat


def break_model(agent: Agent) -> str:
    """Tool that corrupts the executing model mid-run."""
    # Point the model at an unreachable server and clear the cached client
    # so the next API call in the response() loop fails with a connection error.
    agent.model.base_url = "http://localhost:1/v1"  # type: ignore[union-attr]
    agent.model.client = None  # type: ignore[union-attr]
    return "Tool executed successfully."


agent = Agent(
    model=OpenAIChat(id="gpt-5.6-luna"),
    tools=[break_model],
    fallback_models=[Claude(id="claude-sonnet-4-20250514")],
)

if __name__ == "__main__":
    agent.print_response("Call the break_model tool", stream=True)

Use a current fallback model

Before running, replace every Claude(id="claude-sonnet-4-20250514") with Claude(id="claude-sonnet-4-6"). The original Sonnet 4 API model retired on June 15, 2026; see Claude model deprecations. The source is preserved above, and the setup still requires both providers' API keys.

The source docstring describes changing the model ID, but break_model actually changes base_url to an unreachable local address and clears the cached client. If the model selects this tool, the next provider request fails and can trigger fallback. The completed tool action is not rolled back by fallback.

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno anthropic openai

Export your API keys

export ANTHROPIC_API_KEY="your_anthropic_api_key_here"
export OPENAI_API_KEY="your_openai_api_key_here"

Run the example

Save the code above as mid_run_fallback.py, then run:

python mid_run_fallback.py

Full source: cookbook/02_agents/17_fallback_models/03_mid_run_fallback.py