Response Caching

Cache model responses to reduce API calls and costs.

For a conceptual overview of response caching, see Response Caching.

Response caching stores model responses locally, reducing response times and API costs during development and testing.

Basic Usage

Enable caching by setting cache_response=True when initializing the model. This example starts with a temporary, empty cache directory. After a successful first request, the identical second request can reuse its response. Keep the model settings fixed: generation settings such as temperature are not part of the cache key.

cache_model_response.py
from tempfile import TemporaryDirectory
from time import perf_counter

from agno.agent import Agent
from agno.models.openai import OpenAIChat

with TemporaryDirectory() as cache_dir:
    agent = Agent(
        model=OpenAIChat(id="gpt-4o", cache_response=True, cache_dir=cache_dir)
    )
    for i in range(1, 3):
        start = perf_counter()
        response = agent.run(
            "Write me a short story about a cat that can talk and solve problems."
        )
        print(f"Run {i}")
        print(response.content)
        print(f"Elapsed time: {perf_counter() - start:.3f}s")

Usage

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Set your API key

export OPENAI_API_KEY=xxx

Install dependencies

uv pip install -U openai agno

Run Agent

Save the code above as cache_model_response.py, then run:

python cache_model_response.py