Ollama | Agno

Run large language models with Ollama through a local Ollama server or the direct Ollama Cloud API.

Access mode	Host	Authentication	Example model ID
Local Ollama server	`http://localhost:11434`	None	`llama3.1`
Direct Ollama Cloud API	`https://ollama.com`	`OLLAMA_API_KEY`	`gpt-oss:120b`

Ollama also supports cloud-offloaded models through a signed-in local Ollama server. That access mode uses model IDs such as gpt-oss:120b-cloud. See the Ollama Cloud guide.

Ollama supports multiple open-source models. See the Ollama library.

Experiment with different models to find the best fit for your use case. Here are some general recommendations:

gpt-oss:120b is a general-purpose model available through the direct Ollama Cloud API.
llama3.3 models are good for most basic use cases.
qwen models perform particularly well with tool use.
deepseek-r1 models have strong reasoning capabilities.
phi4 models offer strong performance at a small size.

Local and direct-cloud setup are independent alternatives. Local examples below set their host explicitly. Clear OLLAMA_API_KEY in the local terminal; the native Ollama client also reads that environment variable independently of Agno.

Direct Ollama Cloud API authentication

To call Ollama Cloud directly, set your OLLAMA_API_KEY environment variable. Create a key from Ollama.

export OLLAMA_API_KEY=***

When OLLAMA_API_KEY is set, Agno adds bearer authorization and defaults the host to https://ollama.com. Local requests use the Ollama server configured by the client.

Set up a model

Both local and direct cloud usage require the ollama Python package:

uv pip install -U ollama agno

Local Usage

Install the Ollama app and run a model:

run model

unset OLLAMA_API_KEY
ollama run llama3.1

This starts an interactive session with the model.

To download the model for use in an Agno agent:

pull model

unset OLLAMA_API_KEY
ollama pull llama3.1

Direct Ollama Cloud API

Set OLLAMA_API_KEY to access models through https://ollama.com. This direct API path runs without a local Ollama server.

Examples

Local Usage

Once the model is available locally, use the Ollama model class to access it:

from agno.agent import Agent
from agno.models.ollama import Ollama

agent = Agent(
    model=Ollama(id="llama3.1", host="http://localhost:11434", api_key=None),
    markdown=True
)

# Print the response in the terminal
agent.print_response("Share a 2 sentence horror story.")

Direct Ollama Cloud API

When OLLAMA_API_KEY is set, the host defaults to https://ollama.com. You can omit the host parameter.

from agno.agent import Agent
from agno.models.ollama import Ollama

agent = Agent(
    model=Ollama(id="gpt-oss:120b"),
    markdown=True
)

# Print the response in the terminal
agent.print_response("Share a 2 sentence horror story.")

See Ollama usage examples for more.

Params

Parameter	Type	Default	Description
`id`	`str`	`"llama3.1"`	The name of the Ollama model to use
`name`	`str`	`"Ollama"`	The name of the model
`provider`	`str`	`"Ollama"`	The provider of the model
`format`	`Optional[Any]`	`None`	The format to return the response in (e.g., "json")
`options`	`Optional[Any]`	`None`	Additional model options (temperature, top_p, etc.)
`keep_alive`	`Optional[Union[float, str]]`	`None`	How long to keep the model loaded (e.g., "5m", 3600 seconds)
`request_params`	`Optional[Dict[str, Any]]`	`None`	Additional parameters to include in the request
`host`	`Optional[str]`	`None` (`"http://localhost:11434"`; `"https://ollama.com"` when `api_key` is set)	The host URL for the Ollama server
`timeout`	`Optional[Any]`	`None`	Request timeout in seconds
`api_key`	`Optional[str]`	`getenv("OLLAMA_API_KEY")`	API key for Ollama Cloud. When set, the host defaults to Ollama Cloud
`client_params`	`Optional[Dict[str, Any]]`	`None`	Additional parameters for client configuration
`client`	`Optional[OllamaClient]`	`None`	Pre-configured Ollama client
`async_client`	`Optional[AsyncOllamaClient]`	`None`	Pre-configured async Ollama client

Ollama is a subclass of the Model class and has access to the same params.

Responses API

Ollama v0.13.3+ supports the OpenAI Responses API via the /v1/responses endpoint. Use OllamaResponses for this interface. For this local example, keep Ollama v0.13.3 or later running on port 11434, pull the matching model, and install both client packages:

ollama pull gpt-oss:20b
uv pip install -U openai ollama agno

from agno.agent import Agent
from agno.models.ollama import OllamaResponses

agent = Agent(
    model=OllamaResponses(id="gpt-oss:20b", host="http://localhost:11434", api_key=None),
    markdown=True,
)

agent.print_response("Share a 2 sentence horror story")

The Responses API is stateless. Each request is independent with no previous_response_id chaining.

See OllamaResponses reference for full parameters.

On this page