Ollama
Configure the Ollama model class for local Ollama servers or Ollama Cloud.
The Ollama model provides access to open-source models, both locally hosted and via Ollama Cloud.
Local Usage: Run models on your own hardware using the Ollama client. Good for development, privacy-sensitive workloads, and when you want full control over your infrastructure.
Cloud Usage: Access cloud-hosted models via Ollama Cloud with an API key. No local setup required. Set your OLLAMA_API_KEY and start making requests.
Key Features
- Dual Deployment Options: Choose between local hosting for privacy and control, or cloud hosting for scalability
- Switching: Move between local and cloud deployments with minimal code changes
- Auto-configuration: When using an API key, the host automatically defaults to Ollama Cloud
- Wide Model Support: Access to an extensive library of open-source models, including GPT-OSS, Llama, Qwen, DeepSeek, and Phi
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
id | str | "llama3.1" | The name of the Ollama model to use |
name | str | "Ollama" | The name of the model |
provider | str | "Ollama" | The provider of the model |
format | Optional[Any] | None | The format to return the response in (e.g., "json") |
options | Optional[Any] | None | Additional model options (temperature, top_p, etc.) |
keep_alive | Optional[Union[float, str]] | None | How long to keep the model loaded (e.g., "5m", 3600 seconds) |
request_params | Optional[Dict[str, Any]] | None | Additional parameters to include in the request |
host | Optional[str] | None ("http://localhost:11434"; "https://ollama.com" when api_key is set) | The host URL for the Ollama server |
timeout | Optional[Any] | None | Request timeout in seconds |
api_key | Optional[str] | getenv("OLLAMA_API_KEY") | API key for Ollama Cloud. When set, the host defaults to Ollama Cloud |
client_params | Optional[Dict[str, Any]] | None | Additional parameters for client configuration |
client | Optional[OllamaClient] | None | Pre-configured Ollama client |
async_client | Optional[AsyncOllamaClient] | None | Pre-configured async Ollama client |
model_type | ModelType | ModelType.MODEL | Functional role of this model (MODEL, OUTPUT_MODEL, PARSER_MODEL). Set by the agent |
supports_native_structured_outputs | bool | True | Whether the model supports structured outputs natively |
supports_json_schema_outputs | bool | False | Whether the model uses JSON schema for structured outputs |
system_prompt | Optional[str] | None | System prompt from the model added to the agent |
instructions | Optional[List[str]] | None | Instructions from the model added to the agent |
tool_message_role | str | "tool" | Role of tool messages |
assistant_message_role | str | "assistant" | Role of assistant messages |
cache_response | bool | False | Cache model responses to avoid redundant API calls during development |
cache_ttl | Optional[int] | None | Time-to-live for cached responses, in seconds |
cache_dir | Optional[str] | None | Directory for the response cache. Defaults to ~/.agno/cache/model_responses |
retries | int | 0 | Number of retries to attempt before raising a ModelProviderError |
delay_between_retries | int | 1 | Delay between retries, in seconds |
exponential_backoff | bool | False | If True, the delay between retries is doubled each time |
retry_with_guidance | bool | True | Retry a failed model invocation once with a guidance message appended |
retry_with_guidance_limit | int | 1 | Number of times to retry the model invocation with guidance |