LiteLLM
Call providers through the LiteLLM Python SDK with tools, knowledge, history, structured output and media.
LiteLLM runs the multi-provider Python SDK in your process. For a separate proxy server, use the LiteLLMOpenAI examples.
| Example | Description |
|---|---|
| LiteLLM Audio Input Agent | Pass an MP3 file as audio input to a gpt-audio model routed through LiteLLM. |
| LiteLLM Basic | Run a Hugging Face Mistral model through LiteLLM with sync, async, and streaming calls. |
| LiteLLM Basic GPT | Call OpenAI's gpt-5.6-luna through the LiteLLM model class for a basic agent response. |
| DB | Persist LiteLLM agent sessions in SQLite and answer follow-ups with chat history in context. |
| LiteLLM Image Agent | Analyze an image from a URL with gpt-5.6-luna via LiteLLM and pull related news using web search. |
| LiteLLM Image Agent Bytes | Send an image as raw bytes to gpt-5.6-luna via LiteLLM and fetch related news with web search. |
| LiteLLM Knowledge | Answer questions from a PDF knowledge base stored in PgVector using a LiteLLM agent. |
| LiteLLM Memory | Keep up to three previous runs in context and inspect session messages using an in-memory database. |
| LiteLLM Metrics | Inspect per-message and run-level token metrics from a LiteLLM agent using YFinance tools. |
| LiteLLM PDF Input Bytes | Pass a downloaded PDF as raw bytes to a LiteLLM agent and summarize its contents. |
| LiteLLM PDF Input Local | Attach a local PDF file to a LiteLLM agent and ask questions about a specific recipe. |
| LiteLLM PDF Input URL | Attach a PDF by URL to a LiteLLM agent and ask for a recipe from the document. |
| LiteLLM Reasoning Agent Example | Stream DeepSeek V4 Flash reasoning content through the LiteLLM SDK. |
| Retry | Review retry settings and why invalid model IDs cannot reliably exercise the retry path. |
| LiteLLM Structured Output | Request JSON mode and JSON Schema through LiteLLM, then validate responses with MovieScript. |
| LiteLLM Tool Use | Call YFinance tools from a LiteLLM agent with sync, streaming, and async runs. |
| LiteLLM Append Trailing User Message | Append a trailing user turn so Claude 4.6+ models that reject assistant prefill work through LiteLLM. |