Multimodal I/O
Pass images, audio, video, and files to agents.
Agents can process images, audio, video, and files as input and return generated images or audio. Model support varies by modality.
Setup
Install the SDK for each provider you use:
pip install agno openai google-genai anthropic
export OPENAI_API_KEY="your-openai-api-key"
export GOOGLE_API_KEY="your-google-api-key"
export ANTHROPIC_API_KEY="your-anthropic-api-key"Each tab is a separate example. Set the key for its provider, replace the example.com URLs with accessible media URLs, and supply the local files shown in the example (photo.jpg, recording.wav, clip.mp4, report.pdf, or question.wav). Supported formats and size limits depend on the selected model.
Media Classes
| Class | Content Sources | Common Metadata |
|---|---|---|
Image | url, filepath, content | format, mime_type, detail |
Audio | url, filepath, content | format, mime_type, sample_rate, channels |
Video | url, filepath, content | format, mime_type, duration |
File | url, filepath, content, external | filename, format, mime_type |
Quickstart
Pass images via URL, file path, or raw bytes:
from agno.agent import Agent
from agno.media import Image
from agno.models.openai import OpenAIResponses
agent = Agent(model=OpenAIResponses(id="gpt-5.2"))
# From URL
agent.run(
"What's in this image?",
images=[Image(url="https://example.com/photo.jpg")]
)
# From file
agent.run(
"Describe this image",
images=[Image(filepath="./photo.jpg")]
)
# Multiple images
agent.run(
"Compare these two images",
images=[
Image(url="https://example.com/photo1.jpg"),
Image(url="https://example.com/photo2.jpg")
]
)Pass audio files for transcription or analysis:
from agno.agent import Agent
from agno.media import Audio
from agno.models.openai import OpenAIChat
agent = Agent(
model=OpenAIChat(id="gpt-audio", modalities=["text"])
)
# From file
agent.run(
"What is being said in this audio?",
audio=[Audio(filepath="./recording.wav")]
)
# From bytes
with open("recording.wav", "rb") as f:
audio_bytes = f.read()
agent.run(
"Transcribe this audio",
audio=[Audio(content=audio_bytes, format="wav")]
)Pass video files for analysis:
from agno.agent import Agent
from agno.media import Video
from agno.models.google import Gemini
agent = Agent(model=Gemini(id="gemini-3.5-flash"))
agent.run(
"Describe what happens in this video",
videos=[Video(filepath="./clip.mp4")]
)Agno includes video input adapters for Gemini and AWS Bedrock. Choose a model in those providers that supports video.
Pass documents like PDFs:
from agno.agent import Agent
from agno.media import File
from agno.models.anthropic import Claude
agent = Agent(model=Claude(id="claude-sonnet-4-5"))
# From URL
agent.run(
"Summarize this document",
files=[File(url="https://example.com/report.pdf")]
)
# From file path
agent.run(
"What are the key points in this PDF?",
files=[File(filepath="./report.pdf")]
)Generate images with OpenAITools and GPT Image 2:
import base64
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.openai import OpenAITools
from agno.utils.media import save_base64_data
agent = Agent(
model=OpenAIChat(id="gpt-4o"),
tools=[OpenAITools(image_model="gpt-image-2")],
)
response = agent.run("Generate an image of a sunset over mountains")
if response.images and response.images[0].content:
image_base64 = base64.b64encode(
response.images[0].content
).decode("utf-8")
save_base64_data(
image_base64,
"tmp/sunset.png",
)Generate audio responses:
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.utils.audio import write_audio_to_file
agent = Agent(
model=OpenAIChat(
id="gpt-audio",
modalities=["text", "audio"],
audio={"voice": "alloy", "format": "wav"},
),
)
response = agent.run("Tell me a short story")
# Save audio response
if response.response_audio:
write_audio_to_file(
audio=response.response_audio.content,
filename="story.wav"
)Process audio input and generate audio output:
from agno.agent import Agent
from agno.media import Audio
from agno.models.openai import OpenAIChat
from agno.utils.audio import write_audio_to_file
agent = Agent(
model=OpenAIChat(
id="gpt-audio",
modalities=["text", "audio"],
audio={"voice": "alloy", "format": "wav"},
),
)
response = agent.run(
"Respond to this message",
audio=[Audio(filepath="./question.wav")]
)
if response.response_audio:
write_audio_to_file(
audio=response.response_audio.content,
filename="response.wav"
)Learn More
See Multimodal for more examples.