Data extraction
Extract typed Pydantic objects from text, images, audio, video, and PDFs.
Define the schema, pass the input, and check the parsed result before using it.
output_schema requests structured output and Agno attempts to parse it. A failed or unparseable run can leave content as text. Check RunStatus.completed and the expected Pydantic type before reading fields, indexing, or writing downstream. These examples stop on failure; an application can instead send the original input to a review queue. Schema validation checks the shape and constraints, so verify extracted facts against the source separately.
from agno.run.base import RunStatus
from typing import Optional
from agno.agent import Agent
from agno.models.google import Gemini
from pydantic import BaseModel, Field
class Contact(BaseModel):
name: Optional[str] = Field(None, description="Full name as written")
email: Optional[str] = Field(None, description="Email address")
phone: Optional[str] = Field(None, description="Phone number, raw format")
company: Optional[str] = Field(None, description="Company or organization")
title: Optional[str] = Field(None, description="Job title")
agent = Agent(
model=Gemini(id="gemini-3.5-flash"),
instructions=(
"Extract contact information from the input. Use exactly what the "
"text shows. If a field is missing, leave it null. Do not guess."
),
output_schema=Contact,
)
result_run = agent.run(
"Hi - Sarah Johnson, VP of Marketing at Acme Corp. "
"sarah@acme.com / +1-555-0102."
)
if result_run.status != RunStatus.completed or not isinstance(result_run.content, Contact):
raise RuntimeError("No validated Contact; send the input to review before continuing")
result = result_run.content
# Contact(name='Sarah Johnson', email='sarah@acme.com',
# phone='+1-555-0102', company='Acme Corp.', title='VP of Marketing')Tell the agent how to handle missing fields: "If a field is missing, leave it null. Do not guess."
Nested objects
Lists of sub-objects work the same way. Define the inner model and reference it.
from typing import List, Optional
from pydantic import BaseModel, Field
class ActionItem(BaseModel):
owner: str = Field(..., description="Person responsible")
description: str = Field(..., description="What needs to be done")
due_date: Optional[str] = Field(None, description="Due date if stated")
class Meeting(BaseModel):
action_items: List[ActionItem] = Field(default_factory=list)Pass output_schema=Meeting to request a Meeting; check the run status and result type before accessing action_items.
Per-field confidence
When downstream needs to route uncertain fields to a human, wrap each value in a confidence carrier.
from typing import Literal, Optional
from agno.agent import Agent
from agno.models.google import Gemini
from pydantic import BaseModel, Field
class ConfidentField(BaseModel):
value: Optional[str] = None
confidence: Literal["high", "medium", "low"] = Field(
..., description="Confidence in the extracted value"
)
class Contact(BaseModel):
name: ConfidentField
email: ConfidentField
company: ConfidentField
instructions = """\
Extract contact information from the input. For each field:
- value: what the text shows; null if the field is missing
- confidence: high if explicit and unambiguous;
medium if implied or partially formatted;
low if guessed or ambiguous
Use exactly what the text shows. Do not normalize or paraphrase.
"""
confidence_agent = Agent(
model=Gemini(id="gemini-3.5-flash"),
instructions=instructions,
output_schema=Contact,
)Define each confidence level in the instructions so labels use the same criteria across records.
Any modality, same pattern
The input argument changes per modality. Use a model that supports both that modality and structured output. See Multimodal inputs for the input arguments.
| Input | Argument | Cookbook |
|---|---|---|
| Text | agent.run(text) | text_extraction |
| Image | images=[Image(url=...)] | image_extraction |
| Audio | audio=[Audio(content=...)] | audio_extraction |
| Video | videos=[Video(content=..., format="mp4")] | video_extraction |
files=[File(url=...)] | document_extraction |
Extract then index
The image extraction to vector database cookbook is the minimal pipeline. It describes each image as a typed object, flattens the object into searchable text, embeds it, and stores it in LanceDb.
This recipe needs LanceDb on top of the Google provider:
uv pip install lancedb tantivyThe Image Search application expands the same pattern into an ingest workflow, PgVector hybrid search, AgentOS endpoints, and a browser UI.
Next steps
| Task | Guide |
|---|---|
| Assign labels | Classification |
| Build a searchable image library | Image Search |
| Feed non-text input | Multimodal inputs |
| Add a reviewer and adjudicator | Quality pipeline |