Mistral Image OCR With Structured Output

Extract receipt data into a structured schema with Ministral 14B.

The source-fidelity code uses deprecated pixtral-12b-2409. Replace it with ministral-14b-2512 before running the example.

This model configuration sends a JSON Schema request. Agno then parses the returned content against the Pydantic schema; check its type before reading model fields.

image_ocr_with_structured_output.py
"""
Mistral Image Ocr With Structured Output
========================================

Cookbook example for `mistral/image_ocr_with_structured_output.py`.
"""

from typing import List

from agno.agent import Agent
from agno.media import Image
from agno.models.mistral.mistral import MistralChat
from pydantic import BaseModel

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------


class GroceryItem(BaseModel):
    item_name: str
    price: float


class GroceryListElements(BaseModel):
    bill_number: str
    items: List[GroceryItem]
    total_price: float


agent = Agent(
    model=MistralChat(id="pixtral-12b-2409"),
    instructions=[
        "Extract the text elements described by the user from the picture",
    ],
    output_schema=GroceryListElements,
    markdown=True,
)

agent.print_response(
    "From this restaurant bill, extract the bill number, item names and associated prices, and total price and return it as a string in a Json object",
    images=[Image(url="https://i.imghippo.com/files/kgXi81726851246.jpg")],
)

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    pass

output_schema describes the expected type. If parsing or validation fails, result.content can remain a string. Before accessing schema fields in a run result, use isinstance(result.content, YourSchema), replacing YourSchema with the class you passed as output_schema.

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno mistralai

Export your Mistral API key

export MISTRAL_API_KEY="your_mistral_api_key_here"

Update the model

When saving the code, replace pixtral-12b-2409 with ministral-14b-2512.

Run the example

Save the code above as image_ocr_with_structured_output.py, then run:

python image_ocr_with_structured_output.py

Full source: cookbook/90_models/mistral/image_ocr_with_structured_output.py