Vertex AI Image Input Bytes

Send a downloaded image as bytes to Claude on Vertex AI and search the web about it.

The source uses claude-sonnet-4@20250514, which Anthropic lists as deprecated on Google Cloud. Google's Sonnet 4 card does not establish a completed retirement. For a new deployment, choose a current model and its supported settings, such as Sonnet 4.6. Provider availability and lifecycle dates can differ from the direct Anthropic API.

image_input_bytes.py
"""
Vertexai Image Input Bytes
==========================

Cookbook example for `vertexai/claude/image_input_bytes.py`.
"""

from pathlib import Path

from agno.agent import Agent
from agno.media import Image
from agno.models.vertexai.claude import Claude
from agno.tools.websearch import WebSearchTools
from agno.utils.media import download_image

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------

agent = Agent(
    model=Claude(id="claude-sonnet-4@20250514"),
    tools=[WebSearchTools()],
    markdown=True,
)

image_path = Path(__file__).parent.joinpath("sample.jpg")

download_image(
    url="https://upload.wikimedia.org/wikipedia/commons/0/0c/GoldenGateBridge-001.jpg",
    output_path=str(image_path),
)

# Read the image file content as bytes
image_bytes = image_path.read_bytes()

agent.print_response(
    "Tell me about this image and give me the latest news about it.",
    images=[
        Image(content=image_bytes),
    ],
    stream=True,
)

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    pass

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno "anthropic[vertex]" ddgs

Set up Google Cloud access

Use a project with billing and aiplatform.googleapis.com enabled, enable the chosen Claude model in Model Garden, and give your calling identity permission to use it. Follow Google's Claude setup guide. Install the Google Cloud CLI for the local sign-in command below.

Set the project and a location supported by your model. us-east5 is a supported example for Sonnet 4 and Sonnet 4.6; check the model's current availability before switching models or locations.

export ANTHROPIC_VERTEX_PROJECT_ID="your-project-id"
export CLOUD_ML_REGION="us-east5"

Authenticate locally with Application Default Credentials:

gcloud auth application-default login

ADC supplies credentials; it does not enable models or grant IAM permissions. These examples authenticate to Google Cloud without an Anthropic API key.

Run the example

Save the code above as image_input_bytes.py, then run:

python image_input_bytes.py

Full source: cookbook/90_models/vertexai/claude/image_input_bytes.py