Receive and Return WhatsApp Media

Use one multimodal Agent for inbound WhatsApp images, video, audio, and documents. Image and video generation tools return media artifacts that the interface uploads and sends back through the WhatsApp Cloud API.

media.py

"""
Receive and Return WhatsApp Media
=================================

Use one multimodal Agent for inbound WhatsApp images, video, audio, and
documents. Image and video generation tools return media artifacts that the
interface uploads and sends back through the WhatsApp Cloud API.

Prerequisites: GOOGLE_API_KEY, FAL_API_KEY, fal-client, and WhatsApp credentials
Run: .venvs/demo/bin/python cookbook/05_agent_os/19_whatsapp/media.py
Try: Send an image for analysis, or ask for a generated image or short video
"""

from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.models.google import Gemini
from agno.os import AgentOS
from agno.os.interfaces.whatsapp import Whatsapp
from agno.tools.fal import FalTools
from agno.tools.models.gemini import GeminiTools

# ---------------------------------------------------------------------------
# Create the Media WhatsApp AgentOS
# ---------------------------------------------------------------------------

db = SqliteDb(
    id="whatsapp-media-db",
    db_file="tmp/whatsapp_media.db",
)

media_agent = Agent(
    id="whatsapp-media-agent",
    name="WhatsApp Media Agent",
    model=Gemini(id="gemini-3.5-flash"),
    db=db,
    tools=[
        GeminiTools(
            enable_generate_image=True,
            enable_generate_video=False,
        ),
        FalTools(model="fal-ai/hunyuan-video"),
    ],
    add_history_to_context=True,
    num_history_runs=3,
    instructions=[
        "Analyze images, video, audio, and documents that the user sends.",
        "Use generate_image when the user asks for a new still image.",
        "Use generate_media when the user asks for a short generated video.",
        "Keep accompanying text concise because the result is delivered on WhatsApp.",
    ],
)

agent_os = AgentOS(
    id="whatsapp-media-os",
    description="A multimodal AgentOS that receives and returns WhatsApp media.",
    agents=[media_agent],
    interfaces=[
        Whatsapp(
            agent=media_agent,
            media_timeout=60,
        )
    ],
)
app = agent_os.get_app()

# ---------------------------------------------------------------------------
# Run the Media WhatsApp Server
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    agent_os.serve(app=app)

Update the generation tools before running

The retained source uses GeminiTools with imagen-3.0-generate-002, which Google shut down. Remove the GeminiTools import and add this helper before constructing media_agent:

from agno.agent import Agent
from agno.models.google import Gemini
from agno.run.base import RunStatus
from agno.tools.function import ToolResult

async def generate_image(prompt: str) -> ToolResult:
    """Generate a still image with Gemini's native image model."""
    result = await Agent(
        model=Gemini(
            id="gemini-3.1-flash-image",
            response_modalities=["TEXT", "IMAGE"],
        ),
    ).arun(prompt)
    if result.status != RunStatus.completed or not result.images:
        return ToolResult(content="Image generation did not return a completed image.")
    return ToolResult(content="Image generated successfully.", images=result.images)

Replace media_agent's tools argument with:

tools = [generate_image, FalTools(model="fal-ai/hunyuan-video")]

Use that list as tools=tools in the Agent constructor. This helper uses Gemini's native image-generation API and returns an image artifact only after a completed response contains images. A native Gemini model ID cannot be passed to the older Imagen generate_images tool as a drop-in replacement.

For video generation, set both variables shown below: the current FalTools wrapper reads FAL_API_KEY, while the underlying Fal client authenticates with FAL_KEY.

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U "agno[os]" fal-client google-genai

Export environment variables

export FAL_API_KEY="your_fal_api_key_here"
export FAL_KEY="$FAL_API_KEY"
export GOOGLE_API_KEY="your_google_api_key_here"
export WHATSAPP_ACCESS_TOKEN="your_whatsapp_access_token_here"
export WHATSAPP_APP_SECRET="your_whatsapp_app_secret_here"
export WHATSAPP_PHONE_NUMBER_ID="your_whatsapp_phone_number_id_here"
export WHATSAPP_VERIFY_TOKEN="your_whatsapp_verify_token_here"

Expose the server

Start ngrok for port 7777 and copy its public HTTPS URL. Keep ngrok running:

ngrok http 7777

Start AgentOS

Save the code above as media.py, then run:

python media.py

Keep the server running while you configure and verify the webhook.

Configure the webhook

Follow WhatsApp setup. In Meta, set the callback URL to https://<your-ngrok-url>/whatsapp/webhook, use the same verify token as WHATSAPP_VERIFY_TOKEN, and subscribe to the messages field. Verify the webhook while AgentOS and ngrok are running.

Full source: cookbook/05_agent_os/19_whatsapp/media.py