MLX Transcribe Tools

MLX Transcribe: A tool for transcribing audio files using MLX Whisper.

This recipe targets macOS on Apple silicon, the platform described by MLX Whisper. MLX itself also has Linux backends, but installing one alone does not validate this Whisper workflow on Linux.

mlx_transcribe_tools.py
"""
MLX Transcribe: A tool for transcribing audio files using MLX Whisper

Requirements:
1. ffmpeg - Install using:
   - macOS: `brew install ffmpeg`
   - Ubuntu: `sudo apt-get install ffmpeg`
   - Windows: Download from https://ffmpeg.org/download.html

2. mlx-whisper library:
   uv pip install mlx-whisper

Example Usage:
- Place your audio files in the 'storage/audio' directory
    Eg: download https://www.ted.com/talks/reid_hoffman_and_kevin_scott_the_evolution_of_ai_and_how_it_will_impact_human_creativity
- Run this script to transcribe audio files
- Supports various audio formats (mp3, mp4, wav, etc.)
"""

from pathlib import Path

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.mlx_transcribe import MLXTranscribeTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------


# Get audio files from storage/audio directory
agno_root_dir = Path(__file__).parent.parent.parent.resolve()
audio_storage_dir = agno_root_dir.joinpath("storage/audio")

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    if not audio_storage_dir.exists():
        audio_storage_dir.mkdir(exist_ok=True, parents=True)

    agent = Agent(
        name="Transcription Agent",
        model=OpenAIChat(id="gpt-5.6-luna"),
        tools=[MLXTranscribeTools(base_dir=audio_storage_dir)],
        instructions=[
            "To transcribe an audio file, use the `transcribe` tool with the name of the audio file as the argument.",
            "You can find all available audio files using the `read_files` tool.",
        ],
        markdown=True,
    )

    agent.print_response(
        "Summarize the reid hoffman ted talk, split into sections", stream=True
    )

Add a real local audio file and change the prompt to name it, for example Transcribe talk.mp3 and summarize it in sections. The toolkit does not download the TED webpage as audio. Its default model is mlx-community/whisper-large-v3-turbo; the first transcription needs access to download those model weights. The transcription text is passed to the OpenAI model for the summary.

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno mlx-whisper openai

Export your OpenAI API key

export OPENAI_API_KEY="your_openai_api_key_here"

Install ffmpeg

On macOS, run brew install ffmpeg.

Add an audio file

For a standalone file, replace the agno_root_dir assignment with agno_root_dir = Path(__file__).parent.resolve(). Save the code as mlx_transcribe_tools.py, then add the audio file to storage/audio beside the script.

Run the example

Save the code above as mlx_transcribe_tools.py, then run:

python mlx_transcribe_tools.py

Full source: cookbook/91_tools/mlx_transcribe_tools.py