large language

Voxtral Mini 3B

Voxtral Mini is an enhancement of Ministral 3B. It excels at speech transcription, translation and audio understanding.

Model details

Developed by
Mistral AI
Model family
Voxtral
Use case
large language
Version
2507
Variant
Mini
Size
3B
Hardware
H100 MIG 40GB
API
OpenAI SDK
License
Apache 2.0
Readme
View

Example usage

Voxtral Mini 3B accepts text and audio (formatted using mistral_common) through an OpenAI-compatible API. The following example is adapted for Baseten from Voxtral Mini 3B's model page.

Input
from mistral_common.protocol.instruct.messages import (
    TextChunk,
    AudioChunk,
    UserMessage,
    AssistantMessage,
    RawAudio,
)
from mistral_common.audio import Audio
from huggingface_hub import hf_hub_download

from openai import OpenAI

model_id = "12345678"

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url=f"https://model-{model_id}.api.baseten.co/{deploy_env}/sync/v1"
)

models = client.models.list()
model = models.data[0].id

obama_file = hf_hub_download(
    "patrickvonplaten/audio_samples", "obama.mp3", repo_type="dataset"
)
bcn_file = hf_hub_download(
    "patrickvonplaten/audio_samples", "bcn_weather.mp3", repo_type="dataset"
)

def file_to_chunk(file: str) -> AudioChunk:
    audio = Audio.from_file(file, strict=False)
    return AudioChunk.from_audio(audio)

text_chunk = TextChunk(
    text="Which speaker is more inspiring? Why? How are they different from each other?"
)
user_msg = UserMessage(
    content=[file_to_chunk(obama_file), file_to_chunk(bcn_file), text_chunk]
).to_openai()

response = client.chat.completions.create(
    model=model,
    messages=[user_msg],
    temperature=0.2,
    top_p=0.95,
)
content = response.choices[0].message.content

messages = [
    user_msg,
    AssistantMessage(content=content).to_openai(),
    UserMessage(
        content="Ok, now please summarize the content of the first audio."
    ).to_openai(),
]

response = client.chat.completions.create(
    model=model,
    messages=messages,
    temperature=0.2,
    top_p=0.95,
)
print(response.model_dump_json(indent=4))
JSON output
{
    "id": "chatcmpl-7ab89a7c-a0e9-49e2-9b53-08b0158eabf0",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "logprobs": null,
            "message": {
                "content": "The audio is Barack Obama's farewell address to the nation, delivered in Chicago. Here are the key points:\n\n- **Gratitude**: Obama expresses gratitude to the American people for their support and engagement over the past eight years.\n- **Conversations**: He reflects on the conversations he had with Americans across various settings, which kept him honest, inspired, and motivated.\n- **Experiences**: Obama shares personal experiences, such as seeing neighbors helping each other during the economic crisis, hugging cancer survivors with affordable healthcare, and witnessing the resilience of communities like Joplin and Boston.\n- **Hope and Inspiration**: He highlights the hopeful faces of young graduates, military officers, and students who are changing the world. He also mentions finding grace in a Charleston church and seeing scientists helping paralyzed individuals and wounded warriors.\n- **Citizenship**: Obama emphasizes the importance of citizens' participation in democracy, not just during elections but throughout their lives. He encourages people to engage in civic activities, such as organizing, volunteering, and running for office.\n- **Optimism**: Despite the challenges, Obama remains optimistic about the country's promise and looks forward to working alongside citizens for the future.\n- **Call to Action**: He concludes by encouraging everyone to embrace their role as citizens and to work together to improve the nation.\n\nOverall, the audio is a heartfelt and inspiring speech that emphasizes the power of community, hope, and civic engagement.",
                "refusal": null,
                "role": "assistant",
                "annotations": null,
                "audio": null,
                "function_call": null,
                "tool_calls": [],
                "reasoning_content": null
            },
            "stop_reason": null
        }
    ],
    "created": 1752878046,
    "model": "voxtral-mini",
    "object": "chat.completion",
    "service_tier": null,
    "system_fingerprint": null,
    "usage": {
        "completion_tokens": 291,
        "prompt_tokens": 3213,
        "total_tokens": 3504,
        "completion_tokens_details": null,
        "prompt_tokens_details": null
    },
    "prompt_logprobs": null,
    "kv_transfer_params": null
}