[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72021":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":25,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":36,"readmeContent":37,"aiSummary":38,"trendingCount":16,"starSnapshotCount":16,"syncStatus":39,"lastSyncTime":40,"discoverSource":41},72021,"mlx-audio","Blaizzy\u002Fmlx-audio","Blaizzy","A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.","https:\u002F\u002Fblaizzy.github.io\u002Fmlx-audio\u002F",null,"Python",7320,629,52,73,0,97,171,292,291,39.4,"MIT License",false,"main",true,[27,28,29,30,31,32,33,34,35],"apple-silicon","audio-processing","mlx","multimodal","speech-recognition","speech-synthesis","speech-to-text","text-to-speech","transformers","2026-06-12 02:02:57","\n# MLX-Audio\n\n\u003Ca href=\"https:\u002F\u002Ftrendshift.io\u002Frepositories\u002F13625\" target=\"_blank\">\u003Cimg src=\"https:\u002F\u002Ftrendshift.io\u002Fapi\u002Fbadge\u002Frepositories\u002F13625\" alt=\"Blaizzy%2Fmlx-audio | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"\u002F>\u003C\u002Fa>\n\n[![PyPI version](https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fv\u002Fmlx-audio.svg)](https:\u002F\u002Fpypi.org\u002Fproject\u002Fmlx-audio\u002F)\n[![Python](https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fpyversions\u002Fmlx-audio.svg)](https:\u002F\u002Fpypi.org\u002Fproject\u002Fmlx-audio\u002F)\n[![License: MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-blue.svg)](https:\u002F\u002Fopensource.org\u002Flicenses\u002FMIT)\n[![GitHub stars](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FBlaizzy\u002Fmlx-audio.svg?style=social)](https:\u002F\u002Fgithub.com\u002FBlaizzy\u002Fmlx-audio)\n\nThe best audio processing library built on Apple's MLX framework, providing fast and efficient text-to-speech (TTS), speech-to-text (STT), and speech-to-speech (STS) on Apple Silicon.\n\n## Table of Contents\n\n- [Features](#features)\n- [Installation](#installation)\n- [Quick Start](#quick-start)\n- [Supported Models](#supported-models)\n- [Model Examples](#model-examples)\n- [Web Interface \\& API Server](#web-interface--api-server)\n- [Quantization](#quantization)\n- [Swift](#swift)\n- [Requirements](#requirements)\n- [License](#license)\n- [Citation](#citation)\n- [Acknowledgements](#acknowledgements)\n\n## Features\n\n- Fast inference optimized for Apple Silicon (M series chips)\n- Multiple model architectures for TTS, STT, and STS\n- Multilingual support across models\n- Voice customization and cloning capabilities\n- Adjustable speech speed control\n- Interactive web interface with 3D audio visualization\n- OpenAI-compatible REST API\n- Quantization support (3-bit, 4-bit, 6-bit, 8-bit, and more) for optimized performance\n- Swift package for iOS\u002FmacOS integration\n\n## Installation\n\n### Using pip\n```bash\npip install mlx-audio\n```\n\n### Using uv to install only the command line tools\nLatest release from pypi:\n```bash\nuv tool install --force mlx-audio --prerelease=allow\n```\n\nLatest code from github:\n```bash\nuv tool install --force git+https:\u002F\u002Fgithub.com\u002FBlaizzy\u002Fmlx-audio.git --prerelease=allow\n```\n\n### For development or web interface:\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FBlaizzy\u002Fmlx-audio.git\ncd mlx-audio\npip install -e \".[dev]\"\n```\n\n## Quick Start\n\n### Command Line\n\n```bash\n# Basic TTS generation\nmlx_audio.tts.generate --model mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello, world!' --voice Chelsie\n\n# With a different voice and language hint\nmlx_audio.tts.generate --model mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit --text 'Welcome to MLX-Audio!' --voice Ethan --lang_code English\n\n# Play audio immediately\nmlx_audio.tts.generate --model mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --play\n\n# Save to a specific directory\nmlx_audio.tts.generate --model mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --output_path .\u002Fmy_audio\n\n# Stream audio during generation\nmlx_audio.tts.generate --model mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --stream\n\n# Stream audio during generation and save it to disk\nmlx_audio.tts.generate --model mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --stream --save\n\n# Join multiple generated segments into one file\nmlx_audio.tts.generate --model mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit --text $'Hello!\\nHow are you?' --voice Chelsie --join_audio\n```\n\nBy default, when generation yields multiple segments, mlx-audio saves numbered files such as `audio_000.wav` and `audio_001.wav`. Use `--join_audio` to save one combined file instead. When using `--stream`, add `--save` to write the streamed audio to disk.\n\n### Python API\n\n```python\nfrom mlx_audio.tts.utils import load_model\n\n# Load model\nmodel = load_model(\"mlx-community\u002FQwen3-TTS-12Hz-1.7B-Base-8bit\")\n\n# Generate speech\nfor result in model.generate(\n    \"Hello from MLX-Audio!\",\n    voice=\"Chelsie\",\n    lang_code=\"English\",\n):\n    print(f\"Generated {result.audio.shape[0]} samples\")\n    # result.audio contains the waveform as mx.array\n```\n\n## Supported Models\n\n### Text-to-Speech (TTS)\n\n| Model | Description | Languages | Repo |\n|-------|-------------|-----------|------|\n| **Kokoro** | Fast, high-quality multilingual TTS | EN, JA, ZH, FR, ES, IT, PT, HI | [bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FKokoro-82M-bf16), [8bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FKokoro-82M-8bit), [6bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FKokoro-82M-6bit), [4bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FKokoro-82M-4bit) |\n| **Qwen3-TTS** | Alibaba's multilingual TTS with voice design | ZH, EN, JA, KO, + more | [mlx-community\u002FQwen3-TTS-12Hz-1.7B-VoiceDesign-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FQwen3-TTS-12Hz-1.7B-VoiceDesign-bf16) |\n| **CSM** | Conversational Speech Model with voice cloning | EN | [mlx-community\u002Fcsm-1b](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fcsm-1b) |\n| **Dia** | Dialogue-focused TTS | EN | [mlx-community\u002FDia-1.6B-fp16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FDia-1.6B-fp16) |\n| **OuteTTS** | Efficient TTS model | EN | [mlx-community\u002FOuteTTS-1.0-0.6B-fp16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FOuteTTS-1.0-0.6B-fp16) |\n| **Spark** | SparkTTS model | EN, ZH | [mlx-community\u002FSpark-TTS-0.5B-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FSpark-TTS-0.5B-bf16) |\n| **Chatterbox** | Expressive multilingual TTS | EN, ES, FR, DE, IT, PT, PL, TR, RU, NL, CS, AR, ZH, JA, HU, KO | [mlx-community\u002Fchatterbox-fp16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fchatterbox-fp16) |\n| **Soprano** | High-quality TTS | EN | [mlx-community\u002FSoprano-1.1-80M-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FSoprano-1.1-80M-bf16) |\n| **Ming Omni TTS (BailingMM)** | Multimodal generation with voice cloning, style control, and speech\u002Fmusic\u002Fevent generation | EN, ZH | [mlx-community\u002FMing-omni-tts-16.8B-A3B-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FMing-omni-tts-16.8B-A3B-bf16) |\n| **Ming Omni TTS (Dense)** | Lightweight dense Ming Omni variant for voice cloning and style control | EN, ZH | [mlx-community\u002FMing-omni-tts-0.5B-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FMing-omni-tts-0.5B-bf16) |\n| **KugelAudio** | SOTA 7B AR+Diffusion TTS for European languages | EN, DE, FR, ES, IT, PT, NL, PL, RU, UK, + 14 more | [kugelaudio\u002Fkugelaudio-0-open](https:\u002F\u002Fhuggingface.co\u002Fkugelaudio\u002Fkugelaudio-0-open) |\n| **Voxtral TTS** | Mistral's 4B multilingual TTS (20 voices, 9 languages) | EN, FR, ES, DE, IT, PT, NL, AR, HI | [mlx-community\u002FVoxtral-4B-TTS-2603-mlx-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FVoxtral-4B-TTS-2603-mlx-bf16) |\n| **LongCat-AudioDiT** | SOTA diffusion TTS in waveform latent space with voice cloning | ZH, EN | [mlx-community\u002FLongCat-AudioDiT-1B-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FLongCat-AudioDiT-1B-bf16) |\n| **MeloTTS** | Lightweight VITS2-based TTS with streaming | EN (more coming) | [mlx-community\u002FMeloTTS-English-MLX](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FMeloTTS-English-MLX) |\n| **MOSS-TTS** | 8B delay-pattern and 1.7B local-transformer multilingual TTS with voice cloning | 20 languages | [OpenMOSS-Team\u002FMOSS-TTS](https:\u002F\u002Fhuggingface.co\u002FOpenMOSS-Team\u002FMOSS-TTS), [OpenMOSS-Team\u002FMOSS-TTS-Local-Transformer](https:\u002F\u002Fhuggingface.co\u002FOpenMOSS-Team\u002FMOSS-TTS-Local-Transformer) |\n| **MOSS-TTS-Nano** | Tiny multilingual voice-cloning TTS | 20 languages | [mlx-community\u002FMOSS-TTS-Nano-100M](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FMOSS-TTS-Nano-100M) |\n| **Higgs Audio v2** | 3B Llama-backed TTS with real-time voice cloning | EN, ZH, KO, DE, ES | [bf16 (upstream)](https:\u002F\u002Fhuggingface.co\u002Fbosonai\u002Fhiggs-audio-v2-generation-3B-base), [q8](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fhiggs-audio-v2-3B-mlx-q8), [q6](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fhiggs-audio-v2-3B-mlx-q6) |\n\n### Speech-to-Text (STT)\n\n| Model | Description | Languages | Repo |\n|-------|-------------|-----------|------|\n| **Whisper** | OpenAI's robust STT model | 99+ languages | [mlx-community\u002Fwhisper-large-v3-turbo-asr-fp16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fwhisper-large-v3-turbo-asr-fp16) |\n| **Distil-Whisper** | Distilled fast Whisper variants | EN | [distil-whisper\u002Fdistil-large-v3](https:\u002F\u002Fhuggingface.co\u002Fdistil-whisper\u002Fdistil-large-v3) |\n| **Qwen3-ASR** | Alibaba's multilingual ASR | ZH, EN, JA, KO, + more | [mlx-community\u002FQwen3-ASR-1.7B-8bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FQwen3-ASR-1.7B-8bit) |\n| **Qwen3-ForcedAligner** | Word-level audio alignment | ZH, EN, JA, KO, + more | [mlx-community\u002FQwen3-ForcedAligner-0.6B-8bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FQwen3-ForcedAligner-0.6B-8bit) |\n| **Parakeet** | NVIDIA's accurate STT | EN (v2), 25 EU languages (v3) | [mlx-community\u002Fparakeet-tdt-0.6b-v3](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fparakeet-tdt-0.6b-v3) |\n| **Voxtral** | Mistral's speech model | Multiple | [mlx-community\u002FVoxtral-Mini-3B-2507-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FVoxtral-Mini-3B-2507-bf16) |\n| **Voxtral Realtime** | Mistral's 4B streaming STT | Multiple | [4bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FVoxtral-Mini-4B-Realtime-2602-4bit), [fp16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FVoxtral-Mini-4B-Realtime-2602-fp16) |\n| **VibeVoice-ASR** | Microsoft's 9B ASR with diarization & timestamps | Multiple | [mlx-community\u002FVibeVoice-ASR-bf16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FVibeVoice-ASR-bf16) |\n| **Canary** | NVIDIA's multilingual ASR with translation | 25 EU + RU, UK | [README](mlx_audio\u002Fstt\u002Fmodels\u002Fcanary\u002FREADME.md) |\n| **Moonshine** | Useful Sensors' lightweight ASR | EN | [README](mlx_audio\u002Fstt\u002Fmodels\u002Fmoonshine\u002FREADME.md) |\n| **MMS** | Meta's massively multilingual ASR with adapters | 1000+ | [README](mlx_audio\u002Fstt\u002Fmodels\u002Fmms\u002FREADME.md) |\n| **Granite Speech** | IBM's ASR + speech translation | EN, FR, DE, ES, PT, JA | [README](mlx_audio\u002Fstt\u002Fmodels\u002Fgranite_speech\u002FREADME.md) |\n| **Qwen2-Audio** | Alibaba's multimodal audio understanding (ASR, captioning, emotion, translation) | Multiple | [mlx-community\u002FQwen2-Audio-7B-Instruct-4bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FQwen2-Audio-7B-Instruct-4bit) |\n\n\n### Voice Activity Detection \u002F Speaker Diarization (VAD)\n\n| Model | Description | Languages | Repo |\n|-------|-------------|-----------|------|\n| **Silero VAD** | Lightweight speech\u002Fnon-speech detection with streaming state | Language-agnostic | [mlx-community\u002Fsilero-vad](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fsilero-vad) |\n| **Sortformer v1** | NVIDIA's end-to-end speaker diarization (up to 4 speakers) | Language-agnostic | [mlx-community\u002Fdiar_sortformer_4spk-v1-fp32](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fdiar_sortformer_4spk-v1-fp32) |\n| **Sortformer v2.1** | NVIDIA's streaming speaker diarization with AOSC compression | Language-agnostic | [mlx-community\u002Fdiar_streaming_sortformer_4spk-v2.1-fp32](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fdiar_streaming_sortformer_4spk-v2.1-fp32) |\n\nSee the model READMEs for API details, streaming examples, and conversion steps.\n\n### Speech-to-Speech (STS)\n\n| Model | Description | Use Case | Repo |\n|-------|-------------|----------|------|\n| **SAM-Audio** | Text-guided source separation | Extract specific sounds | [mlx-community\u002Fsam-audio-large](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002Fsam-audio-large) |\n| **Liquid2.5-Audio*** | Speech-to-Speech, Text-to-Speech and Speech-to-Text | Speech interactions | [mlx-community\u002FLFM2.5-Audio-1.5B-8bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FLFM2.5-Audio-1.5B-8bit) |\n| **MossFormer2 SE** | Speech enhancement | Noise removal | [starkdmi\u002FMossFormer2_SE_48K_MLX](https:\u002F\u002Fhuggingface.co\u002Fstarkdmi\u002FMossFormer2_SE_48K_MLX) |\n| **DeepFilterNet (1\u002F2\u002F3)** | Speech enhancement | Noise suppression | [mlx-community\u002FDeepFilterNet-mlx](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FDeepFilterNet-mlx) |\n\n## Model Examples\n\n### Qwen3-TTS\n\nAlibaba's state-of-the-art multilingual TTS with voice cloning, emotion control, and voice design capabilities.\n\n```python\nfrom mlx_audio.tts.utils import load_model\n\nmodel = load_model(\"mlx-community\u002FQwen3-TTS-12Hz-0.6B-Base-bf16\")\nresults = list(model.generate(\n    text=\"Hello, welcome to MLX-Audio!\",\n    voice=\"Chelsie\",\n    language=\"English\",\n))\n\naudio = results[0].audio  # mx.array\n```\n\nSee the [Qwen3-TTS README](mlx_audio\u002Ftts\u002Fmodels\u002Fqwen3_tts\u002FREADME.md) for voice cloning, CustomVoice, VoiceDesign, and all available models.\n\n### Ming Omni TTS (BailingMM)\n\n```bash\nmlx_audio.tts.generate \\\n    --model mlx-community\u002FMing-omni-tts-16.8B-A3B-bf16 \\\n    --prompt \"Please generate speech based on the following description.\\n\" \\\n    --text \"This is a quick Ming Omni test.\" \\\n    --lang_code en \\\n    --output_path audio_io \\\n    --file_prefix ming_basic \\\n    --verbose\n```\n\nSee the [Ming Omni TTS README](mlx_audio\u002Ftts\u002Fmodels\u002Fbailingmm\u002FREADME.md) for CLI and Python cookbook examples, and the [Ming Omni Dense README](mlx_audio\u002Ftts\u002Fmodels\u002Fdense\u002FREADME.md) for the `mlx-community\u002FMing-omni-tts-0.5B-bf16` workflow.\n\n### Kokoro TTS\n\nKokoro is a fast, multilingual TTS model with 54 voice presets.\n\n```python\nfrom mlx_audio.tts.utils import load_model\n\nmodel = load_model(\"mlx-community\u002FKokoro-82M-bf16\")\n# Or use a quantized variant for lower memory usage:\n# model = load_model(\"mlx-community\u002FKokoro-82M-8bit\")\n# model = load_model(\"mlx-community\u002FKokoro-82M-4bit\")\n\n# Generate with different voices\nfor result in model.generate(\n    text=\"Welcome to MLX-Audio!\",\n    voice=\"af_heart\",  # American female\n    speed=1.0,\n    lang_code=\"a\"  # American English\n):\n    audio = result.audio\n```\n\n**Available Voices:**\n- American English: `af_heart`, `af_bella`, `af_nova`, `af_sky`, `am_adam`, `am_echo`, etc.\n- British English: `bf_alice`, `bf_emma`, `bm_daniel`, `bm_george`, etc.\n- Japanese: `jf_alpha`, `jm_kumo`, etc.\n- Chinese: `zf_xiaobei`, `zm_yunxi`, etc.\n\nKokoro requires `pip install misaki` for text processing. Japanese and Mandarin may additionally require `pip install misaki[ja]` or `pip install misaki[zh]`.\n\n**Language Codes:**\n| Code | Language | Note |\n|------|----------|------|\n| `a` | American English | Default; requires `pip install misaki` |\n| `b` | British English | Requires `pip install misaki` |\n| `j` | Japanese | Requires `pip install misaki[ja]` |\n| `z` | Mandarin Chinese | Requires `pip install misaki[zh]` |\n| `e` | Spanish | Requires `pip install misaki` |\n| `f` | French | Requires `pip install misaki` |\n\n### CSM (Voice Cloning)\n\nClone any voice using a reference audio sample:\n\n```bash\nmlx_audio.tts.generate \\\n    --model mlx-community\u002Fcsm-1b \\\n    --text \"Hello from Sesame.\" \\\n    --ref_audio .\u002Freference_voice.wav \\\n    --play\n```\n\n### Whisper STT\n\n```python\nfrom mlx_audio.stt.generate import generate_transcription\n\nresult = generate_transcription(\n    model=\"mlx-community\u002Fwhisper-large-v3-turbo-asr-fp16\",\n    audio=\"audio.wav\",\n)\nprint(result.text)\n```\n\n### Qwen3-ASR & ForcedAligner\n\nAlibaba's multilingual speech models for transcription and word-level alignment.\n\n```python\nfrom mlx_audio.stt import load\n\n# Speech recognition\nmodel = load(\"mlx-community\u002FQwen3-ASR-0.6B-8bit\")\nresult = model.generate(\"audio.wav\", language=\"English\")\nprint(result.text)\n\n# Word-level forced alignment\naligner = load(\"mlx-community\u002FQwen3-ForcedAligner-0.6B-8bit\")\nresult = aligner.generate(\"audio.wav\", text=\"I have a dream\", language=\"English\")\nfor item in result:\n    print(f\"[{item.start_time:.2f}s - {item.end_time:.2f}s] {item.text}\")\n```\n\nSee the [Qwen3-ASR README](mlx_audio\u002Fstt\u002Fmodels\u002Fqwen3_asr\u002FREADME.md) for CLI usage, all models, and more examples.\n\n### VibeVoice-ASR\n\nMicrosoft's 9B parameter speech-to-text model with speaker diarization and timestamps. Supports long-form audio (up to 60 minutes) and outputs structured JSON.\n\n```python\nfrom mlx_audio.stt.utils import load\n\nmodel = load(\"mlx-community\u002FVibeVoice-ASR-bf16\")\n\n# Basic transcription\nresult = model.generate(audio=\"meeting.wav\", max_tokens=8192, temperature=0.0)\nprint(result.text)\n# [{\"Start\":0,\"End\":5.2,\"Speaker\":0,\"Content\":\"Hello everyone, let's begin.\"},\n#  {\"Start\":5.5,\"End\":9.8,\"Speaker\":1,\"Content\":\"Thanks for joining today.\"}]\n\n# Access parsed segments\nfor seg in result.segments:\n    print(f\"[{seg['start_time']:.1f}-{seg['end_time']:.1f}] Speaker {seg['speaker_id']}: {seg['text']}\")\n```\n\n**Streaming transcription:**\n\n```python\n# Stream tokens as they are generated\nfor text in model.stream_transcribe(audio=\"speech.wav\", max_tokens=4096):\n    print(text, end=\"\", flush=True)\n```\n\n**With context (hotwords\u002Fmetadata):**\n\n```python\nresult = model.generate(\n    audio=\"technical_talk.wav\",\n    context=\"MLX, Apple Silicon, PyTorch, Transformer\",\n    max_tokens=8192,\n    temperature=0.0,\n)\n```\n\n**CLI usage:**\n\n```bash\n# Basic transcription\npython -m mlx_audio.stt.generate \\\n    --model mlx-community\u002FVibeVoice-ASR-bf16 \\\n    --audio meeting.wav \\\n    --output-path output \\\n    --format json \\\n    --max-tokens 8192 \\\n    --verbose\n\n# With context\u002Fhotwords\npython -m mlx_audio.stt.generate \\\n    --model mlx-community\u002FVibeVoice-ASR-bf16 \\\n    --audio technical_talk.wav \\\n    --output-path output \\\n    --format json \\\n    --max-tokens 8192 \\\n    --context \"MLX, Apple Silicon, PyTorch, Transformer\" \\\n    --verbose\n```\n\n### Parakeet (Multilingual STT)\n\nNVIDIA's high-accuracy speech-to-text model. Parakeet v3 supports 25 European languages.\n\n```python\nfrom mlx_audio.stt.utils import load\n\n# Load the multilingual v3 model\nmodel = load(\"mlx-community\u002Fparakeet-tdt-0.6b-v3\")\n\n# Transcribe audio\nresult = model.generate(\"audio.wav\")\nprint(f\"Text: {result.text}\")\n\n# Access word-level timestamps\nfor sentence in result.sentences:\n    print(f\"[{sentence.start:.2f}s - {sentence.end:.2f}s] {sentence.text}\")\n```\n\n**Streaming transcription:**\n\n```python\nfor chunk in model.generate(\"long_audio.wav\", stream=True):\n    print(chunk.text, end=\"\", flush=True)\n```\n\n**Supported languages (v3):**\nBulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Russian, Ukrainian\n\n**CLI usage:**\n\n```bash\npython -m mlx_audio.stt.generate \\\n    --model mlx-community\u002Fparakeet-tdt-0.6b-v3 \\\n    --audio speech.wav \\\n    --output-path output \\\n    --format json \\\n    --verbose\n```\n\n### KugelAudio\n\nSOTA open-source 7B TTS model for 24 European languages, based on Microsoft VibeVoice.\nUses a hybrid AR + Diffusion architecture (Qwen2.5 LM + SDE-DPM-Solver++ diffusion head + VAE decoder).\n\n```python\nfrom mlx_audio.tts.utils import load_model\n\nmodel = load_model(\"kugelaudio\u002Fkugelaudio-0-open\")\n\nfor result in model.generate(\n    text=\"Hello, welcome to MLX-Audio!\",\n    cfg_scale=3.0,       # Classifier-free guidance (1.0=fast, 3.0=quality)\n    ddpm_steps=10,       # Diffusion steps (5=fast, 10=balanced, 20=max quality)\n):\n    audio = result.audio  # mx.array, 24kHz\n```\n\nThe model loads directly from HuggingFace (weights are remapped automatically via `sanitize()`).\nTo quantize or save in a pre-converted format:\n\n```bash\npython -m mlx_audio.convert \\\n    --hf-path kugelaudio\u002Fkugelaudio-0-open \\\n    --mlx-path .\u002Fkugelaudio-0-open-bf16 \\\n    --dtype bfloat16\n```\n\n**Supported languages (24):** English, German, French, Spanish, Italian, Portuguese, Dutch, Polish, Russian, Ukrainian, Czech, Romanian, Hungarian, Swedish, Danish, Finnish, Norwegian, Greek, Bulgarian, Slovak, Croatian, Serbian, Turkish\n\n> **Note:** Requires ~17GB memory (7B params in bfloat16).\n> Pre-encoded voice presets (voice cloning) are not yet available in the upstream model — the model generates speech with a default voice.\n\n### LongCat-AudioDiT\n\nSOTA diffusion-based TTS operating in the waveform latent space. Uses Conditional Flow Matching with a DiT backbone and WAV-VAE codec at 24kHz. Supports zero-shot voice cloning.\n\n```python\nfrom mlx_audio.tts.utils import load\n\nmodel = load(\"mlx-community\u002FLongCat-AudioDiT-1B-bf16\")\n\n# Zero-shot TTS\nresult = next(model.generate(\"Hello, this is a test of AudioDiT.\"))\naudio = result.audio  # mx.array, 24kHz\n\n# Voice cloning (use \"apg\" guidance for best similarity)\nresult = next(model.generate(\n    text=\"Today is warm turning to rain.\",\n    ref_audio=\"reference.wav\",\n    ref_text=\"Transcript of the reference audio.\",\n    guidance_method=\"apg\",\n    cfg_strength=4.0,\n    steps=16,\n))\n```\n\nSee the [LongCat-AudioDiT README](mlx_audio\u002Ftts\u002Fmodels\u002Flongcat_audiodit\u002FREADME.md) for all parameters and CLI usage.\n\n### Voxtral TTS\n\nMistral's 4B multilingual text-to-speech with 20 voice presets across 9 languages.\n\n```python\nfrom mlx_audio.tts.utils import load\n\nmodel = load(\"mlx-community\u002FVoxtral-4B-TTS-2603-mlx-bf16\")\n\nfor result in model.generate(text=\"Hello, how are you today?\", voice=\"casual_male\"):\n    print(result.audio_duration)\n```\n\nVoices: `casual_male`, `casual_female`, `cheerful_female`, `neutral_male`, `neutral_female`, `fr_male`, `fr_female`, `es_male`, `es_female`, `de_male`, `de_female`, `it_male`, `it_female`, `pt_male`, `pt_female`, `nl_male`, `nl_female`, `ar_male`, `hi_male`, `hi_female`\n\n### Voxtral Realtime\n\nMistral's 4B parameter streaming speech-to-text model, optimized for low-latency transcription.\n\nAvailable variants: [4bit](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FVoxtral-Mini-4B-Realtime-2602-4bit) (smaller\u002Ffaster) | [fp16](https:\u002F\u002Fhuggingface.co\u002Fmlx-community\u002FVoxtral-Mini-4B-Realtime-2602-fp16) (full precision)\n\n```python\nfrom mlx_audio.stt.utils import load\n\n# Use 4bit for faster inference, fp16 for full precision\nmodel = load(\"mlx-community\u002FVoxtral-Mini-4B-Realtime-2602-4bit\")\n\n# Transcribe audio\nresult = model.generate(\"audio.wav\")\nprint(result.text)\n\n# Streaming transcription\nfor chunk in model.generate(\"audio.wav\", stream=True):\n    print(chunk, end=\"\", flush=True)\n\n# Adjust transcription delay (lower = faster but less accurate)\nresult = model.generate(\"audio.wav\", transcription_delay_ms=240)\n```\n\n### MedASR (Medical Transcription)\n\nSpecialized model for medical terms and dictation.\n\n```python\nfrom mlx_audio.stt.utils import load, transcribe\n\nmodel = load(\"mlx-community\u002Fmedasr\")\nresult = transcribe(\"medical_dictation.wav\", model=model)\nprint(result[\"text\"])\n```\n\n**Live Transcription Example:**\n```bash\n# Continuous live transcription with VAD\npython examples\u002Fmedasr_live.py\n```\n\n### SAM-Audio (Source Separation)\n\nSeparate specific sounds from audio using text prompts:\n\n```python\nfrom mlx_audio.sts import SAMAudio, SAMAudioProcessor, save_audio\n\nmodel = SAMAudio.from_pretrained(\"mlx-community\u002Fsam-audio-large\")\nprocessor = SAMAudioProcessor.from_pretrained(\"mlx-community\u002Fsam-audio-large\")\n\nbatch = processor(\n    descriptions=[\"A person speaking\"],\n    audios=[\"mixed_audio.wav\"],\n)\n\nresult = model.separate_long(\n    batch.audios,\n    descriptions=batch.descriptions,\n    anchors=batch.anchor_ids,\n    chunk_seconds=10.0,\n    overlap_seconds=3.0,\n    ode_opt={\"method\": \"midpoint\", \"step_size\": 2\u002F32},\n)\n\nsave_audio(result.target[0], \"voice.wav\")\nsave_audio(result.residual[0], \"background.wav\")\n```\n\n### MossFormer2 (Speech Enhancement)\n\nRemove noise from speech recordings:\n\n```python\nfrom mlx_audio.sts import MossFormer2SEModel, save_audio\n\nmodel = MossFormer2SEModel.from_pretrained(\"starkdmi\u002FMossFormer2_SE_48K_MLX\")\nenhanced = model.enhance(\"noisy_speech.wav\")\nsave_audio(enhanced, \"clean.wav\", 48000)\n```\n\n## Web Interface & API Server\n\nMLX-Audio includes a modern web interface and OpenAI-compatible API.\n\n### Starting the Server\n\n```bash\n# Start API server\nmlx_audio.server --host 0.0.0.0 --port 8000\n\n# Start web UI (in another terminal)\ncd mlx_audio\u002Fui\nnpm install && npm run dev\n```\n\n### API Endpoints\n\n**Text-to-Speech** (OpenAI-compatible):\n```bash\ncurl -X POST http:\u002F\u002Flocalhost:8000\u002Fv1\u002Faudio\u002Fspeech \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d '{\"model\": \"mlx-community\u002FKokoro-82M-bf16\", \"input\": \"Hello!\", \"voice\": \"af_heart\"}' \\\n  --output speech.wav\n```\n\n**Speech-to-Text**:\n```bash\ncurl -X POST http:\u002F\u002Flocalhost:8000\u002Fv1\u002Faudio\u002Ftranscriptions \\\n  -F \"file=@audio.wav\" \\\n  -F \"model=mlx-community\u002Fwhisper-large-v3-turbo-asr-fp16\"\n```\n\n## Quantization\n\nReduce model size and improve performance with quantization using the convert script:\n\n```bash\n# Convert and quantize to 4-bit\npython -m mlx_audio.convert \\\n    --hf-path prince-canuma\u002FKokoro-82M \\\n    --mlx-path .\u002FKokoro-82M-4bit \\\n    --quantize \\\n    --q-bits 4 \\\n    --upload-repo username\u002FKokoro-82M-4bit (optional: if you want to upload the model to Hugging Face)\n\n# Convert with MXFP4 quantization\npython -m mlx_audio.convert \\\n    --hf-path prince-canuma\u002FKokoro-82M \\\n    --mlx-path .\u002FKokoro-82M-mxfp4 \\\n    --quantize \\\n    --q-mode mxfp4\n\n# Convert with specific dtype (bfloat16)\npython -m mlx_audio.convert \\\n    --hf-path prince-canuma\u002FKokoro-82M \\\n    --mlx-path .\u002FKokoro-82M-bf16 \\\n    --dtype bfloat16 \\\n    --upload-repo username\u002FKokoro-82M-bf16 (optional: if you want to upload the model to Hugging Face)\n```\n\n**Options:**\n| Flag | Description |\n|------|-------------|\n| `--hf-path` | Source Hugging Face model or local path |\n| `--mlx-path` | Output directory for converted model |\n| `-q, --quantize` | Enable quantization |\n| `--q-bits` | Bits per weight (optional, defaults depend on `--q-mode`) |\n| `--q-group-size` | Group size for quantization (optional, defaults depend on `--q-mode`) |\n| `--q-mode` | Quantization mode: `affine`, `mxfp4`, `mxfp8`, `nvfp4` |\n| `--dtype` | Weight dtype: `float16`, `bfloat16`, `float32` |\n| `--upload-repo` | Upload converted model to HF Hub |\n\n## Swift\n\nLooking for Swift\u002FiOS support? Check out [mlx-audio-swift](https:\u002F\u002Fgithub.com\u002FBlaizzy\u002Fmlx-audio-swift) for on-device TTS using MLX on macOS and iOS.\n\n## Requirements\n\n- Python 3.10+\n- Apple Silicon Mac (M1\u002FM2\u002FM3\u002FM4)\n- MLX framework\n- **ffmpeg** (required for MP3\u002FFLAC\u002FOGG\u002FOpus\u002FVorbis audio encoding)\n\n### Installing ffmpeg\n\nffmpeg is required for saving audio in MP3, FLAC, OGG, Opus, or Vorbis format. Install it using:\n\n```bash\n# macOS (using Homebrew)\nbrew install ffmpeg\n\n# Ubuntu\u002FDebian\nsudo apt install ffmpeg\n```\n\nWAV format works without ffmpeg.\n\n## License\n\n[MIT License](LICENSE)\n\n## Citation\n\n```bibtex\n@misc{mlx-audio,\n  author = {Canuma, Prince},\n  title = {MLX Audio},\n  year = {2025},\n  howpublished = {\\url{https:\u002F\u002Fgithub.com\u002FBlaizzy\u002Fmlx-audio}},\n  note = {Audio processing library for Apple Silicon with TTS, STT, and STS capabilities.}\n}\n```\n\n## Acknowledgements\n\n- [Apple MLX Team](https:\u002F\u002Fgithub.com\u002Fml-explore\u002Fmlx) for the MLX framework\n","MLX-Audio 是一个基于 Apple 的 MLX 框架构建的文本转语音 (TTS)、语音转文本 (STT) 和语音转语音 (STS) 库，专为 Apple Silicon 设计以提供高效的语音处理。该项目的核心功能包括针对 M 系列芯片优化的快速推理、支持多种模型架构和多语言、自定义和克隆声音的能力、可调节的语速控制以及 3D 音频可视化等。此外，它还提供了与 OpenAI 兼容的 REST API 和 Swift 包，便于 iOS\u002FmacOS 应用集成。适用于需要高效语音处理的应用场景，如智能助手、语音识别系统或任何涉及语音交互的服务。",2,"2026-06-11 03:39:59","high_star"]