[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-93280":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":15,"stars30d":15,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":16,"rankGlobal":9,"rankLanguage":9,"license":17,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":20,"hasPages":18,"topics":21,"createdAt":9,"pushedAt":9,"updatedAt":37,"readmeContent":38,"aiSummary":39,"trendingCount":14,"starSnapshotCount":14,"syncStatus":13,"lastSyncTime":40,"discoverSource":41},93280,"audio.cpp-webui","kigner\u002Faudio.cpp-webui","kigner","audio.cpp with a full-task WebUI - pure C++ audio-model inference engine powered by ggml. TTS, ASR\u002FSTT, VAD, voice conversion, speaker diarization, music generation. No Python dependency.",null,"C++",173,45,2,0,54,70.39,"Other",false,"release-0.1",true,[22,23,24,25,26,27,28,29,30,31,32,33,34,35,36],"asr","audio","cpp","ggml","inference-engine","music-generation","speaker-diarization","speech-recognition","speech-synthesis","speech-to-text","text-to-speech","tts","vad","voice-conversion","webui","2026-07-22 04:02:08","# audio.cpp\n\n> [!NOTE]\n> This repository is a downstream distribution of [**0xShug0\u002Faudio.cpp**](https:\u002F\u002Fgithub.com\u002F0xShug0\u002Faudio.cpp) (Apache-2.0, Copyright ShugoAI LLC), extended with a full-task WebUI and Windows-friendly local launcher scripts, and periodically merged with upstream. All credit for the core inference framework goes to the upstream project — please star and contribute there.\n\n`audio.cpp` is a high-performance C++ audio inference framework built on top of `ggml`, designed to make modern local audio models practical, portable, and fast.\n\nTired of juggling a dozen Conda environments, hundreds of Python packages, and dependency conflicts just to try a few audio models? audio.cpp gives those paths a shared native runtime instead.\n\n> [!IMPORTANT]\n> **CUDA performance headline:** multiple TTS paths already run **1.8x-5.0x faster than their Python reference paths** while cutting end-to-end latency by **45%-80%**.\n>\n> **VibeVoice 1.5B:** generates a **93.9-minute podcast in 18.2 minutes** with **10 diffusion steps** and without quantization, running about **5.15x faster than real time**.\n>\n> **Supertonic 3:** generates about **10 hours of audio in 3 minutes** on RTX5090. Up to 200x+ real-time on CUDA, 6x+ real-time on CPU, and 47 ms TTFT in CUDA streaming mode.\n> [Demo: 10 hours of audio generated in 3 minutes](https:\u002F\u002Fwww.reddit.com\u002Fr\u002FLocalLLaMA\u002Fcomments\u002F1uwpvt9\u002Faudiocpp_10_hours_of_audio_generated_in_3_minutes\u002F).\n>\n> **Real-world ASR win:** In [TranscrIA benchmark](https:\u002F\u002Fgithub.com\u002FMartossien\u002Ftranscria\u002Fblob\u002Fmain\u002Fdocs\u002FSTT_BENCHMARK_REAL_MEETINGS.md) on messy French meeting audio, audio.cpp’s Nemotron 3.5 ASR matched the same WER as other implementations while using about **1\u002F4 of the wall time**. \n\nIt is built for real end-to-end execution rather than one-off model demos: the same runtime powers TTS, voice cloning, voice conversion, ASR, diarization, VAD, source separation, alignment, codec-style models, and higher-level workflows through a common framework surface.\n\nHighlights:\n\n- **Parity.** Strong parity tooling against Python reference paths.\n- **Performance.** Performance-focused execution, reusable sessions, and batch-style offline inference. **Optimized for CUDA**.\n- **Portability.** A portable native stack centered on `ggml`, with CLI and server entry points instead of Python-only deployment paths.\n- **Pipelines.** Experimental JSON pipeline support for higher-level multi-step workflows.\n- **Audio Utilities.** Built-in denoise, enhancement, resampling, and STFT\u002FISTFT utilities for real production-style task paths.\n\n\u003Cp>\u003Cstrong>\u003Cspan style=\"font-size:1.1em;\">The goal of the framework is to provide highly optimized, reusable building blocks for audio-related models, so new model integrations can be brought up faster, shared components can be improved once and benefit many families, and real end-to-end inference paths can stay efficient, maintainable, and portable.\u003C\u002Fspan>\u003C\u002Fstrong>\u003C\u002Fp>\n\naudio.cpp would not be moving this quickly without generous contributors bringing in real fixes, new capabilities, and careful polish. See [CONTRIBUTING.md](CONTRIBUTING.md) for how to contribute and for a shout-out to the people already helping shape the project.\n\n> [!TIP]\n> **Contribution focus:** the most helpful contributions right now are improvements to the UI, API server, and pipeline\u002Fworkflow subsystems. These areas make the existing model surface easier to use, serve, compose, and validate. See [CONTRIBUTING.md](CONTRIBUTING.md) for more details.\n>\n> **New model PRs:** before starting a new model port, **please check the supported model table because several families are already implemented or under testing**. If you do add a model, follow the validation style in [PR #19](https:\u002F\u002Fgithub.com\u002F0xShug0\u002Faudio.cpp\u002Fpull\u002F19): include exact build\u002Frun commands, model paths or package ids, generated outputs, parity or path-test results, and relevant performance or memory notes.\n\n## WebUI and Local Helper Scripts\n\nThe `webui\u002F` directory contains a Windows-friendly local launcher layer for audio.cpp:\n\n- `run_webui.bat` starts the Gradio WebUI at `http:\u002F\u002F127.0.0.1:7860`, with on-demand model loading, model switching, model download support, reference voice upload\u002Frecording, generated controls for advanced model parameters, and inline run\u002Ferror status.\n- `run_realtime.bat` starts the realtime voice chat page backed by local ASR, LLM, and audio.cpp TTS services.\n- `run_server.bat` starts an OpenAI-compatible HTTP API server for other applications; `run_server_asr.bat` is its preset wrapper serving Qwen3-ASR on port 8081.\n- `run_cli_tts.bat` covers quick one-shot TTS; long-text synthesis is built into the WebUI (automatic text chunking and audio concatenation).\n\nFor full setup notes, script arguments, environment variables, API examples, WebUI advanced-parameter behavior, common model ids, and troubleshooting, see [webui\u002FREADME.md](webui\u002FREADME.md).\n\n📺 **Video tutorial:** for a hands-on walkthrough of setting up and using this repo, watch the [video tutorial on YouTube](https:\u002F\u002Fyoutu.be\u002FbDznyUOAzuc).\n\n## News\n\n> [!IMPORTANT]\n> **2026-07-14 - Release 0.3:** This release expands audio.cpp with five new TTS families: IndexTTS2, Irodori-TTS, MOSS-TTS-Nano, MOSS-TTS-Local (thanks to [@justinjohn0306](https:\u002F\u002Fgithub.com\u002Fjustinjohn0306)), and Supertonic 3. Chatterbox also gains voice-conversion support, extending the existing TTS\u002Fvoice-cloning path into a fuller speech workflow.\n>\n> **GGUF support:** audio.cpp now has reusable GGUF loading and conversion support, with tested GGUF paths for multiple ASR and TTS models. Some models can run up to 2× faster with Q8 GGUF, without any parity drift. See [docs\u002Fgguf.md](docs\u002Fgguf.md) for the current support status. Huge thanks to [@mirek190](https:\u002F\u002Fgithub.com\u002Fmirek190) for driving the core GGUF work and model support forward.\n\n**2026-06-25 to 2026-07-08:** audio.cpp grew from the first released model wave into broad TTS, ASR, music generation, source separation, VAD, diarization, codec, and voice-conversion coverage, with VibeVoice 1.5B\u002F7B, LoRA adapter loading, initial streaming support, and major CUDA Conv1DTransp speedups.\n\n## Supported Models\n\n| Family | Task | Supported language(s) | Supported variant(s) in this repo |\n|---|---|---|---|\n| **ace_step** | music generation, music editing | 50+ langs | ACE-Step 1.5 Turbo and Base with acestep-5Hz-lm-1.7B |\n| **chatterbox** | TTS, voice cloning | ar, da, de, el, en, es, fi, fr, hi, it, ko, ms, nl, no, pl, pt, sv, sw, tr | Chatterbox with 0.5B backbone |\n| **citrinet_asr** | ASR | en | Citrinet-256 |\n| **heartmula** | music generation | zh, en, ja, ko, es | HeartMuLa-oss-3B with HeartCodec-oss |\n| **higgs_audio_stt** | ASR | en | Higgs Audio v3 STT |\n| **htdemucs** | source separation | lang agnostic | HTDemucs, HTDemucs_ft |\n| **hviske_asr** | ASR | da | Hviske v5.3 |\n| **marblenet_vad** | VAD | lang agnostic | MarbleNet VAD |\n| **mel_band_roformer** | vocal separation | lang agnostic | Mel-Band RoFormer MLX vocal separation variants |\n| **miocodec** | audio codec, voice conversion backend | lang agnostic | MioCodec v2, 25 Hz, 44.1 kHz |\n| **miotts** | TTS, voice cloning | en, ja | MioTTS-1.7B |\n| **omnivoice** | TTS, voice cloning, voice design | 646+ langs | OmniVoice, Qwen3-0.6B based |\n| **pocket_tts** | TTS, voice cloning | en, de, it, pt, es | PocketTTS-100M |\n| **nemotron_asr** | ASR | 100+ ASR prompt codes incl. auto | Nemotron 3.5 ASR Streaming 0.6B |\n| **qwen3_asr** | ASR | zh, en, yue, ar, de, fr, es, pt, id, it, ko, ru, th, vi, ja, tr, hi, ms, nl, sv, da, fi, pl, cs, fil, fa, el, ro, hu, mk | Qwen3-ASR-0.6B, Qwen3-ASR-1.7B-hf |\n| **qwen3_forced_aligner** | forced alignment | zh, yue, en, de, es, fr, it, pt, ru, ko, ja | Qwen3-ForcedAligner-0.6B |\n| **qwen3_tts** | TTS, voice cloning, voice design | zh, en, fr, de, it, ja, ko, pt, ru, es | Qwen3-TTS-12Hz-0.6B-Base, Qwen3-TTS-12Hz-1.7B-Base, Qwen3-TTS-12Hz-1.7B-CustomVoice, Qwen3-TTS-12Hz-1.7B-VoiceDesign |\n| **seed_vc** | voice conversion | lang agnostic | SeedVC XLS-R + HiFT, SeedVC Whisper-small + BigVGAN |\n| **silero_vad** | VAD | lang agnostic | Silero VAD |\n| **sortformer_diar** | diarization | en | Sortformer-4spk-v1 |\n| **stable_audio** | music generation, sound generation, audio editing | en | Stable Audio 3 Small Music, Stable Audio 3 Small SFX, Stable Audio 3 Medium |\n| **vevo2** | TTS, singing generation, voice conversion, singing conversion, editing | en, zh | Vevo2 with Qwen2.5-0.5B AR model |\n| **vibevoice** | TTS, multi-speaker dialogue TTS | en, zh | VibeVoice-1.5B, VibeVoice-7B |\n| **vibevoice_asr** | ASR | auto | VibeVoice ASR |\n| **voxcpm2** | TTS, voice cloning, voice design | ar, da, de, el, en, es, fi, fr, he, hi, id, it, ja, km, ko, lo, ms, my, nl, no, pl, pt, ru, sv, sw, th, tl, tr, vi, zh | VoxCPM2-2B, 48 kHz |\n| **index_tts2** | TTS, voice cloning, expressive speech | zh, en | IndexTTS-2 |\n| **irodori_tts** | TTS, voice cloning, voice design | ja | Irodori-TTS-500M-v3, Irodori-TTS-600M-v3-VoiceDesign |\n| **moss_tts_nano** | TTS, voice cloning | auto | MOSS-TTS-Nano-100M |\n| **moss_tts_local** | TTS, voice cloning | auto, optional language hint | MOSS-TTS-Local-Transformer-v1.5 |\n| **supertonic** | TTS | en, ko, ja, ar, bg, cs, da, de, el, es, et, fi, fr, hi, hr, hu, id, it, lt, lv, nl, pl, pt, ro, ru, sk, sl, sv, tr, uk, vi, na | Supertonic 3 |\n\nWIP: Higgs Audio v3 TTS 4B, Fish Audio S2 Pro, Voxtral-Mini-4B-Realtime.\n\nPocketTTS language selection is a model-load option. When the model path points at the PocketTTS root, the loader uses `english` unless you pass `--load-option language=\u003Cname>`. Kyutai's normal non-English PocketTTS releases are smaller distilled language models intended for the fast PocketTTS path. The `_24l` variants are larger 24-layer, undistilled preview models that can sound better but are slower. Kyutai currently publishes French only as `french_24l`, not as a normal distilled `french` language directory, so French is not listed as a normal PocketTTS language here.\n\n## Docker\n\nDocker CPU and CUDA images are available for both CLI and server use. See [Docker.md](Docker.md) for build commands and working Docker examples.\n\n## Build\n\n| OS | Requirements |\n|---|---|\n| Linux | GCC 13 or newer, CMake, backend toolchain for CUDA or Vulkan builds |\n| Windows | Visual Studio Build Tools 2022 or newer with C++ desktop workload, MSVC x64 compiler, Windows SDK, CMake, Ninja, MSVC OpenMP components; official NVIDIA CUDA Toolkit for CUDA builds |\n| macOS | Xcode or Xcode Command Line Tools with the Metal compiler available through `xcrun` |\n\n### Linux Build\n\nUse the Linux helper script for CPU, CUDA, or Vulkan builds:\n\n```bash\nscripts\u002Fbuild_linux.sh --backend cuda --target audiocpp_cli --target audiocpp_server\nscripts\u002Fbuild_linux.sh --backend vulkan --target audiocpp_cli --target audiocpp_server\nscripts\u002Fbuild_linux.sh --backend cpu --target audiocpp_cli --target audiocpp_server\n```\n\nThe script writes to aligned build directories such as `build\u002Flinux-cuda-release`, `build\u002Flinux-vulkan-release`, and `build\u002Flinux-cpu-release`.\n\nFor portable CPU kernels on machines where native ISA flags are not suitable:\n\n```bash\nscripts\u002Fbuild_linux.sh --backend cuda --native-cpu OFF --target audiocpp_cli --target audiocpp_server\n```\n\nFor direct CMake commands, see [docs\u002Fbuild\u002Flinux.md](docs\u002Fbuild\u002Flinux.md).\n\n### Windows Build\n\nUse the Windows PowerShell build script:\n\n```powershell\npowershell.exe -NoProfile -ExecutionPolicy Bypass -File .\\scripts\\build_windows.ps1\n```\n\nCommon presets:\n\n```powershell\n.\\scripts\\build_windows.ps1 -Preset windows-cuda-release -Target audiocpp_cli\n.\\scripts\\build_windows.ps1 -Preset windows-cpu-release -Target audiocpp_cli\n.\\scripts\\build_windows.ps1 -Target audiocpp_server -Jobs 16\n```\n\nFrom `cmd.exe`, use the wrapper:\n\n```bat\nscripts\\build_windows.cmd\n```\n\nFor requirements, CPU profiles, CUDA packaging, and release zips, see [docs\u002Fbuild\u002Fwindows.md](docs\u002Fbuild\u002Fwindows.md).\n\n### Metal Build\n\nOn macOS, use the Metal helper script to build against ggml's Metal backend:\n\n```bash\nscripts\u002Fbuild_metal.sh --target audiocpp_cli\n```\n\nThe script configures `build\u002Fmacos-metal-release` by default, enables `ENGINE_ENABLE_METAL=ON`, disables CUDA and Vulkan, embeds the Metal shader library, and builds static libraries plus the requested target.\n\nUseful variants:\n\n```bash\nscripts\u002Fbuild_metal.sh --target audiocpp_server\nscripts\u002Fbuild_metal.sh --build-type Release --archs arm64 --target audiocpp_cli\nscripts\u002Fbuild_metal.sh --with-tests --target audio_dsp_test\nscripts\u002Fbuild_metal.sh --openmp auto --target audiocpp_cli\n```\n\nThe built CLI is written to:\n\n```bash\nbuild\u002Fmacos-metal-release\u002Fbin\u002Faudiocpp_cli\n```\n\n### Build Options\n\n| Option | Meaning | Default |\n|---|---|---|\n| `ENGINE_ENABLE_CUDA` | Enable the ggml CUDA backend. Required for `--backend cuda`. | `OFF` |\n| `ENGINE_ENABLE_VULKAN` | Enable the ggml Vulkan backend. Required for `--backend vulkan`. | `OFF` |\n| `ENGINE_ENABLE_METAL` | Enable the ggml Metal backend. Required for `--backend metal`. | `OFF` on most platforms, `ON` on Apple |\n| `ENGINE_ENABLE_LLAMAFILE` | Enable llamafile SGEMM support in ggml CPU builds. | `ON` |\n| `ENGINE_ENABLE_CUDA_GRAPHS` | Enable ggml CUDA graphs support when CUDA is enabled. | `ON` |\n| `ENGINE_ENABLE_NATIVE_CPU` | Build ggml CPU kernels with native host ISA flags such as `-march=native`. Disable this for portable CPU kernels or toolchains that reject generated CPU instructions. | `ON` |\n| `ENGINE_ENABLE_OPENMP` | Enable OpenMP for host-side parallel work. | `ON` |\n| `ENGINE_BUILD_EXAMPLES` | Build example binaries. | `OFF` |\n| `ENGINE_BUILD_TESTS` | Build framework unit tests. | `OFF` |\n| `ENGINE_BUILD_WARMBENCH` | Build warmbench helper binaries. | `OFF` |\n\n## Usage\n\nFor full setup, CLI, server, and workflow examples, see [docs\u002Fusage.md](docs\u002Fusage.md).\n\n### CLI\n\nThe main CLI binary is:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli\n```\n\nHigh-level command shape:\n\n```bash\naudiocpp_cli --task \u003Ctask> --model \u003Cpath> [--family \u003Cfamily>] [--backend \u003Cbackend>] [--mode \u003Cmode>] [options]\n```\n\nCore selectors:\n\n- `--task vad|asr|diar|sep|gen|tts|clon|vc|s2s|align|vdes|spk|svc`\n- `--model \u003Cpath>`\n- `--family \u003Cname>` optionally narrows model-loader selection when a model path could match more than one family\n- `--backend cpu|cuda|vulkan|metal|best`\n- `--mode offline|streaming`; streaming is available for models whose docs list streaming support\n\nCommon interface options:\n\n- `--load-option key=value` passes model-load options, such as PocketTTS language selection\n- `--session-option key=value` passes session\u002Fruntime options, such as backend-specific weight controls\n- `--request-option key=value` passes per-request model options\n- `--config \u003Cid>` selects a discovered config asset\n- `--weight \u003Cid>` selects a discovered weight asset\n- `--device \u003Cn>` selects the backend device\n- `--threads \u003Cn>` sets backend and OpenMP worker threads\n\nExamples:\n\nText-to-speech:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli \\\n  --task tts \\\n  --family pocket_tts \\\n  --model \u002Fpath\u002Fto\u002Fmodel \\\n  --backend cuda \\\n  --text \"audio.cpp is running PocketTTS locally.\" \\\n  --voice-ref assets\u002Fresources\u002Fsample.wav \\\n  --out build\u002Fout\u002Fpocket_tts.wav\n```\n\nPocketTTS with another language and a built-in voice:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli \\\n  --task tts \\\n  --family pocket_tts \\\n  --model \u002Fpath\u002Fto\u002Fmodels\u002Fpocket-tts \\\n  --backend cuda \\\n  --load-option language=spanish \\\n  --text \"Hola, esta es una prueba corta de Pocket TTS.\" \\\n  --voice-id alba \\\n  --out build\u002Fout\u002Fpocket_tts_spanish.wav\n```\n\nASR:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli \\\n  --task asr \\\n  --family qwen3_asr \\\n  --model \u002Fpath\u002Fto\u002Fmodel \\\n  --backend cuda \\\n  --audio assets\u002Fresources\u002Fsample_16k.wav\n```\n\nVoice conversion:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli \\\n  --task vc \\\n  --family seed_vc \\\n  --model \u002Fpath\u002Fto\u002Fmodel \\\n  --backend cuda \\\n  --audio assets\u002Fresources\u002Fa.wav \\\n  --voice-ref assets\u002Fresources\u002Fb.wav \\\n  --out build\u002Fout\u002Fseed_vc.wav\n```\n\nUseful CLI features:\n\n- `--help` with `--task` shows task-oriented help\n- `--help` with `--model \u003Cpath>` and optional `--family \u003Cfamily>` shows model-owned request, session, and load options\n- `--inspect` prints discovered configs, weights, and capabilities\n- `--list-loaders` prints registered model families\n- `--batch-text-file \u003Ctxt>` runs one offline request per non-empty line\n- `--batch-text-dir \u003Cdir>` runs one offline request per `.txt`, `.md`, or `.json` file, normalizing each file as one paragraph\n- `--batch-audio-dir \u003Cdir>` runs one offline request per `.wav`\n- `--audio-chunk-mode auto` lets ASR\u002Falignment models choose their safe long-audio policy; expert users can override with `fixed`, `vad`, or `none` where supported\n- `--request-sequence \u003Cjson>` runs a multi-request offline session\n- `--batch-merge-audio none|concat` controls batch audio merge behavior\n- `--batch-manifest-out \u003Cjson>` writes a batch output manifest\n- `--pipeline \u003Cjson>` runs a workflow instead of a raw task\n- `--list-pipelines` prints registered workflows\n- `--workflow-input key=value` overrides pipeline inputs\n- `--log` streams framework logs to stdout\n- `--log-file \u003Cpath>` streams framework logs to a file in real time\n- `--segments-out`, `--turns-out`, and `--words-out` write structured JSON outputs\n- `--vad-chunks-out` writes offline VAD-based chunk windows; tune them with `--vad-chunk-max-seconds`, `--vad-chunk-merge-gap-seconds`, and `--vad-chunk-padding-seconds`\n\n### Pipelines\n\nPipelines are an experimental JSON workflow feature for chaining multiple model and audio-processing steps behind one CLI command. A pipeline can define default inputs, let users override them with `--workflow-input key=value`, split long media into model-sized chunks, merge text or audio outputs back together, write intermediate artifacts under `--out-dir`, and copy the declared `final_audio` to `--out`.\n\nThis is the higher-level layer for production-style audio jobs: redubbing, batch cleanup, long-form narration, voice conversion, source-separation workflows, transcription-plus-alignment, and future workflows that combine translation, diarization, denoise, enhancement, or review steps as those model surfaces are wired into the framework.\n\nThe included same-language speech redub pipeline transcribes long speech in chunks with Qwen3 ASR, merges the transcript, then regenerates the speech in a target reference voice with Qwen3 TTS. The default test input `assets\u002Fresources\u002Fspeech.wav` is about 418 seconds long and was generated from an 8,091-character speech text, so it exercises long-audio split and merge behavior rather than a short one-shot request:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli \\\n  --pipeline assets\u002Fpipeline\u002Fspeech_redub.json \\\n  --backend cuda \\\n  --out-dir build\u002Fout\u002Fspeech_redub_pipeline \\\n  --out build\u002Fout\u002Fspeech_redub_pipeline.wav\n```\n\nOverride the source speech or target voice without editing the JSON:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli \\\n  --pipeline assets\u002Fpipeline\u002Fspeech_redub.json \\\n  --backend cuda \\\n  --workflow-input source_audio=\u002Fpath\u002Fto\u002Fspeech.wav \\\n  --workflow-input target_voice=\u002Fpath\u002Fto\u002Fvoice.wav \\\n  --workflow-input language=English \\\n  --out-dir build\u002Fout\u002Fspeech_redub_pipeline \\\n  --out build\u002Fout\u002Fspeech_redub_pipeline.wav\n```\n\n### Tools \u002F Model Manager\n\nThe repository also ships a model manager at `tools\u002Fmodel_manager.py` for downloading supported model packages into the framework expected `models\u002F` layout.\n\nSome models also have GGUF packages available. Current GGUF repositories include [audio-cpp\u002Faudio.cpp-gguf](https:\u002F\u002Fhuggingface.co\u002Faudio-cpp\u002Faudio.cpp-gguf) and [mirek190\u002Faudio.cpp](https:\u002F\u002Fhuggingface.co\u002Fmirek190\u002Faudio.cpp). See [docs\u002Fgguf.md](docs\u002Fgguf.md) for GGUF support status. A dedicated GGUF model-management tool is under development.\n\nDependencies:\n\n- Python 3\n- `torch`\n- `safetensors`\n- `PyYAML`\n- network access to the upstream model sources\n\nThe tool supports three main commands:\n\n- `list` shows the available package ids\n- `info` shows the target layout, required files, and install source for one package\n- `install` downloads or converts one package into a models root\n\nRecommended top-level install packages:\n\n`Yes` means Hugging Face has a ready-to-use repo that the framework can download as-is. `No` means the tool must assemble, convert, or post-process files before the framework can use them.\n\n| Package id | Model | HF ready-to-use repo |\n|---|---|---|\n| `ace_step` | ACE-Step 1.5 Turbo\u002FBase | No |\n| `chatterbox` | Chatterbox | **Yes** |\n| `citrinet_asr` | Citrinet ASR converted layout | No |\n| `heartmula` | HeartMuLa | No |\n| `higgs_audio_stt` | Higgs Audio STT | No |\n| `higgs_audio_v3_tts_4b` | Higgs Audio v3 TTS 4B | **Yes** |\n| `htdemucs` | HTDemucs | No |\n| `hviske_asr` | Hviske ASR | **Yes** |\n| `irodori_tts_500m_v3` | Irodori-TTS 500M v3 | No |\n| `irodori_tts_600m_v3_voice_design` | Irodori-TTS 600M v3 VoiceDesign | No |\n| `kokoro_82m_bf16` | Kokoro 82M bf16 | **Yes** |\n| `marblenet_vad` | MarbleNet VAD converted layout | No |\n| `mel_band_roformer` | Mel-Band RoFormer MLX | **Yes** |\n| `miocodec_25hz_44k_v2` | MioCodec 25Hz 44.1kHz v2 | No |\n| `miotts_1_7b` | MioTTS 1.7B | No |\n| `moss_audio_tokenizer_nano` | MOSS Audio Tokenizer Nano | No |\n| `moss_audio_tokenizer_v2` | MOSS Audio Tokenizer v2 | No |\n| `moss_tts_nano_100m` | MOSS-TTS-Nano 100M | No |\n| `moss_tts_nano_100m_model` | MOSS-TTS-Nano 100M model subcomponent | No |\n| `moss_tts_local_v1_5` | MOSS-TTS-Local Transformer v1.5 | No |\n| `nemotron_asr` | Nemotron ASR | **Yes** |\n| `omnivoice` | OmniVoice | **Yes** |\n| `parakeet_tdt_0_6b_v3` | Parakeet TDT 0.6B v3 | **Yes** |\n| `pocket_tts` | PocketTTS | **Yes** |\n| `qwen3_asr_0_6b` | Qwen3 ASR 0.6B | **Yes** |\n| `qwen3_asr_1_7b_hf` | Qwen3 ASR 1.7B HF | **Yes** |\n| `qwen3_forced_aligner_0_6b` | Qwen3 Forced Aligner 0.6B | **Yes** |\n| `qwen3_tts_0_6b_base` | Qwen3 TTS 12Hz 0.6B Base | **Yes** |\n| `qwen3_tts_1_7b_base` | Qwen3 TTS 12Hz 1.7B Base | **Yes** |\n| `qwen3_tts_1_7b_custom_voice` | Qwen3 TTS 12Hz 1.7B Custom Voice | **Yes** |\n| `qwen3_tts_1_7b_voice_design` | Qwen3 TTS 12Hz 1.7B Voice Design | **Yes** |\n| `seed_vc` | SeedVC-MLX | **Yes** |\n| `sortformer_diar_4spk_v1` | Sortformer diarization 4 speaker v1 | **Yes** |\n| `stable_audio_3_medium` | Stable Audio 3 Medium | **Yes** |\n| `stable_audio_3_small_music` | Stable Audio 3 Small Music | **Yes** |\n| `stable_audio_3_small_sfx` | Stable Audio 3 Small SFX | **Yes** |\n| `supertonic_3` | Supertonic 3 | **Yes** |\n| `vevo2` | Vevo2 | No |\n| `vibevoice_1_5b` | VibeVoice 1.5B | No |\n| `vibevoice_7b` | VibeVoice 7B | No |\n| `vibevoice_asr` | VibeVoice ASR | No |\n| `voxcpm2` | VoxCPM2 | No |\n\n> [!WARNING]\n> PocketTTS is hosted in a gated Hugging Face repo, so the model manager needs a Hugging Face token with access to `kyutai\u002Fpocket-tts`. It currently downloads only the English model and the built-in `alba` voice.\n\n> [!TIP]\n> If you already have the VibeVoice Hugging Face model directory, you do not need to redownload the tokenizer files. Copy `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, and `merges.txt` from `assets\u002Fmodel_manager\u002Fvibevoice_1_5b\u002F` into `VibeVoice-1.5B\u002F`, `VibeVoice-7B\u002F`, or `VibeVoice-ASR\u002F`.\n\nExamples:\n\nList packages:\n\n```bash\npython3 tools\u002Fmodel_manager.py list\n```\n\nShow one package:\n\n```bash\npython3 tools\u002Fmodel_manager.py info qwen3_tts_1_7b_base\n```\n\nInstall one package into the default `models\u002F` directory:\n\n```bash\npython3 tools\u002Fmodel_manager.py install qwen3_tts_1_7b_base\n```\n\nInstall into a custom models root:\n\n```bash\npython3 tools\u002Fmodel_manager.py install vevo2 --models-root \u002Fpath\u002Fto\u002Fmodels\n```\n\nReplace an existing installed package:\n\n```bash\npython3 tools\u002Fmodel_manager.py install pocket_tts --overwrite\n```\n\nSome packages are direct snapshots, while others are composite installs or local-file utilities. Use `info` first when you want to inspect the expected target directory, required files, or whether a package needs extra local source inputs such as `--source-file` or `--source-dir`.\n\nRun a local-file utility:\n\n```bash\npython3 tools\u002Fmodel_manager.py info voxcpm2_audiovae\npython3 tools\u002Fmodel_manager.py install voxcpm2_audiovae --source-file models\u002FVoxCPM2\u002Faudiovae.pth --models-root models --overwrite\n```\n\n### Server\n\nThe server binary is:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_server\n```\n\nBuild:\n\n```bash\ncmake --build build -j$(nproc) --target audiocpp_server\n```\n\nCreate a config file with your own model paths:\n\n```bash\ncat > server.json \u003C\u003C'JSON'\n{\n  \"host\": \"127.0.0.1\",\n  \"port\": 8080,\n  \"backend\": \"cuda\",\n  \"device\": 0,\n  \"threads\": 1,\n  \"lazy_load\": true,\n  \"models\": [\n    {\n      \"id\": \"pocket-tts\",\n      \"family\": \"pocket_tts\",\n      \"path\": \"\u002Fpath\u002Fto\u002Fmodels\u002Fpocket-tts\",\n      \"task\": \"tts\",\n      \"mode\": \"offline\",\n      \"load_options\": {\n        \"language\": \"english\"\n      },\n      \"session_options\": {\n        \"language\": \"english\"\n      }\n    },\n    {\n      \"id\": \"qwen3-asr\",\n      \"family\": \"qwen3_asr\",\n      \"path\": \"\u002Fpath\u002Fto\u002Fmodels\u002FQwen3-ASR-0.6B\",\n      \"task\": \"asr\",\n      \"mode\": \"offline\"\n    }\n  ]\n}\nJSON\n```\n\nSet `\"lazy_load\": true` to register configured model ids at startup while loading each model only on first use. Use per-model `\"lazy\": true` or `\"lazy\": false` to override that default.\n\nSet top-level `\"backend\"` to `\"cuda\"`, `\"cpu\"`, `\"vulkan\"`, or `\"metal\"`. CUDA is the optimized path for audio.cpp; CPU, Vulkan, and Metal are intended for portability and testing when the binary is built with that backend, but performance and model coverage may be lower.\n\n> [!WARNING]\n> Lazy loading does not unload models after a request. Once a model is first used, the server keeps that model and session in memory for reuse until the server exits.\n\nStart:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_server --config server.json\n```\n\nThe server exposes:\n\n- `GET \u002Fhealth`\n- `GET \u002Fv1\u002Fmodels`\n- `POST \u002Fv1\u002Faudio\u002Fspeech`\n- `POST \u002Fv1\u002Faudio\u002Ftranscriptions`\n- `POST \u002Fv1\u002Ftasks\u002Frun`\n\nMore server examples are in [app\u002Fserver\u002FREADME.md](app\u002Fserver\u002FREADME.md).\n\n## Tests\n\nThe repository includes both framework-level parity validation and app-level end-to-end path checks. At a high level, the flow is:\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigure\u002Fparity_test_flow.png\" alt=\"Parity test flow\" width=\"720\" \u002F>\n\u003C\u002Fp>\n\nThe main harness under `tests\u002F` is `tests\u002Fwarmbench.py`. It is used for long-lived multi-request validation, parity checks against Python references, and performance-oriented session reuse scenarios. The `tests\u002F` tree also contains model-specific C++ and Python warmbench entrypoints that `warmbench.py` coordinates.\n\nThe main app-facing test tooling under `tools\u002F` is `tools\u002Faudiocpp_cli\u002Frun_audiocpp_cli_path_tests.py`. It drives `audiocpp_cli` through cataloged offline and streaming cases, verifies expected outputs such as audio or JSON artifacts, and is useful for checking real user-facing request paths rather than just lower-level model components. Streaming coverage is model-specific and applies to models documented with streaming support.\n\nThe Python-reference side of these tests usually requires more time-consuming setup than the C++ path because different models rely on different Python reference repos and dependency stacks. In practice, the framework-side tooling is fast to iterate on once models are installed, while Python parity runs often need extra environment preparation before they are ready.\n\n## Projects\n\n- [TranscrIA](https:\u002F\u002Fgithub.com\u002FMartossien\u002Ftranscria) is a self-hosted meeting transcription platform with diarization and local LLM correction. audio.cpp is integrated as a first-class STT engine in the product.\n- [Pocket TTS Browser Engine](https:\u002F\u002Fgithub.com\u002Fjjmlovesgit\u002Fpocket-tts-browser-engine) uses audio.cpp to bring fully local PocketTTS voices into Chrome and Edge through the browser TTS API.\n- [GuideAnts](https:\u002F\u002Fgithub.com\u002FElumenotion\u002FGuideAnts) uses audio.cpp as the default local AI stack path for basic ASR and TTS, with planned reusable skills for audio.cpp scenarios and model configurations.\n\n## Performance Metrics\n\n> [!WARNING]\n> These Python-relative numbers were measured for the initial release. Several model paths have improved substantially since then, so the figures below should be read as the original release baseline rather than the latest peak performance.\n\nAll performance metrics in this section were measured on Ubuntu with the CUDA backend on an NVIDIA GeForce RTX 5090. The Python-relative one-shot and long-lived-session comparisons come from direct framework\u002Fruntime API benchmark calls, not from `audiocpp_cli`; CLI path tests are separate and include app-layer request parsing, output writing, and other user-facing overhead.\n\n**Absolute RTF depends on the GPU and system setup, but the Python-relative speedups are real because audio.cpp and the matching Python reference paths were measured on the same CUDA setup.**\n\naudio.cpp already shows some genuinely exciting wins against the matching Python reference paths, especially on the TTS side, even when using the original model weights without quantization. The headline win is wall time: several TTS paths run **1.8x-5.0x faster** than Python while cutting end-to-end latency by **45%-80%**.\n\n- In one-shot runs, several TTS-family models already land far ahead of Python:\n  - `vevo2`: **5.03x faster** with **80.11% less wall time**\n  - `pocket tts`: **3.68x faster** with **72.80% less wall time**\n  - `miotts`: **2.73x faster** with **63.39% less wall time**\n  - `moss_tts_local`: **2.33x faster** with **57.07% less wall time**\n  - `qwen3 tts`: **1.83x faster** with **45.34% less wall time**\n  - `vibevoice`: **1.40x faster** with **28.75% less wall time**\n- In long-lived-session runs, where the same loaded session serves multiple requests in sequence, the gains stay strong:\n  - `pocket tts`: **3.22x faster** with **68.91% less wall time**\n  - `qwen3 tts`: **2.74x faster** with **63.47% less wall time**\n  - `moss_tts_local`: **2.66x faster** with **62.35% less wall time**\n  - `miotts`: **2.28x faster** with **56.22% less wall time**\n  - `vibevoice`: **1.77x faster** with **43.55% less wall time**\n  - `vevo2`: **1.75x faster** with **42.72% less wall time**\n- In long-form runs on the shared 6,026-character, 1,028-word passage, the strongest Python-relative wins still show up clearly:\n  - `pocket tts`: **3.15x faster** with **68.23% less wall time**\n  - `qwen3 tts`: **3.06x faster** with **67.33% less wall time**\n  - `vibevoice`: **2.86x faster** with **65.07% less wall time**\n  - `vevo2`: **1.77x faster** with **43.51% less wall time**\n  - `chatterbox`: **1.58x faster** with **36.83% less wall time**\n- These long-lived-session numbers are especially important for real applications, because they reflect the common case where model load, cached state, and reusable runtime setup are amortized across many requests.\n- Bars below the 1.0x line are useful too: they spotlight exactly where more optimization work is still worth doing.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigure\u002Fperf_one_shot_20260630.svg\" alt=\"One-shot\" width=\"720\" \u002F>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Ffigure\u002Fperf_long_lived_session_20260630.svg\" alt=\"Long-lived session\" width=\"720\" \u002F>\n\u003C\u002Fp>\n\nThe figures report `Python wall time \u002F audio.cpp wall time`. The 1.0x line means equal wall time; bars above 1.0x mean audio.cpp is faster than Python, and bars below 1.0x mean it is slower.\n\nFor TTS-family models, the measured one-shot RTF is:\n\n| model | audio len (s) | wall time (s) | RTF | x faster than real time |\n|---|---:|---:|---:|---:|\n| chatterbox | 9.72 | 2.45 | 0.252 | 3.97x |\n| kokoro tts | 10.15 | 0.64 | 0.063 | 15.90x |\n| miotts | 20.40 | 3.30 | 0.162 | 6.18x |\n| moss_tts_local | 9.60 | 0.97 | 0.101 | 9.91x |\n| omnivoice | 9.00 | 1.32 | 0.146 | 6.84x |\n| pocket tts | 8.08 | 0.26 | 0.032 | 31.09x |\n| qwen3 tts | 11.44 | 4.46 | 0.390 | 2.56x |\n| vevo2 | 8.66 | 2.47 | 0.285 | 3.51x |\n| vibevoice | 11.07 | 5.02 | 0.454 | 2.20x |\n| voxcpm2 | 5.60 | 3.09 | 0.551 | 1.81x |\n\nFor long-form TTS tests, each run uses the same 6,026-character, 1,028-word input text (vibevoice uses 106,310 chars, 18,052 words, 4 speakers). Rows are CUDA unless marked CPU. The measured RTF is:\n\n| model | audio len (s) | wall time (s) | RTF | x faster than real time |\n|---|---:|---:|---:|---:|\n| chatterbox | 391.24 | 58.57 | 0.150 | 6.68x |\n| kokoro tts | 371.17 | 7.19 | 0.019 | 51.60x |\n| index tts2 | 422.12 | 139.95 | 0.332 | 3.02x |\n| miotts | 399.16 | 66.59 | 0.167 | 5.99x |\n| moss_tts_nano | 391.20 | 43.16 | 0.110 | 9.06x |\n| moss_tts_local | 375.44 | 73.84 | 0.197 | 5.08x |\n| omnivoice | 357.00 | 17.77 | 0.050 | 20.09x |\n| pocket tts | 353.12 | 7.30 | 0.021 | 48.40x |\n| qwen3 tts | 327.60 | 72.65 | 0.222 | 4.51x |\n| supertonic | 379.32 | 2.02 | 0.005 | 187.62x |\n| supertonic (CPU) | 379.40 | 61.40 | 0.162 | 6.18x |\n| vevo2 | 457.68 | 52.47 | 0.115 | 8.72x |\n| voxcpm2 | 315.84 | 72.70 | 0.230 | 4.34x |\n| vibevoice | 5615.73 | 1376.84 | 0.245 | 4.08x |\n\n## Runtime Memory Options\n\nSome models expose memory-saver session options such as `ace_step.mem_saver=true`, `heartmula.mem_saver=true`, `stable_audio.mem_saver=true`, `omnivoice.mem_saver=true`, and `voxcpm2.mem_saver=true`. These options keep the default output path unchanged while reducing graph workspace VRAM or releasing staged graph\u002Fcache state after request phases; later requests may rebuild released graphs.\n\n## Precision\u002FQuantization Support\n\nMany model sessions expose quantization through `--session-option \u003Cfamily>.weight_type=\u003Cmode>`, and some families also expose more specific knobs such as `...conv_weight_type`, `...talker_weight_type`, or `...speech_decoder_weight_type`. The exact supported modes are model-specific rather than global.\n\nThe framework also has a reusable GGUF tensor source and a streaming converter. The\ncontainer reader is shared by all model families; a family still has to list a `.gguf`\ncheckpoint as one of its accepted assets because model configuration and tensor naming\nremain architecture-specific. Qwen3 ASR, Qwen3 Forced Aligner, Qwen3 TTS, Nemotron\n3.5 ASR, VibeVoice-ASR, Higgs Audio STT, Hviske ASR, and Citrinet ASR currently accept\n`model.gguf` (including `speech_tokenizer\u002Fmodel.gguf` for TTS). The converter recursively embeds sidecar files\nup to 64 MiB by default using binary-safe metadata, including nested tokenizer models,\nand Qwen3 ASR, Nemotron ASR, VibeVoice-ASR, Higgs Audio STT, Hviske ASR, and Citrinet\nASR can load the resulting `model.gguf` as a standalone file. Pass `--no-sidecars` when a\ntensor-only container is desired. A\n`model.safetensors.index.json` is also a first-class tensor source and is merged from\nits routed shards while converting. Exact original tensor ranks are stored separately\nbecause GGML normally collapses trailing singleton dimensions. Rank-0 safetensors\nscalars are stored physically as one-element GGML tensors while their scalar rank is\npreserved in the exact-shape metadata.\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_gguf \\\n  --input models\u002FQwen3-ASR-1.7B-hf\u002Fmodel.safetensors \\\n  --output models\u002FQwen3-ASR-1.7B-hf\u002Fmodel.gguf \\\n  --type q8_0\n```\n\nMulti-component checkpoints can be packed into one GGUF with repeated namespaced\ninputs. Existing component loaders can open a namespace through\n`open_tensor_source(path, \"component\")`, which strips that prefix from the view:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_gguf \\\n  --input gpt=models\u002Findex-tts2-mlx\u002Fgpt.safetensors \\\n  --input s2mel=models\u002Findex-tts2-mlx\u002Fs2mel.safetensors \\\n  --input bigvgan=models\u002Findex-tts2-mlx\u002Fbigvgan\u002Fmodel.safetensors \\\n  --root models\u002Findex-tts2-mlx \\\n  --sidecar models\u002Fshared\u002Fpreprocessor_config.json=preprocessor_config.json \\\n  --output models\u002Findex-tts2-mlx-GGUF\u002Fmodel.gguf \\\n  --type q8_0\n```\n\n`--root` selects the directory whose non-weight sidecars are embedded. Repeat\n`--sidecar source=destination` for required assets outside that root, or to remap an\nasset such as Higgs Audio STT's shared Whisper `preprocessor_config.json` into the\nstandalone model root. For Higgs, that external Whisper file is needed only during GGUF\ncreation; it is not required when loading the completed standalone GGUF.\n\nPacking is a container feature; it does not by itself wire every model loader to the new\nlayout. The packed IndexTTS-2 MLX checkpoint has been conversion-tested, including its\nCAMPPlus rank-0 counters, but the existing IndexTTS-2 runtime still has to open and consume\nthe corresponding namespaces before audio.cpp can synthesize directly from that file.\n\nSupported conversion types are `f16`, `q8_0`, `q2_k`, `q3_k`, `q4_k`, `q5_k`, and\n`q6_k`. Quantized GGUF files use mixed precision: projection matrices are quantized,\nwhile embedding\u002Fcodebook lookup tables and unsupported shapes retain a backend-safe\ntype. If both files exist, Qwen loaders prefer `model.gguf` over `model.safetensors`.\n\nExample:\n\n```bash\nbuild\u002Fbin\u002Faudiocpp_cli --task tts --family qwen3_tts --model \u002Fpath\u002Fto\u002Fmodel --session-option qwen3_tts.weight_type=q8_0\n```\n\nIn practice, lower precision and quantized modes should be treated as model- and route-specific optimizations rather than universally safe defaults.\n\n- **Safety.** Quantization may not be safe on every path even when a model parser accepts the option. For example, in our ACE-Step 1.5 checks, lower-precision runs could fail at runtime with `ACE-Step planner masked decode found no valid token` while higher-precision settings completed normally.\n\n- **Quality Drop.** Output quality can drop a lot. In our VeVo2 checks, non-`fp32` outputs showed noticeably weaker similarity to the `fp32` reference under the repo's existing waveform and log-mel comparison metrics, and even output length could shift.\n\n- **Performance Gain.** The performance gain may be minor relative to that quality risk. For example, `q8_0` was faster than the default setting by only around 3.8% on Qwen3-TTS and around 3.6% on VeVo2. Other models may benefit more, but the tradeoff should be validated per model and per route rather than assumed.\n\n- **Memory Benefit.** Lower precision and quantized weights can still be useful for reducing weight memory footprint and making larger models easier to fit within device limits. For example, in our Qwen3-TTS checks, switching from the default setting to `q8_0` reduced peak RAM by about 3.7% and peak VRAM by about 25.0%. That benefit is real, but it should be evaluated together with runtime stability, output quality, and end-to-end speed rather than assumed from precision alone.\n\n## Notes\n\n- The repo supports multiple backends, but backend and model coverage are model-dependent.\n- GGUF is a container, not a universal architecture adapter. Existing llama.cpp or\n  whisper.cpp GGUF files are not automatically compatible unless their tensor names and\n  model metadata are mapped to the audio.cpp family implementation.\n- `Build_xcframework.sh` is outdated; Metal and Apple XCFramework packaging still need to be retested after the framework refactor.\n","audio.cpp-webui 是一个基于 C++ 和 ggml 的纯本地音频模型推理引擎，提供 Web 界面支持多任务音频处理。核心功能涵盖语音识别（ASR\u002FSTT）、文本转语音（TTS）、语音活动检测（VAD）、说话人分离（diarization）、语音转换与音乐生成，全部无需 Python 依赖。技术特点包括 CUDA 加速优化、低延迟流式推理、跨平台可移植性及内置音频预处理工具。适用于离线环境下的实时语音交互、会议转录、播客生成、边缘设备部署等对性能与隐私有较高要求的场景。","2026-07-15 02:30:08","CREATED_QUERY"]