[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72401":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":15,"forks30d":15,"starsTrendScore":19,"compositeScore":20,"rankGlobal":9,"rankLanguage":9,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":22,"hasPages":22,"topics":24,"createdAt":9,"pushedAt":9,"updatedAt":25,"readmeContent":26,"aiSummary":27,"trendingCount":15,"starSnapshotCount":15,"syncStatus":28,"lastSyncTime":29,"discoverSource":30},72401,"omnilingual-asr","facebookresearch\u002Fomnilingual-asr","facebookresearch","Omnilingual ASR Open-Source Multilingual SpeechRecognition for 1600+ Languages",null,"Python",2824,255,29,35,0,3,11,32,9,73.92,"Other",false,"main",[],"2026-06-12 04:01:05","\u003Cdiv align=\"center\">\n  \u003Cimg src=\".\u002FomniASR_header.jpg\" alt=\"Header image with a collage of on-the-ground photos from the transcription gathering efforts in Pakistan and Liberia.\" width=\"100%\" \u002F>\n  \u003Cp>\u003Ci>Photographs captured during corpus creation efforts in Pakistan and Liberia.\u003C\u002Fi>\u003C\u002Fp>\n\u003C\u002Fdiv>\n\n# Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages\n\nOmnilingual ASR is an open-source speech recognition system supporting over 1,600 languages — including hundreds never previously covered by any ASR technology. Designed for broad accessibility, it enables new languages to be added with just a few paired examples without requiring specialized expertise or large datasets. By combining scalable zero-shot learning with a flexible model family, Omnilingual ASR aims to make speech technology more inclusive and adaptable for communities and researchers worldwide.\n\n* [Huggingface Demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Ffacebook\u002Fomniasr-transcriptions)\n* [Huggingface Dataset](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Ffacebook\u002Fomnilingual-asr-corpus)\n* [Paper](https:\u002F\u002Fai.meta.com\u002Fresearch\u002Fpublications\u002Fomnilingual-asr-open-source-multilingual-speech-recognition-for-1600-languages\u002F)\n* [Blogpost](http:\u002F\u002Fai.meta.com\u002Fblog\u002Fomnilingual-asr-advancing-automatic-speech-recognition)\n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\".\u002Fresult_table.png\" alt=\"Performance results table\" width=\"100%\" \u002F>\n  \u003Cp>\u003Ci>Our 7B-LLM-ASR system achieves state-of-the-art performance across 1,600+ languages, with character error rates (CER) below 10 for 78% of those languages. Per language CER results, plus training hours can be found in \u003Ca href=\".\u002Fper_language_results_table_7B_llm_asr.csv\">this csv\u003C\u002Fa>\u003C\u002Fi>\u003C\u002Fp>\n\u003C\u002Fdiv>\n\n\n## December 2025 Update\nWe release two suites of models:\n- Checkpoints of improved accuracy (CER) for the CTC and LLM-ASR models compared to our existing LLM-ASR model (`omniASR_{CTC,LLM}_{300M,1B,3B,7B}_v2`).\n- A new variant of the LLM-ASR model that supports decoding on unlimited audio length (`omniASR_LLM_Unlimited_{300M,1B,3B,7B}_v2`). The unlimited audio length models are briefly described in the [architecture overview section](src\u002Fomnilingual_asr\u002Fmodels\u002FREADME.md). It's accuracy is comparable to limited audio length models, however finetuning recipies for this model are currently not supported.\n\n## Documentation\n\n### Quick Start\n- **[Installation & Basic Usage](#installation)** - Setup and first transcription\n- **[Inference Pipeline](src\u002Fomnilingual_asr\u002Fmodels\u002Finference\u002FREADME.md)** - Comprehensive transcription guide with batch processing, language conditioning, and context examples\n- **[Supported Languages](#supported-languages)** - View the complete list of 1600+ supported languages\n\n\n### Models & Architecture\n- **[Model Specifications](#model-architectures)** - Available models, parameters, and memory requirements\n- **[Architecture Overview](src\u002Fomnilingual_asr\u002Fmodels\u002FREADME.md)** - Technical details on W2V, CTC, and LLM model families\n- **[Asset Management](src\u002Fomnilingual_asr\u002Fcards\u002FREADME.md)** - Configuration system for models, tokenizers, and datasets\n\n### Training & Data Pipeline\n- **[Data Preparation](workflows\u002Fdataprep\u002FREADME.md)** - End-to-end guide for multilingual dataset preparation, HuggingFace integration, and parquet processing\n- **[Training Recipes](workflows\u002Frecipes\u002Fwav2vec2\u002Fasr\u002FREADME.md)** - Pre-configured workflows for CTC and LLM model training\n\n---\n\n## Installation\n\nThe models were developed using [fairseq2](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Ffairseq2), a research-focused sequence modeling toolkit. While we provide a **reference** inference pipeline that works across platforms, audio support requires [libsndfile](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Ffairseq2?tab=readme-ov-file#system-dependencies) (Mac: `brew install libsndfile`; Windows may need an additional [setup](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Ffairseq2?tab=readme-ov-file#installing-on-windows)).\n\n```bash\n# using pip\npip install omnilingual-asr\n\n# using uv\nuv add omnilingual-asr\n```\n\n## Inference\n\n```python\nfrom omnilingual_asr.models.inference.pipeline import ASRInferencePipeline\n\npipeline = ASRInferencePipeline(model_card=\"omniASR_LLM_Unlimited_7B_v2\")\naudio_files = [\"\u002Fpath\u002Fto\u002Feng_audio1.flac\", \"\u002Fpath\u002Fto\u002Fdeu_audio2.wav\"]\nlang = [\"eng_Latn\", \"deu_Latn\"]\ntranscriptions = pipeline.transcribe(audio_files, lang=lang, batch_size=2)\n```\n\nMore details on running specific models can be found in the [src\u002Fomnilingual_asr\u002Fmodels\u002Finference](\u002Fsrc\u002Fomnilingual_asr\u002Fmodels\u002Finference\u002FREADME.md) directory.\n\n> **⚠️ Important:** Currently only audio files shorter than 40 seconds are accepted for inference on CTC and LLM model suites.\n\n### Supported Languages\n\nTo view the full list of 1600+ supported languages, you can access the language list [programmatically](\u002Fsrc\u002Fomnilingual_asr\u002Fmodels\u002Fwav2vec2_llama\u002Flang_ids.py):\n\n```python\nfrom omnilingual_asr.models.wav2vec2_llama.lang_ids import supported_langs\n\n# Print all supported languages\nprint(f\"Total supported languages: {len(supported_langs)}\")\nprint(supported_langs)\n\n# Check if a specific language is supported\nif \"eng_Latn\" in supported_langs:\n    print(\"English (Latin script) is supported!\")\n```\n\nLanguages follow the format `{language_code}_{script}`, for example `eng_Latn` - English (Latin script), `cmn_Hans` - Mandarin Chinese (Simplified), ...\n\n### Using the HuggingFace Dataset 🤗\n\nWe provide a large-scale multilingual speech dataset on HuggingFace under CC-BY-4.0 License: [`facebook\u002Fomnilingual-asr-corpus`](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Ffacebook\u002Fomnilingual-asr-corpus).\nThis dataset can be directly used with our inference pipeline for evaluation or testing:\n\n```bash\npip install \"omnilingual-asr[data]\"\n```\n\n```python\nfrom datasets import load_dataset\nfrom omnilingual_asr.models.inference.pipeline import ASRInferencePipeline\n\n# Load dataset for a specific language (e.g., Ligurian)\nomni_dataset = load_dataset(\"facebook\u002Fomnilingual-asr-corpus\", \"lij_Latn\", split=\"train\", streaming=True)\nbatch = next(omni_dataset.iter(5))\n\n# Convert to pipeline input format\naudio_data = [{\"waveform\": x[\"array\"], \"sample_rate\": x[\"sampling_rate\"]}\n              for x in batch[\"audio\"]]\n\n# Run inference\npipeline = ASRInferencePipeline(model_card=\"omniASR_LLM_7B_v2\")\ntranscriptions = pipeline.transcribe(audio_data, batch_size=2)\n\n# Display results\nfor i, (transcription, original_text) in enumerate(zip(transcriptions, batch[\"raw_text\"]), 1):\n    print(f\"\\n Sample {i}:\")\n    print(f\"   Ground Truth: {original_text}\")\n    print(f\"   Predicted:    {transcription}\")\n```\n\n\n## Model Architectures\n\n| Model Name          | Features      | Parameters | Download Size (FP32) | Inference VRAM¹ | Real-Time Factor¹ (relative speed)² |\n|---------------------|---------------|------------:|---------------:|---------------:|-----------:|\n| [`omniASR_W2V_300M`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-W2V-300M.pt)      | SSL  | 317_390_592   | 1.2 GiB | | |\n| [`omniASR_W2V_1B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-W2V-1B.pt)          | SSL  | 965_514_752   | 3.6 GiB | | |\n| [`omniASR_W2V_3B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-W2V-3B.pt)          | SSL  | 3_064_124_672 | 12.0 GiB | | |\n| [`omniASR_W2V_7B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-W2V-7B.pt)          | SSL  | 6_488_487_168 | 25.0 GiB | | |\n| [`omniASR_CTC_300M`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-300M.pt)      | ASR  | 325_494_996   | 1.3 GiB   | ~2 GiB  | 0.001 (96x) |\n| [`omniASR_CTC_1B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-1B.pt)          | ASR  | 975_065_300   | 3.7 GiB   | ~3 GiB  | 0.002 (48x) |\n| [`omniASR_CTC_3B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-3B.pt)          | ASR  | 3_080_423_636 | 12.0 GiB  | ~8 GiB  | 0.003 (32x) |\n| [`omniASR_CTC_7B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-7B.pt)          | ASR  | 6_504_786_132 | 25.0 GiB  | ~15 GiB | 0.006 (16x) |\n| [`omniASR_CTC_300M_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-300M-v2.pt)      | ASR  | 325_494_996   | 1.3 GiB   | ~2 GiB  | 0.001 (96x) |\n| [`omniASR_CTC_1B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-1B-v2.pt)          | ASR  | 975_065_300   | 3.7 GiB   | ~3 GiB  | 0.002 (48x) |\n| [`omniASR_CTC_3B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-3B-v2.pt)          | ASR  | 3_080_423_636 | 12.0 GiB  | ~8 GiB  | 0.003 (32x) |\n| [`omniASR_CTC_7B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-CTC-7B-v2.pt)          | ASR  | 6_504_786_132 | 25.0 GiB  | ~15 GiB | 0.006 (16x) |\n| [`omniASR_LLM_300M`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-300M.pt)      | ASR with optional language conditioning  | 1_627_603_584 | 6.1 GiB   | ~5 GiB  | 0.090 (~1x) |\n| [`omniASR_LLM_1B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-1B.pt)          | ASR with optional language conditioning  | 2_275_710_592 | 8.5 GiB   | ~6 GiB  | 0.091 (~1x) |\n| [`omniASR_LLM_3B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-3B.pt)          | ASR with optional language conditioning  | 4_376_679_040 | 17.0 GiB  | ~10 GiB | 0.093 (~1x) |\n| [`omniASR_LLM_7B`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-7B.pt)          | ASR with optional language conditioning  | 7_801_041_536 | 30.0 GiB  | ~17 GiB | 0.092 (~1x) |\n| [`omniASR_LLM_300M_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-300M-v2.pt)      | ASR with optional language conditioning  | 1_627_603_584 | 6.1 GiB   | ~5 GiB  | 0.090 (~1x) |\n| [`omniASR_LLM_1B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-1B-v2.pt)          | ASR with optional language conditioning  | 2_275_710_592 | 8.5 GiB   | ~6 GiB  | 0.091 (~1x) |\n| [`omniASR_LLM_3B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-3B-v2.pt)          | ASR with optional language conditioning  | 4_376_679_040 | 17.0 GiB  | ~10 GiB | 0.093 (~1x) |\n| [`omniASR_LLM_7B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-7B-v2.pt)          | ASR with optional language conditioning  | 7_801_041_536 | 30.0 GiB  | ~17 GiB | 0.092 (~1x) |\n| [`omniASR_LLM_Unlimited_300M_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-Unlimited-300M-v2.pt)      | omniASR_LLM_300M + unlimited audio length  | 1_627_603_584 | 6.1 GiB   | ~5 GiB | 0.092 (~1x) (0.206)³ |\n| [`omniASR_LLM_Unlimited_1B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-Unlimited-1B-v2.pt)          | omniASR_LLM_1B + unlimited audio length  | 2_275_710_592 | 8.5 GiB   | ~6 GiB | 0.097 (~1x) (0.207)³ |\n| [`omniASR_LLM_Unlimited_3B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-Unlimited-3B-v2.pt)          | omniASR_LLM_3B + unlimited audio length  | 4_376_679_040 | 17.0 GiB  | ~10 GiB | 0.095 (~1x) (0.208)³ |\n| [`omniASR_LLM_Unlimited_7B_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-Unlimited-7B-v2.pt)          | omniASR_LLM_7B + unlimited audio length  | 7_801_041_536 | 30.0 GiB  | ~17 GiB | 0.097 (~1x) (0.208)³ |\n| [`omniASR_LLM_7B_ZS`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR-LLM-7B-ZS.pt)    | Zero-Shot ASR | 7_810_900_608 | 30.0 GiB | ~20 GiB | 0.194 (~0.5x) |\n| [`omniASR_tokenizer_v1`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR_tokenizer.model) | Tokenizer for all non-v2 models except omniASR_LLM_7B | - | 100 KiB | - |\n| [`omniASR_tokenizer_v1_variant7`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR_tokenizer_v7.model) | Tokenizer for the omniASR_LLM_7B architecture | - | 100 KiB | - |\n| [`omniASR_tokenizer_written_v2`](https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmms\u002FomniASR_tokenizer_written_v2.model) | Tokenizer for all v2 architectures | - | 100 KiB | - ||\n\n¹ (batch=1, audio_len=30s, BF16, A100)\n\n² Relative speed to `omniASR_LLM_7B`\n\n³ (batch=1, audio_len=15min, BF16, A100)\n\n### Model Download & Storage\n\n- **Automatic Download**: Models are automatically downloaded on first use during training or inference\n- **Storage Location**: Models are saved to [`~\u002F.cache\u002Ffairseq2\u002Fassets\u002F`](https:\u002F\u002Ffacebookresearch.github.io\u002Ffairseq2\u002Fstable\u002Fbasics\u002Fassets.html#the-asset-store-system)\n\n\n### Architecture Documentation\n\nWe provide a high-level model architecture overview in the model directory ([`src\u002Fomnilingual_asr\u002Fmodels`](\u002Fsrc\u002Fomnilingual_asr\u002Fmodels)), with individual configurations for each model family in the respective directories:\n\n- **SSL Models**: [`src\u002Fomnilingual_asr\u002Fmodels\u002Fwav2vec2_ssl`](\u002Fsrc\u002Fomnilingual_asr\u002Fmodels\u002Fwav2vec2_ssl\u002F)\n- **CTC Models**: [`src\u002Fomnilingual_asr\u002Fmodels\u002Fwav2vec2_asr`](\u002Fsrc\u002Fomnilingual_asr\u002Fmodels\u002Fwav2vec2_asr\u002F)\n- **LLM Models**: [`src\u002Fomnilingual_asr\u002Fmodels\u002Fwav2vec2_llama`](\u002Fsrc\u002Fomnilingual_asr\u002Fmodels\u002Fwav2vec2_llama\u002F)\n\n## Training\n\nTo further finetune the released checkpoints on your own data, use our [data preparation guide](\u002Fworkflows\u002Fdataprep\u002FREADME.md) followed by the [finetuning recipe guide](\u002Fworkflows\u002Frecipes\u002Fwav2vec2\u002Fasr\u002FREADME.md).\n\n## License\n\nOmnilingual ASR code and models are released under the [Apache 2.0](.\u002FLICENSE).\n\n## Citation\n\nIf you use the omnilingual ASR model suite in your research and wish to cite us, please use the following BibTeX entry!\n```bibtex\n@misc{omnilingualasrteam2025omnilingualasropensourcemultilingual,\n      title={Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages},\n      author={Omnilingual ASR team and Gil Keren and Artyom Kozhevnikov and Yen Meng and Christophe Ropers and Matthew Setzler and Skyler Wang and Ife Adebara and Michael Auli and Can Balioglu and Kevin Chan and Chierh Cheng and Joe Chuang and Caley Droof and Mark Duppenthaler and Paul-Ambroise Duquenne and Alexander Erben and Cynthia Gao and Gabriel Mejia Gonzalez and Kehan Lyu and Sagar Miglani and Vineel Pratap and Kaushik Ram Sadagopan and Safiyyah Saleem and Arina Turkatenko and Albert Ventayol-Boada and Zheng-Xin Yong and Yu-An Chung and Jean Maillard and Rashel Moritz and Alexandre Mourachko and Mary Williamson and Shireen Yates},\n      year={2025},\n      eprint={2511.09690},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https:\u002F\u002Farxiv.org\u002Fabs\u002F2511.09690},\n}\n```\n","Omnilingual ASR 是一个支持超过1600种语言的开源多语言语音识别系统，包括许多之前未被任何ASR技术覆盖的语言。其核心功能在于通过少量配对示例即可添加新语言，无需专业技能或大量数据集，并结合了可扩展的零样本学习与灵活的模型家族，以实现更广泛的包容性和适应性。该系统特别适合用于那些资源有限、但需要高质量语音转文字服务的社区和研究场景中，如偏远地区或小语种研究领域。",2,"2026-06-11 03:41:53","high_star"]