[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71994":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":18,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":22,"hasPages":22,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":38,"readmeContent":39,"aiSummary":40,"trendingCount":16,"starSnapshotCount":16,"syncStatus":15,"lastSyncTime":41,"discoverSource":42},71994,"SenseVoice","FunAudioLLM\u002FSenseVoice","FunAudioLLM","Multilingual speech understanding: ASR + emotion recognition + audio event detection. 50+ languages, 15x faster than Whisper, non-autoregressive.","https:\u002F\u002Ffunaudiollm.github.io\u002F",null,"Python",8522,778,63,2,0,57,171,391,114.67,"Other",false,"main",[25,26,27,28,29,30,31,32,33,34,35,36,37],"ai","aigc","asr","audio-event-classification","cross-lingual","gpt-4o","llm","multilingual","python","pytorch","speech-emotion-recognition","speech-recognition","speech-to-text","2026-06-12 04:01:03","([简体中文](.\u002FREADME_zh.md)|English|[日本語](.\u002FREADME_ja.md))\n\n\n# Introduction\n\nSenseVoice is a speech foundation model with multiple speech understanding capabilities, including automatic speech recognition (ASR),  spoken language identification (LID), speech emotion recognition (SER), and audio event detection (AED). \n\n\u003Cdiv align=\"center\">  \n\u003Cimg src=\"image\u002Fsensevoice2.png\">\n\u003C\u002Fdiv>\n\n[\u002F\u002F]: # (\u003Cdiv align=\"center\">\u003Cimg src=\"image\u002Fsensevoice.png\" width=\"700\"\u002F> \u003C\u002Fdiv>)\n\n\u003Cdiv align=\"center\">  \n\u003Ch4>\n\u003Ca href=\"https:\u002F\u002Ffunaudiollm.github.io\u002F\"> Homepage \u003C\u002Fa>\n｜\u003Ca href=\"#What's News\"> What's News \u003C\u002Fa>\n｜\u003Ca href=\"#Benchmarks\"> Benchmarks \u003C\u002Fa>\n｜\u003Ca href=\"#Install\"> Install \u003C\u002Fa>\n｜\u003Ca href=\"#Usage\"> Usage \u003C\u002Fa>\n｜\u003Ca href=\"#Community\"> Community \u003C\u002Fa>\n\u003C\u002Fh4>\n\nModel Zoo:\n[modelscope](https:\u002F\u002Fwww.modelscope.cn\u002Fmodels\u002Fiic\u002FSenseVoiceSmall), [huggingface](https:\u002F\u002Fhuggingface.co\u002FFunAudioLLM\u002FSenseVoiceSmall)\n\nOnline Demo:\n[modelscope demo](https:\u002F\u002Fwww.modelscope.cn\u002Fstudios\u002Fiic\u002FSenseVoice), [huggingface space](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FFunAudioLLM\u002FSenseVoice)\n\n\n\u003C\u002Fdiv>\n\n\n\u003Ca name=\"Highligts\">\u003C\u002Fa>\n# Highlights 🎯\n**SenseVoice** focuses on high-accuracy multilingual speech recognition, speech emotion recognition, and audio event detection.\n- **Multilingual Speech Recognition:** Trained with over 400,000 hours of data, supporting more than 50 languages, the recognition performance surpasses that of the Whisper model.\n- **Rich transcribe:** \n  - Possess excellent emotion recognition capabilities, achieving and surpassing the effectiveness of the current best emotion recognition models on test data.\n  - Offer sound event detection capabilities, supporting the detection of various common human-computer interaction events such as bgm, applause, laughter, crying, coughing, and sneezing.\n- **Efficient Inference:** The SenseVoice-Small model utilizes a non-autoregressive end-to-end framework, leading to exceptionally low inference latency. It requires only 70ms to process 10 seconds of audio, which is 15 times faster than Whisper-Large.\n- **Convenient Finetuning:** Provide convenient finetuning scripts and strategies, allowing users to easily address long-tail sample issues according to their business scenarios.\n- **Service Deployment:** Offer service deployment pipeline,  supporting multi-concurrent requests, with client-side languages including Python, C++, HTML, Java, and C#, among others.\n\n\u003Ca name=\"What's News\">\u003C\u002Fa>\n# What's New 🔥\n- 2024\u002F11: Add support for timestamp based on the CTC alignment.\n- 2024\u002F7: Added Export Features for [ONNX](.\u002Fdemo_onnx.py) and [libtorch](.\u002Fdemo_libtorch.py), as well as Python Version Runtimes: [funasr-onnx-0.4.0](https:\u002F\u002Fpypi.org\u002Fproject\u002Ffunasr-onnx\u002F), [funasr-torch-0.1.1](https:\u002F\u002Fpypi.org\u002Fproject\u002Ffunasr-torch\u002F)\n- 2024\u002F7: The [SenseVoice-Small](https:\u002F\u002Fwww.modelscope.cn\u002Fmodels\u002Fiic\u002FSenseVoiceSmall) voice understanding model is open-sourced, which offers high-precision multilingual speech recognition, emotion recognition, and audio event detection capabilities for Mandarin, Cantonese, English, Japanese, and Korean and leads to exceptionally low inference latency.  \n- 2024\u002F7: The CosyVoice for natural speech generation with multi-language, timbre, and emotion control. CosyVoice excels in multi-lingual voice generation, zero-shot voice generation, cross-lingual voice cloning, and instruction-following capabilities. [CosyVoice repo](https:\u002F\u002Fgithub.com\u002FFunAudioLLM\u002FCosyVoice) and [CosyVoice space](https:\u002F\u002Fwww.modelscope.cn\u002Fstudios\u002Fiic\u002FCosyVoice-300M).\n- 2024\u002F7: [FunASR](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002FFunASR) is a fundamental speech recognition toolkit that offers a variety of features, including speech recognition (ASR), Voice Activity Detection (VAD), Punctuation Restoration, Language Models, Speaker Verification, Speaker Diarization and multi-talker ASR.\n\n\u003Ca name=\"Benchmarks\">\u003C\u002Fa>\n# Benchmarks 📝\n\n## Multilingual Speech Recognition\nWe compared the performance of multilingual speech recognition between SenseVoice and Whisper on open-source benchmark datasets, including AISHELL-1, AISHELL-2, Wenetspeech, LibriSpeech, and Common Voice. In terms of Chinese and Cantonese recognition, the SenseVoice-Small model has advantages.\n\n\u003Cdiv align=\"center\">  \n\u003Cimg src=\"image\u002Fasr_results1.png\" width=\"400\" \u002F>\u003Cimg src=\"image\u002Fasr_results2.png\" width=\"400\" \u002F>\n\u003C\u002Fdiv>\n\n## Speech Emotion Recognition\n\nDue to the current lack of widely-used benchmarks and methods for speech emotion recognition, we conducted evaluations across various metrics on multiple test sets and performed a comprehensive comparison with numerous results from recent benchmarks. The selected test sets encompass data in both Chinese and English, and include multiple styles such as performances, films, and natural conversations. Without finetuning on the target data, SenseVoice was able to achieve and exceed the performance of the current best speech emotion recognition models.\n\n\u003Cdiv align=\"center\">  \n\u003Cimg src=\"image\u002Fser_table.png\" width=\"1000\" \u002F>\n\u003C\u002Fdiv>\n\nFurthermore, we compared multiple open-source speech emotion recognition models on the test sets, and the results indicate that the SenseVoice-Large model achieved the best performance on nearly all datasets, while the SenseVoice-Small model also surpassed other open-source models on the majority of the datasets.\n\n\u003Cdiv align=\"center\">  \n\u003Cimg src=\"image\u002Fser_figure.png\" width=\"500\" \u002F>\n\u003C\u002Fdiv>\n\n## Audio Event Detection\n\nAlthough trained exclusively on speech data, SenseVoice can still function as a standalone event detection model. We compared its performance on the environmental sound classification ESC-50 dataset against the widely used industry models BEATS and PANN. The SenseVoice model achieved commendable results on these tasks. However, due to limitations in training data and methodology, its event classification performance has some gaps compared to specialized AED models.\n\n\u003Cdiv align=\"center\">  \n\u003Cimg src=\"image\u002Faed_figure.png\" width=\"500\" \u002F>\n\u003C\u002Fdiv>\n\n## Computational  Efficiency\n\nThe SenseVoice-Small model deploys a non-autoregressive end-to-end architecture, resulting in extremely low inference latency. With a similar number of parameters to the Whisper-Small model, it infers more than 5 times faster than Whisper-Small and 15 times faster than Whisper-Large. \n\n\u003Cdiv align=\"center\">  \n\u003Cimg src=\"image\u002Finference.png\" width=\"1000\" \u002F>\n\u003C\u002Fdiv>\n\n\n# Requirements\n\n```shell\npip install -r requirements.txt\n```\n\n\u003Ca name=\"Usage\">\u003C\u002Fa>\n# Usage\n\n## Inference\n\nSupports input of audio in any format and of any duration.\n\n```python\nfrom funasr import AutoModel\nfrom funasr.utils.postprocess_utils import rich_transcription_postprocess\n\nmodel_dir = \"iic\u002FSenseVoiceSmall\"\n\n\nmodel = AutoModel(\n    model=model_dir,\n    trust_remote_code=True,\n    remote_code=\".\u002Fmodel.py\",    \n    vad_model=\"fsmn-vad\",\n    vad_kwargs={\"max_single_segment_time\": 30000},\n    device=\"cuda:0\",\n)\n\n# en\nres = model.generate(\n    input=f\"{model.model_path}\u002Fexample\u002Fen.mp3\",\n    cache={},\n    language=\"auto\",  # \"zh\", \"en\", \"yue\", \"ja\", \"ko\", \"nospeech\"\n    use_itn=True,\n    batch_size_s=60,\n    merge_vad=True,  #\n    merge_length_s=15,\n)\ntext = rich_transcription_postprocess(res[0][\"text\"])\nprint(text)\n```\n\n\u003Cdetails>\u003Csummary>Parameter Description (Click to Expand)\u003C\u002Fsummary>\n\n- `model_dir`: The name of the model, or the path to the model on the local disk.\n- `trust_remote_code`:\n  - When `True`, it means that the model's code implementation is loaded from `remote_code`, which specifies the exact location of the `model` code (for example, `model.py` in the current directory). It supports absolute paths, relative paths, and network URLs.\n  - When `False`, it indicates that the model's code implementation is the integrated version within [FunASR](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002FFunASR). At this time, modifications made to `model.py` in the current directory will not be effective, as the version loaded is the internal one from FunASR. For the model code, [click here to view](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002FFunASR\u002Ftree\u002Fmain\u002Ffunasr\u002Fmodels\u002Fsense_voice).\n- `vad_model`: This indicates the activation of VAD (Voice Activity Detection). The purpose of VAD is to split long audio into shorter clips. In this case, the inference time includes both VAD and SenseVoice total consumption, and represents the end-to-end latency. If you wish to test the SenseVoice model's inference time separately, the VAD model can be disabled.\n- `vad_kwargs`: Specifies the configurations for the VAD model. `max_single_segment_time`: denotes the maximum duration for audio segmentation by the `vad_model`, with the unit being milliseconds (ms).\n- `use_itn`: Whether the output result includes punctuation and inverse text normalization.\n- `batch_size_s`: Indicates the use of dynamic batching, where the total duration of audio in the batch is measured in seconds (s).\n- `merge_vad`: Whether to merge short audio fragments segmented by the VAD model, with the merged length being `merge_length_s`, in seconds (s).\n- `ban_emo_unk`: Whether to ban the output of the `emo_unk` token.\n\u003C\u002Fdetails>\n\nIf all inputs are short audios (\u003C30s), and batch inference is needed to speed up inference efficiency, the VAD model can be removed, and `batch_size` can be set accordingly.\n```python\nmodel = AutoModel(model=model_dir, trust_remote_code=True, device=\"cuda:0\")\n\nres = model.generate(\n    input=f\"{model.model_path}\u002Fexample\u002Fen.mp3\",\n    cache={},\n    language=\"zh\", # \"zh\", \"en\", \"yue\", \"ja\", \"ko\", \"nospeech\"\n    use_itn=False,\n    batch_size=64, \n)\n```\n\nFor more usage, please refer to [docs](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002FFunASR\u002Fblob\u002Fmain\u002Fdocs\u002Ftutorial\u002FREADME.md)\n\n### Inference directly\n\nSupports input of audio in any format, with an input duration limit of 30 seconds or less.\n\n```python\nfrom model import SenseVoiceSmall\nfrom funasr.utils.postprocess_utils import rich_transcription_postprocess\n\nmodel_dir = \"iic\u002FSenseVoiceSmall\"\nm, kwargs = SenseVoiceSmall.from_pretrained(model=model_dir, device=\"cuda:0\")\nm.eval()\n\nres = m.inference(\n    data_in=f\"{kwargs['model_path']}\u002Fexample\u002Fen.mp3\",\n    language=\"auto\", # \"zh\", \"en\", \"yue\", \"ja\", \"ko\", \"nospeech\"\n    use_itn=False,\n    ban_emo_unk=False,\n    **kwargs,\n)\n\ntext = rich_transcription_postprocess(res[0][0][\"text\"])\nprint(text)\n```\n\n### Export and Test\n\u003Cdetails>\u003Csummary>ONNX and Libtorch Export\u003C\u002Fsummary>\n\n#### ONNX\n```python\n# pip3 install -U funasr funasr-onnx\nfrom pathlib import Path\nfrom funasr_onnx import SenseVoiceSmall\nfrom funasr_onnx.utils.postprocess_utils import rich_transcription_postprocess\n\n\nmodel_dir = \"iic\u002FSenseVoiceSmall\"\n\nmodel = SenseVoiceSmall(model_dir, batch_size=10, quantize=True)\n\n# inference\nwav_or_scp = [\"{}\u002F.cache\u002Fmodelscope\u002Fhub\u002F{}\u002Fexample\u002Fen.mp3\".format(Path.home(), model_dir)]\n\nres = model(wav_or_scp, language=\"auto\", use_itn=True)\nprint([rich_transcription_postprocess(i) for i in res])\n```\nNote: ONNX model is exported to the original model directory.\n\n#### Libtorch\n```python\nfrom pathlib import Path\nfrom funasr_torch import SenseVoiceSmall\nfrom funasr_torch.utils.postprocess_utils import rich_transcription_postprocess\n\n\nmodel_dir = \"iic\u002FSenseVoiceSmall\"\n\nmodel = SenseVoiceSmall(model_dir, batch_size=10, device=\"cuda:0\")\n\nwav_or_scp = [\"{}\u002F.cache\u002Fmodelscope\u002Fhub\u002F{}\u002Fexample\u002Fen.mp3\".format(Path.home(), model_dir)]\n\nres = model(wav_or_scp, language=\"auto\", use_itn=True)\nprint([rich_transcription_postprocess(i) for i in res])\n```\nNote: Libtorch model is exported to the original model directory.\n\u003C\u002Fdetails>\n\n## Service\n\n### Deployment with FastAPI\n```shell\nexport SENSEVOICE_DEVICE=cuda:0\nfastapi run --port 50000\n```\n\n## Finetune\n\n### Requirements\n\n```shell\ngit clone https:\u002F\u002Fgithub.com\u002Falibaba\u002FFunASR.git && cd FunASR\npip3 install -e .\u002F\n```\n## 🐳 Docker Support\n\nSenseVoice can be built and run using Docker to simplify setup, ensure reproducibility, and support both CPU and GPU inference.\n\n### Build with Docker\n```bash\ndocker build -t sensevoice .\n```\n\n### Run (GPU – default)\n```bash\ndocker run --gpus all -p 50000:50000 sensevoice\n```\n### Run (CPU-only)\n```bash\ndocker run -e SENSEVOICE_DEVICE=cpu -p 50000:50000 sensevoice\n```\n### Docker Compose\nDocker Compose provides an easier way to run SenseVoice with persistent model caching, networking etc. \n\n### Start Stack\n```bash\ndocker compose up --build\n```\n### Data prepare\n\nData examples\n\n```text\n{\"key\": \"YOU0000008470_S0000238_punc_itn\", \"text_language\": \"\u003C|en|>\", \"emo_target\": \"\u003C|NEUTRAL|>\", \"event_target\": \"\u003C|Speech|>\", \"with_or_wo_itn\": \"\u003C|withitn|>\", \"target\": \"Including legal due diligence, subscription agreement, negotiation.\", \"source\": \"\u002Fcpfs01\u002Fshared\u002FGroup-speech\u002Fbeinian.lzr\u002Fdata\u002Findustrial_data\u002Fenglish_all\u002Faudio\u002FYOU0000008470_S0000238.wav\", \"target_len\": 7, \"source_len\": 140}\n{\"key\": \"AUD0000001556_S0007580\", \"text_language\": \"\u003C|en|>\", \"emo_target\": \"\u003C|NEUTRAL|>\", \"event_target\": \"\u003C|Speech|>\", \"with_or_wo_itn\": \"\u003C|woitn|>\", \"target\": \"there is a tendency to identify the self or take interest in what one has got used to\", \"source\": \"\u002Fcpfs01\u002Fshared\u002FGroup-speech\u002Fbeinian.lzr\u002Fdata\u002Findustrial_data\u002Fenglish_all\u002Faudio\u002FAUD0000001556_S0007580.wav\", \"target_len\": 18, \"source_len\": 360}\n```\n\nFull ref to `data\u002Ftrain_example.jsonl`\n\n\u003Cdetails>\u003Csummary>Data Prepare Details\u003C\u002Fsummary>\n\nDescription：\n- `key`: audio file unique ID\n- `source`：path to the audio file\n- `source_len`：number of fbank frames of the audio file\n- `target`：transcription\n- `target_len`：length of target\n- `text_language`：language id of the audio file\n- `emo_target`：emotion label of the audio file\n- `event_target`：event label of the audio file\n- `with_or_wo_itn`：whether includes punctuation and inverse text normalization\n\n\n`train_text.txt`\n\n\n```bash\nBAC009S0764W0121 甚至出现交易几乎停滞的情况\nBAC009S0916W0489 湖北一公司以员工名义贷款数十员工负债千万\nasr_example_cn_en 所有只要处理 data 不管你是做 machine learning 做 deep learning 做 data analytics 做 data science 也好 scientist 也好通通都要都做的基本功啊那 again 先先对有一些>也许对\nID0012W0014 he tried to think how it could be\n```\n\n`train_wav.scp`\n\n\n\n```bash\nBAC009S0764W0121 https:\u002F\u002Fisv-data.oss-cn-hangzhou.aliyuncs.com\u002Fics\u002FMaaS\u002FASR\u002Ftest_audio\u002FBAC009S0764W0121.wav\nBAC009S0916W0489 https:\u002F\u002Fisv-data.oss-cn-hangzhou.aliyuncs.com\u002Fics\u002FMaaS\u002FASR\u002Ftest_audio\u002FBAC009S0916W0489.wav\nasr_example_cn_en https:\u002F\u002Fisv-data.oss-cn-hangzhou.aliyuncs.com\u002Fics\u002FMaaS\u002FASR\u002Ftest_audio\u002Fasr_example_cn_en.wav\nID0012W0014 https:\u002F\u002Fisv-data.oss-cn-hangzhou.aliyuncs.com\u002Fics\u002FMaaS\u002FASR\u002Ftest_audio\u002Fasr_example_en.wav\n```\n\n`train_text_language.txt`\n\nThe language ids include `\u003C|zh|>`、`\u003C|en|>`、`\u003C|yue|>`、`\u003C|ja|>` and `\u003C|ko|>`.\n\n```bash\nBAC009S0764W0121 \u003C|zh|>\nBAC009S0916W0489 \u003C|zh|>\nasr_example_cn_en \u003C|zh|>\nID0012W0014 \u003C|en|>\n```\n\n`train_emo.txt`\n\nThe emotion labels include`\u003C|HAPPY|>`、`\u003C|SAD|>`、`\u003C|ANGRY|>`、`\u003C|NEUTRAL|>`、`\u003C|FEARFUL|>`、`\u003C|DISGUSTED|>` and `\u003C|SURPRISED|>`.\n\n```bash\nBAC009S0764W0121 \u003C|NEUTRAL|>\nBAC009S0916W0489 \u003C|NEUTRAL|>\nasr_example_cn_en \u003C|NEUTRAL|>\nID0012W0014 \u003C|NEUTRAL|>\n```\n\n`train_event.txt`\n\nThe event labels include`\u003C|BGM|>`、`\u003C|Speech|>`、`\u003C|Applause|>`、`\u003C|Laughter|>`、`\u003C|Cry|>`、`\u003C|Sneeze|>`、`\u003C|Breath|>` and `\u003C|Cough|>`.\n\n```bash\nBAC009S0764W0121 \u003C|Speech|>\nBAC009S0916W0489 \u003C|Speech|>\nasr_example_cn_en \u003C|Speech|>\nID0012W0014 \u003C|Speech|>\n```\n\n`Command`\n```shell\n# generate train.jsonl and val.jsonl from wav.scp, text.txt, text_language.txt, emo_target.txt, event_target.txt\nsensevoice2jsonl \\\n++scp_file_list='[\"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain_wav.scp\", \"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain_text.txt\", \"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain_text_language.txt\", \"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain_emo.txt\", \"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain_event.txt\"]' \\\n++data_type_list='[\"source\", \"target\", \"text_language\", \"emo_target\", \"event_target\"]' \\\n++jsonl_file_out=\"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain.jsonl\"\n```\n\nIf there is no `train_text_language.txt`, `train_emo_target.txt` and `train_event_target.txt`, the language, emotion and event label will be predicted automatically by using the `SenseVoice` model.\n```shell\n# generate train.jsonl and val.jsonl from wav.scp and text.txt\nsensevoice2jsonl \\\n++scp_file_list='[\"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain_wav.scp\", \"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain_text.txt\"]' \\\n++data_type_list='[\"source\", \"target\"]' \\\n++jsonl_file_out=\"..\u002F..\u002F..\u002Fdata\u002Flist\u002Ftrain.jsonl\" \\\n++model_dir='iic\u002FSenseVoiceSmall'\n```\n\u003C\u002Fdetails>\n\n### Finetune\n\nEnsure to modify the train_tool in finetune.sh to the absolute path of `funasr\u002Fbin\u002Ftrain_ds.py` from the FunASR installation directory you have set up earlier.\n\n```shell\nbash finetune.sh\n```\n\n## WebUI\n\n```shell\npython webui.py\n```\n\n\u003Cdiv align=\"center\">\u003Cimg src=\"image\u002Fwebui.png\" width=\"700\"\u002F> \u003C\u002Fdiv>\n\n\n## Remarkable Third-Party Work\n- Triton (GPU) Deployment Best Practices: Using Triton + TensorRT, tested with FP32, achieving an acceleration ratio of 526 on V100 GPU. FP16 support is in progress. [Repository](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002FFunASR\u002Fblob\u002Fmain\u002Fruntime\u002Ftriton_gpu\u002FREADME.md)\n- Sherpa-onnx Deployment Best Practices: Supports using SenseVoice in 10 programming languages: C++, C, Python, C#, Go, Swift, Kotlin, Java, JavaScript, and Dart. Also supports deploying SenseVoice on platforms like iOS, Android, and Raspberry Pi. [Repository](https:\u002F\u002Fk2-fsa.github.io\u002Fsherpa\u002Fonnx\u002Fsense-voice\u002Findex.html)\n- [SenseVoice.cpp](https:\u002F\u002Fgithub.com\u002Flovemefan\u002FSenseVoice.cpp). Inference of SenseVoice in pure C\u002FC++ based on GGML, supporting 3-bit, 4-bit, 5-bit, 8-bit quantization, etc. with no third-party dependencies.\n- [streaming-sensevoice](https:\u002F\u002Fgithub.com\u002Fpengzhendong\u002Fstreaming-sensevoice) processes inference in chunks. To achieve pseudo-streaming, it employs a truncated attention mechanism, sacrificing some accuracy. Additionally, this technology supports CTC prefix beam search and hot-word boosting features.\n- [OmniSenseVoice](https:\u002F\u002Fgithub.com\u002Flifeiteng\u002FOmniSenseVoice) is optimized for lightning-fast inference and batching process. \n- [SenseVoice Hotword](https:\u002F\u002Fwww.modelscope.cn\u002Fmodels\u002Fdengcunqin\u002FSenseVoiceSmall_hotword)，Neural Network Hotword Enhancement，[Contextualized End-to-End Speech Recognition with Contextual Phrase Prediction Network](https:\u002F\u002Fmp.weixin.qq.com\u002Fs\u002F1QkIvh8j7rrUjRyWOgAvdA)。\n\u003Ca name=\"Community\">\u003C\u002Fa>\n# Community\nIf you encounter problems in use, you can directly raise Issues on the github page.\n\nYou can also scan the following DingTalk group QR code to join the community group for communication and discussion.\n\n|                          FunASR                          |\n|:--------------------------------------------------------:|\n| \u003Cimg src=\"image\u002Fdingding_funasr.png\" width=\"250\"\u002F>\u003C\u002Fdiv> |\n\n\n","SenseVoice 是一个多语言语音理解模型，具备自动语音识别（ASR）、语言识别（LID）、语音情感识别（SER）和音频事件检测（AED）等功能。该模型使用超过40万小时的数据进行训练，支持50多种语言，并在多语言语音识别性能上超越了Whisper模型。此外，SenseVoice还拥有出色的情感识别能力，能够检测背景音乐、掌声、笑声等常见的人机交互事件。其小规模版本采用非自回归端到端框架，实现了极低的推理延迟，处理10秒音频仅需70毫秒。项目提供了便捷的微调脚本和服务部署管道，适用于需要高精度多语言语音处理及情感分析的应用场景，如客户服务、智能助手等。","2026-06-11 03:39:53","high_star"]