[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72083":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":16,"starSnapshotCount":16,"syncStatus":29,"lastSyncTime":30,"discoverSource":31},72083,"parler-tts","huggingface\u002Fparler-tts","huggingface","Inference and training library for high-quality TTS models.","",null,"Python",5576,590,56,116,0,1,4,3,39.31,"Apache License 2.0",false,"main",true,[],"2026-06-12 02:02:58","# Parler-TTS\n\nParler-TTS is a lightweight text-to-speech (TTS) model that can generate high-quality, natural sounding speech in the style of a given speaker (gender, pitch, speaking style, etc). It is a reproduction of work from the paper [Natural language guidance of high-fidelity text-to-speech with synthetic annotations](https:\u002F\u002Fwww.text-description-to-speech.com) by Dan Lyth and Simon King, from Stability AI and Edinburgh University respectively.\n\nContrarily to other TTS models, Parler-TTS is a **fully open-source** release. All of the datasets, pre-processing, training code and weights are released publicly under permissive license, enabling the community to build on our work and develop their own powerful TTS models.\n\nThis repository contains the inference and training code for Parler-TTS. It is designed to accompany the [Data-Speech](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fdataspeech) repository for dataset annotation.\n\n> [!IMPORTANT]\n> **08\u002F08\u002F2024:** We are proud to release two new Parler-TTS checkpoints:\n> 1. [Parler-TTS Mini](https:\u002F\u002Fhuggingface.co\u002Fparler-tts\u002Fparler-tts-mini-v1), an 880M parameter model.\n> 2. [Parler-TTS Large](https:\u002F\u002Fhuggingface.co\u002Fparler-tts\u002Fparler-tts-large-v1), a 2.3B parameter model.\n>\n> These checkpoints have been trained on 45k hours of audiobook data.\n>\n> In addition, the code is optimized for much faster generation: we've added SDPA and Flash Attention 2 compatibility, as well as the ability to compile the model.\n\n## 📖 Quick Index\n* [Installation](#installation)\n* [Usage](#usage)\n  - [🎲 Using a random voice](#-random-voice)\n  - [🎯 Using a specific speaker](#-using-a-specific-speaker)\n* [Training](#training)\n* [Demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fparler-tts\u002Fparler_tts)\n* [Model weights and datasets](https:\u002F\u002Fhuggingface.co\u002Fparler-tts)\n* [Optimizing inference](#-optimizing-inference-speed)\n\n## Installation\n\nParler-TTS has light-weight dependencies and can be installed in one line:\n\n```sh\npip install git+https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fparler-tts.git\n```\n\nApple Silicon users will need to run a follow-up command to make use the nightly PyTorch (2.4) build for bfloat16 support:\n\n```sh\npip3 install --pre torch torchaudio --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fnightly\u002Fcpu\n```\n\n## Usage\n\n> [!TIP]\n> You can directly try it out in an interactive demo [here](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fparler-tts\u002Fparler_tts)!\n\nUsing Parler-TTS is as simple as \"bonjour\". Simply install the library once:\n\n```sh\npip install git+https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fparler-tts.git\n```\n\n### 🎲 Random voice\n\n\n**Parler-TTS** has been trained to generate speech with features that can be controlled with a simple text prompt, for example:\n\n```py\nimport torch\nfrom parler_tts import ParlerTTSForConditionalGeneration\nfrom transformers import AutoTokenizer\nimport soundfile as sf\n\ndevice = \"cuda:0\" if torch.cuda.is_available() else \"cpu\"\n\nmodel = ParlerTTSForConditionalGeneration.from_pretrained(\"parler-tts\u002Fparler-tts-mini-v1\").to(device)\ntokenizer = AutoTokenizer.from_pretrained(\"parler-tts\u002Fparler-tts-mini-v1\")\n\nprompt = \"Hey, how are you doing today?\"\ndescription = \"A female speaker delivers a slightly expressive and animated speech with a moderate speed and pitch. The recording is of very high quality, with the speaker's voice sounding clear and very close up.\"\n\ninput_ids = tokenizer(description, return_tensors=\"pt\").input_ids.to(device)\nprompt_input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(device)\n\ngeneration = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)\naudio_arr = generation.cpu().numpy().squeeze()\nsf.write(\"parler_tts_out.wav\", audio_arr, model.config.sampling_rate)\n```\n\n### 🎯 Using a specific speaker\n\nTo ensure speaker consistency across generations, this checkpoint was also trained on 34 speakers, characterized by name. The full list of available speakers includes:\nLaura, Gary, Jon, Lea, Karen, Rick, Brenda, David, Eileen, Jordan, Mike, Yann, Joy, James, Eric, Lauren, Rose, Will, Jason, Aaron, Naomie, Alisa, Patrick, Jerry, Tina, Jenna, Bill, Tom, Carol, Barbara, Rebecca, Anna, Bruce, Emily.\n\nTo take advantage of this, simply adapt your text description to specify which speaker to use: `Jon's voice is monotone yet slightly fast in delivery, with a very close recording that almost has no background noise.`\n\nYou can replace \"Jon\" with any of the names from the list above to utilize different speaker characteristics. Each speaker has unique vocal qualities that can be leveraged to suit your specific needs. For more detailed information on speaker performance with voice consistency, please refer [inference guide](INFERENCE.md#speaker-consistency).\n\n```py\nimport torch\nfrom parler_tts import ParlerTTSForConditionalGeneration\nfrom transformers import AutoTokenizer\nimport soundfile as sf\n\ndevice = \"cuda:0\" if torch.cuda.is_available() else \"cpu\"\n\nmodel = ParlerTTSForConditionalGeneration.from_pretrained(\"parler-tts\u002Fparler-tts-mini-v1\").to(device)\ntokenizer = AutoTokenizer.from_pretrained(\"parler-tts\u002Fparler-tts-mini-v1\")\n\nprompt = \"Hey, how are you doing today?\"\ndescription = \"Jon's voice is monotone yet slightly fast in delivery, with a very close recording that almost has no background noise.\"\n\ninput_ids = tokenizer(description, return_tensors=\"pt\").input_ids.to(device)\nprompt_input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(device)\n\ngeneration = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)\naudio_arr = generation.cpu().numpy().squeeze()\nsf.write(\"parler_tts_out.wav\", audio_arr, model.config.sampling_rate)\n```\n\n**Tips**:\n* Include the term \"very clear audio\" to generate the highest quality audio, and \"very noisy audio\" for high levels of background noise\n* Punctuation can be used to control the prosody of the generations, e.g. use commas to add small breaks in speech\n* The remaining speech features (gender, speaking rate, pitch and reverberation) can be controlled directly through the prompt\n\n### ✨ Optimizing Inference Speed\n\nWe've set up an [inference guide](INFERENCE.md) to make generation faster. Think SDPA, torch.compile and streaming!\n\n\nhttps:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fparler-tts\u002Fassets\u002F52246514\u002F251e2488-fe6e-42c1-81cd-814c5b7795b0\n\n## Training\n\n\u003Ca target=\"_blank\" href=\"https:\u002F\u002Fgithub.com\u002Fylacombe\u002Fscripts_and_notebooks\u002Fblob\u002Fmain\u002FFinetuning_Parler_TTS_v1_on_a_single_speaker_dataset.ipynb\"> \n  \u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" alt=\"Open In Colab\"\u002F> \n\u003C\u002Fa>\n\nThe [training folder](\u002Ftraining\u002F) contains all the information to train or fine-tune your own Parler-TTS model. It consists of:\n- [1. An introduction to the Parler-TTS architecture](\u002Ftraining\u002FREADME.md#1-architecture)\n- [2. The first steps to get started](\u002Ftraining\u002FREADME.md#2-getting-started)\n- [3. A training guide](\u002Ftraining\u002FREADME.md#3-training)\n\n> [!IMPORTANT]\n> **TL;DR:** After having followed the [installation steps](\u002Ftraining\u002FREADME.md#requirements), you can reproduce the Parler-TTS Mini v1 training recipe with the following command line:\n\n```sh\naccelerate launch .\u002Ftraining\u002Frun_parler_tts_training.py .\u002Fhelpers\u002Ftraining_configs\u002Fstarting_point_v1.json\n```\n\n> [!IMPORTANT]\n> You can also follow [this fine-tuning guide](https:\u002F\u002Fgithub.com\u002Fylacombe\u002Fscripts_and_notebooks\u002Fblob\u002Fmain\u002FFinetuning_Parler_TTS_v1_on_a_single_speaker_dataset.ipynb) on a mono-speaker dataset example.\n\n## Acknowledgements\n\nThis library builds on top of a number of open-source giants, to whom we'd like to extend our warmest thanks for providing these tools!\n\nSpecial thanks to:\n- Dan Lyth and Simon King, from Stability AI and Edinburgh University respectively, for publishing such a promising and clear research paper: [Natural language guidance of high-fidelity text-to-speech with synthetic annotations](https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.01912).\n- the many libraries used, namely [🤗 datasets](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fdatasets\u002Fv2.17.0\u002Fen\u002Findex), [🤗 accelerate](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Faccelerate\u002Fen\u002Findex), [jiwer](https:\u002F\u002Fgithub.com\u002Fjitsi\u002Fjiwer), [wandb](https:\u002F\u002Fwandb.ai\u002F), and [🤗 transformers](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Ftransformers\u002Findex).\n- Descript for the [DAC codec model](https:\u002F\u002Fgithub.com\u002Fdescriptinc\u002Fdescript-audio-codec)\n- Hugging Face 🤗 for providing compute resources and time to explore!\n\n\n## Citation\n\nIf you found this repository useful, please consider citing this work and also the original Stability AI paper:\n\n```\n@misc{lacombe-etal-2024-parler-tts,\n  author = {Yoach Lacombe and Vaibhav Srivastav and Sanchit Gandhi},\n  title = {Parler-TTS},\n  year = {2024},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fparler-tts}}\n}\n```\n\n```\n@misc{lyth2024natural,\n      title={Natural language guidance of high-fidelity text-to-speech with synthetic annotations},\n      author={Dan Lyth and Simon King},\n      year={2024},\n      eprint={2402.01912},\n      archivePrefix={arXiv},\n      primaryClass={cs.SD}\n}\n```\n\n## Contribution\n\nContributions are welcome, as the project offers many possibilities for improvement and exploration.\n\nNamely, we're looking at ways to improve both quality and speed:\n- Datasets:\n    - Train on more data\n    - Add more features such as accents\n- Training:\n    - Add PEFT compatibility to do Lora fine-tuning.\n    - Add possibility to train without description column.\n    - Add notebook training.\n    - Explore multilingual training.\n    - Explore mono-speaker finetuning.\n    - Explore more architectures.\n- Optimization:\n    - Compilation and static cache\n    - Support to FA2 and SDPA\n- Evaluation:\n    - Add more evaluation metrics\n\n","Parler-TTS 是一个轻量级的文本转语音（TTS）模型，能够生成高质量、自然的声音，并模仿特定说话人的风格（如性别、音调、说话方式等）。该项目的核心功能包括完全开源的训练代码和数据集，支持用户自定义开发强大的TTS模型。技术特点上，它兼容SDPA和Flash Attention 2，优化了生成速度，并且提供了不同规模的预训练模型（880M参数的小型模型和2.3B参数的大型模型），这些模型基于45,000小时的有声书数据训练而成。Parler-TTS适用于需要高质量语音合成的应用场景，例如虚拟助手、有声读物制作或任何需要自然语音输出的项目。",2,"2026-06-11 03:40:16","high_star"]