[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-74074":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},74074,"heartlib","HeartMuLa\u002Fheartlib","HeartMuLa","HeartMuLa Official Repo: The Most Powerful Open-Source Music Generation Model of 2026","",null,"Python",3671,408,62,75,0,14,45,109,42,109.84,"Apache License 2.0",false,"main",true,[],"2026-06-12 04:01:13","\u003Cp align=\"center\">\n    \u003Cpicture>\n        \u003Csource srcset=\".\u002Fassets\u002Flogo.png\" media=\"(prefers-color-scheme: dark)\">\n        \u003Cimg src=\".\u002Fassets\u002Flogo.png\" width=\"30%\">\n    \u003C\u002Fpicture>\n    \n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fheartmula.github.io\u002F\">Demo 🎶\u003C\u002Fa>  |  📑 \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2601.10547\">Paper\u003C\u002Fa>\n    \u003Cbr>\n    \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002FHeartMuLa\u002FHeartMuLa-oss-3B-happy-new-year\">HeartMuLa-oss-3B-happy-new-year 🤗\u003C\u002Fa>  |  \u003Ca href=\"https:\u002F\u002Fmodelscope.cn\u002Fmodels\u002FHeartMuLa\u002FHeartMuLa-oss-3B-happy-new-year\">HeartMuLa-oss-3B-happy-new-year \u003Cpicture>\n        \u003Csource srcset=\".\u002Fassets\u002Fbadge.svg\" media=\"(prefers-color-scheme: dark)\">\n        \u003Cimg src=\".\u002Fassets\u002Fbadge.svg\" width=\"20px\">\n    \u003C\u002Fpicture>\u003C\u002Fa>\n    \u003Cbr>\n    \n    \n\u003C\u002Fp>\n\n---\n# HeartMuLa: A Family of Open Sourced Music Foundation Models\n\nHeartMuLa is a family of open sourced music foundation models including: \n1. HeartMuLa: a music language model that generates music conditioned on lyrics and tags with multilingual support covering almost all languages.\n2. HeartCodec: a 12.5 hz music codec with high reconstruction fidelity;\n3. HeartTranscriptor: a whisper-based model specifically tuned for lyrics transcription; Check [this page](.\u002Fexamples\u002FREADME.md) for its usage.\n4. HeartCLAP: an audio–text alignment model that establishes a unified embedding space for music descriptions and cross-modal retrieval.\n---\n\n\nBelow shows the experiment result of our oss-3B version compared with other baselines.\n\u003Cp align=\"center\">\n    \u003Cpicture>\n        \u003Csource srcset=\".\u002Fassets\u002Fexp-new.png\" media=\"(prefers-color-scheme: dark)\">\n        \u003Cimg src=\".\u002Fassets\u002Fexp-new.png\" width=\"90%\">\n    \u003C\u002Fpicture>\n    \n\u003C\u002Fp>\n\n---\n\n## 🔥 Highlight\n\nOur latest internal version of HeartMuLa-7B achieves **comparable performance with Suno** in terms of musicality, fidelity and controllability. \n\n## 📰 News\nJoin on Discord! [\u003Cimg alt=\"join discord\" src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F842440537755353128?color=%237289da&logo=discord\"\u002F>](https:\u002F\u002Fdiscord.gg\u002FrkC4VmpH)\n\n- 🚀 **10 Apr. 2026**\n\n  We launched online demo spaces on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FHeartMuLa\u002Fheartmula) and [ModelScope](https:\u002F\u002Fwww.modelscope.cn\u002Fstudios\u002FHeartMuLa\u002Fheartmula\u002F).\n\n- 🚀 **13 Feb. 2026**\n\n  We released our **HeartMuLa-oss-3B-happy-new-year** version. This version is currently the best open-sourced model in terms of lyrics controllability and music quality. We recommend using **HeartMuLa-oss-3B-happy-new-year** and **HeartCodec-oss-20260123** for music generation.\n\n- ⚖️ **03 Feb. 2026**\n\n  We have released our [HeartMuLa-Benchmark](https:\u002F\u002Fmodelscope.cn\u002Fdatasets\u002FHeartMuLa\u002FHeartMuLa-Benchmark) (referred to as **HeartBeats Benchmark** in our paper) as introduced in our [paper](https:\u002F\u002Farxiv.org\u002Fpdf\u002F2601.10547). This benchmark comprises heterogeneous AI-generated lyrics and tags across diverse languages and genres, providing a rigorous and fair evaluation framework.\n  \n- 🚀 **23 Jan. 2026**\n\n    By leveraging Reinforcement Learning, we have continuously refined our model and are proud to officially release **HeartMuLa-RL-oss-3B-20260123**. This version is designed to achieve more precise control over styles and tags. Simultaneously, we are launching **HeartCodec-oss-20260123**, which optimizes audio decoding quality.\n\n- 🫶 **20 Jan. 2026** \n    \n    [Benji](https:\u002F\u002Fgithub.com\u002Fbenjiyaya) has created a wonderful [ComfyUI custom node](https:\u002F\u002Fgithub.com\u002Fbenjiyaya\u002FHeartMuLa_ComfyUI) for HeartMuLa. Thanks Benji!\n\n- ⚖️ **20 Jan. 2026** \n\n    License update: We update the license of this repo and all related model weights to **Apache 2.0**.\n\n- 🚀 **14 Jan. 2026**  \n    The official release of **HeartTranscriptor-oss** and the first **HeartMuLa-oss-3B** version along with our **HeartCodec-oss**.\n\n---\n## 🧭 TODOs\n\n- ⏳ Release scripts for inference acceleration and streaming inference. The current inference speed is around RTF $\\approx 1.0$.\n- ⏳ Support **reference audio conditioning**, **fine-grained controllable music generation**, **hot song generation**.\n- ⏳ Release the **HeartMuLa-oss-7B** version.\n- ✅ Release inference code and pretrained checkpoints of  \n  **HeartCodec-oss, HeartMuLa-oss-3B, and HeartTranscriptor-oss**.\n\n---\n\n## 🛠️ Local Deployment\n\n### ⚙️ Environment Setup\n\nWe recommend using `python=3.10` for local deployment.\n\nClone this repo and install locally.\n\n```\ngit clone https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib.git\ncd heartlib\npip install -e .\n```\n\nDownload our pretrained checkpoints from huggingface or modelscope using the following command:\n\n```\n# if you are using huggingface\nhf download --local-dir '.\u002Fckpt' 'HeartMuLa\u002FHeartMuLaGen'\nhf download --local-dir '.\u002Fckpt\u002FHeartMuLa-oss-3B' 'HeartMuLa\u002FHeartMuLa-oss-3B-happy-new-year'\nhf download --local-dir '.\u002Fckpt\u002FHeartCodec-oss' HeartMuLa\u002FHeartCodec-oss-20260123\n\n\n# if you are using modelscope\nmodelscope download --model 'HeartMuLa\u002FHeartMuLaGen' --local_dir '.\u002Fckpt'\nmodelscope download --model 'HeartMuLa\u002FHeartMuLa-oss-3B-happy-new-year' --local_dir '.\u002Fckpt\u002FHeartMuLa-oss-3B'\nmodelscope download --model 'HeartMuLa\u002FHeartCodec-oss-20260123' --local_dir '.\u002Fckpt\u002FHeartCodec-oss'\n\n```\n\nAfter downloading, the `.\u002Fckpt` subfolder should structure like this:\n```\n.\u002Fckpt\u002F\n├── HeartCodec-oss\u002F\n├── HeartMuLa-oss-3B\u002F\n├── gen_config.json\n└── tokenizer.json\n```\n\n\n### ▶️ Example Usage\n\nTo generate music, run:\n\n```\npython .\u002Fexamples\u002Frun_music_generation.py --model_path=.\u002Fckpt --version=\"3B\"\n```\n\nBy default this command will generate a piece of music conditioned on lyrics and tags provided in `.\u002Fassets` folder. The output music will be saved at `.\u002Fassets\u002Foutput.mp3`.\n\n#### FAQs\n\n1. How to specify lyrics and tags?\n\n    The model will load lyrics from the txt file `--lyrics` link to (by default `.\u002Fassets\u002Flyrics.txt`). If you would like to use your own lyrics, just modify the content in `.\u002Fassets\u002Flyrics.txt`. If you would like to save your lyrics to another path, e.g. `my_awesome_lyrics.txt`, remember to input arguments `--lyrics my_awesome_lyrics.txt`.\n\n    For tags it's basically the same.\n\n2. CUDA out of memory?\n\n    If you have multi-GPUs (e.g. 2 4090s), we recommend placing the params of HeartMuLa and HeartCodec separately on different devices. You can do it by typing `--mula_device cuda:0 --codec_device cuda:1`\n\n    If you are running on a single GPU, use `--lazy_load true` so that modules will be loaded on demand and deleted once inference completed to save GPU memory.\n\nAll parameters:\n\n- `--model_path` (required): Path to the pretrained model checkpoint\n- `--lyrics`: Path to lyrics file (default: `.\u002Fassets\u002Flyrics.txt`)\n- `--tags`: Path to tags file (default: `.\u002Fassets\u002Ftags.txt`)\n- `--save_path`: Output audio file path (default: `.\u002Fassets\u002Foutput.mp3`)\n- `--max_audio_length_ms`: Maximum audio length in milliseconds (default: 240000)\n- `--topk`: Top-k sampling parameter for generation (default: 50)\n- `--temperature`: Sampling temperature for generation (default: 1.0)\n- `--cfg_scale`: Classifier-free guidance scale (default: 1.5)\n- `--version`: The version of HeartMuLa, choose between [`3B`, `7B`]. (default: `3B`) # `7B` version not released yet.\n- `--mula_device\u002F--codec_device`: The device where params will be placed. Both are set to `cuda` by default. You can use `--mula_device cuda:0 --codec_device cuda:1` to explicitly place different modules to different devices.\n- `--mula_dtype\u002F--codec_dtype`: Inference dtype. By default is `bf16` for HeartMuLa and `fp32` for HeartCodec. Setting `bf16` for HeartCodec may result in the degradation of audio quality.\n- `--lazy_load`: Whether or not to use lazy loading (default: false). If turned on, modules will be loaded on demand to save GPU usage. \nRecommended format of lyrics and tags:\n```txt\n[Intro]\n\n[Verse]\nThe sun creeps in across the floor\nI hear the traffic outside the door\nThe coffee pot begins to hiss\nIt is another morning just like this\n\n[Prechorus]\nThe world keeps spinning round and round\nFeet are planted on the ground\nI find my rhythm in the sound\n\n[Chorus]\nEvery day the light returns\nEvery day the fire burns\nWe keep on walking down this street\nMoving to the same steady beat\nIt is the ordinary magic that we meet\n\n[Verse]\nThe hours tick deeply into noon\nChasing shadows,chasing the moon\nWork is done and the lights go low\nWatching the city start to glow\n\n[Bridge]\nIt is not always easy,not always bright\nSometimes we wrestle with the night\nBut we make it to the morning light\n\n[Chorus]\nEvery day the light returns\nEvery day the fire burns\nWe keep on walking down this street\nMoving to the same steady beat\n\n[Outro]\nJust another day\nEvery single day\n```\n\nRegarding tags, check this [issue](https:\u002F\u002Fgithub.com\u002FHeartMuLa\u002Fheartlib\u002Fissues\u002F17) for reference.\nOur different tags are comma-separated without spaces as illustrated below:\n```txt\npiano,happy,wedding,synthesizer,romantic\n```\n\n---\n\n\n## ⚖️ License\n\nThis repository is licensed under the Apache 2.0 License.\n\n---\n\n## 📚 Citation\n\n```\n@misc{yang2026heartmulafamilyopensourced,\n      title={HeartMuLa: A Family of Open Sourced Music Foundation Models}, \n      author={Dongchao Yang and Yuxin Xie and Yuguo Yin and Zheyu Wang and Xiaoyu Yi and Gongxi Zhu and Xiaolong Weng and Zihan Xiong and Yingzhe Ma and Dading Cong and Jingliang Liu and Zihang Huang and Jinghan Ru and Rongjie Huang and Haoran Wan and Peixu Wang and Kuoxi Yu and Helin Wang and Liming Liang and Xianwei Zhuang and Yuanyuan Wang and Haohan Guo and Junjie Cao and Zeqian Ju and Songxiang Liu and Yuewen Cao and Heming Weng and Yuexian Zou},\n      year={2026},\n      eprint={2601.10547},\n      archivePrefix={arXiv},\n      primaryClass={cs.SD},\n      url={https:\u002F\u002Farxiv.org\u002Fabs\u002F2601.10547}, \n}\n```\n\n## 📬 Contact\nIf you are interested in HeartMuLa, feel free to reach us at heartmula.ai@gmail.com\n\nWelcome to join us through [discord](https:\u002F\u002Fdiscord.gg\u002FBKXF5FgH) or Wechat group.\n\nScan the QR code on the left to join our Wechat group. If it expires, feel free to raise an issue to remind us of updating. \n\nIf the number of group members exceeds 200, joining the group via directly scanning the QR code is restricted by WeChat. In this case, scan our team member's QR code on the right and send a request writing **HeartMuLa Group Invite**. We will invite you into the group manually.\n\u003Cp align=\"center\">\n    \u003Cpicture>\n        \u003Csource srcset=\".\u002Fassets\u002Fgroup_wx.jpeg\" media=\"(prefers-color-scheme: dark)\">\n        \u003Cimg src=\".\u002Fassets\u002Fgroup_wx.jpeg\" width=\"40%\">\n    \u003C\u002Fpicture>\n    \u003Cpicture>\n        \u003Csource srcset=\".\u002Fassets\u002Flead_wx.jpeg\" media=\"(prefers-color-scheme: dark)\">\n        \u003Cimg src=\".\u002Fassets\u002Flead_wx.jpeg\" width=\"40%\">\n    \u003C\u002Fpicture>\n\u003C\u002Fp>\n","HeartMuLa 是一个开源音乐生成模型，旨在根据歌词和标签生成高质量的音乐，并支持多语言。项目包括四个核心组件：HeartMuLa 音乐语言模型、HeartCodec 高保真音乐编解码器、HeartTranscriptor 歌词转录模型以及 HeartCLAP 音频-文本对齐模型。这些组件共同提供了从音乐创作到转录的一系列功能，尤其在音乐生成的质量和可控性方面表现出色。适用于需要高质量音乐生成、音频处理及歌词转录的应用场景，如音乐制作软件、在线音乐平台等。",2,"2026-06-11 03:48:41","high_star"]