[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-79475":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},79475,"avtr-1","avaturn-live\u002Favtr-1","avaturn-live","AVTR-1: Avatars that listen back","",null,"Python",328,52,6,1,0,4,28,273,15,5.17,"Other",false,"main",true,[],"2026-06-12 02:03:54","\u003Cdiv align=\"center\">\n\n# AVTR-1\n\n[![Project Page](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject%20Page-AVTR--1-8A2BE2)](https:\u002F\u002Favaturn-live.github.io\u002Favtr-1-projectpage\u002F)\n[![Managed API](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FManaged%20API-avaturn.live-blue?logo=data:image\u002Fpng;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAQAAADZc7J\u002FAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAACYktHRAD\u002Fh4\u002FMvwAAAAd0SU1FB+oFFA8fAzOfxeUAAAOsSURBVEjHfZVPSFxXFMa\u002F+wYpJGqofxCrhBKcpBMTozGRdFNCSDYtpRFCFu0mWYRgNor54za7JILgspa6SALZl9JCVgndFGqqVWmpmkKjUWGCCiGOzrx376+L++Y5o5Oe1TDvfueec+53vs+oJDAKjJVo1ef6TCf0keokbehfzeg3\u002FWCWJVJyBlUKAkmig+\u002FZoFJkGaN75+RueEpiP8PkAbBEOJzDgcMSYQHYYoQaf3ovPMMkAKFHRcnVMdoRAjBFelcKUhI9LHmwgwS8RS5JAg7rbAgs0lOSgkCijRUoRDuHs9ylnQCRYSjO7aMQASsciWeBIaCaaY+z5AiBpzQgTnKDPk4hGnkKhORwxRtm2EeA8eWPAAXLAw4hWrlKLZ1MJDOYoYMa+jmISDNKwZczLBEIQyc57KbrRZzhJhcQ4jqL5LE4tnnFFYQ4Rz9diF62HZYCGT+DMSAaRdwD4DHiBEIcIMMxqlH8zyMA7iHu+zbGJUOL+yOot0qbek0IvdVhNep3LehHzWpVTq06ri+U1mm90Z+ql\u002FSpVvWSlOFtcFT0AXYTMUAEDCKeVSTic8QdIOIW4h1Y4Jp4CEQRTZwF5qjiYkydEIvDxb8scAnDHHCeBkLfxBMxCaGF24gXfEOKl7iYt6VhccxTxWUmEQNA6IBZse5ryVJHK2KwhIflEQFDiCaaWcajWNPOx28Rn5DDebJUrGGDDOK7kktKFjOQZJSXVHndjaQPhCRXtodrvphlmmlCDP1vC4OIFurIFltYF7N+HAOISS5Txfx7h\u002FgPKb7mBeI2fvRMiSdAFNLAeWAOwyXA7nnGCLhIFX8DZ2ki8iU9FNcA+w5xiwi4g3hesYVnyRsNIDZ9D32GFveXqbW0mWb9KmlN7WrUhBb0k2b1WoGadVxfKq1uvdG8amV0WmtaICW3FnRKYhyI7vsVAR4ly1TNMTIcSJbpcbJMo76BMf8OGQrYbdeL6KKfcwhxhVds47DkWeQ6QlzgJmcQvWw6LDm6MF7QhoGwwChpxEH6qaGDmaT7CTqp5SqtiEM8wEIBGJFIeUnbx4x\u002FaJdIWiPiFH3c4CSiIZE0W6TENNUEmKKoHmF1R1RDYIkhMoiAdu6SLaFXIQJWaSsxmFjWF8GGtkzWc2yVyXrsDUtlsp6kOMxUXIAjpk4R7GDHWCbJvM+bahiJr\u002FRmVrQ2l1hbnmH2V4CXmGs3Y2QrUnGDcTp2m6upaO8t+ko96tDH+lDSulY0rV\u002F0s3m9197\u002FA6bOU5Zxdgd9AAAAAElFTkSuQmCC)](https:\u002F\u002Favaturn.live)\n[![Weights](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHuggingFace-Weights-orange?logo=huggingface)](https:\u002F\u002Fhuggingface.co\u002Favaturn-live\u002Favtr-1)\n[![Demo](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDemo-Try%20Now-brightgreen?logo=data:image\u002Fpng;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAQAAADZc7J\u002FAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAACYktHRAD\u002Fh4\u002FMvwAAAAd0SU1FB+oFFA8fAzOfxeUAAAOsSURBVEjHfZVPSFxXFMa\u002F+wYpJGqofxCrhBKcpBMTozGRdFNCSDYtpRFCFu0mWYRgNor54za7JILgspa6SALZl9JCVgndFGqqVWmpmkKjUWGCCiGOzrx376+L++Y5o5Oe1TDvfueec+53vs+oJDAKjJVo1ef6TCf0keokbehfzeg3\u002FWCWJVJyBlUKAkmig+\u002FZoFJkGaN75+RueEpiP8PkAbBEOJzDgcMSYQHYYoQaf3ovPMMkAKFHRcnVMdoRAjBFelcKUhI9LHmwgwS8RS5JAg7rbAgs0lOSgkCijRUoRDuHs9ylnQCRYSjO7aMQASsciWeBIaCaaY+z5AiBpzQgTnKDPk4hGnkKhORwxRtm2EeA8eWPAAXLAw4hWrlKLZ1MJDOYoYMa+jmISDNKwZczLBEIQyc57KbrRZzhJhcQ4jqL5LE4tnnFFYQ4Rz9diF62HZYCGT+DMSAaRdwD4DHiBEIcIMMxqlH8zyMA7iHu+zbGJUOL+yOot0qbek0IvdVhNep3LehHzWpVTq06ri+U1mm90Z+ql\u002FSpVvWSlOFtcFT0AXYTMUAEDCKeVSTic8QdIOIW4h1Y4Jp4CEQRTZwF5qjiYkydEIvDxb8scAnDHHCeBkLfxBMxCaGF24gXfEOKl7iYt6VhccxTxWUmEQNA6IBZse5ryVJHK2KwhIflEQFDiCaaWcajWNPOx28Rn5DDebJUrGGDDOK7kktKFjOQZJSXVHndjaQPhCRXtodrvphlmmlCDP1vC4OIFurIFltYF7N+HAOISS5Txfx7h\u002FgPKb7mBeI2fvRMiSdAFNLAeWAOwyXA7nnGCLhIFX8DZ2ki8iU9FNcA+w5xiwi4g3hesYVnyRsNIDZ9D32GFveXqbW0mWb9KmlN7WrUhBb0k2b1WoGadVxfKq1uvdG8amV0WmtaICW3FnRKYhyI7vsVAR4ly1TNMTIcSJbpcbJMo76BMf8OGQrYbdeL6KKfcwhxhVds47DkWeQ6QlzgJmcQvWw6LDm6MF7QhoGwwChpxEH6qaGDmaT7CTqp5SqtiEM8wEIBGJFIeUnbx4x\u002FaJdIWiPiFH3c4CSiIZE0W6TENNUEmKKoHmF1R1RDYIkhMoiAdu6SLaFXIQJWaSsxmFjWF8GGtkzWc2yVyXrsDUtlsp6kOMxUXIAjpk4R7GDHWCbJvM+bahiJr\u002FRmVrQ2l1hbnmH2V4CXmGs3Y2QrUnGDcTp2m6upaO8t+ko96tDH+lDSulY0rV\u002F0s3m9197\u002FA6bOU5Zxdgd9AAAAAElFTkSuQmCC)](https:\u002F\u002Favaturn.live\u002Fdemo)\n\n\u003C\u002Fdiv>\n\n**AVTR-1** is a flow-matching-based autoregressive model for live dialogue. Given a portrait image and dual-stream audio, it renders lip-synced speech and active listening at 25 fps on a single GPU. Built for production deployment: model weights, TensorRT-accelerated inference, and the live-session backend - available as an API or fully self-hosted\n\n\u003Cdiv align=\"center\">\n  \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F5e4f96af-973a-4aa4-a8be-e0ce6f44a5d1\" muted=false width=\"50%\">\u003C\u002Fvideo>\n\u003C\u002Fdiv>\n\n---\n\n## 📑 What's included\n\n- [x] Model weights\n- [x] Inference code\n- [x] Interactive streaming demo\n- [ ] Technical report (Coming soon)\n- [ ] Production-ready back-end (Coming soon)\n\n---\n\n## Table of Contents\n\n1. [Quick Start](#1-quick-start)\n2. [Performance](#2-performance)\n3. [Troubleshooting](#troubleshooting)\n\n---\n\n## 1. 🚀 Quick Start\n\n### Prerequisites\n\n- Linux\n- NVIDIA GPU (Ampere or later recommended)\n- CUDA 12.x + TensorRT 10.x\n- [pixi](https:\u002F\u002Fprefix.dev\u002F) — `curl -fsSL https:\u002F\u002Fpixi.sh\u002Finstall.sh | sh`\n\n### Install\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Favaturn-live\u002Favtr-1.git\ncd avtr-1\npixi install\n```\n\n### Set storage path (Optional)\n\n```bash\nexport AVTR1_LOCAL_STORAGE=\u002Fpath\u002Fto\u002Favtr1_storage\n```\n\nAll downloaded weights and built engines go here. Defaults to `\u003Cproject_root>\u002Fartifacts\u002F` (the repo checkout, not the caller's working directory) when unset.\n\n### Download weights\n\n```bash\npixi run download\n```\n\nFirst run will prompt for a HuggingFace login via `hf auth login`\n(automatically invoked as a dependency of `download`).\n\n### Build TRT engines\n\nWeights are pulled from two public HF repos by the previous step:\nAVTR-1 weights from [avaturn-live\u002Favtr-1](https:\u002F\u002Fhuggingface.co\u002Favaturn-live\u002Favtr-1)\nand LivePortrait weights repackaged as ONNX graphs from\n[digital-avatar\u002Fditto-talkinghead](https:\u002F\u002Fhuggingface.co\u002Fdigital-avatar\u002Fditto-talkinghead).\nTRT engines are compute-capability specific and built locally — run the scripts\nbelow once per machine; outputs land under `$AVTR1_LOCAL_STORAGE`.\n\n```bash\n# Build everything at once\npixi run build-trt-engines\n\n# Or individually\npixi run build-trt-engines-avtr1\npixi run build-trt-engines-renderer\npixi run build-trt-engines-hubert\n```\n\n### Run interactive demo\n```bash\npixi run interactive-demo\n```\n\n\n### Run offline generation\n\n**Single speaker.** Avatar lip-syncs the given audio track.\n\n```bash\npixi run generate_offline --speech example\u002Fspeaker_1.ogg\n\n# with a custom avatar and background:\npixi run generate_offline --speech example\u002Fspeaker_1.ogg --avatar maria --bg minimal_office\n```\n\n**Two-speaker dialogue.** Avatar voices `--speech` and reacts (active listening) to the peer audio on `--listen`. Run twice with the tracks swapped to render both sides of the conversation.\n\n```bash\n# avatar = speaker 1 (elena)\npixi run generate_offline --speech example\u002Fspeaker_1.ogg --listen example\u002Fspeaker_2.ogg --avatar elena  --out elena.mp4\n# avatar = speaker 2 (marcus)\npixi run generate_offline --speech example\u002Fspeaker_2.ogg --listen example\u002Fspeaker_1.ogg --avatar marcus --out marcus.mp4\n\n# stitch both sides into a single side-by-side video:\nffmpeg -i elena.mp4 -i marcus.mp4 -filter_complex \\\n  \"[0:v][1:v]hstack=inputs=2[v];[0:a][1:a]amix=inputs=2[a]\" \\\n  -map \"[v]\" -map \"[a]\" dialogue.mp4\n```\n\n**Silence \u002F idle motion.** No audio — renders idle micro-motion for the given duration.\n\n```bash\npixi run generate_offline --duration 10\n```\n\nAvailable avatars are the filenames (without `.png`) inside\n`$AVTR1_LOCAL_STORAGE\u002Fv1\u002Favatars_artifacts\u002Freference_frames\u002F` after downloading.\n---\n\n## 2. Performance\n\n### Per-chunk latency\n\nAVTR-1 generates motion in 5-frame chunks end-to-end. At 25 fps that's 200 ms\nof output per chunk, so any GPU under that line runs in real-time.\n\n| GPU         | Latency \u002F 5-frame chunk | Real-time factor |\n| ----------- | ----------------------- | ---------------- |\n| L40         | 84 ms                   | 2.4×             |\n| A100        | 91 ms                   | 2.2×             |\n| RTX 4060 Ti | 166 ms                  | 1.2×             |\n| RTX 3070    | 181 ms                  | 1.1×             |\n| L4          | 202 ms                  | 0.99×            |\n| RTX 3060 Ti | 206 ms                  | 0.97×            |\n| RTX 4060    | 232 ms                  | 0.86×            |\n\nReal-time factor = 200 ms \u002F latency. ≥ 1.0× means the GPU keeps up with 25 fps.\n\n---\n\n## Troubleshooting\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>TURN server setup\u003C\u002Fb> (optional)\u003C\u002Fsummary>\n\nICE tries direct UDP first (host candidates + STUN-reflexive\ncandidates from a public STUN server) and only needs a TURN relay when the\nnetwork in between can't pass UDP between browser and streamer — typical\nwhen the streamer lives on a cloud VM whose security group blocks inbound\nUDP, or when one peer is behind symmetric NAT.\n\nIf direct UDP works for your setup you can skip this section entirely. The\nbrowser's connectivity card after the engine dropdown tells you which path\nICE actually picked, and the same UI links back here when the verdict is\n\"only TURN works\" or \"nothing worked\".\n\nThe project is wired for **Cloudflare's Realtime TURN**. The free tier is\ngenerous enough for development; no credit card required.\n\n**1. Create a TURN application on Cloudflare**\n\n- Sign in to [dash.cloudflare.com](https:\u002F\u002Fdash.cloudflare.com).\n- Navigate to **Realtime → TURN Server**.\n- Click **Create TURN App**, give it a name (e.g. `avtr1-dev`), and submit.\n\n**2. Copy the two credential values**\n\nOn the application's detail page you'll see:\n\n- **Turn Key ID** — short identifier (looks like a UUID without dashes).\n- **API Token** — long secret shown only once at creation. Save it before\n  navigating away.\n\n**3. Put them in `.env`**\n\n```dotenv\nCLOUDFLARE_TURN_KEY_ID=\"\u003CTurn Key ID>\"\nCLOUDFLARE_TURN_KEY_TOKEN=\"\u003CAPI Token>\"\n```\n\nThat's it. On the next `\u002Fice-servers` request the streamer mints a fresh,\nshort-lived TURN credential per session via Cloudflare's\n`\u002Fv1\u002Fturn\u002Fkeys\u002F{kid}\u002Fcredentials\u002Fgenerate` endpoint — the long-lived API\ntoken never leaves the server. You can verify it picked up the keys by\nwatching the streamer log for `ice: using Cloudflare TURN` on the first\nbrowser request.\n\nThe browser-side connectivity probe (the small status card under the\ncontrols) tells you which ICE path actually wins:\n\n- ✓ **host** — the browser saw its own local interface; always present.\n- ✓ **server-reflexive via STUN** — the browser learned its public IP via\n  STUN; doesn't prove the streamer is reachable on UDP from the browser.\n- ✓ **relay via TURN** — the browser successfully allocated a Cloudflare\n  TURN relay; required when direct UDP can't traverse the network in\n  between.\n\nIf the relay check fails while TURN is configured the most likely cause is\nwrong credentials — re-check that you copied the full **API Token** (not\nthe Key ID twice) into `CLOUDFLARE_TURN_KEY_TOKEN`.\n\n**Alternatives.** Anything that speaks the standard TURN protocol works.\nSet `TURN_URL` (and optionally `TURN_USERNAME` \u002F `TURN_CREDENTIAL`) instead\nof the Cloudflare variables and `resolve_ice_servers()` will use it\nverbatim — e.g. a self-hosted [coturn](https:\u002F\u002Fgithub.com\u002Fcoturn\u002Fcoturn) on\na small VM. STUN-only also works *if* you can open the appropriate UDP\nport range inbound on whatever firewall sits in front of the streamer.\n\n\u003C\u002Fdetails>\n\n---\n\n## License\n\nThis repository contains three separately licensed components:\n\n- **`scripts\u002F`** — build and demo tooling, released under the **AVTR-1\n  Community License** ([LICENSE-MODEL.md](LICENSE-MODEL.md)). Permits\n  commercial use by entities under USD 10M annual revenue; entities at or\n  above that threshold need a commercial agreement. The same license governs\n  the AVTR-1 weights distributed at\n  [avaturn-live\u002Favtr-1](https:\u002F\u002Fhuggingface.co\u002Favaturn-live\u002Favtr-1).\n- **`src\u002Favtr1_renderer\u002F`** — Avaturn Renderer (inference pipeline), released\n  under the **PolyForm Noncommercial License 1.0.0** with a Required Notice\n  ([LICENSE-RENDERER.md](LICENSE-RENDERER.md)). **Noncommercial use only**,\n  regardless of revenue; any commercial use needs a separate Renderer\n  Commercial License.\n- **`src\u002Favaturn_live_streamer\u002F`** — Avaturn Streamer (orchestration\n  backend), released under the **PolyForm Noncommercial License 1.0.0** with\n  a Required Notice and patent reservation\n  ([LICENSE-STREAMER.md](LICENSE-STREAMER.md),\n  [PATENTS.md](PATENTS.md)). **Noncommercial use only**, regardless of\n  revenue; any commercial use needs a separate Streamer Commercial License.\n\nSee [LICENSE.md](LICENSE.md) for the full component map and the consequences\nof the multi-license structure. In any conflict between this summary and the\nunderlying license files, the license files control.\n\n### Non-commercial dependency\n\nThe pipeline uses InsightFace's pretrained SCRFD detector and 2D106 landmark\nmodel, which are licensed for **non-commercial research use only**. To use\nAVTR-1 commercially you must either obtain a commercial license from\nInsightFace (deepinsight@gmail.com) or replace these models with\npermissively-licensed alternatives (e.g., MediaPipe). See\n[THIRD-PARTY-NOTICES.md](THIRD-PARTY-NOTICES.md) for the full picture.\n\n**Commercial inquiries:** hello@avaturn.me\n","AVTR-1 是一个能够响应用户语音并生成相应动作的虚拟形象项目。该项目利用 Python 语言开发，通过先进的语音识别技术和深度学习模型来实现虚拟形象对用户语音的智能反馈。其核心功能包括实时语音处理、情感分析以及基于内容的动作生成，使得虚拟形象不仅能够“听懂”用户的话，还能以自然的方式作出回应。适合应用于在线教育、虚拟客服、娱乐互动等多种场景中，为用户提供更加生动和个性化的交互体验。",2,"2026-06-11 03:58:05","CREATED_QUERY"]