[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-93064":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":14,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":16,"rankGlobal":10,"rankLanguage":10,"license":17,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":20,"hasPages":18,"topics":21,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":15,"starSnapshotCount":15,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},93064,"worldcupvoice","zicojiao\u002Fworldcupvoice","zicojiao","AI live commentator for sports streams. It watches the Agora RTC video feed and adds AI voice commentary.","",null,"Python",56,9,1,0,43.1,"MIT License",false,"main",true,[22,23,24,25,26],"agora","agora-sdk","livestream","rtmp","voice-ai","2026-07-22 04:02:08","\u003Cdiv align=\"center\">\n\n# WorldCupVoice\n\n**AI live commentator for sports streams.**\n\nAdd real-time AI commentary to a live feed: the AI watches the same RTC video as\nviewers and speaks over the action live. The World Cup is the showcase scenario,\nbut the same pipeline works for any live stream.\n\n[![License: MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-blue)](.\u002FLICENSE)\n![Version](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fversion-v0.1-orange)\n![AI commentator](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FAI-commentator-7c3aed)\n![Live media](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flive%20media-Agora%20RTC-0ea5e9)\n![Ingest](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fingest-OBS%20%2B%20RTMP-111827)\n![Built for World Cup](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fbuilt%20for-World%20Cup-16a34a)\n\n**English** · [简体中文](.\u002FREADME.zh-CN.md)\n\n\u003C\u002Fdiv>\n\n---\n\nWorldCupVoice is a live AI commentator for sports streams: the broadcast enters a\nreal-time media channel, the AI watches the same feed as viewers, and generated\nspeech returns to the room as live commentary.\n\nIt is built for live sports moments like the World Cup, and one use case it fits\nespecially well is accessibility: AI live commentary can give blind and\nlow-vision fans field-level detail that human live commentary often skips, such\nas where the ball is, which side pressure is forming from, and who is making a\nrun. The goal is not to replace human commentators.\n\n## Demo\n\nWatch WorldCupVoice turn a live match feed into AI commentary:\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F307ca759-29f7-40d8-b3be-e9d5e5104e48\n\n## Architecture\n\n```mermaid\nflowchart LR\n  Encoder[\"Live source\u003Cbr\u002F>OBS \u002F encoder\"]\n  RTMP[\"RTMP push\"]\n  Gateway[\"Agora Media Gateway\"]\n  Channel[\"Agora RTC channel\"]\n  Viewer[\"Browser viewer\u003Cbr\u002F>Agora Web SDK\"]\n  Agent[\"AI commentator backend\u003Cbr\u002F>Agora Python Server SDK\"]\n  Vision[\"Vision model\"]\n  Voice[\"TTS voice\"]\n\n  Encoder --> RTMP --> Gateway --> Channel\n  Channel --> Viewer\n  Channel --> Agent\n  Agent --> Vision --> Voice --> Channel\n```\n\nRuntime roles:\n\n- Live feed: the RTMP\u002FGateway publisher inside the RTC channel.\n- AI commentator: the backend participant that samples frames, writes the call,\n  and publishes generated speech.\n- Browser viewer: the web client that plays the live feed, AI audio, transcript,\n  and booth status.\n\n## Features\n\n- Live AI play-by-play from the same RTC feed viewers are watching.\n- Any live source (OBS, encoder, ...) ingests through RTMP and Agora Media Gateway.\n- Server-side frame sampling for visually grounded match commentary.\n- Configurable commentator voice with OpenAI TTS, ElevenLabs, or Fish Audio.\n- AI audio published back into RTC with synced transcript and booth status.\n- Explicit `Start AI` \u002F `Stop AI`, viewer heartbeat, and hard session TTL to\n  control AI spend.\n\n## Quick Start\n\n### Frontend Env\n\nInstall frontend dependencies:\n\n```bash\npnpm install\ncp .env.example .env.local\n```\n\nFill `.env.local`:\n\nGenerate one backend secret first:\n\n```bash\nopenssl rand -hex 32\n```\n\nPaste the same generated value into both frontend and backend env files:\n\n```bash\nNEXT_PUBLIC_AGORA_APP_ID=\nNEXT_AGORA_APP_CERTIFICATE=\nAGENT_BACKEND_URL=http:\u002F\u002Flocalhost:8000\nBACKEND_API_SECRET=\u003Csame-generated-secret>\nACCESS_PASSWORD=\u003Cchoose-a-local-access-code>\n```\n\n`NEXT_AGORA_APP_CERTIFICATE` (frontend) and `AGORA_APP_CERTIFICATE` (backend) are\nthe same Agora certificate, named per each side's convention.\n\nOptional overrides:\n\n```bash\nNEXT_PUBLIC_LIVE_CHANNEL_NAME=worldcup-live\nNEXT_PUBLIC_MATCH_FEED_UID=234567\nNEXT_PUBLIC_AGENT_UID=123456\n# Signs the access cookie. Optional locally (defaults to ACCESS_PASSWORD); set a\n# dedicated value in production and add it to Vercel.\nACCESS_SESSION_SECRET=\n```\n\n### Backend\n\nPrepare the backend environment:\n\n```bash\ncd server\npython -m venv .venv\nsource .venv\u002Fbin\u002Factivate\npip install -r requirements.txt -r requirements-dev.txt\ncp .env.example .env.local\n```\n\nFill `server\u002F.env.local`:\n\n```bash\nAGORA_APP_ID=\nAGORA_APP_CERTIFICATE=\nBACKEND_API_SECRET=\u003Csame-generated-secret>\nOPENAI_API_KEY=\n```\n\n### TTS Voice\n\nThe default OpenAI TTS path is enough to run the project, but the voice is a big\npart of the AI commentator effect. For demos and production-style streams,\nElevenLabs is recommended because a purpose-built sportscaster voice makes the\ncommentary feel far more like a real broadcast booth.\n\nFish Audio can also be used as the TTS provider, and is worth trying for Chinese\ncommentary voices:\n\n```bash\nTTS_PROVIDER=fish_audio\nFISH_AUDIO_API_KEY=\nFISH_AUDIO_VOICE_ID_ZH_MEME=\nFISH_AUDIO_VOICE_ID_ZH_TACTICAL=\n```\n\nThe open-source project ships commentator profiles and prompts, not private\nvoice IDs. Create your own Fish Audio voice and put its ID in the matching env\nvariable. If neither a profile-specific nor generic third-party voice ID is configured,\nthe backend falls back to OpenAI TTS so local setup still works.\n\nBuilt-in profile voice variables:\n\n| Profile | Provider | Voice env |\n| --- | --- | --- |\n| English Sportscaster | ElevenLabs | `ELEVENLABS_VOICE_ID_EN_SPORTSCASTER` |\n| French Sportscaster | ElevenLabs | `ELEVENLABS_VOICE_ID_FR_SPORTSCASTER` |\n| Chinese Meme Commentary | Fish Audio | `FISH_AUDIO_VOICE_ID_ZH_MEME` |\n| Chinese Tactical Commentary | Fish Audio | `FISH_AUDIO_VOICE_ID_ZH_TACTICAL` |\n\nCreate your own ElevenLabs voice and put its voice ID in `server\u002F.env.local`:\n\n```bash\nTTS_PROVIDER=elevenlabs\nELEVENLABS_API_KEY=\nELEVENLABS_VOICE_ID=\nELEVENLABS_VOICE_ID_EN_SPORTSCASTER=\nELEVENLABS_VOICE_ID_FR_SPORTSCASTER=\n```\n\nIn ElevenLabs:\n\n1. Open **VoiceLab**.\n2. Click **Create Voice**.\n3. Choose **Voice Design**.\n4. Paste the prompt below and generate a sportscaster voice.\n5. Save the voice, then copy its **Voice ID** into `ELEVENLABS_VOICE_ID`.\n\nThe ElevenLabs voice prompt used to generate the voice in the demo:\n\n```text\nNative English, neutral American broadcast style. Male, 35-50. Broadcast quality.\n\nPersona: elite sports commentator. Emotion: explosive, urgent, passionate.\n\nA powerful, resonant, high-energy voice built for live football and basketball commentary. Deep but agile timbre, crisp articulation, close-mic broadcast presence, and clean studio-quality audio. Speaks at a fast, rhythmic pace during live action, with sudden bursts of excitement, sharp emphasis on player names, and dramatic pauses after huge moments. The delivery should feel like a professional television play-by-play announcer calling a World Cup final: intense, emotionally invested, breathless during attacks, and thunderous when a goal or game-changing moment happens.\n```\n\nStart the backend after the env file is filled:\n\n```bash\npython -m uvicorn app.main:app --reload --host 127.0.0.1 --port 8000\n```\n\n### Run The App\n\nStart the frontend:\n\n```bash\npnpm dev\n```\n\nOpen [http:\u002F\u002Flocalhost:3000](http:\u002F\u002Flocalhost:3000), enter the booth, then push\na stream from OBS or a local clip you provide.\n\n### Media Gateway Stream Key\n\nAgora Media Gateway needs two RTMP values: a server domain name and a streaming\nkey. The Console page enables Media Gateway, but it does not show a ready-made\nstream key. When you use Agora's unified RTMP domain, create the stream key with\nthe Media Gateway REST API.\n\nFor this project, generate the key for the default live feed:\n\n```text\nChannel: worldcup-live\nUID: 234567\n```\n\nIf you changed `NEXT_PUBLIC_LIVE_CHANNEL_NAME` or `NEXT_PUBLIC_MATCH_FEED_UID`,\nuse those values instead.\n\nIn [Agora Console](https:\u002F\u002Fconsole.agora.io\u002F):\n\n1. Open **Projects** from the Console sidebar and select your project.\n2. Enable **Media Gateway** from the project's feature list.\n3. Open **Developer Toolkit -> RESTful API** in Console and create or copy a\n   Customer ID and Customer Secret. Agora documents this flow in\n   [RESTful authentication](https:\u002F\u002Fdocs.agora.io\u002Fen\u002Fsignaling\u002Frest-api\u002Frestful-authentication).\n4. Add them to local `.env.local` only:\n\n```bash\nAGORA_CUSTOMER_ID=\nAGORA_CUSTOMER_SECRET=\nAGORA_MEDIA_GATEWAY_REGION=\u003Cregion>\n```\n\nChoose the Media Gateway region closest to your encoder or cloud RTMP source,\nfor example `eu`, `na`, `as`, `cn`, `jp`, or `in`.\n\nCreate the stream key:\n\n```bash\npnpm run media-gateway:key\n```\n\nCopy the generated RTMP details into whichever source you use next:\n\n```text\nRTMP server: rtmp:\u002F\u002Frtls-ingress-prod-\u003Cregion>.agoramdn.com\u002Flive\nStream key: \u003Cgenerated stream key>\n```\n\nKeep the Customer Secret and stream key private. Do not commit them to GitHub or\nput them in Vercel.\n\nAgora's official docs explain that the unified RTMP server uses the\n`rtls-ingress-prod-\u003Cregion>.agoramdn.com\u002Flive` domain and that the stream key is\ncreated through the Media Gateway REST API. See\n[Media Gateway quickstart](https:\u002F\u002Fdocs.agora.io\u002Fen\u002Fmedia-gateway\u002Fget-started\u002Fquickstart)\nand [Create streaming key](https:\u002F\u002Fdocs.agora.io\u002Fen\u002Fmedia-gateway\u002Freference\u002Frest-api\u002Fendpoints\u002Fstreaming-key\u002Fcreate-streaming-key).\n\n### Choose an RTMP Source\n\nAfter you have an RTMP server and stream key, choose one RTMP source:\n\n| Source | Best for | Runs where | Notes |\n| --- | --- | --- | --- |\n| `pnpm run stream:sample` | Fast local smoke tests | Your laptop | Uses ffmpeg to loop a local clip you provide. Stop it with `Ctrl+C`. |\n| OBS | Manual demos and real camera\u002Fscreen feeds | Your laptop or encoder machine | Good for interactive control, overlays, audio routing, and screen capture. |\n| StreamFlow or another cloud RTMP encoder | Longer prerecorded or scheduled streams | A VPS or cloud host | More stable than a laptop for 24\u002F7 prerecorded streams, but still just an RTMP producer. |\n\nAll three options push to the same Agora Media Gateway RTMP server and stream\nkey. The app and AI commentator always consume the result as live Agora RTC\nvideo.\n\n### Push A Local Clip\n\nNo video is shipped with this repo — broadcast match footage is copyrighted and\ncannot be redistributed, so **bring your own football clip** (any 16:9 `.mp4`).\nThe clip is only a local broadcast source; it is not played directly by the app.\n\nFor better commentary, describe your match first so the AI can identify\nplayers. Each match is one JSON file under\n[`data\u002Fmatches\u002F`](.\u002Fdata\u002Fmatches\u002F) (teams, jersey colors, rosters, storyline) —\ncopy `_template.json`, edit it, and register it in\n[`lib\u002Fcommentary.ts`](.\u002Flib\u002Fcommentary.ts). See\n[`data\u002Fmatches\u002FREADME.md`](.\u002Fdata\u002Fmatches\u002FREADME.md) for the full guide.\n\nInstall ffmpeg, then push your clip through Agora Media Gateway:\n\n```bash\nbrew install ffmpeg\n\nRTMP_STREAM_KEY=\u003Cgenerated stream key> \\\nRTMP_INPUT=\u002Fpath\u002Fto\u002Fyour-match.mp4 \\\npnpm run stream:sample\n```\n\nIf `RTMP_INPUT` is unset, the script streams the first `.mp4` it finds in\n`samples\u002F`. It loops the clip until you press `Ctrl+C`; the browser and AI\ncommentator receive it as live Agora RTC video.\n\n> The clip used during local testing came from\n> \u003Chttps:\u002F\u002Fwww.youtube.com\u002Fwatch?v=RgqKdplLIk4>. Obtain and use any third-party\n> footage at your own responsibility.\n\n### Cloud RTMP With StreamFlow\n\n[StreamFlow](https:\u002F\u002Fgithub.com\u002Fbangtutorial\u002Fstreamflow) is an optional\nself-hosted web app for managing prerecorded live streams. It provides video\nupload\u002Fmanagement, scheduled streaming, loop mode, bitrate\u002FFPS\u002Fresolution\nsettings, and custom RTMP output.\n\nUse StreamFlow when you want a production-like prerecorded broadcast that keeps\nrunning from a VPS instead of your laptop:\n\n1. Deploy StreamFlow on a VPS or Docker host.\n2. Upload your own football clip.\n3. Use the RTMP server and stream key printed by `pnpm run media-gateway:key`.\n4. Enable loop or schedule settings as needed.\n5. Start the stream from StreamFlow.\n\nThis does not replace Agora Media Gateway. StreamFlow is only the upstream RTMP\nproducer; Agora Media Gateway still converts that RTMP input into the RTC feed\nwatched by the browser and AI commentator.\n\n### OBS Setup\n\nUse a custom streaming service:\n\n```text\nService: Custom\nServer: rtmp:\u002F\u002Frtls-ingress-prod-\u003Cregion>.agoramdn.com\u002Flive\nStream Key: \u003Cyour Agora Media Gateway stream key>\n```\n\nRecommended starting settings:\n\n- Encoder: H.264\n- FPS: 30\n- Keyframe interval: 2 seconds\n- Rate control: CBR\n- Bitrate: 4500-6500 Kbps for 1080p\n- Audio: AAC, 48 kHz\n\n### Deployment\n\nFrontend:\n\n```bash\nvercel link\nvercel env add NEXT_PUBLIC_AGORA_APP_ID production\nvercel env add NEXT_AGORA_APP_CERTIFICATE production\nvercel env add AGENT_BACKEND_URL production\nvercel env add BACKEND_API_SECRET production\nvercel env add ACCESS_PASSWORD production\nvercel env add ACCESS_SESSION_SECRET production\nvercel deploy --prod\n```\n\nBackend:\n\n- Deploy `server\u002F` as a Railway Docker service.\n- Set the variables from [`server\u002F.env.example`](.\u002Fserver\u002F.env.example).\n- Use the same `BACKEND_API_SECRET` value in Railway and Vercel.\n- Set frontend `AGENT_BACKEND_URL` to the Railway public URL.\n\n## Runtime Cost Controls\n\nThe backend does not spend AI tokens just because it is deployed.\n\n- `\u002Fsessions\u002Fstart` starts the AI commentator.\n- `\u002Fsessions\u002Fheartbeat` keeps it alive while a viewer is present.\n- `\u002Fsessions\u002Fstop` stops it explicitly.\n- `BACKEND_API_SECRET` protects the public Python backend from direct session\n  control calls.\n- Stale sessions stop automatically when viewer heartbeats disappear.\n- Every session has a hard maximum runtime.\n\nThe timeout, model, voice, frame sampling, and audio pacing defaults live in\n[`server\u002Fapp\u002Fconfig.py`](.\u002Fserver\u002Fapp\u002Fconfig.py). Override them only when you are\ndeliberately tuning the runtime.\n\nThe booth monitor shows whether AI is idle, waiting for video, active, stopped,\nor missing.\n\n## Project Layout\n\n```text\napp\u002Fapi\u002F                  Next.js API routes for tokens, access, and session proxying\ncomponents\u002F               Live booth UI, transcript, metrics, and lobby\ndata\u002Fmatches\u002F             Per-match context (one JSON per game) sent to the AI\nlib\u002Fcommentary.ts         Loads the match JSON and builds the AI prompt context\nserver\u002Fapp\u002F               FastAPI backend and AI commentator\nserver\u002Ftests\u002F             Backend smoke tests\npublic\u002F                   Logos, icons, and poster imagery\n```\n\n## Roadmap\n\n- [ ] Lower latency between the video frame and the AI commentary output.\n- [ ] Multi-commentator: several AI commentators working together.\n- [ ] A dedicated accessibility mode for blind and low-vision fans, with denser field\u002Fspatial detail.\n- [ ] More voices and languages: different countries' languages and vocal styles.\n\n## License\n\nMIT\n","WorldCupVoice 是一个为体育直播流提供实时AI语音解说的技术方案。它通过接入Agora RTC视频流，利用服务器端视觉模型分析帧画面生成比赛描述，并调用TTS服务合成语音，再将音频实时回传至同一RTC频道，实现与观众观看画面同步的AI解说。项目支持RTMP推流接入、多TTS引擎配置（如OpenAI、ElevenLabs）、帧级视觉理解及显式启停控制，专为足球等实时体育赛事设计，特别适用于无障碍场景——为视障用户提供球位、跑位、压迫方向等细粒度场况信息。",2,"2026-07-11 02:30:47","CREATED_QUERY"]