[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-83845":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":18,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":23,"hasPages":21,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":45,"readmeContent":46,"aiSummary":10,"trendingCount":15,"starSnapshotCount":15,"syncStatus":47,"lastSyncTime":48,"discoverSource":49},83845,"ai-shortVideo-pipeline","myccarl\u002Fai-shortVideo-pipeline","myccarl","End-to-end AI short-video production pipeline. FastAPI orchestration + Spring Boot gateway with multi-model failover, circuit breaker, metering, and full-stack observability. AI quality gating: prompt anchoring, CLIP consistency, AV sync auto-rescue.","",null,"Python",139,1,53,0,18,82,99,0.9,"MIT License",false,"main",true,[25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44],"ai","ai-pipeline","circuit-breaker","clip-model","deepseek","docker-compose","failover","fastapi","ffmpeg","java","kling","langfuse","llm","multi-model","prompt-engineering","python","resilience4j","short-video","spring-boot","video-generation","2026-06-12 02:04:35","# myAiVideos\n\n> An end-to-end automated short-video production pipeline. One command in, a publish-ready video out.\n\n**English** | [简体中文](README.zh-CN.md)\n\n[![License: MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-yellow.svg)](LICENSE)\n\n## Overview\n\n`myAiVideos` is an open-source engineering project for automated Chinese-language short-video production, covering the full pipeline from **topic discovery → creative generation → visuals → audio → post-production → distribution**. It uses FastAPI as the orchestration core and a Java gateway as the platform governance layer (auth \u002F routing \u002F circuit breaker \u002F metering \u002F observability), coordinating multiple AI models for generation and quality gating.\n\nSuitable content formats:\n\n- Hot-news commentary\n- Knowledge explainers\n- Emotional storytelling\n- Curiosity \u002F strange facts\n- Social insight clips\n\n## Key Features\n\n- **Seven-layer pipeline architecture**: Topic (L1) → Creative (L2) → Visual (L3) → Audio (L4) → Post-production (L5) → Distribution (L6) → Optimization (L7). Layers are decoupled and independently replaceable.\n- **Multi-model failover + circuit breaker**: Java gateway aggregates DeepSeek \u002F Qwen \u002F GLM with Resilience4j protection; failed providers are automatically rotated.\n- **AI consistency governance**\n  - Prompt anchoring: maintain visual identity of subjects across video segments\n  - CLIP text-image consistency gating: reject off-prompt keyframes early\n  - Audio-video sync auto-rescue: 4-tier strategy (audio tempo \u002F video pad \u002F narration rewrite)\n- **Full-stack observability**: `trace_id` propagated across Java\u002FPython; Langfuse call-tree visibility.\n- **Metering & rate limiting**: AOP aspect captures token \u002F cost without code changes, aggregated by tenant.\n- **Production-ready**: SSE progress streaming, single-segment regeneration, rolling log archives.\n\n## Tech Stack\n\n| Layer | Components |\n|---|---|\n| Orchestration (Python) | FastAPI · asyncio · Pydantic · SQLAlchemy (asyncpg) · Alembic · ARQ |\n| Gateway (Java) | Spring Boot 3.5 · WebClient · Resilience4j · Caffeine · Prometheus |\n| Frontend | Vue 3 · Vite · Vue Flow · Pinia · Tailwind |\n| Data | PostgreSQL · Redis · MinIO |\n| Observability | Langfuse · cross-language trace context |\n| AI Models | DeepSeek · Zhipu GLM-4V · Kling v2.5 · Volcengine \u002F MiniMax TTS · faster-whisper |\n| Containers | Docker Compose |\n\n## Architecture\n\n```\n                +----------------+\n                |  Client \u002F Bot  |\n                +--------+-------+\n                         |\n                +--------v-------+\n                |  Java Gateway  |  JWT \u002F TraceId \u002F Router \u002F Breaker \u002F Meter\n                +--------+-------+\n                         |\n                +--------v-------+\n                | FastAPI orch.  |  webhook · session · job orchestration\n                +--+---------+---+\n                   |         |\n        +----------v---+   +-v-----------+\n        |  ARQ Worker  |   |  7-layer    |\n        +-----+--------+   |  pipeline   |\n              |            +-+-----------+\n              |              |\n   +----------v--+    +------v-----------------+\n   | Redis(queue)|    | DeepSeek \u002F Kling \u002F TTS |\n   +-------------+    +------------------------+\n```\n\nFor details, see [`architecture.md`](architecture.md).\n\n## Quick Start\n\n### Prerequisites\n\n- Docker Desktop ≥ 24\n- Node.js ≥ 20 (only for frontend development)\n- Python ≥ 3.11 (only if running locally without Docker, or running tests on host)\n\n### One-Click Start\n\n```bash\n# 1. Clone and prepare env\ngit clone \u003Cyour-fork>.git\ncd myAiVideos\ncp .env.example .env\n# Edit .env to fill in at least the required AI provider API keys.\n\n# 2. Start backend (postgres + redis + orchestrator + worker)\ndocker compose up -d postgres redis orchestrator worker\n\n# 3. Run database migrations\ndocker compose exec -T orchestrator alembic upgrade head\n\n# 4. (Optional) Start Java gateway\ndocker compose up -d gateway\n\n# 5. Start frontend\ncd sl-vue && npm install && npm run dev\n```\n\nHealth checks:\n\n- Python API: \u003Chttp:\u002F\u002Flocalhost:8000\u002Fhealth>\n- Java Gateway: \u003Chttp:\u002F\u002Flocalhost:8080\u002Factuator\u002Fhealth>\n- Frontend: \u003Chttp:\u002F\u002Flocalhost:5173>\n\nFor Windows \u002F macOS \u002F Linux specific scripts, scenario presets, and troubleshooting, see [`docs\u002F启动-停止脚本.md`](docs\u002F启动-停止脚本.md).\n\n### Configuration\n\nCore environment variables (full list in `.env.example`):\n\n| Variable | Purpose |\n|---|---|\n| `DEEPSEEK_API_KEY` | Text LLM |\n| `GLM_API_KEY` | Multimodal moderation |\n| `KLING_ACCESS_KEY` \u002F `KLING_SECRET_KEY` | Text-to-image \u002F image-to-video |\n| `VOLCENGINE_TTS_*` or `MINIMAX_TTS_*` | Speech synthesis |\n| `LANGFUSE_PUBLIC_KEY` \u002F `LANGFUSE_SECRET_KEY` | Tracing (optional) |\n| `GATEWAY_AUTH_JWT_SECRET` | JWT signing key for Java gateway (≥ 32 bytes) |\n\n## Project Structure\n\n```\nmyAiVideos\u002F\n├── api\u002F                  FastAPI routes (webhook, storyboard, clip)\n├── config\u002F               Settings, Skill templates, characters, styles\n├── core\u002F                 trace context, langfuse client, logging, scheduler\n├── db\u002F                   SQLAlchemy models, Alembic migrations\n├── integrations\u002F         External model client wrappers\n├── layers\u002F               Seven-layer pipeline\n│   ├── L1_trending\u002F      Topic discovery \u002F ranking\n│   ├── L2_creative\u002F      Script generation, prompt orchestration, anchoring\n│   ├── L3_visual\u002F        Text-to-image, image-to-video, CLIP consistency\n│   ├── L4_audio\u002F         TTS, mixing, duration planning\n│   ├── L5_postprod\u002F      AV sync, rescue, captions, cover\n│   ├── L6_distribution\u002F  Delivery back to client\n│   └── L7_optimization\u002F  Quality scoring, data feedback\n├── scripts\u002F              Ops & backfill scripts\n├── sl-vue\u002F               Vue 3 frontend\n├── gateway\u002F              Java Spring Boot gateway (see gateway\u002FREADME.md)\n├── tests\u002F                pytest test suite\n├── docs\u002F                 Module reference docs\n├── docker-compose.yaml   Multi-container orchestration\n├── Dockerfile            Python image\n└── main_v2.py            FastAPI entrypoint\n```\n\n## Documentation Index\n\n- [`architecture.md`](architecture.md) — Overall architecture and data flow\n- [`docs\u002Foperations-manual.md`](docs\u002Foperations-manual.md) — Operations manual: Feishu integration, testing, daily maintenance (中文: [`docs\u002F操作使用手册.md`](docs\u002F操作使用手册.md))\n- [`docs\u002F启动-停止脚本.md`](docs\u002F启动-停止脚本.md) — Cross-platform start \u002F stop \u002F log \u002F troubleshooting guide\n- [`gateway\u002FREADME.md`](gateway\u002FREADME.md) — Java gateway (auth, routing, failover, circuit breaker, metering)\n\n## Development\n\n### Run tests\n\n```bash\ndocker compose exec -T orchestrator pytest tests\u002F -q\n```\n\n### After Python code changes\n\n```bash\ndocker compose build orchestrator\ndocker compose up -d --force-recreate --no-build orchestrator worker\n```\n\n> Note: `docker compose restart` does **not** rebuild the image. Use the two commands above for source code changes.\n\n### Database migrations\n\n```bash\ndocker compose exec -T orchestrator alembic upgrade head\n# Create a new migration\ndocker compose exec -T orchestrator alembic revision -m \"your message\"\n```\n\n## Contributing\n\nIssues and pull requests are welcome. For substantial changes, please open an issue first to discuss the design.\n\n## License\n\nMIT — see [`LICENSE`](LICENSE).\n\n## Acknowledgements\n\nThis project integrates several excellent open-source models and commercial AI services, including but not limited to: DeepSeek, Zhipu GLM, Kling AI, Volcengine TTS, faster-whisper, Langfuse, Resilience4j.\n",2,"2026-06-11 04:11:38","CREATED_QUERY"]