[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-79979":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":13,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":14,"forks30d":14,"starsTrendScore":18,"compositeScore":19,"rankGlobal":9,"rankLanguage":9,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":23,"topics":24,"createdAt":9,"pushedAt":9,"updatedAt":25,"readmeContent":26,"aiSummary":27,"trendingCount":14,"starSnapshotCount":14,"syncStatus":13,"lastSyncTime":28,"discoverSource":29},79979,"WBench","meituan-longcat\u002FWBench","meituan-longcat","WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation",null,"Python",130,4,2,0,6,13,43,21,2.1,"MIT License",false,"main",true,[],"2026-06-12 02:03:56","\u003Cdiv align=\"center\">\n  \u003Cimg src=\"assets\u002Flongcat-logo-full.png\" width=\"250\">\n\u003C\u002Fdiv>\n\n\u003Cdiv align=\"center\">\n  \u003Ch1>WBench: A Comprehensive Multi-turn Benchmark for\u003Cbr>Interactive Video World Model Evaluation\u003C\u002Fh1>\n\u003C\u002Fdiv>\n\n\u003Cdiv align=\"center\">\n\n[![Homepage](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHomepage-blue?style=for-the-badge&logo=google-chrome&logoColor=white)](https:\u002F\u002Fmeituan-longcat.github.io\u002FWBench\u002F)\n[![Paper](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPaper-red?style=for-the-badge&logo=arxiv&logoColor=white)](https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.25874)\n[![HF Daily Paper](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDaily_Paper_%232-FFD21E?style=for-the-badge&logo=huggingface&logoColor=white&color=FF9D00)](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2605.25874)\n[![Leaderboard](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLeaderboard-32CD32?style=for-the-badge&logo=google-chrome&logoColor=white)](https:\u002F\u002Fmeituan-longcat.github.io\u002FWBench\u002F#leaderboard)\n[![Datasets](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDatasets-4285F4?style=for-the-badge&logo=huggingface&logoColor=white)](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fmeituan-longcat\u002FWBench)\n[![Weights](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FWeights-FF9D00?style=for-the-badge&logo=huggingface&logoColor=white)](https:\u002F\u002Fhuggingface.co\u002Fmeituan-longcat\u002FWBench-weights)\n[![ModelScope](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FModelScope-6B4EFF?style=for-the-badge&logo=data:image\u002Fsvg+xml;base64,PHN2ZyBmaWxsPSJ3aGl0ZSIgZmlsbC1ydWxlPSJldmVub2RkIiBoZWlnaHQ9IjFlbSIgc3R5bGU9ImZsZXg6bm9uZTtsaW5lLWhlaWdodDoxIiB2aWV3Qm94PSIwIDAgMjQgMjQiIHdpZHRoPSIxZW0iIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyI+PHRpdGxlPk1vZGVsU2NvcGU8L3RpdGxlPjxwYXRoIGQ9Ik0yLjY2NyA1LjNIOHYyLjY2N0g1LjMzM3YyLjY2NkgyLjY2N1Y4LjQ2N0guNXYyLjE2NmgyLjE2N1YxMy4zSDBWNy45NjdoMi42NjdWNS4zek0yLjY2NyAxMy4zaDIuNjY2djIuNjY3SDh2Mi42NjZIMi42NjdWMTMuM3pNOCAxMC42MzNoMi42NjdWMTMuM0g4di0yLjY2N3pNMTMuMzMzIDEzLjN2Mi42NjdoLTIuNjY2VjEzLjNoMi42NjZ6TTEzLjMzMyAxMy4zdi0yLjY2N0gxNlYxMy4zaC0yLjY2N3oiPjwvcGF0aD48cGF0aCBjbGlwLXJ1bGU9ImV2ZW5vZGQiIGQ9Ik0yMS4zMzMgMTMuM3YtMi42NjdoLTIuNjY2VjcuOTY3SDE2VjUuM2g1LjMzM3YyLjY2N0gyNFYxMy4zaC0yLjY2N3ptMC0yLjY2N0gyMy41VjguNDY3aC0yLjE2N3YyLjE2NnoiPjwvcGF0aD48cGF0aCBkPSJNMjEuMzMzIDEzLjN2NS4zMzNIMTZ2LTIuNjY2aDIuNjY3VjEzLjNoMi42NjZ6Ij48L3BhdGg+PC9zdmc+&logoColor=white)](https:\u002F\u002Fmodelscope.cn\u002Fdatasets\u002Fmeituan-longcat\u002FWBench)\n[![中文解读](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F中文解读-07C160?style=for-the-badge&logo=wechat&logoColor=white)](https:\u002F\u002Fmp.weixin.qq.com\u002Fs\u002Fbr3RlOBGtReolLZc5YW2HA)\n[![WeChat Group](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FWeChat_Group-07C160?style=for-the-badge&logo=wechat&logoColor=white)](assets\u002Fwx_qr.png)\n\n\u003C\u002Fdiv>\n\n\u003Cdiv align=\"center\">\n  \u003Ci>Is Your World Model an All-Round Player?\u003C\u002Fi>\n\u003C\u002Fdiv>\n\n---\n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\"assets\u002Fteaser.png\" width=\"90%\">\n\u003C\u002Fdiv>\n\n\u003Cp align=\"center\" style=\"color: grey;\">\n\u003Cb>TL;DR\u003C\u002Fb> — WBench evaluates 20 video world models across 5 dimensions and 22 metrics.\n\u003C\u002Fp>\n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\"assets\u002Fqr_code.png\" width=\"300\">\n\u003C\u002Fdiv>\n\n## 📢 News\n\n- **[2026\u002F05\u002F29]** Our paper ranked **#2** on [Hugging Face Daily Papers](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2605.25874)!\n- **[2026\u002F05\u002F28]** Our paper is now available on [arXiv](https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.25874)!\n- **[2026\u002F05\u002F28]** [Homepage](https:\u002F\u002Fmeituan-longcat.github.io\u002FWBench\u002F) with interactive [leaderboard](https:\u002F\u002Fmeituan-longcat.github.io\u002FWBench\u002F#leaderboard) and [dataset gallery](https:\u002F\u002Fmeituan-longcat.github.io\u002FWBench\u002F#gallery) is live!\n- **[2026\u002F05\u002F28]** We release the full [WBench dataset](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fmeituan-longcat\u002FWBench), [evaluation code](https:\u002F\u002Fgithub.com\u002Fmeituan-longcat\u002FWBench), and [model weights](https:\u002F\u002Fhuggingface.co\u002Fmeituan-longcat\u002FWBench-weights).\n\n## ✨ Contributions\n\n- A **comprehensive evaluation framework** with 289 cases, 1,058 interaction turns, covering 4 interaction types (navigation, subject action, event editing, perspective switching) across diverse scenes and perspectives.\n- A **unified navigation protocol** that bridges text, 6-DoF camera pose, and discrete-action interfaces, enabling fair comparison across model families.\n- **22 automatic metrics** spanning 5 complementary dimensions, validated against human judgments, ensuring reliable automatic evaluation at scale.\n- **Systematic diagnosis of 20 models** revealing that current world models have not yet unified high-fidelity rendering with reliable controllability, consistency, and physics compliance.\n\n## 🏆 Leaderboard\n\n**20 Models — Navigation Split (5 Dimensions, sorted by average)**\n\n| # | Model | **Average** | Quality | Setting | Interaction | Consistency | Physical |\n|:---:|:---|:---:|:---:|:---:|:---:|:---:|:---:|\n| 1 | \u003Cimg src=\"assets\u002Ficon\u002Fkling.jpeg\" height=\"18\"> Kling 3.0 | **79.2 🥇** | 83.0 🥈 | 91.0 🥈 | 70.3    | 82.5    | 69.3 🥉 |\n| 2 | \u003Cimg src=\"assets\u002Ficon\u002Flingbot.png\" height=\"18\"> LingBot-World | **78.8 🥈** | 81.5    | 72.6    | 79.8    | 88.9 🥇 | 71.2 🥈 |\n| 3 | \u003Cimg src=\"assets\u002Ficon\u002Fwan.png\" height=\"18\"> Wan 2.7 | **78.5 🥉** | 82.6 🥉 | 91.4 🥇 | 66.0    | 80.5    | 71.8 🥇 |\n| 4 | \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-World 1.5 | **78.4**      | 80.2    | 72.2    | 87.5 🥇 | 86.0    | 66.3    |\n| 5 | \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-Video 1.5 | **78.2**    | 79.7    | 85.6 🥉 | 71.8    | 86.7 🥉 | 67.4    |\n| 6 | \u003Cimg src=\"assets\u002Ficon\u002Falibaba.png\" height=\"18\"> Happy Oyster | **77.1**    | 79.3    | 74.2    | 85.1 🥈 | 83.3    | 63.5    |\n| 7 | \u003Cimg src=\"assets\u002Ficon\u002Fbytedance.png\" height=\"18\"> Seedance 1.5 | **76.5**    | 83.2 🥇 | 82.9    | 68.0    | 80.2    | 68.4    |\n| 8 | \u003Cimg src=\"assets\u002Ficon\u002Fcosmos.png\" height=\"18\"> Cosmos 2.5 | **75.2**    | 75.6    | 83.3    | 64.1    | 85.6    | 67.4    |\n| 9 | \u003Cimg src=\"assets\u002Ficon\u002Flightrix.jpeg\" height=\"18\"> LTX 2.3 | **74.4**    | 78.7    | 85.2    | 67.6    | 75.6    | 64.9    |\n| 10 | \u003Cimg src=\"assets\u002Ficon\u002Finspatio.jpeg\" height=\"18\"> InSpatio-World | **74.3**    | 74.9    | 71.4    | 72.8    | 87.4 🥈 | 65.2    |\n| 11 | \u003Cimg src=\"assets\u002Ficon\u002Famap.png\" height=\"18\"> Fantasy-World | **74.2**    | 75.5    | 71.3    | 72.1    | 85.3    | 66.8    |\n| 12 | \u003Cimg src=\"assets\u002Ficon\u002Fgoogle.png\" height=\"18\"> Genie 3 | **74.1**    | 77.4    | 72.5    | 73.3    | 81.4    | 65.7    |\n| 13 | \u003Cimg src=\"assets\u002Ficon\u002Flongcat.png\" height=\"18\"> LongCat-Video | **73.7**    | 78.2    | 72.3    | 63.1    | 85.9    | 68.9    |\n| 14 | \u003Cimg src=\"assets\u002Ficon\u002Fshlab.png\" height=\"18\"> YUME 1.5 | **73.5**    | 79.5    | 72.4    | 72.0    | 78.6    | 65.2    |\n| 15 | \u003Cimg src=\"assets\u002Ficon\u002Fmeituan.png\" height=\"18\"> Infinite-World | **72.9**    | 78.7    | 69.3    | 75.9    | 78.7    | 62.1    |\n| 16 | \u003Cimg src=\"assets\u002Ficon\u002Fskywork.jpeg\" height=\"18\"> MatrixGame3 | **71.2**    | 76.9    | 63.6    | 83.5 🥉 | 72.9    | 59.3    |\n| 17 | \u003Cimg src=\"assets\u002Ficon\u002Fkairos.png\" height=\"18\"> Kairos 3.0 | **70.7**    | 76.4    | 70.3    | 65.1    | 81.4    | 60.4    |\n| 18 | \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-GameCraft | **68.5**    | 74.9    | 66.6    | 67.8    | 70.6    | 62.4    |\n| 19 | \u003Cimg src=\"assets\u002Ficon\u002Fskywork.jpeg\" height=\"18\"> MatrixGame2 | **68.5**    | 75.7    | 67.1    | 80.6    | 62.0    | 57.2    |\n| 20 | \u003Cimg src=\"assets\u002Ficon\u002Fthu.png\" height=\"18\"> Astra | **64.0**    | 69.7    | 59.6    | 67.7    | 71.6    | 51.4    |\n\n\n**9 Text-driven Models — Full Split (5 Dimensions, sorted by average)**\n\n| # | Model | **Average** | Quality | Setting | Interaction | Consistency | Physical |\n|:---:|:---|:---:|:---:|:---:|:---:|:---:|:---:|\n| 1 | \u003Cimg src=\"assets\u002Ficon\u002Fkling.jpeg\" height=\"18\"> Kling 3.0 | **79.5 🥇** | 81.8 🥉 | 91.0 🥈 | 73.1 🥇 | 82.6    | 69.2 🥈 |\n| 2 | \u003Cimg src=\"assets\u002Ficon\u002Fwan.png\" height=\"18\"> Wan 2.7 | **78.2 🥈** | 82.2 🥈 | 91.4 🥇 | 72.1 🥈 | 73.8    | 71.6 🥇 |\n| 3 | \u003Cimg src=\"assets\u002Ficon\u002Fbytedance.png\" height=\"18\"> Seedance 1.5 | **76.2 🥉** | 83.0 🥇 | 82.9    | 68.3 🥉 | 78.5    | 68.2    |\n| 4 | \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-Video 1.5 | **74.6**    | 78.9    | 85.6 🥉 | 54.7    | 86.8 🥇 | 67.1    |\n| 5 | \u003Cimg src=\"assets\u002Ficon\u002Flightrix.jpeg\" height=\"18\"> LTX 2.3 | **71.0**    | 78.8    | 85.2    | 49.4    | 76.4    | 65.1    |\n| 6 | \u003Cimg src=\"assets\u002Ficon\u002Fcosmos.png\" height=\"18\"> Cosmos 2.5 | **70.8**    | 74.6    | 83.3    | 43.5    | 85.4 🥉 | 67.0    |\n| 7 | \u003Cimg src=\"assets\u002Ficon\u002Flongcat.png\" height=\"18\"> LongCat-Video | **70.2**    | 79.7    | 72.3    | 45.1    | 85.5 🥈 | 68.4 🥉 |\n| 8 | \u003Cimg src=\"assets\u002Ficon\u002Fshlab.png\" height=\"18\"> YUME 1.5 | **69.0**    | 79.7    | 72.4    | 48.4    | 79.3    | 65.4    |\n| 9 | \u003Cimg src=\"assets\u002Ficon\u002Fkairos.png\" height=\"18\"> Kairos 3.0 | **66.0**    | 75.8    | 70.3    | 41.6    | 81.9    | 60.5    |\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>20 Models — Navigation Split (19 metrics)\u003C\u002Fb>\u003C\u002Fsummary>\n\n| Model | Aesthetic Quality | Imaging Quality | Background Consistency | Temporal Flickering | Dynamic Degree | Motion Smoothness | HPSv3 Quality | Scene Adherence | Subject Adherence | Navigation Trajectory | Spatial Consistency | Gated Spatial Consistency | Perspective Consistency | Segment Continuity | Geometric Consistency | Photometric Consistency | Subject Consistency Cross-Model | Visual Plausibility | Causal Fidelity |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-Video 1.5 | 63.4    | 67.4    | 92.1    | 94.2    | 73.9    | 98.7    | 68.0    | 77.5    | 93.6    | 71.8    | 79.2    | 75.1    | 86.6    | 99.4    | 94.6    | 80.3    | 91.6    | 59.7    | 75.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fkling.jpeg\" height=\"18\"> Kling 3.0 | 63.0    | 68.1    | 92.3    | 93.2    | 97.5    | 97.6    | 69.1    | 89.0    | 92.9    | 70.3    | 75.2    | 75.1    | 76.8    | 93.0    | 88.9    | 79.9    | 88.5    | 60.7    | 78.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fcosmos.png\" height=\"18\"> Cosmos 2.5 | 61.8    | 66.9    | 92.3    | 94.8    | 49.0    | 98.2    | 66.5    | 72.4    | 94.2    | 64.1    | 78.1    | 74.3    | 84.3    | 94.3    | 94.6    | 81.6    | 92.3    | 60.1    | 74.7    |\n| \u003Cimg src=\"assets\u002Ficon\u002Flightrix.jpeg\" height=\"18\"> LTX 2.3 | 57.9    | 61.0    | 88.3    | 93.2    | 98.1    | 96.4    | 56.1    | 81.3    | 89.2    | 67.6    | 70.2    | 70.2    | 69.8    | 75.8    | 76.9    | 79.2    | 87.2    | 55.7    | 74.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fbytedance.png\" height=\"18\"> Seedance 1.5 | 61.0    | 69.3    | 89.6    | 92.4    | 99.4    | 97.5    | 73.0    | 71.6    | 94.2    | 68.0    | 72.7    | 72.4    | 70.5    | 96.2    | 82.4    | 76.8    | 90.1    | 60.7    | 76.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fwan.png\" height=\"18\"> Wan 2.7 | 61.4    | 68.0    | 89.4    | 92.2    | 100.0    | 96.3    | 71.1    | 88.3    | 94.6    | 66.0    | 71.0    | 71.0    | 78.2    | 92.4    | 83.7    | 76.4    | 90.7    | 60.3    | 83.3    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fkairos.png\" height=\"18\"> Kairos 3.0 | 59.9    | 62.7    | 91.1    | 95.4    | 70.1    | 97.5    | 58.5    | 52.2    | 88.5    | 65.1    | 76.8    | 62.0    | 76.3    | 94.3    | 89.0    | 80.8    | 90.8    | 58.0    | 62.7    |\n| \u003Cimg src=\"assets\u002Ficon\u002Flongcat.png\" height=\"18\"> LongCat-Video | 66.5    | 69.6    | 95.1    | 94.8    | 45.9    | 97.9    | 77.6    | 53.1    | 91.5    | 63.1    | 83.3    | 66.2    | 81.5    | 99.4    | 95.4    | 82.2    | 93.4    | 61.8    | 76.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fshlab.png\" height=\"18\"> YUME 1.5 | 58.7    | 63.3    | 90.3    | 93.0    | 96.8    | 97.0    | 57.0    | 53.1    | 91.7    | 72.0    | 71.5    | 71.4    | 48.0    | 99.4    | 88.0    | 83.3    | 88.8    | 57.7    | 72.7    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fthu.png\" height=\"18\"> Astra | 48.6    | 52.5    | 85.3    | 96.0    | 79.6    | 97.7    | 28.0    | 43.4    | 75.9    | 67.7    | 64.7    | 63.3    | 30.0    | 86.6    | 85.6    | 87.5    | 83.5    | 54.6    | 48.3    |\n| \u003Cimg src=\"assets\u002Ficon\u002Famap.png\" height=\"18\"> Fantasy-World | 63.0    | 62.8    | 94.2    | 95.8    | 49.0    | 97.9    | 65.8    | 52.4    | 90.1    | 72.1    | 80.6    | 64.2    | 79.8    | 100.0    | 95.3    | 84.8    | 92.5    | 59.7    | 74.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-GameCraft | 52.6    | 58.7    | 86.5    | 93.7    | 96.8    | 97.6    | 38.3    | 50.6    | 82.5    | 67.8    | 60.5    | 60.5    | 17.9    | 99.4    | 88.3    | 85.0    | 82.6    | 56.5    | 68.3    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fgoogle.png\" height=\"18\"> Genie 3 | 51.6    | 59.3    | 90.7    | 95.0    | 92.4    | 97.8    | 55.2    | 61.1    | 83.8    | 73.3    | 79.9    | 78.4    | 54.5    | 93.6    | 88.6    | 84.5    | 90.4    | 59.7    | 71.7    |\n| \u003Cimg src=\"assets\u002Ficon\u002Falibaba.png\" height=\"18\"> Happy Oyster | 56.6    | 63.9    | 91.4    | 94.0    | 94.2    | 97.0    | 58.3    | 57.4    | 91.1    | 85.1    | 77.7    | 75.8    | 75.0    | 96.2    | 87.2    | 79.8    | 91.5    | 57.6    | 69.3    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-World 1.5 | 60.1    | 65.4    | 92.7    | 93.5    | 91.1    | 98.1    | 60.5    | 53.5    | 90.8    | 87.5    | 90.6    | 84.9    | 62.5    | 100.0    | 92.0    | 83.1    | 89.1    | 58.6    | 74.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fmeituan.png\" height=\"18\"> Infinite-World | 58.7    | 66.1    | 88.8    | 94.1    | 82.8    | 98.0    | 62.3    | 54.0    | 84.5    | 75.9    | 74.9    | 74.4    | 33.8    | 100.0    | 94.3    | 85.1    | 88.4    | 57.2    | 67.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Finspatio.jpeg\" height=\"18\"> InSpatio-World | 64.4    | 67.6    | 95.0    | 96.0    | 26.1    | 98.8    | 76.1    | 51.7    | 91.1    | 72.8    | 93.8    | 66.5    | 72.5    | 100.0    | 97.3    | 87.4    | 94.4    | 63.1    | 67.3    |\n| \u003Cimg src=\"assets\u002Ficon\u002Flingbot.png\" height=\"18\"> LingBot-World | 66.9    | 67.9    | 96.9    | 94.1    | 66.2    | 96.9    | 81.4    | 51.6    | 93.6    | 79.8    | 92.7    | 67.1    | 90.9    | 99.4    | 95.4    | 83.3    | 93.5    | 64.8    | 77.7    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fskywork.jpeg\" height=\"18\"> MatrixGame2 | 54.0    | 60.3    | 86.9    | 94.6    | 94.9    | 98.2    | 41.0    | 49.4    | 84.9    | 80.6    | 64.5    | 64.5    | 29.2    | 21.0    | 86.1    | 81.3    | 87.2    | 55.0    | 59.3    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fskywork.jpeg\" height=\"18\"> MatrixGame3 | 46.4    | 70.0    | 85.7    | 86.3    | 97.5    | 95.4    | 57.1    | 48.9    | 78.4    | 83.5    | 81.0    | 80.4    | 13.3    | 89.8    | 87.6    | 75.3    | 83.0    | 54.0    | 64.7    |\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>9 Text-driven Models — Full Split (22 metrics)\u003C\u002Fb>\u003C\u002Fsummary>\n\n| Model | Aesthetic Quality | Imaging Quality | Background Consistency | Temporal Flickering | Dynamic Degree | Motion Smoothness | HPSv3 Quality | Scene Adherence | Subject Adherence | Navigation Trajectory | Event Edit Adherence | Subject Action Adherence | Perspective Switch Adherence | Spatial Consistency | Gated Spatial Consistency | Perspective Consistency | Segment Continuity | Geometric Consistency | Photometric Consistency | Subject Consistency Cross-Model | Visual Plausibility | Causal Fidelity |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| \u003Cimg src=\"assets\u002Ficon\u002Fhunyuan.png\" height=\"18\"> HY-Video 1.5 | 61.9    | 67.4    | 92.4    | 95.5    | 68.8    | 98.8    | 67.5    | 77.5    | 93.6    | 71.8    | 63.8    | 55.6    | 27.6    | 79.2    | 75.1    | 86.6    | 99.3    | 94.4    | 81.4    | 91.5    | 59.3    | 75.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fkling.jpeg\" height=\"18\"> Kling 3.0 | 61.3    | 67.7    | 92.7    | 94.5    | 89.9    | 97.9    | 68.8    | 89.0    | 92.9    | 70.3    | 81.4    | 85.6    | 55.0    | 75.2    | 75.1    | 76.8    | 92.7    | 89.4    | 80.4    | 88.5    | 60.4    | 78.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fcosmos.png\" height=\"18\"> Cosmos 2.5 | 60.1    | 67.2    | 92.3    | 96.0    | 42.4    | 98.3    | 65.9    | 72.4    | 94.2    | 64.1    | 48.2    | 41.6    | 20.0    | 78.1    | 74.3    | 84.3    | 93.1    | 94.2    | 82.1    | 91.8    | 59.3    | 74.7    |\n| \u003Cimg src=\"assets\u002Ficon\u002Flightrix.jpeg\" height=\"18\"> LTX 2.3 | 56.9    | 62.3    | 89.3    | 94.1    | 94.4    | 96.8    | 57.7    | 81.3    | 89.2    | 67.6    | 53.0    | 51.8    | 25.0    | 70.2    | 70.2    | 69.8    | 77.8    | 81.1    | 79.4    | 86.7    | 56.2    | 74.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fbytedance.png\" height=\"18\"> Seedance 1.5 | 59.7    | 69.8    | 89.6    | 93.4    | 98.3    | 97.6    | 72.9    | 71.6    | 94.2    | 68.0    | 80.4    | 80.0    | 45.0    | 72.7    | 72.4    | 62.7    | 92.4    | 83.5    | 76.7    | 89.3    | 60.5    | 76.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fwan.png\" height=\"18\"> Wan 2.7 | 59.6    | 68.1    | 89.5    | 93.0    | 99.3    | 96.5    | 69.4    | 88.3    | 94.6    | 66.0    | 84.0    | 83.4    | 55.0    | 71.0    | 71.0    | 62.2    | 65.6    | 82.6    | 75.5    | 88.7    | 59.8    | 83.3    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fkairos.png\" height=\"18\"> Kairos 3.0 | 58.4    | 63.6    | 91.8    | 96.3    | 63.5    | 97.9    | 58.8    | 52.2    | 88.5    | 65.1    | 46.8    | 41.4    | 13.3    | 76.8    | 62.0    | 76.3    | 94.1    | 91.5    | 82.1    | 90.7    | 58.2    | 62.7    |\n| \u003Cimg src=\"assets\u002Ficon\u002Flongcat.png\" height=\"18\"> LongCat-Video | 64.7    | 69.8    | 94.7    | 94.9    | 59.7    | 97.7    | 76.3    | 53.1    | 91.5    | 63.1    | 50.4    | 48.4    | 18.3    | 83.3    | 66.2    | 81.5    | 98.6    | 94.7    | 81.5    | 92.4    | 60.8    | 76.0    |\n| \u003Cimg src=\"assets\u002Ficon\u002Fshlab.png\" height=\"18\"> YUME 1.5 | 59.3    | 65.7    | 92.0    | 94.8    | 86.1    | 97.7    | 62.0    | 53.1    | 91.7    | 72.0    | 57.8    | 47.0    | 16.7    | 71.5    | 71.4    | 48.0    | 99.3    | 91.1    | 84.1    | 89.4    | 58.1    | 72.7    |\n\n\u003C\u002Fdetails>\n\n## 🚀 Quick Start\n\n```bash\n# Install\ngit clone --recursive https:\u002F\u002Fgithub.com\u002Fmeituan-longcat\u002FWBench.git\ncd WBench\n\n# Download data and weights\npip install huggingface_hub\nhf download meituan-longcat\u002FWBench --repo-type dataset --local-dir data\u002F --exclude \"splits\u002F*\"\nhf download meituan-longcat\u002FWBench-weights --local-dir weights\u002F\n\n# Environment 1: wbench-main (all metrics except visual_plausibility)\nbash tools\u002Finstall.sh wbench-main\nconda activate wbench-main\nexport LD_LIBRARY_PATH=$CONDA_PREFIX\u002Flib:$LD_LIBRARY_PATH\n\n# Environment 2: wbench-vp (visual_plausibility only, requires vLLM)\nbash tools\u002Finstall_vp.sh wbench-vp\n\n# Verify\nconda activate wbench-main\npython tools\u002Fverify_install.py\n\n# Run evaluation (auto multi-GPU)\npython main.py --model your_model\n```\n\nSee [docs\u002Finstallation.md](docs\u002Finstallation.md) for detailed setup instructions.\n\n## 🎮 Evaluate Your Model\n\n1. Generate multi-turn videos → place in `work_dirs\u002F\u003Cmodel>\u002Fvideos\u002Fcase_{id}_combined.mp4`\n2. Run the 3-phase pipeline:\n\n```bash\n# Full pipeline (precompute → GPU metrics → VLM metrics → report)\npython main.py --model my_model --gpus 0,1,2,3,4,5,6,7\n\n# Or run phases independently:\npython main.py --model my_model --phase precompute    # SAM2 + DA3 + MegaSAM\npython main.py --model my_model --phase gpu           # GPU metrics (per-metric)\npython main.py --model my_model --phase vlm           # VLM metrics (API)\npython main.py --model my_model --phase report        # Aggregate report\n```\n\n3. Results: `work_dirs\u002F\u003Cmodel>\u002Fevaluation\u002F{metric}\u002Fcase_{id}.json` + `report.json`\n\n```bash\n# Run specific metrics (by name or dimension)\npython main.py --model my_model --phase gpu --metrics hpsv3_quality\npython main.py --model my_model --phase gpu --metrics renderer        # all 6 video quality\npython main.py --model my_model --phase gpu --metrics consistency     # all consistency metrics\n\n# Skip pre-computation if already done\npython main.py --model my_model --phase gpu --skip_megasam --skip_sam2 --skip_da3\n\n# Single video evaluation\npython main.py --video video.mp4 --case data\u002Fcases\u002Fcase_1.json\n```\n\n**Dimensions** (`--metrics` supports these as shorthand):\n| Dimension | Metrics |\n|:---|:---|\n| `quality` | aesthetic_quality, imaging_quality, temporal_flickering, dynamic_degree, motion_smoothness, hpsv3_quality |\n| `consistency` | background_consistency, segment_continuity, perspective_consistency, subject_consistency, geometric_consistency, photometric_consistency, spatial_consistency, gated_spatial_consistency |\n| `interaction` | navigation_trajectory, event_edit_adherence, subject_action_adherence, perspective_switch_adherence |\n| `setting` | scene_adherence, subject_adherence |\n| `physical` | visual_plausibility, causal_fidelity |\n\nSet environment variables for VLM metrics (we use [Doubao-Seed-2.0-lite](https:\u002F\u002Fconsole.volcengine.com\u002Fark\u002Fregion:ark+cn-beijing\u002Fmodel\u002Fdetail?Id=doubao-seed-2-0-lite) via [Volcengine ARK](https:\u002F\u002Fwww.volcengine.com\u002Fdocs\u002F82379\u002F1099475)):\n```bash\nexport VLM_API_KEY=\"\u003Cyour-ark-api-key>\"\n# Optional (defaults shown):\n# export VLM_API_URL=\"https:\u002F\u002Fark.cn-beijing.volces.com\u002Fapi\u002Fv3\"\n# export VLM_MODEL_NAME=\"doubao-seed-2-0-lite-260215\"\n```\n\nFor `visual_plausibility`, use the separate `wbench-vp` environment:\n```bash\npython tools\u002Frun_visual_plausibility.py --model my_model  # uses all available GPUs\n```\n\n\n## 🔌 Implement Your Model\n\nWBench supports 3 model types with different control interfaces:\n\n| Type | Input | Cases | Status |\n|:---|:---|:---:|:---:|\n| **Text-conditioned** | Text prompt + first-frame image | 289 (all) | ✅ Implemented |\n| **Camera-conditioned** | First-frame image + 6-DoF camera pose | 158 (navi) | 🚧 Coming soon |\n| **Action-conditioned** | First-frame image + discrete action | 158 (navi) | 🚧 Coming soon |\n\n### Text-conditioned models\n\n```python\nfrom src.models import get_model\n\n# Available: wan, kling, seedance (or register your own)\nmodel = get_model(\"wan\")\n\n# Generate multi-turn video from a case\nresult = model.generate_multi_turn(\n    case=case_dict,\n    output_path=\"work_dirs\u002Fwan\u002Fvideos\u002Fcase_1_combined.mp4\",\n    data_root=\"data\u002F\",\n)\n```\n\nEach turn: build prompt from interaction → call I2V API → extract last frame → next turn.\n\nSet API credentials:\n```bash\nexport VIDEO_API_URL=\"https:\u002F\u002Fyour-video-api.com\"\nexport VIDEO_API_KEY=\"your-key\"\n```\n\n### Camera-conditioned models\n\n🚧 Coming soon — will accept camera extrinsics per turn.\n\n### Action-conditioned models\n\n🚧 Coming soon — will accept discrete action signals per frame\u002Fturn.\n\n## 📋 TODO\n\n- [x] Text-conditioned model generation (Wan, Kling, Seedance)\n- [x] Homepage with interactive leaderboard\n- [x] Dataset and weights release on HuggingFace\n- [ ] Camera-conditioned model generation example\n- [ ] Action-conditioned model generation example\n- [ ] Model submission portal (auto-evaluation)\n- [x] ArXiv paper release\n\n## 📝 Citation\n\nIf you find our work useful, please consider citing:\n\n```bibtex\n@article{ying2026wbenchcomprehensivemultiturnbenchmark,\n  title={WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation},\n  author={Ying, Kaining and Hu, Hengrui and Ren, Siyu and Li, Jiamu and Chen, Fengjiao and Wang, Ziwen and Cao, Xuezhi and Cai, Xunliang and Ding, Henghui},\n  journal={arXiv preprint arXiv:2605.25874},\n  year={2026}\n}\n```\n\n## 🙏 Acknowledgement\n\nThis project builds upon the following excellent works:\n\n- [WorldScore](https:\u002F\u002Fgithub.com\u002FWorldScore\u002FWorldScore) — World model evaluation framework\n- [VBench](https:\u002F\u002Fgithub.com\u002FVchitect\u002FVBench) — Video quality metrics\n- [SAM2](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsam2) — Segment Anything Model 2 for mask tracking\n- [Depth-Anything-V3](https:\u002F\u002Fgithub.com\u002FDepthAnything\u002FDepth-Anything-V3) — Monocular depth estimation\n- [MegaSAM](https:\u002F\u002Fgithub.com\u002Fmega-sam\u002Fmega-sam) — Camera pose estimation\n- [DreamSim](https:\u002F\u002Fgithub.com\u002Fssundaram21\u002Fdreamsim) — Perceptual similarity metric\n- [HPSv3](https:\u002F\u002Fgithub.com\u002Ftgxs002\u002FHPSv2) — Human Preference Score\n- [AMT](https:\u002F\u002Fgithub.com\u002FMCG-NKU\u002FAMT) — Frame interpolation for motion smoothness\n- [RAFT](https:\u002F\u002Fgithub.com\u002Fprinceton-vl\u002FRAFT) — Optical flow estimation\n- [TransNetV2](https:\u002F\u002Fgithub.com\u002FsoCzech\u002FTransNetV2) — Scene boundary detection\n- ... and many other excellent open-source projects\n\n## 📧 Contact\n\nFeel free to open an [Issue](https:\u002F\u002Fgithub.com\u002Fmeituan-longcat\u002FWBench\u002Fissues) or [Pull Request](https:\u002F\u002Fgithub.com\u002Fmeituan-longcat\u002FWBench\u002Fpulls). You can also reach us directly:\n\n- **Kaining Ying**: `kaining.ying.cv@gmail.com`\n- **Siyu Ren**: `rensiyu07@meituan.com`\n\n## 📄 License\n\nCode and data: [MIT License](LICENSE). Model weights retain their [original licenses](https:\u002F\u002Fhuggingface.co\u002Fmeituan-longcat\u002FWBench-weights\u002Fblob\u002Fmain\u002FLICENSE_NOTICE.md).\n","WBench 是一个用于评估交互式视频世界模型的多轮综合基准。它通过5个维度和22个指标对20种视频世界模型进行全面评测，旨在帮助研究人员和开发者了解不同模型在各种任务中的表现。项目采用Python语言编写，具备强大的数据处理与分析能力，并提供了详细的评测结果和排行榜。适合需要对视频理解、生成及交互模型进行性能对比分析的研究场景或工业应用环境使用。","2026-06-11 03:58:46","CREATED_QUERY"]