[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-2315":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":16,"stars30d":17,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":18,"rankGlobal":10,"rankLanguage":10,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":40,"readmeContent":41,"aiSummary":42,"trendingCount":16,"starSnapshotCount":16,"syncStatus":15,"lastSyncTime":43,"discoverSource":44},2315,"VEFX-Bench","Visko-Platform\u002FVEFX-Bench","Visko-Platform","VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects","https:\u002F\u002Fxiangbogaobarry.github.io\u002FVEFX-Bench\u002F",null,"Python",216,17,16,2,0,75,51.27,"Apache License 2.0",false,"main",true,[24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39],"agent","ai","artificial-intelligence","generative-ai","image","image-editing","image-generation","skills","vfx","video","video-editing","video-gen","video-generation","video-processing","video-quality-assessment","visualization","2026-06-12 04:00:14","## 🔥 News\n\n> **[May 16, 2026]** 🚀 **VEFX-Reward-32B is now publicly available!** Our 32B reward modelDownload it now: [🤗 viskoplatform\u002FVEFX-Reward-32B](https:\u002F\u002Fhuggingface.co\u002Fviskoplatform\u002FVEFX-Reward-32B). One-click inference: `VEFXReward(\"32B\")`.\n\n---\n\n\u003Cdiv align=\"center\">\n\n# VEFX-Bench\n\n### Benchmarking Generic Video Editing and Visual Effects\n\n[📄 Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.16272) •\n[💻 Code](https:\u002F\u002Fgithub.com\u002FVisko-Platform\u002FVEFX-Bench) •\n[🤗 Dataset](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fxiangbog\u002FVEFX-Bench) •\n[🤗 Model (4B)](https:\u002F\u002Fhuggingface.co\u002Fxiangbog\u002FVEFX-Reward-4B) •\n[🤗 Model (32B)](https:\u002F\u002Fhuggingface.co\u002Fviskoplatform\u002FVEFX-Reward-32B) •\n[🏆 Leaderboard](https:\u002F\u002Fvefx-leaderboard.com\u002F) •\n[🌐 Project Page](https:\u002F\u002Fxiangbogaobarry.github.io\u002FVEFX-Bench\u002F)\n\n\u003C\u002Fdiv>\n\n**VEFX-Bench** is a comprehensive benchmark for evaluating text-driven video editing and visual effects. It includes **5,049 annotated examples** spanning **9 categories** and **32 subcategories**, evaluated by **VEFX-Reward** — a VLM-based reward model that scores edits across three dimensions on a 1–4 scale:\n\n| Dimension | What it measures |\n|---|---|\n| **Instructional Following (IF)** | Does the edit accurately reflect the editing instruction? |\n| **Render Quality (RQ)** | Visual clarity, temporal consistency, and physical plausibility |\n| **Edit Exclusivity (EE)** | Were only the intended regions modified, without side-effects? |\n\n---\n\n## 🏆 Model Leaderboard\n\nVEFX-Reward scores on 1–4 scale. Ranked by **GeoAgg** (α=2 for IF, β=1 for RQ, γ=1 for EE). Higher is better.\n\n> **📅 Updated: May 2, 2026** — For the latest results & submissions, visit the **[live leaderboard →](https:\u002F\u002Fvefx-leaderboard.com\u002F)**\n\n| Rank | Model | Type | IF ↑ | RQ ↑ | EE ↑ | GeoAgg ↑ |\n|:---:|---|---|:---:|:---:|:---:|:---:|\n| 🥇 | **Kling o3 Omni** | Commercial | 3.033 | **3.588** | 3.043 | **3.057** |\n| 🥈 | **Kling o1** | Commercial | **3.040** | 3.534 | 2.976 | 2.985 |\n| 🥉 | **Runway Gen-4.5** | Commercial | 2.817 | 3.319 | 2.923 | 2.912 |\n| 4 | Seedance 2.0 | Commercial | 2.811 | 3.421 | 3.088 | 2.766 |\n| 5 | Grok Imagine | Commercial | 2.606 | 3.346 | **3.376** | 2.723 |\n| 6 | Luma Ray 3 | Commercial | 2.702 | 3.403 | 2.705 | 2.717 |\n| 7 | UniVideo | Open-source | 2.294 | 3.266 | 3.091 | 2.516 |\n| 8 | Wan 2.6 | Commercial | 2.012 | 3.317 | 2.446 | 2.146 |\n| 9 | Luma Ray 2 | Commercial | 2.038 | 2.532 | 1.363 | 1.804 |\n| 10 | VACE | Open-source | 2.027 | 3.172 | 1.180 | 1.775 |\n\n---\n\n## 🎬 Demo Videos\n\nEach demo shows the **original video** (left) alongside the **edited video** (right).\n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd align=\"center\">\u003Cb>Attribute Change\u003C\u002Fb>\u003Cbr>\u003Csub>\"Change the color of the red industrial trailer to a bright yellow while maintaining the texture and appearance of the metal surface.\"\u003C\u002Fsub>\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>Object Removal\u003C\u002Fb>\u003Cbr>\u003Csub>\"Remove the woman with the grey backpack walking on the right side of the frame.\"\u003C\u002Fsub>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd align=\"center\">\u003Cimg src=\"assets\u002Fdemo_attribute_change.gif\" width=\"400\">\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cimg src=\"assets\u002Fdemo_object_removal.gif\" width=\"400\">\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd align=\"center\">\u003Cb>Style Transfer\u003C\u002Fb>\u003Cbr>\u003Csub>\"Restore the natural, realistic colors to the entire scene, replacing the current black and white style with a full-color rendition.\"\u003C\u002Fsub>\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>Camera Motion\u003C\u002Fb>\u003Cbr>\u003Csub>\"Perform a smooth zoom in on the distant snowy mountain peaks to create a more immersive view.\"\u003C\u002Fsub>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd align=\"center\">\u003Cimg src=\"assets\u002Fdemo_style_transfer.gif\" width=\"400\">\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cimg src=\"assets\u002Fdemo_camera_zoom.gif\" width=\"400\">\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n---\n\n## 📊 Benchmark at a Glance\n\n| | |\n|---|---|\n| 📝 **5,049** Annotated Examples | 🎬 **1,419** Source Videos |\n| 📂 **9 \u002F 32** Categories \u002F Subcategories | 🤖 **10** Editing Systems |\n| 📐 **3** Quality Dimensions (IF, RQ, EE) | 🧪 **300** Benchmark Test Pairs |\n\n---\n\n## 🤗 VEFX-Reward Models\n\n| Model | Backbone | Params | HuggingFace | Status |\n|---|---|---|---|---|\n| **VEFX-Reward-4B** | Qwen3-VL-4B-Instruct | 4B | [xiangbog\u002FVEFX-Reward-4B](https:\u002F\u002Fhuggingface.co\u002Fxiangbog\u002FVEFX-Reward-4B) | ✅ Available |\n| **VEFX-Reward-32B** | Qwen3-VL-32B-Instruct | 32B | [viskoplatform\u002FVEFX-Reward-32B](https:\u002F\u002Fhuggingface.co\u002Fviskoplatform\u002FVEFX-Reward-32B) | ✅ Available |\n\n> The 32B model needs ~65 GB VRAM in bfloat16. Use `VEFXReward(\"viskoplatform\u002FVEFX-Reward-32B\", device=\"cuda\")` for one-line inference.\n\n---\n\n## 🚀 Quick Start\n\n### Installation\n\n```bash\nconda create -n vefx-bench python=3.10 -y\nconda activate vefx-bench\n\n# Install PyTorch first (match your CUDA version)\n# See https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F for the right command\npip install torch torchvision --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fcu124\n\n# Install remaining dependencies\npip install -r requirements.txt\n\n# Install the package\npip install -e .\n```\n\n> **Requirements:** Python ≥ 3.10, CUDA GPU. The 4B model needs ~10 GB VRAM, the 32B model needs ~65 GB VRAM (bfloat16). Make sure your PyTorch CUDA version matches your driver.\n\n### Pick your model size\n\n```python\nfrom vefx_reward import VEFXReward\n\n# Fast \u002F low-VRAM (~10 GB)\nmodel = VEFXReward(\"xiangbog\u002FVEFX-Reward-4B\", device=\"cuda\")\n\n# Higher accuracy (~65 GB VRAM, e.g. a single H100 80 GB)\nmodel = VEFXReward(\"viskoplatform\u002FVEFX-Reward-32B\", device=\"cuda\")\n```\n\n### Score a Video Edit (Python API)\n\n```python\nfrom vefx_reward import VEFXReward\n\n# 4B (default, ~10 GB VRAM)\nmodel = VEFXReward(\"xiangbog\u002FVEFX-Reward-4B\", device=\"cuda\")\n\n# Or one-click 32B (highest accuracy, ~65 GB VRAM)\n# model = VEFXReward(\"viskoplatform\u002FVEFX-Reward-32B\", device=\"cuda\")\n# model = VEFXReward(\"32B\", device=\"cuda\")  # same thing — short alias\n\nscores = model.score(\n    original_video=\"examples\u002Fsample_videos\u002Fobject_removal_original.mp4\",\n    edited_video=\"examples\u002Fsample_videos\u002Fobject_removal_edited.mp4\",\n    instruction=\"Remove the woman with the grey backpack walking on the right side of the frame.\",\n)\nprint(scores)\n# {'IF': 2.34, 'RQ': 1.93, 'EE': 1.82, 'Overall': 6.09}\n```\n\n### CLI Usage\n\n```bash\n# 4B\npython examples\u002Fquick_start.py \\\n    --original examples\u002Fsample_videos\u002Fobject_removal_original.mp4 \\\n    --edited examples\u002Fsample_videos\u002Fobject_removal_edited.mp4 \\\n    --instruction \"Remove the woman with the grey backpack walking on the right side of the frame.\"\n\n# 32B (one-click — alias automatically resolves to viskoplatform\u002FVEFX-Reward-32B)\npython examples\u002Fquick_start.py --model 32B --run_samples\n```\n\n### Score All Included Samples\n\nThe repo includes 4 sample video pairs with prompts. Score them all:\n\n```python\nimport json\nfrom vefx_reward import VEFXReward\n\nmodel = VEFXReward(\"xiangbog\u002FVEFX-Reward-4B\", device=\"cuda\")\n\nwith open(\"examples\u002Fsample_videos\u002Fprompts.json\") as f:\n    samples = json.load(f)\n\nfor sample in samples:\n    scores = model.score(\n        original_video=f\"examples\u002Fsample_videos\u002F{sample['original']}\",\n        edited_video=f\"examples\u002Fsample_videos\u002F{sample['edited']}\",\n        instruction=sample[\"instruction\"],\n    )\n    print(f\"[{sample['category']}] IF={scores['IF']:.2f}  RQ={scores['RQ']:.2f}  EE={scores['EE']:.2f}\")\n```\n\n### Batch Scoring\n\nPrepare a CSV with columns `original_video`, `edited_video`, `instruction`:\n\n```bash\npython examples\u002Fbatch_scoring.py --csv edits.csv --output results.csv\n```\n\n### Multi-GPU Scoring\n\nFor large-scale evaluation across multiple GPUs:\n\n```bash\npython examples\u002Fmulti_gpu_scoring.py --csv edits.csv --num_gpus 4 --output results.csv\n```\n\n---\n\n## 📖 API Reference\n\n### `VEFXReward`\n\n```python\nVEFXReward(\n    model_path=\"xiangbog\u002FVEFX-Reward-4B\",  # HuggingFace ID or local path\n    device=\"cuda\",                           # \"cuda\", \"cuda:0\", \"cpu\"\n    dtype=torch.bfloat16,                    # torch.bfloat16 or torch.float16\n    fps=4.0,                                 # Video sampling rate\n    max_frame_pixels=399360,                 # Max pixels per frame\n)\n```\n\n#### `model.score(original_video, edited_video, instruction) → dict`\n\nScore a single video edit. Returns `{'IF': float, 'RQ': float, 'EE': float, 'Overall': float}`.\n\n#### `model.score_batch(original_videos, edited_videos, instructions) → list[dict]`\n\nScore multiple edits sequentially. Each sample is processed independently to avoid OOM.\n\n---\n\n## 📝 Citation\n\n```bibtex\n@article{gao2026vefx,\n  title={VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects},\n  author={Gao, Xiangbo and Jiang, Sicong and Liu, Bangya and Chen, Xinghao and Yang, Minglai and Yang, Siyuan and Wu, Mingyang and Yu, Jiongze and Zheng, Qi and Wang, Haozhi and others},\n  journal={arXiv preprint arXiv:2604.16272},\n  year={2026}\n}\n```\n\n## License\n\nThis project is licensed under the Apache License 2.0. See [LICENSE](LICENSE) for details.\n","VEFX-Bench 是一个全面的基准测试工具，用于评估基于文本驱动的视频编辑和视觉效果。该项目提供了一个包含5,049个标注示例的数据集，覆盖了9个大类和32个子类。核心功能包括使用名为VEFX-Reward的VLM（视觉-语言模型）奖励模型对编辑结果进行评分，该模型从指令遵循度、渲染质量和编辑专属性三个维度对编辑效果进行1-4分的评价。VEFX-Bench 适合于需要高质量视频编辑与视觉特效评估的研究人员、开发者以及商业应用中，尤其是在生成式AI领域内寻求提升视频处理技术准确性和视觉表现力的场景下。项目采用Python编写，并在Apache License 2.0许可下开源。","2026-06-11 02:49:28","CREATED_QUERY"]