[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92378":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":8,"htmlUrl":8,"language":9,"languages":8,"totalLinesOfCode":8,"stars":10,"forks":11,"watchers":12,"openIssues":11,"contributorsCount":11,"subscribersCount":11,"size":11,"stars1d":11,"stars7d":11,"stars30d":13,"stars90d":11,"forks30d":11,"starsTrendScore":11,"compositeScore":14,"rankGlobal":8,"rankLanguage":8,"license":8,"archived":15,"fork":15,"defaultBranch":16,"hasWiki":15,"hasPages":15,"topics":17,"createdAt":8,"pushedAt":8,"updatedAt":18,"readmeContent":19,"aiSummary":20,"trendingCount":11,"starSnapshotCount":11,"syncStatus":21,"lastSyncTime":22,"discoverSource":23},92378,"WorldDirector","pPetrichor\u002FWorldDirector","pPetrichor",null,"Python",57,0,51,6,34.6,false,"main",[],"2026-07-22 04:02:06","# WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory\n\n**[[📄 Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.02517)]**\n**[[🌐 Project Page](https:\u002F\u002Fworlddirector.github.io\u002F)]**\n**[[🤗 Model Weights](https:\u002F\u002Fhuggingface.co\u002Fhlwang06\u002FWorldDirector)]**\n\n\u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fa8fa3819-db91-4767-9e5a-44b99e5272af\" controls width=\"100%\">\u003C\u002Fvideo>\n\n***[Hanlin Wang](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=0uO4fzkAAAAJ&hl=zh-CN), [Hao Ouyang](https:\u002F\u002Fken-ouyang.github.io\u002F), [Qiuyu Wang](https:\u002F\u002Fgithub.com\u002Fqiuyu96), [Wen Wang](https:\u002F\u002Fgithub.com\u002Fencounter1997), [Qingyan Bai](https:\u002F\u002Fbqy.info\u002F), [Ka Leong Cheng](https:\u002F\u002Ffelixcheng97.github.io\u002F), [Yue Yu](https:\u002F\u002Fbruceyy.com\u002F), [Yixuan Li](https:\u002F\u002Fyixuanli98.github.io\u002F), [Yihao Meng](https:\u002F\u002Fyihao-meng.github.io\u002F), [Zichen Liu](https:\u002F\u002Fzliucz.github.io\u002F), [Yanhong Zeng](https:\u002F\u002Fzengyh1900.github.io\u002F), [Yujun Shen](https:\u002F\u002Fshenyujun.github.io\u002F), [Qifeng Chen](https:\u002F\u002Fcqf.io\u002F)***\n\n# TLDR\n\nWorldDirector is a controllable video world model framework for persistent dynamic object memory and unrestricted viewpoint exploration. It decouples semantic motion orchestration from visual generation: an LLM plans 3D object and camera trajectories, these plans are projected into 2D location controls, and appearance binding preserves object identity when dynamic entities leave the view and later re-enter. This enables long-horizon, causally generated world simulation with controllable object actions, camera motion, object permanence, and stable visual appearance.\n\n**Strongly recommend seeing our [demo page](https:\u002F\u002Fworlddirector.github.io\u002F).**\n\nIf you enjoyed the videos we created, please consider giving us a star 🌟.\n\n## 🚀 Open-Source Plan\n\n### ✅ Released\n\n- Full inference code\n- `WorldDirector-14B`\n\n# Setup\n\nThis codebase follows an environment setup similar to [LingBot-World](https:\u002F\u002Fgithub.com\u002Frobbyant\u002Flingbot-world).\n\n### Installation\n\nClone the repo:\n\n```shell\ngit clone https:\u002F\u002Fgithub.com\u002FpPetrichor\u002FWorldDirector\ncd WorldDirector\n```\n\nInstall dependencies:\n\n```shell\n# Ensure torch >= 2.4.0\npip install -r generate\u002Frequirements.txt\n```\n\nInstall FlashAttention:\n\n```shell\npip install flash-attn --no-build-isolation\n```\n\n### Model Download\n\n\n| Model                 | Resolution | Download Links                                                  |\n| --------------------- | ---------- | --------------------------------------------------------------- |\n| **WorldDirector-14B** | 480P       | 🤗 [HuggingFace](https:\u002F\u002Fhuggingface.co\u002Fhlwang06\u002FWorldDirector) |\n\n\nDownload models using `huggingface-cli`:\n\n```shell\npip install \"huggingface_hub[cli]\"\nhuggingface-cli download hlwang06\u002FWorldDirector  --local-dir .\u002FWorldDirector-14B\n```\n\n# Inference\n\nWorldDirector first uses an LLM to plan the motion of dynamic objects and the camera trajectory. This planning stage can be customized by users, as long as the final output follows a JSON format similar to `LLM_plan.json`, which describes all dynamic object motions and camera movements throughout the video.\n\nBelow is our inference pipeline.\n\n### 1. Plan Dynamic Objects and Camera Motion\n\nFirst, choose an input image to operate on. If the image already contains the dynamic objects you want to control, estimate their initial 3D bounding boxes before LLM planning. We provide a simple SAM- and DepthAnything-based tool for this step; please see `3D_bbox_infer` for details. If the input image does not contain the dynamic objects to be designed, this step can be skipped.\n\nNext, use an LLM to generate the JSON file that describes the full-video motion of all dynamic objects and the camera. We provide our complete prompt template in `prompt.txt`. After running the code generated by Gemini 3.1 Pro, you should obtain an output similar to `LLM_plan.json`, together with a 3D box visualization video.\n\nAt this stage, users can freely adjust the prompt, including the video duration, camera motion, and object behaviors. The LLM can also design 3D bounding-box trajectories for new dynamic objects, even if those objects do not appear in the input image. The planned video duration should be a multiple of 5 seconds, and the total frame count must satisfy 80*n+1 (for example, 81, 161, or 241 frames at 16 FPS). \n\n### 2. Prepare Latents and Conditions\n\nAfter LLM planning, run the following command to extract video latents and compute the box condition and appearance condition used for generation. Please refer to the paper for more details.\n\n```shell\ncd worldlatent_prepare\npython extract_games_hacking_from_json.py \\\n  --input_json \u003CPATH_TO_LLM_GENERATED_JSON> \\\n  --output_dir \u003COUTPUT_DIR> \\\n  --vae_path \u003CPATH_TO_Wan2.1_VAE.pth>\n```\n\nThe output directory should contain several `segment_{n}` folders, where each folder represents a 5-second video segment. WorldDirector then causally generates the full video chunk by chunk.\n\nBefore generation, configure `eval_dataset.json` by setting the text prompt for each segment. The `text` field should be a list: the first item is the global prompt, and the following items are local prompt descriptions for each dynamic object. Use the generated `text_subject_map.json` to determine the order of the local object prompts. You also need to fill in the initial `image_path` in the first entry of `eval_dataset.json` with the path to the input image. We provide three example prompt settings on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fhlwang06\u002FWorldDirector\u002Ftree\u002Fmain\u002FWorldDirector_pt_examples) for reference. If you want to directly test these examples, please update `image_path` and all file paths in the corresponding `eval_dataset.json` files to your local paths.\n\n### 3. Causal Video Generation\n\nFinally, run causal video generation:\n\n```shell\ncd ..\u002Fgenerate\ntorchrun --nproc_per_node=8 generate_channel.py \\\n  --task i2v-A14B \\\n  --size 480*832 \\\n  --ckpt_dir \u003CPATH_TO_WEIGHTS> \\\n  --eval_dataset_path \u003CPATH_TO_EVAL_DATASET_JSON> \\\n  --save_dir \u003COUTPUT_DIR> \\\n  --frame_num \u003CGENERATED_VIDEO_FRAME_NUM> \\\n  --sample_steps 50 \\\n  --sample_shift 10.0 \\\n  --sample_guide_scale 1.0 \\\n  --fps 16 \\\n  --height 480 \\\n  --width 832 \\\n  --box_attn_bias_value 3.5 \\\n  --prefix_num_total 10 \\\n  --ulysses_size 8 \\\n  --dit_fsdp \\\n  --t5_fsdp\n```\n\nHere, `\u003CPATH_TO_WEIGHTS>` is the local directory containing the downloaded WorldDirector model weights, and `\u003CPATH_TO_EVAL_DATASET_JSON>` is the path to the configured `eval_dataset.json` generated in the previous step.\n\n### Demo Results\n\nWe provide example generation results below. Each video shows the generated result on the left and the corresponding 3D plan on the right.\n\n\u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F1b04e7d3-7f77-4e13-bb15-a9b0f20f9024\" controls width=\"100%\">\u003C\u002Fvideo>\n\n\u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F1f9fe79b-0b9b-428f-8556-3f97106c75c9\" controls width=\"100%\">\u003C\u002Fvideo>\n\n\u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F376fbb76-c501-41d4-bae5-66df951a3b21\" controls width=\"100%\">\u003C\u002Fvideo>\n\nThe generated videos preserve object permanence and autonomous motion for dynamic entities, while maintaining consistent dynamic object memory.\n\nUsers are not limited to objects that appear in the input image. The LLM can also design new objects and use text prompts to control their generation, as shown below:\n\n\u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F9fa29ee5-7792-4ee6-a4b6-438a353c3a49\" controls width=\"100%\">\u003C\u002Fvideo>\n\n## 📚 Related Projects\n\n- [LingBot-World](https:\u002F\u002Fgithub.com\u002Frobbyant\u002Flingbot-world)\n- [WorldCanvas](https:\u002F\u002Fgithub.com\u002FpPetrichor\u002FWorldCanvas)\n\n## ✨ Acknowledgement\n\nWe would like to express our gratitude to the Wan Team for open-sourcing their code and models. Their contributions have been instrumental to the development of this project.\n\nWe also thank [LingBot-World](https:\u002F\u002Fgithub.com\u002Frobbyant\u002Flingbot-world) for its open-source world-model codebase and environment setup, which helped support this project.\n\n# Citation\n\nIf you find this work useful, please consider citing our paper:\n\n```bibtex\n@article{wang2026worlddirector,\n  title={WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory},\n  author={Hanlin Wang and Hao Ouyang and Qiuyu Wang and Wen Wang and Qingyan Bai and Ka Leong Cheng and Yue Yu and Yixuan Li and Yihao Meng and Zichen Liu and Yanhong Zeng and Yujun Shen and Qifeng Chen},\n  journal={arXiv preprint arXiv:2607.02517},\n  year={2026}\n}\n```\n\n# License\n\nThis project is licensed under the CC BY-NC-SA 4.0 ([Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](https:\u002F\u002Fcreativecommons.org\u002Flicenses\u002Fby-nc-sa\u002F4.0\u002F)).\n\nThe code is provided for academic research purposes only.\n\nFor any questions, please contact [hwangif@connect.ust.hk](mailto:hwangif@connect.ust.hk).\n","WorldDirector 是一个面向可控视频世界建模的开源框架，旨在构建具备持久动态记忆能力的可交互虚拟世界模拟器。其核心技术采用解耦式架构：由大语言模型（LLM）规划3D物体与相机运动轨迹，再投影为2D空间控制信号，并通过外观绑定机制保障物体离开视野后重入时的身份一致性与视觉稳定性。项目支持长时序、因果连贯的世界生成，涵盖物体动作控制、自由视角探索、对象永久性及跨帧外观保持。适用于具身智能仿真、AI代理训练环境构建、可控视频生成及三维场景理解等研究场景。",2,"2026-07-08 04:30:13","CREATED_QUERY"]