[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71094":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":32,"readmeContent":33,"aiSummary":34,"trendingCount":16,"starSnapshotCount":16,"syncStatus":35,"lastSyncTime":36,"discoverSource":37},71094,"ProPainter","sczhou\u002FProPainter","sczhou","[ICCV 2023] ProPainter: Improving Propagation and Transformer for Video Inpainting","https:\u002F\u002Fshangchenzhou.com\u002Fprojects\u002FProPainter\u002F",null,"Python",6737,798,64,72,0,5,16,59,15,88.61,"Other",false,"main",true,[27,28,29,30,31],"object-removal","video-completion","video-inpainting","video-outpainting","watermark-removal","2026-06-12 04:00:58","\u003Cdiv align=\"center\">\n\n\u003Cdiv class=\"logo\">\n   \u003Ca href=\"https:\u002F\u002Fshangchenzhou.com\u002Fprojects\u002FProPainter\u002F\">\n      \u003Cimg src=\"assets\u002Fpropainter_logo1_glow.png\" style=\"width: 180px\">\n   \u003C\u002Fa>\n\u003C\u002Fdiv>\n\n\u003Ch1>ProPainter: Improving Propagation and Transformer for Video Inpainting\u003C\u002Fh1>\n\n\u003Cdiv>\n    \u003Ca href='https:\u002F\u002Fshangchenzhou.com\u002F' target='_blank'>Shangchen Zhou\u003C\u002Fa> \n    \u003Ca href='https:\u002F\u002Fli-chongyi.github.io\u002F' target='_blank'>Chongyi Li\u003C\u002Fa> \n    \u003Ca href='https:\u002F\u002Fckkelvinchan.github.io\u002F' target='_blank'>Kelvin C.K. Chan\u003C\u002Fa> \n    \u003Ca href='https:\u002F\u002Fwww.mmlab-ntu.com\u002Fperson\u002Fccloy\u002F' target='_blank'>Chen Change Loy\u003C\u002Fa>\n\u003C\u002Fdiv>\n\u003Cdiv>\n    S-Lab, Nanyang Technological University  \n\u003C\u002Fdiv>\n\n\u003Cdiv>\n    \u003Cstrong>ICCV 2023\u003C\u002Fstrong>\n\u003C\u002Fdiv>\n\n\u003Cdiv>\n    \u003Ch4 align=\"center\">\n        \u003Ca href=\"https:\u002F\u002Fshangchenzhou.com\u002Fprojects\u002FProPainter\" target='_blank'>\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🐳-Project%20Page-blue\">\n        \u003C\u002Fa>\n        \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2309.03897\" target='_blank'>\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2309.03897-b31b1b.svg\">\n        \u003C\u002Fa>\n        \u003Ca href=\"https:\u002F\u002Fyoutu.be\u002F92EHfgCO5-Q\" target='_blank'>\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDemo%20Video-%23FF0000.svg?logo=YouTube&logoColor=white\">\n        \u003C\u002Fa>\n        \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fsczhou\u002FProPainter\" target='_blank'>\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDemo-%F0%9F%A4%97%20Hugging%20Face-blue\">\n        \u003C\u002Fa>\n        \u003Ca href=\"https:\u002F\u002Fopenxlab.org.cn\u002Fapps\u002Fdetail\u002FShangchenZhou\u002FProPainter\" target='_blank'>\n        \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDemo-%F0%9F%91%A8%E2%80%8D%F0%9F%8E%A8%20OpenXLab-blue\">\n        \u003C\u002Fa>\n        \u003Cimg src=\"https:\u002F\u002Fapi.infinitescript.com\u002Fbadgen\u002Fcount?name=sczhou\u002FProPainter\">\n    \u003C\u002Fh4>\n\u003C\u002Fdiv>\n\n⭐ If ProPainter is helpful to your projects, please help star this repo. Thanks! 🤗\n\n:open_book: For more visual results, go checkout our \u003Ca href=\"https:\u002F\u002Fshangchenzhou.com\u002Fprojects\u002FProPainter\u002F\" target=\"_blank\">project page\u003C\u002Fa>\n\n\n---\n\n\u003C\u002Fdiv>\n\n\n## Update\n- **2023.11.09**: Integrated to :man_artist: [OpenXLab](https:\u002F\u002Fopenxlab.org.cn\u002Fapps). Try out online demo! [![OpenXLab](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDemo-%F0%9F%91%A8%E2%80%8D%F0%9F%8E%A8%20OpenXLab-blue)](https:\u002F\u002Fopenxlab.org.cn\u002Fapps\u002Fdetail\u002FShangchenZhou\u002FProPainter)\n- **2023.11.09**: Integrated to :hugs: [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fspaces). Try out online demo! [![Hugging Face](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDemo-%F0%9F%A4%97%20Hugging%20Face-blue)](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fsczhou\u002FProPainter)\n- **2023.09.24**: We remove the watermark removal demos officially to prevent the misuse of our work for unethical purposes.\n- **2023.09.21**: Add features for memory-efficient inference. Check our [GPU memory](https:\u002F\u002Fgithub.com\u002Fsczhou\u002FProPainter#-memory-efficient-inference) requirements. 🚀\n- **2023.09.07**: Our code and model are publicly available. 🐳\n- **2023.09.01**: This repo is created.\n\n\n### TODO\n- [ ] Make a Colab demo.\n- [x] ~~Make a interactive Gradio demo.~~\n- [x] ~~Update features for memory-efficient inference.~~\n  \n## Results\n\n#### 👨🏻‍🎨 Object Removal\n\u003Ctable>\n\u003Ctr>\n   \u003Ctd> \n      \u003Cimg src=\"assets\u002Fobject_removal1.gif\">\n   \u003C\u002Ftd>\n   \u003Ctd> \n      \u003Cimg src=\"assets\u002Fobject_removal2.gif\">\n   \u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n#### 🎨 Video Completion\n\u003Ctable>\n\u003Ctr>\n   \u003Ctd> \n      \u003Cimg src=\"assets\u002Fvideo_completion1.gif\">\n   \u003C\u002Ftd>\n   \u003Ctd> \n      \u003Cimg src=\"assets\u002Fvideo_completion2.gif\">\n   \u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n   \u003Ctd> \n      \u003Cimg src=\"assets\u002Fvideo_completion3.gif\">\n   \u003C\u002Ftd>\n   \u003Ctd> \n      \u003Cimg src=\"assets\u002Fvideo_completion4.gif\">\n   \u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\n\n## Overview\n![overall_structure](assets\u002FProPainter_pipeline.png)\n\n\n## Dependencies and Installation\n\n1. Clone Repo\n\n   ```bash\n   git clone https:\u002F\u002Fgithub.com\u002Fsczhou\u002FProPainter.git\n   ```\n\n2. Create Conda Environment and Install Dependencies\n\n   ```bash\n   # create new anaconda env\n   conda create -n propainter python=3.8 -y\n   conda activate propainter\n\n   # install python dependencies\n   pip3 install -r requirements.txt\n   ```\n\n   - CUDA >= 9.2\n   - PyTorch >= 1.7.1\n   - Torchvision >= 0.8.2\n   - Other required packages in `requirements.txt`\n\n## Get Started\n### Prepare pretrained models\nDownload our pretrained models from [Releases V0.1.0](https:\u002F\u002Fgithub.com\u002Fsczhou\u002FProPainter\u002Freleases\u002Ftag\u002Fv0.1.0) to the `weights` folder. (All pretrained models can also be automatically downloaded during the first inference.)\n\nThe directory structure will be arranged as:\n```\nweights\n   |- ProPainter.pth\n   |- recurrent_flow_completion.pth\n   |- raft-things.pth\n   |- i3d_rgb_imagenet.pt (for evaluating VFID metric)\n   |- README.md\n```\n\n### 🏂 Quick test\nWe provide some examples in the [`inputs`](.\u002Finputs) folder. \nRun the following commands to try it out:\n```shell\n# The first example (object removal)\npython inference_propainter.py --video inputs\u002Fobject_removal\u002Fbmx-trees --mask inputs\u002Fobject_removal\u002Fbmx-trees_mask \n# The second example (video completion)\npython inference_propainter.py --video inputs\u002Fvideo_completion\u002Frunning_car.mp4 --mask inputs\u002Fvideo_completion\u002Fmask_square.png --height 240 --width 432\n```\n\nThe results will be saved in the `results` folder.\nTo test your own videos, please prepare the input `mp4 video` (or `split frames`) and `frame-wise mask(s)`.\n\nIf you want to specify the video resolution for processing or avoid running out of memory, you can set the video size of `--width` and `--height`:\n```shell\n# process a 576x320 video; set --fp16 to use fp16 (half precision) during inference.\npython inference_propainter.py --video inputs\u002Fvideo_completion\u002Frunning_car.mp4 --mask inputs\u002Fvideo_completion\u002Fmask_square.png --height 320 --width 576 --fp16\n```\n\n#### 💃🏻 Interactive Demo\n\nWe also provide an interactive demo for object removal, allowing users to select any object they wish to remove from a video. You can try the demo on [Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fsczhou\u002FProPainter) or run it [locally](https:\u002F\u002Fgithub.com\u002Fsczhou\u002FProPainter\u002Ftree\u002Fmain\u002Fweb-demos\u002Fhugging_face). \n\n\u003Cdiv align=\"center\">\n  \u003Cimg src=\".\u002Fweb-demos\u002Fhugging_face\u002Fassets\u002Fdemo.gif\" alt=\"Demo GIF\" style=\"max-width: 512px; height: auto;\">\n\u003C\u002Fdiv>\n\n*Please note that the demo's interface and usage may differ from the GIF animation above. For detailed instructions, refer to the [user guide](https:\u002F\u002Fgithub.com\u002Fsczhou\u002FProPainter\u002Fblob\u002Fmain\u002Fweb-demos\u002Fhugging_face\u002FREADME.md).*\n\n### 🚀 Memory-efficient inference\n\nVideo inpainting typically requires a significant amount of GPU memory. Here, we offer various features that facilitate memory-efficient inference, effectively avoiding the Out-Of-Memory (OOM) error. You can use the following options to reduce memory usage further:\n\n   - Reduce the number of local neighbors through decreasing the `--neighbor_length` (default 10).\n   - Reduce the number of global references by increasing the `--ref_stride` (default 10).\n   - Set the `--resize_ratio` (default 1.0) to resize the processing video.\n   - Set a smaller video size via specifying the `--width` and `--height`.\n   - Set `--fp16` to use fp16 (half precision) during inference.\n   - Reduce the frames of sub-videos `--subvideo_length` (default 80), which effectively decouples GPU memory costs and video length.\n\nBlow shows the estimated GPU memory requirements for different sub-video lengths with fp32\u002Ffp16 precision: \n\n| Resolution | 50 frames | 80 frames |\n| :---       | :----:    | :----:    |\n| 1280 x 720 | 28G \u002F 19G | OOM \u002F 25G |\n| 720 x 480  | 11G \u002F 7G  | 13G \u002F 8G  |\n| 640 x 480  | 10G \u002F 6G  | 12G \u002F 7G  |\n| 320 x 240  | 3G  \u002F 2G  | 4G  \u002F 3G  | \n\n\n## Dataset preparation\n\u003Ctable>\n\u003Cthead>\n  \u003Ctr>\n    \u003Cth>Dataset\u003C\u002Fth>\n    \u003Cth>YouTube-VOS\u003C\u002Fth>\n    \u003Cth>DAVIS\u003C\u002Fth>\n  \u003C\u002Ftr>\n\u003C\u002Fthead>\n\u003Ctbody>\n  \u003Ctr>\n    \u003Ctd>Description\u003C\u002Ftd>\n    \u003Ctd>For training (3,471) and evaluation (508)\u003C\u002Ftd>\n    \u003Ctd>For evaluation (50 in 90)\u003C\u002Ftd>\n  \u003Ctr>\n    \u003Ctd>Images\u003C\u002Ftd>\n    \u003Ctd> [\u003Ca href=\"https:\u002F\u002Fcompetitions.codalab.org\u002Fcompetitions\u002F19544#participate-get-data\">Official Link\u003C\u002Fa>] (Download train and test all frames) \u003C\u002Ftd>\n    \u003Ctd> [\u003Ca href=\"https:\u002F\u002Fdata.vision.ee.ethz.ch\u002Fcsergi\u002Fshare\u002Fdavis\u002FDAVIS-2017-trainval-480p.zip\">Official Link\u003C\u002Fa>] (2017, 480p, TrainVal) \u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>Masks\u003C\u002Ftd>\n    \u003Ctd colspan=\"2\"> [\u003Ca href=\"https:\u002F\u002Fdrive.google.com\u002Ffile\u002Fd\u002F1dFTneS_zaJAHjglxU10gYzr1-xALgHa4\u002Fview?usp=sharing\">Google Drive\u003C\u002Fa>] [\u003Ca href=\"https:\u002F\u002Fpan.baidu.com\u002Fs\u002F1JC-UKmlQfjhVtD81196cxA?pwd=87e3\">Baidu Disk\u003C\u002Fa>] (For reproducing paper results; provided in \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2309.03897\">ProPainter\u003C\u002Fa> paper) \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftbody>\n\u003C\u002Ftable>\n\nThe training and test split files are provided in `datasets\u002F\u003Cdataset_name>`. For each dataset, you should place `JPEGImages` to `datasets\u002F\u003Cdataset_name>`. Resize all video frames to size `432x240` for training. Unzip downloaded mask files to `datasets`.\n\nThe `datasets` directory structure will be arranged as: (**Note**: please check it carefully)\n```\ndatasets\n   |- davis\n      |- JPEGImages_432_240\n         |- \u003Cvideo_name>\n            |- 00000.jpg\n            |- 00001.jpg\n      |- test_masks\n         |- \u003Cvideo_name>\n            |- 00000.png\n            |- 00001.png   \n      |- train.json\n      |- test.json\n   |- youtube-vos\n      |- JPEGImages_432_240\n         |- \u003Cvideo_name>\n            |- 00000.jpg\n            |- 00001.jpg\n      |- test_masks\n         |- \u003Cvideo_name>\n            |- 00000.png\n            |- 00001.png\n      |- train.json\n      |- test.json   \n```\n\n## Training\nOur training configures are provided in [`train_flowcomp.json`](.\u002Fconfigs\u002Ftrain_flowcomp.json) (for Recurrent Flow Completion Network) and [`train_propainter.json`](.\u002Fconfigs\u002Ftrain_propainter.json) (for ProPainter).\n\nRun one of the following commands for training:\n```shell\n # For training Recurrent Flow Completion Network\n python train.py -c configs\u002Ftrain_flowcomp.json\n # For training ProPainter\n python train.py -c configs\u002Ftrain_propainter.json\n```\nYou can run the **same command** to **resume** your training.\n\nTo speed up the training process, you can precompute optical flow for the training dataset using the following command:\n```shell\n # Compute optical flow for training dataset\n python scripts\u002Fcompute_flow.py --root_path \u003Cdataset_root> --save_path \u003Csave_flow_root> --height 240 --width 432\n```\n\n## Evaluation\nRun one of the following commands for evaluation:\n```shell\n # For evaluating flow completion model\n python scripts\u002Fevaluate_flow_completion.py --dataset \u003Cdataset_name> --video_root \u003Cvideo_root> --mask_root \u003Cmask_root> --save_results\n # For evaluating ProPainter model\n python scripts\u002Fevaluate_propainter.py --dataset \u003Cdataset_name> --video_root \u003Cvideo_root> --mask_root \u003Cmask_root> --save_results\n```\n\nThe scores and results will also be saved in the `results_eval` folder.\nPlease `--save_results` for further [evaluating temporal warping error](https:\u002F\u002Fgithub.com\u002Fphoenix104104\u002Ffast_blind_video_consistency#evaluation).\n\n\n\n## Citation\n\n   If you find our repo useful for your research, please consider citing our paper:\n\n   ```bibtex\n   @inproceedings{zhou2023propainter,\n      title={{ProPainter}: Improving Propagation and Transformer for Video Inpainting},\n      author={Zhou, Shangchen and Li, Chongyi and Chan, Kelvin C.K and Loy, Chen Change},\n      booktitle={Proceedings of IEEE International Conference on Computer Vision (ICCV)},\n      year={2023}\n   }\n   ```\n\n\n## License\n\n#### Non-Commercial Use Only Declaration\nThe ProPainter is made available for use, reproduction, and distribution strictly for non-commercial purposes. The code and models are licensed under \u003Ca rel=\"license\" href=\".\u002FLICENSE\">NTU S-Lab License 1.0\u003C\u002Fa>. Redistribution and use should follow this license.\n\nFor inquiries or to obtain permission for commercial use, please consult Dr. Shangchen Zhou (shangchenzhou@gmail.com).\n\n\n## Projects that use ProPainter\n\nIf you develop or use ProPainter in your projects, feel free to let me know. Also, please include this [ProPainter](https:\u002F\u002Fgithub.com\u002Fsczhou\u002FProPainter) repo link, authorship information, and our [S-Lab license](https:\u002F\u002Fgithub.com\u002Fsczhou\u002FProPainter\u002Fblob\u002Fmain\u002FLICENSE) (with link).\n\n#### Projects\u002FApplications from the Community\n\n- Streaming ProPainter: https:\u002F\u002Fgithub.com\u002Fosmr\u002Fpropainter\n- Faster ProPainter: https:\u002F\u002Fgithub.com\u002Fhalfzm\u002Ffaster-propainter\n- ProPainter WebUI: https:\u002F\u002Fgithub.com\u002Fhalfzm\u002FProPainter-Webui\n- ProPainter ComfyUI: https:\u002F\u002Fgithub.com\u002Fdaniabib\u002FComfyUI_ProPainter_Nodes\n- Cutie (video segmentation): https:\u002F\u002Fgithub.com\u002Fhkchengrex\u002FCutie\n- Cinetransfer (character transfer): https:\u002F\u002Fvirtualfilmstudio.github.io\u002Fprojects\u002Fcinetransfer\n- Motionshop (character transfer): https:\u002F\u002Faigc3d.github.io\u002Fmotionshop\n\n\n\n#### PyPI\n- propainter: https:\u002F\u002Fpypi.org\u002Fproject\u002Fpropainter\n- pytorchcv: https:\u002F\u002Fpypi.org\u002Fproject\u002Fpytorchcv\n\n## Contact\nIf you have any questions, please feel free to reach me out at shangchenzhou@gmail.com. \n\n## Acknowledgement\n\nThis code is based on [E\u003Csup>2\u003C\u002Fsup>FGVI](https:\u002F\u002Fgithub.com\u002FMCG-NKU\u002FE2FGVI) and [STTN](https:\u002F\u002Fgithub.com\u002Fresearchmm\u002FSTTN). Some code are brought from [BasicVSR++](https:\u002F\u002Fgithub.com\u002Fckkelvinchan\u002FBasicVSR_PlusPlus). Thanks for their awesome works. \n\nSpecial thanks to [Yihang Luo](https:\u002F\u002Fgithub.com\u002FLuo-Yihang) for his valuable contributions to build and maintain the Gradio demos for ProPainter.\n","ProPainter 是一个用于视频修复的工具，旨在通过改进传播机制和Transformer来提高视频修复的质量。该项目利用Python语言开发，核心功能包括对象移除、视频补全及扩展等，并且特别强调了在保持高效内存使用的同时实现高质量的视频修复效果。ProPainter适合于需要进行视频内容编辑或修复的场景，如电影后期制作、视频广告制作以及个人视频编辑等领域。此外，该项目还提供了在线演示版本，方便用户直接体验其功能。",2,"2026-06-11 03:35:51","high_star"]