[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72064":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":23,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":16,"starSnapshotCount":16,"syncStatus":29,"lastSyncTime":30,"discoverSource":31},72064,"ToonCrafter","Doubiiu\u002FToonCrafter","Doubiiu","[SIGGRAPH Asia 2024, Journal Track] ToonCrafter: Generative Cartoon Interpolation","https:\u002F\u002Fdoubiiu.github.io\u002Fprojects\u002FToonCrafter\u002F",null,"Python",5967,530,57,55,0,3,4,7,9,39.18,"Apache License 2.0",false,"main",[],"2026-06-12 02:02:58","## ___***ToonCrafter: Generative Cartoon Interpolation***___\n\u003C!-- ![](.\u002Fassets\u002Flogo_long.png#gh-light-mode-only){: width=\"50%\"} -->\n\u003C!-- ![](.\u002Fassets\u002Flogo_long_dark.png#gh-dark-mode-only=100x20) -->\n\u003Cdiv align=\"center\">\n\u003Cimg src='assets\u002Flogo\u002Flogo2.png' style=\"height:100px\">\u003C\u002Fimg>\n\n \u003Ca href='https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.17933'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2405.17933-b31b1b.svg'>\u003C\u002Fa>  \n \u003Ca href='https:\u002F\u002Fdoubiiu.github.io\u002Fprojects\u002FToonCrafter\u002F'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject-Page-Green'>\u003C\u002Fa>  \n\u003Ca href='https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=u3F35do93_8'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FYoutube-Video-b31b1b.svg'>\u003C\u002Fa>\u003Cbr>\n\u003Ca href='https:\u002F\u002Freplicate.com\u002Ffofr\u002Ftooncrafter'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Freplicate-Demo-blue'>\u003C\u002Fa>  \n\u003Ca href='https:\u002F\u002Fgithub.com\u002Fcamenduru\u002FToonCrafter-jupyter'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FColab-Demo-Green'>\u003C\u002Fa> \n\u003Ca href='https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FDoubiiu\u002Ftooncrafter'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Hugging%20Face%20ToonCrafter-Demo-blue'>\u003C\u002Fa>\n\n\n_**[Jinbo Xing](https:\u002F\u002Fdoubiiu.github.io\u002F), [Hanyuan Liu](https:\u002F\u002Fgithub.com\u002Fhyliu), [Menghan Xia](https:\u002F\u002Fmenghanxia.github.io), [Yong Zhang](https:\u002F\u002Fyzhang2016.github.io), [Xintao Wang](https:\u002F\u002Fxinntao.github.io\u002F), [Ying Shan](https:\u002F\u002Fscholar.google.com\u002Fcitations?hl=en&user=4oXBp9UAAAAJ&view_op=list_works&sortby=pubdate), [Tien-Tsin Wong](https:\u002F\u002Fttwong12.github.io\u002Fmyself.html)**_\n\u003Cbr>\u003Cbr>\nFrom CUHK and Tencent AI Lab.\n\n\u003Cstrong>at SIGGRAPH Asia 2024, Journal Track\u003C\u002Fstrong>\n\n\n\u003C\u002Fdiv>\n \n## 🔆 Introduction\n\n⚠️ We have not set up any official profit-making projects or web applications. Please be cautious!!!\n\n🤗 ToonCrafter can interpolate two cartoon images by leveraging the pre-trained image-to-video diffusion priors. Please check our project page and paper for more information. \u003Cbr>\n\n\n\n\n\n\n\n### 1.1 Showcases (512x320)\n\u003Ctable class=\"center\">\n    \u003Ctr style=\"font-weight: bolder;text-align:center;\">\n        \u003Ctd>Input starting frame\u003C\u002Ftd>\n        \u003Ctd>Input ending frame\u003C\u002Ftd>\n        \u003Ctd>Generated video\u003C\u002Ftd>\n    \u003C\u002Ftr>\n  \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F72109_125.mp4_00-00.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F72109_125.mp4_00-01.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F00.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n\n   \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002FJapan_v2_2_062266_s2_frame1.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002FJapan_v2_2_062266_s2_frame3.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F03.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002FJapan_v2_1_070321_s3_frame1.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002FJapan_v2_1_070321_s3_frame3.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F02.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr> \n  \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F74302_1349_frame1.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F74302_1349_frame3.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F01.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n### 1.2 Sparse sketch guidance\n\u003Ctable class=\"center\">\n    \u003Ctr style=\"font-weight: bolder;text-align:center;\">\n        \u003Ctd>Input starting frame\u003C\u002Ftd>\n        \u003Ctd>Input ending frame\u003C\u002Ftd>\n        \u003Ctd>Input sketch guidance\u003C\u002Ftd>\n        \u003Ctd>Generated video\u003C\u002Ftd>\n    \u003C\u002Ftr>\n  \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F72105_388.mp4_00-00.png width=\"200\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F72105_388.mp4_00-01.png width=\"200\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F06.gif width=\"200\">\n  \u003C\u002Ftd>\n   \u003Ctd>\n    \u003Cimg src=assets\u002F07.gif width=\"200\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n  \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F72110_255.mp4_00-00.png width=\"200\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F72110_255.mp4_00-01.png width=\"200\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F12.gif width=\"200\">\n  \u003C\u002Ftd>\n   \u003Ctd>\n    \u003Cimg src=assets\u002F13.gif width=\"200\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n\n\u003C\u002Ftable>\n\n\n### 2. Applications\n#### 2.1 Cartoon Sketch Interpolation (see project page for more details)\n\u003Ctable class=\"center\">\n    \u003Ctr style=\"font-weight: bolder;text-align:center;\">\n        \u003Ctd>Input starting frame\u003C\u002Ftd>\n        \u003Ctd>Input ending frame\u003C\u002Ftd>\n        \u003Ctd>Generated video\u003C\u002Ftd>\n    \u003C\u002Ftr>\n\n  \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002Fframe0001_10.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002Fframe0016_10.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F10.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n\n   \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002Fframe0001_11.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002Fframe0016_11.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F11.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n\u003C\u002Ftable>\n\n\n#### 2.2 Reference-based Sketch Colorization\n\u003Ctable class=\"center\">\n    \u003Ctr style=\"font-weight: bolder;text-align:center;\">\n        \u003Ctd>Input sketch\u003C\u002Ftd>\n        \u003Ctd>Input reference\u003C\u002Ftd>\n        \u003Ctd>Colorization results\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \n  \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F04.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002Fframe0001_05.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F05.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n\n   \u003Ctr>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F08.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002Fframe0001_09.png width=\"250\">\n  \u003C\u002Ftd>\n  \u003Ctd>\n    \u003Cimg src=assets\u002F09.gif width=\"250\">\n  \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n\u003C\u002Ftable>\n\n\n\n\n\n\n\n## 📝 Changelog\n- [ ] Add sketch control and colorization function.\n- __[2024.05.29]__: 🔥🔥 Release code and model weights.\n- __[2024.05.28]__: Launch the project page and update the arXiv preprint.\n\u003Cbr>\n\n\n## 🧰 Models\n\n|Model|Resolution|GPU Mem. & Inference Time (A100, ddim 50steps)|Checkpoint|\n|:---------|:---------|:--------|:--------|\n|ToonCrafter_512|320x512| ~24G & 24s (`perframe_ae=True`)|[Hugging Face](https:\u002F\u002Fhuggingface.co\u002FDoubiiu\u002FToonCrafter\u002Fblob\u002Fmain\u002Fmodel.ckpt)|\n\nWe get the feedback from issues that the model may consume about 24G~27G GPU memory in this implementation, but the community has lowered the consumption to ~10GB.\n\nCurrently, our ToonCrafter can support generating videos of up to 16 frames with a resolution of 512x320. The inference time can be reduced by using fewer DDIM steps.\n\n\n\n## ⚙️ Setup\n\n### Install Environment via Anaconda (Recommended)\n```bash\nconda create -n tooncrafter python=3.8.5\nconda activate tooncrafter\npip install -r requirements.txt\n```\n\n\n## 💫 Inference\n### 1. Command line\n\nDownload pretrained ToonCrafter_512 and put the `model.ckpt` in `checkpoints\u002Ftooncrafter_512_interp_v1\u002Fmodel.ckpt`.\n```bash\n  sh scripts\u002Frun.sh\n```\n\n\n### 2. Local Gradio demo\n\nDownload the pretrained model and put it in the corresponding directory according to the previous guidelines.\n```bash\n  python gradio_app.py \n```\n\n\n\n\n\n\n## 🤝 Community Support\n1. ComfyUI and pruned models (fp16): [ComfyUI-DynamiCrafterWrapper](https:\u002F\u002Fgithub.com\u002Fkijai\u002FComfyUI-DynamiCrafterWrapper) (Thanks to [kijai](https:\u002F\u002Ftwitter.com\u002Fkijaidesign))\n\n|Model|Resolution|GPU Mem. |Checkpoint|\n|:---------|:---------|:--------|:--------|\n|ToonCrafter|512x320|12GB |[Hugging Face](https:\u002F\u002Fhuggingface.co\u002FKijai\u002FDynamiCrafter_pruned\u002Fblob\u002Fmain\u002Ftooncrafter_512_interp-fp16.safetensors)|\n\n2. ComfyUI. [ComfyUI-ToonCrafter](https:\u002F\u002Fgithub.com\u002FAIGODLIKE\u002FComfyUI-ToonCrafter) (Thanks to [Yorha4D](https:\u002F\u002Fgithub.com\u002FYorha4D))\n\n3. Colab. [Code](https:\u002F\u002Fgithub.com\u002Fcamenduru\u002FToonCrafter-jupyter) (Thanks to [camenduru](https:\u002F\u002Fgithub.com\u002Fcamenduru)), [Code](https:\u002F\u002Fgist.github.com\u002F0smboy\u002Fbaef995b8f5974f19ac114ec20ac37d5) (Thanks to [0smboy](https:\u002F\u002Fgithub.com\u002F0smboy))\n\n4. Windows platform support: [ToonCrafter-for-windows](https:\u002F\u002Fgithub.com\u002Fsdbds\u002FToonCrafter-for-windows) (Thanks to [sdbds](https:\u002F\u002Fgithub.com\u002Fsdbds))\n\n5. Sketch-guidance implementation: [ToonCrafter_with_SketchGuidance](https:\u002F\u002Fgithub.com\u002Fmattyamonaca\u002FToonCrafter_with_SketchGuidance) (Thanks to [mattyamonaca](https:\u002F\u002Fgithub.com\u002Fmattyamonaca))\n\n## 😉 Citation\nPlease consider citing our paper if our code is useful:\n```bib\n@article{xing2024tooncrafter,\n  title={Tooncrafter: Generative cartoon interpolation},\n  author={Xing, Jinbo and Liu, Hanyuan and Xia, Menghan and Zhang, Yong and Wang, Xintao and Shan, Ying and Wong, Tien-Tsin},\n  journal={ACM Transactions on Graphics (TOG)},\n  volume={43},\n  number={6},\n  pages={1--11},\n  year={2024}\n}\n```\n\n\n## 🙏 Acknowledgements\nWe would like to thank [Xiaoyu](https:\u002F\u002Fengineering.purdue.edu\u002Fpeople\u002Fxiaoyu.xiang.1) for providing the [sketch extractor](https:\u002F\u002Fgithub.com\u002FMukosame\u002FAnime2Sketch), and [supraxylon](https:\u002F\u002Fgithub.com\u002Fsupraxylon) for the Windows batch script.\n\n\u003Ca name=\"disc\">\u003C\u002Fa>\n## 📢 Disclaimer\nCalm down. Our framework opens up the era of generative cartoon interpolation, but due to the variaity of generative video prior, the success rate is not guaranteed.\n\n⚠️This is an open-source research exploration, instead of commercial products. It can't meet all your expectations.\n\nThis project strives to impact the domain of AI-driven video generation positively. Users are granted the freedom to create videos using this tool, but they are expected to comply with local laws and utilize it responsibly. The developers do not assume any responsibility for potential misuse by users.\n****\n","ToonCrafter 是一个用于生成卡通插值视频的工具，能够基于两张输入的卡通图像生成平滑过渡的动画。项目利用预训练的图像到视频扩散先验技术实现这一功能，支持用户自定义起始和结束帧，进而生成高质量的中间帧序列。该技术特别适用于需要快速制作卡通动画但又缺乏专业动画制作技能的场景，如教育、娱乐内容创作等领域。此外，ToonCrafter 提供了多种在线演示平台（如Colab、Hugging Face等），方便用户直接体验其效果。",2,"2026-06-11 03:40:13","high_star"]