[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72378":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},72378,"MultiTalk","MeiGen-AI\u002FMultiTalk","MeiGen-AI","[NeurIPS 2025] Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation","https:\u002F\u002Fmeigen-ai.github.io\u002Fmulti-talk\u002F",null,"Python",2945,488,66,150,0,5,7,24,15,30.07,"Apache License 2.0",false,"main",true,[],"2026-06-12 02:03:02","\u003Cdiv align=\"center\">\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Flogo2.jpeg\" alt=\"MultiTalk\" width=\"240\"\u002F>\n\u003C\u002Fp>\n\n\u003Ch1>Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation (NeurIPS 2025)\u003C\u002Fh1>\n\n\n[Zhe Kong*](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=4X3yLwsAAAAJ&hl=zh-CN) · [Feng Gao*](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=lFkCeoYAAAAJ) ·[Yong Zhang](https:\u002F\u002Fyzhang2016.github.io\u002F)\u003Csup>✉\u003C\u002Fsup> · [Zhuoliang Kang](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=W1ZXjMkAAAAJ&hl=en) · [Xiaoming Wei](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=JXV5yrZxj5MC&hl=zh-CN) · [Xunliang Cai](https:\u002F\u002Fopenreview.net\u002Fprofile?id=~Xunliang_Cai1)  \n\n[Guanying Chen](https:\u002F\u002Fguanyingc.github.io\u002F) · [Wenhan Luo](https:\u002F\u002Fwhluo.github.io\u002F)\u003Csup>✉\u003C\u002Fsup>\n\n\u003Csup>*\u003C\u002Fsup>Equal Contribution\n\u003Csup>✉\u003C\u002Fsup>Corresponding Authors\n\n\n\u003Ca href='https:\u002F\u002Fmeigen-ai.github.io\u002Fmulti-talk\u002F'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject-Page-green'>\u003C\u002Fa>\n\u003Ca href='https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.22647'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTechnique-Report-red'>\u003C\u002Fa>\n\u003Ca href='https:\u002F\u002Fhuggingface.co\u002FMeiGen-AI\u002FMeiGen-MultiTalk'>\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Hugging%20Face-Model-blue'>\u003C\u002Fa>\n\u003C\u002Fdiv>\n\n> **TL; DR:**  MultiTalk is an audio-driven multi-person conversational video generation​​. It enables the video creation of multi-person conversation 💬, singing  🎤,  interaction control 👬, and cartoon 🙊.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fpipe.png\">\n\u003C\u002Fp>\n\n## Video Demos\n\n\u003Ctable border=\"0\" style=\"width: 100%; text-align: left; margin-top: 20px;\">\n  \u003Ctr>\n      \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fe55952e6-e1b2-44a5-9887-a89307a378da\" width=\"320\" controls loop>\u003C\u002Fvideo>\n      \u003C\u002Ftd>\n      \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Ff0396c19-d459-42aa-9d78-34fdea10de18\" width=\"320\" controls loop>\u003C\u002Fvideo>\n      \u003C\u002Ftd>\n       \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F3576fd04-3e5f-4933-ac7b-1c4e6a601379\" width=\"320\" controls loop>\u003C\u002Fvideo>\n     \u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n      \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F5589056e-3202-442d-a62a-2cad7a7ecb19\" width=\"320\" controls loop>\u003C\u002Fvideo>\n      \u003C\u002Ftd>\n      \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F554bfbe7-0090-492c-94be-329f5e39e175\" width=\"320\" controls loop>\u003C\u002Fvideo>\n      \u003C\u002Ftd>\n       \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F9e961f35-9413-4846-a806-8186d54061da\" width=\"320\" controls loop>\u003C\u002Fvideo>\n     \u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n      \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F342595ab-cf75-4872-8182-f20fe8c95611\" width=\"320\" controls loop>\u003C\u002Fvideo>\n      \u003C\u002Ftd>\n      \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F6476f9f0-35e0-4484-91a4-8aa646aa994a\" width=\"320\" controls loop>\u003C\u002Fvideo>\n      \u003C\u002Ftd>\n       \u003Ctd>\n          \u003Cvideo src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fd8fc8e94-0cba-4c25-9f3a-a8d7e0a785e1\" width=\"320\" controls loop>\u003C\u002Fvideo>\n     \u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\n\n\n\n## ✨ Key Features\n\nWe propose **MultiTalk** , a novel framework for audio-driven multi-person conversational video generation. Given a multi-stream audio input, a reference image and a prompt, MultiTalk generates a video containing interactions following the prompt, with consistent lip motions aligned with the audio.\n\n> - 💬 **​​Realistic Conversations**​​ - Support single & multi-person generation\n> - 👥 ​​**​​Interactive Character Control**​​​​ - Direct virtual humans via prompts\n> - 🎤 ​​**​​Generalization Performances**​​​​ - Support the generation of cartoon character and singing \n> - 📺 **​​​​Resolution Flexibility​​**​​: 480p & 720p output at arbitrary aspect ratios\n> - ⏱️ **Long Video Generation**: Support video generation up to 15 seconds\n\n## 🔥 Latest News\n* Dec 16, 2025: 🚀 We are excited to announce the release of **[LongCat-Video-Avatar](https:\u002F\u002Fgithub.com\u002FMeiGen-AI\u002FLongCat-Video-Avatar)**, a unified model that delivers expressive and highly dynamic audio-driven character animation, supporting native tasks including Audio-Text-to-Video, Audio-Text-Image-to-Video, and Video Continuation with seamless compatibility for both single-stream and multi-stream audio inputs. The release includes our Technical Report, [code](https:\u002F\u002Fgithub.com\u002Fmeituan-longcat\u002FLongCat-Video), [model weights](https:\u002F\u002Fhuggingface.co\u002Fmeituan-longcat\u002FLongCat-Video-Avatar), and [project page](https:\u002F\u002Fmeigen-ai.github.io\u002FLongCat-Video-Avatar\u002F).\n* Aug 19, 2025:  🔥🔥 We released [**InfiniteTalk**](https:\u002F\u002Fgithub.com\u002FMeiGen-AI\u002FInfiniteTalk), a novel new paradigm for video dubbing.  InfiniteTalk supports infinite-length video-to-video generation and image-to-video generation. Models, code, gradio, and comfyui have all been released. \n* July 11, 2025: 🔥🔥 `MultiTalk` supports INT8 [quantization](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Foptimum-quanto) and [SageAttention2.2](https:\u002F\u002Fgithub.com\u002Fthu-ml\u002FSageAttention), and updates the CFG strategy (2 NFE per step) for FusionX LoRA, \n* July 01, 2025: 🔥🔥 `MultiTalk` supports input audios with TTS, [FusioniX](https:\u002F\u002Fhuggingface.co\u002Fvrgamedevgirl84\u002FWan14BT2VFusioniX\u002Fblob\u002Fmain\u002FFusionX_LoRa\u002FWan2.1_I2V_14B_FusionX_LoRA.safetensors) and [lightx2v](https:\u002F\u002Fhuggingface.co\u002FKijai\u002FWanVideo_comfy\u002Fblob\u002Fmain\u002FWan21_T2V_14B_lightx2v_cfg_step_distill_lora_rank32.safetensors) LoRA acceleration (requires only 4~8 steps), and Gradio.\n* June 14, 2025: 🔥🔥 We release `MultiTalk` with support for `multi-GPU inference`, `teacache acceleration`, `APG` and `low-VRAM inference` (enabling 480P video generation on a single RTX 4090). [APG](https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.02416) is used to alleviate the color error accumulation in long video generation. TeaCache is capable of increasing speed by approximately 2~3x.\n* June 9, 2025: 🔥🔥 We release the [weights](https:\u002F\u002Fhuggingface.co\u002FMeiGen-AI\u002FMeiGen-MultiTalk) and inference code of **MultiTalk** \n* May 29, 2025: We release the [Technique-Report](https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.22647) of **MultiTalk** \n* May 29, 2025: We release the [project page](https:\u002F\u002Fmeigen-ai.github.io\u002Fmulti-talk\u002F) of **MultiTalk** \n\n## 🌐 Community  Works\n- [Wan2GP](https:\u002F\u002Fgithub.com\u002Fdeepbeepmeep\u002FWan2GP): thank [deepbeepmeep](https:\u002F\u002Fgithub.com\u002Fdeepbeepmeep) for providing the project [Wan2GP](https:\u002F\u002Fgithub.com\u002Fdeepbeepmeep\u002FWan2GP) that enables Multitalk on very low VRAM hardware (8 GB of VRAM) and combines it with the capabilities of Vace. \n- [Replicate](https:\u002F\u002Freplicate.com\u002Fzsxkib\u002Fmultitalk): thank [zsxkib](https:\u002F\u002Fgithub.com\u002Fzsxkib) for pushing MultiTalk to Replicate platform, try it! Please refer to [cog-MultiTalk](https:\u002F\u002Fgithub.com\u002Fzsxkib\u002Fcog-MultiTalk) for details.\n- [Gradio Demo](https:\u002F\u002Fgithub.com\u002FMeiGen-AI\u002FMultiTalk): thank [fffiloni](https:\u002F\u002Fgithub.com\u002Ffffiloni) for developing this gradio demo on Hugging Face. Please refer to the [issue](https:\u002F\u002Fgithub.com\u002FMeiGen-AI\u002FMultiTalk\u002Fissues\u002F39) for details. \n- [ComfyUI](https:\u002F\u002Fgithub.com\u002Fkijai\u002FComfyUI-WanVideoWrapper\u002Ftree\u002Fmultitalk): thank [kijai](https:\u002F\u002Fgithub.com\u002Fkijai) for integrating MultiTalk into ComfyUI-WanVideoWrapper. [Rudra](https:\u002F\u002Fgithub.com\u002FRudra-ai-coder) found something interesting that MultiTalk can be combined with Wanx T2V and VACE in the [issue](https:\u002F\u002Fgithub.com\u002Fkijai\u002FComfyUI-WanVideoWrapper\u002Fissues\u002F635). \n- [Google Colab example](https:\u002F\u002Fcolab.research.google.com\u002Fdrive\u002F185OyRIpJDlpnRjhBRb7FnaRlq11BLZTa?usp=sharing), an exmaple for inference on A100 provided by [Braffolk](https:\u002F\u002Fgithub.com\u002FBraffolk).\n\n## 📑 Todo List\n\n- [x] Release the technical report\n- [x] Inference\n- [x] Checkpoints\n- [x] Multi-GPU Inference\n- [ ] Inference acceleration\n  - [x] TeaCache\n  - [x] int8 quantization\n  - [ ] LCM distillation\n  - [ ] Sparse Attention\n- [x] Run with very low VRAM\n- [x] TTS integration\n- [x] Gradio demo\n- [ ] ComfyUI\n- [ ] 1.3B model\n\n## Quick Start\n\n### 🛠️Installation\n\n#### 1. Create a conda environment and install pytorch, xformers\n```\nconda create -n multitalk python=3.10\nconda activate multitalk\npip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fcu121\npip install -U xformers==0.0.28 --index-url https:\u002F\u002Fdownload.pytorch.org\u002Fwhl\u002Fcu121\n```\n#### 2. Flash-attn installation:\n```\npip install misaki[en]\npip install ninja \npip install psutil \npip install packaging \npip install flash_attn==2.7.4.post1\n```\n\n#### 3. Other dependencies\n```\npip install -r requirements.txt\nconda install -c conda-forge librosa\n```\n\n#### 4. FFmeg installation\n```\nconda install -c conda-forge ffmpeg\n```\nor\n```\nsudo yum install ffmpeg ffmpeg-devel\n```\n\n### 🧱Model Preparation\n\n#### 1. Model Download\n\n| Models        |                       Download Link                                           |    Notes                      |\n| --------------|-------------------------------------------------------------------------------|-------------------------------|\n| Wan2.1-I2V-14B-480P  |      🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002FWan-AI\u002FWan2.1-I2V-14B-480P)       | Base model\n| chinese-wav2vec2-base |      🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002FTencentGameMate\u002Fchinese-wav2vec2-base)          | Audio encoder\n| Kokoro-82M      |      🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002Fhexgrad\u002FKokoro-82M)              | TTS weights\n| MeiGen-MultiTalk      |      🤗 [Huggingface](https:\u002F\u002Fhuggingface.co\u002FMeiGen-AI\u002FMeiGen-MultiTalk)              | Our audio condition weights\n\nDownload models using huggingface-cli:\n``` sh\nhuggingface-cli download Wan-AI\u002FWan2.1-I2V-14B-480P --local-dir .\u002Fweights\u002FWan2.1-I2V-14B-480P\nhuggingface-cli download TencentGameMate\u002Fchinese-wav2vec2-base --local-dir .\u002Fweights\u002Fchinese-wav2vec2-base\nhuggingface-cli download TencentGameMate\u002Fchinese-wav2vec2-base model.safetensors --revision refs\u002Fpr\u002F1 --local-dir .\u002Fweights\u002Fchinese-wav2vec2-base\nhuggingface-cli download hexgrad\u002FKokoro-82M --local-dir .\u002Fweights\u002FKokoro-82M\nhuggingface-cli download MeiGen-AI\u002FMeiGen-MultiTalk --local-dir .\u002Fweights\u002FMeiGen-MultiTalk\n```\n\n#### 2. Link or Copy MultiTalk Model to Wan2.1-I2V-14B-480P Directory\n\nLink through:\n```\nmv weights\u002FWan2.1-I2V-14B-480P\u002Fdiffusion_pytorch_model.safetensors.index.json weights\u002FWan2.1-I2V-14B-480P\u002Fdiffusion_pytorch_model.safetensors.index.json_old\nsudo ln -s {Absolute path}\u002Fweights\u002FMeiGen-MultiTalk\u002Fdiffusion_pytorch_model.safetensors.index.json weights\u002FWan2.1-I2V-14B-480P\u002F\nsudo ln -s {Absolute path}\u002Fweights\u002FMeiGen-MultiTalk\u002Fmultitalk.safetensors weights\u002FWan2.1-I2V-14B-480P\u002F\n```\n\nOr, copy through:\n```\nmv weights\u002FWan2.1-I2V-14B-480P\u002Fdiffusion_pytorch_model.safetensors.index.json weights\u002FWan2.1-I2V-14B-480P\u002Fdiffusion_pytorch_model.safetensors.index.json_old\ncp weights\u002FMeiGen-MultiTalk\u002Fdiffusion_pytorch_model.safetensors.index.json weights\u002FWan2.1-I2V-14B-480P\u002F\ncp weights\u002FMeiGen-MultiTalk\u002Fmultitalk.safetensors weights\u002FWan2.1-I2V-14B-480P\u002F\n```\n### 🔑 Quick Inference\n\nOur model is compatible with both 480P and 720P resolutions. The current code only supports 480P inference. 720P inference requires multiple GPUs, and we will provide an update soon.\n> Some tips\n> - Lip synchronization accuracy:​​ Audio CFG works optimally between 3–5. Increase the audio CFG value for better synchronization.\n> - ​​Video clip length:​​ The model was trained on 81-frame videos at 25 FPS. For optimal prompt following performance, generate clips at 81 frames. Generating up to 201 frames is possible, though longer clips might reduce prompt-following performance.\n> - ​​Long video generation:​​ Audio CFG influences color tone consistency across segments. Set this value to 3 to alleviate tonal variations.\n> - Sampling steps: If you want to generate a video fast, you can decrease the sampling steps to even 10 that will not hurt the lip synchronization accuracy, but affects the motion and visual quality. More sampling steps, better video quality.\n> - TeaCache accelerate:​​ The optimal range for `--teacache_thresh` is between 0.2 and 0.5. Increasing this value can further improve acceleration, but may also lead to a decline in the quality of the generated video.\n\n#### Usage of MultiTalk\n```\n--mode streaming: long video generation.\n--mode clip: generate short video with one chunk. \n--use_teacache: run with TeaCache.\n--size multitalk-480: generate 480P video.\n--size multitalk-720: generate 720P video.\n--use_apg: run with APG.\n--teacache_thresh: A coefficient used for TeaCache acceleration\n—-sample_text_guide_scale： When not using LoRA, the optimal value is 5. After applying LoRA, the recommended value is 1.\n—-sample_audio_guide_scale： When not using LoRA, the optimal value is 4. After applying LoRA, the recommended value is 2.\n```\n\n#### 1. Single-Person\n\n##### 1) Run with single GPU\n\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fsingle_example_1.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --use_teacache \\\n    --save_file single_long_exp\n```\n\n##### 2) Run with very low VRAM\n\nIf you want run with very low VRAM, set `--num_persistent_param_in_dit 0`:\n\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fsingle_example_1.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --num_persistent_param_in_dit 0 \\\n    --use_teacache \\\n    --save_file single_long_lowvram_exp\n```\n\n##### 3) Multi-GPU inference\n\n```\nGPU_NUM=8\ntorchrun --nproc_per_node=$GPU_NUM --standalone generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --dit_fsdp --t5_fsdp \\\n    --ulysses_size=$GPU_NUM \\\n    --input_json examples\u002Fsingle_example_1.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --use_teacache \\\n    --save_file single_long_multigpu_exp\n```\n\n##### 4) Run with TTS\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fsingle_example_tts_1.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --num_persistent_param_in_dit 0 \\\n    --use_teacache \\\n    --save_file single_long_lowvram_tts_exp \\\n    --audio_mode tts\n```\n\n\n#### 2. Multi-Person\n\n##### 1) Run with single GPU\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fmultitalk_example_2.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --use_teacache \\\n    --save_file multi_long_exp\n```\n##### 2) Run with very low VRAM\n\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fmultitalk_example_2.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --num_persistent_param_in_dit 0 \\\n    --use_teacache \\\n    --save_file multi_long_lowvram_exp\n```\n\n##### 3) Multi-GPU inference\n\n```\nGPU_NUM=8\ntorchrun --nproc_per_node=$GPU_NUM --standalone generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --dit_fsdp --t5_fsdp --ulysses_size=$GPU_NUM \\\n    --input_json examples\u002Fmultitalk_example_2.json \\\n    --sample_steps 40 \\\n    --mode streaming --use_teacache \\\n    --save_file multi_long_multigpu_exp\n```\n\n##### 4) Run with TTS\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fmultitalk_example_tts_1.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --num_persistent_param_in_dit 0 \\\n    --use_teacache \\\n    --save_file multi_long_lowvram_tts_exp \\\n    --audio_mode tts\n```\n\n\n#### 3. Run with FusioniX and CausVid(Require only 4~8 steps)\n\n[FusioniX](https:\u002F\u002Fhuggingface.co\u002Fvrgamedevgirl84\u002FWan14BT2VFusioniX\u002Fblob\u002Fmain\u002FFusionX_LoRa\u002FWan2.1_I2V_14B_FusionX_LoRA.safetensors) require 8 steps and [lightx2v](https:\u002F\u002Fhuggingface.co\u002FKijai\u002FWanVideo_comfy\u002Fblob\u002Fmain\u002FWan21_T2V_14B_lightx2v_cfg_step_distill_lora_rank32.safetensors) requires only 4 steps.\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fsingle_example_1.json \\\n    --lora_dir weights\u002FWan2.1_I2V_14B_FusionX_LoRA.safetensors \\\n    --lora_scale 1.0 \\\n    --sample_text_guide_scale 1.0 \\\n    --sample_audio_guide_scale 2.0 \\\n    --sample_steps 8 \\\n    --mode streaming \\\n    --num_persistent_param_in_dit 0 \\\n    --save_file single_long_lowvram_fusionx_exp \\\n    --sample_shift 2\n```\n\nor \n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fmultitalk_example_2.json \\\n    --lora_dir weights\u002FWan2.1_I2V_14B_FusionX_LoRA.safetensors \\\n    --lora_scale 1.0 \\\n    --sample_text_guide_scale 1.0 \\\n    --sample_audio_guide_scale 2.0 \\\n    --sample_steps 8 \\\n    --mode streaming \\\n    --num_persistent_param_in_dit 0 \\\n    --save_file multi_long_lowvram_fusionx_exp \\\n\n```\n\n#### 4. Run with the quantization model (Only support run with single gpu)\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fmultitalk_example_2.json \\\n    --sample_steps 40 \\\n    --mode streaming \\\n    --use_teacache \\\n    --quant int8 \\\n    --quant_dir weights\u002FMeiGen-MultiTalk \\\n    --num_persistent_param_in_dit 0 \\\n    --save_file multi_long_lowvram_exp_quant\n```\n\nRun with lora:\n\n```\npython generate_multitalk.py \\\n    --ckpt_dir weights\u002FWan2.1-I2V-14B-480P \\\n    --wav2vec_dir 'weights\u002Fchinese-wav2vec2-base' \\\n    --input_json examples\u002Fmultitalk_example_1.json \\\n    --quant int8 \\\n    --quant_dir weights\u002FMeiGen-MultiTalk \\\n    --lora_dir weights\u002FMeiGen-MultiTalk\u002Fquant_models\u002Fquant_model_int8_FusionX.safetensors \\\n    --sample_text_guide_scale 1.0 \\\n    --sample_audio_guide_scale 2.0 \\\n    --sample_steps 8 \\\n    --mode streaming \\\n    --num_persistent_param_in_dit 0 \\\n    --save_file multi_long_lowvram_fusionx_exp_quant \\\n    --sample_shift 2\n```\n\n#### 5. Run with Gradio\n\n\n\n```\npython app.py \\\n    --lora_dir weights\u002FWan2.1_I2V_14B_FusionX_LoRA.safetensors \\\n    --lora_scale 1.0 \\\n    --num_persistent_param_in_dit 0 \\\n    --sample_shift 2\n```\n\nor\n\n```\npython app.py --num_persistent_param_in_dit 0 \n```\n\nor \n\n```\npython app.py \\\n    --quant int8 \\\n    --quant_dir weights\u002FMeiGen-MultiTalk \\\n    --lora_dir weights\u002FMeiGen-MultiTalk\u002Fquant_models\u002Fquant_model_int8_FusionX.safetensors \\\n    --sample_shift 2 \\\n    --num_persistent_param_in_dit 0\n```\n\n## 🚀Computational Efficiency\n\n#### 1) Non quantitative results\n\nThe results are evaluated on A100 GPUs for multi-person generation. Single-person generation uses less memory and provides faster inference.\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fefficiency.png\">\n\u003C\u002Fp>\nTeaCache is capable of increasing speed by approximately 2~3x.\n\n#### 2) Quantitative results\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fnone_quant_efficiency.png\">\n\u003C\u002Fp>\n\n\n## 📚 Citation\n\nIf you find our work useful in your research, please consider citing:\n\n```\n@article{kong2025let,\n  title={Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation},\n  author={Kong, Zhe and Gao, Feng and Zhang, Yong and Kang, Zhuoliang and Wei, Xiaoming and Cai, Xunliang and Chen, Guanying and Luo, Wenhan},\n  journal={arXiv preprint arXiv:2505.22647},\n  year={2025}\n}\n```\n\n## 📜 License\nThe models in this repository are licensed under the Apache 2.0 License. We claim no rights over the your generated contents, \ngranting you the freedom to use them while ensuring that your usage complies with the provisions of this license. \nYou are fully accountable for your use of the models, which must not involve sharing any content that violates applicable laws, \ncauses harm to individuals or groups, disseminates personal information intended for harm, spreads misinformation, or targets vulnerable populations. \n\n","MultiTalk 是一个基于音频驱动的多人对话视频生成项目。它能够根据输入的音频生成多个人物之间的对话、唱歌、互动控制以及卡通角色的视频，支持多种场景下的应用。该项目采用Python语言开发，具备强大的音视频处理能力，可以实现高质量的面部表情和口型同步。适合用于虚拟会议、在线教育、娱乐内容创作等需要自动生成对话视频的场景。",2,"2026-06-11 03:41:35","high_star"]