[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80817":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":13,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":18,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":23,"hasPages":21,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":33,"readmeContent":34,"aiSummary":35,"trendingCount":15,"starSnapshotCount":15,"syncStatus":36,"lastSyncTime":37,"discoverSource":38},80817,"LeapAlign_Code","RockeyCoss\u002FLeapAlign_Code","RockeyCoss","[CVPR2026] LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories","https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.15311",null,"Python",49,1,37,0,10,12,30,0.9,"Apache License 2.0",false,"main",true,[25,26,27,28,29,30,31,32],"diffusion","diffusion-model","diffusion-models","diffusion-rl","flow-matching","flow-matching-models","flux","text-to-image","2026-06-12 02:04:07","## LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories\u003Cbr>\u003Csub>Official PyTorch Implementation\u003C\u002Fsub>\n\n![method](assets\u002Fimages\u002Fmethod.webp)\n\n> [**LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories**](https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.15311)\u003Cbr>\n> [Zhanhao Liang](https:\u002F\u002Fgithub.com\u002FRockeyCoss)\u003Csup>1,2,*\u003C\u002Fsup>, [Tao Yang](https:\u002F\u002Fgithub.com\u002FThomasMrY)\u003Csup>2,*,†\u003C\u002Fsup>, [Jie Wu](https:\u002F\u002Fwujie1010.github.io\u002F)\u003Csup>2\u003C\u002Fsup>, [Chengjian Feng](https:\u002F\u002Ffcjian.github.io\u002F)\u003Csup>2\u003C\u002Fsup>, [Liang Zheng](https:\u002F\u002Fzheng-lab-anu.github.io\u002F)\u003Csup>1\u003C\u002Fsup>\u003Cbr>\n> \u003Csup>1\u003C\u002Fsup>ANU, \u003Csup>2\u003C\u002Fsup>ByteDance Seed\u003Cbr>\n> \u003Csup>*\u003C\u002Fsup>Equal contribution, \u003Csup>†\u003C\u002Fsup>Project Lead\u003Cbr>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.15311\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPaper-arXiv-red?style=for-the-badge\" alt=\"Paper\" height=\"24\">\u003C\u002Fa>\n     \n  \u003Ca href=\"https:\u002F\u002Frockeycoss.github.io\u002Fleapalign\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject-Page-blue?style=for-the-badge\" alt=\"Project Page\" height=\"24\">\u003C\u002Fa>\n     \n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Frockeycoss\u002FLeapAlign\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHugging-Face-yellow?style=for-the-badge\" alt=\"Hugging Face\" height=\"24\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n## Overview\nLeapAlign is a post-training method for aligning flow matching models with human preferences by backpropagating reward gradients through the differentiable generation process. Since backpropagation through the full sampling trajectory is memory-intensive and prone to gradient explosion, early generation steps that determine the global structure and layout of the final image are difficult to optimize. LeapAlign addresses this by constructing a two-step **leap trajectory** and backpropagating gradients through it instead of the full sampling trajectory, enabling gradient propagation to **any** generation step.\n\n## Dependency Setup\n### 1. Create the Conda environment\n```bash\nconda create --name leapalign python=3.11 -y\nconda activate leapalign\n```\n\n### 2. Install project dependencies\n```bash\n.\u002Fsetup_env.sh\n```\n\nThis script installs the Python dependencies for LeapAlign, installs `python3-tk`, and clones and installs `HPSv2`.\n\n### 3. Download checkpoints\n```bash\nmkdir -p data\nhuggingface-cli download black-forest-labs\u002FFLUX.1-dev --local-dir .\u002Fdata\u002Fflux\n\nmkdir -p hps_ckpt\nhuggingface-cli download xswu\u002FHPSv2 HPS_v2.1_compressed.pt --local-dir .\u002Fhps_ckpt\nhuggingface-cli download laion\u002FCLIP-ViT-H-14-laion2B-s32B-b79K open_clip_pytorch_model.bin --local-dir .\u002Fhps_ckpt\n```\n\n### 4. Prepare prompt embeddings\nThe preprocessing scripts use 8 GPUs by default.\n\nPrepare HPDv2 prompt embeddings:\n```bash\nbash scripts\u002Fpreprocess\u002Fpreprocess_flux_hpdv2_rl_embeddings.sh\n```\n\nPrepare GenEval prompt embeddings:\n```bash\nbash scripts\u002Fpreprocess\u002Fpreprocess_flux_geneval_rl_embeddings.sh\n```\n\n## Fine-tune with LeapAlign\nThe provided training scripts are configured for a 2x8 setup by default, i.e., 2 nodes with 8 GPUs each.\n\n### HPDv2\nLaunch on node 0:\n```bash\nMASTER_ADDR=\u003Cnode0_host> MASTER_PORT=29500 NNODES=2 NODE_RANK=0 NPROC_PER_NODE=8 \\\nbash scripts\u002Ffinetune\u002Ftrain_flux_leapalign_hpdv2_hpsv2.sh\n```\n\nLaunch on node 1:\n```bash\nMASTER_ADDR=\u003Cnode0_host> MASTER_PORT=29500 NNODES=2 NODE_RANK=1 NPROC_PER_NODE=8 \\\nbash scripts\u002Ffinetune\u002Ftrain_flux_leapalign_hpdv2_hpsv2.sh\n```\n\n### GenEval\nLaunch on node 0:\n```bash\nMASTER_ADDR=\u003Cnode0_host> MASTER_PORT=29500 NNODES=2 NODE_RANK=0 NPROC_PER_NODE=8 \\\nbash scripts\u002Ffinetune\u002Ftrain_flux_leapalign_geneval_hpsv2.sh\n```\n\nLaunch on node 1:\n```bash\nMASTER_ADDR=\u003Cnode0_host> MASTER_PORT=29500 NNODES=2 NODE_RANK=1 NPROC_PER_NODE=8 \\\nbash scripts\u002Ffinetune\u002Ftrain_flux_leapalign_geneval_hpsv2.sh\n```\n\n## Evaluation\n\n### HPDv2\n#### 1. Sample images\nThe example below uses 8 GPUs.\n\n```bash\ntorchrun --nproc_per_node=8 fastvideo\u002Fevaluation\u002Fsample_flux_images.py \\\n    --model_path \u003Cpath_to_finetuned_transformer> \\\n    --output_dir .\u002Foutput\u002Fflux_leapalign_hpsv2\n```\n\n`--model_path` should point to a checkpoint directory saved during fine-tuning.\n\n#### 2. Compute average HPSv2.1 reward\n```bash\npython fastvideo\u002Fevaluation\u002Fevaluate_reward_values.py \\\n    --results_dir .\u002Foutput\u002Fflux_leapalign_hpsv2 \\\n    --output_dir .\u002Foutput\u002Frewards_log\n```\n\nThe script prints the average HPSv2.1 score and writes `\u003Cresults_dir_basename>_rewards_log.json` to `--output_dir`.\n\n### GenEval\n#### 1. Sample images\nThe example below uses 8 GPUs.\n\n```bash\ntorchrun --nproc_per_node=8 fastvideo\u002Fevaluation\u002Fsample_flux_images_geneval.py \\\n    --model_path \u003Cpath_to_finetuned_transformer> \\\n    --outdir .\u002Foutput\u002Fflux_leapalign_geneval\n```\n\n#### 2. Score with the GenEval evaluator\nFollow the setup and scoring instructions in the [official GenEval repo](https:\u002F\u002Fgithub.com\u002Fdjghosh13\u002Fgeneval) and run their evaluator against the directory produced in step 1.\n\n## Inference with Released Checkpoints\n\nWe release **FLUX checkpoints fine-tuned with LeapAlign** on the [Hugging Face Hub](https:\u002F\u002Fhuggingface.co\u002Frockeycoss\u002FLeapAlign). All checkpoints are trained by applying **LeapAlign** on top of **FLUX.1-dev**.\n\n- `flux_geneval_hpsv2`: FLUX + LeapAlign + **GenEval** prompt set + **HPSv2.1** reward\n- `flux_hpdv2_hpsv2`: FLUX + LeapAlign + **HPDv2** prompt set + **HPSv2.1** reward\n- `flux_mjhq30k_hpsv3`: FLUX + LeapAlign + **MJHQ-30k** prompt set + **HPSv3** reward\n\n### 1. Download checkpoints\n\n```bash\n# Download into the current directory (creates local folders)\nhuggingface-cli download rockeycoss\u002FLeapAlign --local-dir . --include \"flux_geneval_hpsv2\u002F*\"\nhuggingface-cli download rockeycoss\u002FLeapAlign --local-dir . --include \"flux_hpdv2_hpsv2\u002F*\"\nhuggingface-cli download rockeycoss\u002FLeapAlign --local-dir . --include \"flux_mjhq30k_hpsv3\u002F*\"\n```\n\n### 2. Run inference\n\n```python\nimport torch\n\nfrom diffusers import FluxPipeline, FluxTransformer2DModel\n\nbase_id = \"black-forest-labs\u002FFLUX.1-dev\"\nckpt_dir = \"flux_geneval_hpsv2\"  # or \"flux_hpdv2_hpsv2\" \u002F \"flux_mjhq30k_hpsv3\"\n\npipe = FluxPipeline.from_pretrained(\n    base_id,\n    torch_dtype=torch.bfloat16,\n    use_safetensors=True,\n)\n\n# Load LeapAlign-aligned transformer weights from local folder\npipe.transformer = FluxTransformer2DModel.from_pretrained(\n    ckpt_dir,\n    torch_dtype=torch.bfloat16,\n)\n\npipe.to(\"cuda\")\n\nimage = pipe(\n    \"a very cute cat\",\n    guidance_scale=3.5,\n    height=720,\n    width=720,\n    num_inference_steps=50,\n    max_sequence_length=512,\n).images[0]\n\nimage.save(\"out.png\")\n```\n\n## Acknowledgments\nLeapAlign builds upon several excellent open-source projects. We would like to thank the authors and contributors of:\n- [DanceGRPO](https:\u002F\u002Fgithub.com\u002FXueZeyue\u002FDanceGRPO\u002Ftree\u002Fmain)\n- [FastVideo](https:\u002F\u002Fgithub.com\u002Fhao-ai-lab\u002Ffastvideo)\n- [Diffusers](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fdiffusers)\n- [MixGRPO](https:\u002F\u002Fgithub.com\u002FTencent-Hunyuan\u002FMixGRPO)\n- [GenEval](https:\u002F\u002Fgithub.com\u002Fdjghosh13\u002Fgeneval)\n- [HPSv2](https:\u002F\u002Fgithub.com\u002Ftgxs002\u002FHPSv2\u002Ftree\u002Fmaster)\n\n## :mailbox_with_mail: Citation\nIf you find this code useful in your research, please consider citing:\n\n```\n@article{liang2026leapalign,\n  title={LeapAlign: Post-Training Flow Matching Models at Any Generation Step by Building Two-Step Trajectories},\n  author={Liang, Zhanhao and Yang, Tao and Wu, Jie and Feng, Chengjian and Zheng, Liang},\n  journal={arXiv preprint arXiv:2604.15311},\n  year={2026}\n}\n```\n","LeapAlign 是一种用于在任意生成步骤通过构建两步轨迹来对齐流匹配模型的后训练方法。该项目利用反向传播奖励梯度通过可微生成过程，以调整模型使其更符合人类偏好；其核心在于通过创建一个“跳跃轨迹”而不是整个采样路径来进行梯度回传，从而有效解决早期生成阶段优化难题及内存消耗问题。适用于需要提高图像生成质量且注重全局结构与布局控制的场景中，如文本到图像转换等应用领域。项目基于 PyTorch 实现，并采用 Apache License 2.0 开源许可协议发布。",2,"2026-06-11 04:02:27","CREATED_QUERY"]