[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-105312":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":17,"stars30d":17,"stars90d":16,"forks30d":16,"starsTrendScore":18,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":23,"hasPages":21,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":30,"readmeContent":31,"aiSummary":10,"trendingCount":16,"starSnapshotCount":16,"syncStatus":15,"lastSyncTime":32,"discoverSource":33},105312,"AReA","KooroshRH\u002FAReA","KooroshRH","Official implementation of \"AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers\"","https:\u002F\u002Frajabi2001.github.io\u002Farea\u002F",null,"Python",149,5,4,2,0,29,87,75.73,"Apache License 2.0",false,"main",true,[25,26,27,28,29],"diffusion-models","efficient-inference","high-resolution","image-generation","prompt-dataset","2026-11-01 00:14:02","\u003Cp align=\"center\">\n  \u003Cpicture>\n    \u003Csource media=\"(prefers-color-scheme: dark)\" srcset=\"assets\u002Ftitle-dark.png\">\n    \u003Cimg src=\"assets\u002Ftitle-light.png\" alt=\"AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers\" width=\"640\">\n  \u003C\u002Fpicture>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Frajabi2001.github.io\u002Farea\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject-Page-blue\" alt=\"Project Page\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.17227\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2610.17227-b31b1b.svg\" alt=\"arXiv\">\u003C\u002Fa>\n  \u003Ca href=\"LICENSE\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache%202.0-green.svg\" alt=\"License: Apache 2.0\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fkooroshrh\u002FAReA\">\u003Cimg src=\"assets\u002Fopen-in-spaces.svg\" alt=\"Open in Spaces\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fteaser.jpg\" alt=\"AReA samples across FLUX.1, Qwen-Image, FLUX.2 and Ideogram 4\" width=\"900\"\u002F>\n\u003C\u002Fp>\n\n**AReA** is a training-free attention mechanism for generating images beyond the training resolution of diffusion transformers. Attention's effective receptive field narrows during denoising, from global in early steps that form structure to local in later steps that refine details. AReA follows this progression in two phases:\n\n- **Phase I, cross-scale coarse attention.** Queries stay at full resolution while keys and values keep one token per *f × f* block, placed on a grid close to the model's native resolution. Every token still sees the whole image, at 1\u002F*f*² of the cost.\n- **Phase II, adaptive windowed fine attention.** Attention runs inside local windows that shrink at every step down to the native size. Windows over smooth regions of a wavelet (DWT) detail map are split into quadrants, a narrow **Halo** of neighbouring tokens extends each window's keys and values, and a **global-KV bridge** adds distinctive tokens from the rest of the image.\n\nAReA never computes full attention at the target resolution and needs no RoPE extrapolation, extra parameters or fine-tuning. It is model agnostic, and its speedup over full attention grows with resolution, up to 3.2× in our experiments.\n\n## News\n\n- **[2026-10-30]** Qwen-Image support.\n- **[2026-10-19]** Ideogram 4 support, and `--multi-gpu` to spread a model over several GPUs.\n- **[2026-10]** Code released with FLUX.1 support, together with the HRGen-1K benchmark prompts.\n- **Coming soon:** FLUX.2.\n\n## Supported models\n\n| Model | `--model` | Default checkpoint | Status |\n|---|---|---|---|\n| FLUX.1 | `flux1` | [`black-forest-labs\u002FFLUX.1-Krea-dev`](https:\u002F\u002Fhuggingface.co\u002Fblack-forest-labs\u002FFLUX.1-Krea-dev) | Available |\n| Qwen-Image | `qwen-image` | [`Qwen\u002FQwen-Image`](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen-Image) | Available |\n| FLUX.2 | | | Coming soon |\n| Ideogram 4 | `ideogram4` | [`ideogram-ai\u002Fideogram-4-nf4-diffusers`](https:\u002F\u002Fhuggingface.co\u002Fideogram-ai\u002Fideogram-4-nf4-diffusers) | Available |\n\nAReA scales up from a *base resolution*: Phase I pools keys and values onto a grid that fits it, and Phase II windows shrink down to it. The base is 1024 px for FLUX.1. Qwen-Image pools onto a grid that fits its 1328 px native resolution, and its windows shrink to 1024 px. For Ideogram 4, whose native resolution is 2048 px, it is 1024 px up to 4096 × 4096 and 2048 px, the largest it can be, above. AReA works best for targets about 3× the base or more.\n\n## Installation\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FKooroshRH\u002FAReA.git\ncd AReA\n\n# Install PyTorch for your CUDA version first, then:\npip install -r requirements.txt\n```\n\nModel weights are downloaded from Hugging Face on first use. Some checkpoints are gated: accept their license on Hugging Face and log in with `huggingface-cli login`.\n\n## Quick start\n\nTo try AReA without installing anything, use the [Hugging Face demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fkooroshrh\u002FAReA) (FLUX.1, 2K to 4K).\n\n```bash\npython generate.py --model flux1 \\\n  --prompt \"A vibrant village with colorful houses and a picturesque castle is set against a backdrop of mountains and a clear blue sky\" \\\n  --height 4096 --width 4096 --seed 0\n```\n\nThe image is written to `outputs\u002F`. `--model` selects the model (see [Supported models](#supported-models)), and each model runs with its own default checkpoint, number of steps and guidance scale. `--checkpoint` loads another checkpoint of the same model, given as a Hugging Face repo id or a local path:\n\n```bash\npython generate.py --model flux1 --checkpoint black-forest-labs\u002FFLUX.1-dev \\\n  --prompt \"...\" --height 4096 --width 4096\n```\n\nWith `--model ideogram4`, the prompt is first expanded by Ideogram 4's prompt enhancer. With `--model qwen-image`, a quality suffix is appended to every prompt: Qwen-Image's own \", Ultra HD, 4K, cinematic composition\", followed by \", faithful to the described content, quantities, style and medium, coherent fine detail, consistent scale, proportion and perspective, believable light and shadow, smooth tonal gradations.\" It names no content, lighting or style, so it suits any prompt; `--no-prompt-suffix` turns it off.\n\nOn GPUs with less than 60 GB of memory, idle models are offloaded to the CPU automatically. `--multi-gpu` spreads the model over all visible GPUs instead, for canvases that do not fit on one.\n\n## Using AReA in your own code\n\nAReA attaches in place to a [diffusers](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fdiffusers) pipeline of a supported model, and the pipeline is then called as usual. For example, with FLUX.1:\n\n```python\nimport torch\nfrom diffusers import FluxPipeline\nfrom area import AReA\n\npipe = FluxPipeline.from_pretrained(\"black-forest-labs\u002FFLUX.1-Krea-dev\", torch_dtype=torch.bfloat16)\npipe.vae.enable_tiling()\npipe.to(\"cuda\")              # or pipe.enable_model_cpu_offload()\n\narea = AReA(pipe)\n\nimage = pipe(\n    \"A vibrant village with colorful houses and a picturesque castle\",\n    height=4096,\n    width=4096,\n    num_inference_steps=28,\n    guidance_scale=4.5,\n    generator=torch.Generator(\"cuda\").manual_seed(0),\n).images[0]\nimage.save(\"village.png\")\n\narea.remove()                # back to the original attention\n```\n\nQwen-Image runs true classifier-free guidance only when a `negative_prompt` is given (an empty string works); `true_cfg_scale` sets its strength. `generate.py` also appends a quality suffix (`QWEN_SUFFIX` in `generate.py`) to every prompt, so add it to your prompts when you call the pipeline yourself. For Ideogram 4, load the pipeline with `area.pipelines.SlimIdeogram4Pipeline` instead of diffusers' `Ideogram4Pipeline`; it uses the same weights and fits much larger canvases in memory.\n\nCanvases at or below the model's native resolution keep the original attention. Height and width must divide into whole patches, and the two patch counts must share a factor of at least 2, since Phase I pools *f × f* blocks.\n\n## Options\n\n| `AReA(...)` argument | `generate.py` flag | Default | Description |\n|---|---|---|---|\n| `pooling` | `--pooling` | `subsample` | Phase I: the token kept for each *f × f* block. `subsample` (block centre), `mean`, `medoid` or `random`. |\n| `switch_sigma` | `--switch-sigma` | per model | Noise level σ\\* at which Phase II takes over: Phase II starts at the first step whose σ is at or below σ\\*. FLUX.1 0.89 (step 8 of 28), Qwen-Image 0.87 (step 20 of 40 at 4096 × 4096), Ideogram 4 0.92 (step 20 of 48 at 4096 × 4096). |\n| `halo` | `--halo` | 8 | Largest Halo width δ\u003Csub>max\u003C\u002Fsub>, in patches. |\n| `bridge_ratio` | `--bridge-ratio` | 0.5 | Global-KV bridge ratio ρ: each window of side *w* gets (ρ*w*)² bridge tokens. |\n| `seed` | `--seed` | 0 | Seed for window boundary jitter, Halo widths and `random` pooling. `generate.py` uses the generation seed. |\n\nThe switch is a noise level rather than a step count because models differ in noise schedules and step counts (Appendix A.3 of the paper), so one σ\\* lands on different steps for different models, step counts and, for Qwen-Image and Ideogram 4, resolutions. To try AReA on another model, search σ\\* between 0.86 and 0.96, where the defaults of the models we tuned lie, instead of reusing another model's number of Phase I steps.\n\nOther `generate.py` flags: `--model`, `--checkpoint`, `--height`, `--width`, `--steps`, `--guidance`, `--offload`, `--multi-gpu`, `--no-prompt-suffix`, `--output-dir`, and `--prompt-file` to generate a whole prompt list. Run `python generate.py --help` for details and defaults.\n\n## HRGen-1K\n\n[`HRGen-1K\u002Fprompts.jsonl`](HRGen-1K\u002Fprompts.jsonl) holds the 1,000 prompts of HRGen-1K, our benchmark for high-resolution generation. The prompts span natural scenes, built environments, people, animals, objects and imaginative worlds, and target failure modes that single-subject prompts rarely reveal: complex scenes with many interacting subjects, spatial relations, explicit object counts, reflections and transparency, and 125 text-rendering prompts. Prompt length ranges from 9 to 91 words (32.5 on average). Each line is a JSON object with an `id` and a `prompt`:\n\n```bash\npython generate.py --model flux1 --prompt-file HRGen-1K\u002Fprompts.jsonl --height 4096 --width 4096 --seed 0\n```\n\n## Repository layout\n\n```text\nAReA\u002F\n├── area\u002F\n│   ├── schedule.py      # Phase I pooling ramp and Phase II window sizes\n│   ├── attention.py     # cross-scale coarse attention and windowed fine attention\n│   ├── tiling.py        # base tiling, DWT detail map, splits, Halo, merge weights\n│   ├── bridge.py        # global-KV bridge token selection\n│   ├── multi_gpu.py     # spreading a model over several GPUs\n│   ├── backbones\u002F       # one adapter per model family\n│   └── pipelines\u002F       # memory-efficient pipelines (Ideogram 4)\n├── generate.py          # command-line generation\n├── HRGen-1K\u002F            # benchmark prompts\n└── assets\u002F\n```\n\nThe method lives in the model-independent modules; a backbone adapter only describes how a model lays out its tokens and positions, so supporting a new model means adding one adapter.\n\n## AReA-Bootstrap\n\nAReA-Bootstrap is available for FLUX.1 only. It replaces Phase I with a generation at the native resolution, as multi-stage methods do: the native-resolution latent is upsampled to the target size, re-noised to the level at which Phase II begins, and denoised with Phase II.\n\n```bash\npython generate.py --model flux1 --prompt \"...\" --height 4096 --width 4096 --bootstrap\n```\n\n```python\nimage = area.bootstrap(\"...\", height=4096, width=4096, seed=0).images[0]\n```\n\n## Citation\n\n```bibtex\n@misc{rajabi2026area,\n      title={AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers},\n      author={Javad Rajabi and Koorosh Roohi and David B. Lindell and Babak Taati},\n      year={2026},\n      eprint={2610.17227},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.17227},\n}\n```\n","2026-10-31 04:26:25","CREATED_QUERY"]