[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-94544":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":9,"rankLanguage":9,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":9,"pushedAt":9,"updatedAt":23,"readmeContent":24,"aiSummary":25,"trendingCount":15,"starSnapshotCount":15,"syncStatus":26,"lastSyncTime":27,"discoverSource":28},94544,"h3.c","antirez\u002Fh3.c","antirez","MiniMax H3 inference engine for Mac computers",null,"C",1875,122,9,4,0,807,64.27,"MIT License",false,"main",true,[],"2026-08-24 04:01:22","# h3-metal\n\nNative MiniMax-H3 inference for Apple Silicon. The project is being built as a\nsequence of working vertical slices: deterministic host\u002Fmodel metadata first,\nthen portable Metal block parity, prompt encoding, prompt-to-video\u002Faudio, and\nfirst\u002Flast-frame conditioning and then ordered references.\n\nPrompt-to-video\u002Faudio, first\u002Flast-frame conditioning, and ordered Ref2VA\nimage\u002Fvideo\u002Faudio references work end to end. The current work is incremental\nH3-specific Metal performance and memory optimization on M3 Max and M5 Max.\n\n## Tutorial\n\n### 1. Build and inspect the model\n\nThe examples assume that the Hugging Face snapshot is in `.\u002FMiniMax-H3` and\nthat FFmpeg and FFprobe are available on `PATH`.\n\n```sh\nmake -j8\nmkdir -p outputs\n.\u002Fh3 --info -d .\u002FMiniMax-H3\n```\n\n`--info` checks the model layout and prints the selected Metal device without\nmapping all weights or generating media. Run `.\u002Fh3 --help` for the complete CLI\nreference.\n\nWithout `-p`, the same binary starts an Iris-style interactive session:\n\n```sh\n.\u002Fh3 -d .\u002FMiniMax-H3 --width 512 --height 512 --steps 6\n```\n\nType a prompt to generate a numbered video. The session keeps the exact BF16\nprompt conditioning, prepared DiT, and video decoder in memory, so repeating a\nprompt with another seed avoids loading and encoding them again. Useful commands\nare `!status`, `!seed random`, `!seconds 2`, `!show`, `!save output.mp4`, and\n`!cache`. Use `!help` for the full, short list.\n\nFirst\u002Flast-frame conditioning is persistent in the session:\n\n```text\nh3> !first opening.png\nh3> !last ending.png\nh3> The camera moves slowly around the subject.\n```\n\nUse `!first clear` or `!last clear` to remove an anchor. Generated videos are\nwritten to the session directory printed at startup.\n\nFor a general Ref2VA conditioning image, use `!ref-image PATH` instead. Images\nare appended in order and exposed to the model as `\u003CPicture 1>`, `\u003CPicture 2>`,\nand so on; filenames have no meaning to the model.\n\n```text\nh3> !ref-image person.png\nh3> Make the person shown in Picture 1 wave to the camera.\n```\n\n`!refs` lists the current order, `!ref-remove N` removes one entry, and\n`!refs clear` removes them all. Ref2VA references cannot be mixed with\n`!first`\u002F`!last` anchors.\n\n### 2. Make a first fast video\n\nStart with the validated balanced preset. It generates 22 frames at 24 fps\n(about 0.92 seconds), displays the evolving middle-video frame after every\ndenoising transition in a supported graphical terminal, and prints phase\ntimings:\n\n```sh\n.\u002Fh3 --profile \\\n  -d .\u002FMiniMax-H3 \\\n  -p \"A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur, soft footsteps and wind.\" \\\n  --width 512 --height 512 \\\n  --frames 22 --steps 20 \\\n  --layers 45 --reuse 2 \\\n  --show \\\n  -o outputs\u002Ffox-fast.mp4\n```\n\nThis is deliberately not the most aggressive configuration:\n\n- `--steps 20` performs the default 20 denoising passes.\n- `--reuse 2` computes 11 fresh denoiser velocities instead of all 20 and\n  extrapolates the skipped transitions.\n- `--layers 45` runs 45 of the 50 transformer blocks, reducing both time and\n  unified-memory use.\n- `--show` is optional. It supports Kitty\u002FGhostty and\n  iTerm2\u002FWezTerm\u002FKonsole graphical protocols. It loads a resident preview VAE,\n  displays one representative middle-video frame after every Euler transition,\n  and then displays all final frames. Display dimensions default to 2x so the\n  image has its intended logical size on macOS Retina screens; use `--zoom 1`\n  on a non-HiDPI display. This adds preview decode time and roughly 10 GiB of\n  temporary model residency; runs without `--show` are unchanged.\n- `--profile` is optional and does not select a different generation path.\n\nThe first process invocation also pays model loading and filesystem-cache\ncosts. Compare performance using repeated runs, and alternate variants when\nthe machines are warming up because this workload is sensitive to thermal\nthrottling.\n\nFor a very short iteration, request four denoising passes directly:\n\n```sh\n.\u002Fh3 --profile \\\n  -d .\u002FMiniMax-H3 \\\n  -p \"A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur.\" \\\n  --width 512 --height 512 --frames 22 \\\n  --steps 4 --layers 50 --reuse 1 \\\n  --show \\\n  -o outputs\u002Ffox-four-step.mp4\n```\n\n`--steps N` always means exactly N denoising passes. Four through seven passes\nuse the same schedule that won the low-budget comparison; increasing from 4\nto 7 progressively improves detail and motion. Keep `--reuse 1` at such small\nbudgets so every requested pass runs the model. `--show` displays one preview\nafter each pass.\n\nSeveral tail-heavy schedules were evaluated because most visible cleanup\nhappens late in a long run. They preserved too few early composition updates\nand produced woven texture, weak motion, or clipped colors. The retained mode\nuses the released linear base grid with one terminal point. On the 512-square,\n22-frame fox test, the selected four-pass result had 0.556 full-video SSIM\nagainst a 29-pass reference; an independent surfer test measured 0.547. The\nfour-pass denoise took about 3.5 seconds on M5 Max, versus 26.4 seconds for the\nreference.\n\nFor a low-memory run, add `--ssd-streaming`:\n\n```sh\n.\u002Fh3 --profile \\\n  -d .\u002FMiniMax-H3 \\\n  -p \"A red fox walks through fresh snow in a pine forest.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --layers 50 --reuse 1 --ssd-streaming \\\n  -o outputs\u002Ffox-ssd.mp4\n```\n\nThis uses the original BF16 checkpoint without conversion or quantization. It\nkeeps two DiT blocks in memory and reads the next block from SSD while the GPU\nruns the current one. On M5 Max, tracked DiT storage fell from about 36.5 GiB to\n2.0 GiB at 512 square and 2.1 GiB at 864x480. A warm 50-block forward measured\n1.35 versus 2.49 seconds at 512 square (84% slower), and 2.14 versus 2.68\nseconds at 864x480 (26% slower). These are comparisons against the same\nfull-residency BF16 path, and the results were byte-identical in both checks.\n\nThe 2.0--2.1 GiB figure is the DiT's tracked tensor storage, not total system\nRAM. Prompt encoding and the two VAEs run in separate phases rather than adding\ntheir full peaks to it; the OS, media buffers, and output resolution still need\nheadroom. `--show` keeps a preview VAE resident and adds roughly 10 GiB, so omit\nit for the lowest-memory run.\n\nSSD streaming is an explicit memory\u002Fspeed tradeoff and is not the default. It\ncannot be combined with `--use-int8-row-fc2`. In an interactive session, use\n`!ssd-streaming on`.\n\n### 3. Move toward reference quality\n\nChange one control at a time when evaluating quality. First restore all layers,\nthen all denoiser evaluations, and finally raise the default 20-pass schedule\nto the slower 50-pass reference:\n\n```sh\n.\u002Fh3 --profile \\\n  -d .\u002FMiniMax-H3 \\\n  -p \"A red fox walks through fresh snow in a pine forest. Medium tracking shot, natural winter light, realistic fur, soft footsteps and wind.\" \\\n  --width 512 --height 512 \\\n  --frames 22 --steps 50 \\\n  --layers 50 --reuse 1 \\\n  -o outputs\u002Ffox-close.mp4\n```\n\nThe defaults are `--steps 20 --layers 50 --reuse 1`; keep `--steps 50`\nexplicit for this close path. It performs 50 complete 50-block denoiser\nforwards and is much more expensive than the default, but is the right oracle\nwhen a fast mode changes the subject, anatomy, motion, or composition.\nNumerical pixel identity with MLX is not expected because the random-number and\nexecution engines differ; the depicted content and motion should agree.\n\n### 4. Choose a speed\u002Fquality preset\n\nThese controls are independent unless noted otherwise:\n\n| Control | Slow reference | Default | Aggressive | Main impact |\n|---|---:|---:|---:|---|\n| Denoising passes | `--steps 50` | `--steps 20` | `--steps 4..7` | The number always names actual denoising passes. |\n| Whole denoiser reuse | `--reuse 1` | `--reuse 2` | `--reuse 3` | At 20 steps: 20, 11, or 8 fresh DiT evaluations. |\n| Active DiT blocks | `--layers 50` | `--layers 45` | `--layers 40` | Fewer blocks reduce compute and resident transformer weights. |\n| Core residual reuse | `--core-reuse 1` | `--core-reuse 4` | `--core-reuse 6` | Refreshes patch\u002Fhead work every step but runs the expensive core less often. |\n| Token reduction | off | optional | `--token-reduction` | Pairs horizontal video tokens inside middle blocks; faster but may change composition. |\n| Internal canvas | output size | `384x384` for 512 square output | `320x320` | Runs DiT\u002FVAE smaller, then upscales with vImage. |\n\nOn M5, `--use-int8-row-fc2` uses one activation scale per FC2 row and a single\nfull-width TensorOps product. It is optional because it is less numerically\nconservative than grouped int8. It reduced complete denoiser forwards by about\n2.6% in reciprocal tests. Matched four-step fox and surfer videos kept the same\nsubjects, setting, and motion (full-video SSIM 0.919 and 0.828). In the\ninteractive session, use `!int8-row-fc2 on`.\n\n`--reuse` and `--core-reuse` are mutually exclusive. Layer thinning can be\ncombined with either one.\n\nTo make the first command faster while keeping its output resolution, add\ntoken reduction:\n\n```sh\n.\u002Fh3 --profile \\\n  -d .\u002FMiniMax-H3 \\\n  -p \"A surfer riding inside a sharp blue ocean wave, one rider and one white board, realistic spray.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --layers 45 --reuse 2 --token-reduction \\\n  -o outputs\u002Fsurfer-fast.mp4\n```\n\nAt the validated 512 square shape, token reduction cut the `45 layers + reuse\n2` denoise profile from 16.69 to 12.60 seconds on the IT M5 Max. Independent\nfox and surfer renders stayed coherent, but composition can diverge more from\nthe close path.\n\nFor an aggressive preview, render internally at 320 square and upscale to the\nrequested 512 square output:\n\n```sh\n.\u002Fh3 --profile \\\n  -d .\u002FMiniMax-H3 \\\n  -p \"A red fox walking through snow, realistic, tracking shot.\" \\\n  --width 512 --height 512 \\\n  --render-width 320 --render-height 320 \\\n  --frames 22 --steps 20 --layers 40 --reuse 3 \\\n  -o outputs\u002Ffox-aggressive.mp4\n```\n\nThis combination produced a clean, recognizable 22-frame fox in validation,\nbut loses fine detail and can change framing. Do **not** add `--token-reduction`\nto both `--layers 40` and `--reuse 3`: that tested combination produced color\nringing, outlines, and ghosted limbs.\n\nAs an alternative to whole-velocity reuse, this keeps the timestep-dependent\npatch and output heads fresh at every transition:\n\n```sh\n.\u002Fh3 --profile \\\n  -d .\u002FMiniMax-H3 \\\n  -p \"A surfer riding a blue ocean wave.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --layers 45 --core-reuse 4 \\\n  -o outputs\u002Fsurfer-core-reuse.mp4\n```\n\nUse `--core-reuse 6` only as an aggressive preview. Values above 6 are not\nexposed because validation lost subject fidelity.\n\n### 5. Pick resolution and duration\n\nWidth and height must each be multiples of 32, at least 32, and their product\nmust not exceed `768 * 1344` pixels. Those are mechanical limits, not a promise\nthat every tiny canvas has good model quality. H3-Base is a 768p model.\n\n| Canvas | Current guidance |\n|---|---|\n| `512x512` | Safest development size; repeatedly validated with multiple prompts. |\n| `768x768` | Validated close-quality square output; substantially more expensive. |\n| `1344x768`, `768x1344` | Released 768p-class landscape\u002Fportrait limit. |\n| `1024x768`, `768x1024` | Valid 4:3 and 3:4 768p-class canvases. |\n| `384x384` internal to `512x512` | Validated fast-quality scaling point. |\n| `320x320` internal to `512x512` | Validated aggressive scaling point. |\n| `256x256` | Native fast-preview canvas with automatic low-resolution RoPE adaptation. |\n\nFor a fast native 256-square preview:\n\n```sh\n.\u002Fh3 -d .\u002FMiniMax-H3 \\\n  -p \"A red fox walks through fresh snow in a pine forest.\" \\\n  --width 256 --height 256 \\\n  --frames 22 --steps 20 \\\n  --layers 50 --reuse 1 \\\n  -o outputs\u002Ffox-256.mp4\n```\n\nAt 256 square, H3 has only an `8x8` effective spatial-token grid, so it has less\nroom for fine detail and complex composition. H3 automatically halves spatial\nRoPE coordinates at exactly 256 square. This removed repeating lattice\nartifacts in long fox renders and stayed coherent on an independent portrait,\nwithout adding tokens or runtime. Use `--use-reference-rope` to restore the\nreleased\u002FMLX coordinates for parity checks. Keep token reduction off at this\nsize. Native 128 square remains unsupported: its `4x4` token grid did not\nrecover a recognizable subject even with adjusted RoPE.\n\n`--render-width` and `--render-height` must be set together, must have the same\naspect ratio as the output, and cannot exceed the output dimensions. The model\nand VAE use the internal size; terminal frames and the encoded video retain the\nrequested output size.\n\nH3 emits 24 fps and aligns frame requests upward to `5 + 17*n`:\n\nUse `--seconds N` for a duration-oriented request, or `--frames N` for direct\nframe control; the two options are mutually exclusive. Fractional seconds are\naccepted. Seconds are converted at 24 fps and then rounded upward to the next\nlegal H3 temporal shape, so `--seconds 10` produces 243 frames (10.125 seconds).\n\n| Frames | Approximate video duration |\n|---:|---:|\n| 22 | 0.917 seconds |\n| 39 | 1.625 seconds |\n| 56 | 2.333 seconds |\n| 107 | 4.458 seconds |\n| 243 | 10.125 seconds |\n| 362 | 15.083 seconds |\n\nShort clips are useful for development. The released workflow is intended for\nroughly 4–15 second videos. A request such as `--frames 23` is rounded up to 39\nframes rather than producing an arbitrary temporal shape.\n\n### 6. Improve the prompt\n\nA short prompt works, but the released system expects a Context-IR-like\ndescription. State the subject, action, setting, camera, lighting\u002Fstyle, and\ndesired sound. For example:\n\n```text\nScene: a single red fox in a snow-covered pine forest at dawn.\nAction: the fox walks steadily left to right and looks toward the camera once.\nCamera: medium-height lateral tracking shot, 50 mm lens, stable framing.\nLook: photorealistic fur, cold blue ambient light, warm sunrise rim light.\nAudio: soft footsteps in snow, light wind through pine branches, no music.\n```\n\nKeep identity and object counts explicit when they matter. `--seed N` controls\nthe native random stream; the default is 42. Compare options with the same\nprompt, seed, resolution, frame count, and step count.\n\n### 7. Preview frames and diagnose performance\n\n- `--show` displays a representative frame after every denoising transition,\n  followed by all frames from the completed video. Like Iris, it advertises 2x\n  display dimensions by default for Retina terminals; `--zoom N` changes that\n  factor without resizing the generated video or the encoded terminal image.\n- `--frames-dir DIR` writes final callback frames as PPM files. Intermediate\n  `--show` previews are not written there.\n- `-o ''` disables MP4 encoding; combine it with `--frames-dir` when FFmpeg is\n  unavailable.\n- `--profile` reports phase wall time, Metal encoding\u002Fwait time, peak live\n  tensor storage, cumulative allocation, and dispatch counts.\n\nFor example:\n\n```sh\n.\u002Fh3 --profile -d .\u002FMiniMax-H3 -p \"A hummingbird hovering over red flowers.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --layers 45 --reuse 2 --frames-dir outputs\u002Fhummingbird-frames \\\n  -o ''\n```\n\n### 8. Add image, video, and audio references\n\nFirst\u002Flast-frame anchors select the FL2VA path:\n\n```sh\n.\u002Fh3 -d .\u002FMiniMax-H3 -p \"The fox keeps walking through the snow.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --layers 45 --reuse 2 \\\n  --first-frame fox.png --last-frame fox-later.png \\\n  -o outputs\u002Ffox-anchored.mp4\n```\n\nOrdered references select the distinct Ref2VA checkpoint. Use the flag matching\nthe media semantics:\n\n```sh\n# One image reference.\n.\u002Fh3 -d .\u002FMiniMax-H3 -p \"Use the animal and setting in the reference.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --ref-image fox.png -o outputs\u002Ffox-reference.mp4\n\n# Continue a clip but ignore its soundtrack.\n.\u002Fh3 -d .\u002FMiniMax-H3 -p \"Continue the motion in this clip.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --ref-silent-video fox.mp4 -o outputs\u002Ffox-video-reference.mp4\n\n# Preserve the clip's embedded audio.\n.\u002Fh3 -d .\u002FMiniMax-H3 -p \"Continue this audiovisual scene.\" \\\n  --width 512 --height 512 --frames 56 --steps 20 \\\n  --ref-video fox-with-audio.mp4 -o outputs\u002Ffox-video-audio.mp4\n\n# Replace a video's soundtrack explicitly.\n.\u002Fh3 -d .\u002FMiniMax-H3 -p \"Continue the scene with the supplied music.\" \\\n  --width 512 --height 512 --frames 56 --steps 20 \\\n  --ref-video-audio silent-fox.mp4 replacement.wav \\\n  -o outputs\u002Ffox-replaced-audio.mp4\n\n# An ordered image plus standalone audio reference.\n.\u002Fh3 -d .\u002FMiniMax-H3 -p \"Use the animal and music from the references.\" \\\n  --width 512 --height 512 --frames 56 --steps 20 \\\n  --ref-image fox.png --ref-audio music.wav \\\n  -o outputs\u002Ffox-image-audio.mp4\n```\n\nReference flags may be repeated and their command-line order is preserved.\nStandalone audio must accompany an image or video reference. Audio references\nmust be 2–15 seconds; at most three audio inputs are accepted and their total\ndecoded duration is capped at 15 seconds.\n\n## Tests and runtime requirements\n\n```sh\nmake test\nmake parity\n```\n\n`make test` runs the deterministic host suite and, when the ignored MLX fixture\nis installed under `misc\u002Ffixtures\u002F`, compiles the Metal source at runtime and\nchecks a complete toy H3 block against named MLX outputs. Runtime compilation is\nintentional: it follows Iris and does not require Xcode's optional offline Metal\ntoolchain. The test covers both an F32 diagnosis path and the production BF16\nstorage path; wide BF16 matrix products and SDPA use cached MPSGraph graphs, with\ndirect Metal correctness fallbacks. `make parity` runs only those Metal\u002FMLX\nchecks.\n\nFFmpeg and FFprobe must be available on `PATH` for media inputs and MP4 output\n(`H3_FFMPEG` and `H3_FFPROBE` may select explicit executables). Generated RGB24 and\n32 kHz stereo F32 PCM are fed through concurrent pipes; no intermediate\nuncompressed media file is created.\n\n## Implementation and performance notes\n\nThe remainder documents the implementation behind the tutorial presets and the\nenvironment variables retained for exact A\u002FB diagnosis.\n\n### Sampler and DiT controls\n\nThe default sampler uses the released shifted video\u002Faudio schedule. `--steps`\nalways names the number of denoising passes, with terminal zero added after the\nlast pass. Whole-denoiser reuse evaluates the first and last pass plus every\nrequested interval, then extrapolates skipped video and audio velocities on\ntheir independent schedules. With very small step counts, keep `--reuse 1`.\n\nFor the low-budget path, the released linear base grid won against\nactual-video-sigma linear spacing,\nquadratic and cubic warps, exact 30-point tail subsets, mild power warps,\nzero-order held full-grid velocities, linear velocity extrapolation, and RES.\nThe more tail-heavy candidates often sharpened the subject but damaged motion\nor left a repetitive woven background; sparse RES and long extrapolation\nintervals failed much more visibly.\n\nLayer thinning ranks the checkpoint's actual AdaLN gates while protecting\nstructurally important first and final blocks. Unused weights and schedule\ntensors are not retained, so `--layers 45` and `--layers 40` reduce both\ntransformer time and unified-memory use. Core reuse holds the previous full\ntransformer residual while refreshing the patch projection and timestep-aware\nhead; it remains mutually exclusive with whole-velocity reuse.\n\n### Exact DiT fusions\n\nEvery active DiT block fuses its attention residual gate with the following MLP\nAdaLN. The rounded BF16 residual is still written exactly, but the same row is\nkept in threadgroup memory for normalization, eliminating one dispatch and one\nglobal reread. Away from token-reduction boundaries, the MLP residual gate also\nproduces the next block's attention AdaLN and carries that normalized state\nacross the loop. `H3_DISABLE_FUSED_GATE_ADALN=1` and\n`H3_DISABLE_FUSED_CROSS_BLOCK_ADALN=1` restore the two-kernel oracles.\nThe final audio\u002Fvideo AdaLN kernels bind directly to offsets in the residual\nstream, avoiding two slice blits and 18.8 MiB of scratch at 512x512 (29.4 MiB\nat the 864-class benchmark shape).\n`H3_DISABLE_FUSED_FINAL_SLICE=1` restores the copy-plus-AdaLN oracle at load.\nThe BF16 final heads then apply AdaLN while loading their 16x16 projection\ntiles, preserving the standalone rounding and accumulation order while\nremoving another equally sized normalized activation. The two optimizations\ntogether save 37.5\u002F58.9 MiB. `H3_DISABLE_FUSED_FINAL_HEAD=1` restores the\noffset-AdaLN-plus-linear oracle at load.\n\n### Token-reduction internals\n\n`--token-reduction` is an independent aggressive DiT mode. After block 3 it\npairs adjacent horizontal target-video tokens while leaving text, audio,\nconditions, and reference tokens exact. The complete full-resolution state is\nkept as a bypass. During the first ten noisy evaluations it restores before\nblock 40; subsequent detail-forming evaluations restore before block 30. Each\ntoken returns as its original value plus the update learned by its pair, so\nwithin-pair detail is not discarded.\nThe pooling kernel writes only true-pair baselines into a dense tail of the\nalready allocated attention scratch buffer; odd-width singleton tokens need no\nbaseline. The full bypass uses the oversized QKV tail when it fits, with a\nguarded dedicated fallback only for reference-heavy layouts. Common text-only\ncanvases therefore add no activation arena at any token-grid width. Pooling\nalso snapshots both source tokens while their BF16 values are already in\nregisters, avoiding a separate full-hidden blit and redundant source read. The\nsame entry kernel keeps each pooled row in threadgroup memory and emits the\nfirst reduced block's attention AdaLN, eliminating another global residual read.\nAt the restore boundary, the first full-resolution attention AdaLN is fused\ninto expansion: a 10.5 KiB threadgroup row avoids a global residual reread while\nstill writing the exact bypass needed by the following residual branch.\nOn a thermal-balanced 512x512x22, 19-forward IT M5 Max A\u002FB this reduced denoise\ntime from 39.13 to 28.06 seconds (28.3%). Final video\u002Faudio latent relative L2\nwas 5.56%\u002F15.14%. First\u002Fmiddle\u002Flast fox frames retained one clean muzzle,\ncoherent legs, and sharp fur; an independent surfer remained consistent with\none rider and board through the wave spray. It changes composition and is\ntherefore opt-in rather than the close-reference default.\n`H3_TOKEN_REDUCTION_BLOCKS` can override the later `4:30` interval;\n`H3_TOKEN_REDUCTION_EARLY=STEPS:END` overrides the early schedule and `0`\ndisables it. `H3_DISABLE_TOKEN_REDUCTION=1` provides an in-context exact oracle.\n`H3_DISABLE_FUSED_TOKEN_POOL_ADALN=1` and\n`H3_DISABLE_FUSED_TOKEN_ADALN=1` independently restore the two-kernel entry and\nexit boundaries for diagnosis.\nToken reduction composes cleanly with the validated `--layers 45 --reuse 2`\nsettings: on the same 512 benchmark it reduced that profile from 16.69 to\n12.60 seconds (24.5% marginal), and independent fox and surfer renders stayed\ncoherent. Do not combine it with both `--layers 40` and `--reuse 3`; that\n6.47-second experiment produced chromatic ringing and ghosted limbs despite\nacceptable latent norms.\n\n### Internal canvas and video VAE\n\n`--render-width` and `--render-height` run the model and VAE on a lower\nsame-aspect internal canvas, then high-quality vImage-scale RGB frames to the\nrequested output size before callbacks, terminal display, and encoding. This is an\nexplicit quality\u002Fspeed tradeoff: a measured 384-to-512 prompt render reduced\nM5 DiT time by 33% and video-VAE time by 18% while retaining a clean,\nrecognizable photorealistic result. Both values must be multiples of 32; the\nexact output canvas remains the default.\nFor square 512 output, 384 is the fast-quality point and 320 is the validated\naggressive point. The latter produced a coherent walking fox and repeated at\n8.02 seconds of DiT versus about 15.82 seconds natively. Native 256 uses the\nsame-cost spatial-RoPE adaptation described above; it remains a fast composition\npreview rather than a substitute for a 512- or 768-class final render.\nThe video VAE automatically chooses a 256-320 pixel spatial tile from the\nrequested canvas geometry, minimizing repeated overlap work while keeping peak\nstorage bounded. `H3_VAE_TILE_PIXELS=256` restores the original conservative\ntile plan for close-reference diagnosis.\n\n### Weight residency and streamed prompt encoding\n\nOn M5-class GPUs, persistent transformer weights are mapped directly from their\nsafetensor shards instead of copied into anonymous shared buffers. This keeps\nthe 37 GiB model file-backed\u002Freclaimable and slightly improves total transformer\ntime; M3 uses the faster copied-buffer path. `H3_ZERO_COPY_WEIGHTS=0` disables\nthe M5 selection for diagnostics.\nThe streamed Qwen text encoder preallocates a small ring of future layer\nbuffers and fills them on eight I\u002FO workers while Metal executes the current\nlayer. The default ring depth is two layers on M3\u002Folder hardware and three on\nM5, where the target machine has 128 GiB. `H3_QWEN_PREFETCH=0` restores the\nsingle-layer synchronous reference path; values 1-8 select the worker count,\nand `H3_QWEN_PREFETCH_DEPTH=1` through `6` overrides the ring depth.\n\n`--ssd-streaming` is a separate, more aggressive residency mode for the DiT.\nOnly its small per-block normalization weights remain resident. Two complete\nBF16 matrix slots alternate while a background reader fills the next slot in\ncheckpoint-offset order; the current Metal command buffer runs concurrently.\nDarwin uncached reads avoid retaining a second copy in the filesystem cache.\nThe first active block is prefetched again during the final block, so a cached\ninteractive DiT is ready for its next denoiser evaluation. Measurements reached\nabout 13--14.6 GiB\u002Fs from the internal SSD. `H3_PROFILE=1` reports total bytes,\nread throughput, and the part of the read wait that was not hidden by GPU work.\n\n### Metal 4 and TensorOps paths\n\nM5 GPUs automatically use native BF16 Metal 4\u002FTensorOps for the DiT QKV and\nattention-output projections at sequence lengths up to 2,048. The compact\nMorton schedule routes Q\u002FK\u002FV directly into head-major attention inputs, avoids\nthree MPSGraph input transposes, and is byte-identical to the portable path. It\nimproves a complete 512x512 50-block forward by about 2% across repeated IT\u002FUS\nM5 Max runs. For 2,049-3,072 rows, including 864x480, two row-offset Morton\ndispatches preserve the efficient tile geometry and improve the complete\nforward by about 2% in balanced runs. Still larger sequences stay on MPSGraph.\n`H3_NAX=0` disables TensorOps for exact A\u002FB diagnosis. The selection is guarded\nat runtime and falls back to the unchanged portable library if compilation is\nunavailable.\n\n`H3_NAX=1` forces the broader native BF16 linear path. It passes the complete\n50-block MLX fixture, but remains opt-in: exact-shape microbenchmarks favor its\n128-row tile while full DiT runs currently favor MPSGraph scheduling. This\nkeeps a working NAX integration available for later quantized\u002Ffused kernels\nwithout making a benchmark regression the default.\n`H3_NAX=mlp` selects a more specialized Metal 4 path: paired FC1 gate\u002Fup\nTensorOps tiles apply SwiGLU in threadgroup memory and write only the\n14,336-wide activated intermediate, then FC2 also stays on TensorOps.\n`H3_DISABLE_NAX_MLP=1` keeps the MPSGraph MLP in a context created this way for\nsame-process A\u002FB testing. The path is deliberately opt-in because scheduling\ndepends on the OS GPU stack: the primary macOS 26.5.2 M5 Max gained 1.3-2.0%\nin isolated real-weight MLP runs but lost about 1-3% in a complete 50-block forward,\nwhile an otherwise identical macOS 26.5 M5 Max gained 1.4% in a same-context\nforward A\u002FB. The resulting 50-block velocities were close (1.9% video and 2.4%\naudio relative L2), but not byte-identical.\n\n### Specialized projection kernels\n\nThe narrow DiT audio\u002Fvideo output heads convert their small released F32\nweights to BF16 once and use the Iris-derived 16x16 tiled linear directly on\nBF16 activations. At the production 320-render geometry, isolated paired-head\nmeasurements are 2.30x faster on M3 Max and 1.83x faster on M5 Max, with\nrelative L2 `8.64e-4`; the absolute M5 saving is about 0.6 ms per evaluated\nstep. Full fox and surfer sequences remained clean and measured 29.9\u002F38.4 dB\nagainst the F32-head renders. `H3_DIT_F32_FINAL=1` restores the close-reference\nhead and its extra activation buffers.\nThe F32 `96->5376` video and `32->5376` audio patch projections use a dedicated\n16x16 cooperative tile, retaining F32 weights, inputs and accumulation while\nrounding the tile result directly to BF16.\nPaired production-shape measurements are 1.77x faster on M3 and 1.62-1.78x\non M5; the complete generated RGB stream is byte-identical to the scalar path.\nFusing the final cast improves the 2835-row tile itself from 2.499 to 1.734 ms\non M3 and 1.555 to 1.186 ms on M5, and removes 38.27\u002F59.66 MiB of F32 scratch\nat 512\u002F864-class geometry. `H3_DISABLE_FUSED_PATCH_CAST=1` restores the tiled\nF32 output plus standalone cast; `H3_SCALAR_PATCH=1` selects the scalar\ndiagnostic path.\nThe same tile binds its output directly into the packed hidden stream, removing\nthe BF16 media staging buffers and their blits. This saves another 19.13\u002F29.83\nMiB and improves the 2835-row boundary from 1.847 to 1.730 ms on M3 and 1.282\nto 1.184 ms on M5. Contiguous T2VA uses byte offsets; FL2VA\u002FRef2VA use compact\ndestination-row maps so each modality remains one large dispatch. A complete\nsix-segment Ref2VA M5 ABBA remained byte-identical and improved 5.067 to 5.033\nseconds per measured forward pair. `H3_DISABLE_FUSED_PATCH_PACK=1` restores the\nstaging buffers and packing blits.\n\n### Scheduling and activation memory\n\nThe DiT core is split into two ordered Metal command buffers so GPU execution\nof the first part overlaps CPU encoding of the second. Thermal-balanced ABBA\nmeasurements select a 60%-depth split on M5 (30\u002F50, 27\u002F45, and 24\u002F40), with\nroughly 0.5-1.8% wins; M3 automatically splits only the validated 30\u002F50 case,\nwhich measured 1.2% faster, because 24\u002F40 regressed there. The operation order\nand generated bytes are unchanged. `H3_DIT_COMMAND_BLOCKS=0` restores one\ncommand buffer; values 1-50 override the split for further tuning.\nDiT activation buffers also follow their actual intra-block lifetimes: the QKV\nprojection arena is reused first for attention heads and then for the normalized\nMLP input, while the current attention-output arena becomes the MLP output after\nits branch has been consumed. This removes 61.25 MiB at 512-class geometry and\n99.63 MiB at 864-class geometry without changing dispatches or arithmetic.\n`H3_DISABLE_DIT_ACTIVATION_ALIAS=1` restores separate diagnostic buffers.\nMPSGraph tensor-data wrappers for immutable DiT weights and biases are retained\nwith their resident buffers. This avoids rebuilding the same binding metadata\nfor every block and denoiser evaluation without copying tensor storage; measured\nABBA gains were 1.6% on M3 Max and 0.4-1.1% on M5 Max. Activation wrappers stay\ntransient because retaining them regressed the M5. The outputs remain\nbyte-identical, and `H3_DISABLE_GRAPH_DATA_CACHE=1` restores transient wrappers\nfor all tensors.\nOn M3\u002Folder hardware, the four MPSGraph segments in each DiT block also reuse\none `MPSCommandBuffer` wrapper for their shared underlying Metal command buffer.\nRepeated thermal-balanced runs measured 1.0-1.6% faster on M3 Max; M5 measured\nneutral, so it retains fresh wrappers. `H3_REUSE_MPS_COMMAND=0` or `1` overrides\nthe automatic selection. Results are byte-identical.\nOn M5, the serving Euler sampler keeps its patch-packed F32 latents and cached\nBF16 velocities in Metal buffers. Each selected denoiser refresh is completed\nbefore the next is encoded, avoiding MPSGraph back-pressure while removing all\nintermediate latent\u002Fvelocity readbacks and repacking. Two warm eight-run A\u002FB\nsequences measured small 0.1% and 0.3% gains with byte-identical final latents;\nthe path also saves roughly 16 bytes of transient host state per video-latent\nelement (about 136 MB at the 768p shape). M3 and older GPUs retain the CPU\nsampler by default. `H3_CPU_SAMPLER=1` restores it on M5;\n`H3_GPU_SAMPLER=1` selects the GPU-state path explicitly, and\n`H3_GPU_SAMPLER_WINDOW=0` enables the slower unbounded encode-ahead diagnostic.\n\n### Checkpoint layout and media pipeline\n\nThe released checkpoint stores DiT QKV rows interleaved per attention head.\nNative Metal consumes that layout directly in the fused QK-normalization\u002FRoPE\nkernel, avoiding a checkpoint transpose and extra RAM. The earlier identity\ninterpretation was the cause of the noisy diagnostic outputs.\n\nThe public generation path decodes the joint audio latent with a streamed native\nBigVGAN\u002FAudioVAE and writes synchronized H.264 plus 32 kHz stereo AAC. The native\nwaveform agrees with the corrected MLX oracle to relative L2 `6.94e-5`.\n`--first-frame`, `--last-frame`, and their combination use the released visual\nVAE encoder, Qwen3-VL vision tower and three-deepstack multimodal presentation,\n0.999 condition augmentation, and fixed condition rows in the native DiT. The\nfirst image is stretched to the target canvas; the last image is aspect-cover\nscaled and center cropped, matching the reference implementation. `--ref-image`\nselects the distinct Ref2VA transformer, preserves ordered `\u003CPicture N>`\npresentation, and uses the released down-only aspect-preserving reference canvas.\n`--ref-silent-video` additionally performs bounded 24 fps decoding, the visual\nVAE's causal `ceil(T\u002F4)` compression, two-frame Qwen sampling, and timestamped\n`\u003CVideo N>` presentation. `--ref-video` preserves an embedded soundtrack,\n`--ref-video-audio VIDEO AUDIO` supplies an explicit replacement, and\n`--ref-audio` appends an ordered standalone clip. Reference audio is decoded as\n32 kHz stereo F32, encoded by the native AudioVAE posterior-mean path, mixed as\n0.999 clean latent plus 0.001 seeded noise, pinned to the audio condition\ntimestep 1.0, and packed as width-32 rows on the same rotary timeline as visual\nreferences. Audio inputs are 2-15 seconds, at most three are\naccepted, their total decoded duration is capped at 15 seconds, and a standalone\naudio reference must be combined with an image or video reference.\n\nThe native audio encoder matches the corrected MLX oracle at relative L2\n`3.59e-6` on a real two-second stereo fixture. The correction is important: the\noriginal MLX reshape interleaved left\u002Fright samples, whereas the official\nPyTorch\u002FSGLang path folds intact stereo channels into the batch dimension. On\nthe 128 GB M5 Max, clean end-to-end image+audio and embedded-video+audio renders\ncompleted in 74.58 and 76.99 seconds respectively, each with about a 40.1 GB\npeak physical footprint and zero swaps.\n\n### Profiling and diagnostic paths\n\n`--profile` reports each Metal-backed phase separately: wall time, CPU-side\ncommand encoding, complete commit-to-fence wait, root-command GPU timestamps,\npeak live tensor storage, cumulative allocation, and dispatch counts. The wait\nmeasurement is the complete command turnaround; the root GPU timestamp alone\ncan omit child buffers scheduled internally by MPSGraph and is labeled\naccordingly.\n\nThe DiT fast path evaluates each BF16 `fc1 -> SwiGLU -> fc2` block as one cached\ngraph, avoiding separate graph boundaries and persistent intermediate tensors.\nSet `H3_DISABLE_FUSED_MLP=1` to retain the close-reference operation boundaries\nfor numerical diagnosis.\n\nOn supported M5 Metal 4 TensorOps hardware, the native int8 MLP engine is the\ndefault. It dynamically quantizes activations, uses per-output-channel weight\nscales, and gives the sensitive FC2 input one scale per 1,024 channels.\nThe selected FC2 kernel keeps scaled partial products in private cooperative\nfragments instead of repeatedly spilling a 32 KiB threadgroup tile. A fixed\n50-layer, 19-transition 512x512 render measured 36.30 seconds with BF16 MPS and\n25.80 seconds with int8 on M5 Max. Beginning, middle, and final decoded frames\nretained the same subject, composition, and motion; small edge and fur details\ncan differ. The current diagnostic implementation retains both BF16 and int8\nMLP weights only when an A\u002FB diagnostic requests them. Normal int8 loading\nreleases each block's BF16 FC1\u002FFC2 buffers after their submitted quantization\nfinishes, reducing measured peak tensor storage to 25.9 GiB from the BF16\npath's 36.4 GiB. Runtime weight quantization still adds startup time.\n\nThe fastest M5 path also quantizes each DiT QKV projection and writes its\nQ\u002FK\u002FV tiles directly in head-major attention layout before the existing Q\u002FK\nnormalization and RoPE kernel. In a fixed 50-layer, 19-transition 512x512\nrender this reduced denoising again, from 25.80 to 19.32 seconds. Sampled\nbeginning, middle, and final frames remained a coherent detailed fox walking\nthrough snow; quantized attention can change framing and fine detail. Use\n`--use-slower-bf16-qkv` for the close-reference BF16 projection. Normal int8\nloading releases the redundant BF16 QKV weights after quantization.\n\nThe following attention-output projection is int8 as well on the default M5\npath. Crossed same-model tests improve a complete forward by another 4.5-5.5%\nat 512 and 864. A decoded fox render remained clean and closely matched the\nint8-QKV-only composition; its thermally hot denoise measured 19.18 seconds.\nUse `--use-slower-bf16-attention-output` to retain that projection in BF16.\n\nOn that int8 path, SDPA now leaves its result in native\n`[head,row,dimension]` order. A specialized 256-thread kernel gathers and\nquantizes each H3 row directly into the projection's row-major int8 buffer,\neliminating the intervening full-width BF16 transpose without changing any\noutput byte. Thermally controlled crossed runs improve complete 512 and 864\nforwards by roughly 0.2-1.2%. Use\n`--use-slower-row-major-attention-output` to restore the explicit BF16\nrow-major SDPA output and ordinary quantizer.\n\nThe M5 path also folds QKV and MLP activation quantization into the preceding\ngated AdaLN kernel. This removes 99 standalone quantizer dispatches per\n50-layer forward while preserving the previous output bytes, improving crossed\n512\u002F864 measurements by about 0.3-0.6%. Use\n`--use-slower-unfused-int8-inputs` to restore the standalone quantizers.\n\nThe fused gated-AdaLN path loads its full 5,376-wide H3 rows as BF16x4 vectors\nand writes int8x4. It stages the rounded values locally before computing the\noriginal per-thread RMS sequence, so the reduction tree and every output byte\nremain unchanged. Crossed measurements save roughly another 0.1-0.5%. The\nexisting `--use-slower-unfused-int8-inputs` option retains the portable scalar\nand standalone-quantizer fallback.\n\nQ\u002FK RMS normalization and RoPE are performed inside the int8 QKV projection\ntile as well. The fused epilogue is byte-identical and improves complete\nforwards by 2.1-3.2% at 512 and 1.0-1.8% at 864 in crossed M5 measurements.\nUse `--use-slower-unfused-qkv-rope` to restore the separate Q\u002FK kernel.\n\nThat epilogue processes four adjacent Q\u002FK dimensions per work item with\nBF16x4 loads and stores. The per-element arithmetic and BF16 rounding order are\nunchanged, while crossed cool-state measurements improve complete forwards by\nabout 0.4-1.0% at both 512 and 864. The same\n`--use-slower-unfused-qkv-rope` option restores the scalar standalone path.\n\nAt up to 2,048 rows, the exact RMS loop uses BF16x4 loads followed by four\nexplicit ordered FMAs. This preserves every output bit and improves 512-class\nforwards by another 0.5-0.6%; larger shapes retain scalar loads because the two\nforms tie there. Use `--use-slower-scalar-qkv-rms` to force scalar loads.\n\nThe int8 attention-output projection caches its 128 row and column scales in\n1 KiB of threadgroup memory instead of rereading them for every cooperative\nfragment element. Above 2,048 rows the fused QKV kernel uses the same idea and\nthen recycles that storage for inverse RMS values; smaller QKV shapes retain\ndirect loads because the two forms tie there. Both are byte-identical and\nimprove complete forwards by about 0.2-0.7% where selected. Use\n`--use-slower-uncached-int8-scales` to restore direct device-scale loads.\n\nFor sequences of at most 2,048 rows, the H3 attention-output projection also\ncompiles its 7,168-by-5,376 shape into the TensorOps kernel. The result remains\nbyte-identical while saving about 0.2-0.8% in crossed complete 512-forward\nmeasurements. Larger sequences retain the dynamic-shape kernel because the\nspecialization regresses there. `--use-slower-uncached-int8-scales` restores\nthe general dynamic, direct-scale-load implementation.\n\nFC1 also uses an H3-specialized, compile-time 5,376-wide TensorOps loop. It is\nbyte-identical to the generic loop and saves about 0.1-0.4% in crossed complete\nforwards. Use `--use-slower-dynamic-fc1-k` to restore the runtime-bound loop.\n\n```sh\n.\u002Fh3 --profile -d .\u002FMiniMax-H3 \\\n  -p \"A red fox walks through fresh snow.\" \\\n  --width 512 --height 512 --frames 22 --steps 20 \\\n  --layers 50 --reuse 1 -o outputs\u002Ffox-int8.mp4\n```\n\nUse `--use-slower-bf16-mlp` to force the portable close-reference MPS\u002FBF16 MLP\npath for numerical comparison. Older Metal hardware selects that path\nautomatically when the required native TensorOps kernels are unavailable.\nFor FC2 activation quantization, sequences of at most 2,048 rows use an exact\n128-thread reduction. Each thread retains its eight BF16 input values while\ncomputing the group maximum, avoiding a second device-memory read when it emits\nthe int8 values; crossed M5 measurements improved complete 512 forwards by\nabout 0.2-0.8% without changing any output byte. Larger sequences retain the\nmeasured 256-thread kernel. `--use-slower-grouped-quantizer` forces the latter\nat every size for A\u002FB comparison.\n\nThe native baseline targets the original `FL2VA\u002F` and `Ref2VA\u002F` checkpoint\ntrees. Model phases are loaded and released separately so the 33B transformer,\nQwen encoder, and decoders never have to coexist in unified memory.\n","h3.c 是一个专为 Apple Silicon（M-series）Mac 设计的轻量级 MiniMax H3 多模态生成模型推理引擎，基于 C 语言与 Metal 框架实现。核心功能包括 prompt-to-video\u002Faudio 生成、首尾帧条件控制（first\u002Flast-frame conditioning）及有序图像\u002F视频\u002F音频参考注入（Ref2VA），支持 BF16 精度、内存复用与交互式会话缓存。技术特点涵盖 Metal 原生加速、确定性元数据加载、分阶段 denoising 优化及终端实时中间帧预览。适用于本地化、低延迟的多模态内容创作场景，如快速视频草稿生成、教育演示素材制作或创意工作流原型验证。",2,"2026-08-12 02:30:03","CREATED_QUERY"]