[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-93063":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":14,"stars30d":12,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":15,"rankGlobal":9,"rankLanguage":9,"license":16,"archived":17,"fork":17,"defaultBranch":18,"hasWiki":19,"hasPages":17,"topics":20,"createdAt":9,"pushedAt":9,"updatedAt":21,"readmeContent":22,"aiSummary":23,"trendingCount":14,"starSnapshotCount":14,"syncStatus":13,"lastSyncTime":24,"discoverSource":25},93063,"dfine-cpp","PogChamper\u002Fdfine-cpp","PogChamper","C++\u002FTensorRT inference runtime for the D-FINE object detector: explicit-deform export (+10.5 AP on TRT), strongly-typed FP16, single-cudaGraphLaunch frozen pipeline, C ABI + Python bindings",null,"Python",69,13,2,0,44.74,"Apache License 2.0",false,"main",true,[],"2026-07-22 04:02:08","# D-FINE-cpp\n\n**Production C++\u002FTensorRT inference for the [D-FINE](https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE) object detector.**\nA zero-Python, OpenCV-free runtime that runs the full pipeline — CUDA preprocessing → TensorRT engine →\nC++ decode — at up to **1550+ FPS** (D-FINE-N) and **2.47 ms** end-to-end batch-1 latency on an\nRTX 4070 Ti SUPER, matching PyTorch mAP to within **0.002 AP** in the default configuration.\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"#quickstart\">Quickstart\u003C\u002Fa> ·\n  \u003Ca href=\"#supported-hardware--prerequisites\">Supported hardware\u003C\u002Fa> ·\n  \u003Ca href=\"#benchmarks\">Benchmarks\u003C\u002Fa> ·\n  \u003Ca href=\"#using-the-library\">C++ API\u003C\u002Fa> ·\n  \u003Ca href=\"#from-python\">Python\u003C\u002Fa> ·\n  \u003Ca href=\"#precision-guide\">Precision\u003C\u002Fa> ·\n  \u003Ca href=\"docs\u002FRESEARCH_MATRIX.md\">Research matrix\u003C\u002Fa> ·\n  \u003Ca href=\"docs\u002FROADMAP.md\">Roadmap\u003C\u002Fa>\n\u003C\u002Fp>\n\n[![CI](https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Factions\u002Fworkflows\u002Fci.yml\u002Fbadge.svg?branch=main)](https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Factions\u002Fworkflows\u002Fci.yml)\n[![Release](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fv\u002Frelease\u002FPogChamper\u002Fdfine-cpp)](https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases)\n[![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-Apache--2.0-blue.svg)](LICENSE)\n![C++17](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FC%2B%2B-17-blue.svg?logo=cplusplus&logoColor=white)\n![TensorRT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensorRT-10.x-76B900?logo=nvidia&logoColor=white)\n![CUDA](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FCUDA-12.x-76B900?logo=nvidia&logoColor=white)\n\n![PyTorch vs D-FINE-cpp throughput](assets\u002Fdemo_video.gif)\n*All three panels race through the same clip at 12× slow motion; frame counters advance at each\nbackend's measured batch-8 throughput (D-FINE-M, RTX 4070 Ti SUPER). Detections are identical —\nthe C++ surgical-FP16 runtime produced every panel. Reproduce:\n`trt-files\u002Fscripts\u002Fmake_demo_gif.py` (clip: Mixkit free license).*\n\nAll five model sizes, three precision\u002Fspeed tiers, end-to-end (preprocess + inference + decode),\nmAP on full COCO val2017, throughput = batch-8 img\u002Fs (medians of 3×500-iter rounds), RTX 4070 Ti SUPER.\n**prod fp16** = the v0.2.0 production tier (decoder kept FP32); **surgical** = FP16 including the\ndecoder (`convert_fp16_surgical.py`; the shipped `--slim` variant trims the FP32 island further);\n**fast** = slim + export sliders (`--num-queries 200 --cascade 1:100`):\n\n| model | PyTorch mAP | prod fp16 b8 · mAP | **surgical b8 · mAP** | fast b8 · mAP |\n|---|---|---|---|---|\n| D-FINE-N | 0.4279 | 1234 · 0.4280 | **1309** · 0.4276 | **1556** · 0.4231 |\n| D-FINE-S | 0.5073 | 637 · 0.5069 | **758** · 0.5065 | **880** · 0.5021 |\n| D-FINE-M | 0.5509 | 469 · 0.5500 | **526** · 0.5502 | **598** · 0.5448 |\n| D-FINE-L | 0.5724 | 357 · 0.5723 | **390** · 0.5724 | **453** · 0.5647 |\n| D-FINE-X | 0.5931 | 244 · 0.5927 | **264** · 0.5929 | **292** · 0.5855 |\n\nThe release ONNX is the **`--slim`** surgical variant — separately gated lossless on all five\nsizes (full-val n 0.4272 \u002F s 0.5060 \u002F m 0.5500 \u002F l 0.5723 \u002F x 0.5926), +2-3% faster at batch 8;\nthe surgical column above is the conservative non-slim measurement. The m ladder tops out at\n**686 img\u002Fs — 10.4× PyTorch** (the `max` preset: fast + `--eval-idx 2` + `--opt-batch 8`, −0.89 AP);\nevery cell (b1-b8, VRAM, and the closed negatives: FP8, INT8, plugins) is in\n**[docs\u002FRESEARCH_MATRIX.md](docs\u002FRESEARCH_MATRIX.md)** with one-command reproduce paths.\nPython users: [three commands to first detection](#python--three-commands-no-checkout).\n\n---\n\n## Why\n\nD-FINE is a state-of-the-art real-time detector, but getting it to run *fast and correct* on TensorRT is\nnon-trivial — a naive export silently loses ~10 AP, and naive FP16 loses another ~7. This library solves both\nand ships a lean, dependency-light C++ runtime:\n\n- **Correct on TensorRT.** The stock `grid_sample` deformable-attention export costs **−10.5 AP** on TRT\n  (full-val 0.5507 → 0.4455) — TRT compiles it divergently in-context. We export an **explicit gather-bilinear**\n  core (same math, `Gather`+arithmetic) that is TRT-exact with **no plugin and no latency cost**.\n- **FP16 that actually holds mAP.** Naive `kFP16` costs a fixed **−6.8 AP** on D-FINE regardless of layer\n  pinning — the *builder flag itself* leaks lossy reformats onto the FDR box-decode. The fix is **strong\n  typing** (precision baked into ONNX types): **1.6–2.2× faster at −0.2% mAP**, validated on all five sizes.\n- **Lean runtime.** `libdfine.so` takes a raw `ImageU8` (HWC uint8) — **no OpenCV** in the core — and hides\n  all TensorRT\u002FCUDA behind a PIMPL. RAII everywhere, sanitizer-clean, `-Werror` clean.\n\n## Quickstart\n\nTwo paths to the same runtime. Both start from the release assets: prebuilt ONNX for all five\nsizes — the FP32 opset-19 base (`dfine_\u003Csize>_op19`) and the surgical-FP16 production build\n(`dfine_\u003Csize>_slim`), each with its `.json` sidecar, plus `SHA256SUMS` — attached to\n[GitHub Releases](https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases). What the names and\nsidecars mean — and which one is the source of truth — is spelled out in\n[docs\u002FNAMING.md](docs\u002FNAMING.md). Running on a GPU we haven't validated? A\n[docs\u002FVALIDATION.md](docs\u002FVALIDATION.md) report takes ten minutes and earns a\nrow in the compatibility table.\n\n### Python — three commands, no checkout\n\nNo compiler, no OpenCV, no repo clone — the wheel bundles the C++ runtime and an engine-build\nscript; the engine compiles on your GPU in one command (one-time, 1-3 min). Wheel = sm_89 \u002F\nlinux_x86_64 (see [Supported hardware](#supported-hardware--prerequisites)); other GPUs build from\nsource first, then everything below is identical.\n\n```sh\npip install \"dfine[cli] @ https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases\u002Fdownload\u002Fv0.3.2\u002Fdfine-0.3.2-py3-none-linux_x86_64.whl\" \"tensorrt-cu12==10.13.*\"\ncurl -LO https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases\u002Fdownload\u002Fv0.3.2\u002Fdfine_m_slim.onnx \\\n     -LO https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases\u002Fdownload\u002Fv0.3.2\u002Fdfine_m_slim.json\ndfine build --model m --onnx dfine_m_slim.onnx --output dfine_m_slim.engine\n```\n\n```python\nfrom dfine import Detector\n\ndet = Detector(\"dfine_m_slim.engine\", threshold=0.5)\nfor d in det.detect(frame):                  # frame: numpy HWC uint8 (RGB)\n    print(d.class_name, round(d.score, 3), d.box.as_tuple())\n```\n\nExpected output (COCO `000000039769.jpg` — the two-cats picture):\n\n```\ncat 0.961 (11.9, 53.6, 316.9, 472.9)\ncouch 0.955 (0.6, 2.1, 639.4, 474.1)\ncat 0.949 (346.4, 24.1, 639.8, 371.7)\nremote 0.928 (40.1, 73.1, 175.8, 117.7)\nremote 0.883 (333.1, 76.6, 370.6, 187.8)\n```\n\nThat engine is the lossless production build (full-val 0.5500, exactly the FP16 reference; its\ntier measures 288 img\u002Fs b1 \u002F 526 img\u002Fs b8 on a 4070 Ti SUPER — PyTorch does 31 \u002F 66). Notebook\nversion: [examples\u002Fpython_quickstart.ipynb](examples\u002Fpython_quickstart.ipynb); one-shot CLI:\n`dfine predict --engine dfine_m_slim.engine --image dog.jpg --out annotated.jpg`.\n\n### C++ — build, engine, run\n\n```sh\n# 1. Build (see Supported hardware for CUDA_ARCH values; 'native' probes the local GPU).\n#    Building needs TensorRT HEADERS — `pip install tensorrt-cu12` is runtime-only;\n#    build.sh checks first and prints the fix (details: docs\u002FTROUBLESHOOTING.md)\n.\u002Fbuild.sh                                              # or: CUDA_ARCH=86 .\u002Fbuild.sh\n# — or plain CMake:\ncmake -B build -S . -DCMAKE_CUDA_ARCHITECTURES=native   # [-DTENSORRT_DIR=\u002Fpath\u002Fto\u002Ftensorrt]\ncmake --build build -j\n\n# 2. Get the prebuilt ONNX + sidecar, compile the engine on your GPU\ncurl -LO https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases\u002Fdownload\u002Fv0.3.2\u002Fdfine_m_slim.onnx\ncurl -LO https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases\u002Fdownload\u002Fv0.3.2\u002Fdfine_m_slim.json\npython trt-files\u002Fscripts\u002Fbuild_engine.py --strongly-typed --no-tf32 --max-batch 8 \\\n    --onnx dfine_m_slim.onnx --output trt-files\u002Fengines\u002Fdfine_m_slim.engine\n#   (add --opt-batch 8 for batch serving: +6-10% b8 throughput, costs some b1 latency)\n\n# 3. Run — libnvinfer\u002Flibcudart must be on LD_LIBRARY_PATH; any TensorRT 10.x libs work,\n#    e.g. `python -m pip install \"tensorrt-cu12==10.13.*\"` then the venv's ...\u002Fsite-packages\u002Ftensorrt_libs\nexport LD_LIBRARY_PATH=\u002Fpath\u002Fto\u002Ftensorrt\u002Flib:$LD_LIBRARY_PATH\n.\u002Fbuild\u002Fdfine_detect --engine trt-files\u002Fengines\u002Fdfine_m_slim.engine \\\n    --image 000000039769.jpg --threshold 0.5\n```\n\nExpected output (excerpt — the timing line of this one-shot tool includes engine warm-up;\nsteady-state numbers come from `dfine_bench`):\n\n```\nengine: trt-files\u002Fengines\u002Fdfine_m_slim.engine  variant=m  input=640x640  queries=300  classes=80\n5 detections (thr=0.50)\n  cat              0.961  [11.9, 53.6, 316.9, 472.9]\n  couch            0.955  [0.6, 2.1, 639.4, 474.1]\n  cat              0.949  [346.4, 24.1, 639.8, 371.7]\n  remote           0.928  [40.1, 73.1, 175.8, 117.7]\n  remote           0.883  [333.1, 76.6, 370.6, 187.8]\n```\n\nFor an FP32 engine, drop `--strongly-typed` and point `--onnx` at the base file\n(`dfine_m_op19.onnx`). The v0.2.0 `fp16_st` assets (decoder kept FP32) remain on the\n[v0.2.0 release](https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases\u002Ftag\u002Fv0.2.0) and are still a\nvalid, slightly slower build.\n\n### Using dfine from your own CMake project\n\nThe library installs as a config package — no copying headers around:\n\n```sh\ncmake --install build --prefix \u002Fopt\u002Fdfine        # after the build above\n```\n\n```cmake\nfind_package(dfine CONFIG REQUIRED)               # -DCMAKE_PREFIX_PATH=\u002Fopt\u002Fdfine\ntarget_link_libraries(your_app PRIVATE dfine::dfine)\n```\n\nThe public headers are TensorRT\u002FCUDA-free (PIMPL), so your project needs no\nTensorRT include paths — only the TRT\u002FCUDA libraries at link and run time (the\nshipped config resolves them via `find_dependency`). A complete out-of-tree\nconsumer lives in [examples\u002Fconsumer](examples\u002Fconsumer\u002FCMakeLists.txt) and is\nbuilt by CI against every commit.\n\n### Docker\n\n```sh\ndocker build --build-arg CUDA_ARCH=89 -t dfine-cpp .    # 86 = RTX 30xx, 89 = RTX 40xx\ndocker run --rm --gpus all -v \u002Fpath\u002Fto\u002Fengines:\u002Fengines dfine-cpp \\\n    dfine_detect --engine \u002Fengines\u002Fdfine_m_slim.engine --image dog.jpg\n```\n\nThe image builds and runs the C++ consumer of an already-built `.engine`; ONNX export is not included\n(it needs the D-FINE-seg source — see the Dockerfile header for the exact limitations).\n\n### Export from a checkpoint\n\nTo build the ONNX yourself (fine-tuned weights, non-COCO class counts), you need the training\u002Fexport source\nrepo [D-FINE-seg](https:\u002F\u002Fgithub.com\u002FArgoHA\u002FD-FINE-seg) — for this step only; `build_engine` \u002F\n`convert_fp16` \u002F `profile` \u002F `coco_eval` and the C++ runtime are self-contained.\n\n```sh\ngit clone https:\u002F\u002Fgithub.com\u002FArgoHA\u002FD-FINE-seg                       # export-time dependency only\npython trt-files\u002Fscripts\u002Fexport_dfine_onnx.py --model-name m --opset 19 \\\n    --checkpoint dfine_m_obj2coco.pt --dfine-src .\u002FD-FINE-seg \\\n    --output trt-files\u002Fonnx\u002Fdfine_m_op19.onnx                        # FP32 base + .json sidecar\npython trt-files\u002Fscripts\u002Fconvert_fp16_surgical.py --slim \\\n    --onnx trt-files\u002Fonnx\u002Fdfine_m_op19.onnx \\\n    --output trt-files\u002Fonnx\u002Fdfine_m_slim.onnx                        # production surgical FP16\n```\n\n`--opset 19` matters: the surgical converter refuses opset-16 exports (their decomposed LayerNorm\nis miscompiled by TensorRT in FP16 — see the [Precision guide](#precision-guide)). The export also\ntakes accuracy\u002Fspeed sliders (`--num-queries`, `--eval-idx`, `--cascade K:KEEP`) that shrink the\ngraph itself — the `fast` column in the hero table is `--num-queries 200 --cascade 1:100`; the\ncost\u002Fgain of every slider is tabulated in [docs\u002FRESEARCH_MATRIX.md](docs\u002FRESEARCH_MATRIX.md).\nThe older `convert_fp16.py` (decoder kept FP32) remains available as the legacy opset-16 tier\n(`dfine export --precision fp16-legacy`).\n\nStandard checkpoints (`dfine_\u003Csize>_\u003Cdataset>.pt`) can be fetched from Hugging Face with D-FINE-seg's\n`ensure_pretrained` helper (`src\u002Fd_fine\u002Futils.py` in that repo); nano has no obj2coco checkpoint — use\n`dfine_n_coco.pt`. Any other checkpoint: pass its path to `--checkpoint` (the loader is strict — a\nclass-count or variant mismatch stops the export with a hint instead of exporting random weights;\npass `--num-classes`\u002F`--class-names` for custom label sets). The `dfine export` CLI (below) runs this\nexact recipe: `--precision fp16` = opset-19 export + surgical `--slim`.\n\n## Supported hardware & prerequisites\n\n**Validated:** Linux x86_64 (native + WSL2) · **Ada 89 \u002F Ampere 86 \u002F Blackwell 120** ·\nTensorRT 10.13. README numbers are the Ada reference; the Ampere (RTX 3090) and Blackwell\n(RTX 5080) full-methodology runs live in [docs\u002FVALIDATION.md](docs\u002FVALIDATION.md).\n**Expected (not yet validated):** Turing, Jetson Orin. **Not supported:** Windows native\n(use WSL2), macOS. The release wheel bundles `libdfine.so` built for **sm_89** — it runs\nas-is on newer arches via PTX JIT (verified on RTX 5080 with identical detections); older\nGPUs are one `.\u002Fbuild.sh` from source, after which every command works identically.\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>Full platform \u002F GPU \u002F dependency tables\u003C\u002Fb> (statuses, CUDA_ARCH values, versions)\u003C\u002Fsummary>\n\n**Platforms** (three statuses: *validated* = benchmarked\u002FCI here; *expected* = no known blocker,\nnot benchmarked here; *not supported* = known blocker):\n\n| Platform | Status |\n|---|---|\n| Linux x86_64 (Ubuntu 22.04+) | ✅ **validated** — native Ubuntu 22.04 (the Ampere\u002FBlackwell full runs) and the WSL2 guest below + CI builds on ubuntu-latest |\n| **WSL2** (Windows 11, Ubuntu guest) | ✅ **validated** — the development & benchmark environment itself |\n| Other Linux x86_64 distros | expected (plain CMake + CUDA 12 + TRT 10 stack) |\n| Jetson Orin (aarch64, JetPack ≥ 6.1) | expected, **not validated** — JetPack 6.1+ ships TensorRT 10.3 + CUDA 12.6; the runtime is plain CUDA + TRT with no x86-specific code. Note: the `tensorrt` pip wheel does not exist for Jetson — use JetPack's TensorRT for both build and run |\n| Windows native | ❌ not supported (Linux-only build scripts and `.so` packaging) — use WSL2 |\n| macOS | ❌ n\u002Fa (no NVIDIA\u002FCUDA) |\n\n**GPU** — any GPU TensorRT 10.x supports (compute capability **7.5 \u002F Turing or newer**, per\nNVIDIA's TRT support matrix); this project is validated on **8.9 (Ada)**:\n\n| GPU family | `CUDA_ARCH` | Status \u002F notes |\n|---|:---:|---|\n| RTX 40xx (Ada) | `89` | ✅ validated (RTX 4070 Ti SUPER — the reference machine) |\n| RTX 30xx (Ampere) | `86` | ✅ validated (RTX 3090: full tier ladder, lossless slim — [docs\u002FVALIDATION.md](docs\u002FVALIDATION.md)) |\n| RTX 20xx \u002F T4 (Turing) | `75` | expected (TRT 10 floor; FP16 tensor cores present) |\n| RTX 50xx (Blackwell) | `120` | ✅ validated (RTX 5080; building needs driver ≥ 570 + CUDA ≥ 12.8 — the release wheel itself runs via PTX JIT) |\n| Jetson Orin | `87` | expected, not validated (see platform row) |\n\n`.\u002Fbuild.sh` targets your local GPU by default (`CUDA_ARCH=native`); pass one value\n(`CUDA_ARCH=86 .\u002Fbuild.sh`) for headless\u002FCI builds or a semicolon list (`CUDA_ARCH=\"86;89\"`) for a\nfat binary.\n\n**Dependencies** — what you need and *when* you need it:\n\n| Dependency | Version | Needed for |\n|---|---|---|\n| NVIDIA driver | CUDA-12-capable (R525+; RTX 50xx needs R570+\u002FCUDA 12.8 stack) | build + run |\n| CUDA toolkit (`nvcc`) | 12.x (sm_120 needs ≥ 12.8) | build only |\n| TensorRT | **10.x** — validated on 10.13; any 10.x ≥ your GPU's floor works, e.g. `pip install \"tensorrt-cu12==10.13.*\"` as a lib source. **TensorRT 11 (2026) is not yet validated** — it makes strongly-typed networks the default (exactly this repo's approach), but nothing here has been re-gated on it | build + run |\n| CMake | ≥ 3.24 for `native` arch \u002F ≥ 3.20 with explicit arch | build only |\n| C++ compiler | C++17 — any GCC\u002FClang your CUDA 12.x `nvcc` accepts (CUDA 12.9: GCC ≤ 14, Clang ≤ 19) | build only |\n| Python | 3.9+ — engine build & ONNX export scripts only, **never at inference** | tooling only |\n| OpenCV | — | **not required, ever** |\n\n**Prebuilt binaries:** the release wheel bundles `libdfine.so` for **sm_89 \u002F linux_x86_64 only**;\nevery other GPU\u002Fplatform builds from source (one `.\u002Fbuild.sh`) — the Python package, CLI, and every\ncommand below then work identically. A system TensorRT is found automatically\n([cmake\u002FFindTensorRT.cmake](cmake\u002FFindTensorRT.cmake) searches `$TENSORRT_DIR`,\n`\u002Fusr\u002Flocal\u002FTensorRT`, `\u002Fopt\u002Ftensorrt`, `\u002Fusr`); alternatively populate `third_party\u002Ftensorrt`\n([third_party\u002FREADME.md](third_party\u002FREADME.md)). Engines are machine-specific build outputs —\nalways compiled locally from the release ONNX (that is why we ship ONNX, not engines).\n\n\u003C\u002Fdetails>\n\n## Benchmarks\n\n![Throughput over COCO stills](assets\u002Fdemo.gif)\n*D-FINE-N over COCO val2017 stills at 30× slow motion: the `fast`-preset engine (1556 img\u002Fs b8)\nagainst PyTorch FP32 (191 img\u002Fs b8, measured with `profile.py --backends torch --model-name n`);\nidentical detections both panels.*\n\nThe full m ladder, batch-8: **PyTorch 66 → C++ fp32 230 → fp16 469 → surgical 526 (561 with\n`--opt-batch 8`) → fast 598 → max 686 img\u002Fs (10.4×)**; at the latency end, the full-pipeline CUDA\ngraph holds **2.47 ms** end-to-end batch-1 with byte-identical detections. Per-batch curves, VRAM,\nand every intermediate point: [docs\u002FRESEARCH_MATRIX.md](docs\u002FRESEARCH_MATRIX.md).\n\nThe same ladder across generations (m surgical, b8): **RTX 3090 487 · 4070 Ti SUPER 526 ·\nRTX 5080 676 img\u002Fs** — full-methodology tables in [docs\u002FVALIDATION.md](docs\u002FVALIDATION.md).\nBatch-1 numbers here carry a WSL2 dispatch tax the native-Linux runs don't: the RTX 5080\nholds n-surgical at **0.91 ms** end to end.\n\nRTX 4070 Ti SUPER · COCO val2017 (5000 imgs) · D-FINE-M · latency = e2e p50 ms — the v0.2.0\n`profile.py` session, a couple of percent below the hero table's newer medians.\n\n| backend | e2e (b1) | **FPS b1** | **FPS b8** | GPU MiB | mAP |\n|---|---|---|---|---|---|\n| PyTorch (FP32) | 32.0 | 31 | 66 | — | 0.5509 |\n| ONNXRuntime-GPU (FP32) | 25.0 | 40 | 89 | — | 0.5509 |\n| TensorRT FP32 (Python) | 8.0 | 125 | 160 | 640 | 0.5507 |\n| **C++ FP32** | 5.7 | 176 | 227 | 642 | 0.5506 |\n| **C++ FP16** | **3.7** | **272** | **459** | **488** | 0.5500 |\n\nFP16 (strongly typed) is lossless to ≤0.0006 AP on every size at 1.3–2.8× the FP32 throughput; per-size\nFP32 columns and batch scaling are in the [nightly report](trt-files\u002Fscripts\u002Fovernight_bench.sh) output.\n\n**Frozen pipeline (opt-in), for steady-state streaming:** `gpu_decode` decodes on the GPU so only\nsurviving detections cross PCIe; `freeze()` \u002F `FreezeSpec` locks the memory footprint (zero steady-state\nallocation, VRAM Δ = +0 B over full runs); `full_pipeline_graph` captures H2D → preprocess → infer →\ndecode → D2H into **one `cudaGraphLaunch` per frame** (needs a `--max-aux-streams 0` engine and a fixed\nsource resolution; byte-identical to the split path, validated on 1061 real 640×480 COCO images).\nMeasured on m FP16: batch-1 e2e wall **−34.3%**, CPU per frame **4.30 → 0.195 ms** (dispatch\n4.18 → 0.12 ms); batch-8 frees 19.2 ms of CPU per call. The score threshold stays a live per-call knob\ninside the captured graph.\n\n```cpp\ndfine::DetectorOptions o;\no.full_pipeline_graph = true;                     \u002F\u002F implies gpu_decode\ndfine::DFineDetector det(\"dfine_m_slim.engine\", o);\ndet.freeze(dfine::FreezeSpec{1, 1920, 1080});     \u002F\u002F batch, source WxH — captures + locks\ndet.detect(frame);                                \u002F\u002F one cudaGraphLaunch per call\n```\n\n`last_timings()` reports per-stage CPU cost — the `dispatch_ms` column is what the graph collapses.\nExact contract: [include\u002Fdfine\u002Ftasks\u002Fdetector.hpp](include\u002Fdfine\u002Ftasks\u002Fdetector.hpp).\n\n### Reproduce\n\n```sh\n# PyTorch vs ONNXRuntime-GPU vs TensorRT vs C++ side by side (latency, FPS, GPU mem, mAP):\npython trt-files\u002Fscripts\u002Fprofile.py --backends torch onnx trt cpp cpp-graph \\\n       --engine trt-files\u002Fengines\u002Fdfine_m_fp16_st.engine --full --batches 1 8\nbash trt-files\u002Fscripts\u002Fovernight_bench.sh    # the full n→x sweep behind the tables above\n.\u002Fbuild\u002Fdfine_bench --pipeline-compare ...   # per-stage CPU cost, split vs full graph\n```\n\n(The `torch` backend and `.pt`→ONNX export need the\n[D-FINE-seg](https:\u002F\u002Fgithub.com\u002FArgoHA\u002FD-FINE-seg) source on `PYTHONPATH`; everything else is\nself-contained.)\n\n## Using the library\n\n```cpp\n#include \"dfine\u002Ftasks\u002Fdetector.hpp\"\n\ndfine::DFineDetector det(\"dfine_m_slim.engine\");             \u002F\u002F loads engine + .json sidecar\ndfine::ImageU8 img{data, height, width, 3, width * 3, \u002F*bgr=*\u002Ffalse};   \u002F\u002F your HWC uint8 buffer\nfor (const dfine::Detection& d : det.detect(img, \u002F*threshold=*\u002F0.5f)) {\n    \u002F\u002F d.box (xyxy pixel space), d.class_id (0..79 COCO), d.score\n}\n```\n\n`detect_batch(std::vector\u003CImageU8>)` runs dynamic batch (N=1..8). `DetectorOptions.use_cuda_graph` enables\nCUDA-graph replay (needs a `--max-aux-streams 0` engine). Link `dfine::dfine`; no OpenCV required.\n\n### From C (stable ABI)\n\n`libdfine.so` exposes a pure-C ABI ([`include\u002Fdfine\u002Fc_api.h`](include\u002Fdfine\u002Fc_api.h), built by default;\n`-DDFINE_BUILD_C_API=OFF` to skip) — opaque handle, no exceptions cross the boundary, thread-local\n`dfine_last_error()`, heap result sets freed by `dfine_detections_free()`. It's the foundation for FFI from\nany language:\n\n```c\ndfine_detector_t* det = dfine_detector_create(\"dfine_m_slim.engine\", NULL);\ndfine_detections_t* r = dfine_detector_detect(det, rgb, w, h, w*3, 3, \u002F*is_bgr=*\u002F0, 0.5f);\nfor (int i = 0; i \u003C r->count; ++i)\n    printf(\"%s %.2f\\n\", dfine_class_name(r->detections[i].class_id), r->detections[i].score);\ndfine_detections_free(r);\ndfine_detector_destroy(det);\n```\n\n### From Python\n\nA dependency-light [`dfine`](python\u002F) package wraps the C ABI via `ctypes` (no compile step; loads the\nprebuilt `.so`). See [`python\u002FREADME.md`](python\u002FREADME.md).\n\n```sh\npip install \"dfine @ https:\u002F\u002Fgithub.com\u002FPogChamper\u002Fdfine-cpp\u002Freleases\u002Fdownload\u002Fv0.3.2\u002Fdfine-0.3.2-py3-none-linux_x86_64.whl\" \"tensorrt-cu12==10.13.*\"\n```\n\nThe wheel bundles `libdfine.so` built for sm_89 \u002F linux_x86_64 plus a snapshot of\n`build_engine.py`, so `dfine build` works without a repo checkout; other GPUs and platforms build\nfrom source (`.\u002Fbuild.sh`, then `pip install -e python\u002F`).\n\n```python\nimport numpy as np\nfrom dfine import Detector\n\nwith Detector(\"dfine_m_slim.engine\", threshold=0.4) as det:\n    for d in det.detect(rgb_hwc_uint8):        # numpy HWC uint8 (is_bgr=True for BGR)\n        print(d.class_name, d.score, d.box.as_tuple())\n```\n\nThe C result set is freed after every call; `with`\u002F`__del__` release the engine. `detect_batch()` returns\nper-image results. Detections are **byte-identical to the C++ `dfine_detect`** (verified by `pytest`).\n\n### Zero-setup CLI\n\nThe package installs a `dfine` command that resolves an engine from `--engine`, an on-disk cache\n(`~\u002F.cache\u002Fdfine`, keyed by the source ONNX's content hash + batch profile + GPU arch + TRT version,\nso a stale engine can never shadow a fresh export), the dev-tree, or builds one on demand:\n\n```sh\ndfine predict --model m --image dog.jpg --threshold 0.5 --out annotated.jpg   # detect + draw\ndfine info    --model m                                                        # introspection\ndfine build   --model m --precision fp16                                       # ONNX -> .engine (cached)\ndfine export  --model m --precision fp16                                       # .pt -> surgical\u002Fslim ONNX\ndfine bench   --model m --batches 1,2,4,8                                       # latency\u002Fthroughput\ndfine doctor                                                                    # environment diagnostics\n```\n\n`dfine export --precision fp16` produces the production surgical\u002Fslim tier (the hero-table numbers);\nthe v0.2 decoder-FP32 tier remains available as `--precision fp16-legacy`. Custom checkpoints:\n`--checkpoint model.pt --num-classes 3 --class-names a,b,c`. `predict --json` keeps stdout pure JSON\n(diagnostics go to stderr), so it pipes into `jq` even when the first run builds an engine.\n\n## Apps\n\n`dfine_detect` (single image) · `dfine_coco_eval` (mAP) · `dfine_bench` (per-stage latency \u002F FPS \u002F GPU-mem,\n`--cuda-graph`, `--graph-compare`, `--pipeline-compare`) · `dfine_build` (pure-C++ engine build, FP32) ·\n`dfine_inspect` \u002F `dfine_smoke`.\n\n## Architecture\n\nThe whole network is frozen into a `.engine`; C++ reimplements only the parts that must be fast or Python-free:\n\n```\nImageU8 (HWC u8) ──► CUDA preprocess (stretch-resize + \u002F255, BGR→RGB, →CHW)  [src\u002Fcore\u002Fcuda_preprocess.cu]\n                 ──► TensorRT engine (backbone+encoder+decoder+FDR\u002FIntegral\u002FLQE)  [frozen; owns deform core]\n                 ──► C++ decode (sigmoid → top-300 → cxcywh→xyxy → scale)  [src\u002Fcore\u002Fpostprocess.cpp]\n```\n\nPreprocessing is **`\u002F255` only — no ImageNet mean\u002Fstd** (a D-FINE quirk; copying a normal detector's kernel\ncollapses mAP). The FDR\u002FIntegral\u002FLQE box math stays *inside* the engine and is never reimplemented in C++.\n\nResize is stretch by default (the training convention). Pipelines that standardize on aspect-preserving\ninputs can opt into **letterbox** — `DetectorOptions.preprocess` or the sidecar `resize` field — with\nconfigurable anchor (center\u002Ftop-left), padding value, and upscale on\u002Foff; box coordinates are un-mapped\nand clipped automatically, including under the GPU decode and the full-pipeline graph. Measured cost on\nthe stretch-trained weights: **−1.7…−2.0 AP** (`trt-files\u002Fscripts\u002Fletterbox_eval.py`); the CUDA path\nreproduces the host reference to +0.0002 AP.\n\n## Precision guide\n\n| mode | mAP (full-val, m) | b8 vs FP32 | how |\n|---|---|---|---|\n| **FP32** | 0.5506 | 1× | `build_engine.py --no-tf32` |\n| FP16 (decoder FP32) | 0.5500 | 2.0× | `convert_fp16.py` → `build_engine.py --strongly-typed` (**not** the `kFP16` flag) |\n| **surgical FP16** ✅ | lossless, all 5 sizes (m: 0.5502 surgical \u002F 0.5500 `--slim`) | 2.3× (2.4× opt8) | opset-19 export → `convert_fp16_surgical.py` (release ships `--slim`) |\n| FP8 | −17.6 AP (subset) | 1.8× (7-9% *slower* than FP16) | rejected: E4M3 mantissa + GeForce Ada runs FP8 at FP16 rate |\n| INT8 | −3.2 AP | 2.26× | closed for v0.3 on Ada: slower than surgical FP16 with real accuracy cost under the PTQ recipes tried (`convert_int8.py` kept for reference); later calibration research re-opened the accuracy side — the closure is scoped, not physics |\n| BF16 | −0.0012 (sim) | — | no win over FP16 paths; the old \"−27 AP\" was a weak-typing artifact |\n\n**Opset 19 is mandatory for surgical FP16**: opset-16 exports decompose LayerNorm into primitive\nops, and TensorRT 10.13 miscompiles that decomposition in FP16 (mAP collapses to ~0.005 while\nONNXRuntime stays healthy — a TRT-side bug; minimal repro archived, NVIDIA report in preparation).\nThe converter hard-errors on opset \u003C 19.\n\n**Export sliders** — accuracy traded back for speed at export time (m, full-val, gains vs\nsurgical b8 = 526 img\u002Fs; every point in [docs\u002FRESEARCH_MATRIX.md](docs\u002FRESEARCH_MATRIX.md)):\n\n| slider | cost | gain | note |\n|---|---|---|---|\n| `--cascade 1:150` | −0.18 AP | +8% | top-150 queries after layer 1, ranked by the trained aux head — the best single slider |\n| `--num-queries 200` | −0.13 AP | +7% | decode cost halves; near-free on few-class fine-tunes |\n| `--eval-idx 2` | −0.57 AP | +4% | drops a decoder layer; best b1 latency without a CUDA graph |\n| `fast` = Q200+cascade 1:100 | −0.44…−0.77 AP | +11…19% across the lineup | the hero-table column |\n| `max` = fast + E2 + `--opt-batch 8` | −0.89 AP | +30% (686 img\u002Fs; +46% over v0.2.0 prod fp16) | m ladder top, 10.4× PyTorch |\n\nThe through-line: **D-FINE's FDR box-decode is acutely FP-precision-sensitive** — it amplifies tiny\nupstream rounding into box error. That single fact explains the grid_sample trap, the kFP16-flag\ntrap, the surgical converter's FP32 island (FDR scopes + deform *index* math), and why FP8\u002FINT8\nfail here. Full forensics in [docs\u002Fimpl\u002FM0_STATUS.md](docs\u002Fimpl\u002FM0_STATUS.md),\n[docs\u002FRESEARCH_MATRIX.md](docs\u002FRESEARCH_MATRIX.md), and [docs\u002FHANDOFF.md](docs\u002FHANDOFF.md).\n\n## Status & roadmap\n\n**Done & validated:** M0 (export→engine→validate, all 5 sizes) · M1 (C++ detector, AP 0.5506 == reference) ·\nM2 (FP16 + CUDA-graph) · **M4 bindings** (stable C ABI + Python `ctypes` package + zero-setup `dfine`\nCLI — detections byte-identical to the C++ path) · **intensive-core P1–P3** (GPU decode, `freeze()`,\nfull-pipeline graph) · optional letterbox preprocessing · **v0.3.0 precision campaign** (surgical FP16\nlossless on all five sizes, export sliders, cascade pruning; FP8\u002FINT8\u002Fdeform-plugin closed with\nmeasurements — [docs\u002FRESEARCH_MATRIX.md](docs\u002FRESEARCH_MATRIX.md)).\nM3 instance segmentation is shelved. A browser demo is under investigation (the explicit-gather export\nremoves the GridSample blocker, but operator coverage on browser runtimes is unproven — no promise until\na feasibility spike passes). Hardware validation now spans Ampere\u002FAda\u002FBlackwell\n([docs\u002FVALIDATION.md](docs\u002FVALIDATION.md)); near-term focus is distribution and a real-time video demo —\nsee **[docs\u002FROADMAP.md](docs\u002FROADMAP.md)**. [docs\u002FHANDOFF.md](docs\u002FHANDOFF.md) is the historical lab\njournal behind these decisions.\n\n## Troubleshooting\n\nFirst command: **`dfine doctor`** — environment facts, every library-discovery path with its\nverdict, and exactly what to attach to a bug report. The two most common trips:\n\n- **`libnvinfer.so.10: cannot open shared object file`** — TensorRT libs are not on `LD_LIBRARY_PATH`;\n  any TRT 10.x works, e.g. a `pip install \"tensorrt-cu12==10.13.*\"` venv's `tensorrt_libs` dir (Quickstart\n  step 3).\n- **Engine fails to deserialize** — `.engine` files are GPU-arch- and TRT-version-specific; rebuild from\n  the ONNX on the target machine (`build_engine.py`).\n\nEverything else — the `tensorrt` pip package resolving to CUDA-13, missing TensorRT headers,\nconda toolchain clashes, driver recovery on hosted boxes, export reproducibility:\n**[docs\u002FTROUBLESHOOTING.md](docs\u002FTROUBLESHOOTING.md)**.\n\nContributions: see [CONTRIBUTING.md](CONTRIBUTING.md) — the validation bar (warning-clean,\nsanitizer-clean, mAP-neutral) is spelled out there.\n\n## Credits & license\n\nC++\u002FTensorRT port of **D-FINE** (Peng et al.), Apache-2.0; export path built on\n[D-FINE-seg](https:\u002F\u002Fgithub.com\u002FArgoHA\u002FD-FINE-seg). The runtime scaffolding (TensorRT session wrapper,\nCUDA RAII helpers, logging, C ABI surface, preprocessing kernel core, CPU decode skeleton) was initially\nderived from [rf-detr-cpp](https:\u002F\u002Fgithub.com\u002Finfracv\u002Frf-detr-cpp) (Apache-2.0) and substantially adapted;\nthe explicit-gather deformable-attention export, surgical FP16 conversion, GPU decode, frozen-memory\ncontract, and full-pipeline CUDA graph are original to this project. Vendored: `stb_image` (public\ndomain); nlohmann\u002Fjson (MIT) is found on the system or fetched at configure time. Links NVIDIA\nTensorRT\u002FCUDA (install separately). This project is **Apache-2.0** — see [LICENSE](LICENSE) and\n[NOTICE](NOTICE).\n","D-FINE-cpp 是一个面向生产环境的 C++\u002FTensorRT 推理运行时，专为 D-FINE 目标检测模型优化。它提供端到端流水线（CUDA 预处理 → TensorRT 引擎 → C++ 后处理解码），支持显式形变导出、强类型 FP16 精度、单 cudaGraph 启动及冻结式执行，兼顾高精度（mAP 误差仅 0.002）与极致性能（最高 1550+ FPS）。项目采用零 Python、无 OpenCV 依赖设计，提供 C ABI 接口和 Python 绑定，适用于对延迟敏感、需嵌入式或服务端部署的实时视觉检测场景，如工业质检、智能交通与边缘安防。","2026-07-11 02:30:47","CREATED_QUERY"]