[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92368":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":10,"rankLanguage":10,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":19,"hasPages":19,"topics":21,"createdAt":10,"pushedAt":10,"updatedAt":35,"readmeContent":36,"aiSummary":37,"trendingCount":15,"starSnapshotCount":15,"syncStatus":38,"lastSyncTime":39,"discoverSource":40},92368,"rf-detr-cpp","infracv\u002Frf-detr-cpp","infracv","Production-ready C++\u002FTensorRT inference engine for RF-DETR. Object detection and instance segmentation with FP32\u002FFP16\u002FINT8 support. Optimized for NVIDIA GPUs, Jetson (Orin, AGX Thor).","https:\u002F\u002Finfracv.com",null,"C++",165,17,1,0,15,45.27,"Apache License 2.0",false,"develop",[22,23,24,25,26,27,28,29,30,31,32,33,34],"computer-vision","cpp","cuda","deep-learning","deployment","docker","instance-segmentation","nvidia-gpu","nvidia-jetson","object-detection","onnx","rf-detr","tensorrt","2026-07-22 04:02:05","\u003Cimg width=\"1540\" height=\"324\" alt=\"RF-DETR CPP\" src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F2b21c5d5-2ba7-498d-94f1-fe3f8e2bf871\" \u002F>\n\n\u003Ch3 align=\"center\">Production-ready C++\u002FTensorRT inference engine for RF-DETR.\u003C\u002Fh3>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"#quick-start\">Quick Start\u003C\u002Fa> ·\n  \u003Ca href=\"#supported-models--tasks\">Models\u003C\u002Fa> ·\n  \u003Ca href=\"#api-reference\">API\u003C\u002Fa> ·\n  \u003Ca href=\"#benchmarks\">Benchmarks\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FTensorRT-%E2%89%A5%2010.0-76B900?logo=nvidia&logoColor=white\" alt=\"TensorRT\"\u002F>\n  \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FCUDA-%E2%89%A5%2012.0-76B900?logo=nvidia&logoColor=white\" alt=\"CUDA\"\u002F>\n  \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FC%2B%2B-17-blue.svg?logo=cplusplus&logoColor=white\" alt=\"C++17\"\u002F>\n  \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-Apache--2.0-ef4444\" alt=\"License\"\u002F>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Finfracv\u002Frf-detr-cpp\u002Freleases\">\u003Cimg alt=\"GitHub release\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease\u002Finfracv\u002Frf-detr-cpp.svg\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\n---\n\n## Overview\n\n**RF-DETR C++** is a production-grade TensorRT inference engine for [RF-DETR](https:\u002F\u002Fgithub.com\u002Froboflow\u002Frf-detr), Roboflow's transformer-based real-time object detection model built on a DINOv2 backbone.\n\n```cpp\n#include \"rfdetr\u002Ftasks\u002Fdetector.hpp\"\n\nrfdetr::RFDetrDetector detector(\"rf-detr-nano-fp32.engine\");\nrfdetr::Detections dets = detector.detect(cv::imread(\"image.jpg\"), 0.5f);\n```\n\nUnlike YOLO models, RF-DETR uses a DETR-style architecture with **no NMS, no anchor grids, no letterboxing**. This requires a different inference pipeline, which this library implements entirely in C++ with zero Python at runtime.\n\nMost RF-DETR deployments run Python at inference time. This library eliminates that dependency:\n\n| | RF-DETR C++ | Python (PyTorch) |\n|---|:---:|:---:|\n| Runtime dependency | **C++ only** | Python + PyTorch + CUDA |\n| Preprocessing | **GPU CUDA kernel** | CPU \u002F torchvision |\n| Host-to-device transfer | **Async** (pinned memory) | Synchronous |\n| Precision | **FP32 \u002F FP16 \u002F INT8** | FP32 \u002F FP16 |\n| Inference dispatch | **CUDA Graph replay** | Per-frame forward pass |\n| Latency (RTX 5070 Ti) | **2.0 ms (FP16)** | ~15–30 ms |\n\n---\n\n## Quick Start\n\n```sh\n# 1. Clone & build\ngit clone https:\u002F\u002Fgithub.com\u002Finfracv\u002Frf-detr-cpp.git\ncd rf-detr-cpp\ncmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120\ncmake --build build -j$(nproc)\n# Replace 120 with your GPU arch: RTX 30xx=86, RTX 40xx=89, RTX 50xx=120\n# Tarball TensorRT: add -DTENSORRT_DIR=\u002Fpath\u002Fto\u002FTensorRT\n\n# 2. Export ONNX and build TRT engine (one-time, Python)\npip install -r requirements.txt\npython trt-files\u002Fscripts\u002Fexport_onnx.py --variant nano --out-dir trt-files\u002Fonnx\n\n# FP32 engine\n.\u002Fbuild\u002Frfdetr_build --onnx trt-files\u002Fonnx\u002Frf-detr-nano.onnx --precision fp32\n\n# FP16 engine\n.\u002Fbuild\u002Frfdetr_build --onnx trt-files\u002Fonnx\u002Frf-detr-nano.onnx --precision fp16\n\n# 3. Run inference\n.\u002Fbuild\u002Frfdetr_detect \\\n    --engine trt-files\u002Fonnx\u002Frf-detr-nano-fp16.engine \\\n    --image  asset\u002Ftest_img.jpg --out out\u002Fresult.jpg\n```\n\n---\n\n## Supported Models & Tasks\n\n| Task | Variants |\n|:-----|:---------|\n| Object Detection | `nano`, `small`, `medium`, `base`, `large` |\n| Instance Segmentation | `seg-nano`, `seg-small`, `seg-medium`, `seg-large`, `seg-xlarge`, `seg-2xlarge` |\n\n### Precision support\n\n| Precision | TRT \u003C 11 | TRT 11+ | Notes |\n|:---------:|:--------:|:-------:|-------|\n| FP32 | ✅ | ✅ | Default |\n| FP16 | ✅ (`kFP16` flag) | ✅ (`convert_fp16.py` → pre-converted ONNX) | ~25% faster than FP32 |\n| INT8 | ✅ (calibration cache) | ✅ (QDQ ONNX via `convert_int8.py`) | Lowest memory |\n\n> **FP16 NaN sanity check.** RF-DETR's transformer attention can produce values that exceed the FP16 range (±65,504), causing NaNs to propagate through the network and yield zero detections. After building an FP16 engine, run a single detection on `asset\u002Ftest_img.jpg`. If you get zero detections on an image where FP32 finds objects, your FP16 engine is hitting overflow. Fall back to FP32 (`rfdetr_build --precision fp32`) or rebuild with mixed-precision settings.\n\n---\n\n## Installation\n\n### Prerequisites\n\n| Dependency | Version | Notes |\n|:-----------|:-------:|:------|\n| NVIDIA GPU | CC ≥ 8.0 | RTX 30xx \u002F 40xx \u002F 50xx |\n| CUDA Toolkit | ≥ 12.0 | |\n| TensorRT | ≥ 10.0 | TRT 11 supported (strongly-typed networks) |\n| OpenCV | ≥ 4.5 | core, imgproc, imgcodecs, videoio, highgui |\n| CMake | ≥ 3.20 | |\n| C++ compiler | C++17 | GCC 9+, Clang 10+ |\n\n### Build from Source\n\n```sh\ngit clone https:\u002F\u002Fgithub.com\u002Finfracv\u002Frf-detr-cpp.git\ncd rf-detr-cpp\ncmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120\ncmake --build build -j$(nproc)\n# Tarball TensorRT: add -DTENSORRT_DIR=\u002Fpath\u002Fto\u002FTensorRT\n```\n\nReplace `120` with your GPU's compute capability:\n\n| GPU family | `CMAKE_CUDA_ARCHITECTURES` |\n|:-----------|:--------------------------:|\n| RTX 30xx (Ampere) | `86` |\n| RTX 40xx (Ada Lovelace) | `89` |\n| RTX 50xx (Blackwell) | `120` |\n| Jetson Orin | `87` |\n| Thor \u002F GH200 | `101` |\n\nThe default build (no `-DCMAKE_CUDA_ARCHITECTURES`) compiles for all five (86 87 89 101 120). Pass a single value to keep build times short.\n\n> **Note:** Always pass `-DCMAKE_CUDA_ARCHITECTURES` explicitly. If your environment has a stale value set (e.g. from a conda env), CMake will use that instead and the CUDA preprocessing kernel won't be optimized for your GPU.\n\n## Model Conversion\n\nPython is only needed for this one-time conversion step. The C++ runtime requires no Python.\n\n```sh\n# Python deps (uv recommended)\nuv venv --python 3.12 .venv && source .venv\u002Fbin\u002Factivate\nuv pip install -r requirements.txt\n\n# Export RF-DETR PyTorch → ONNX\npython trt-files\u002Fscripts\u002Fexport_onnx.py --variant nano --out-dir trt-files\u002Fonnx\n\n# FP32 engine\n.\u002Fbuild\u002Frfdetr_build --onnx trt-files\u002Fonnx\u002Frf-detr-nano.onnx --precision fp32\n\n# FP16 engine\n.\u002Fbuild\u002Frfdetr_build --onnx trt-files\u002Fonnx\u002Frf-detr-nano.onnx --precision fp16\n```\n\nFor INT8 see [trt-files\u002FINT8_QUANTIZATION.md](.\u002Ftrt-files\u002FINT8_QUANTIZATION.md).\n\n---\n\n## API Reference\n\n### Object Detection\n\n```cpp\n#include \"rfdetr\u002Ftasks\u002Fdetector.hpp\"\n\nrfdetr::RFDetrDetector detector(\"rf-detr-nano-fp32.engine\");\n\ncv::Mat image = cv::imread(\"image.jpg\");\nrfdetr::Detections dets = detector.detect(image, \u002F*threshold=*\u002F0.5f);\n\nfor (const auto& d : dets)\n    std::printf(\"class=%d  score=%.2f  box=[%.1f %.1f %.1f %.1f]\\n\",\n                d.class_id, d.score, d.box.x1, d.box.y1, d.box.x2, d.box.y2);\n\nrfdetr::draw_detections(image, dets);\ncv::imwrite(\"out.jpg\", image);\n\n\u002F\u002F Check whether CUDA Graph was captured at construction:\nif (detector.cuda_graph_active())\n    std::puts(\"CUDA Graph active: lowest dispatch latency\");\n```\n\n### Instance Segmentation\n\n```cpp\n#include \"rfdetr\u002Ftasks\u002Fsegmenter.hpp\"\n\nrfdetr::RFDetrSegmenter segmenter(\"rf-detr-seg-nano-fp16.engine\");\n\ncv::Mat image = cv::imread(\"image.jpg\");\nrfdetr::Detections dets = segmenter.segment(image, \u002F*threshold=*\u002F0.5f);\n\nfor (const auto& d : dets)\n    std::printf(\"class=%d  score=%.2f  mask=%dx%d (CV_8UC1)\\n\",\n                d.class_id, d.score, d.mask.cols, d.mask.rows);\n\nrfdetr::draw_segmentations(image, dets);\nrfdetr::draw_detections(image, dets);\ncv::imwrite(\"out.jpg\", image);\n```\n\nEach `Detection.mask` is a `CV_8UC1` cv::Mat of the original image size, with values 0 or 255.\n\n### Batch Inference\n\n```cpp\nstd::vector\u003Ccv::Mat> frames = { img0, img1, img2, img3 };\n\nrfdetr::RFDetrDetector detector(\"rf-detr-nano-fp32.engine\");\nauto batch = detector.detect_batch(frames, 0.5f);\n\nfor (std::size_t i = 0; i \u003C batch.size(); ++i)\n    std::printf(\"frame %zu: %zu detections\\n\", i, batch[i].size());\n```\n\n### C ABI (Python \u002F Rust \u002F Go FFI)\n\nBuild with `-DRFDETR_BUILD_C_API=ON` to produce `librfdetr_c.so`:\n\n```c\n#include \"rfdetr\u002Fc_api.h\"\n\nrfdetr_detector_t* det = rfdetr_detector_create(\"model.engine\", NULL);\nrfdetr_detections_t* res = rfdetr_detector_detect(det, bgr_data, w, h, w*3, 0.5f);\n\nfor (int i = 0; i \u003C res->count; ++i)\n    printf(\"cls=%d  score=%.2f\\n\", res->detections[i].class_id, res->detections[i].score);\n\nrfdetr_detections_free(res);\nrfdetr_detector_destroy(det);\n```\n\n```python\n# Python via ctypes\nimport ctypes\nlib = ctypes.CDLL(\"librfdetr_c.so\")\n# See include\u002Frfdetr\u002Fc_api.h for the full binding\n```\n\n---\n\n## Benchmarks\n\nNVIDIA RTX 5070 Ti, 500 iters, 50-iter warm-up, Batch 1.\n\n| Task | Precision | FPS | Avg Latency | P50 | P99 | GPU Memory |\n|:-----|:---------:|:---:|:-----------:|:---:|:---:|:----------:|\n| Detection (`nano`) | **FP32** | 406 | 2.462 ms | 2.444 ms | 2.778 ms | 859 MB |\n| Detection (`nano`) | **FP16** | 514 | 1.944 ms | 1.911 ms | 2.508 ms | 836 MB |\n| Segmentation (`seg-nano`) | **FP32** | 112 | 8.906 ms | 8.817 ms | 10.243 ms | 892 MB |\n| Segmentation (`seg-nano`) | **FP16** | 157 | 6.364 ms | 6.314 ms | 7.211 ms | 886 MB |\n\n> Numbers include the full pipeline: preprocessing, inference, and postprocessing. Segmentation mask decoding runs entirely on the GPU via a dedicated CUDA kernel.\n\n\u003Cdetails>\n\u003Csummary>Jetson Orin NX 16GB — TensorRT 10.3.0 \u002F CUDA 12.6\u003C\u002Fsummary>\n\n500 iters, 50-iter warm-up, Batch 1.\n\n| Task | Precision | FPS | Avg Latency | P50 | P99 |\n|:-----|:---------:|:---:|:-----------:|:---:|:---:|\n| Detection (`nano`) | **FP16** | 120 | 8.302 ms | 7.766 ms | 12.829 ms |\n| Detection (`nano`) | **FP32** | 50 | 19.984 ms | 18.858 ms | 33.003 ms |\n| Segmentation (`seg-nano`) | **FP16** | 55 | 18.093 ms | 17.008 ms | 23.871 ms |\n| Segmentation (`seg-nano`) | **FP32** | 24 | 42.292 ms | 42.047 ms | 45.436 ms |\n\n> GPU Memory column is omitted — Jetson uses unified CPU\u002FGPU memory, so `cudaMemGetInfo` reports system memory rather than dedicated VRAM and the delta reads as zero. See the [Benchmarking Guide](.\u002Fbenchmarks\u002FBENCHMARKING.md#unified-memory-caveat) for details.\n\n\u003C\u002Fdetails>\n\nSee [benchmarks\u002FBENCHMARKING.md](.\u002Fbenchmarks\u002FBENCHMARKING.md) to reproduce these numbers or run your own benchmarks on any GPU.\n\n---\n\n## License\n\nApache-2.0. See [LICENSE](.\u002FLICENSE).\n\n---\n\n## Acknowledgments\n\n- [Roboflow RF-DETR](https:\u002F\u002Fgithub.com\u002Froboflow\u002Frf-detr) for the model architecture and pretrained weights\n- [NVIDIA TensorRT](https:\u002F\u002Fdeveloper.nvidia.com\u002Ftensorrt) for optimised GPU inference\n- [OpenCV](https:\u002F\u002Fgithub.com\u002Fopencv\u002Fopencv) for image I\u002FO and visualisation\n\n---\n\n## Contributing\n\nWe welcome and appreciate all contributions. If you notice any issues or bugs, have questions, or would like to suggest new features, please open an issue or pull request. By sharing your ideas and improvements, you help make RF-DETR C++ better for everyone.\n\nSee [CONTRIBUTING.md](.\u002F.github\u002FCONTRIBUTING.md) for guidelines on setting up the dev environment, coding standards, and the PR process.\n\n---\n\n\u003Cp align=\"center\">\n  ⭐ If RF-DETR C++ is useful to you, consider giving it a star on \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Finfracv\u002Frf-detr-cpp\">GitHub\u003C\u002Fa>. It helps others find the project.\n\u003C\u002Fp>\n\n","这是一个面向生产环境的RF-DETR模型C++推理引擎，基于NVIDIA TensorRT实现高性能目标检测与实例分割。核心支持FP32\u002FFP16\u002FINT8精度推理，集成GPU端CUDA预处理、异步内存传输和CUDA Graph加速，完全消除Python依赖；专为NVIDIA GPU（含Jetson Orin\u002FAGX Thor）优化，适用于低延迟、高吞吐的嵌入式或边缘AI部署场景，如智能安防、工业质检与机器人视觉系统。",2,"2026-07-08 04:30:11","CREATED_QUERY"]