[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72347":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":30,"readmeContent":31,"aiSummary":32,"trendingCount":16,"starSnapshotCount":16,"syncStatus":33,"lastSyncTime":34,"discoverSource":35},72347,"D-FINE","Peterande\u002FD-FINE","Peterande","D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement  [ICLR 2025 Spotlight]","",null,"Python",3168,303,42,161,0,5,14,44,15,29.45,"Apache License 2.0",false,"master",true,[27,28,29],"d-fine","detr","object-detection","2026-06-12 02:03:02","\u003C!--# [D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement](https:\u002F\u002Farxiv.org\u002Fabs\u002Fxxxxxx) -->\n\nEnglish | [简体中文](README_cn.md) | [日本語](README_ja.md) | [English Blog](src\u002Fzoo\u002Fdfine\u002Fblog.md) | [中文博客](src\u002Fzoo\u002Fdfine\u002Fblog_cn.md)\n\n\u003Ch2 align=\"center\">\n  D-FINE: Redefine Regression Task of DETRs as Fine‑grained Distribution Refinement\n\u003C\u002Fh2>\n\n\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Fdeveloper0hye\u002FD-FINE\">\n        \u003Cimg alt=\"hf\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Hugging%20Face-Spaces-blue\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fblob\u002Fmaster\u002FLICENSE\">\n        \u003Cimg alt=\"license\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLICENSE-Apache%202.0-blue\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fpulls\">\n        \u003Cimg alt=\"prs\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues-pr\u002FPeterande\u002FD-FINE\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\">\n        \u003Cimg alt=\"issues\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues\u002FPeterande\u002FD-FINE?color=olive\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13842\">\n        \u003Cimg alt=\"arXiv\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2410.13842-red\">\n    \u003C\u002Fa>\n\u003C!--     \u003Ca href=\"mailto: pengyansong@mail.ustc.edu.cn\">\n        \u003Cimg alt=\"email\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcontact_me-email-yellow\">\n    \u003C\u002Fa> -->\n      \u003Ca href=\"https:\u002F\u002Fresults.pre-commit.ci\u002Flatest\u002Fgithub\u002FPeterande\u002FD-FINE\u002Fmaster\">\n        \u003Cimg alt=\"pre-commit.ci status\" src=\"https:\u002F\u002Fresults.pre-commit.ci\u002Fbadge\u002Fgithub\u002FPeterande\u002FD-FINE\u002Fmaster.svg\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\">\n        \u003Cimg alt=\"stars\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002FPeterande\u002FD-FINE\">\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\n\n\u003Cp align=\"center\">\n    📄 This is the official implementation of the paper:\n    \u003Cbr>\n    \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13842\">D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement\u003C\u002Fa>\n\u003C\u002Fp>\n\n\n\n\u003Cp align=\"center\">\nYansong Peng, Hebei Li, Peixi Wu, Yueyi Zhang, Xiaoyan Sun, and Feng Wu\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\nUniversity of Science and Technology of China\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fpaperswithcode.com\u002Fsota\u002Freal-time-object-detection-on-coco?p=d-fine-redefine-regression-task-in-detrs-as\">\n        \u003Cimg alt=\"sota\" src=\"https:\u002F\u002Fimg.shields.io\u002Fendpoint.svg?url=https:\u002F\u002Fpaperswithcode.com\u002Fbadge\u002Fd-fine-redefine-regression-task-in-detrs-as\u002Freal-time-object-detection-on-coco\">\n    \u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003C!-- \u003Ctable>\u003Ctr>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Flatency.png border=0 width=333>\u003C\u002Ftd>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Fparams.png border=0 width=333>\u003C\u002Ftd>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Fblob\u002Fmaster\u002Fflops.png border=0 width=333>\u003C\u002Ftd>\n\u003C\u002Ftr>\u003C\u002Ftable> -->\n\n\u003Cp align=\"center\">\n\u003Cstrong>If you like D-FINE, please give us a ⭐! Your support motivates us to keep improving!\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Fmaster\u002Ffigs\u002Fstats_padded.png\" width=\"1000\">\n\u003C\u002Fp>\n\nD-FINE is a powerful real-time object detector that redefines the bounding box regression task in DETRs as Fine-grained Distribution Refinement (FDR) and introduces Global Optimal Localization Self-Distillation (GO-LSD), achieving outstanding performance without introducing additional inference and training costs.\n\n\u003Cdetails open>\n\u003Csummary> Video \u003C\u002Fsummary>\n\nWe conduct object detection using D-FINE and YOLO11 on a complex street scene video from [YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=CfhEWj9sd9A). Despite challenging conditions such as backlighting, motion blur, and dense crowds, D-FINE-X successfully detects nearly all targets, including subtle small objects like backpacks, bicycles, and traffic lights. Its confidence scores and the localization precision for blurred edges are significantly higher than those of YOLO11.\n\n\u003C!-- We use D-FINE and YOLO11 on a street scene video from [YouTube](https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=CfhEWj9sd9A). Despite challenges like backlighting, motion blur, and dense crowds, D-FINE-X outperforms YOLO11x, detecting more objects with higher confidence and better precision. -->\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fe5933d8e-3c8a-400e-870b-4e452f5321d9\n\n\u003C\u002Fdetails>\n\n## 🚀 Updates\n- [x] **\\[2024.10.18\\]** Release D-FINE series.\n- [x] **\\[2024.10.25\\]** Add custom dataset finetuning configs ([#7](https:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fissues\u002F7)).\n- [x] **\\[2024.10.30\\]** Update D-FINE-L (E25) pretrained model, with performance improved by 2.0%.\n- [x] **\\[2024.11.07\\]** Release **D-FINE-N**, achiving 42.8% AP\u003Csup>val\u003C\u002Fsup> on COCO @ 472 FPS\u003Csup>T4\u003C\u002Fsup>!\n\n## Model Zoo\n\n### COCO\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑N** | COCO | **42.8** | 4M | 2.12ms | 7 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_n_coco.yml) | [42.8](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_n_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_n_coco_log.txt)\n**D‑FINE‑S** | COCO | **48.5** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_s_coco.yml) | [48.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_s_coco_log.txt)\n**D‑FINE‑M** | COCO | **52.3** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_m_coco.yml) | [52.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_m_coco_log.txt)\n**D‑FINE‑L** | COCO | **54.0** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_l_coco.yml) | [54.0](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_l_coco_log.txt)\n**D‑FINE‑X** | COCO | **55.8** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_x_coco.yml) | [55.8](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fcoco\u002Fdfine_x_coco_log.txt)\n\n\n### Objects365+COCO\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑S** | Objects365+COCO | **50.7** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_s_obj2coco.yml) | [50.7](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_s_obj2coco_log.txt)\n**D‑FINE‑M** | Objects365+COCO | **55.1** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_m_obj2coco.yml) | [55.1](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_m_obj2coco_log.txt)\n**D‑FINE‑L** | Objects365+COCO | **57.3** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj2coco.yml) | [57.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj2coco_e25.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_l_obj2coco_log_e25.txt)\n**D‑FINE‑X** | Objects365+COCO | **59.3** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_x_obj2coco.yml) | [59.3](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_obj2coco.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj2coco\u002Fdfine_x_obj2coco_log.txt)\n\n**We highly recommend that you use the Objects365 pre-trained model for fine-tuning:**\n\n⚠️ **Important**: Please note that this is generally beneficial for complex scene understanding. If your categories are very simple, it might lead to overfitting and suboptimal performance.\n\u003Cdetails>\n\u003Csummary>\u003Cstrong> 🔥 Pretrained Models on Objects365 (Best generalization) \u003C\u002Fstrong>\u003C\u002Fsummary>\n\n| Model | Dataset | AP\u003Csup>val\u003C\u002Fsup> | AP\u003Csup>5000\u003C\u002Fsup> | #Params | Latency | GFLOPs | config | checkpoint | logs |\n| :---: | :---: | :---: |  :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n**D‑FINE‑S** | Objects365 | **31.0** | **30.5** | 10M | 3.49ms | 25 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_s_obj365.yml) | [30.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_s_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_s_obj365_log.txt)\n**D‑FINE‑M** | Objects365 | **38.6** | **37.4** | 19M | 5.62ms | 57 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_m_obj365.yml) | [37.4](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_m_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_m_obj365_log.txt)\n**D‑FINE‑L** | Objects365 | - | **40.6** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj365.yml) | [40.6](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_l_obj365_log.txt)\n**D‑FINE‑L (E25)** | Objects365 | **44.7** | **42.6** | 31M | 8.07ms | 91 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_l_obj365.yml) | [42.6](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_l_obj365_e25.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_l_obj365_log_e25.txt)\n**D‑FINE‑X** | Objects365 | **49.5** | **46.5** | 62M | 12.89ms | 202 | [yml](.\u002Fconfigs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_x_obj365.yml) | [46.5](https:\u002F\u002Fgithub.com\u002FPeterande\u002Fstorage\u002Freleases\u002Fdownload\u002Fdfinev1.0\u002Fdfine_x_obj365.pth) | [url](https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Frefs\u002Fheads\u002Fmaster\u002Flogs\u002Fobj365\u002Fdfine_x_obj365_log.txt)\n- **E25**: Re-trained and extended the pretraining to 25 epochs.\n- **AP\u003Csup>val\u003C\u002Fsup>** is evaluated on *Objects365* full validation set.\n- **AP\u003Csup>5000\u003C\u002Fsup>** is evaluated on the first 5000 samples of the *Objects365* validation set.\n\u003C\u002Fdetails>\n\n**Notes:**\n- **AP\u003Csup>val\u003C\u002Fsup>** is evaluated on *MSCOCO val2017* dataset.\n- **Latency** is evaluated on a single T4 GPU with $batch\\\\_size = 1$, $fp16$, and $TensorRT==10.4.0$.\n- **Objects365+COCO** means finetuned model on *COCO* using pretrained weights trained on *Objects365*.\n\n\n\n## Quick start\n\n### Setup\n\n```shell\nconda create -n dfine python=3.11.9\nconda activate dfine\npip install -r requirements.txt\n```\n\n\n### Data Preparation\n\n\u003Cdetails>\n\u003Csummary> COCO2017 Dataset \u003C\u002Fsummary>\n\n1. Download COCO2017 from [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FCOCO_2017) or [COCO](https:\u002F\u002Fcocodataset.org\u002F#download).\n1. Modify paths in [coco_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fcoco_detection.yml)\n\n    ```yaml\n    train_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Ftrain2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_train2017.json\n    val_dataloader:\n        img_folder: \u002Fdata\u002FCOCO2017\u002Fval2017\u002F\n        ann_file: \u002Fdata\u002FCOCO2017\u002Fannotations\u002Finstances_val2017.json\n    ```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Objects365 Dataset \u003C\u002Fsummary>\n\n1. Download Objects365 from [OpenDataLab](https:\u002F\u002Fopendatalab.com\u002FOpenDataLab\u002FObjects365).\n\n2. Set the Base Directory:\n```shell\nexport BASE_DIR=\u002Fdata\u002FObjects365\u002Fdata\n```\n\n3. Extract and organize the downloaded files, resulting directory structure:\n\n```shell\n${BASE_DIR}\u002Ftrain\n├── images\n│   ├── v1\n│   │   ├── patch0\n│   │   │   ├── 000000000.jpg\n│   │   │   ├── 000000001.jpg\n│   │   │   └── ... (more images)\n│   ├── v2\n│   │   ├── patchx\n│   │   │   ├── 000000000.jpg\n│   │   │   ├── 000000001.jpg\n│   │   │   └── ... (more images)\n├── zhiyuan_objv2_train.json\n```\n\n```shell\n${BASE_DIR}\u002Fval\n├── images\n│   ├── v1\n│   │   ├── patch0\n│   │   │   ├── 000000000.jpg\n│   │   │   └── ... (more images)\n│   ├── v2\n│   │   ├── patchx\n│   │   │   ├── 000000000.jpg\n│   │   │   └── ... (more images)\n├── zhiyuan_objv2_val.json\n```\n\n4. Create a New Directory to Store Images from the Validation Set:\n```shell\nmkdir -p ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\n```\n\n5. Copy the v1 and v2 folders from the val directory into the train\u002Fimages_from_val directory\n```shell\ncp -r ${BASE_DIR}\u002Fval\u002Fimages\u002Fv1 ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\u002F\ncp -r ${BASE_DIR}\u002Fval\u002Fimages\u002Fv2 ${BASE_DIR}\u002Ftrain\u002Fimages_from_val\u002F\n```\n\n6. Run remap_obj365.py to merge a subset of the validation set into the training set. Specifically, this script moves samples with indices between 5000 and 800000 from the validation set to the training set.\n```shell\npython tools\u002Fremap_obj365.py --base_dir ${BASE_DIR}\n```\n\n\n7. Run the resize_obj365.py script to resize any images in the dataset where the maximum edge length exceeds 640 pixels. Use the updated JSON file generated in Step 5 to process the sample data. Ensure that you resize images in both the train and val datasets to maintain consistency.\n```shell\npython tools\u002Fresize_obj365.py --base_dir ${BASE_DIR}\n```\n\n8. Modify paths in [obj365_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fobj365_detection.yml)\n\n    ```yaml\n    train_dataloader:\n        img_folder: \u002Fdata\u002FObjects365\u002Fdata\u002Ftrain\n        ann_file: \u002Fdata\u002FObjects365\u002Fdata\u002Ftrain\u002Fnew_zhiyuan_objv2_train_resized.json\n    val_dataloader:\n        img_folder: \u002Fdata\u002FObjects365\u002Fdata\u002Fval\u002F\n        ann_file: \u002Fdata\u002FObjects365\u002Fdata\u002Fval\u002Fnew_zhiyuan_objv2_val_resized.json\n    ```\n\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>CrowdHuman\u003C\u002Fsummary>\n\nDownload COCO format dataset here: [url](https:\u002F\u002Faistudio.baidu.com\u002Fdatasetdetail\u002F231455)\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>Custom Dataset\u003C\u002Fsummary>\n\nTo train on your custom dataset, you need to organize it in the COCO format. Follow the steps below to prepare your dataset:\n\n1. **Set `remap_mscoco_category` to `False`:**\n\n    This prevents the automatic remapping of category IDs to match the MSCOCO categories.\n\n    ```yaml\n    remap_mscoco_category: False\n    ```\n\n2. **Organize Images:**\n\n    Structure your dataset directories as follows:\n\n    ```shell\n    dataset\u002F\n    ├── images\u002F\n    │   ├── train\u002F\n    │   │   ├── image1.jpg\n    │   │   ├── image2.jpg\n    │   │   └── ...\n    │   ├── val\u002F\n    │   │   ├── image1.jpg\n    │   │   ├── image2.jpg\n    │   │   └── ...\n    └── annotations\u002F\n        ├── instances_train.json\n        ├── instances_val.json\n        └── ...\n    ```\n\n    - **`images\u002Ftrain\u002F`**: Contains all training images.\n    - **`images\u002Fval\u002F`**: Contains all validation images.\n    - **`annotations\u002F`**: Contains COCO-formatted annotation files.\n\n3. **Convert Annotations to COCO Format:**\n\n    If your annotations are not already in COCO format, you'll need to convert them. You can use the following Python script as a reference or utilize existing tools:\n\n    ```python\n    import json\n\n    def convert_to_coco(input_annotations, output_annotations):\n        # Implement conversion logic here\n        pass\n\n    if __name__ == \"__main__\":\n        convert_to_coco('path\u002Fto\u002Fyour_annotations.json', 'dataset\u002Fannotations\u002Finstances_train.json')\n    ```\n\n4. **Update Configuration Files:**\n\n    Modify your [custom_detection.yml](.\u002Fconfigs\u002Fdataset\u002Fcustom_detection.yml).\n\n    ```yaml\n    task: detection\n\n    evaluator:\n      type: CocoEvaluator\n      iou_types: ['bbox', ]\n\n    num_classes: 777 # your dataset classes\n    remap_mscoco_category: False\n\n    train_dataloader:\n      type: DataLoader\n      dataset:\n        type: CocoDetection\n        img_folder: \u002Fdata\u002Fyourdataset\u002Ftrain\n        ann_file: \u002Fdata\u002Fyourdataset\u002Ftrain\u002Ftrain.json\n        return_masks: False\n        transforms:\n          type: Compose\n          ops: ~\n      shuffle: True\n      num_workers: 4\n      drop_last: True\n      collate_fn:\n        type: BatchImageCollateFunction\n\n    val_dataloader:\n      type: DataLoader\n      dataset:\n        type: CocoDetection\n        img_folder: \u002Fdata\u002Fyourdataset\u002Fval\n        ann_file: \u002Fdata\u002Fyourdataset\u002Fval\u002Fann.json\n        return_masks: False\n        transforms:\n          type: Compose\n          ops: ~\n      shuffle: False\n      num_workers: 4\n      drop_last: False\n      collate_fn:\n        type: BatchImageCollateFunction\n    ```\n\n\u003C\u002Fdetails>\n\n\n## Usage\n\u003Cdetails open>\n\u003Csummary> COCO2017 \u003C\u002Fsummary>\n\n\u003C!-- \u003Csummary>1. Training \u003C\u002Fsummary> -->\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --use-amp --seed=0\n```\n\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n3. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --test-only -r model.pth\n```\n\n\u003C!-- \u003Csummary>3. Tuning \u003C\u002Fsummary> -->\n4. Tuning\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --use-amp --seed=0 -t model.pth\n```\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Objects365 to COCO2017 \u003C\u002Fsummary>\n\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training on Objects365\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj365.yml --use-amp --seed=0\n```\n\n3. Tuning on COCO2017\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj2coco.yml --use-amp --seed=0 -t model.pth\n```\n\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n4. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml --test-only -r model.pth\n```\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Custom Dataset \u003C\u002Fsummary>\n\n1. Set Model\n```shell\nexport model=l  # n s m l x\n```\n\n2. Training on Custom Dataset\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --use-amp --seed=0\n```\n\u003C!-- \u003Csummary>2. Testing \u003C\u002Fsummary> -->\n3. Testing\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --test-only -r model.pth\n```\n\n4. Tuning on Custom Dataset\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fobjects365\u002Fdfine_hgnetv2_${model}_obj2custom.yml --use-amp --seed=0 -t model.pth\n```\n\n5. **[Optional]** Modify Class Mappings:\n\nWhen using the Objects365 pre-trained weights to train on your custom dataset, the example assumes that your dataset only contains the classes `'Person'` and `'Car'`. For faster convergence, you can modify `self.obj365_ids` in `src\u002Fsolver\u002F_solver.py` as follows:\n\n\n```python\nself.obj365_ids = [0, 5]  # Person, Cars\n```\nYou can replace these with any corresponding classes from your dataset. The list of Objects365 classes with their corresponding IDs:\nhttps:\u002F\u002Fgithub.com\u002FPeterande\u002FD-FINE\u002Fblob\u002F352a94ece291e26e1957df81277bef00fe88a8e3\u002Fsrc\u002Fsolver\u002F_solver.py#L330\n\nNew training command:\n\n```shell\nCUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --master_port=7777 --nproc_per_node=4 train.py -c configs\u002Fdfine\u002Fcustom\u002Fdfine_hgnetv2_${model}_custom.yml --use-amp --seed=0 -t model.pth\n```\n\nHowever, if you don't wish to modify the class mappings, the pre-trained Objects365 weights will still work without any changes. Modifying the class mappings is optional and can potentially accelerate convergence for specific tasks.\n\n\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Customizing Batch Size \u003C\u002Fsummary>\n\nFor example, if you want to double the total batch size when training D-FINE-L on COCO2017, here are the steps you should follow:\n\n1. **Modify your [dataloader.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdataloader.yml)** to increase the `total_batch_size`:\n\n    ```yaml\n    train_dataloader:\n        total_batch_size: 64  # Previously it was 32, now doubled\n    ```\n\n2. **Modify your [dfine_hgnetv2_l_coco.yml](.\u002Fconfigs\u002Fdfine\u002Fdfine_hgnetv2_l_coco.yml)**. Here’s how the key parameters should be adjusted:\n\n    ```yaml\n    optimizer:\n    type: AdamW\n    params:\n        -\n        params: '^(?=.*backbone)(?!.*norm|bn).*$'\n        lr: 0.000025  # doubled, linear scaling law\n        -\n        params: '^(?=.*(?:encoder|decoder))(?=.*(?:norm|bn)).*$'\n        weight_decay: 0.\n\n    lr: 0.0005  # doubled, linear scaling law\n    betas: [0.9, 0.999]\n    weight_decay: 0.0001  # need a grid search\n\n    ema:  # added EMA settings\n        decay: 0.9998  # adjusted by 1 - (1 - decay) * 2\n        warmups: 500  # halved\n\n    lr_warmup_scheduler:\n        warmup_duration: 250  # halved\n    ```\n\n\u003C\u002Fdetails>\n\n\n\u003Cdetails>\n\u003Csummary> Customizing Input Size \u003C\u002Fsummary>\n\nIf you'd like to train **D-FINE-L** on COCO2017 with an input size of 320x320, follow these steps:\n\n1. **Modify your [dataloader.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdataloader.yml)**:\n\n    ```yaml\n\n    train_dataloader:\n    dataset:\n        transforms:\n            ops:\n                - {type: Resize, size: [320, 320], }\n    collate_fn:\n        base_size: 320\n    dataset:\n        transforms:\n            ops:\n                - {type: Resize, size: [320, 320], }\n    ```\n\n2. **Modify your [dfine_hgnetv2.yml](.\u002Fconfigs\u002Fdfine\u002Finclude\u002Fdfine_hgnetv2.yml)**:\n\n    ```yaml\n    eval_spatial_size: [320, 320]\n    ```\n\n\u003C\u002Fdetails>\n\n## Tools\n\u003Cdetails>\n\u003Csummary> Deployment \u003C\u002Fsummary>\n\n\u003C!-- \u003Csummary>4. Export onnx \u003C\u002Fsummary> -->\n1. Setup\n```shell\npip install onnx onnxsim\nexport model=l  # n s m l x\n```\n\n2. Export onnx\n```shell\npython tools\u002Fdeployment\u002Fexport_onnx.py --check -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth\n```\n\n3. Export [tensorrt](https:\u002F\u002Fdocs.nvidia.com\u002Fdeeplearning\u002Ftensorrt\u002Finstall-guide\u002Findex.html)\n```shell\ntrtexec --onnx=\"model.onnx\" --saveEngine=\"model.engine\" --fp16\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Inference (Visualization) \u003C\u002Fsummary>\n\n\n1. Setup\n```shell\npip install -r tools\u002Finference\u002Frequirements.txt\nexport model=l  # n s m l x\n```\n\n\n\u003C!-- \u003Csummary>5. Inference \u003C\u002Fsummary> -->\n2. Inference (onnxruntime \u002F tensorrt \u002F torch)\n\nInference on images and videos is now supported.\n```shell\npython tools\u002Finference\u002Fonnx_inf.py --onnx model.onnx --input image.jpg  # video.mp4\npython tools\u002Finference\u002Ftrt_inf.py --trt model.engine --input image.jpg\npython tools\u002Finference\u002Ftorch_inf.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth --input image.jpg --device cuda:0\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Benchmark \u003C\u002Fsummary>\n\n1. Setup\n```shell\npip install -r tools\u002Fbenchmark\u002Frequirements.txt\nexport model=l  # n s m l x\n```\n\n\u003C!-- \u003Csummary>6. Benchmark \u003C\u002Fsummary> -->\n2. Model FLOPs, MACs, and Params\n```shell\npython tools\u002Fbenchmark\u002Fget_info.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml\n```\n\n2. TensorRT Latency\n```shell\npython tools\u002Fbenchmark\u002Ftrt_benchmark.py --COCO_dir path\u002Fto\u002FCOCO2017 --engine_dir model.engine\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Fiftyone Visualization  \u003C\u002Fsummary>\n\n1. Setup\n```shell\npip install fiftyone\nexport model=l  # n s m l x\n```\n4. Voxel51 Fiftyone Visualization ([fiftyone](https:\u002F\u002Fgithub.com\u002Fvoxel51\u002Ffiftyone))\n```shell\npython tools\u002Fvisualization\u002Ffiftyone_vis.py -c configs\u002Fdfine\u002Fdfine_hgnetv2_${model}_coco.yml -r model.pth\n```\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Others \u003C\u002Fsummary>\n\n1. Auto Resume Training\n```shell\nbash reference\u002Fsafe_training.sh\n```\n\n2. Converting Model Weights\n```shell\npython reference\u002Fconvert_weight.py model.pth\n```\n\u003C\u002Fdetails>\n\n## Figures and Visualizations\n\n\u003Cdetails>\n\u003Csummary> FDR and GO-LSD \u003C\u002Fsummary>\n\n1. Overview of D-FINE with FDR. The probability distributions that act as a more fine-\ngrained intermediate representation are iteratively refined by the decoder layers in a residual manner.\nNon-uniform weighting functions are applied to allow for finer localization.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Fmaster\u002Ffigs\u002Ffdr-1.jpg\" alt=\"Fine-grained Distribution Refinement Process\" width=\"1000\">\n\u003C\u002Fp>\n\n2. Overview of GO-LSD process. Localization knowledge from the final layer’s refined\ndistributions is distilled into earlier layers through DDF loss with decoupled weighting strategies.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Fmaster\u002Ffigs\u002Fgo_lsd-1.jpg\" alt=\"GO-LSD Process\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\u003Cdetails open>\n\u003Csummary> Distributions \u003C\u002Fsummary>\n\nVisualizations of FDR across detection scenarios with initial and refined bounding boxes, along with unweighted and weighted distributions.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Fmaster\u002Ffigs\u002Fmerged_image.jpg\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary> Hard Cases \u003C\u002Fsummary>\n\nThe following visualization demonstrates D-FINE's predictions in various complex detection scenarios. These include cases with occlusion, low-light conditions, motion blur, depth of field effects, and densely populated scenes. Despite these challenges, D-FINE consistently produces accurate localization results.\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Fmaster\u002Ffigs\u002Fhard_case-1.jpg\" alt=\"D-FINE Predictions in Challenging Scenarios\" width=\"1000\">\n\u003C\u002Fp>\n\n\u003C\u002Fdetails>\n\n\n\u003C!-- \u003Cdiv style=\"display: flex; flex-wrap: wrap; justify-content: center; margin: 0; padding: 0;\">\n    \u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Fmaster\u002Ffigs\u002Fmerged_image.jpg\" style=\"width:99.96%; margin: 0; padding: 0;\" \u002F>\n\u003C\u002Fdiv>\n\n\u003Ctable>\u003Ctr>\n\u003Ctd>\u003Cimg src=https:\u002F\u002Fraw.githubusercontent.com\u002FPeterande\u002Fstorage\u002Fmaster\u002Ffigs\u002Fmerged_image.jpg border=0 width=1000>\u003C\u002Ftd>\n\u003C\u002Ftr>\u003C\u002Ftable> -->\n\n\n\n\n## Citation\nIf you use `D-FINE` or its methods in your work, please cite the following BibTeX entries:\n\u003Cdetails open>\n\u003Csummary> bibtex \u003C\u002Fsummary>\n\n```latex\n@misc{peng2024dfine,\n      title={D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},\n      author={Yansong Peng and Hebei Li and Peixi Wu and Yueyi Zhang and Xiaoyan Sun and Feng Wu},\n      year={2024},\n      eprint={2410.13842},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\u003C\u002Fdetails>\n\n## Acknowledgement\nOur work is built upon [RT-DETR](https:\u002F\u002Fgithub.com\u002Flyuwenyu\u002FRT-DETR).\nThanks to the inspirations from [RT-DETR](https:\u002F\u002Fgithub.com\u002Flyuwenyu\u002FRT-DETR), [GFocal](https:\u002F\u002Fgithub.com\u002Fimplus\u002FGFocal), [LD](https:\u002F\u002Fgithub.com\u002FHikariTJU\u002FLD), and [YOLOv9](https:\u002F\u002Fgithub.com\u002FWongKinYiu\u002Fyolov9).\n\n✨ Feel free to contribute and reach out if you have any questions! ✨\n","D-FINE 项目旨在通过细粒度分布细化重新定义 DETR（Detection Transformer）中的回归任务。该项目的核心功能是改进目标检测模型的回归精度，采用的技术特点包括对 DETR 模型进行优化，使其在保持高精度的同时减少计算资源消耗。适用于需要高效、精确目标检测的应用场景，如自动驾驶、视频监控和工业自动化等。此项目使用 Python 编写，并在 ICLR 2025 上获得了 Spotlight 论文的认可。",2,"2026-06-11 03:41:27","high_star"]