[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-70993":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":23,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":31,"readmeContent":32,"aiSummary":33,"trendingCount":16,"starSnapshotCount":16,"syncStatus":34,"lastSyncTime":35,"discoverSource":36},70993,"GroundingDINO","IDEA-Research\u002FGroundingDINO","IDEA-Research","[ECCV 2024] Official implementation of the paper \"Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection\"","https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499",null,"Python",10247,1037,49,287,0,19,56,153,57,119.05,"Apache License 2.0",false,"main",[26,27,28,29,30],"object-detection","open-world","open-world-detection","vision-language","vision-language-transformer","2026-06-12 04:00:58","\u003Cdiv align=\"center\">\n  \u003Cimg src=\".\u002F.asset\u002Fgrounding_dino_logo.png\" width=\"30%\">\n\u003C\u002Fdiv>\n\n# :sauropod: Grounding DINO \n\n[![PWC](https:\u002F\u002Fimg.shields.io\u002Fendpoint.svg?url=https:\u002F\u002Fpaperswithcode.com\u002Fbadge\u002Fgrounding-dino-marrying-dino-with-grounded\u002Fzero-shot-object-detection-on-mscoco)](https:\u002F\u002Fpaperswithcode.com\u002Fsota\u002Fzero-shot-object-detection-on-mscoco?p=grounding-dino-marrying-dino-with-grounded) [![PWC](https:\u002F\u002Fimg.shields.io\u002Fendpoint.svg?url=https:\u002F\u002Fpaperswithcode.com\u002Fbadge\u002Fgrounding-dino-marrying-dino-with-grounded\u002Fzero-shot-object-detection-on-odinw)](https:\u002F\u002Fpaperswithcode.com\u002Fsota\u002Fzero-shot-object-detection-on-odinw?p=grounding-dino-marrying-dino-with-grounded) \\\n[![PWC](https:\u002F\u002Fimg.shields.io\u002Fendpoint.svg?url=https:\u002F\u002Fpaperswithcode.com\u002Fbadge\u002Fgrounding-dino-marrying-dino-with-grounded\u002Fobject-detection-on-coco-minival)](https:\u002F\u002Fpaperswithcode.com\u002Fsota\u002Fobject-detection-on-coco-minival?p=grounding-dino-marrying-dino-with-grounded) [![PWC](https:\u002F\u002Fimg.shields.io\u002Fendpoint.svg?url=https:\u002F\u002Fpaperswithcode.com\u002Fbadge\u002Fgrounding-dino-marrying-dino-with-grounded\u002Fobject-detection-on-coco)](https:\u002F\u002Fpaperswithcode.com\u002Fsota\u002Fobject-detection-on-coco?p=grounding-dino-marrying-dino-with-grounded)\n\n\n**[IDEA-CVR, IDEA-Research](https:\u002F\u002Fgithub.com\u002FIDEA-Research)** \n\n[Shilong Liu](http:\u002F\u002Fwww.lsl.zone\u002F), [Zhaoyang Zeng](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=U_cvvUwAAAAJ&hl=zh-CN&oi=ao), [Tianhe Ren](https:\u002F\u002Frentainhe.github.io\u002F), [Feng Li](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=ybRe9GcAAAAJ&hl=zh-CN), [Hao Zhang](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=B8hPxMQAAAAJ&hl=zh-CN), [Jie Yang](https:\u002F\u002Fgithub.com\u002Fyangjie-cv), [Chunyuan Li](https:\u002F\u002Fscholar.google.com\u002Fcitations?user=Zd7WmXUAAAAJ&hl=zh-CN&oi=ao), [Jianwei Yang](https:\u002F\u002Fjwyang.github.io\u002F), [Hang Su](https:\u002F\u002Fscholar.google.com\u002Fcitations?hl=en&user=dxN1_X0AAAAJ&view_op=list_works&sortby=pubdate), [Jun Zhu](https:\u002F\u002Fscholar.google.com\u002Fcitations?hl=en&user=axsP38wAAAAJ), [Lei Zhang](https:\u002F\u002Fwww.leizhang.org\u002F)\u003Csup>:email:\u003C\u002Fsup>.\n\n\n[[`Paper`](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499)] [[`Demo`](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FShilongLiu\u002FGrounding_DINO_demo)] [[`BibTex`](#black_nib-citation)]\n\n\nPyTorch implementation and pretrained models for Grounding DINO. For details, see the paper **[Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499)**.\n\n- 🔥 **[Grounded SAM 2](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGrounded-SAM-2)** is released now, which combines Grounding DINO with [SAM 2](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything-2) for any object tracking in open-world scenarios.\n- 🔥 **[Grounding DINO 1.5](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGrounding-DINO-1.5-API)** is released now, which is IDEA Research's **Most Capable** Open-World Object Detection Model!\n- 🔥 **[Grounding DINO](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499)** and **[Grounded SAM](https:\u002F\u002Farxiv.org\u002Fabs\u002F2401.14159)** are now supported in Huggingface. For more convenient use, you can refer to [this documentation](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Ftransformers\u002Fmodel_doc\u002Fgrounding-dino)\n\n## :sun_with_face: Helpful Tutorial\n\n- :grapes: [[Read our arXiv Paper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499)]\n- :apple:  [[Watch our simple introduction video on YouTube](https:\u002F\u002Fyoutu.be\u002FwxWDt5UiwY8)]\n- :blossom:    [[Try the Colab Demo](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Froboflow-ai\u002Fnotebooks\u002Fblob\u002Fmain\u002Fnotebooks\u002Fzero-shot-object-detection-with-grounding-dino.ipynb)]\n- :sunflower: [[Try our Official Huggingface Demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FShilongLiu\u002FGrounding_DINO_demo)]\n- :maple_leaf: [[Watch the Step by Step Tutorial about GroundingDINO by Roboflow AI](https:\u002F\u002Fyoutu.be\u002FcMa77r3YrDk)]\n- :mushroom: [[GroundingDINO: Automated Dataset Annotation and Evaluation by Roboflow AI](https:\u002F\u002Fyoutu.be\u002FC4NqaRBz_Kw)]\n- :hibiscus: [[Accelerate Image Annotation with SAM and GroundingDINO by Roboflow AI](https:\u002F\u002Fyoutu.be\u002FoEQYStnF2l8)]\n- :white_flower: [[Autodistill: Train YOLOv8 with ZERO Annotations based on Grounding-DINO and Grounded-SAM by Roboflow AI](https:\u002F\u002Fgithub.com\u002Fautodistill\u002Fautodistill)]\n\n\u003C!-- Grounding DINO Methods | \n[![arXiv](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2303.05499-b31b1b.svg)](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499) \n[![YouTube](https:\u002F\u002Fbadges.aleen42.com\u002Fsrc\u002Fyoutube.svg)](https:\u002F\u002Fyoutu.be\u002FwxWDt5UiwY8) -->\n\n\u003C!-- Grounding DINO Demos |\n[![Open In Colab](https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg)](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Froboflow-ai\u002Fnotebooks\u002Fblob\u002Fmain\u002Fnotebooks\u002Fzero-shot-object-detection-with-grounding-dino.ipynb) -->\n\u003C!-- [![YouTube](https:\u002F\u002Fbadges.aleen42.com\u002Fsrc\u002Fyoutube.svg)](https:\u002F\u002Fyoutu.be\u002FcMa77r3YrDk)\n[![HuggingFace space](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗-HuggingFace%20Space-cyan.svg)](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FShilongLiu\u002FGrounding_DINO_demo)\n[![YouTube](https:\u002F\u002Fbadges.aleen42.com\u002Fsrc\u002Fyoutube.svg)](https:\u002F\u002Fyoutu.be\u002FoEQYStnF2l8)\n[![YouTube](https:\u002F\u002Fbadges.aleen42.com\u002Fsrc\u002Fyoutube.svg)](https:\u002F\u002Fyoutu.be\u002FC4NqaRBz_Kw) -->\n\n## :sparkles: Highlight Projects\n\n- [Semantic-SAM: a universal image segmentation model to enable segment and recognize anything at any desired granularity.](https:\u002F\u002Fgithub.com\u002FUX-Decoder\u002FSemantic-SAM), \n- [DetGPT: Detect What You Need via Reasoning](https:\u002F\u002Fgithub.com\u002FOptimalScale\u002FDetGPT)\n- [Grounded-SAM: Marrying Grounding DINO with Segment Anything](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGrounded-Segment-Anything)\n- [Grounding DINO with Stable Diffusion](demo\u002Fimage_editing_with_groundingdino_stablediffusion.ipynb)\n- [Grounding DINO with GLIGEN for Controllable Image Editing](demo\u002Fimage_editing_with_groundingdino_gligen.ipynb)\n- [OpenSeeD: A Simple and Strong Openset Segmentation Model](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FOpenSeeD)\n- [SEEM: Segment Everything Everywhere All at Once](https:\u002F\u002Fgithub.com\u002FUX-Decoder\u002FSegment-Everything-Everywhere-All-At-Once)\n- [X-GPT: Conversational Visual Agent supported by X-Decoder](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FX-Decoder\u002Ftree\u002Fxgpt)\n- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https:\u002F\u002Fgithub.com\u002Fgligen\u002FGLIGEN)\n- [LLaVA: Large Language and Vision Assistant](https:\u002F\u002Fgithub.com\u002Fhaotian-liu\u002FLLaVA)\n\n\u003C!-- Extensions | [Grounding DINO with Segment Anything](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGrounded-Segment-Anything); [Grounding DINO with Stable Diffusion](demo\u002Fimage_editing_with_groundingdino_stablediffusion.ipynb); [Grounding DINO with GLIGEN](demo\u002Fimage_editing_with_groundingdino_gligen.ipynb)  -->\n\n\n\n\u003C!-- Official PyTorch implementation of [Grounding DINO](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499), a stronger open-set object detector. Code is available now! -->\n\n\n## :bulb: Highlight\n\n- **Open-Set Detection.** Detect **everything** with language!\n- **High Performance.** COCO zero-shot **52.5 AP** (training without COCO data!). COCO fine-tune **63.0 AP**.\n- **Flexible.** Collaboration with Stable Diffusion for Image Editting.\n\n\n\n\n## :fire: News\n- **`2023\u002F07\u002F18`**: We release [Semantic-SAM](https:\u002F\u002Fgithub.com\u002FUX-Decoder\u002FSemantic-SAM), a universal image segmentation model to enable segment and recognize anything at any desired granularity. **Code** and **checkpoint** are available!\n- **`2023\u002F06\u002F17`**: We provide an example to evaluate Grounding DINO on COCO zero-shot performance.\n- **`2023\u002F04\u002F15`**: Refer to [CV in the Wild Readings](https:\u002F\u002Fgithub.com\u002FComputer-Vision-in-the-Wild\u002FCVinW_Readings) for those who are interested in open-set recognition!\n- **`2023\u002F04\u002F08`**: We release [demos](demo\u002Fimage_editing_with_groundingdino_gligen.ipynb) to combine [Grounding DINO](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499) with [GLIGEN](https:\u002F\u002Fgithub.com\u002Fgligen\u002FGLIGEN)  for more controllable image editings.\n- **`2023\u002F04\u002F08`**: We release [demos](demo\u002Fimage_editing_with_groundingdino_stablediffusion.ipynb) to combine [Grounding DINO](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499) with [Stable Diffusion](https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableDiffusion) for image editings.\n- **`2023\u002F04\u002F06`**: We build a new demo by marrying GroundingDINO with [Segment-Anything](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsegment-anything) named **[Grounded-Segment-Anything](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGrounded-Segment-Anything)** aims to support segmentation in GroundingDINO.\n- **`2023\u002F03\u002F28`**: A YouTube [video](https:\u002F\u002Fyoutu.be\u002FcMa77r3YrDk) about Grounding DINO and basic object detection prompt engineering. [[SkalskiP](https:\u002F\u002Fgithub.com\u002FSkalskiP)]\n- **`2023\u002F03\u002F28`**: Add a [demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FShilongLiu\u002FGrounding_DINO_demo) on Hugging Face Space!\n- **`2023\u002F03\u002F27`**: Support CPU-only mode. Now the model can run on machines without GPUs.\n- **`2023\u002F03\u002F25`**: A [demo](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Froboflow-ai\u002Fnotebooks\u002Fblob\u002Fmain\u002Fnotebooks\u002Fzero-shot-object-detection-with-grounding-dino.ipynb) for Grounding DINO is available at Colab. [[SkalskiP](https:\u002F\u002Fgithub.com\u002FSkalskiP)]\n- **`2023\u002F03\u002F22`**: Code is available Now!\n\n\u003Cdetails open>\n\u003Csummary>\u003Cfont size=\"4\">\nDescription\n\u003C\u002Ffont>\u003C\u002Fsummary>\n \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499\">Paper\u003C\u002Fa> introduction.\n\u003Cimg src=\".asset\u002Fhero_figure.png\" alt=\"ODinW\" width=\"100%\">\nMarrying \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\">Grounding DINO\u003C\u002Fa> and \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fgligen\u002FGLIGEN\">GLIGEN\u003C\u002Fa>\n\u003Cimg src=\"https:\u002F\u002Fhuggingface.co\u002FShilongLiu\u002FGroundingDINO\u002Fresolve\u002Fmain\u002FGD_GLIGEN.png\" alt=\"gd_gligen\" width=\"100%\">\n\u003C\u002Fdetails>\n\n## :star: Explanations\u002FTips for Grounding DINO Inputs and Outputs\n- Grounding DINO accepts an `(image, text)` pair as inputs.\n- It outputs `900` (by default) object boxes. Each box has similarity scores across all input words. (as shown in Figures below.)\n- We defaultly choose the boxes whose highest similarities are higher than a `box_threshold`.\n- We extract the words whose similarities are higher than the `text_threshold` as predicted labels.\n- If you want to obtain objects of specific phrases, like the `dogs` in the sentence `two dogs with a stick.`, you can select the boxes with highest text similarities with `dogs` as final outputs. \n- Note that each word can be split to **more than one** tokens with different tokenlizers. The number of words in a sentence may not equal to the number of text tokens.\n- We suggest separating different category names with `.` for Grounding DINO.\n![model_explain1](.asset\u002Fmodel_explan1.PNG)\n![model_explain2](.asset\u002Fmodel_explan2.PNG)\n\n## :label: TODO \n\n- [x] Release inference code and demo.\n- [x] Release checkpoints.\n- [x] Grounding DINO with Stable Diffusion and GLIGEN demos.\n- [ ] Release training codes.\n\n## :hammer_and_wrench: Install \n\n**Note:**\n\n0. If you have a CUDA environment, please make sure the environment variable `CUDA_HOME` is set. It will be compiled under CPU-only mode if no CUDA available.\n\nPlease make sure following the installation steps strictly, otherwise the program may produce: \n```bash\nNameError: name '_C' is not defined\n```\n\nIf this happened, please reinstalled the groundingDINO by reclone the git and do all the installation steps again.\n \n#### how to check cuda:\n```bash\necho $CUDA_HOME\n```\nIf it print nothing, then it means you haven't set up the path\u002F\n\nRun this so the environment variable will be set under current shell. \n```bash\nexport CUDA_HOME=\u002Fpath\u002Fto\u002Fcuda-11.3\n```\n\nNotice the version of cuda should be aligned with your CUDA runtime, for there might exists multiple cuda at the same time. \n\nIf you want to set the CUDA_HOME permanently, store it using:\n\n```bash\necho 'export CUDA_HOME=\u002Fpath\u002Fto\u002Fcuda' >> ~\u002F.bashrc\n```\nafter that, source the bashrc file and check CUDA_HOME:\n```bash\nsource ~\u002F.bashrc\necho $CUDA_HOME\n```\n\nIn this example, \u002Fpath\u002Fto\u002Fcuda-11.3 should be replaced with the path where your CUDA toolkit is installed. You can find this by typing **which nvcc** in your terminal:\n\nFor instance, \nif the output is \u002Fusr\u002Flocal\u002Fcuda\u002Fbin\u002Fnvcc, then:\n```bash\nexport CUDA_HOME=\u002Fusr\u002Flocal\u002Fcuda\n```\n**Installation:**\n\n1.Clone the GroundingDINO repository from GitHub.\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO.git\n```\n\n2. Change the current directory to the GroundingDINO folder.\n\n```bash\ncd GroundingDINO\u002F\n```\n\n3. Install the required dependencies in the current directory.\n\n```bash\npip install -e .\n```\n\n4. Download pre-trained model weights.\n\n```bash\nmkdir weights\ncd weights\nwget -q https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\u002Freleases\u002Fdownload\u002Fv0.1.0-alpha\u002Fgroundingdino_swint_ogc.pth\ncd ..\n```\n\n## :arrow_forward: Demo\nCheck your GPU ID (only if you're using a GPU)\n\n```bash\nnvidia-smi\n```\nReplace `{GPU ID}`, `image_you_want_to_detect.jpg`, and `\"dir you want to save the output\"` with appropriate values in the following command\n```bash\nCUDA_VISIBLE_DEVICES={GPU ID} python demo\u002Finference_on_a_image.py \\\n-c groundingdino\u002Fconfig\u002FGroundingDINO_SwinT_OGC.py \\\n-p weights\u002Fgroundingdino_swint_ogc.pth \\\n-i image_you_want_to_detect.jpg \\\n-o \"dir you want to save the output\" \\\n-t \"chair\"\n [--cpu-only] # open it for cpu mode\n```\n\nIf you would like to specify the phrases to detect, here is a demo:\n```bash\nCUDA_VISIBLE_DEVICES={GPU ID} python demo\u002Finference_on_a_image.py \\\n-c groundingdino\u002Fconfig\u002FGroundingDINO_SwinT_OGC.py \\\n-p .\u002Fgroundingdino_swint_ogc.pth \\\n-i .asset\u002Fcat_dog.jpeg \\\n-o logs\u002F1111 \\\n-t \"There is a cat and a dog in the image .\" \\\n--token_spans \"[[[9, 10], [11, 14]], [[19, 20], [21, 24]]]\"\n [--cpu-only] # open it for cpu mode\n```\nThe token_spans specify the start and end positions of a phrases. For example, the first phrase is `[[9, 10], [11, 14]]`. `\"There is a cat and a dog in the image .\"[9:10] = 'a'`, `\"There is a cat and a dog in the image .\"[11:14] = 'cat'`. Hence it refers to the phrase `a cat` . Similarly, the `[[19, 20], [21, 24]]` refers to the phrase `a dog`.\n\nSee the `demo\u002Finference_on_a_image.py` for more details.\n\n**Running with Python:**\n\n```python\nfrom groundingdino.util.inference import load_model, load_image, predict, annotate\nimport cv2\n\nmodel = load_model(\"groundingdino\u002Fconfig\u002FGroundingDINO_SwinT_OGC.py\", \"weights\u002Fgroundingdino_swint_ogc.pth\")\nIMAGE_PATH = \"weights\u002Fdog-3.jpeg\"\nTEXT_PROMPT = \"chair . person . dog .\"\nBOX_TRESHOLD = 0.35\nTEXT_TRESHOLD = 0.25\n\nimage_source, image = load_image(IMAGE_PATH)\n\nboxes, logits, phrases = predict(\n    model=model,\n    image=image,\n    caption=TEXT_PROMPT,\n    box_threshold=BOX_TRESHOLD,\n    text_threshold=TEXT_TRESHOLD\n)\n\nannotated_frame = annotate(image_source=image_source, boxes=boxes, logits=logits, phrases=phrases)\ncv2.imwrite(\"annotated_image.jpg\", annotated_frame)\n```\n**Web UI**\n\nWe also provide a demo code to integrate Grounding DINO with Gradio Web UI. See the file `demo\u002Fgradio_app.py` for more details.\n\n**Notebooks**\n\n- We release [demos](demo\u002Fimage_editing_with_groundingdino_gligen.ipynb) to combine [Grounding DINO](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499) with [GLIGEN](https:\u002F\u002Fgithub.com\u002Fgligen\u002FGLIGEN)  for more controllable image editings.\n- We release [demos](demo\u002Fimage_editing_with_groundingdino_stablediffusion.ipynb) to combine [Grounding DINO](https:\u002F\u002Farxiv.org\u002Fabs\u002F2303.05499) with [Stable Diffusion](https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableDiffusion) for image editings.\n\n## COCO Zero-shot Evaluations\n\nWe provide an example to evaluate Grounding DINO zero-shot performance on COCO. The results should be **48.5**.\n\n```bash\nCUDA_VISIBLE_DEVICES=0 \\\npython demo\u002Ftest_ap_on_coco.py \\\n -c groundingdino\u002Fconfig\u002FGroundingDINO_SwinT_OGC.py \\\n -p weights\u002Fgroundingdino_swint_ogc.pth \\\n --anno_path \u002Fpath\u002Fto\u002Fannoataions\u002Fie\u002Finstances_val2017.json \\\n --image_dir \u002Fpath\u002Fto\u002Fimagedir\u002Fie\u002Fval2017\n```\n\n\n## :luggage: Checkpoints\n\n\u003C!-- insert a table -->\n\u003Ctable>\n  \u003Cthead>\n    \u003Ctr style=\"text-align: right;\">\n      \u003Cth>\u003C\u002Fth>\n      \u003Cth>name\u003C\u002Fth>\n      \u003Cth>backbone\u003C\u002Fth>\n      \u003Cth>Data\u003C\u002Fth>\n      \u003Cth>box AP on COCO\u003C\u002Fth>\n      \u003Cth>Checkpoint\u003C\u002Fth>\n      \u003Cth>Config\u003C\u002Fth>\n    \u003C\u002Ftr>\n  \u003C\u002Fthead>\n  \u003Ctbody>\n    \u003Ctr>\n      \u003Cth>1\u003C\u002Fth>\n      \u003Ctd>GroundingDINO-T\u003C\u002Ftd>\n      \u003Ctd>Swin-T\u003C\u002Ftd>\n      \u003Ctd>O365,GoldG,Cap4M\u003C\u002Ftd>\n      \u003Ctd>48.4 (zero-shot) \u002F 57.2 (fine-tune)\u003C\u002Ftd>\n      \u003Ctd>\u003Ca href=\"https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\u002Freleases\u002Fdownload\u002Fv0.1.0-alpha\u002Fgroundingdino_swint_ogc.pth\">GitHub link\u003C\u002Fa> | \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002FShilongLiu\u002FGroundingDINO\u002Fresolve\u002Fmain\u002Fgroundingdino_swint_ogc.pth\">HF link\u003C\u002Fa>\u003C\u002Ftd>\n      \u003Ctd>\u003Ca href=\"https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\u002Fblob\u002Fmain\u002Fgroundingdino\u002Fconfig\u002FGroundingDINO_SwinT_OGC.py\">link\u003C\u002Fa>\u003C\u002Ftd>\n    \u003C\u002Ftr>\n    \u003Ctr>\n      \u003Cth>2\u003C\u002Fth>\n      \u003Ctd>GroundingDINO-B\u003C\u002Ftd>\n      \u003Ctd>Swin-B\u003C\u002Ftd>\n      \u003Ctd>COCO,O365,GoldG,Cap4M,OpenImage,ODinW-35,RefCOCO\u003C\u002Ftd>\n      \u003Ctd>56.7 \u003C\u002Ftd>\n      \u003Ctd>\u003Ca href=\"https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\u002Freleases\u002Fdownload\u002Fv0.1.0-alpha2\u002Fgroundingdino_swinb_cogcoor.pth\">GitHub link\u003C\u002Fa>  | \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002FShilongLiu\u002FGroundingDINO\u002Fresolve\u002Fmain\u002Fgroundingdino_swinb_cogcoor.pth\">HF link\u003C\u002Fa> \n      \u003Ctd>\u003Ca href=\"https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\u002Fblob\u002Fmain\u002Fgroundingdino\u002Fconfig\u002FGroundingDINO_SwinB_cfg.py\">link\u003C\u002Fa>\u003C\u002Ftd>\n    \u003C\u002Ftr>\n  \u003C\u002Ftbody>\n\u003C\u002Ftable>\n\n## :medal_military: Results\n\n\u003Cdetails open>\n\u003Csummary>\u003Cfont size=\"4\">\nCOCO Object Detection Results\n\u003C\u002Ffont>\u003C\u002Fsummary>\n\u003Cimg src=\".asset\u002FCOCO.png\" alt=\"COCO\" width=\"100%\">\n\u003C\u002Fdetails>\n\n\u003Cdetails open>\n\u003Csummary>\u003Cfont size=\"4\">\nODinW Object Detection Results\n\u003C\u002Ffont>\u003C\u002Fsummary>\n\u003Cimg src=\".asset\u002FODinW.png\" alt=\"ODinW\" width=\"100%\">\n\u003C\u002Fdetails>\n\n\u003Cdetails open>\n\u003Csummary>\u003Cfont size=\"4\">\nMarrying Grounding DINO with \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableDiffusion\">Stable Diffusion\u003C\u002Fa> for Image Editing\n\u003C\u002Ffont>\u003C\u002Fsummary>\nSee our example \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\u002Fblob\u002Fmain\u002Fdemo\u002Fimage_editing_with_groundingdino_stablediffusion.ipynb\">notebook\u003C\u002Fa> for more details.\n\u003Cimg src=\".asset\u002FGD_SD.png\" alt=\"GD_SD\" width=\"100%\">\n\u003C\u002Fdetails>\n\n\n\u003Cdetails open>\n\u003Csummary>\u003Cfont size=\"4\">\nMarrying Grounding DINO with \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fgligen\u002FGLIGEN\">GLIGEN\u003C\u002Fa> for more Detailed Image Editing.\n\u003C\u002Ffont>\u003C\u002Fsummary>\nSee our example \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FGroundingDINO\u002Fblob\u002Fmain\u002Fdemo\u002Fimage_editing_with_groundingdino_gligen.ipynb\">notebook\u003C\u002Fa> for more details.\n\u003Cimg src=\".asset\u002FGD_GLIGEN.png\" alt=\"GD_GLIGEN\" width=\"100%\">\n\u003C\u002Fdetails>\n\n## :sauropod: Model: Grounding DINO\n\nIncludes: a text backbone, an image backbone, a feature enhancer, a language-guided query selection, and a cross-modality decoder.\n\n![arch](.asset\u002Farch.png)\n\n\n## :hearts: Acknowledgement\n\nOur model is related to [DINO](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002FDINO) and [GLIP](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FGLIP). Thanks for their great work!\n\nWe also thank great previous work including DETR, Deformable DETR, SMCA, Conditional DETR, Anchor DETR, Dynamic DETR, DAB-DETR, DN-DETR, etc. More related work are available at [Awesome Detection Transformer](https:\u002F\u002Fgithub.com\u002FIDEACVR\u002Fawesome-detection-transformer). A new toolbox [detrex](https:\u002F\u002Fgithub.com\u002FIDEA-Research\u002Fdetrex) is available as well.\n\nThanks [Stable Diffusion](https:\u002F\u002Fgithub.com\u002FStability-AI\u002FStableDiffusion) and [GLIGEN](https:\u002F\u002Fgithub.com\u002Fgligen\u002FGLIGEN) for their awesome models.\n\n\n## :black_nib: Citation\n\nIf you find our work helpful for your research, please consider citing the following BibTeX entry.   \n\n```bibtex\n@article{liu2023grounding,\n  title={Grounding dino: Marrying dino with grounded pre-training for open-set object detection},\n  author={Liu, Shilong and Zeng, Zhaoyang and Ren, Tianhe and Li, Feng and Zhang, Hao and Yang, Jie and Li, Chunyuan and Yang, Jianwei and Su, Hang and Zhu, Jun and others},\n  journal={arXiv preprint arXiv:2303.05499},\n  year={2023}\n}\n```\n\n\n\n\n","Grounding DINO 是一个基于 PyTorch 实现的开放集目标检测模型，结合了 DINO 与有监督预训练技术。该项目提供了官方实现代码和预训练模型，支持零样本目标检测任务，在多个基准数据集上取得了领先性能。其核心功能包括通过视觉-语言转换器实现对未知类别的物体进行准确检测，适用于需要在开放环境下识别未见过物体的应用场景，如自动驾驶、智能监控等。此外，项目还扩展了 Grounded SAM 2 和 Grounding DINO 1.5 等版本，进一步增强了模型的能力和适用范围。",2,"2026-06-11 03:35:21","high_star"]