[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92366":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":10,"rankLanguage":10,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":10,"pushedAt":10,"updatedAt":23,"readmeContent":24,"aiSummary":25,"trendingCount":15,"starSnapshotCount":15,"syncStatus":26,"lastSyncTime":27,"discoverSource":28},92366,"depth-anything.cpp","mudler\u002Fdepth-anything.cpp","mudler","A from-scratch C++17\u002Fggml port of Depth Anything 3 (ByteDance)","",null,"C++",253,28,1,0,100,54.39,"MIT License",false,"master",true,[],"2026-07-22 04:02:05","# depth-anything.cpp\n\n**Brought to you by the [LocalAI](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI) team**, the folks behind LocalAI, the open-source AI engine that runs any model (LLMs, vision, voice, image, video) on any hardware, no GPU required.\n\n[![Model on Hugging Face](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fhuggingface\u002Fbadges\u002Fresolve\u002Fmain\u002Fmodel-on-hf-md.svg)](https:\u002F\u002Fhuggingface.co\u002Fmudler\u002Fdepth-anything.cpp-gguf)\n[![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-green)](LICENSE)\n[![LocalAI](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLocalAI-Run_Locally-orange)](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI)\n\nA from-scratch C++17\u002F[ggml](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fggml) port of [Depth Anything 3](https:\u002F\u002Fgithub.com\u002Fbytedance-seed\u002Fdepth-anything-3) (ByteDance) for **dependency-free monocular metric depth + camera pose** inference. One self-contained GGUF file, no Python, no PyTorch, no CUDA toolkit at inference, just a small native library and CLI, and now **faster than PyTorch on CPU**, bit-exact against the original.\n\n![depth-anything.cpp vs PyTorch on CPU: same depth, ggml finishes first](benchmarks\u002Fmedia\u002Fdepth_race.gif)\n\n> The same photo, depth computed side by side on CPU: identical output, depth-anything.cpp gets there first ([full clip](benchmarks\u002Fmedia\u002Fdepth_race.mp4)).\n\nGiven an image it recovers a dense **metric depth** map, per-pixel **confidence**, the camera **extrinsics (3x4)** and **intrinsics (3x3)**, an optional **sky** mask, a back-projected **3D point cloud**, and exports to **glb \u002F COLMAP \u002F PLY**. Everything is verified numerically equal to the reference DA3 forward (correlation 1.0), component by component.\n\n---\n\n## Features\n\n- **Monocular metric depth + camera pose** from a single image, plus multi-view depth+pose.\n- **Full DA3 family.** small (ViT-S), base (ViT-B), large (ViT-L), giant (ViT-g), metric-large, mono-large, and the nested giant+large metric model - all driven by metadata baked into the GGUF.\n- **The whole output surface:** depth, confidence, sky mask, extrinsics\u002Fintrinsics, ray-based pose, 3D Gaussians \u002F point cloud, and `glb` \u002F `COLMAP` \u002F `PLY` export.\n- **Self-contained GGUF.** Every dimension, hyperparameter and preprocessing constant lives inside the file. The loader reads them; nothing is hardcoded, no external config or vocab is shipped.\n- **Quantization** to f16 \u002F q8_0 \u002F q6_k \u002F q5_k \u002F q4_k - q4_k is **99 MB** (0.25x the f32) and near-lossless.\n- **CPU-first, GPU-ready.** Tuned CPU path (tinyBLAS, Winograd, flash-attention) plus CUDA \u002F Metal \u002F Vulkan ggml backends.\n- **Flat C API** (`include\u002Fda_capi.h`) - embed from C, C++, Go, or Rust. Powers the [LocalAI](#use-it-from-localai) backend.\n- **Parity-first.** Every component is gated against PyTorch-dumped reference tensors; the end-to-end depth matches the real `net()` at correlation 1.0.\n\n---\n\n## Supported models\n\nConvert any of the official [Depth-Anything-3](https:\u002F\u002Fhuggingface.co\u002Fdepth-anything) checkpoints to GGUF. All run through the same metadata-driven engine:\n\n| Model | Backbone | Output | Notes |\n|-------|----------|--------|-------|\n| `DA3-SMALL` | ViT-S | depth + conf + pose | smallest \u002F fastest |\n| `DA3-BASE` | ViT-B | depth + conf + pose | the default anchor |\n| `DA3-LARGE` | ViT-L | depth + conf + pose | higher quality |\n| `DA3-GIANT` | ViT-g | depth + conf + pose + 3D Gaussians | reconstruction |\n| `DA3MONO-LARGE` | ViT-L | depth + **sky** | monocular DPT head |\n| `DA3METRIC-LARGE` | ViT-L | metric depth + sky | metric branch |\n| `DA3NESTED-GIANT-LARGE` | ViT-g + ViT-L | aligned **metric** depth + pose | two-branch alignment |\n\n### Depth Anything V2\n\nThe same engine also runs [Depth Anything **V2**](https:\u002F\u002Fhuggingface.co\u002Fdepth-anything) checkpoints (single-image depth only — no confidence, pose or sky). Relative models emit inverse depth (ReLU); metric models emit depth in metres (Sigmoid × `max_depth`).\n\n| Model | Backbone | Output | Notes |\n|-------|----------|--------|-------|\n| `Depth-Anything-V2-Small` | ViT-S | relative depth | smallest \u002F fastest |\n| `Depth-Anything-V2-Base` | ViT-B | relative depth | |\n| `Depth-Anything-V2-Large` | ViT-L | relative depth | higher quality |\n| `Depth-Anything-V2-Metric-Hypersim-{Small,Base,Large}` | ViT-S\u002FB\u002FL | metric depth (m), indoor | `max_depth=20` |\n| `Depth-Anything-V2-Metric-VKITTI-{Small,Base,Large}` | ViT-S\u002FB\u002FL | metric depth (m), outdoor | `max_depth=80` |\n\n> DA2 is depth only (no pose\u002Fconfidence). The ViT-g (Giant) DA2 checkpoint is not shipped — its `Depth-Anything-V2-Giant` HF repo is gated\u002Funreleased.\n\n---\n\n## Performance\n\ndepth-anything.cpp is now **faster than PyTorch on CPU**: **1.20x at f32** and **1.31x at q8_0** on the production @504 path, while also running the same model in **half the memory**, **loading ~6.7x faster**, shipping as a **99 MB** quantized file, and needing **no Python \u002F PyTorch \u002F CUDA** at inference, all while staying **bit-exact** (correlation 1.0 vs the reference).\n\nCPU, AMD Ryzen 9 9950X3D (16-core \u002F 32-thread x86), `threads=16`, 504x336, sustained (`repeat=25`), PyTorch f32 baseline (full methodology + the CPU-optimization history in [`benchmarks\u002FBENCHMARK.md`](benchmarks\u002FBENCHMARK.md)):\n\n| engine | quant | model MB | load ms | infer ms | peak RAM MB | vs PyTorch |\n|--------|-------|---------:|--------:|---------:|------------:|-----------:|\n| PyTorch | f32 | 516 | 749 | 416.9 | 1328 | 1.00x |\n| **C++\u002Fggml** | f32 | 393 | **112** | **346.4** | **614** | **1.20x** |\n| **C++\u002Fggml** | q8_0 | 142 | **40** | **319.4** | **363** | **1.31x** |\n| **C++\u002Fggml** | q4_k | **99** | **25** | 395.2 | **320** | 1.05x |\n\nWhat flipped it: two positional embeddings (the DPT head's UV embedding ~90 ms, the backbone's bicubic pos-embed ~10 ms) were recomputed every forward with single-threaded scalar sin\u002Fcos and bicubic loops, even though they depend only on the input geometry and are identical every call. Caching them removed ~95 ms of per-forward host overhead (PyTorch builds the same embeddings with vectorized ops). These are x86 + oneDNN numbers; see [`benchmarks\u002FBENCHMARK.md`](benchmarks\u002FBENCHMARK.md). Camera pose adds a few ms on top of depth at f32. q8_0 is near-lossless; q4_k is the smallest. Bit-exact parity holds at every quant down to f16.\n\nOn **GPU** (NVIDIA GB10, via `-DDA_GGML_CUDA=ON`) the ggml CUDA path with flash attention **ties PyTorch's tuned cuDNN** (47.3 vs 47.3 ms\u002Fforward, ~47 ms across every quant) and loads 1.75-2.9x faster, so it wins the cold start (~548 vs ~926 ms). Details in [`benchmarks\u002FBENCHMARK.md`](benchmarks\u002FBENCHMARK.md#gpu-nvidia-gb10-grace-blackwell).\n\n![GPU inference speed](benchmarks\u002Fmedia\u002Fgpu_speed.png) ![GPU load time](benchmarks\u002Fmedia\u002Fgpu_load.png)\n\n![inference speed](benchmarks\u002Fmedia\u002Finfer_speed.png) ![peak memory](benchmarks\u002Fmedia\u002Fmemory.png)\n\n### See it run\n\nReal photos through the actual CLI, input next to the colorized depth (turbo):\n\n![real-photo depth maps from depth-anything.cpp](benchmarks\u002Fmedia\u002Fdepth_demo.png)\n\nMore plots: [load time](benchmarks\u002Fmedia\u002Fload_time.png), [model size](benchmarks\u002Fmedia\u002Fmodel_size.png), [quantization tradeoff](benchmarks\u002Fmedia\u002Fquant_tradeoff.png).\n\n---\n\n## Build\n\n```sh\ngit clone --recursive https:\u002F\u002Fgithub.com\u002Fmudler\u002Fdepth-anything.cpp\ncd depth-anything.cpp\ncmake -B build -DDA_BUILD_CLI=ON\ncmake --build build -j\n# -> build\u002Fexamples\u002Fcli\u002Fda3-cli\n```\n\n### CMake options\n\n| Option | Default | Effect |\n|--------|---------|--------|\n| `DA_BUILD_CLI` | ON | build the `da3-cli` tool |\n| `DA_BUILD_TESTS` | OFF | build the ctest parity suite |\n| `DA_SHARED` | OFF | build `libdepthanything.so` (static ggml, PIC) for embedding |\n| `DA_GGML_LLAMAFILE` | ON | tinyBLAS AVX-512\u002FAVX2 matmul kernels (faster CPU) |\n| `DA_GGML_CUDA` | OFF | CUDA backend (`-DCMAKE_CUDA_ARCHITECTURES=native` auto) |\n| `DA_GGML_METAL` | OFF | Apple Metal backend |\n| `DA_GGML_VULKAN` | OFF | Vulkan backend |\n\nGPU example: `cmake -B build -DDA_GGML_CUDA=ON && cmake --build build -j`. See [`docs\u002FGPU.md`](docs\u002FGPU.md).\n\n---\n\n## Python environment setup (conversion only)\n\nConversion and parity checks need a Python env; **inference does not**.\n\n```sh\npython3 -m venv .venv && source .venv\u002Fbin\u002Factivate\npip install -r requirements.txt\npython scripts\u002Fdownload_model.py --repo depth-anything\u002FDA3-BASE --out models\u002FDA3-BASE\n```\n\n## Converting a model\n\n```sh\n# depth + pose models (small\u002Fbase\u002Flarge\u002Fgiant)\npython scripts\u002Fconvert_da3_to_gguf.py --model models\u002FDA3-BASE --output models\u002Fdepth-anything-base-f32.gguf\n\n# monocular (depth + sky) - DA3MONO-LARGE\npython scripts\u002Fconvert_mono_to_gguf.py --model models\u002FDA3MONO-LARGE --output models\u002Fdepth-anything-mono-large-f32.gguf\n\n# nested two-branch metric model\npython scripts\u002Fconvert_nested_to_gguf.py --model models\u002FDA3NESTED-GIANT-LARGE --output-prefix models\u002Fdepth-anything-nested\n\n# Depth Anything V2 — relative (encoder vits\u002Fvitb\u002Fvitl)\npython scripts\u002Fconvert_da2_to_gguf.py --encoder vitl --ckpt models\u002Fdepth_anything_v2_vitl.pth \\\n    --output models\u002Fdepth-anything2-large-f32.gguf --name Depth-Anything-V2-Large\n\n# Depth Anything V2 — metric (add --max-depth: 20 for Hypersim\u002Findoor, 80 for VKITTI\u002Foutdoor)\npython scripts\u002Fconvert_da2_to_gguf.py --encoder vits --ckpt models\u002Fdepth_anything_v2_metric_hypersim_vits.pth \\\n    --output models\u002Fdepth-anything2-metric-hypersim-small-f32.gguf \\\n    --name Depth-Anything-V2-Metric-Hypersim-Small --max-depth 20\npython scripts\u002Fconvert_da2_to_gguf.py --encoder vitl --ckpt models\u002Fdepth_anything_v2_metric_vkitti_vitl.pth \\\n    --output models\u002Fdepth-anything2-metric-vkitti-large-f32.gguf \\\n    --name Depth-Anything-V2-Metric-VKITTI-Large --max-depth 80\n```\n\n## Quantization\n\n```sh\nbuild\u002Fexamples\u002Fcli\u002Fda3-cli quantize models\u002Fdepth-anything-base-f32.gguf models\u002Fdepth-anything-base-q4_k.gguf q4_k\n# types: f16 | q8_0 | q6_k | q5_k | q4_k\n```\n\n---\n\n## Running inference\n\n```sh\nCLI=build\u002Fexamples\u002Fcli\u002Fda3-cli\nM=models\u002Fdepth-anything-base-f32.gguf\n\n# Depth -> PFM (lossless float) + a colorizable grayscale PNG\n$CLI depth --model $M --input photo.jpg --pfm depth.pfm --png depth.png\n\n# Depth + camera pose (extrinsics 3x4 \u002F intrinsics 3x3) as JSON\n$CLI depth --model $M --input photo.jpg --pose pose.json\n\n# Ray-based pose (solved from the auxiliary ray field; needs a --with-aux GGUF)\n$CLI depth --model models\u002Fdepth-anything-base-aux-f32.gguf --input photo.jpg --ray-pose --pose pose.json\n\n# Monocular model: depth + sky mask\n$CLI depth --model models\u002Fdepth-anything-mono-large-f32.gguf --input photo.jpg --pfm depth.pfm --sky sky.pfm\n\n# Nested metric-scale depth (two GGUFs)\n$CLI depth --model nested-anyview.gguf --metric-model nested-metric.gguf --input photo.jpg --pfm metric.pfm\n\n# Multi-view depth + pose\n$CLI depth --model $M --input a.jpg --input b.jpg --out-prefix scene\n\n# 3D export from a single image\n$CLI depth --model $M --input photo.jpg --glb scene.glb --colmap colmap_out\u002F\n$CLI reconstruct --model models\u002Fdepth-anything-giant-f32.gguf --input photo.jpg --ply cloud.ply\n\n# Model metadata (arch, dims, head config, quant)\n$CLI info --model $M\n```\n\nThe `.glb` (glTF 2.0) and COLMAP `cameras\u002Fimages\u002Fpoints3D` writers are dependency-free (no trimesh \u002F pycolmap) and parity-checked against the reference exporters. See [`docs\u002FEXPORT.md`](docs\u002FEXPORT.md).\n\n---\n\n## Use it from LocalAI\n\ndepth-anything.cpp ships as a native [LocalAI](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI) backend (Go gRPC + purego, the `.so` static-links ggml - no external `libggml`). It exposes a typed **`Depth`** gRPC RPC and a **`POST \u002Fv1\u002Fdepth`** REST endpoint returning the *full* output - per-pixel depth, confidence, sky, extrinsics\u002Fintrinsics, the 3D point cloud, and glb\u002FCOLMAP exports - not just a normalized PNG.\n\n```sh\n# once the backend + models are in the gallery:\nlocal-ai run depth-anything-3-base\n```\n\n```sh\n# REST: full depth field + pose + points in one typed response\ncurl http:\u002F\u002Flocalhost:8080\u002Fv1\u002Fdepth -H 'Content-Type: application\u002Fjson' -d '{\n  \"model\": \"depth-anything-3-base\",\n  \"src\": \"photo.jpg\",\n  \"include_depth\": true, \"include_pose\": true, \"include_points\": true\n}'\n```\n\nGallery entries cover base (q4_k\u002Fq8_0\u002Ff16\u002Ff32), small, large, giant, and mono-large. The LocalAI backend lives in the [LocalAI repo](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI) under `backend\u002Fgo\u002Fdepth-anything-cpp\u002F`.\n\n---\n\n## C API\n\nA flat C ABI (`include\u002Fda_capi.h`, `abi_version` 4) over `libdepthanything.so`:\n\n```c\nda_ctx* ctx = da_capi_load(\"model.gguf\", \u002F*threads*\u002F 8);\n\u002F\u002F nested metric model (two branches): da_capi_load_nested(anyview, metric, threads)\nint h, w, is_metric;\nfloat *depth, *conf, *sky, ext[12], intr[9];\nda_capi_depth_dense(ctx, \"photo.jpg\", &h, &w, &depth, &conf, &sky, ext, intr, &is_metric);\n\u002F\u002F ... use depth[h*w], conf, pose ...\nda_capi_free_floats(depth);\n\nint n; float *xyz; unsigned char *rgb;\nda_capi_points(ctx, \"photo.jpg\", \u002F*conf_thresh*\u002F 1.0f, &n, &xyz, &rgb);   \u002F\u002F 3D cloud\nda_capi_export_glb(ctx, \"photo.jpg\", \"scene.glb\");\nda_capi_free(ctx);\n```\n\nOpaque handles, C types only, `da_capi_last_error` for diagnostics. Build it with `-DDA_SHARED=ON`.\n\n---\n\n## Parity & tests\n\nThe port is verified **numerically equal to the reference**, per component, not just end-to-end. The ctest suite (`-DDA_BUILD_TESTS=ON`, 37 tests) gates the preprocessing, backbone, attention, DPT head, depth, pose, ray head, ray→pose solver, exporters, and the C API against PyTorch-dumped tensors. End-to-end depth correlates 1.0 with the real DA3 forward; the full verification and parity numbers are in [`docs\u002FVERIFICATION.md`](docs\u002FVERIFICATION.md).\n\n```sh\ncmake -B build -DDA_BUILD_TESTS=ON && cmake --build build -j && ctest --test-dir build\n```\n\n---\n\n## Why depth-anything.cpp\n\nDepth Anything 3 is a great model, but running it for inference drags in a heavy Python\u002FPyTorch\u002FCUDA stack. This is a clean C++17\u002Fggml port focused purely on inference:\n\n- **No Python at inference.** A single `libdepthanything.so` behind a flat C API, easy to embed from C, C++, Go, or Rust.\n- **Faster than PyTorch on CPU** (1.20x f32, 1.31x q8_0), in **half the memory**, with **~6.7x faster load** and **bit-exact** output.\n- **Small and portable.** Self-contained GGUF with f16 \u002F q8_0 \u002F K-quants, on CPU and any ggml GPU backend.\n- **The whole model.** Depth, confidence, pose, sky, ray-pose, 3D point cloud, and glb\u002FCOLMAP\u002FPLY export, all parity-checked, across the full DA3 model family.\n\n---\n\n## Citation\n\nIf you use depth-anything.cpp, please cite this repository and the original model:\n\n```bibtex\n@software{depth_anything_cpp,\n  title  = {depth-anything.cpp: a C++\u002Fggml inference engine for Depth Anything 3},\n  author = {Di Giacinto, Ettore and Palethorpe, Richard},\n  url    = {https:\u002F\u002Fgithub.com\u002Fmudler\u002Fdepth-anything.cpp},\n  year   = {2026}\n}\n```\n\nDepth Anything 3 is by ByteDance Seed ([bytedance-seed\u002Fdepth-anything-3](https:\u002F\u002Fgithub.com\u002Fbytedance-seed\u002Fdepth-anything-3)).\n\n## Author\n\nEttore Di Giacinto ([@mudler](https:\u002F\u002Fgithub.com\u002Fmudler)).\n\n## License\n\ndepth-anything.cpp is released under the [MIT License](LICENSE). The Depth Anything 3 model weights are governed by their original license (Apache-2.0) - check each model card on HuggingFace.\n\n---\n\nBuilt by the [LocalAI](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI) team. If you want to run depth (and LLMs, vision, voice, image, and video models) locally on any hardware with an OpenAI-compatible API, [give LocalAI a star](https:\u002F\u002Fgithub.com\u002Fmudler\u002FLocalAI).\n","这是一个基于C++17和ggml框架实现的Depth Anything V3模型轻量级推理引擎，专注于单目图像的度量深度估计与相机位姿联合求解。核心功能包括生成像素级度量深度图、置信度图、天空掩膜、相机内外参矩阵、3D点云及GLB\u002FCOLMAP\u002FPLY格式导出；支持ViT-S\u002FB\u002FL\u002FG全系列模型及多种量化精度（如q4_k仅99MB），完全脱离Python\u002FPyTorch\u002FCUDA依赖，纯CPU运行且速度优于原PyTorch CPU实现。适用于边缘设备部署、机器人SLAM预处理、AR场景重建及无需GPU的离线三维视觉分析等场景。",2,"2026-07-08 04:30:11","CREATED_QUERY"]