[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-79907":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":14,"stars7d":14,"stars30d":13,"stars90d":15,"forks30d":15,"starsTrendScore":16,"compositeScore":17,"rankGlobal":9,"rankLanguage":9,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":19,"hasPages":19,"topics":21,"createdAt":9,"pushedAt":9,"updatedAt":22,"readmeContent":23,"aiSummary":24,"trendingCount":15,"starSnapshotCount":15,"syncStatus":14,"lastSyncTime":25,"discoverSource":26},79907,"deepreefmap","eceo-epfl\u002Fdeepreefmap","eceo-epfl","Tools for reconstructing and mapping coral reef scenes from underwater images",null,"Python",94,9,4,2,0,6,3,"Apache License 2.0",false,"main",[],"2026-06-12 02:03:55","# DeepReefMap\n\n[DeepReefMap](https:\u002F\u002Fbesjournals.onlinelibrary.wiley.com\u002Fdoi\u002Ffull\u002F10.1111\u002F2041-210X.14307) is a software for rapid 3D semantic mapping of coral reefs from handheld cameras. \nRepository maintained by [Hugues Sibille](https:\u002F\u002Fgithub.com\u002FHuguesSib) (EPFL) and [Jonathan Sauder](https:\u002F\u002Fjosauder.github.io\u002F) (MIT\u002FEPFL).\n\n![DeepReefMap 3D viewer](assets\u002Fdeepreefmap_view_3d_2x.gif)\n\n## What you get\n\nFrom one input video, a run produces:\n\n- A semantic 3D point cloud of the reef (`.ply`) - that you can open in Meshlab or CloudCompare.\n- An ortho-mosaic image (`ortho.png`)\n- Benthic cover statistics per class (`benthic_cover.json`)\n- An interactive 3D viewer to inspect the result\n\n## Quickstart\n\nExample input clip (10s GoPro Hero 10, Linear mode):\n\n![Example input clip](assets\u002Fdemo_input.gif)\n\nGet a first reconstruction running in three commands. This uses the lightest backend (`scsfmlearner`) and the bundled GoPro Hero 10 profile.\n\n```bash\n# 1. Install\nuv sync\n\n# 2. Run a reconstruction\nuv run deepreefmap reconstruct \\\n  --videos assets\u002Fdemo_input.mp4 \\\n  --camera-profile gopro_hero_10 \\\n  --mapping scsfmlearner \\\n  --out out \\\n  --viser\n\n# 3. Reopen the interactive viewer later\nuv run deepreefmap view-run --run-dir out --viser-port 8080\n```\n\nDon't have a GoPro Hero 10? See [Camera setup](#camera-setup-and-calibration) to calibrate your own. \nWant better quality? See the [LoGeR backend](#loger-higher-quality-more-setup).\n\n## How it works\n\nAt a high level, a run does four things:\n\n1. Reads one or more videos in order.\n2. Rectifies frames using a camera profile.\n3. Runs semantic segmentation and depth\u002Fpose reconstruction.\n4. Exports point clouds, ortho products, and reports.\n\n## Requirements\n\n- Python 3.10, 3.11, or 3.12\n- `[uv](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F)` for dependency management\n- FFmpeg (pulled in via `imageio[ffmpeg]`)\n- **GPU**: strongly recommended. CPU-only runs work with `scsfmlearner` but are slow. The `loger` \u002F `loger_star` backends require CUDA.\n\n## Installation\n\n```bash\nuv sync\n```\n\nOptional extras:\n\n```bash\nuv sync --extra gopro --extra train\n```\n\n### Choose a reconstruction backend\n\nTo run `deepreefmap reconstruct`, you need at least one reconstruction backend:\n\n- `scsfmlearner`: easiest to start with, no LoGeR checkpoint setup, but poorer reconstruction quality.\n- `loger` (or `loger_star`): higher quality reconstruction, but requires CUDA + GPU and checkpoint download.\n\nImportant performance note:\n\n- Without a GPU, all reconstruction backends will be slow.\n- LoGeR specifically requires CUDA and a compatible GPU.\n\n### SC-SfMLearner path (simplest)\n\nUse `--mapping scsfmlearner`. By default, the checkpoint is downloaded from Hugging Face (`EPFL-ECEO\u002Fdeepreefmap-sfm-net\u002Fscsfmlearner.pt`).\n\n```bash\nuv run deepreefmap reconstruct \\\n  --videos GX010001.MP4 \\\n  --mapping scsfmlearner \\\n  --camera-profile gopro_hero_10 \\\n  --tsdf \\\n  --out out_scsfm\n```\n\n### LoGeR path (higher quality, more setup)\n\nLoGeR upstream (`https:\u002F\u002Fgithub.com\u002FJunyi42\u002FLoGeR`) is vendored as a submodule at `third_party\u002FLoGeR`.\n\nInstall dependencies and initialize submodule:\n\n```bash\ngit submodule update --init --recursive\nuv sync --extra loger\n\n# Download checkpoints\ncurl -L -C - \"https:\u002F\u002Fhuggingface.co\u002FJunyi42\u002FLoGeR\u002Fresolve\u002Fmain\u002FLoGeR\u002Flatest.pt?download=true\" \\\n  -o third_party\u002FLoGeR\u002Fckpts\u002FLoGeR\u002Flatest.pt\ncurl -L -C - \"https:\u002F\u002Fhuggingface.co\u002FJunyi42\u002FLoGeR\u002Fresolve\u002Fmain\u002FLoGeR_star\u002Flatest.pt?download=true\" \\\n  -o third_party\u002FLoGeR\u002Fckpts\u002FLoGeR_star\u002Flatest.pt\n```\n\nAnd then you can run:\n\n```bash\nuv run deepreefmap reconstruct \\\n  --videos GX010001.MP4 \\\n  --mapping loger_star \\\n  --camera-profile gopro_hero_10 \\\n  --out out_loger \\\n```\n\n- **DINOv3-based** (`coralscapes-vit-*-dpt`): higher quality, **requires Hugging Face authentication** (gated models).\n- **SegFormer**: lighter and faster, no authentication needed.\n\nSelect with `--segmentation \u003Cmodel_name>`. List all available models:\n\n```bash\nuv run deepreefmap list-models\n```\n\n### Using DINOv3 models (authentication)\n\n1. Request access on Hugging Face: see [gated model docs](https:\u002F\u002Fhuggingface.co\u002Fdocs\u002Fhub\u002Fmodels-gated).\n2. Authenticate locally:\n\n```bash\nuv run huggingface-cli login\n```\n\n## Camera setup and calibration\n\n### GoPro Hero 10 in Linear mode with GoPro casing\n\nIf your footage is from a GoPro Hero 10 in Linear mode with the GoPro casing setup used by this project, use the built-in profile:\n\n- Camera profile: `gopro_hero_10` (bundled JSON: `deepreefmap\u002Fresources\u002Fcamera_profiles\u002Fgopro_hero_10.json`). You can also override or add profiles with `.\u002Fcamera_profiles\u002F\u003Cname>.json` in the current working directory.\n\nExample:\n\n```bash\nuv run deepreefmap reconstruct \\\n  --videos GX010001.MP4 \\\n  --segmentation coralscapes-vit-b-dpt \\\n  --camera-profile gopro_hero_10 \\\n  --mapping scsfmlearner \\\n  --out out\n```\n\n## Camera setup and calibration\n\n### Bundled profile: GoPro Hero 10 (Linear mode, GoPro casing)\n\nIf your footage matches this setup, use `--camera-profile gopro_hero_10` (bundled at `deepreefmap\u002Fresources\u002Fcamera_profiles\u002Fgopro_hero_10.json`). You can also drop your own profile at `.\u002Fcamera_profiles\u002F\u003Cname>.json` in the working directory.\n\n### Calibrating a different camera\n\nRun a calibration clip through the built-in COLMAP-based calibrator:\n\n```bash\nuv run deepreefmap calibrate \u002Fpath\u002Fto\u002Fnew_video.mp4 \\\n  --name my_new_camera \\\n  --n-frames 120 \\\n  --fps 8 \\\n  --begin 30.0 \\\n  --end 120.0\n```\n\nTips for a good calibration:\n\n- Pick a clip with **strong camera translation** (moving through the scene), not mostly rotation — COLMAP needs parallax.\n- Use `--begin` \u002F `--end` to trim to the cleanest section.\n\nValidate, then use it:\n\n```bash\nuv run deepreefmap verify-calibration my_new_camera\n\nuv run deepreefmap reconstruct \\\n  --videos \u002Fpath\u002Fto\u002Fnew_video.mp4 \\\n  --camera-profile my_new_camera \\\n  --mapping loger \\\n  --out out_new_camera\n```\n\n## Outputs\n\nEach run writes:\n\n- `frames\u002F`, `labels\u002F`, `masks\u002F` — rectified frames, semantic labels, keep masks.\n- `mapping_outputs.npz` — depth, poses, intrinsics, confidence, frame indices.\n- `semantic_reference_cloud.ply` — filtered semantic point cloud.\n- `tsdf_cloud.ply`, `semantic_tsdf_cloud.ply` — when `--tsdf` is enabled.\n- `ortho.png`, `ortho.npz` — aggregated ortho products.\n- `benthic_cover.json` — class counts and cover fractions.\n- `geometry_cloud.ply` — geometry-only cloud (when `--skip-segmentation`).\n- `run_manifest.json` — canonical run manifest (`semantic` or `geometry_only`).\n\n## Interactive viewer (viser)\n\nLive during reconstruction with `--viser`, or open an existing run:\n\n```bash\nuv run deepreefmap view-run --run-dir out --viser-port 8080\n```\n\nIn the viewer you can:\n\n- Click a camera frustum to jump to that point in the timeline.\n- Inspect RGB, segmentation, and depth per frame.\n- Toggle class visibility and switch between RGB and semantic colors.\n- Use **Accumulate** to overlay filtered points up to the current timeline index.\n\n## CLI reference\n\n```bash\nuv run deepreefmap list-models           # available segmentation + mapping models\nuv run deepreefmap list-profiles         # available camera profiles\nuv run deepreefmap reconstruct ...       # main pipeline\nuv run deepreefmap calibrate VIDEO ...   # camera calibration via COLMAP\nuv run deepreefmap verify-calibration NAME\nuv run deepreefmap render-video --run-dir out\nuv run deepreefmap view-run --run-dir out --viser-port 8080\n```\n\nUseful `reconstruct` flags:\n\n- `--grid-bins`: ortho aggregation resolution.\n- `--keep-viser-open` \u002F `--no-keep-viser-open`: keep viewer running after processing.\n- `--require-gravity-telemetry`: fail if gravity telemetry cannot be loaded\u002Faligned.\n- `--preprocess-batch-size`: segmentation batch size during frame preparation.\n- `--transect-length` and `--transect-crop-width`: crop outputs around dominant transect.\n- `--skip-segmentation`: geometry-only run (no semantics).\n\n## Reconstruction outputs\n\nEach run writes cached and derived artifacts:\n\n- `frames\u002F`, `labels\u002F`, `masks\u002F`: rectified frames, semantic labels, and keep masks.\n- `mapping_outputs.npz`: depth, poses, intrinsics, confidence, frame indices.\n- `semantic_reference_cloud.ply`: filtered semantic point cloud.\n- `tsdf_cloud.ply` and `semantic_tsdf_cloud.ply`: optional TSDF outputs when `--tsdf` is enabled.\n- `ortho.png` and `ortho.npz`: aggregated ortho products.\n- `benthic_cover.json`: class counts and cover fractions.\n- `geometry_cloud.ply`: geometry-only cloud from `--skip-segmentation`.\n- `run_manifest.json`: canonical run manifest (`semantic` or `geometry_only`).\n\n## Viser app (interactive viewer)\n\nYou can use live viewing during reconstruction (`--viser`) or open an existing run:\n\n```bash\nuv run deepreefmap view-run --run-dir out --viser-port 8080\n```\n\nViewer highlights:\n\n- Click a camera frustum to jump timeline.\n- Inspect RGB, segmentation, and depth for each frame.\n- Toggle class visibility and switch color mode (RGB vs semantic colors).\n- Use `Accumulate` to overlay filtered points up to current timeline index.\n\n## Citation\n\nIf you use this repository or build on it, please cite DeepReefMap:\n\n```bibtex\n@article{sauder2024scalable,\n  title={Scalable semantic 3D mapping of coral reefs with deep learning},\n  author={Sauder, Jonathan and Banc-Prandi, Guilhem and Meibom, Anders and Tuia, Devis},\n  journal={Methods in Ecology and Evolution},\n  volume={15},\n  number={5},\n  pages={916--934},\n  year={2024},\n  publisher={Wiley Online Library}\n}\n```\n\nThe segmentation models are trained on the [Coralscapes](https:\u002F\u002Fjosauder.github.io\u002Fcoralscapes\u002F) dataset. If you use them, please cite\n\n```bibtex\n@inproceedings{sauder2025coralscapes,\n  title={The Coralscapes Dataset: Semantic scene understanding in coral reefs},\n  author={Sauder, Jonathan and Domazetoski, Viktor and Banc-Prandi, Guilhem and Perna, Gabriela and Meibom, Anders and Tuia, Devis},\n  booktitle={ICCV Joint Workshop on Marine Vision},\n  year={2025}\n}\n```\n\nIf you use the **LoGeR** backend (`--mapping loger` or `loger_star`), please also cite:\n\n```bibtex\n@article{zhang2026loger,\n  title={LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory},\n  author={Zhang, Junyi and Herrmann, Charles and Hur, Junhwa and Sun, Chen and Yang, Ming-Hsuan and Cole, Forrester and Darrell, Trevor and Sun, Deqing},\n  journal={arXiv preprint arXiv:2603.03269},\n  year={2026}\n}\n```\n\n## Acknowledgements\n\nDeepReefMap builds on:\n\n- [LoGeR](https:\u002F\u002Fgithub.com\u002FJunyi42\u002FLoGeR) by Zhang et al. — high-quality reconstruction backend.\n- [viser](https:\u002F\u002Fgithub.com\u002Fnerfstudio-project\u002Fviser) — interactive 3D viewer.\n\n## License\n\nDeepReefMap is licensed under the [Apache License 2.0](LICENSE).\n\nVendored or optional third-party components (notably `third_party\u002FLoGeR` and downloaded checkpoints) carry their own terms; see `THIRD_PARTY_NOTICES.md` before redistribution.","DeepReefMap 是一个用于从水下图像重建和绘制珊瑚礁场景的软件。其核心功能包括通过手持相机视频生成语义3D点云、正射影像图以及生物覆盖统计，并提供一个交互式3D查看器来检查结果。项目采用Python开发，支持多种后端进行深度\u002F姿态重建与语义分割，推荐使用GPU以获得更高效的处理速度。适用于海洋生物学研究、环境保护监测及教育展示等场景。","2026-06-11 03:58:29","CREATED_QUERY"]