[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72300":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":23,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":35,"readmeContent":36,"aiSummary":37,"trendingCount":16,"starSnapshotCount":16,"syncStatus":38,"lastSyncTime":39,"discoverSource":40},72300,"map-anything","facebookresearch\u002Fmap-anything","facebookresearch","MapAnything: Universal Feed-Forward Metric 3D Reconstruction","",null,"Python",3475,260,33,28,0,14,45,123,42,109.25,"Apache License 2.0",false,"main",[26,27,28,29,30,31,32,33,34],"3d-reconstruction","ai","calibration","depth-completion","depth-estimation","image-to-3d","multi-view-stereo","robotics","sfm","2026-06-12 04:01:04","\u003Cdiv align=\"center\">\n\u003Ch1>MapAnything: Universal Feed-Forward Metric \u003Cbr>3D Reconstruction\u003C\u002Fh1>\n\u003Ca href=\"https:\u002F\u002Fmap-anything.github.io\u002Fassets\u002FMapAnything.pdf\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPaper-blue\" alt=\"Paper\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.13414\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2509.13414-b31b1b\" alt=\"arXiv\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fmap-anything.github.io\u002F\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject_Page-green\" alt=\"Project Page\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fx.com\u002FNik__V__\u002Fstatus\u002F1968316841618518371\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FX_Thread-1DA1F2\" alt=\"X Thread\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Ffacebook\u002Fmap-anything\">\u003Cimg src='https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F%F0%9F%A4%97%20Hugging%20Face-Demo-blue'>\u003C\u002Fa>\n\u003Cbr>\n\u003Cbr>\n\u003Cstrong>\n\u003Ca href=\"https:\u002F\u002Fnik-v9.github.io\u002F\">Nikhil Keetha\u003Csup>1,2\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fsirwyver.github.io\u002F\">Norman Müller\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fdemuc.de\u002F\">Johannes Schönberger\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Florenzoporzi\">Lorenzo Porzi\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Finfinity1096.github.io\u002F\">Yuchen Zhang\u003Csup>2\u003C\u002Fsup>\u003C\u002Fa>\n\u003Cbr>\n\u003Ca href=\"https:\u002F\u002Ftobiasfshr.github.io\u002F\">Tobias Fischer\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Farno-knapitsch\">Arno Knapitsch\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fduncan-zauss\">Duncan Zauss\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fethanweber.me\u002F\">Ethan Weber\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fnelsonantunes7\">Nelson Antunes\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n\u003Cbr>\n\u003Ca href=\"https:\u002F\u002Fx.com\u002Fjonathonluiten?lang=en\">Jonathon Luiten\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fm.lopezantequera.com\u002F\">Manuel Lopez-Antequera\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fscholar.google.com\u002Fcitations?user=484sccEAAAAJ\">Samuel Rota Bulò\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Frichardt.name\u002F\">Christian Richardt\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n\u003Cbr>\n\u003Ca href=\"https:\u002F\u002Fwww.cs.cmu.edu\u002F~deva\u002F\">Deva Ramanan\u003Csup>2\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Ftheairlab.org\u002Fteam\u002Fsebastian\u002F\">Sebastian Scherer\u003Csup>2\u003C\u002Fsup>\u003C\u002Fa>\n  \n\u003Ca href=\"https:\u002F\u002Fwww.linkedin.com\u002Fin\u002Fpeter-kontschieder-2a6410134\">Peter Kontschieder\u003Csup>1\u003C\u002Fsup>\u003C\u002Fa>\n\u003Cbr>\n\u003Cbr>\n\u003Csup>1\u003C\u002Fsup> Meta   \n\u003Csup>2\u003C\u002Fsup> Carnegie Mellon University\n\u003C\u002Fstrong>\n\n\u003C\u002Fdiv>\n\n## Overview\n\nMapAnything is an **open-source research framework** for universal metric 3D reconstruction. At its core is a simple, end-to-end trained transformer model that directly regresses the factored metric 3D geometry of a scene given various types of inputs (images, calibration, poses, or depth). A single feed-forward model supports over 12 different 3D reconstruction tasks including multi-image sfm, multi-view stereo, monocular metric depth estimation, registration, depth completion and more.\n\nThe framework provides the **complete stack**—data processing, training, inference, and profiling—with a **modular design** that allows different 3D reconstruction models (VGGT, DUSt3R, MASt3R, MUSt3R, Pi3-X, and more) to be used interchangeably through a unified interface.\n\n![Overview](.\u002Fassets\u002Fteaser.png)\n\n## Table of Contents\n\n- [Overview](#overview)\n- [Quick Start](#quick-start)\n  - [Installation](#installation)\n  - [Image-Only Inference](#image-only-inference)\n  - [Multi-Modal Inference](#multi-modal-inference)\n  - [Running External Models](#running-external-models)\n    - [Available Models](#available-models)\n    - [Installation](#external-model-installation)\n    - [Quick Start Example](#external-model-quick-start)\n    - [Running Inference](#running-inference)\n    - [Unified Output Format](#unified-output-format)\n    - [Notes on Input Requirements](#notes-on-input-requirements)\n- [Interactive Demos](#interactive-demos)\n  - [Online Demo](#online-demo)\n  - [Local Gradio Demo](#local-gradio-demo)\n  - [Rerun Demo](#rerun-demo)\n  - [Demo Inference on COLMAP outputs](#demo-inference-on-colmap-outputs)\n- [Profiling](#profiling)\n  - [Profiling Results](#profiling-results)\n  - [Basic Profiling](#basic-profiling)\n  - [Comparing with External Models](#comparing-with-external-models)\n  - [Command-Line Arguments](#command-line-arguments)\n  - [Output Files](#output-files)\n- [COLMAP & GSplat Support](#colmap--gsplat-support)\n  - [Exporting to COLMAP Format](#exporting-to-colmap-format)\n  - [Visualizing COLMAP Reconstruction in Rerun](#visualizing-colmap-reconstruction-in-rerun)\n  - [Integration with Gaussian Splatting](#integration-with-gaussian-splatting)\n- [Data Processing for Training & Benchmarking](#data-processing-for-training--benchmarking)\n- [Training](#training)\n- [Benchmarking](#benchmarking)\n  - [Available Benchmarks](#available-benchmarks)\n- [Code License](#code-license)\n- [Models](#models)\n  - [Hugging Face Hub Models](#-hugging-face-hub-models)\n  - [Hugging Face Hub Models (V1 Release)](#-hugging-face-hub-models-used-for-v1-release-in-september-2025)\n  - [Model Selection Guide](#model-selection-guide)\n  - [Optional Checkpoint Conversion](#optional-checkpoint-conversion)\n- [Building Blocks for MapAnything](#building-blocks-for-mapanything)\n  - [Related Research](#related-research)\n- [Acknowledgments](#acknowledgments)\n- [Citation](#citation)\n\n## Quick Start\n\n### Installation\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fmap-anything.git\ncd map-anything\n\n# Create and activate conda environment\nconda create -n mapanything python=3.12 -y\nconda activate mapanything\n\n# Optional: Install torch, torchvision & torchaudio specific to your system\n# Install MapAnything\npip install -e .\n\n# For all optional dependencies\n# This includes external model support (VGGT, DUSt3R, MASt3R, MUSt3R, Pi3-X, DA3, etc.)\n# See \"Running External Models\" section for more details\n# See pyproject.toml for more details on installed packages\npip install -e \".[all]\"\npre-commit install\n```\n\nNote that we don't pin a specific version of PyTorch or CUDA in our requirements. Please feel free to install PyTorch based on your specific system.\n\n### Image-Only Inference\n\nFor metric 3D reconstruction from images without additional geometric inputs:\n\n```python\n# Optional config for better memory efficiency\nimport os\nos.environ[\"PYTORCH_CUDA_ALLOC_CONF\"] = \"expandable_segments:True\"\n\n# Required imports\nimport torch\nfrom mapanything.models import MapAnything\nfrom mapanything.utils.image import load_images\n\n# Get inference device\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n# Init model - This requires internet access or the huggingface hub cache to be pre-downloaded\n# For Apache 2.0 license model, use \"facebook\u002Fmap-anything-apache\"\nmodel = MapAnything.from_pretrained(\"facebook\u002Fmap-anything\").to(device)\n\n# Load and preprocess images from a folder or list of paths\nimages = \"path\u002Fto\u002Fyour\u002Fimages\u002F\"  # or [\"path\u002Fto\u002Fimg1.jpg\", \"path\u002Fto\u002Fimg2.jpg\", ...]\nviews = load_images(images)\n\n# Run inference\npredictions = model.infer(\n    views,                            # Input views\n    memory_efficient_inference=True,  # Trades off speed for more views (up to 2000 views on 140 GB). Trade off is negligible - see profiling section\n    minibatch_size=None,              # Minibatch size for memory-efficient inference (use 1 for smallest GPU memory consumption). Default is dynamic computation based on available GPU memory.\n    use_amp=True,                     # Use mixed precision inference (recommended)\n    amp_dtype=\"bf16\",                 # bf16 inference (recommended; falls back to fp16 if bf16 not supported)\n    apply_mask=True,                  # Apply masking to dense geometry outputs\n    mask_edges=True,                  # Remove edge artifacts by using normals and depth\n    apply_confidence_mask=False,      # Filter low-confidence regions\n    confidence_percentile=10,         # Remove bottom 10 percentile confidence pixels\n    use_multiview_confidence=False,   # Enable multi-view depth consistency based confidence in place of learning-based one\n)\n\n# Access results for each view - Complete list of metric outputs\nfor i, pred in enumerate(predictions):\n    # Geometry outputs\n    pts3d = pred[\"pts3d\"]                     # 3D points in world coordinates (B, H, W, 3)\n    pts3d_cam = pred[\"pts3d_cam\"]             # 3D points in camera coordinates (B, H, W, 3)\n    depth_z = pred[\"depth_z\"]                 # Z-depth in camera frame (B, H, W, 1)\n    depth_along_ray = pred[\"depth_along_ray\"] # Depth along ray in camera frame (B, H, W, 1)\n\n    # Camera outputs\n    ray_directions = pred[\"ray_directions\"]   # Ray directions in camera frame (B, H, W, 3)\n    intrinsics = pred[\"intrinsics\"]           # Recovered pinhole camera intrinsics (B, 3, 3)\n    camera_poses = pred[\"camera_poses\"]       # OpenCV (+X - Right, +Y - Down, +Z - Forward) cam2world poses in world frame (B, 4, 4)\n    cam_trans = pred[\"cam_trans\"]             # OpenCV (+X - Right, +Y - Down, +Z - Forward) cam2world translation in world frame (B, 3)\n    cam_quats = pred[\"cam_quats\"]             # OpenCV (+X - Right, +Y - Down, +Z - Forward) cam2world quaternion in world frame (B, 4)\n\n    # Quality and masking\n    confidence = pred[\"conf\"]                 # Per-pixel confidence scores (B, H, W)\n    mask = pred[\"mask\"]                       # Combined validity mask (B, H, W, 1)\n    non_ambiguous_mask = pred[\"non_ambiguous_mask\"]                # Non-ambiguous regions (B, H, W)\n    non_ambiguous_mask_logits = pred[\"non_ambiguous_mask_logits\"]  # Mask logits (B, H, W)\n\n    # Scaling\n    metric_scaling_factor = pred[\"metric_scaling_factor\"]  # Applied metric scaling (B,)\n\n    # Original input\n    img_no_norm = pred[\"img_no_norm\"]         # Denormalized input images for visualization (B, H, W, 3)\n```\n\n### Multi-Modal Inference\n\nMapAnything supports flexible combinations of geometric inputs for enhanced metric reconstruction. Steps to try it out:\n\n**Initialize the model:**\n\n```python\n# Optional config for better memory efficiency\nimport os\nos.environ[\"PYTORCH_CUDA_ALLOC_CONF\"] = \"expandable_segments:True\"\n\n# Required imports\nimport torch\nfrom mapanything.models import MapAnything\n\n# Get inference device\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n\n# Init model - This requires internet access or the huggingface hub cache to be pre-downloaded\n# For Apache 2.0 license model, use \"facebook\u002Fmap-anything-apache\"\nmodel = MapAnything.from_pretrained(\"facebook\u002Fmap-anything\").to(device)\n```\n\n**Initialize the inputs:**\n\n```python\n# MapAnything is extremely flexible and supports any combination of inputs.\nviews_example = [\n    {\n        # View 0: Images + Calibration\n        \"img\": image, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics, # (3, 3)\n    },\n    {\n        # View 1: Images + Calibration + Depth\n        \"img\": image, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics, # (3, 3)\n        \"depth_z\": depth_z, # (H, W)\n        \"is_metric_scale\": torch.tensor([True], device=device), # (1,)\n    },\n    {\n        # View 2: Images + Calibration + Depth + Pose\n        \"img\": image, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics, # (3, 3)\n        \"depth_z\": depth_z, # (H, W)\n        \"camera_poses\": camera_poses, # (4, 4) or tuple of (quats, trans) in OpenCV cam2world convention\n        \"is_metric_scale\": torch.tensor([True], device=device), # (1,)\n    },\n    ...\n]\n```\n\nNote that MapAnything expects the input camera poses to follow the OpenCV (+X - Right, +Y - Down, +Z - Forward) cam2world convention.\n\n\u003C!-- TODO: We provide a helper function `mapanything.utils.geometry.convert_cam_convention` to convert from other conventions to OpenCV (see the function in [geometry utils](.\u002Fmapanything\u002Futils\u002Fgeometry.py) for more details). -->\n\n\u003Cdetails>\n\u003Csummary>Expand to show more examples\u003C\u002Fsummary>\n\n```python\n# Example 1: Images + Camera Intrinsics\nviews_example = [\n    {\n        \"img\": image_tensor,  # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics_tensor,  # (3, 3)\n    },\n    ...\n]\n\n# Example 2: Images + Intrinsics + Depth\nviews_example = [\n    {\n        \"img\": image_tensor, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics_tensor, # (3, 3)\n        \"depth_z\": depth_tensor,  # (H, W)\n        \"is_metric_scale\": torch.tensor([True]), # (1,)\n    },\n    ...\n]\n\n# Example 3: Images + Intrinsics + Camera Poses\nviews_example = [\n    {\n        \"img\": image_tensor, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics_tensor, # (3, 3)\n        \"camera_poses\": pose_matrices,  # (4, 4) or tuple of (quats, trans) in OpenCV cam2world convention\n        \"is_metric_scale\": torch.tensor([True]), # (1,)\n    },\n    ...\n]\n\n# Example 4: Images + Ray Directions + Depth (alternative to intrinsics)\nviews_example = [\n    {\n        \"img\": image_tensor, # (H, W, 3) - [0, 255]\n        \"ray_directions\": ray_dirs_tensor,  # (H, W, 3)\n        \"depth_z\": depth_tensor, # (H, W)\n    }\n    ...\n]\n\n# Example 5: Full Multi-Modal (Images + Intrinsics + Depth + Poses)\nviews_example = [\n    {\n        \"img\": image_tensor, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics_tensor, # (3, 3)\n        \"depth_z\": depth_tensor, # (H, W)\n        \"camera_poses\": pose_matrices, # (4, 4) or tuple of (quats, trans) in OpenCV cam2world convention\n        \"is_metric_scale\": torch.tensor([True]), # (1,)\n    }\n    ...\n]\n\n# Example 6: Adaptive Mixed Inputs\nviews_example = [\n    {\n        # View 0: Images + Pose\n        \"img\": images, # (H, W, 3) - [0, 255]\n        \"camera_poses\": camera_poses, # (4, 4) or tuple of (quats, trans) in OpenCV cam2world convention\n    },\n    {\n        # View 1: Images + Calibration\n        \"img\": images, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics, # (3, 3)\n    },\n    {\n        # View 2: Images + Calibration + Depth\n        \"img\": images, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics, # (3, 3)\n        \"depth_z\": depth_z, # (H, W)\n        \"is_metric_scale\": torch.tensor([True], device=device), # (1,)\n    },\n    {\n        # View 3: Images + Calibration + Depth + Pose\n        \"img\": images, # (H, W, 3) - [0, 255]\n        \"intrinsics\": intrinsics, # (3, 3)\n        \"depth_z\": depth_z, # (H, W)\n        \"camera_poses\": camera_poses, # (4, 4) or tuple of (quats, trans) in OpenCV cam2world convention\n        \"is_metric_scale\": torch.tensor([True], device=device), # (1,)\n    },\n    ...\n]\n```\n\u003C\u002Fdetails>\n\n\u003Cbr>\n\n**Run model inference:**\n\n```python\nfrom mapanything.utils.image import preprocess_inputs\n\n# Preprocess inputs to the expected format\nprocessed_views = preprocess_inputs(views_example)\n\n# Run inference with any combination of inputs\npredictions = model.infer(\n    processed_views,                  # Any combination of input views\n    memory_efficient_inference=True, # Trades off speed for more views (up to 2000 views on 140 GB). Trade off is negligible - see profiling section\n    minibatch_size=None,              # Minibatch size for memory-efficient inference (use 1 for smallest GPU memory consumption). Default is dynamic computation based on available GPU memory.\n    use_amp=True,                     # Use mixed precision inference (recommended)\n    amp_dtype=\"bf16\",                 # bf16 inference (recommended; falls back to fp16 if bf16 not supported)\n    apply_mask=True,                  # Apply masking to dense geometry outputs\n    mask_edges=True,                  # Remove edge artifacts by using normals and depth\n    apply_confidence_mask=False,      # Filter low-confidence regions\n    confidence_percentile=10,         # Remove bottom 10 percentile confidence pixels\n    use_multiview_confidence=False,   # Enable multi-view depth consistency based confidence in place of learning-based one\n    # Control which inputs to use\u002Fignore\n    # By default, all inputs are used when provided\n    # If is_metric_scale flag is not provided, all inputs are assumed to be in metric scale\n    ignore_calibration_inputs=False,\n    ignore_depth_inputs=False,\n    ignore_pose_inputs=False,\n    ignore_depth_scale_inputs=False,\n    ignore_pose_scale_inputs=False,\n)\n\n# Access results for each view - Complete list of metric outputs\n# Outputs are the same as above (image-only input case), but with additional inputs used\n```\n\n**Input requirements for `model.infer`:**\n- `img`: RGB images normalized according to `data_norm_type`\n- `data_norm_type`: Normalization type (must match model's encoder requirements)\n\n**Optional geometric inputs supported by `model.infer`:**\n- `intrinsics` OR `ray_directions`: Camera calibration (cannot provide both since they are redundant)\n- `depth_z`: Z-depth maps (requires calibration info)\n- `camera_poses`: OpenCV (+X - Right, +Y - Down, +Z - Forward) cam2world poses as 4×4 matrices or (quaternions, translations)\n- `is_metric_scale`: Whether inputs are in metric scale\n\n**Key constraints for `model.infer`:**\n- If `depth_z` is provided, must also provide `intrinsics` or `ray_directions`\n- If any view has `camera_poses`, the first view (reference) must also have them\n- Cannot provide both `intrinsics` and `ray_directions` simultaneously (they are redundant)\n\nThe above constraints are enforced in the inference API. However, if desired, the underlying `model.forward` can support any arbitrary combination of inputs (a total of 64 configurations; without counting per view flexibility).\n\n### Running External Models\n\nThe MapAnything codebase is **modular** — different 3D reconstruction models can be used interchangeably through a unified model factory interface. All model wrappers produce outputs in a **unified format** following MapAnything conventions (`pts3d`, `pts3d_cam`, `ray_directions`, `depth_along_ray`, `cam_trans`, `cam_quats`, `conf`, etc.). This enables fair comparison, benchmarking, and easy experimentation across methods.\n\n#### Available Models\n\n| Model Key | Description | Resolution | Data Norm Type | Install Extra |\n|-----------|-------------|------------|----------------|---------------|\n| `mapanything` | MapAnything | 518 | `dinov2` | (base) |\n| `mapanything_ablations` | MapAnything ablations | 518 | `dinov2` | (base) |\n| `modular_dust3r` | ModularDUSt3R | 512 | `dust3r` | (base) |\n| `vggt` | VGGT 1B | 518 | `identity` | (base) |\n| `dust3r` | DUSt3R + Global BA | 512 | `dust3r` | `dust3r` |\n| `mast3r` | MASt3R + SGA | 512 | `dust3r` | `mast3r` |\n| `moge` | MoGe | 518 | `identity` | (base) |\n| `must3r` | MUSt3R | 512 | `dust3r` | `must3r` |\n| `pi3` | π³ | 518 | `identity` | (base) |\n| `pi3x` | π³-X | 518 | `identity` | `pi3` |\n| `pow3r` | Pow3R | 512 | `dust3r` | `pow3r` |\n| `pow3r_ba` | Pow3R + BA | 512 | `dust3r` | `pow3r` |\n| `anycalib` | AnyCalib | - | - | `anycalib` |\n| `da3` | Depth Anything 3 | 504 | `dinov2` | `depth-anything-3` |\n\n#### Installation {#external-model-installation}\n\nInstall optional dependencies for external models:\n\n```bash\n# Install specific external model dependencies\npip install -e \".[dust3r]\"           # DUSt3R\npip install -e \".[mast3r]\"           # MASt3R\npip install -e \".[pi3]\"              # π³-X (note: π³ base works without this)\npip install -e \".[pow3r]\"            # Pow3R\npip install -e \".[anycalib]\"         # AnyCalib\npip install -e \".[must3r]\"           # MUSt3R\npip install -e \".[depth-anything-3]\" # Depth Anything 3\n\n# Or install all external model dependencies\npip install -e \".[all]\"\n```\n\n#### Quick Start Example {#external-model-quick-start}\n\n**Example 1: Using `init_model_from_config` (recommended)**\n\n```python\nfrom mapanything.models import init_model_from_config\n\n# Initialize model by name - handles Hydra config composition automatically\n# Options are based on configs available in configs\u002Fmodel\n# The model is returned on the specified device\nmodel = init_model_from_config(\"vggt\", device=\"cuda\")\n\n# Other examples:\n# model = init_model_from_config(\"pi3x\", device=\"cuda\")\n# model = init_model_from_config(\"da3_nested\", device=\"cuda\")\n# Note: For MUSt3R, the values in configs\u002Fmachine\u002Fdefault.yaml need to be populated to enable checkpoint loading\n# model = init_model_from_config(\"must3r\", device=\"cuda\")\n```\n\n**Example 2: Using `model_factory` directly**\n\n```python\nfrom mapanything.models import model_factory, get_available_models\n\n# List all available models\nprint(get_available_models())\n# ['mapanything', 'mapanything_ablations', 'modular_dust3r', 'anycalib',\n#  'da3', 'dust3r', 'mast3r', 'moge', 'must3r', 'pi3', 'pi3x', 'pow3r',\n#  'pow3r_ba', 'vggt']\n\n# Initialize external model\n# Requires passing in additional model config arguments as kwargs\n# model = model_factory(\"model_name\", **model_config_kwargs)\nmodel = model_factory(\"vggt\", name=\"vggt\", torch_hub_force_reload=False)\nmodel = model.to(\"cuda\")\n```\n\n#### Running Inference\n\nAll model wrappers follow the same `forward()` interface. The model expects a list of view dictionaries as input and returns a list of prediction dictionaries:\n\n```python\nimport torch\nfrom mapanything.utils.image import load_images\n\n# Load and preprocess images from a folder\n# This handles resizing and normalization based on model requirements\nviews = load_images(\n    folder_or_list=\"path\u002Fto\u002Fimages\",  # Folder path or list of image paths\n    resolution_set=518,               # Model-specific resolution (see table above)\n    norm_type=\"dinov2\",               # Model-specific normalization (see table above)\n    patch_size=14,\n)\n\n# Run inference\nmodel.eval()\nwith torch.no_grad():\n    with torch.autocast(\"cuda\"):\n        predictions = model(views)\n\n# predictions is a list of dicts, one per input view\n# See the Unified Output Format section below for more details\n```\n\nEach input view dictionary must contain at minimum an `img` tensor of shape `(B, 3, H, W)` with the appropriate normalization applied. The `load_images` utility handles this preprocessing automatically. For more advanced inference patterns including loss computation and device handling, see the `loss_of_one_batch_multi_view` function in `mapanything\u002Futils\u002Finference.py`.\n\n#### Unified Output Format\n\nAll model wrappers produce outputs in a consistent format:\n\n| Output Key | Description |\n|------------|-------------|\n| `pts3d` | 3D points in world coordinates |\n| `pts3d_cam` | 3D points in camera coordinates |\n| `ray_directions` | Ray directions in camera frame |\n| `depth_along_ray` | Depth along ray |\n| `cam_trans` | Camera translation (cam2world) |\n| `cam_quats` | Camera quaternion (cam2world) |\n| `conf` | Per-pixel confidence scores |\n\nThis unified output format allows:\n- Fair comparison between methods\n- Easy swapping of models for experimentation\n- Consistent downstream processing pipelines\n\n#### Notes on Input Requirements\n\nDifferent models have different input requirements:\n\n**Resolution - Longest Side**:\n- 518px: MapAnything, VGGT, MoGe, Pi3, Pi3X\n- 512px: DUSt3R, MASt3R, MUSt3R, Pow3R\n- 504px: Depth Anything 3\n\n**Data Normalization (`data_norm_type`)**:\n- `dinov2`: MapAnything, Depth Anything 3\n- `identity`: VGGT, MoGe, Pi3, Pi3X\n- `dust3r`: DUSt3R, MASt3R, MUSt3R, Pow3R\n\nFor training and fine-tuning external models, see the [Training README](train.md) for detailed instructions.\n\n## Interactive Demos\n\nWe provide multiple interactive demos to try out MapAnything!\n\n### Online Demo\n\nTry our online demo without installation: [🤗 Hugging Face Demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Ffacebook\u002Fmap-anything)\n\n### Local Gradio Demo\n\nWe provide a script to launch our Gradio app. The interface and GUI mirrors our [online demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002Ffacebook\u002Fmap-anything) where you can upload images\u002Fvideos, run reconstruction and interactively view them. You can launch this using:\n\n```bash\n# Install requirements for the app\npip install -e \".[gradio]\"\n\n# Launch app locally\npython scripts\u002Fgradio_app.py\n```\n\n\u003Cdetails>\n\u003Csummary>Expand to preview the Gradio demo interface\u003C\u002Fsummary>\n\n![Gradio Interface Preview](.\u002Fassets\u002Fgradio_example.png)\n\u003C\u002Fdetails>\n\n### Rerun Demo\n\nWe provide a demo script for interactive 3D visualization of metric reconstruction results using [Rerun](https:\u002F\u002Frerun.io\u002F).\n\n```bash\n# Terminal 1: Start the Rerun server\nrerun --serve --port 2004 --web-viewer-port 2006\n\n# Terminal 2: Run MapAnything demo\npython scripts\u002Fdemo_images_only_inference.py \\\n    --image_folder \u002Fpath\u002Fto\u002Fyour\u002Fimages \\\n    --viz \\\n    --save_glb \\\n    --output_path \u002Fpath\u002Fto\u002Foutput.glb\n\n# Terminal 3 or Local Machine: Open web viewer at http:\u002F\u002F127.0.0.1:2006 (You might need to port forward if using a remote server)\n```\n\nAdditional options:\n- `--apache`: Use the Apache 2.0 licensed model\n- `--video_viz_for_rerun`: Enable video-style visualization with time indexing\n- `--log_only_imgs_for_rerun_cams`: Log only images for Rerun cameras (skip depth\u002Fmask)\n\nOptionally, if rerun is installed locally, local rerun viewer can be spawned using: `rerun --connect rerun+http:\u002F\u002F127.0.0.1:2004\u002Fproxy`.\n\n### Demo Inference on COLMAP outputs\n\nWe provide a demo script to run MapAnything inference on COLMAP outputs. The script runs MapAnything in MVS mode by default (using COLMAP calibration and poses as input). Use the `--help` flag for more info.\n\n```bash\n# Terminal 1: Start the Rerun server\nrerun --serve --port 2004 --web-viewer-port 2006\n\n# Terminal 2: Run MapAnything inference on COLMAP output folder\npython scripts\u002Fdemo_inference_on_colmap_outputs.py \\\n    --colmap_path \u002Fpath\u002Fto\u002Fyour\u002Fcolmap_output \\\n    --viz\n\n# Terminal 3 or Local Machine: Open web viewer at http:\u002F\u002F127.0.0.1:2006 (You might need to port forward if using a remote server)\n```\n\nAdditional options:\n- `--apache`: Use the Apache 2.0 licensed model\n- `--stride N`: Load every Nth image (default: 1)\n- `--ext .bin\u002F.txt`: COLMAP file extension (default: .bin)\n- `--ignore_calibration_inputs`: Ignore COLMAP calibration (use only images and poses)\n- `--ignore_pose_inputs`: Ignore COLMAP poses (use only images and calibration)\n- `--save_colmap`: Export results in COLMAP format\n- `--save_glb`: Save reconstruction as GLB file\n- `--output_directory`: Output directory for COLMAP or GLB exports (default: colmap_mapanything_output)\n- `--verbose`: Enable verbose loading output\n\nOptionally, if rerun is installed locally, local rerun viewer can be spawned using: `rerun --connect rerun+http:\u002F\u002F127.0.0.1:2004\u002Fproxy`.\n\n## Profiling\n\nProfile GPU memory usage and inference speed of MapAnything across different view counts. The profiling script supports comparison with external models and outputs both JSON results and visualizations.\n\n### Profiling Results\n\nMapAnything achieves the best speed and memory profile compared to existing methods, enabling efficient inference across a wide range of view counts.\n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd>\u003Cimg src=\".\u002Fassets\u002Fprofiling_memory.png\" alt=\"Memory Profiling\" width=\"100%\">\u003C\u002Ftd>\n\u003Ctd>\u003Cimg src=\".\u002Fassets\u002Fprofiling_speed.png\" alt=\"Speed Profiling\" width=\"100%\">\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd align=\"center\">\u003Cb>Peak GPU Memory vs Number of Views\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>Inference Speed vs Number of Views\u003C\u002Fb>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n**Note on Memory Efficient Mode:** MapAnything (Mem Efficient) in the plots refers to using `memory_efficient_inference=True` with mini batch size 1 in the `.infer()` and `.forward()` calls. This mode trades off speed for reduced memory consumption, enabling inference on a larger number of views (up to 2000 views on 140 GB). As seen in the plots, the speed trade off is negligible.\n\n### Basic Profiling\n\n```bash\n# Profile MapAnything (default and memory-efficient modes)\npython scripts\u002Fprofile_memory_runtime.py \\\n    --output_dir \u002Fpath\u002Fto\u002Fresults\n\n# Profile with a specific checkpoint\npython scripts\u002Fprofile_memory_runtime.py \\\n    --output_dir \u002Fpath\u002Fto\u002Fresults \\\n    --mapanything_checkpoint \u002Fpath\u002Fto\u002Fcheckpoint.pth\n\n# Use Apache 2.0 licensed model\npython scripts\u002Fprofile_memory_runtime.py \\\n    --output_dir \u002Fpath\u002Fto\u002Fresults \\\n    --apache\n```\n\n### Comparing with External Models\n\nExternal models are loaded using their Hydra config files from `configs\u002Fmodel\u002F\u003Cmodel_name>.yaml`. Make sure the required dependencies are installed (see [Running External Models](#running-external-models)).\n\n```bash\n# Compare MapAnything with external models\npython scripts\u002Fprofile_memory_runtime.py \\\n    --output_dir \u002Fpath\u002Fto\u002Fresults \\\n    --external_models vggt pi3x must3r\n\n# Custom view counts\npython scripts\u002Fprofile_memory_runtime.py \\\n    --output_dir \u002Fpath\u002Fto\u002Fresults \\\n    --num_views 2 4 8 16 32 64\n```\n\n**Available external models:** `vggt`, `pi3`, `pi3x`, `dust3r`, `mast3r`, `must3r`, `pow3r`, `pow3r_ba`, `da3`, `da3_nested`, `moge_1`, `moge_2`\n\n### Command-Line Arguments\n\n| Argument | Description | Default |\n|----------|-------------|---------|\n| `--output_dir` | Directory to save results | Required |\n| `--num_views` | List of view counts to profile | 2 4 8 16 24 32 50 100 200 500 1000 |\n| `--external_models` | External model names to compare | None |\n| `--mapanything_checkpoint` | Path to MapAnything checkpoint | None (uses HuggingFace) |\n| `--apache` | Use Apache 2.0 licensed model | False |\n| `--warmup_runs` | Number of warmup iterations | 3 |\n| `--timed_runs` | Number of timed iterations | 5 |\n| `--skip_mem_efficient` | Skip memory-efficient mode profiling | False |\n\n### Output Files\n\nThe script generates the following outputs in the specified directory:\n\n- `profiling_results.json`: Raw profiling data with memory and timing statistics\n- `profiling_memory.png`: Plot of peak GPU memory usage vs number of views\n- `profiling_speed.png`: Plot of inference frequency (Hz) vs number of views\n\n## COLMAP & GSplat Support\n\nMapAnything predictions can be exported to COLMAP format for use with Gaussian Splatting and other downstream applications.\n\n### Exporting to COLMAP Format\n\nMapAnything's predictions can directly be converted to COLMAP format by using:\n\n```bash\n# Install requirements for this specific demo\npip install -e \".[colmap]\"\n\n# Export MapAnything predictions to COLMAP format\npython scripts\u002Fdemo_colmap.py --images_dir=\u002FYOUR\u002FIMAGES_DIR\u002F --output_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F\n\n# With custom voxel fraction (default: 0.01 = 1% of IQR-based scene extent)\npython scripts\u002Fdemo_colmap.py --images_dir=\u002FYOUR\u002FIMAGES_DIR\u002F --output_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --voxel_fraction=0.002\n\n# With explicit voxel size in meters (overrides --voxel_fraction)\npython scripts\u002Fdemo_colmap.py --images_dir=\u002FYOUR\u002FIMAGES_DIR\u002F --output_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --voxel_size=0.01\n\n# Use Apache 2.0 licensed model\npython scripts\u002Fdemo_colmap.py --images_dir=\u002FYOUR\u002FIMAGES_DIR\u002F --output_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --apache\n\n# Also save dense reconstruction as GLB file\npython scripts\u002Fdemo_colmap.py --images_dir=\u002FYOUR\u002FIMAGES_DIR\u002F --output_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --save_glb\n\n# Skip Point2D backprojection for faster export\npython scripts\u002Fdemo_colmap.py --images_dir=\u002FYOUR\u002FIMAGES_DIR\u002F --output_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --skip_point2d\n```\n\nThe output is a self-contained COLMAP reconstruction with processed images (at model inference resolution) and camera parameters:\n\n```\nOUTPUT_DIR\u002F\n├── images\u002F           # Processed images (matching intrinsics resolution)\n│   ├── img1.jpg\n│   └── img2.jpg\n└── sparse\u002F\n    ├── cameras.bin\n    ├── images.bin\n    ├── points3D.bin\n    └── points.ply\n```\n\n### Visualizing COLMAP Reconstruction in Rerun\n\nYou can visualize the exported COLMAP reconstruction using Rerun:\n\n```bash\n# Terminal 1: Start the Rerun server\nrerun --serve --port 2004 --web-viewer-port 2006\n\n# Terminal 2: Visualize the COLMAP reconstruction\npython scripts\u002Fvisualize_colmap_format_in_rerun.py --scene_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --connect\n\n# With images and keypoints\npython scripts\u002Fvisualize_colmap_format_in_rerun.py --scene_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --show_images --show_keypoints --connect\n\n# Filter noisy points (by track length, primarily useful for traditional SfM outputs)\npython scripts\u002Fvisualize_colmap_format_in_rerun.py --scene_dir=\u002FYOUR\u002FOUTPUT_DIR\u002F --filter --min_track_length=4 --connect\n\n# Terminal 3 or Local Machine: Open web viewer at http:\u002F\u002F127.0.0.1:2006\n```\n\nOptionally, if Rerun is installed locally, the local Rerun viewer can be spawned using: `rerun --connect rerun+http:\u002F\u002F127.0.0.1:2004\u002Fproxy`.\n\n### Integration with Gaussian Splatting\n\nThe exported COLMAP files can be directly used with [gsplat](https:\u002F\u002Fgithub.com\u002Fnerfstudio-project\u002Fgsplat) for Gaussian Splatting training. Install `gsplat` following their official instructions (we recommend `gsplat==1.3.0`):\n\nAn example command to train the model is:\n```\ncd \u003Cpath_to_gsplat>\npython examples\u002Fsimple_trainer.py  default --data_factor 1 --data_dir \u002FYOUR\u002FOUTPUT_DIR\u002F --result_dir \u002FYOUR\u002FRESULT_DIR\u002F\n```\n\n## Data Processing for Training & Benchmarking\n\nWe provide details in the [Data Processing README](data_processing\u002FREADME.md).\n\n## Training\n\nWe provide comprehensive training instructions, scripts, and configurations to reproduce MapAnything and train custom models. See [Training README](train.md) for detailed training instructions, including:\n- Data setup and processing for all 13 training datasets used in the paper\n- Quick start examples with memory optimization tips\n- All main model and ablation training scripts from the paper\n- Fine-tuning support for other geometry estimation models like MoGe-2, VGGT, π³ showcasing the modularity of our framework\n\n## Benchmarking\n\nWe provide comprehensive benchmarking scripts and instructions for evaluating MapAnything across multiple tasks and datasets. All original bash scripts used for benchmarking are available in the `\u002Fbash_scripts\u002Fbenchmarking\u002F` folder.\n\n### Available Benchmarks\n\n1. **Dense Up-to-N-View Reconstruction Benchmark**\n   See [Dense Up-to-N-View Benchmark README](benchmarking\u002Fdense_n_view\u002FREADME.md) for detailed instructions on evaluating dense multi-view metric reconstruction.\n\n2. **Single-View Image Calibration Benchmark**\n   See [Calibration Benchmark README](benchmarking\u002Fcalibration\u002FREADME.md) for detailed instructions on evaluating camera intrinsic prediction from single images.\n\n3. **RobustMVD Benchmark**\n   See [RMVD Benchmark README](benchmarking\u002Frmvd_mvs_benchmark\u002FREADME.md) for detailed instructions on using the RobustMVD benchmark.\n\n## Code License\n\nThis code is licensed under an open-source [Apache 2.0 license](LICENSE).\n\n## Models\n\nWe release **two variants** of the pre-trained MapAnything models on Hugging Face Hub, each with different licensing based on the underlying training datasets:\n\n### 🤗 Hugging Face Hub Models\n\n1. **[facebook\u002Fmap-anything](https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fmap-anything)** (CC-BY-NC 4.0 License)\n2. **[facebook\u002Fmap-anything-apache](https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fmap-anything-apache)** (Apache 2.0 License)\n\n### 🤗 Hugging Face Hub Models used for V1 Release in September 2025\n\n1. **[facebook\u002Fmap-anything-v1](https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fmap-anything-v1)** (CC-BY-NC 4.0 License)\n2. **[facebook\u002Fmap-anything-apache-v1](https:\u002F\u002Fhuggingface.co\u002Ffacebook\u002Fmap-anything-apache-v1)** (Apache 2.0 License)\n\n### Model Selection Guide\n\n- **For Research & Academic Use**: Use `facebook\u002Fmap-anything` for the best performance\n- **For Commercial Use**: Use `facebook\u002Fmap-anything-apache` for commercial-friendly licensing\n\nBoth models support the same API and functionality. The only difference is the training data composition and resulting license terms. Please see our paper for detailed information about the specific datasets used in each model variant.\n\n### Optional Checkpoint Conversion\n\nThe MapAnything training\u002Fbenchmarking framework expects trained checkpoints in a specific format with a `model` key. The HuggingFace checkpoints can be easily converted to the expected format using:\n\n```bash\n# Convert default CC-BY-NC model\npython scripts\u002Fconvert_hf_to_benchmark_checkpoint.py \\\n    --output_path checkpoints\u002Ffacebook_map-anything.pth\n\n# Convert Apache 2.0 model for commercial use\npython scripts\u002Fconvert_hf_to_benchmark_checkpoint.py \\\n    --apache \\\n    --output_path checkpoints\u002Ffacebook_map-anything-apache.pth\n```\n\n## Building Blocks for MapAnything\n\nUniCeption & WorldAI (WAI) Data are two crucial building blocks and have been developed for ease of use by the community:\n\n🌍 [UniCeption](https:\u002F\u002Fgithub.com\u002Fcastacks\u002FUniCeption\u002Ftree\u002Fmain) is a library which contains modular, config-swappable components for assembling end-to-end networks.\n\n🌍 [WAI](.\u002Fdata_processing\u002FREADME.md) is a unified data format for all things 3D, 4D & Spatial AI. It enables easy, scalable and reproducible data processing.\n\nWe strongly encourage the community to build on top of the tools and submit PRs! This also enables us to release stronger models (both apache and research use) as the community adds more datasets to WAI and builds on top of UniCeption\u002FMapAnything.\n\n### Related Research\n\nCheck out our related work which also use UniCeption & WAI:\n\n🚀 [UFM: A Simple Path towards Unified Dense Correspondence with Flow](https:\u002F\u002Funiflowmatch.github.io\u002F)\n\n🚀 [FlowR: Flowing from Sparse to Dense 3D Reconstructions](https:\u002F\u002Ftobiasfshr.github.io\u002Fpub\u002Fflowr\u002F)\n\n## Acknowledgments\n\nWe thank the following projects for their open-source code: [DUSt3R](https:\u002F\u002Fgithub.com\u002Fnaver\u002Fdust3r), [MASt3R](https:\u002F\u002Fgithub.com\u002Fnaver\u002Fmast3r), [RayDiffusion](https:\u002F\u002Fgithub.com\u002Fjasonyzhang\u002FRayDiffusion), [MoGe](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fmoge), [VGGSfM](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fvggsfm), [VGGT](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fvggt), [MaRePo](https:\u002F\u002Fgithub.com\u002Fnianticlabs\u002Fmarepo), and [DINOv2](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fdinov2).\n\n## Citation\n\nIf you find our repository useful, please consider giving it a star ⭐ and citing our paper in your work:\n\n```bibtex\n@inproceedings{keetha2026mapanything,\n  title={{MapAnything}: Universal Feed-Forward Metric {3D} Reconstruction},\n  author={Nikhil Keetha and Norman M\\\"{u}ller and Johannes Sch\\\"{o}nberger and Lorenzo Porzi and Yuchen Zhang and Tobias Fischer and Arno Knapitsch and Duncan Zauss and Ethan Weber and Nelson Antunes and Jonathon Luiten and Manuel Lopez-Antequera and Samuel Rota Bul\\`{o} and Christian Richardt and Deva Ramanan and Sebastian Scherer and Peter Kontschieder},\n  booktitle={International Conference on 3D Vision (3DV)},\n  year={2026},\n  organization={IEEE}\n}\n```\n","MapAnything 是一个用于通用度量3D重建的开源研究框架。其核心是一个端到端训练的Transformer模型，能够直接从多种输入（如图像、校准、姿态或深度）中回归出场景的因子化度量3D几何结构。该框架支持超过12种不同的3D重建任务，包括多图像SFM、多视图立体视觉、单目度量深度估计、配准和深度补全等。MapAnything 提供了完整的数据处理、训练、推理和性能分析工具链，并采用模块化设计，便于用户根据需求进行定制和扩展。适用于需要高精度3D重建的应用场景，如机器人导航、虚拟现实、增强现实以及自动驾驶等领域。",2,"2026-06-11 03:41:14","high_star"]