[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80664":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":13,"stars30d":13,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":16,"rankGlobal":10,"rankLanguage":10,"license":17,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":18,"hasPages":18,"topics":20,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":15,"starSnapshotCount":15,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},80664,"MuGS","Renforce-Dynamics\u002FMuGS","Renforce-Dynamics","MuGS: MuJoCo Gaussian Splatting","",null,"Python",50,1,49,0,0.9,"Other",false,"master",[21,22,23,24,25,26],"gaussian-splatting","graphics","mjlab","mujoco","render","simulation","2026-06-12 02:04:05","# MuGS: MuJoCo + 3D Gaussian Splatting for Photorealistic Robot Simulation\n\n**MuGS** is a hybrid rendering pipeline that combines MuJoCo physics simulation with 3D Gaussian Splatting (3DGS) photorealistic backgrounds to create photo-realistic robot simulation environments for Vision-Language-Action (VLA) model training and evaluation.\n\n## Demo — DISCOVERSE 3DGS rooms × AndroidTwin multi-cam rollout\n\nFour cameras, one rollout, one shared `world_T_gs` rigid alignment. The\nG1 + Inspire FTP stands via AMO inside the DISCOVERSE `lab3` 3DGS room;\neach frame is captured from the robot's first-person `head_cam` plus\nthree world cams (front-diagonal, over-the-shoulder, low side hero).\nThe composite below stitches all four panes — every panel is a hybrid\nMuGS render (3DGS bg + MuJoCo fg, alpha-blended per camera).\n\n![4-cam DISCOVERSE lab3 composite](docs\u002Fmedia\u002Fdiscoverse_lab3\u002F4cam_composite.gif)\n\n| Camera | View |\n|---|---|\n| `robot\u002Fhead_cam` (first-person, D435 mount, 47° down, 42° vfov) | ![head_cam](docs\u002Fmedia\u002Fdiscoverse_lab3\u002Fhead_cam.gif) |\n| `world_cam` (front-diagonal) | ![world_cam](docs\u002Fmedia\u002Fdiscoverse_lab3\u002Fworld_cam.gif) |\n| `world_back_cam` (over-the-shoulder back) | ![world_back_cam](docs\u002Fmedia\u002Fdiscoverse_lab3\u002Fworld_back_cam.gif) |\n| `world_side_cam` (low hero shot) | ![world_side_cam](docs\u002Fmedia\u002Fdiscoverse_lab3\u002Fworld_side_cam.gif) |\n\nTwo pieces of plumbing make this work compared to the kitchen demo\nbelow:\n\n- **One alignment for all cams.** `MuGSRecorder` accepts a `world_T_gs`\n  4×4 transform that maps MJ-world → GS-world. Every simultaneous camera\n  applies the same transform to its live MJ pose, so the four views see\n  the same room from coherent perspectives without per-cam initial-pose\n  snapping.\n- **OpenGL → OpenCV Y-flip.** gsplat is OpenCV (+Y down). MuGS internally\n  negates column 2 of the cam rotation (Z forward), so the recorder\n  pre-negates column 1 in the `world_T_gs` branch. Without this, every\n  level-horizon world cam renders upside-down; the first-person head cam\n  survives unflipped only because its 47° pitch coincidentally lines up.\n\nGet the scene:\n\n```bash\nbash scripts\u002Fdata_collection\u002Fdownload_discoverse_scenes.sh   # default: lab3\n```\n\nRender the demo (from the AndroidTwin repo, scene path adjusted):\n\n```bash\n.venv\u002Fbin\u002Fpython examples\u002Fdiscoverse_room_rollout.py \\\n  --bg-ply \u002Fpath\u002Fto\u002FMuGS\u002Fassets\u002Fscenes\u002Fdiscoverse_unpacked\u002Flab3\u002Fpoint_cloud.ply \\\n  --out-dir outputs\u002Fdiscoverse_lab3 \\\n  --num-steps 120 --width 480 --height 360 \\\n  --gs-fx 380 --gs-fy 380 --yaw-deg 180\n```\n\nSee [`docs\u002Fdiscoverse_scenes.md`](docs\u002Fdiscoverse_scenes.md) for the\nscene catalog, alignment knobs (`yaw_deg`, GS focal, floor 5%-percentile,\nbbox-center vs median), and the training-cam convex-hull caveat that\nmakes some camera placements render dim\u002Fnoisy.\n\n## What is MuGS?\n\nMuGS enables **photorealistic robot simulation** by compositing physically accurate MuJoCo robot renders with real-world 3DGS backgrounds at **5000+ FPS**. It bridges the Sim2Real gap for vision-based robot learning by providing training data that looks like real photos while maintaining perfect physics simulation.\n\n**Key Innovation**: Two-stage rendering pipeline\n- **Stage 1**: MuJoCo renders robot\u002Fobjects (CPU-based, fast)\n- **Stage 2**: 3DGS renders background (GPU-based, photorealistic)\n- **Compositing**: Alpha-blending with automatic segmentation masks\n\n![MuGS Showcase](docs\u002Fimages\u002Fshowcase.jpg)\n\n### AndroidTwin × MuGS — G1 humanoid in INRIA kitchen\n\nHybrid render demo from the **AndroidTwin** humanoid bench\n(Unitree G1 + Inspire FTP 5-finger hands, 53 dof) composited on the\nINRIA mip-NeRF 360 *kitchen* 3DGS scene.\nTask: `p1_amo_table_grasp` (4 GraspNet objects on a table, AMO\nwhole-body controller, head_cam first-person view), zero-policy\n30-step rollout.\n\n#### Pipeline — body-prefix masking\n\n![pipeline](docs\u002Fimages\u002Fshowcase\u002Fandroidtwin_g1\u002Fpanel_pipeline.png)\n\n`scene_inspire.xml` leaves geoms unnamed, so the recorder resolves\nforeground pixels by **body name prefix** instead of geom name.\nSceneCfg attaches each entity with a namespace prefix (`robot\u002F`,\n`table\u002F`, `obj_NNN\u002F`, `grasp_cube\u002F`…); any geom whose parent body\nmatches one of the configured prefixes joins the fg mask.\n\n| panel | what |\n|-------|------|\n| 1. MuJoCo foreground | `mujoco.Renderer.render()` at `robot\u002Fhead_cam` |\n| 2. Geom segmentation | `enable_segmentation_rendering()` raw geom-id channel |\n| 3. FG mask | `np.isin(seg, fg_geom_ids)` → 29.3% coverage on this view |\n\n#### Hybrid render — bg cam tracks MuJoCo head\n\n![tracking](docs\u002Fimages\u002Fshowcase\u002Fandroidtwin_g1\u002Fpanel_tracking.png)\n\n3DGS scenes live in COLMAP world frames disjoint from MuJoCo world.\nThe recorder pins the **initial** GS bg pose to a training cam\n(kitchen `cam[0]`) and *assumes* it coincides with the initial MuJoCo\nhead_cam pose; on each capture it applies the MuJoCo-frame head_cam\ndelta (rotated into GS frame) on top of that initial pose so the bg\ntracks head motion. Intrinsics stay fixed at the kitchen training\ncam's fx\u002Ffy.\n\n```\nR_align = R_gs0 · R_mj0ᵀ           # one-shot at first capture\npos_t   = pos_gs0 + R_align · (pos_mj_t − pos_mj_0)\nR_t     = R_align · R_mj_t\n```\n\n#### Static vs tracked background\n\n![before\u002Fafter](docs\u002Fimages\u002Fshowcase\u002Fandroidtwin_g1\u002Fpanel_before_after.png)\n\nSame MuJoCo frame, two bg policies:\n\n- **Left (static)** — `bg_t == bg_0` for every capture; bg-region\n  pixel diff stays at 0.05\u002F255 (just libx264 noise).\n- **Right (tracked)** — bg follows head motion; bg-region pixel\n  diff rises to ~90\u002F255 by frame 5 (kitchen content shifts as expected).\n\n#### Animation\n\n![hybrid tracking gif](docs\u002Fimages\u002Fshowcase\u002Fandroidtwin_g1\u002Fhybrid_tracking.gif)\n\n16-frame loop @ 10 fps, 320×240 (downsampled from 31-frame mp4 at\n640×480). Top of frame turns black mid-rollout because the head\npitches down enough that the bg cam looks above the kitchen ply's\ntrained volume — a known tradeoff of the no-scale-calibration\nassumption.\n\n#### Reproduce\n\n```bash\nuv run at-eval \\\n  --task p1_amo_table_grasp --policy zero \\\n  --num-episodes 1 --max-episode-steps 30 \\\n  --camera robot\u002Fhead_cam \\\n  --render-backend mugs --mugs-mode hybrid \\\n  --mugs-ply  \u002Fpath\u002Fto\u002Fkitchen\u002Fpoint_cloud.ply \\\n  --mugs-bg-cam-json \u002Fpath\u002Fto\u002Fkitchen\u002Fcameras.json \\\n  --mugs-bg-cam-idx 0 \\\n  --eval-dir outputs\u002Fevals_mugs_hybrid\n```\n\n`MUJOCO_GL=egl` required on headless servers. The wrapper\n(`MuGSRecorder`) lives at `androidtwin\u002Fenvs\u002Fmugs_recorder.py` in the\nAndroidTwin repo and uses MuGS's standalone `GaussianSensor` API.\n\n## Features\n\n### Core Rendering\n- **Hybrid Rendering**: 3DGS background + MuJoCo foreground @ 5000+ FPS\n- **Physics-Accurate**: Full MuJoCo physics simulation with contact dynamics\n- **Photorealistic**: 3D Gaussian Splatting for realistic backgrounds from real-world scans\n- **Camera Aligned**: Automatic camera parameter extraction and coordinate system conversion\n- **Flexible Modes**: `hybrid`, `3dgs_only`, `mujoco_only` rendering modes\n\n### Integration & Usability\n- **Easy Integration**: Drop-in `GaussianSensor` API for standalone use\n- **Batch Rendering**: mjlab integration for parallel multi-environment rendering (`mugs_mjlab.tasks`: ready-to-use env configs + benchmark runner — 4096 envs in one batched warp + gsplat call)\n- **Automatic Masking**: Body-prefix or geom-name based foreground segmentation\n- **Camera Tracking**: Dynamic background camera follows MuJoCo camera motion\n\n### Post-Processing (Optional)\n- **Super-Resolution**: AI upscaling (Real-ESRGAN) for photorealistic detail\n  - Low-res rendering (320×240) → High-res output (1280×960)\n  - 4x upscaling with photorealistic texture enhancement\n  - Modular design: lazy-loaded, GPU-accelerated, batch processing support\n\n### Asset Support\n- **Pretrained Scenes**: INRIA kitchen (mip-NeRF 360 dataset)\n- **External Assets**: GS-Playground scenes, DISCOVERSE tasks (via download scripts)\n- **Custom Scenes**: Support for any 3DGS PLY files from COLMAP \u002F Nerfstudio — shoot your own with [`docs\u002Fguides\u002FGS_VIDEO_CAPTURE.md`](docs\u002Fguides\u002FGS_VIDEO_CAPTURE.md) (phone-camera 三圈拍法, 中文实操向)\n\n## Quick Start\n\n### Basic Rendering\n\n```python\nfrom mugs.sensors import GaussianSensor, GaussianSensorConfig\n\n# 1. Configure sensor\nconfig = GaussianSensorConfig(\n    width=640,\n    height=480,\n    background_ply_path=\"path\u002Fto\u002Fpoint_cloud.ply\",\n    render_mode=\"hybrid\",  # \"hybrid\" | \"3dgs_only\" | \"mujoco_only\"\n    robot_geom_names=[\"link1\", \"link2\", \"gripper\"],\n)\n\nsensor = GaussianSensor(config)\n\n# 2. Render\nresult = sensor.render(model, data, camera_name, return_components=True)\n\n# 3. Access components\nrgb = result['rgb']              # Final hybrid image (H, W, 3) uint8\nforeground = result['foreground'] # MuJoCo only\nbackground = result['background'] # 3DGS only\nmask = result['mask']            # Blending mask\n```\n\n### Batched parallel rendering (mjlab)\n\nRun **thousands of envs in parallel** with one batched mjwarp render + one\ngsplat rasterization call. `mugs_mjlab.tasks` ships a YAM lift-cube task\nconfig with a `GaussianSensorMjlab` pre-attached to the wrist camera, plus a\ngeneric benchmark runner that works for any `ManagerBasedRlEnvCfg` factory:\n\n```python\nfrom mjlab.envs import ManagerBasedRlEnv\nfrom mugs_mjlab import (\n    make_yam_lift_cube_gs_env_cfg,\n    kitchen_ply_path,\n    benchmark_env_cfg,\n    print_benchmark_table,\n)\n\n# 1. Build a 4096-env photorealistic YAM lift-cube env.\ncfg = make_yam_lift_cube_gs_env_cfg(\n    ply_path=kitchen_ply_path(),  # bundled INRIA kitchen scene\n    num_envs=4096,\n    width=160, height=120,\n    render_mode=\"hybrid\",         # 3DGS bg + MuJoCo fg + alpha-blend\n)\nenv = ManagerBasedRlEnv(cfg=cfg, device=\"cuda\")\nobs, _ = env.reset()\n# obs['gs_rgb'].rgb.shape == (4096, 120, 160, 3)  uint8\n\n# 2. Sweep batch sizes and print a throughput table (RTX 4090: ~12.5k env-FPS @ 4096).\nresults = benchmark_env_cfg(\n    lambda n: make_yam_lift_cube_gs_env_cfg(kitchen_ply_path(), num_envs=n),\n    env_counts=(1, 64, 1024, 4096),\n)\nprint_benchmark_table(results)\n```\n\n`benchmark_env_cfg` accepts any `int -> ManagerBasedRlEnvCfg` factory, so the\nsame runner works for your own mjlab task. CLI shim:\n\n```bash\npython scripts\u002Fevaluation\u002Fbenchmark_mjlab_envs.py --env-counts 1 64 1024 4096\n# Override scene \u002F resolution \u002F mode:\npython scripts\u002Fevaluation\u002Fbenchmark_mjlab_envs.py \\\n    --ply \u002Fpath\u002Fto\u002Fscene.ply --width 320 --height 240 --render-mode hybrid\n```\n\n### With Super-Resolution (Optional)\n\n```python\nfrom mugs.postprocess import SuperResolution, SuperResolutionConfig\n\n# 1. Low-res rendering (fast)\nsensor = GaussianSensor(GaussianSensorConfig(width=320, height=240, ...))\nimg_lr = sensor.render(model, data, camera_name)  # 320×240\n\n# 2. AI upscaling (photorealistic)\nsr = SuperResolution(SuperResolutionConfig(\n    model_name=\"RealESRGAN_x4plus\",\n    scale=4,\n))\nimg_hr = sr.upscale(img_lr)  # 1280×960\n\n# Installation: pip install realesrgan basicsr\n# Download model: python scripts\u002Fdownload_sr_models.py --model RealESRGAN_x4plus\n```\n\n## Demos\n\n### YAM Manipulation\n```bash\nTORCH_CUDA_ARCH_LIST=\"8.6\" python examples\u002Fyam_standalone_demo.py\n```\n\n### Quality Comparison\n```bash\nTORCH_CUDA_ARCH_LIST=\"8.6\" python examples\u002Fquality_comparison_demo.py\n```\n\n### Wrist Camera (Task View)\n```bash\nTORCH_CUDA_ARCH_LIST=\"8.6\" python examples\u002Fyam_wrist_camera_demo.py\n```\n\n### Super-Resolution Pipeline\n```bash\n# 1. Install SR dependencies\npip install realesrgan basicsr\n\n# 2. Download pretrained models\npython scripts\u002Fdownload_sr_models.py --model RealESRGAN_x4plus\n\n# 3. Run demo\nTORCH_CUDA_ARCH_LIST=\"8.6\" python examples\u002Fsr_pipeline_demo.py\n```\n\n### Using External Assets\n```bash\n# List available external asset sources\npython scripts\u002Fdownload_external_assets.py list\n\n# Download GS-Playground assets\npython scripts\u002Fdownload_external_assets.py gs-playground\n\n# Download Bridge-GS dataset\npython scripts\u002Fdownload_external_assets.py bridge-gs\n\n# Run examples with external assets\nTORCH_CUDA_ARCH_LIST=\"8.6\" python examples\u002Fuse_external_assets.py\n```\n\nSee `docs\u002FEXTERNAL_ASSETS.md` for detailed tutorial.\n\n## Performance\n\n### Hardware\n\n| Component | Spec |\n|---|---|\n| GPU | NVIDIA GeForce RTX 4090 (24 GB, sm_89) |\n| Driver \u002F CUDA runtime | 580.126.09 \u002F CUDA 13.0 |\n| PyTorch | 2.11.0 + cu130 |\n| gsplat | 1.5.3 (JIT-compiled with `TORCH_CUDA_ARCH_LIST=8.6+PTX`) |\n| Build toolchain | CUDA 11.6 nvcc → driver-JIT to sm_89 |\n\n### Single-camera \u002F single-env throughput\n\n| Stage | Latency | Notes |\n|---|---|---|\n| MuJoCo RGB | 0.21 ms | foreground render |\n| Segmentation | 1.33 ms | body-prefix mask |\n| Mask extract | 0.15 ms | `np.isin` |\n| **3DGS GPU** | **0.12 ms** | 6.18k gaussians @ 160×120 |\n| Composite | 0.13 ms | alpha-blend |\n| **Total** | **1.94 ms** | **515 Hz end-to-end** |\n\n### Parallel-env 3DGS rendering (RTX 4090, 160×120)\n\nEach environment gets an independent camera pose (sphere of look-at views).\nSee `scripts\u002Fevaluation\u002Fbenchmark_gsplat_parallel.py`.\n\n| Envs | 6k gaussians | 50k gaussians | 100k gaussians | 500k gaussians |\n|---:|---:|---:|---:|---:|\n| 1 | 10,479 | 10,421 | 10,479 | 9,625 |\n| 16 | 123,513 | 126,422 | 120,159 | 30,148 |\n| 64 | 489,251 | 249,321 | 137,871 | 24,685 |\n| 256 | **1,101,650** | 206,304 | 114,741 | 25,196 |\n| 1024 | 883,496 | 187,414 | 109,543 | OOM |\n| 2048 | 756,534 | 196,071 | 112,714 | OOM |\n| 4096 | 757,980 | 188,734 | **109,481** | OOM |\n\n(numbers = total env-FPS; per-env latency at 4096 envs is **1.32 µs** for kitchen-scale)\n\n### 4096 envs × resolution (6k gaussians)\n\n| Resolution | Latency | Env-FPS | Peak VRAM |\n|---|---:|---:|---:|\n| 80×60 | 2.60 ms | 1,575,441 | 21.0 GB |\n| 160×120 | 5.30 ms | 773,220 | 3.6 GB |\n| 224×224 | 11.01 ms | 372,049 | 7.4 GB |\n| 320×240 | 15.84 ms | 258,643 | 10.6 GB |\n\n### End-to-end mjlab + GaussianSensorMjlab (hybrid, 160×120, 6k gaussians)\n\nFull pipeline: mjlab `step()` (physics + observations + reward) **plus**\n`GaussianSensorMjlab` in hybrid mode (mjwarp foreground + 3DGS background +\nalpha-composite). `yam_lift_cube` manipulation task with a wrist-mounted\ncamera. Reproduce in one command:\n\n```bash\npython scripts\u002Fevaluation\u002Fbenchmark_mjlab_envs.py\n```\n\n| Envs | Step latency | Env-FPS | Peak VRAM |\n|---:|---:|---:|---:|\n| 1 | 4.34 ms | 231 | 0.10 GB |\n| 4 | 4.44 ms | 901 | 0.21 GB |\n| 16 | 5.66 ms | 2,825 | 0.21 GB |\n| 64 | 9.85 ms | 6,499 | 0.21 GB |\n| 256 | 25.97 ms | 9,859 | 0.21 GB |\n| 1024 | 84.18 ms | 12,164 | 0.31 GB |\n| 4096 | 326.32 ms | **12,552** | 0.32 GB |\n\n### Takeaways\n\n- Single-env 3DGS clears the **5000 FPS** project target by ~2× at any realistic scene size.\n- Saturation point on a 4090: **batch ≈256 at 6k gaussians**, **batch ≈64 at 100k gaussians** (kernel becomes memory-bandwidth-bound thereafter).\n- 4096 parallel envs is feasible up to ~100k gaussians (22 GB peak); 500k gaussians OOMs past ~256 envs on 24 GB.\n- End-to-end mjlab + MuGS hybrid pipeline saturates at **~12.5k env-FPS** for 1024–4096 envs at 160×120 — bounded by mjlab's physics step + mjwarp segmentation render, not by 3DGS (which costs \u003C0.4 ms per batch at this scale).\n- Background loading: ~2 s, cached for subsequent renders.\n\n## Technical Details\n\n### Camera Alignment\n\nMuGS automatically handles camera parameter extraction and coordinate system conversion:\n\n1. **FOV Handling**: Respects MuJoCo's `\u003Ccompiler angle=\"radian\"\u002F>` directive\n2. **Coordinate System**: Converts MuJoCo (+Z forward) to OpenGL (-Z forward)\n3. **View Matrix**: Automatic construction from camera position and rotation\n\nSee `docs\u002FCAMERA_ALIGNMENT_FIX.md` for details.\n\n### Rendering Pipeline\n\n1. Extract camera parameters from MuJoCo\n2. Render MuJoCo foreground (robot + objects)\n3. Render 3DGS background with aligned camera\n4. Alpha-composite using robot segmentation mask\n\n## Documentation\n\n### Getting Started\n- **[`docs\u002FOVERVIEW.md`](docs\u002FOVERVIEW.md)** - 📘 Project overview (what MuGS does, repo structure, use cases)\n- **[`docs\u002FAPI_QUICKSTART.md`](docs\u002FAPI_QUICKSTART.md)** - 🚀 Quick start guide (中文)\n- **[`docs\u002FAPI_REFERENCE.md`](docs\u002FAPI_REFERENCE.md)** - 📖 Complete API reference\n\n### Technical Guides\n- **[`docs\u002FSUPER_RESOLUTION.md`](docs\u002FSUPER_RESOLUTION.md)** - 🎨 AI upscaling guide (Real-ESRGAN)\n- **[`docs\u002FCAMERA_ALIGNMENT_FIX.md`](docs\u002FCAMERA_ALIGNMENT_FIX.md)** - 📐 Camera parameter handling\n- **[`docs\u002FEXTERNAL_ASSETS.md`](docs\u002FEXTERNAL_ASSETS.md)** - 📦 Using GS-Playground, DISCOVERSE assets\n- **[`docs\u002FSHOWCASE.md`](docs\u002FSHOWCASE.md)** - 🎬 Creating demonstration materials\n\n### Capturing your own 3DGS scenes\n- **[`docs\u002Fguides\u002FINSTALLATION.md`](docs\u002Fguides\u002FINSTALLATION.md)** - 🛠️ Install MuGS + 3DGS training stack\n- **[`docs\u002Fguides\u002FQUICK_START.md`](docs\u002Fguides\u002FQUICK_START.md)** - ⚡ End-to-end \"first scene\" walkthrough\n- **[`docs\u002Fguides\u002FGS_VIDEO_CAPTURE.md`](docs\u002Fguides\u002FGS_VIDEO_CAPTURE.md)** - 📱 Phone-camera shooting guide (中文, 实操向)\n- **[`docs\u002Fguides\u002FGS_DATA_COLLECTION.md`](docs\u002Fguides\u002FGS_DATA_COLLECTION.md)** - 🎞️ Full capture → COLMAP → 3DGS pipeline\n- **[`docs\u002Fguides\u002F3DGS_ROOM_DATASETS.md`](docs\u002Fguides\u002F3DGS_ROOM_DATASETS.md)** - 🏠 Public 3DGS room datasets to start from\n- **[`docs\u002Fguides\u002FPRETRAINED_MODELS.md`](docs\u002Fguides\u002FPRETRAINED_MODELS.md)** - 🧠 Pretrained scenes shipped with MuGS\n- **[`docs\u002Fguides\u002FASSET_ACQUISITION.md`](docs\u002Fguides\u002FASSET_ACQUISITION.md)** - 🗂️ Where to source robots \u002F scenes \u002F objects\n- **[`docs\u002Fguides\u002FSEGMENT_ID_SYSTEM.md`](docs\u002Fguides\u002FSEGMENT_ID_SYSTEM.md)** - 🎭 Segmentation ID conventions\n\n### Design & Status\n- **[`docs\u002Fdesign\u002FPROJECT_ARCHITECTURE.md`](docs\u002Fdesign\u002FPROJECT_ARCHITECTURE.md)** - 🏗️ System architecture\n- **[`docs\u002Fdesign\u002FTECHNICAL_DESIGN.md`](docs\u002Fdesign\u002FTECHNICAL_DESIGN.md)** - 🔧 Technical design notes\n- **[`docs\u002FPROJECT_STATUS.md`](docs\u002FPROJECT_STATUS.md)** - 📊 Current status\n- **[`docs\u002Fdevelopment\u002FTODO.md`](docs\u002Fdevelopment\u002FTODO.md)** - 📋 Development roadmap\n\n## Installation\n\n```bash\n# 1. Clone and install MuGS (standalone GaussianSensor API).\ngit clone https:\u002F\u002Fgithub.com\u002FRenforce-Dynamics\u002FMuGS.git\ncd MuGS && pip install -e .\n\n# 2. (Optional) install the mjlab integration for batched multi-env rendering.\npip install -e . -f pyproject_mugs_mjlab.toml   # needs mjlab + mujoco_warp\n```\n\nFor RTX 4090 + CUDA 11.x build toolchains, set\n`TORCH_CUDA_ARCH_LIST=\"8.6+PTX\"` so gsplat JIT-compiles to sm_89 via PTX.\nOn headless servers, `MUJOCO_GL=egl` is required.\n\n## Requirements\n\n- Python 3.10+\n- MuJoCo 3.0+\n- PyTorch 2.0+\n- gsplat 1.5+\n- (mjlab integration only) mjlab + mujoco_warp on CUDA 12.x runtime\n\n## Citation\n\n```bibtex\n@article{mugs2026,\n  title={MuGS: Photorealistic Simulation for Vision-Language-Action Models},\n  author={},\n  journal={},\n  year={2026}\n}\n```\n\n## License\n\nMIT License\n\n## Acknowledgments\n\n- MuJoCo physics engine\n- gsplat rendering library\n- Kitchen scene from 3DGS benchmark\n","MuGS是一个结合了MuJoCo物理仿真与3D高斯点绘技术的混合渲染管线，旨在为视觉-语言-动作模型的训练和评估创建逼真的机器人模拟环境。其核心功能在于通过将基于物理的精确机器人渲染与现实世界的3D背景相结合，实现了超过5000帧\u002F秒的高效渲染速度，同时保持高度的真实感。该方案特别适用于需要在复杂或特定场景下测试机器人行为的研究项目，如实验室环境下的多摄像头同步观察等。此外，MuGS还解决了不同视角间的一致性问题以及坐标系转换难题，使得从多个角度观察同一场景成为可能。",2,"2026-06-11 04:01:32","CREATED_QUERY"]