[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-77836":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":8,"htmlUrl":8,"language":9,"languages":8,"totalLinesOfCode":8,"stars":10,"forks":11,"watchers":12,"openIssues":13,"contributorsCount":13,"subscribersCount":13,"size":13,"stars1d":13,"stars7d":14,"stars30d":15,"stars90d":13,"forks30d":13,"starsTrendScore":13,"compositeScore":12,"rankGlobal":8,"rankLanguage":8,"license":16,"archived":17,"fork":17,"defaultBranch":18,"hasWiki":19,"hasPages":19,"topics":20,"createdAt":8,"pushedAt":8,"updatedAt":21,"readmeContent":22,"aiSummary":23,"trendingCount":13,"starSnapshotCount":13,"syncStatus":14,"lastSyncTime":24,"discoverSource":25},77836,"TideGS","sponge-lab\u002FTideGS","sponge-lab",null,"Python",146,9,3,0,2,79,"Apache License 2.0",false,"main",true,[],"2026-06-12 02:03:45","# TideGS\n\n[![arXiv](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FarXiv-2605.20150-b31b1b.svg)](https:\u002F\u002Farxiv.org\u002Fabs\u002F2605.20150)\n[![Hugging Face](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHugging%20Face-Paper-yellow.svg)](https:\u002F\u002Fhuggingface.co\u002Fpapers\u002F2605.20150)\n[![Project Page](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FProject-Page-blue.svg)](https:\u002F\u002Fsponge-lab.github.io\u002FTideGS)\n[![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache--2.0-green.svg)](LICENSE)\n\nTideGS is a system for training large-scale 3D Gaussian Splatting scenes with\nSSD-based out-of-core optimization. It keeps the full Gaussian parameter array\non SSD, uses CPU DRAM as a tiered cache, and materializes only the active\nresident blocks in GPU memory.\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fteaser.png\" alt=\"TideGS teaser\" width=\"680\">\n\u003C\u002Fp>\n\n## Features\n\n- Train city-scale 3DGS scenes without keeping the full Gaussian set in GPU memory.\n- Stream Gaussian blocks between SSD, CPU memory, and GPU resident buffers.\n- Reuse prebuilt SSD bases for repeated experiments without reprocessing the PLY.\n- Resume training from incremental checkpoints without copying the full base file.\n\n## Method Overview\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Foverview.png\" alt=\"TideGS method overview\" width=\"900\">\n\u003C\u002Fp>\n\n## Visual Comparison\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"assets\u002Fcomparison_tidegs_vs_vanilla_focus.gif\" alt=\"TideGS vs. vanilla 3DGS visual comparison\" width=\"900\">\n\u003C\u002Fp>\n\n## Installation\n\nThe release experiments used Python 3.10 with `torch==2.4.0+cu124`,\n`torchvision==0.19.0+cu124`, and `torchaudio==2.4.0+cu124`. Install a matching\nPyTorch stack for your CUDA\u002Fplatform first, then install the remaining Python\ndependencies and project extensions:\n\n```bash\npip install -r requirements.txt\npip install --no-build-isolation submodules\u002Fclm_kernels\npip install submodules\u002Ffast-tsp\npip install --no-build-isolation submodules\u002Fgsplat\npip install --no-build-isolation submodules\u002Fsimple-knn\n```\n\nSet PyTorch allocation behavior before training:\n\n```bash\nexport PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True\n```\n\n## Data Preparation\n\nTideGS experiments use MatrixCity-style aerial\u002Fstreet scenes. Download the RGB,\ncamera-pose, and depth resources from the official\n[MatrixCity repository](https:\u002F\u002Fgithub.com\u002Fcity-super\u002FMatrixCity), then follow\nits data-generation instructions to produce the initial dense point cloud.\n\nThe camera directory passed to `-s` \u002F `--src` must contain MatrixCity transform\nfiles:\n\n```text\n\u003Cscene_dir>\u002F\n  transforms_train.json\n  transforms_test.json\n```\n\nEach frame in the transform files should reference an image through\n`file_name` or `file_path`. The loader resolves MatrixCity paths relative to the\ntransform directory and the split folder, so a typical layout is:\n\n```text\n\u003Cdataset_root>\u002F\n  pose\u002Fall_blocks\u002F\n    transforms_train.json\n    transforms_test.json\n  train\u002F\n    0000.png\n    0001.png\n    ...\n  test\u002F\n    ...\n  point_cloud\u002F\n    matrixcity_1b.ply\n```\n\nThe exact folder names can differ as long as `transforms_train.json` and\n`transforms_test.json` point to valid image files. During the first run, images\nare decoded into raw files under `--decode-dataset-path`; put this cache on a\nlarge local or shared SSD.\n\n## Recommended Paths\n\nThe release scripts do not hard-code local dataset paths. Set these variables\nfor your machine:\n\n```bash\nexport TIDEGS_ROOT=\u002Fpath\u002Fto\u002Ftidegs_outputs\nexport MATRIXCITY_SCENE_DIR=\u002Fpath\u002Fto\u002FMatrixCity\u002Fpose\u002Fall_blocks\nexport TIDEGS_DENSE_PLY=\u002Fpath\u002Fto\u002Fmatrixcity_1b.ply\nexport TIDEGS_DECODE_CACHE=$TIDEGS_ROOT\u002Fdecoded_cache\u002Fmatrixcity\n```\n\nIf you already built an SSD base, also set:\n\n```bash\nexport TIDEGS_PREBUILT_MANIFEST=\u002Fpath\u002Fto\u002Fstreaming_init_manifest.json\n```\n\n## Build Or Reuse An SSD Base\n\nFor a fresh run without `TIDEGS_PREBUILT_MANIFEST`, the training command streams\n`$TIDEGS_DENSE_PLY` into an SSD base before training. This is correct but can be\nslow for billion-point scenes.\n\nFor repeated experiments, reuse a prebuilt SSD base by passing the generated\n`streaming_init_manifest.json`:\n\n```bash\n--manifest $TIDEGS_PREBUILT_MANIFEST\n```\n\nA prebuilt manifest points to:\n\n```text\nbase_file.bin\nblock_bounds.npy\nstreaming_init_manifest.json\n```\n\n`base_file.bin` stores the immutable initial `[N, 59]` float32 block array.\nPatch logs and checkpoints are written to the current run's SSD cache directory.\n\n## Training\n\nRun the accepted full-camera MatrixCity 1B configuration:\n\n```bash\nGPU=0 \\\nRUN_TAG=$(date +\"%Y%m%d_%H%M%S\")_tidegs_1b_train \\\nbash scripts\u002Ftrain_matrixcity_1b.sh \\\n  --mode train \\\n  --iterations 240 \\\n  --bsz 16 \\\n  --capacity 2048 \\\n  --resident-policy topc_balanced \\\n  --resident-lambda 0.3 \\\n  --resident-decay 0.95 \\\n  --balanced-seed-fraction 0.25 \\\n  --debug-max-train-cameras -1 \\\n  --debug-camera-sample-mode contiguous \\\n  --src \"$MATRIXCITY_SCENE_DIR\" \\\n  --ply \"$TIDEGS_DENSE_PLY\" \\\n  --manifest \"$TIDEGS_PREBUILT_MANIFEST\" \\\n  --decode-dataset-path \"$TIDEGS_DECODE_CACHE\" \\\n  --root \"$TIDEGS_ROOT\"\n```\n\nThe `debug-max-train-cameras` option controls the camera cap. In release commands,\n`--debug-max-train-cameras -1` disables the camera cap and uses all training\ncameras. Positive values are only for quick smoke or locality diagnostic runs.\n\nThe runner is quiet by default: the terminal shows progress bars, while detailed\ntraining stdout is written to `python.log`. Use `--debug-logging` to add detailed\nruntime markers to `python.log`. Use `--verbose-terminal` only when actively\ndebugging and you want the training subprocess to stream to the terminal.\n\nThe accepted 1B configuration is:\n\n```text\nbatch size: 16\nresident block capacity: 2048\nresident policy: balanced TopC\nresident lambda: 0.3\nrecency decay: 0.95\nbalanced seed fraction: 0.25\nprojection camera chunk: 2\nRAM cache budget: 32 GB\ncheckpoint mode: incremental\n```\n\n## Checkpoint And Resume\n\nRun 1000 iterations with an incremental checkpoint at 500:\n\n```bash\nGPU=0 \\\nRUN_TAG=$(date +\"%Y%m%d_%H%M%S\")_tidegs_ckpt1000 \\\nbash scripts\u002Ftrain_matrixcity_1b.sh \\\n  --mode checkpoint \\\n  --bsz 16 \\\n  --capacity 2048 \\\n  --checkpoint-iter 500 \\\n  --debug-max-train-cameras -1 \\\n  --debug-camera-sample-mode contiguous \\\n  --src \"$MATRIXCITY_SCENE_DIR\" \\\n  --ply \"$TIDEGS_DENSE_PLY\" \\\n  --manifest \"$TIDEGS_PREBUILT_MANIFEST\" \\\n  --decode-dataset-path \"$TIDEGS_DECODE_CACHE\" \\\n  --root \"$TIDEGS_ROOT\"\n```\n\nResume from the checkpoint:\n\n```bash\nCKPT=\u002Fpath\u002Fto\u002Frun\u002Fcheckpoints\u002F500\n\nGPU=0 \\\nRUN_TAG=$(date +\"%Y%m%d_%H%M%S\")_tidegs_resume500_to1500 \\\nbash scripts\u002Ftrain_matrixcity_1b.sh \\\n  --mode resume \\\n  --start-checkpoint \"$CKPT\" \\\n  --resume-to-iter 1500 \\\n  --bsz 16 \\\n  --capacity 2048 \\\n  --debug-max-train-cameras -1 \\\n  --debug-camera-sample-mode contiguous \\\n  --src \"$MATRIXCITY_SCENE_DIR\" \\\n  --ply \"$TIDEGS_DENSE_PLY\" \\\n  --decode-dataset-path \"$TIDEGS_DECODE_CACHE\" \\\n  --root \"$TIDEGS_ROOT\"\n```\n\nIncremental checkpoints save the training state, the log-structured storage\nindex, and the patch files needed by the latest block versions. They do not copy\nthe immutable 1B `base_file.bin`.\n\n## Outputs\n\nTraining logs, configurations, checkpoints, and SSD cache files are written under\n`$TIDEGS_ROOT\u002Foutput` and `$TIDEGS_ROOT\u002Fssd_cache`.\n\n## Acknowledgements\n\nThis repository builds on and takes important reference from\n[CLM-GS](https:\u002F\u002Fgithub.com\u002Fnyu-systems\u002FCLM-GS) and\n[gsplat](https:\u002F\u002Fgithub.com\u002Fnerfstudio-project\u002Fgsplat). We thank the authors\nfor releasing their code.\n\n## License\n\nTideGS is released under the Apache License 2.0. Third-party submodules and\ndependencies are governed by their own licenses.\n\n## Citation\n\n```bibtex\n@inproceedings{zhong2026tidegs,\n  title={{TideGS}: Scalable Training of Over One Billion 3D Gaussian Splatting Primitives via Out-of-Core Optimization},\n  author={Zhong, Chonghao and Shi, Linfeng and Chen, Hua and Sun, Tiecheng and Zhao, Hao and Yuan, Binhang and Li, Chaojian},\n  booktitle={International Conference on Machine Learning},\n  year={2026},\n  organization={PMLR}\n}\n```\n","TideGS 是一个用于训练大规模3D高斯点云场景的系统，采用基于SSD的外存优化技术。其核心功能包括在不将完整的高斯参数集保留在GPU内存中的情况下训练城市规模的3D高斯点云场景，以及在SSD、CPU内存和GPU驻留缓冲区之间流式传输高斯块。此外，它支持重复实验时重用预构建的SSD基底，并可以从增量检查点恢复训练而无需复制整个基底文件。TideGS适用于需要处理大规模3D数据集但受限于GPU内存容量的应用场景，如城市级3D重建或虚拟现实环境创建。","2026-06-11 03:56:10","CREATED_QUERY"]