[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-82788":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":14,"forks30d":14,"starsTrendScore":18,"compositeScore":19,"rankGlobal":9,"rankLanguage":9,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":23,"hasPages":21,"topics":24,"createdAt":9,"pushedAt":9,"updatedAt":25,"readmeContent":26,"aiSummary":27,"trendingCount":14,"starSnapshotCount":14,"syncStatus":28,"lastSyncTime":29,"discoverSource":30},82788,"lerobot-rgb-rgbd-vla-dataset-toolkit","dsta022\u002Flerobot-rgb-rgbd-vla-dataset-toolkit","dsta022","Toolkit for collecting, merging, auditing, visualizing, and publishing RGB\u002FRGB-D LeRobot VLA datasets.",null,"Python",363,93,17,0,6,137,267,62,5.92,"MIT License",false,"main",true,[],"2026-06-12 02:04:28","# LeRobot RGB\u002FRGB-D VLA Dataset Toolkit\n\n[![Hugging Face Dataset](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHugging%20Face-DerekLX%2Flerobot__derek__depth-yellow)](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FDerekLX\u002Flerobot_derek_depth)\n[![Python](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPython-3.12%2B-blue)](https:\u002F\u002Fwww.python.org\u002F)\n[![LeRobot](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FData-LeRobot%20RGB%2FRGB--D-green)](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Flerobot)\n\nThis software project provides a LeRobot RGB\u002FRGB-D dataset toolkit for generating and auditing VLA train-ready datasets. It supports existing RGB-only datasets in the standard LeRobot ecosystem and RGB-D datasets with optional depth sidecars. RGB video is handled as a first-class data stream together with robot actions, states, metadata, and depth when available.\n\nThe toolkit is designed for Orbbec RGB-D collection, merging multiple same-structure LeRobot datasets into one real dataset, auditing RGB video\u002Faction\u002Fmetadata quality with optional depth checks, building a cleaned copy from reviewed episode lists, and uploading the final dataset to Hugging Face.\n\nIt also includes an Orbbec RGB-D data collection overlay that can be copied directly into the official Hugging Face LeRobot repository and used from the LeRobot repo root: https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Flerobot. A visuo-tactile data collection pipeline is also in progress, and its dataset and toolkit will be released open source when ready.\n\n## Dataset\n\nThe merged RGB-D VLA dataset is available on Hugging Face:\n\n```text\nhttps:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FDerekLX\u002Flerobot_derek_depth\n```\n\nRepository:\n\n```text\nDerekLX\u002Flerobot_derek_depth\n```\n\nThe Hugging Face dataset root is intended to contain the LeRobot subdirectories directly:\n\n```text\ndata\u002F\ndepth_sidecar\u002F\nmeta\u002F\nvideos\u002F\n```\n\n## Dataset Preview\n\nThe examples below are sampled from one `pick_up_cups_dataset` episode with two cups in the scene.\n\n![Dataset preview: global robot collection view, wrist camera view, and front depth visualization](assets\u002Freadme\u002Fthree_view_preview.png)\n\nLeft to right: global robot collection view, wrist camera view, and front depth visualization.\n\nTask data collection:\n\n\u003Cvideo src=\"assets\u002Freadme\u002Ftask_demo.mp4\" controls width=\"720\">\u003C\u002Fvideo>\n\nIf the video does not render in your Markdown viewer, open [task_demo.mp4](assets\u002Freadme\u002Ftask_demo.mp4) directly.\n\nAligned episode overview matrices:\n\n| Front camera, 9 sampled episodes |\n| --- |\n| ![Front camera episode matrix](assets\u002Freadme\u002Ffront_episode_matrix.jpg) |\n\n| Wrist camera, 9 sampled episodes |\n| --- |\n| ![Wrist camera episode matrix](assets\u002Freadme\u002Fwrist_episode_matrix.jpg) |\n\n| Front depth matrix, 9 sampled episodes |\n| --- |\n| ![Front depth episode matrix](assets\u002Freadme\u002Fdepth_episode_matrix.png) |\n\n## Overview\n\nMain capabilities:\n\n- collect Orbbec RGB-D data through a LeRobot overlay;\n- merge multiple LeRobot-style datasets into one physical dataset;\n- keep the output as real files, not junctions or symbolic links;\n- rewrite episode indices, frame indices, task indices, metadata, parquet tables, video references, and depth sidecars;\n- audit RGB-only LeRobot datasets and RGB-D datasets with depth sidecars;\n- audit RGB videos, optional depth sidecars, action\u002Fstate tables, metadata completeness, and deterministic quality failures;\n- separate episodes into `keep`, `review`, and `drop` lists;\n- build a clean train-ready dataset from `drop_episodes.txt`;\n- upload the dataset to Hugging Face with resumable large-folder upload support;\n- visualize depth PNG files for inspection.\n\n\n## Getting Started\n\nWe recommend using a dedicated Python environment. Python 3.12 or newer is recommended if you also use the Orbbec RGB-D capture overlay.\n\nCreate and activate a conda environment:\n\n```powershell\nconda create -n vla_data_check python=3.12 -y\nconda activate vla_data_check\n```\n\nInstall Python dependencies:\n\n```powershell\npython -m pip install --upgrade pip\npython -m pip install -r requirements.txt\n```\n\nFull video decoding and video trimming require `ffmpeg` and `ffprobe` command-line tools. If they are not already available in the environment, install them with conda:\n\n```powershell\nconda install -c conda-forge ffmpeg -y\nffmpeg -version\nffprobe -version\n```\n\nEnable faster Hugging Face uploads:\n\n```powershell\n$env:HF_XET_HIGH_PERFORMANCE=\"1\"\n```\n\n## Orbbec RGB-D Data Collection\n\nFolder:\n\n```text\nlerobot_rgbd_capture\u002F\n```\n\nPurpose: add Orbbec Femto Bolt RGB-D recording to a LeRobot checkout. This makes the pipeline cover capture -> merge -> audit -> clean -> upload.\n\nScope: this folder is for data collection only. It provides an Orbbec camera backend, RGB-D recording scripts, and depth sidecar storage helpers. It does not contain this project's depth model training or inference implementation.\n\nThe capture overlay is designed to be copied directly into the official Hugging Face LeRobot repository and used from there:\n\n```text\nhttps:\u002F\u002Fgithub.com\u002Fhuggingface\u002Flerobot\n```\n\nCopy the overlay contents into the root of a `huggingface\u002Flerobot` checkout. The target is the directory that contains LeRobot's own `pyproject.toml` and `src\u002Flerobot\u002F`.\n\nDo not copy `lerobot_rgbd_capture\u002F` as a nested subfolder under LeRobot. Merge its contents into the LeRobot repo root:\n\n```powershell\n$LEROBOT_ROOT=\"E:\\path\\to\\lerobot\"\nCopy-Item -Recurse -Force .\\lerobot_rgbd_capture\\examples $LEROBOT_ROOT\\\nCopy-Item -Recurse -Force .\\lerobot_rgbd_capture\\src $LEROBOT_ROOT\\\nCopy-Item -Force .\\lerobot_rgbd_capture\\pyproject.toml $LEROBOT_ROOT\\\n```\n\nAfter copying, the required files should appear directly under the LeRobot checkout:\n\n```text\n\u003Clerobot_root>\u002Fexamples\u002Frgbd_vla_depth\u002F\n\u003Clerobot_root>\u002Fsrc\u002Flerobot\u002Fcameras\u002Forbbec\u002F\n\u003Clerobot_root>\u002Fsrc\u002Flerobot\u002Fscripts\u002F\n\u003Clerobot_root>\u002Fsrc\u002Flerobot\u002Frobots\u002Fso_follower\u002F\n\u003Clerobot_root>\u002Fpyproject.toml\n```\n\nInstall inside the LeRobot repo root:\n\n```powershell\ncd $LEROBOT_ROOT\npython -m pip install -e \".[feetech,orbbec,rgbd_vla]\" -e .\\examples\\rgbd_vla_depth\n```\n\nFind the Orbbec camera:\n\n```powershell\nlerobot-find-cameras orbbec\n```\n\nWrite the camera serial and robot ports into:\n\n```text\nexamples\u002Frgbd_vla_depth\u002Fconfigs\u002Frecord_rgbd.yaml\n```\n\nRecord:\n\n```powershell\npython .\\examples\\rgbd_vla_depth\\record.py --config_path=.\\examples\\rgbd_vla_depth\\configs\\record_rgbd.yaml\n```\n\nDepth is saved as lossless uint16 PNG sidecars under `depth_sidecar\u002F`, with a recording manifest at `meta\u002Frgbd_vla_depth_recording.json`.\n\n## Merge Datasets\n\nUse `merge_lerobot_datasets.py` to merge multiple `*_dataset` folders into one real LeRobot dataset.\n\n```powershell\npython .\\merge_lerobot_datasets.py `\n  --src-root . `\n  --out-dir .\\lerobot_derek_depth `\n  --dataset-glob \"*_dataset\" `\n  --copy-mode copy\n```\n\nOverwrite an existing output:\n\n```powershell\npython .\\merge_lerobot_datasets.py --out-dir .\\lerobot_derek_depth --overwrite\n```\n\n## Audit Dataset Quality\n\nUse `lerobot_data_quality_audit.py` to run conservative quality checks. Hard deterministic failures are marked as `drop`; suspicious but uncertain episodes are marked as `review`. The audit never deletes source data.\n\nRGB is in scope. The audit checks RGB video existence, real-file status, empty files, ffprobe dimensions\u002Fcodec when available, full ffmpeg decode when requested, and sampled black\u002Fwhite\u002Fnear-constant\u002Ffrozen video content. Existing RGB-only LeRobot datasets do not need a `depth_sidecar\u002F` directory. When depth sidecars are present, the audit checks them separately through PNG structure, dimensions, optional full decode, invalid-depth ratio, and constant-depth samples.\n\n```powershell\npython .\\lerobot_data_quality_audit.py .\\lerobot_derek_depth\n```\n\nDefault audit output:\n\n```text\nquality_audit_reports\u002Flerobot_derek_depth\u002F\n  quality_report.csv\n  quality_summary.json\n  keep_episodes.txt\n  review_episodes.txt\n  drop_episodes.txt\n```\n\nRun with video decode and sampled content checks:\n\n```powershell\npython .\\lerobot_data_quality_audit.py .\\lerobot_derek_depth `\n  --depth-check header `\n  --video-check decode `\n  --video-content-check sample `\n  --sample-frames 8 `\n  --depth-sample-frames 8\n```\n\n## Check Completeness\n\nUse `utils\u002Fcheck_lerobot_completeness.py` for structure-only validation after merging or cleaning.\n\n```powershell\npython .\\utils\\check_lerobot_completeness.py .\\lerobot_derek_depth\n```\n\n## Build a Clean Dataset\n\nUse `build_clean_lerobot_dataset.py` to create a new physical dataset from `drop_episodes.txt`. The original dataset is preserved.\n\nRecommended workflow:\n\n1. Run the quality audit.\n2. Inspect `quality_report.csv` and `review_episodes.txt`.\n3. Add manually confirmed bad episodes to `drop_episodes.txt`.\n4. Build the clean dataset.\n\n```powershell\npython .\\build_clean_lerobot_dataset.py `\n  .\\lerobot_derek_depth `\n  --drop-list .\\quality_audit_reports\\lerobot_derek_depth\\drop_episodes.txt `\n  --out-dir .\\lerobot_derek_depth_clean `\n  --video-mode trim-reencode\n```\n\nNotes:\n\n- `trim-reencode` trims videos per kept episode and physically removes dropped episode segments.\n- `copy-referenced` is faster, but copied mp4 files may still contain unreferenced old segments.\n- The output includes `cleaning_manifest.json`.\n\n## Upload to Hugging Face\n\nUse `push_lerobot_dataset_to_hf.py` to upload a local dataset directory to a Hugging Face Dataset repository.\n\nRecommended token setup:\n\n```powershell\n$env:HF_TOKEN=\"hf_your_write_token\"\n```\n\nThe script also contains editable local defaults:\n\n```python\nHF_USERNAME = \"your_huggingface_username\"\nHF_WRITE_TOKEN = \"hf_your_write_token_here\"\nHF_DATASET_NAME = \"your_dataset_name\"\n```\n\nAlways run a dry run first:\n\n```powershell\npython .\\push_lerobot_dataset_to_hf.py --dry-run\n```\n\nUpload:\n\n```powershell\n$env:HF_XET_HIGH_PERFORMANCE=\"1\"\npython .\\push_lerobot_dataset_to_hf.py\n```\n\nUpload a cleaned dataset instead:\n\n```powershell\npython .\\push_lerobot_dataset_to_hf.py `\n  --dataset-dir .\\lerobot_derek_depth_clean `\n  --dataset-name lerobot_derek_depth_clean\n```\n\n## Visualize Depth\n\nUse `visualize_lerobot_depth.py` to inspect depth PNG statistics or generate visualization images.\n\nPrint statistics:\n\n```powershell\npython .\\visualize_lerobot_depth.py .\\lerobot_derek_depth\\depth_sidecar --stats\n```\n\nGenerate visualization images:\n\n```powershell\npython .\\visualize_lerobot_depth.py .\\lerobot_derek_depth\\depth_sidecar --out .\\depth_vis\n```\n\n## Run Tests\n\nThe repository includes a small synthetic LeRobot RGB-D dataset test. It does not use the real local datasets.\n\n```powershell\npython -m unittest discover -s tests\n```\n\n## Dataset Citation\n\nIf this dataset is useful for your work, please cite or link the Hugging Face dataset:\n\n```bibtex\n@dataset{dereklx_lerobot_derek_depth_2026,\n  author    = {DerekLX},\n  title     = {lerobot_derek_depth},\n  year      = {2026},\n  publisher = {Hugging Face},\n  url       = {https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FDerekLX\u002Flerobot_derek_depth}\n}\n```\n","LeRobot RGB\u002FRGB-D VLA Dataset Toolkit 是一个用于收集、合并、审核、可视化和发布 LeRobot VLA 数据集的工具包。它支持标准 LeRobot 生态系统中的 RGB 和 RGB-D 数据集，能够处理视频流与机器人动作、状态、元数据及深度信息（如果可用）。该工具包特别设计用于 Orbbec RGB-D 数据采集，可以将多个相同结构的 LeRobot 数据集合并为一个真实数据集，并对 RGB 视频、动作和元数据的质量进行审核，同时可选地检查深度信息。此外，它还提供了从已审查剧集列表构建清理副本的功能，并能将最终数据集上传至 Hugging Face。适用于需要高质量视觉-触觉数据的研究者或开发者，在机器人学习和计算机视觉领域尤为有用。",2,"2026-06-11 04:09:13","CREATED_QUERY"]