[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80843":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":13,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":14,"forks30d":14,"starsTrendScore":17,"compositeScore":18,"rankGlobal":9,"rankLanguage":9,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":9,"pushedAt":9,"updatedAt":24,"readmeContent":25,"aiSummary":26,"trendingCount":14,"starSnapshotCount":14,"syncStatus":27,"lastSyncTime":28,"discoverSource":29},80843,"DAWN","COOWAI\u002FDAWN","COOWAI","official repo of \"The DAWN of World-Action Interactive Models\"",null,"Python",59,5,1,0,3,23,9,52.13,"MIT License",false,"main",true,[],"2026-06-12 04:01:30","\u003Cdiv align=\"center\">\n\n# The DAWN of World-Action Interactive Models\n\n**Hongbo Lu**\u003Csup>1,2,\\*\u003C\u002Fsup>, **Liang Yao**\u003Csup>1,\\*\u003C\u002Fsup>, **Chenghao He**\u003Csup>1,\\*\u003C\u002Fsup>, **Haoyu Wang**\u003Csup>1,\\*\u003C\u002Fsup>, \n\n**Xiang Gu**\u003Csup>3\u003C\u002Fsup>, **Xianfei Li**\u003Csup>1,2\u003C\u002Fsup>, **Wenlong Liao**\u003Csup>1,†\u003C\u002Fsup>, **Tao He**\u003Csup>1\u003C\u002Fsup>, **Pai Peng**\u003Csup>1,†,‡\u003C\u002Fsup>\n\n\u003Csup>1\u003C\u002Fsup>COWARobot Co. Ltd    \u003Csup>2\u003C\u002Fsup>Shanghai Jiao Tong University    \u003Csup>3\u003C\u002Fsup>Hohai University\n\n\\* Equal Contribution    † Corresponding Author    ‡ Project Lead\n\n\u003C\u002Fdiv>\n\n---\n\n# nuScenes Inference\n\nOffline inference\u002Fevaluation repo for world-model planners on nuScenes data converted to the NavSim PKL layout.\n\nThe entrypoint loads a training YAML, builds the encoder, predictor, and planner, restores a checkpoint, runs the `val_command.py` validation forward path, and writes metrics plus optional trajectory visualizations.\n\n## Setup\n\n```bash\npip install -e .\n```\n\nThe code expects the converted nuScenes\u002FNavSim PKL paths and model checkpoints referenced by the YAML to exist on the machine.\n\n## Model Weights\n\nDownload the required model weights before running inference.\n\n- Pretrained weights (used for `config.meta.pretrain_checkpoint_full`):\n  - `pretrained.pt`: https:\u002F\u002Fpan.baidu.com\u002Fs\u002F1SgmdMop50yd-Vv2nl0k2Mg Code: ukx8 \n- Inference checkpoint weights (used for `--checkpoint` or `best_open_loop.pt`):\n  - `best_open_loop.pt`: https:\u002F\u002Fpan.baidu.com\u002Fs\u002F1uVHHEGp8qEJR9Gz3XGS05Q Code: xhku \n\nFor example:\n\n```bash\nmkdir -p checkpoints\nwget -O checkpoints\u002Fpretrained.pt https:\u002F\u002Fexample.com\u002Fpath\u002Fto\u002Fpretrained.pt\nwget -O checkpoints\u002Fbest_open_loop.pt https:\u002F\u002Fexample.com\u002Fpath\u002Fto\u002Fbest_open_loop.pt\n```\n\nThen update your YAML or command line `--checkpoint` path accordingly.\n\n## Data Preparation\n\nConvert raw nuScenes data (labels JSON + GT box NPZ + CAN bus) to the NavSim PKL layout used by `NavSimWorldModelDataset`:\n\n```bash\npython tools\u002Fconvert_nuscenes_to_navsim_pkl.py \\\n  --nuscenes-root \u002Fpath\u002FnuScenes \\\n  --output-root \u002Fpath\u002FnuScenes\u002Fnavsim_format \\\n  --split trainval \\\n  --workers 8\n```\n\n**Input directory layout** (`--nuscenes-root`):\n\n```\n\u002Fpath\u002FnuScenes\u002F\n├── labels\u002F\n│   ├── scene-0001.json          # per-scene frame metadata\n│   ├── scene-0001\u002F              # per-scene GT box NPZ files\n│   │   ├── 0001.npz\n│   │   └── ...\n│   └── scene-0002.json\n├── samples\u002F                     # original camera images (symlinked, not copied)\n│   └── CAM_FRONT\u002F...\n└── can_bus\u002F\n    └── scene-0001_pose.json     # CAN bus ego pose (velocity, acceleration)\n```\n\n**Output directory layout** (`--output-root`):\n\n```\n\u002Fpath\u002FnuScenes\u002Fnavsim_format\u002F\n├── train\u002F\n│   ├── scene-0001.pkl           # List[Dict], each entry = one keyframe (2 Hz)\n│   └── ...\n├── val\u002F\n│   ├── scene-0003.pkl\n│   └── ...\n└── sensor_blobs\u002F\n    └── scene-0001\u002F\n        └── CAM_F0\u002F\n            └── n015-2018-...jpg  # symlinks to original images\n```\n\n**What gets converted per keyframe:**\n\n- `ego2global_translation` \u002F `ego2global_rotation` — from nuScenes ego pose matrix\n- `ego_dynamic_state` (`[vx, vy, ax, ay]` in ego frame) — interpolated from CAN bus pose data\n- `driving_command` (one-hot: `[GO_STRAIGHT \u002F TURN_LEFT \u002F TURN_RIGHT \u002F U_TURN]`) — inferred from cumulative yaw change over the scene\n- `cams` — only `CAM_FRONT` is retained; images are symlinked, not copied\n- `anns` (`GT boxes`) — NPZ annotations converted to NavSim format; categories mapped (`[vehicle\u002Fpedestrian\u002Fbicycle]`); barriers and traffic cones are filtered out\n\nScene split follows the standard nuScenes v1.0-trainval partition: 700 train + 150 val scenes.\n\n## Run\n\n```bash\nPYTHONPATH=\"$PWD\" \u002Fusr\u002Fbin\u002Fpython3 scripts\u002Finfer_nuscenes_val.py \\\n  --config configs\u002Fnuscenes_inference_example.yaml \\\n  --checkpoint \u002Fpath\u002Fto\u002Fbest_open_loop.pt \\\n  --output-dir outputs\u002Fnuscenes_eval \\\n  --disable-vis\n```\n\nIf `--checkpoint` is omitted, the script tries `\u003Cconfig.folder>\u002Fbest_open_loop.pt`, then `\u003Cconfig.folder>\u002Flatest.pt`, then `meta.resume_checkpoint`.\n\n## Data Contract\n\nThis repo targets nuScenes converted to the local NavSim-style PKL format used by `NavSimWorldModelDataset`:\n\n- `data.navsim.val_data_path`: directory containing validation `.pkl` scene files\n- `data.navsim.val_sensor_blobs_path`: camera image root\n- `data.navsim.camera_name`: camera folder name, usually `CAM_F0`\n\nThe validation path computes ADE\u002FFDE\u002FminADE@K\u002FminFDE@K, World4Drive L2 horizons, and collision metrics when BEV segmentation is present in the dataloader batch.\n","DAWN项目是一个用于世界-行为交互模型的官方仓库，主要基于nuScenes数据集进行离线推理和评估。该项目使用Python开发，支持构建编码器、预测器和规划器，并通过加载训练配置文件来恢复检查点，执行验证路径并输出度量结果及可选轨迹可视化。其核心功能包括处理转换后的nuScenes\u002FNavSim PKL格式数据，适用于自动驾驶领域的研究与开发场景中，特别是对于需要测试或改进世界模型规划算法的研究者而言非常有用。此外，提供了详细的设置指南和预训练权重下载链接，方便用户快速上手使用。",2,"2026-06-11 04:02:31","CREATED_QUERY"]