[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-82798":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},82798,"cosmos-framework","NVIDIA\u002Fcosmos-framework","NVIDIA","Our inference and training framework to run on the Cosmos Models","https:\u002F\u002Fgithub.com\u002Fnvidia\u002Fcosmos",null,"Python",222,29,5,4,0,6,67,168,38,86.43,"Other",false,"main",true,[],"2026-06-12 04:01:39","\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F28f2d612-bbd6-44a3-8795-833d05e9f05f\" width=\"274\" alt=\"NVIDIA Cosmos\"\u002F>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FCosmos\">NVIDIA Cosmos\u003C\u002Fa> |\n    🤗 \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Fnvidia\u002Fcosmos3\">Cosmos 3 \u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n    Part of the \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FCosmos\">NVIDIA Cosmos\u003C\u002Fa> project family — the training and serving framework repository.\n\u003C\u002Fp>\n\n# Cosmos-Framework\n\n**Cosmos-Framework** is an end-to-end framework for training and serving world models, including the **Cosmos3** model family. Everything lives in a single top-level [`cosmos_framework\u002F`](.\u002Fcosmos_framework) Python package:\n\n- **Training** — distributed FSDP \u002F TP \u002F CP \u002F PP trainer, native DCP checkpoints with HuggingFace `safetensors` import\u002Fexport, JSONL \u002F WebDataset \u002F LeRobot dataset adapters. Entry point: `cosmos_framework.scripts.train`. See [`docs\u002Ftraining.md`](.\u002Fdocs\u002Ftraining.md).\n- **Inference** — Diffusers \u002F Transformers \u002F vLLM backends with offline batch generation and online serving (Ray + Gradio). Entry point: `cosmos_framework.scripts.inference`. Ecosystem-facing shim libraries (lightweight standalone wrappers for downstream projects) live under [`packages\u002F`](.\u002Fpackages).\n\n## Cosmos 3\n\n**Cosmos 3** is our newest model family [[Report]](https:\u002F\u002Fresearch.nvidia.com\u002Flabs\u002Fcosmos-lab\u002Fcosmos3\u002Ftechnical-report.pdf) [[Website]](https:\u002F\u002Fresearch.nvidia.com\u002Flabs\u002Fcosmos-lab\u002Fcosmos3\u002F). It is a suite of omnimodal world models designed to jointly process and generate language, images, video, audio, and action sequences within a unified Mixture-of-Transformers architecture. By supporting highly flexible input-output configurations, it seamlessly unifies critical modalities for Physical AI — effectively subsuming vision-language models, video generators, world simulators, and world-action models into a single framework. For a guided experience to test out Cosmos3, please visit [[Cosmos]](https:\u002F\u002Fgithub.com\u002Fnvidia\u002Fcosmos).\n\n## Framework Documentation\n\n- [Quickstart](#setup)\n- [Setup](.\u002Fdocs\u002Fsetup.md)\n- [Training (Supervised Fine-Tuning)](.\u002Fdocs\u002Ftraining.md)\n  - [JSONL Dataset](.\u002Fdocs\u002Fdataset_jsonl.md)\n- [Inference](.\u002Fdocs\u002Finference.md)\n- Reference\n  - [Code Structure](.\u002Fdocs\u002Fcode_structure.md)\n  - [Environment Variables](.\u002Fdocs\u002Fenvironment_variables.md)\n  - [FAQ](.\u002Fdocs\u002Ffaq.md)\n  - [AGENTS.md](.\u002FAGENTS.md)\n\n## Setup\n\nFor more details and alternative installation methods, see [Setup](.\u002Fdocs\u002Fsetup.md#installation). Before installing, make sure your machine meets the [System Requirements](.\u002Fdocs\u002Fsetup.md#system-requirements). If you want a curated PyTorch + CUDA environment, start from the [recommended NVIDIA NGC base image](.\u002Fdocs\u002Fsetup.md#recommended-base-image).\n\nInstall system dependencies:\n\n```shell\nsudo apt-get install -y --no-install-recommends curl ffmpeg git-lfs libx11-dev tree wget\n```\n\nInstall the package with `uv` (pick the dependency group that matches your CUDA toolkit — see [CUDA Variants](.\u002Fdocs\u002Fsetup.md#cuda-variants)):\n\n```shell\n# CUDA 13.0 (recommended)\nuv sync --all-extras --group=cu130-train\n# Or, for CUDA 12.8:\n# uv sync --all-extras --group=cu128-train\nsource .venv\u002Fbin\u002Factivate && export LD_LIBRARY_PATH=\n```\n\nIf you are starting from the recommended NGC image (`nvcr.io\u002Fnvidia\u002Fpytorch:25.09-py3`), see the [one-shot quickstart](.\u002Fdocs\u002Fsetup.md#quickstart-from-the-recommended-base-image).\n\n## Training\n\nFor the full guide (data preparation, base-checkpoint conversion, parallelism strategies, mixed precision, resuming), see [Training](.\u002Fdocs\u002Ftraining.md). The number of GPUs required depends on the recipe; the shipped recipes under [`examples\u002F`](.\u002Fexamples\u002FREADME.md) are 8-GPU configurations (tested on 8× H100 80 GB) launched via their paired launch shells, e.g.:\n\n```shell\nbash examples\u002Flaunch_sft_vision_nano.sh\n```\n\nUsers may adjust the GPU count to match their model and underlying hardware architecture — tune `NPROC_PER_NODE` and the parallelism degrees (DP\u002FCP\u002FFSDP shard) in the recipe accordingly.\n\n## Inference\n\nSee [Inference](.\u002Fdocs\u002Finference.md) for the full guide — launch commands, supported modes, parallelism presets, and troubleshooting.\n\nQuick single-GPU launch:\n\n```shell\npython -m cosmos_framework.scripts.inference \\\n    --parallelism-preset=latency \\\n    -i \"inputs\u002Fomni\u002Ft2v.json\" \\\n    -o outputs\u002Fomni_nano \\\n    --checkpoint-path Cosmos3-Nano \\\n    --seed=0\n```\n\n## Reference\n\n| Topic                                                        | What it covers                                                                                                           |\n| ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |\n| [Setup](.\u002Fdocs\u002Fsetup.md)                                     | Hardware\u002Fsoftware prerequisites, `uv` install paths, CUDA variants, Docker base image, and base-checkpoint downloading.  |\n| [Code Structure](.\u002Fdocs\u002Fcode_structure.md)                   | Repository layout and a per-subpackage tour of `cosmos_framework\u002F` — where each concern lives and where to add new code. |\n| [Training](.\u002Fdocs\u002Ftraining.md)                               | Launching multi-GPU and multi-node runs; parallelism strategies; mixed precision; resuming.                              |\n| [Inference (from a trained checkpoint)](.\u002Fdocs\u002Finference.md) | Loading a trained checkpoint into one of the inference backends.                                                         |\n| [FAQ](.\u002Fdocs\u002Ffaq.md)                                         | Troubleshooting (OOM, NCCL hangs, slow training), environment variables, and common pitfalls.                            |\n","NVIDIA\u002Fcosmos-framework 是一个用于训练和部署世界模型的端到端框架，特别支持Cosmos3模型系列。其核心功能包括分布式训练（支持FSDP、TP、CP、PP）和多种推理后端（如Diffusers、Transformers、vLLM），并提供HuggingFace `safetensors`格式的数据检查点导入导出以及JSONL\u002FWebDataset\u002FLeRobot数据集适配器。该框架还具备离线批量生成与在线服务能力（通过Ray + Gradio实现）。此项目非常适合需要处理多模态数据（如文本、图像、视频、音频等）的应用场景，尤其是在物理AI领域内寻求统一架构解决方案的研究者和开发者使用。",2,"2026-06-11 04:09:16","CREATED_QUERY"]