[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-79859":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":9,"totalLinesOfCode":9,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":9,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":19,"rankGlobal":9,"rankLanguage":9,"license":9,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":20,"hasPages":20,"topics":9,"createdAt":9,"pushedAt":9,"updatedAt":22,"readmeContent":23,"aiSummary":24,"trendingCount":16,"starSnapshotCount":16,"syncStatus":25,"lastSyncTime":26,"discoverSource":27},79859,"nixl","ai-dynamo\u002Fnixl","ai-dynamo","NVIDIA Inference Xfer Library (NIXL)",null,"https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl","C++",1077,350,31,51,0,13,20,61.14,false,"main","2026-06-11 04:06:55","\u003C!--\nSPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\nSPDX-License-Identifier: Apache-2.0\n-->\n\n# NVIDIA Inference Xfer Library (NIXL)\n\nNVIDIA Inference Xfer Library (NIXL) is targeted for accelerating point to point communications in AI inference frameworks such as NVIDIA Dynamo, while providing an abstraction over various types of memory (e.g., CPU and GPU) and storage (e.g., file, block and object store) through a modular plug-in architecture.\n\n[![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-Apache_2.0-blue.svg)](https:\u002F\u002Fopensource.org\u002Flicenses\u002FApache-2.0)\n[![GitHub Release](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fv\u002Frelease\u002Fai-dynamo\u002Fnixl)](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Freleases\u002Flatest)\n\n## Documentation and Resources\n\n* [NIXL overview](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Fblob\u002Fmain\u002Fdocs\u002Fnixl.md) - Core concepts\u002Farchitecture overview (`docs\u002Fnixl.md`)\n\n* [Python API](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Fblob\u002Fmain\u002Fdocs\u002Fpython_api.md) - Python API usage and examples (`docs\u002Fpython_api.md`)\n\n* [Backend guide](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Fblob\u002Fmain\u002Fdocs\u002FBackendGuide.md) - Backend\u002Fplugin development guide (`docs\u002FBackendGuide.md`)\n\n* [Telemetry](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Fblob\u002Fmain\u002Fdocs\u002Ftelemetry.md) - Observability and telemetry details (`docs\u002Ftelemetry.md`)\n\n* [Doxygen guide](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Fblob\u002Fmain\u002Fdocs\u002Fdoxygen\u002Fnixl_doxygen.md) - API\u002Fclass diagrams overview (`docs\u002Fdoxygen\u002Fnixl_doxygen.md`)\n\n* [Doxygen images](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Ftree\u002Fmain\u002Fdocs\u002Fdoxygen) - Diagram assets (`docs\u002Fdoxygen\u002F`)\n\n* [NIXLBench docs](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Fblob\u002Fmain\u002Fbenchmark\u002Fnixlbench\u002FREADME.md) - Benchmark usage guide (`benchmark\u002Fnixlbench\u002FREADME.md`)\n\n* [KVBench docs](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Ftree\u002Fmain\u002Fbenchmark\u002Fkvbench\u002Fdocs) - KVBench workflows and tutorials (`benchmark\u002Fkvbench\u002Fdocs\u002F`)\n\n## Supported Platforms\nNIXL is supported on a Linux environment only. It is tested on Ubuntu (22.04\u002F24.04) and Fedora. macOS and Windows are not currently supported; use a Linux host or container\u002FVM.\n\n## Pre-build Distributions\n### PyPI Wheel\n\nThe nixl python API and libraries, including UCX, are available directly through PyPI.\nFor example, if you have a GPU running on a Linux host, container, or VM, you can do the following install:\n\nInstall with:\n\n```bash\npip install nixl\n```\n\nThis installs both CUDA 12 and CUDA 13 backends. At runtime, the correct backend is selected automatically based on the CUDA version reported by PyTorch.\n\n## Prerequisites for source build (Linux)\n### Ubuntu:\n\n`$ sudo apt install build-essential cmake pkg-config`\n\n### Fedora:\n\n`$ sudo dnf install gcc-c++ cmake pkg-config`\n\n### Python\n\n`$ pip3 install meson ninja pybind11 tomlkit`\n\n### UCX\n\nNIXL was tested with UCX version 1.21.x.\n\n[GDRCopy](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fgdrcopy) is available on Github and is necessary for maximum performance, but UCX and NIXL will work without it.\n\n```\n$ git clone https:\u002F\u002Fgithub.com\u002Fopenucx\u002Fucx.git\n$ cd ucx\n$ git checkout v1.21.x\n$ .\u002Fautogen.sh\n$ .\u002Fcontrib\u002Fconfigure-release-mt       \\\n    --enable-shared                    \\\n    --disable-static                   \\\n    --disable-doxygen-doc              \\\n    --enable-optimizations             \\\n    --enable-cma                       \\\n    --enable-devel-headers             \\\n    --with-cuda=\u003Ccuda install>         \\\n    --with-verbs                       \\\n    --with-dm                          \\\n    --with-gdrcopy=\u003Cgdrcopy install>\n$ make -j\n$ make -j install-strip\n$ ldconfig\n```\n\n### ETCD (Optional)\nNIXL can use ETCD for metadata distribution and coordination between nodes in distributed environments. To use ETCD with NIXL:\n#### ETCD Server and Client\n ```\n$ sudo apt install etcd etcd-server etcd-client\n\n# Or use Docker\n$ docker run -d -p 2379:2379 quay.io\u002Fcoreos\u002Fetcd:v3.5.1\n```\n\n#### ETCD CPP API\nInstalled from https:\u002F\u002Fgithub.com\u002Fetcd-cpp-apiv3\u002Fetcd-cpp-apiv3\n\n```\n$ sudo apt install libgrpc-dev libgrpc++-dev libprotobuf-dev protobuf-compiler-grpc\n$ sudo apt install libcpprest-dev\n$ git clone https:\u002F\u002Fgithub.com\u002Fetcd-cpp-apiv3\u002Fetcd-cpp-apiv3.git\n$ cd etcd-cpp-apiv3\n$ mkdir build && cd build\n$ cmake ..\n$ make -j$(nproc) && make install\n```\n\n### Additional plugins\n\nSome plugins may have additional build requirements, see them here:\n\n- [Mooncake](src\u002Fplugins\u002Fmooncake\u002FREADME.md)\n- [POSIX](src\u002Fplugins\u002Fposix\u002FREADME.md)\n- [GDS](src\u002Fplugins\u002Fcuda_gds\u002FREADME.md)\n\n## Getting started\n### Build & install\n\n```\n$ meson setup \u003Cname_of_build_dir>\n$ cd \u003Cname_of_build_dir>\n$ ninja\n$ ninja install\n```\n\n### Build Options\n\n#### Release build (default)\n\n```bash\n$ meson setup \u003Cname_of_build_dir>\n```\n\n#### Debug build\n\n```bash\n$ meson setup \u003Cname_of_build_dir> --buildtype=debug\n```\n\n#### NIXL-specific build options\n\n```bash\n# Example with custom options\n$ meson setup \u003Cname_of_build_dir> \\\n    -Dbuild_docs=true \\           # Build Doxygen documentation\n    -Ducx_path=\u002Fpath\u002Fto\u002Fucx \\     # Custom UCX installation path\n    -Dinstall_headers=true \\      # Install development headers\n    -Ddisable_gds_backend=false   # Enable GDS backend\n```\n\nCommon build options:\n- `build_docs`: Build Doxygen documentation (default: false)\n- `ucx_path`: Path to UCX installation (default: system path)\n- `install_headers`: Install development headers (default: true)\n- `disable_gds_backend`: Disable GDS backend (default: false)\n- `cudapath_inc`, `cudapath_lib`: Custom CUDA paths\n- `static_plugins`: Comma-separated list of plugins to build statically\n- `enable_plugins`: Comma-separated list of plugins to build (e.g. `-Denable_plugins=UCX,POSIX`). Cannot be used with `disable_plugins`.\n- `disable_plugins`: Comma-separated list of plugins to exclude (e.g. `-Ddisable_plugins=GDS`). Cannot be used with `enable_plugins`.\n\n#### Environment Variables\n\nThere are a few environment variables that can be set to configure the build:\n- `NIXL_NO_STUBS_FALLBACK`: If not set or 0, build NIXL stub library if the library build fails\n\n### Building Documentation\n\nIf you have Doxygen installed, you can build the documentation:\n\n```bash\n# Configure with documentation enabled\n$ meson setup \u003Cname_of_build_dir> -Dbuild_docs=true\n$ cd \u003Cname_of_build_dir>\n$ ninja\n\n# Documentation will be generated in \u003Cname_of_build_dir>\u002Fhtml\n# After installation (ninja install), documentation will be available in \u003Cprefix>\u002Fshare\u002Fdoc\u002Fnixl\u002F\n```\n\n### Python Interface\n\nNIXL provides Python bindings through pybind11. For detailed Python API documentation, see [docs\u002Fpython_api.md](docs\u002Fpython_api.md).\n\nThe preferred way to install the Python bindings is through pip from PyPI:\n\n```bash\npip install nixl\n```\n\nThis installs both CUDA 12 and CUDA 13 backends. At runtime, the correct backend is selected automatically based on the CUDA version reported by PyTorch.\n\n#### Installation from source\n\nPrerequisites:\n\n- `uv`: https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002Fgetting-started\u002Finstallation\u002F\n- `tomlkit`: https:\u002F\u002Fpypi.org\u002Fproject\u002Ftomlkit\u002F\n- `PyTorch`: https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F\n\n`uv` is always required *even if* you have another kind of Python virtual environment manager or if you are using a system-wide Python installation without using a virtual environment.\n\nExample with `uv` Python virtual environment:\n\n```\ncurl -LsSf https:\u002F\u002Fastral.sh\u002Fuv\u002Finstall.sh | sh\nexport PATH=\"$HOME\u002F.local\u002Fbin:${PATH}\"\n\nuv venv .venv --python 3.12\nsource .venv\u002Fbin\u002Factivate\nuv pip install tomlkit\n```\n\nExample with python-virtualenv:\n\n```\ncurl -LsSf https:\u002F\u002Fastral.sh\u002Fuv\u002Finstall.sh | sh\nexport PATH=\"$HOME\u002F.local\u002Fbin:${PATH}\"\n\npython3 -m venv .venv\nsource .venv\u002Fbin\u002Factivate\npip install tomlkit\n```\n\nExample with system-wide Python installation without using a virtual environment:\n\n```\ncurl -LsSf https:\u002F\u002Fastral.sh\u002Fuv\u002Finstall.sh | sh\nexport PATH=\"$HOME\u002F.local\u002Fbin:${PATH}\"\n\npip install tomlkit\n```\n\nThen install PyTorch following the instructions on the PyTorch website: https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F\n\nAfter installing the prerequisites, you can build and install the NIXL binaries and the Python bindings from source. You have to:\n\n1. Build NIXL binaries and install them\n2. Build and install the CUDA platform-specific package (`nixl-cu12` or `nixl-cu13`)\n3. Build and install the `nixl` meta-package\n\n**For CUDA 12:**\n\n```\npip install .\nmeson setup build\nninja -C build install\npip install build\u002Fsrc\u002Fbindings\u002Fpython\u002Fnixl-meta\u002Fnixl-*-py3-none-any.whl\n```\n\n**For CUDA 13:**\n\n```\npip install .\n.\u002Fcontrib\u002Ftomlutil.py --wheel-name nixl-cu13 pyproject.toml\nmeson setup build\nninja -C build install\npip install build\u002Fsrc\u002Fbindings\u002Fpython\u002Fnixl-meta\u002Fnixl-*-py3-none-any.whl\n```\n\nTo check if the installation is successful, you can run the following command:\n\n```\npython3 -c \"import nixl; agent = nixl.nixl_agent('agent1')\"\n```\n\nwhich should print:\n\n```\n2026-01-08 13:36:27 NIXL INFO    _api.py:363 Backend UCX was instantiated\n2026-01-08 13:36:27 NIXL INFO    _api.py:253 Initialized NIXL agent: agent1\n```\n\nYou can also run a complete Python example to test the installation:\n\n```\npython3 examples\u002Fpython\u002Fexpanded_two_peers.py --mode=target --use_cuda=true --ip=127.0.0.1 --port=4242 &\nsleep 5\npython3 examples\u002Fpython\u002Fexpanded_two_peers.py --mode=initiator --use_cuda=true --ip=127.0.0.1 --port=4242\n```\n\nFor more Python examples, see [examples\u002Fpython\u002F](examples\u002Fpython\u002F).\n\n### Rust Bindings\n#### Build\n- Use `-Drust=true` meson option to build rust bindings.\n- Use `--buildtype=debug` for a debug build (default is release).\n- Or build manually:\n    ```bash\n    $ cargo build --release\n    ```\n#### Install\nThe bindings will be installed under `nixl-sys` in the configured installation prefix.\nCan be done using ninja, from project build directory:\n```bash\n$ ninja install\n```\n\n#### Test\n```\n# Rust bindings tests\n$ cargo test\n```\n\nUse in your project by adding to `Cargo.toml`:\n```toml\n[dependencies]\nnixl-sys = { path = \"path\u002Fto\u002Fnixl\u002Fbindings\u002Frust\" }\n```\n\n### Other build options\nSee [contrib\u002FREADME.md](contrib\u002FREADME.md) for more build options.\n\n### Building Docker container\nTo build the docker container, first clone the current repository. Also make sure you are able to pull docker images to your machine before attempting to build the container.\n\nRun the following from the root folder of the cloned NIXL repository:\n```\n$ .\u002Fcontrib\u002Fbuild-container.sh\n```\n\nBy default, the container is built with Ubuntu 24.04. To build a container for Ubuntu 22.04 use the --os option as follows:\n```\n$ .\u002Fcontrib\u002Fbuild-container.sh --os ubuntu22\n```\n\nTo see all the options supported by the container use:\n```\n$ .\u002Fcontrib\u002Fbuild-container.sh -h\n```\n\nThe container also includes a prebuilt python wheel in \u002Fworkspace\u002Fdist if required for installing\u002Fdistributing. Also, the wheel can be built with a separate script (see below).\n\n### Building the python wheel\nThe contrib folder also includes a script to build the python wheel with the UCX dependencies. Note, that UCX and other NIXL dependencies are required to be installed.\n```\n$ .\u002Fcontrib\u002Fbuild-wheel.sh\n```\n\n## Running with ETCD\nNIXL can use ETCD for metadata exchange between distributed nodes. This is especially useful in containerized or cloud-native environments.\n\n### Environment Setup\nTo use ETCD with NIXL, set the following environment variables:\n\n```bash\n# Set ETCD endpoints (required) - replace localhost with the hostname of the etcd server\nexport NIXL_ETCD_ENDPOINTS=\"http:\u002F\u002Flocalhost:2379\"\n\n# Set ETCD namespace (optional, defaults to \u002Fnixl\u002Fagents)\nexport NIXL_ETCD_NAMESPACE=\"\u002Fnixl\u002Fagents\"\n```\n\n### Running the ETCD Example\nNIXL includes an example demonstrating metadata exchange and data transfer using ETCD:\n\n```bash\n# Start an ETCD server if not already running\n# For example:\n# docker run -d -p 2379:2379 quay.io\u002Fcoreos\u002Fetcd:v3.5.1\n\n# Set the ETCD env variables as above\n\n# Run the example. The two agents in the example will exchange metadata through ETCD\n# and perform data transfers\n.\u002F\u003Cnixl_build_path>\u002Fexamples\u002Fnixl_etcd_example\n```\n\n### nixlbench Benchmark\nFor more comprehensive testing, the nixlbench benchmarking tool supports ETCD for worker coordination:\n\n```bash\n# Build nixlbench (see benchmark\u002Fnixlbench\u002FREADME.md for details)\ncd benchmark\u002Fnixlbench\nmeson setup build && cd build && ninja\n\n# Run benchmark with ETCD\n.\u002Fnixlbench --etcd-endpoints http:\u002F\u002Flocalhost:2379 --backend UCX --initiator_seg_type VRAM\n```\n\n## Code Examples\n\n* [C++ examples](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Ftree\u002Fmain\u002Fexamples\u002Fcpp)\n\n* [Python examples](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Ftree\u002Fmain\u002Fexamples\u002Fpython)\n\n## Contributing\n\nFor contribution guidelines, see [CONTRIBUTING.md](https:\u002F\u002Fgithub.com\u002Fai-dynamo\u002Fnixl\u002Fblob\u002Fmain\u002FCONTRIBUTING.md) (`CONTRIBUTING.md`).\n\n## Third-Party Components\n\nThis project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.\n","NVIDIA Inference Xfer Library (NIXL) 是一个用于加速AI推理框架中点对点通信的库，特别针对NVIDIA Dynamo等应用。它通过模块化的插件架构提供对多种类型内存（如CPU和GPU）和存储（如文件、块和对象存储）的抽象支持。NIXL使用C++开发，具有Python API接口，并且在设计上考虑了可扩展性和灵活性，使得开发者能够轻松集成不同的后端存储解决方案。此项目适用于需要高性能数据传输的AI应用场景，尤其是在Linux环境下进行深度学习模型部署时。",2,"2026-06-11 03:58:20","trending"]