[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-9726":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":25,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":34,"readmeContent":35,"aiSummary":36,"trendingCount":16,"starSnapshotCount":16,"syncStatus":37,"lastSyncTime":38,"discoverSource":39},9726,"cutlass","NVIDIA\u002Fcutlass","NVIDIA","CUDA Templates and Python DSLs for High-Performance Linear Algebra","https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Findex.html",null,"C++",9882,1903,122,483,0,8,67,202,40,111.84,"Other",false,"main",true,[27,28,29,30,31,32,33],"cpp","cuda","deep-learning","deep-learning-library","gpu","nvidia","python","2026-06-12 04:00:46","![ALT](.\u002Fmedia\u002Fimages\u002Fgemm-hierarchy-with-epilogue-no-labels.png \"Complete CUDA GEMM decomposition\")\n# Overview\n\n# CUTLASS 4.5.0\n\n_CUTLASS 4.5.0 - May 2026_\n\nCUTLASS is a collection of abstractions for implementing high-performance matrix-matrix multiplication (GEMM)\nand related computations at all levels and scales within CUDA. It incorporates strategies for\nhierarchical decomposition and data movement. CUTLASS decomposes these \"moving parts\" into reusable, modular\nsoftware components and abstractions.\n\nPrimitives for different levels of a conceptual parallelization hierarchy can be specialized and tuned\nvia custom tiling sizes, data types, and other algorithmic policy. The resulting flexibility simplifies\ntheir use as building blocks within custom kernels and applications.\n\nCUTLASS has been providing CUDA C++ template abstractions for high-performance linear algebra since 2017 and\nthese abstractions provide extensive support for a wide range of computations including\nmixed-precision computations, specialized data-movement (async copy) and\nmultiply-accumulate abstractions for FP64, FP32, TF32, FP16, BF16,\n[FP32 emulation via tensor core instruction](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fcutlass\u002Ftree\u002Fmain\u002Fexamples\u002F27_ampere_3xtf32_fast_accurate_tensorop_gemm),\n 8b floating point types (e5m2 and e4m3),\n block scaled data types (NVIDIA NVFP4 and OCP standard MXFP4, MXFP6, MXFP8),\n narrow integer types (4 and 8b signed and unsigned integers),\n and binary 1b data types (where architectures allow for the\nnative support of such data types) across NVIDIA's Volta, Turing, Ampere, Ada, Hopper, and Blackwell architectures.\n\nTo this rich ecosystem of C++ based kernel programming abstractions, CUTLASS 4 adds CUTLASS DSLs. These are Python native interfaces for writing high-performance CUDA kernels based on core CUTLASS and CuTe concepts without any performance compromises. This allows for a much smoother learning curve, orders of magnitude faster compile times, native integration with DL frameworks without writing glue code, and much more intuitive metaprogramming that does not require deep C++ expertise.\n\nOverall we envision CUTLASS DSLs as a family of domain-specific languages (DSLs). With the release of 4.0, we are releasing the first of these in CuTe DSL. This is a low level programming model that is fully consistent with CuTe C++ abstractions — exposing core concepts such as layouts, tensors, hardware atoms, and full control over the hardware thread and data hierarchy.\n\nCuTe DSL demonstrates optimal matrix multiply and other linear algebra operations\ntargeting the programmable, high-throughput _Tensor Cores_ implemented by\nNVIDIA's Ampere, Hopper, and Blackwell architectures.\n\nWe believe it will become an indispensable tool for students, researchers, and performance\nengineers alike — flattening the learning curve of GPU programming, rapidly prototyping kernel\ndesigns, and bringing optimized solutions into production.\n\nCuTe DSL is currently in public beta and will graduate out of beta by end of summer 2025.\n\nTo get started quickly - please refer :\n  - [CUTLASS C++ Quick Start Guide](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fquickstart.html).\n  - [CuTe DSL Quick Start Guide](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002FpythonDSL\u002Fquick_start.html).\n\n# What's New in CUTLASS 4.5\n\n## CuTe DSL\n* New features\n  - New Block API `block_copy()` to simplify TMA and S2T copy. Users can ignore detail about multicast and 2CTA partition for TMA by `block_copy()` and need not to invoke `tma_partition()`. And users can remove bulk of S2T initialization to simplify S2T copy.\n  - MXF8F6F4 mixed precision supoort\n    - BlockScaled MMA now supports MXF8*MXF4 or MXF8*MXF6\n  - Block Scaled MMA for SM120 now works on Spark\n  - EFC broadcast semantics support\n    -  EFC epilogue functions can now broadcast and remap tensor modes via `C.remap_modes[:, 0, 1]` subscript syntax (where `:` marks a broadcast dimension and integers select source mode indices). Covers scalar broadcast, row\u002Fcolumn broadcast, and arbitrary mode permutations (e.g. transpose). The PyTorch reference evaluator mirrors the same transformations.\n  - Initial linter support: Improved type hints on CuTe DSL APIs to support static type checkers like MyPy\n  - dataclasses.dataclass is now supported for JIT compilaton and cute.compile for both plain and tvm-ffi path\n  - cute.copy now supports user specified loop unrolling\n\n* Bug fixing and improvements\n  - Improved source code correlation for profiling\u002Fdebugging\n  - Fixed an aarch64 segfault issue with tvm-ffi\n  - Re-organization for CuTe DSL examples\u002Ftutorials for better discoverability\n\n* More examples of authorizing peak-performance kernels\n  - MOE examles\n    - A new style of grouped-gemm that aligns to torch's grouped_mm and scaled_groued_mm interface.\n    - Expert-wise tensormap descriptor setup by a cheap helper kernel (~2us) to avoid long latency in tile switching, kernel structure is much more closer to a normal GEMM.\n    - Compared to torch_210_cu13, very few problem has worse perf in B200.\n        - mxfp8_2dx3d: avg 1.29 speedup;\n        - mxfp8_2dx2d: avg 1.41 speedup;\n            - nvfp4_2dx3d: avg 1.11 speedup;\n        - nvfp4_2dx2d: avg 1.12 speedup (worst case 0.98)\n        - bf16_2dx3d: avg 1.15 speedup (worst case 0.98)\n        - bf16_2dx2d: avg 1.17 speedup (worst case 0.96)\n        - Note: The perf is measured from torch profiler, this impl includes the helper kernel + main kernel, while torch's includes its setup kernel and cutlass_cpp main kernel.\n\n* API changes\n  - ab_dtype is deprecated in make_trivial_tiled_mma and make_blockscaled_trivial_tiled_mma from blackwell_helpers.py. Please specify a_dtype and b_dtype separately instead.\n\n## CUTLASS C++\n* Add 2SM MMA instruction support to mixed TMA+CpAsync SM100 vanilla GEMM kernels.\n  - Mixed TMA+CpAsync can now accept static, but non trivial cluster shapes.\n  - Uses TMA multicast for A tile when using non-trivial cluster size along N mode.\n  - Uses an additional barrier (mma_trampoline_barrier) to track cp.async arrivals in both CTAs.\n  - Changes included in [example 92](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fcutlass\u002Ftree\u002Fmain\u002Fexamples\u002F92_blackwell_moe_gemm).\n* Add support for 128x32xK and 128x64xK tile sizes for SM120 blockscaled MMA collective builders, yielding up to 30% performance improvement on Blackwell SM121 related kernels.\n* Add static load to tensor memory support, included in [example 77](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fcutlass\u002Ftree\u002Fmain\u002Fexamples\u002F77_blackwell_fmha\u002F).\n* Use 64-bit adds for SM100 MMA descriptor offsets and reduce move instructions for improved code generation.\n* Add [example 95](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fcutlass\u002Ftree\u002Fmain\u002Fexamples\u002F95_blackwell_gemm_green_context) to support green context SM partition\n  - Enables launching GEMM on stream with partial SM allocation.\n* Fix some kernel issues:\n  - Fix l2_capacity=0 handling in Blackwell SM100\u002FSM120 kernel templates\n  - Fix CUTLASS clang build issues\n  - Fix atomicCAS read-modify-write loop in `ConstSubbyteReference`\n  - Replace `__nv_atomic_load_n` with `volatile` for CUDA 11.4 compatibility in subbyte reference\n  - Remove `PipelineStorage` shadowing in SM100 complex epilogue\n  - Fix build issue in SM90 epilogue fusion visitor TMA warpspecialized\n* Fix some profiler issues:\n  - Add missing reference kernels for blockwise GEMM profiler\n* Various improvements and fixes from the community and CUTLASS team. Thanks to everyone who submitted PRs!\n* Optimal code generation with CUDA toolkit versions 13.2.\n\nNote: CUTLASS 4.x builds are known to be down on Windows platforms for all CUDA toolkits.\nCUTLASS team is working on a fix.\n\n**See the [CHANGELOG](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002FCHANGELOG.html) for details of all past releases and updates.**\n\n# Performance\n\nCUTLASS primitives are very efficient.  When used to construct device-wide GEMM kernels,\nthey exhibit nearly optimal utilization of peak theoretical throughput. The figure below\nshows CUTLASS 3.8's performance as a % of theoretical peak utilization\non various input and output data types when run on NVIDIA Blackwell SM100 architecture GPU.\n\n![ALT](media\u002Fimages\u002Fcutlass-3.8-blackwell-gemm-peak-performance.svg \"\")\n\nThe two figures below show the continual CUTLASS performance improvements\non an [NVIDIA H100](https:\u002F\u002Fwww.nvidia.com\u002Fen-us\u002Fdata-center\u002Fh100\u002F) (NVIDIA Hopper architecture) since\nCUTLASS 3.1.\nCUTLASS 3.5.1 was compiled with the [CUDA 12.5u1 Toolkit](https:\u002F\u002Fdeveloper.nvidia.com\u002Fcuda-downloads).\nTensor Core operations are implemented using CUDA's\n[mma](https:\u002F\u002Fdocs.nvidia.com\u002Fcuda\u002Fparallel-thread-execution\u002Findex.html#warp-level-matrix-instructions-mma) and\n[wgmma](https:\u002F\u002Fdocs.nvidia.com\u002Fcuda\u002Fparallel-thread-execution\u002Findex.html#asynchronous-warpgroup-level-matrix-instructions) instructions.\n\n![ALT](media\u002Fimages\u002Fcutlass-3.5.1-gemm-peak-performance.png \"\")\n![ALT](media\u002Fimages\u002Fcutlass-3.5.1-gemm-peak-performance-fp8.png \"\")\n\n# CuTe\n\nCUTLASS 3.0 introduced a new core library, CuTe, to describe and manipulate tensors of threads and data.\nCuTe is a collection of C++ CUDA template abstractions for\ndefining and operating on hierarchically multidimensional layouts of threads and data.\nCuTe provides `Layout` and `Tensor` objects that compactly package the type,\nshape, memory space, and layout of data, while performing the complicated indexing for the user.\nThis lets programmers focus on the logical descriptions of their algorithms while\nCuTe does the mechanical bookkeeping for them. With these tools, we can quickly design,\nimplement, and modify all dense linear algebra operations.\n\nThe core abstractions of CuTe are hierarchically multidimensional layouts\nwhich can be composed with data arrays to represent tensors.\nThe representation of layouts is powerful enough to represent nearly\neverything we need to implement efficient dense linear algebra.\nLayouts can also be combined and manipulated via functional composition, on which we build a large set of common operations such as tiling and partitioning.\n\nCUTLASS 3.0 and beyond adopts CuTe throughout the GEMM hierarchy in its templates.\nThis greatly simplifies the design and improves code composability and readability.\nMore documentation specific to CuTe can be found in its\n[dedicated documentation directory](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fcute\u002F00_quickstart.html).\n\n# Compatibility\n\nMinimum requirements:\n\n- Architecture: Volta (compute capability 7.0)\n- Compiler: Must support at least C++17\n- CUDA Toolkit version: 11.4\n\nCUTLASS requires a C++17 host compiler and\nperforms best when built with the [**CUDA 12.8 Toolkit**](https:\u002F\u002Fdeveloper.nvidia.com\u002Fcuda-downloads).\nIt is also compatible with CUDA 11.4, CUDA 11.5, CUDA 11.6, CUDA 11.7, CUDA 11.8, and all other CUDA 12.x versions.\n\n## Operating Systems\n\nWe have tested the following environments.\n\n|**Operating System** | **Compiler** |\n|-----------------|----------|\n| Ubuntu 18.04 | GCC 7.5.0  |\n| Ubuntu 20.04 | GCC 10.3.0 |\n| Ubuntu 22.04 | GCC 11.2.0 |\n\nNote: GCC 8.5.0 has known regressions regarding fold expressions and overloaded operators. Using GCC 7.5.0 or (preferred) GCC >= 9 is recommended.\n\nNote: CUTLASS 3.x builds are known to be down on Windows platforms for all CUDA toolkits.\nCUTLASS team is working on a fix.\n\n## Hardware\n\nCUTLASS runs successfully on the following NVIDIA GPUs, and it is expected to be efficient on Volta, Turing, Ampere, Ada, and Hopper architecture based NVIDIA GPUs.\n\n|**GPU**|**CUDA Compute Capability**|**Minimum CUDA Toolkit Required by CUTLASS-3**|\n|---|---|---|\n|NVIDIA V100 Tensor Core GPU            |7.0|11.4|\n|NVIDIA TitanV                          |7.0|11.4|\n|NVIDIA GeForce RTX 20x0 series         |7.5|11.4|\n|NVIDIA T4                              |7.5|11.4|\n|NVIDIA A100 Tensor Core GPU            |8.0|11.4|\n|NVIDIA A10                             |8.6|11.4|\n|NVIDIA GeForce RTX 30x0 series         |8.6|11.4|\n|NVIDIA GeForce RTX 40x0 series         |8.9|11.8|\n|NVIDIA L40                             |8.9|11.8|\n|NVIDIA H100 Tensor Core GPU            |9.0|11.8|\n|NVIDIA H200 Tensor Core GPU            |9.0|11.8|\n|NVIDIA B200 Tensor Core GPU            |10.0|12.8|\n|NVIDIA B300 Tensor Core GPU            |10.3|13.0|\n|NVIDIA DRIVE Thor                      |11.0|13.0|\n|NVIDIA GeForce RTX 50x0 series         |12.0|12.8|\n|NVIDIA DGX Spark                       |12.1|13.0|\n\n## Target Architecture\n\nIn general, PTX code generated for one target architecture can be run on future architectures\n(i.e., it is forward compatible).\nHowever, CUDA 12.0 introduced the concept of \"architecture-accelerated features\" whose\nPTX does not have forward compatibility guarantees.\nSeveral Hopper and Blackwell PTX instructions fall under this category of\narchitecture-accelerated features, and thus require a `sm_90a` or `sm100a` target architecture\n(note the \"a\" appended). For more details on this and other architecture-accelerated instructions,\nplease refer to the [CUDA Documentation](https:\u002F\u002Fdocs.nvidia.com\u002Fcuda\u002Fcuda-c-programming-guide\u002Findex.html#feature-availability).\n\nThe target architecture information is passed on to CUTLASS via the cmake flag\n`CUTLASS_NVCC_ARCHS`. In order to maximize performance on Hopper GH100,\nusers are required to build CUTLASS with `90a` as the target architecture.\nIf a user accidentally builds a kernel which uses SM90a features\n(e.g. Hopper Tensor Core Instructions), using the SM90 target\n(note the lack of \"a\"), with either CUDA Toolkit 12 or 11.8,\nthe kernel is expected to fail with a runtime error.\n\n```\ncmake .. -DCUTLASS_NVCC_ARCHS=\"90a\"\n```\nOr\n\n```\ncmake .. -DCUTLASS_NVCC_ARCHS=\"100a\"\n```\n\nNote: The NVIDIA Blackwell SM100 architecture used in the datacenter\nproducts has a different compute capability than the one underpinning\nNVIDIA Blackwell GeForce RTX 50 series GPUs (SM120). As a result, kernels\ncompiled for Blackwell SM100 architecture with arch conditional features\n(using `sm100a`) are not compatible with RTX 50 series GPUs.\n\nPlease refer to the [functionality documentation](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Ffunctionality.html)\nfor details on which kernels require which target architectures.\n\n# Documentation\n\nCUTLASS is described in the following documents and the accompanying\n[Doxygen documentation](https:\u002F\u002Fnvidia.github.io\u002Fcutlass).\n\n- [Quick Start Guide](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fquickstart.html) - basics of building and running CUTLASS\n- [Functionality](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Ffunctionality.html) - summarizes functionality available in CUTLASS\n- [Efficient GEMM in CUDA](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fefficient_gemm.html) - describes how GEMM kernels may be implemented efficiently in CUDA\n- [CUTLASS 3.x Design](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fcutlass_3x_design.html) - describes the CUTLASS 3.x design, its benefits, and how CuTe enables us to write much more composable components\n- [GEMM API 3.x](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fgemm_api_3x.html) - describes the CUTLASS 3.x GEMM model and C++ template concepts\n- [GEMM API 2.x](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fgemm_api.html) - describes the CUTLASS 2.x GEMM model and C++ template concepts\n- [Implicit GEMM Convolution](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fimplicit_gemm_convolution.html) - describes 2-D and 3-D convolution in CUTLASS\n- [Code Organization](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fcode_organization.html) - describes the organization and contents of the CUTLASS project\n- [Terminology](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fterminology.html) - describes terms used in the code\n- [Programming Guidelines](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fprogramming_guidelines.html) - guidelines for writing efficient modern CUDA C++\n- [Fundamental types](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Ffundamental_types.html) - describes basic C++ classes used in CUTLASS to represent numeric quantities and arrays\n- [Layouts](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Flayout.html) - describes layouts of matrices and tensors in memory\n- [Tile Iterators](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Ftile_iterator_concept.html) - describes C++ concepts for iterating over tiles of matrices in memory\n- [CUTLASS Profiler](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fprofiler.html) - command-line driven profiling application\n- [CUTLASS Utilities](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Futilities.html) - additional templates used to facilitate rapid development\n- [Dependent kernel launch](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fdependent_kernel_launch.html) - describes a new feature in Hopper which allows overlapping dependent\nkernels in the same stream, and how it is used in CUTLASS.\n\n# Resources\nWe have also described the structure of an efficient GEMM in our talk at the\n[GPU Technology Conference 2018](http:\u002F\u002Fon-demand.gputechconf.com\u002Fgtc\u002F2018\u002Fpresentation\u002Fs8854-cutlass-software-primitives-for-dense-linear-algebra-at-all-levels-and-scales-within-cuda.pdf).\n\n- [CUTLASS: Software Primitives for Dense Linear Algebra at All Levels and Scales within CUDA](https:\u002F\u002Fwww.nvidia.com\u002Fen-us\u002Fon-demand\u002Fsession\u002Fgtcsiliconvalley2018-s8854\u002F)\n- [Developing CUDA Kernels to Push Tensor Cores to the Absolute Limit on NVIDIA A100](https:\u002F\u002Fwww.nvidia.com\u002Fen-us\u002Fon-demand\u002Fsession\u002Fgtcsj20-s21745\u002F)\n- [Accelerating Convolution with Tensor Cores in CUTLASS](https:\u002F\u002Fwww.nvidia.com\u002Fen-us\u002Fon-demand\u002Fsession\u002Fgtcspring21-s31883\u002F)\n- [Accelerating Backward Data Gradient by Increasing Tensor Core Utilization in CUTLASS](https:\u002F\u002Fwww.nvidia.com\u002Fen-us\u002Fon-demand\u002Fsession\u002Fgtcspring22-s41996\u002F)\n- [CUTLASS: Python API, Enhancements, and NVIDIA Hopper](https:\u002F\u002Fwww.nvidia.com\u002Fen-us\u002Fon-demand\u002Fsession\u002Fgtcfall22-a41131\u002F)\n\n# Building CUTLASS\n\nCUTLASS is a header-only template library and does not need to be built to be used by other\nprojects. Client applications should target CUTLASS's `include\u002F` directory in their include\npaths.\n\nCUTLASS unit tests, examples, and utilities can be build with CMake.\nThe minimum version of CMake is given in the [Quickstart guide](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fquickstart.html).\nMake sure the `CUDACXX` environment  variable points to NVCC in the CUDA Toolkit installed\non your system.\n\n```bash\n$ export CUDACXX=${CUDA_INSTALL_PATH}\u002Fbin\u002Fnvcc\n```\n\nCreate a build directory within the CUTLASS project, then run CMake. By default CUTLASS will build kernels\nfor CUDA architecture versions 5.0, 6.0, 6.1, 7.0, 7.5, 8.0, 8.6, 8.9, and 9.0.\nTo reduce compile time you can specify\nthe architectures to build CUTLASS for by changing the CMake configuration setting\n`CUTLASS_NVCC_ARCHS`.\n\n```bash\n$ mkdir build && cd build\n\n$ cmake .. -DCUTLASS_NVCC_ARCHS=80               # compiles for NVIDIA's Ampere Architecture\n```\n\nFrom the `build\u002F` directory, compile and run the CUTLASS unit tests by building the target `test_unit` with make.\n\nThe unit tests are organized as several binaries mirroring the top-level namespaces of CUTLASS,\nand they may be executed in parallel via make's `-j` command line argument.\n\n```bash\n$ make test_unit -j\n...\n...\n...\n[----------] Global test environment tear-down\n[==========] 946 tests from 57 test cases ran. (10812 ms total)\n[  PASSED  ] 946 tests.\n```\n\nAll tests should pass on supported platforms, though the exact number of tests may vary over time.\n\n\n# Project Structure\n\nCUTLASS is arranged as a header-only library along with Utilities, Tools, Examples, and unit tests.\n[Doxygen documentation](https:\u002F\u002Fnvidia.github.io\u002Fcutlass) provides a complete list of files, classes,\nand template concepts defined in the CUTLASS project.\n\nA detailed explanation of the source code organization may be found in the\n[CUTLASS documentation](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fcode_organization.html), but several main components are summarized below.\n\n## CUTLASS Template Library\n\n```\ninclude\u002F                     # client applications should target this directory in their build's include paths\n\n  cutlass\u002F                   # CUDA Templates for Linear Algebra Subroutines and Solvers - headers only\n\n    arch\u002F                    # direct exposure of architecture features (including instruction-level GEMMs)\n\n    conv\u002F                    # code specialized for convolution\n\n    epilogue\u002F                # code specialized for the epilogue of gemm\u002Fconvolution\n\n    gemm\u002F                    # code specialized for general matrix product computations\n\n    layout\u002F                  # layout definitions for matrices, tensors, and other mathematical objects in memory\n\n    platform\u002F                # CUDA-capable Standard Library components\n\n    reduction\u002F               # bandwidth-limited reduction kernels that do not fit the \"gemm\" model\n\n    thread\u002F                  # simt code that can be performed within a CUDA thread\n\n    transform\u002F               # code specialized for layout, type, and domain transformations\n\n    *                        # core vocabulary types, containers, and basic numeric operations\n\n  cute\u002F                      # CuTe Layout, layout algebra, MMA\u002FCopy atoms, tiled MMA\u002FCopy\n\n    algorithm\u002F               # Definitions of core operations such as copy, gemm, and operations on cute::tuples\n\n    arch\u002F                    # Bare bones PTX wrapper structs for copy and math instructions\n\n    atom\u002F                    # Meta-information either link to or built from arch\u002F operators\n\n      mma_atom.hpp           # cute::Mma_Atom and cute::TiledMma\n\n      copy_atom.hpp          # cute::Copy_Atom and cute::TiledCopy\n\n      *sm*.hpp               # Arch specific meta-information for copy and math operations\n\n    *                        # Core library types such as Shape, Stride, Layout, Tensor, and associated operations\n\n```\n\n### CUTLASS SDK Examples\n\n[CUTLASS SDK examples](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002Fcutlass\u002Ftree\u002Fmain\u002Fexamples) apply CUTLASS templates to implement basic computations.\n\n### Tools\n\n```\ntools\u002F\n  library\u002F                   # CUTLASS Instance Library - contains instantiations of all supported CUTLASS templates\n    include\u002F\n      cutlass\u002F\n        library\u002F\n\n  profiler\u002F                  # CUTLASS Profiler         - command-line utility for executing operations in the\n                             #                            CUTLASS Library\n\n  util\u002F                      # CUTLASS Utilities        - contains numerous helper classes for\n    include\u002F                 #                            managing tensors in device memory, reference\n      cutlass\u002F               #                            implementations for GEMM, random initialization\n        util\u002F                #                            of tensors, and I\u002FO.\n```\n\n### Test\n\nThe `test\u002Funit\u002F` directory consist of unit tests implemented with Google Test that demonstrate\nbasic usage of Core API components and complete tests of the CUTLASS GEMM computations.\n\nInstructions for building and running the Unit tests are described in the [Quickstart guide](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fquickstart.html).\n\n# Performance Profiling\n\nThe `tools\u002Fprofiler\u002F` directory contains a command-line utility for launching each of the GEMM kernels.\nIt can be built as follows:\n\n```bash\n$ make cutlass_profiler -j16\n```\n## Building all GEMM and Convolution kernels (_long_ build times)\n\nBy default, only one tile size is instantiated for each data type, math instruction, and layout.\nTo instantiate all, set the following environment variable when running CMake from an empty `build\u002F` directory.\nBeware, this results in *tens of thousands* of kernels and long build times.\nThis would also result in a large binary size and on some platforms linker to fail on building the library.\nTherefore, it's highly recommended to generate only a subset of kernels as demonstrated in the sub-section below.\n```bash\n$ cmake .. -DCUTLASS_NVCC_ARCHS=90a -DCUTLASS_LIBRARY_KERNELS=all\n...\n$ make cutlass_profiler -j16\n```\n\n## Building a subset of GEMM and Convolution kernels (_reduced_ build times)\n\nTo compile strictly one kernel or a small set of kernels, a comma-delimited list of kernel names with\nwildcard characters may be used to reduce the set of kernels. The following examples show building exactly one\nor a subset of kernels for NVIDIA Ampere and Turing architecture:\n\n### Building a subset Tensor Core GEMM kernels\n\nTo compile a subset of Tensor Core GEMM kernels with FP32 accumulation and FP16 input targeting NVIDIA Ampere and Turing architecture,\nuse the below cmake command line:\n```bash\n$ cmake .. -DCUTLASS_NVCC_ARCHS='75;80' -DCUTLASS_LIBRARY_KERNELS=cutlass_tensorop_s*gemm_f16_*_nt_align8\n...\n$ make cutlass_profiler -j16\n```\n\nExample command line for profiling a subset of Tensor Core GEMM kernels is as follows:\n```bash\n.\u002Ftools\u002Fprofiler\u002Fcutlass_profiler --kernels=cutlass_tensorop_s*gemm_f16_*_nt_align8 --m=3456 --n=4096 --k=4096\n\n...\n=============================\n  Problem ID: 1\n\n        Provider: CUTLASS\n   OperationKind: gemm\n       Operation: cutlass_tensorop_s1688gemm_f16_256x128_32x2_nt_align8\n\n          Status: Success\n    Verification: ON\n     Disposition: Passed\n\nreference_device: Passed\n          cuBLAS: Passed\n\n       Arguments: --gemm_kind=universal --m=3456 --n=4096 --k=4096 --A=f16:column --B=f16:row --C=f32:column --alpha=1  \\\n                  --beta=0 --split_k_slices=1 --batch_count=1 --op_class=tensorop --accum=f32 --cta_m=256 --cta_n=128  \\\n                  --cta_k=32 --stages=2 --warps_m=4 --warps_n=2 --warps_k=1 --inst_m=16 --inst_n=8 --inst_k=8 --min_cc=75  \\\n                  --max_cc=1024\n\n           Bytes: 118489088  bytes\n           FLOPs: 115992428544  flops\n\n         Runtime: 1.55948  ms\n          Memory: 70.7616 GiB\u002Fs\n\n            Math: 74378.8 GFLOP\u002Fs\n\n\n\n=============================\n...\n```\n\n### Building one CUDA Core GEMM kernel\n\nTo compile one SGEMM kernel targeting NVIDIA Ampere and Turing architecture, use the below cmake command line:\n```bash\n$ cmake .. -DCUTLASS_NVCC_ARCHS='75;80' -DCUTLASS_LIBRARY_KERNELS=cutlass_simt_sgemm_128x128_8x2_nn_align1\n...\n$ make cutlass_profiler -j16\n```\n\nExample command line for profiling single SGEMM CUDA kernel is as follows:\n```bash\n$ .\u002Ftools\u002Fprofiler\u002Fcutlass_profiler --kernels=sgemm --m=3456 --n=4096 --k=4096\n\n=============================\n  Problem ID: 1\n\n        Provider: CUTLASS\n   OperationKind: gemm\n       Operation: cutlass_simt_sgemm_128x128_8x2_nn_align1\n\n          Status: Success\n    Verification: ON\n     Disposition: Passed\n\n          cuBLAS: Passed\n\n       Arguments: --m=3456 --n=4096 --k=4096 --A=f32:column --B=f32:column --C=f32:column --alpha=1 --beta=0 --split_k_slices=1  \\\n                  --batch_count=1 --op_class=simt --accum=f32 --cta_m=128 --cta_n=128 --cta_k=8 --stages=2 --warps_m=4  \\\n                  --warps_n=2 --warps_k=1 --inst_m=1 --inst_n=1 --inst_k=1 --min_cc=50 --max_cc=1024\n\n           Bytes: 180355072  bytes\n           FLOPs: 115992428544  flops\n\n         Runtime: 6.73655  ms\n          Memory: 24.934 GiB\u002Fs\n\n            Math: 17218.4 GFLOP\u002Fs\n\n=============================\n```\n\n### Building a subset of Tensor Core Convolution kernels\n\nTo compile a subset of Tensor core convolution kernels implementing forward propagation (fprop) with FP32 accumulation\nand FP16 input targeting NVIDIA Ampere and Turing architecture, use the below cmake command line:\n```bash\n$ cmake .. -DCUTLASS_NVCC_ARCHS='75;80' -DCUTLASS_LIBRARY_KERNELS=cutlass_tensorop_s*fprop_optimized_f16\n...\n$ make cutlass_profiler -j16\n```\n\nExample command line for profiling a subset of Tensor Core convolution kernels is as follows:\n\n```bash\n$ .\u002Ftools\u002Fprofiler\u002Fcutlass_profiler --kernels=cutlass_tensorop_s*fprop_optimized_f16 --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3\n\n...\n=============================\n  Problem ID: 1\n\n        Provider: CUTLASS\n   OperationKind: conv2d\n       Operation: cutlass_tensorop_s16816fprop_optimized_f16_128x128_32x5_nhwc\n\n          Status: Success\n    Verification: ON\n     Disposition: Passed\n\nreference_device: Passed\n\n       Arguments: --conv_kind=fprop --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3 --p=224 --q=224 --pad_h=1 --pad_w=1  \\\n                  --stride_h=1 --stride_w=1 --dilation_h=1 --dilation_w=1 --Activation=f16:nhwc --Filter=f16:nhwc --Output=f32:nhwc  \\\n                  --conv_mode=cross --iterator_algorithm=optimized --alpha=1 --beta=0 --split_k_mode=serial --split_k_slices=1  \\\n                  --eq_gemm_provider=none --op_class=tensorop --accum=f32 --cta_m=128 --cta_n=128 --cta_k=32 --stages=5  \\\n                  --warps_m=2 --warps_n=2 --warps_k=1 --inst_m=16 --inst_n=8 --inst_k=16 --min_cc=80 --max_cc=1024\n\n           Bytes: 1130659840  bytes\n           FLOPs: 118482796544  flops\n\n         Runtime: 0.711496  ms\n          Memory: 1479.99 GiB\u002Fs\n\n            Math: 166526 GFLOP\u002Fs\n\n=============================\n...\n```\n\n\n### Building one Convolution CUDA kernel\n\nTo compile and run one CUDA Core convolution kernel implementing forward propagation (fprop) with F32 accumulation\nand FP32 input targeting NVIDIA Ampere and Turing architecture, use the below cmake command line:\n```bash\n$ cmake .. -DCUTLASS_NVCC_ARCHS='75;80' -DCUTLASS_LIBRARY_KERNELS=cutlass_simt_sfprop_optimized_128x128_8x2_nhwc\n...\n$ make cutlass_profiler -j16\n```\n\nExample command line for profiling one CUDA Core convolution kernel:\n\n```bash\n$ .\u002Ftools\u002Fprofiler\u002Fcutlass_profiler --kernels=cutlass_simt_sfprop_optimized_128x128_8x2_nhwc --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3\n\n\n=============================\n  Problem ID: 1\n\n        Provider: CUTLASS\n   OperationKind: conv2d\n       Operation: cutlass_simt_sfprop_optimized_128x128_8x2_nhwc\n\n          Status: Success\n    Verification: ON\n     Disposition: Passed\n\nreference_device: Passed\n\n       Arguments: --conv_kind=fprop --n=8 --h=224 --w=224 --c=128 --k=128 --r=3 --s=3 --p=224 --q=224 --pad_h=1 --pad_w=1  \\\n                  --stride_h=1 --stride_w=1 --dilation_h=1 --dilation_w=1 --Activation=f32:nhwc --Filter=f32:nhwc --Output=f32:nhwc  \\\n                  --conv_mode=cross --iterator_algorithm=optimized --alpha=1 --beta=0 --split_k_mode=serial --split_k_slices=1  \\\n                  --eq_gemm_provider=none --op_class=simt --accum=f32 --cta_m=128 --cta_n=128 --cta_k=8 --stages=2 --warps_m=4  \\\n                  --warps_n=2 --warps_k=1 --inst_m=1 --inst_n=1 --inst_k=1 --min_cc=50 --max_cc=1024\n\n           Bytes: 2055798784  bytes\n           FLOPs: 118482796544  flops\n\n         Runtime: 7.34266  ms\n          Memory: 260.752 GiB\u002Fs\n\n            Math: 16136.2 GFLOP\u002Fs\n\n\n=============================\n\n```\n\n## More Details on Compiling CUTLASS Kernels and CUTLASS Profiler\n- Please follow the links for more CMake examples on selectively compiling CUTLASS kernels:\n  - [GEMM CMake Examples](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fquickstart.html#gemm-cmake-examples)\n  - [Implicit GEMM convolution CMake Examples](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fquickstart.html#convolution-cmake-examples)\n- [Further details about the CUTLASS Profiler are described here.](https:\u002F\u002Fdocs.nvidia.com\u002Fcutlass\u002Flatest\u002Fmedia\u002Fdocs\u002Fcpp\u002Fprofiler.html)\n\n\n# About\n\nCUTLASS is released by NVIDIA Corporation as Open Source software under the\n[3-clause \"New\" BSD license](LICENSE.txt).\n\n# Contributors\n\nThe official list of CUTLASS developers and contributors is available here: [CONTRIBUTORS](CONTRIBUTORS.md).\n\n# Copyright\n\nCopyright (c) 2017 - 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\nSPDX-License-Identifier: BSD-3-Clause\n\n```\n  Redistribution and use in source and binary forms, with or without\n  modification, are permitted provided that the following conditions are met:\n\n  1. Redistributions of source code must retain the above copyright notice, this\n  list of conditions and the following disclaimer.\n\n  2. Redistributions in binary form must reproduce the above copyright notice,\n  this list of conditions and the following disclaimer in the documentation\n  and\u002For other materials provided with the distribution.\n\n  3. Neither the name of the copyright holder nor the names of its\n  contributors may be used to endorse or promote products derived from\n  this software without specific prior written permission.\n\n  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\"\n  AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE\n  IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE\n  DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE\n  FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL\n  DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR\n  SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER\n  CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,\n  OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE\n  OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n```\n","CUTLASS 是一个用于实现高性能矩阵乘法（GEMM）及相关计算的 CUDA 模板库。它通过层次分解和数据移动策略，将复杂的计算任务拆解为可重用的模块化组件，支持多种数据类型及混合精度计算，适用于 NVIDIA 的多个 GPU 架构。此外，CUTLASS 4.0 引入了 Python 原生接口（DSLs），使得用户无需深入掌握 C++ 即可编写高效 CUDA 内核代码，大幅降低了学习曲线，并提供了与深度学习框架无缝集成的能力。该项目特别适合需要在GPU上进行大规模线性代数运算的应用场景，如深度学习、科学计算等。",2,"2026-06-11 03:24:24","top_topic"]