CUTLASS 2.0
CUTLASS 2.0 - November 2019
CUTLASS is a collection of CUDA C++ template abstractions for implementing high-performance matrix-multiplication (GEMM) at all levels and scales within CUDA. It incorporates strategies for hierarchical decomposition and data movement similar to those used to implement cuBLAS. CUTLASS decomposes these "moving parts" into reusable, modular software components abstracted by C++ template classes. These thread-wide, warp-wide, block-wide, and device-wide primitives can be specialized and tuned via custom tiling sizes, data types, and other algorithmic policy. The resulting flexibility simplifies their use as building blocks within custom kernels and applications.
To support a wide variety of applications, CUTLASS provides extensive support for mixed-precision computations, providing specialized data-movement and multiply-accumulate abstractions for 8-bit integer, half-precision floating point (FP16), single-precision floating point (FP32), and double-precision floating point (FP64) types. Furthermore, CUTLASS demonstrates warp-synchronous matrix multiply operations for targeting the programmable, high-throughput Tensor Cores implemented by NVIDIA's Volta and Turing architectures.
What's New in CUTLASS 2.0
CUTLASS 2.0 is a substantial refactoring from the previous version, intended to offer:
- Better performance over 1.x, particularly for kernels targeting Turing Tensor Cores
- Robust and durable templates that reliably span the design space
- Encapsulated functionality that may be reusable in other contexts
Example CUTLASS GEMM
The following illustrates an example function that defines a CUTLASS GEMM kernel with single-precision inputs and outputs. This is an exercpt from the CUTLASS SDK basic_gemm example.
cudaError_t cutlass_sgemm_nn(
int M,
int N,
int K,
float alpha,
float const *A,
int lda,
float const *B,
int ldb,
float beta,
float *C,
int ldc) {
ColumnMajor,
float,
ColumnMajor,
float,
ColumnMajor>;
CutlassGemm gemm_operator;
CutlassGemm::Arguments args({M , N, K},
{A, lda},
{B, ldb},
{C, ldc},
{C, ldc},
{alpha, beta});
return cudaErrorUnknown;
}
return cudaSuccess;
}
Copyright
Copyright (c) 2017-2019, NVIDIA CORPORATION. All rights reserved.
1 Redistribution and use in source and binary forms, with or without modification, are permitted
2 provided that the following conditions are met:
3 * Redistributions of source code must retain the above copyright notice, this list of
4 conditions and the following disclaimer.
5 * Redistributions in binary form must reproduce the above copyright notice, this list of
6 conditions and the following disclaimer in the documentation and/or other materials
7 provided with the distribution.
8 * Neither the name of the NVIDIA CORPORATION nor the names of its contributors may be used
9 to endorse or promote products derived from this software without specific prior written
12 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR
13 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
14 FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL NVIDIA CORPORATION BE LIABLE
15 FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
16 BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
17 OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
18 STRICT LIABILITY, OR TOR (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
19 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.