NVIDIA has officially introduced the CUDA Rust toolkit, enabling developers to utilize the Rust programming language to create kernels that run on the GPU. This toolkit is designed to offer compile-time memory safety and prevent race conditions. Plans are in place to bring CUDA Rust to a level suitable for the development of functional projects by next year, aligning it with existing tools like CUDA C++ and CUDA Python.
The CUDA Rust toolkit supports two parallel executable kernel development models – SIMT (Single Instruction, Multiple Threads) and Tale. The SIMT model allows for defining single thread logic at a low level and managing numerous threads, while the Tale model provides a higher level of abstraction by defining algorithms for handling data blocks (tiles) instead of individual threads. The Tile IR compiler takes charge of tasks such as thread management, access synchronization, memory management, and data distribution among tensor cores.
For Rust development using the SIMT model, a compiler called cuda-oxide is being developed. This compiler allows for Rust code compilation directly into instructions for execution in a CUDA PTX (Parallel Thread Execution) virtual machine. CUDA kernels can be written in regular Rust, operating in a no_std environment and limited to functions from the libcore library and specialized Rust abstractions, excluding access to the standard Rust library (libstd).
CUDA kernels in Rust offer safety through the type system, support for unsafe blocks, and access to low-level hardware instructions. To ensure security, the DisjointSlice type guarantees that each thread exclusively accesses its own data. The cuda-oxide code is available under the Apache 2.0 license.
For those opting to use the Tale model, the library cutile-rs is available. This library enables the use of idiomatic Rust language to directly compile code into CUDA kernels. By enforcing Rust’s strict ownership and borrowing rules, cutile-rs allows developers to work with single-threaded programs that operate on data blocks, with the compiler handling thread breakdown and synchronization. Threads share access to immutable tensors and securely access mutable tensors by dividing them into disjoint blocks. The cutile-rs code is also distributed under the Apache 2.0 license.