Harnessing Rust's std::thread for GPU Programming

At VectorWare, we're pioneering the use of Rust's std::thread on GPUs, marking a significant leap in GPU programming. Traditionally, CPUs and GPUs have distinct execution models; CPUs start with a single thread and spawn others as needed, while GPUs launch kernels with many parallel instances. This inherent concurrency in GPUs makes programming challenging, as developers must manually manage data indexing and avoid race conditions.

In Rust, GPU kernels are modeled as functions, similar to CPU functions, but they execute thousands of times in parallel. This parallel execution conflicts with Rust's ownership model, which is designed for single-threaded CPU execution. To bridge this gap, we use a CPU harness to simulate GPU execution, allowing tools like miri to check for undefined behavior.

Implementing std::thread on the GPU is complex due to the fundamental differences between CPU threads and GPU lanes. A GPU thread, or lane, is part of a warp and advances in lockstep with others, unlike independent CPU threads. Mapping std::thread to GPU lanes would violate Rust's semantics and slow down execution due to divergence.

Our solution maps each std::thread to a GPU warp. Initially, only Warp 0 is active, running the main function. Other warps are activated on demand, mimicking CPU thread behavior. This approach prevents divergence by ensuring each warp runs the same code, and it leverages Rust's borrow checker and lifetimes, making GPU programming more intuitive for Rust developers.

This innovation opens up a vast portion of the Rust ecosystem for GPU use, enabling libraries that rely on threads for parallelism or async for I/O to run on GPUs with minimal changes. While there are downsides, such as limited warp resources and expensive thread synchronization, the benefits of integrating Rust's concurrency models with GPU programming are substantial.

Looking ahead, we aim to develop GPU-native applications that fully exploit GPU hardware capabilities, moving beyond simply porting CPU software to GPUs. Although our focus is on Rust, we plan to support multiple languages, recognizing Rust's unique strengths in building high-performance, reliable GPU applications.