PantheonSim / Run CUDA without a GPU

How to run CUDA programs without a GPU

PantheonSim, the GPU simulator for CUDA and ROCm, lets you compile CUDA code with the real nvcc and run the unmodified binary on a computer that has no NVIDIA GPU. It answers every call the program makes and runs its kernels on the CPU. This page shows the shortest ways to try it, the commands to run your own program, and what to expect.

What this is, and what it is not

PantheonSim is an open source simulator (Apache-2.0) that pretends to be an NVIDIA GPU, such as a T4, an H100 or a B200. It implements the CUDA driver API, the runtime API and the common CUDA libraries, so a program built with NVIDIA's own compiler loads them the way it loads the real ones. Its kernels are interpreted on the CPU, from the machine code (SASS) in the binary when there is some and from PTX otherwise.

It is a functional simulator. It tells you what your code does: whether it compiles, whether the kernels give the right answer, whether it reads past the end of a buffer, whether two warps race on shared memory. It does not tell you how fast the code would run, because it has no timing model. A run on a physical GPU before a release still counts. If you need cycle-level performance numbers, a research simulator is the better tool; the comparison page lists them.

Fastest way: the browser playground

The playground gives you a Linux machine in the browser with simulated GPUs, an editor, a terminal and nvcc for CUDA 12.6 or 13.0. There is nothing to install and no sign-up. Pick a card, paste a kernel, compile it and run it. A session lasts up to an hour, and ten idle minutes end it.

Install it on your own Linux machine

You need Linux, CMake 3.20 or newer and a C++20 compiler (g++ 13 or newer, or clang++ 17). No GPU and no NVIDIA driver are needed. To build the CUDA libraries the simulator also needs a CUDA toolkit on the machine, because they are built against it and carry its version. make doctor checks all of this, installs nothing, and prints the command that would fix what is missing.

git clone https://github.com/pantheongpu/pantheonsim
cd pantheonsim
make doctor
make build
make demo

The demo runs a built-in vector-add kernel on a simulated H100 and ends with a line that starts with PASS. That is your first simulation. If you would rather not build anything, a container image runs the simulator on any Linux host with Docker, with no GPU and no --gpus flag:

docker run --rm ghcr.io/pantheongpu/pantheonsim:latest vgpu list-gpus

That image has no compiler (neither nvcc nor hipcc), so a program has to arrive already built, or you copy the simulator into an image that has your compiler. The container notes show how.

Build and run your own program

Here is a complete program that adds two vectors on the GPU and checks the answer. Save it as vectoradd.cu:

#include <cstdio>

__global__ void add(const float* a, const float* b, float* c, int n) {
  int i = blockIdx.x * blockDim.x + threadIdx.x;
  if (i < n) c[i] = a[i] + b[i];
}

int main() {
  const int n = 1024;
  float ha[n], hb[n], hc[n];
  for (int i = 0; i < n; i++) { ha[i] = (float)i; hb[i] = 2.0f * i; }

  float *a, *b, *c;
  cudaMalloc(&a, sizeof ha); cudaMalloc(&b, sizeof hb); cudaMalloc(&c, sizeof hc);
  cudaMemcpy(a, ha, sizeof ha, cudaMemcpyHostToDevice);
  cudaMemcpy(b, hb, sizeof hb, cudaMemcpyHostToDevice);

  add<<<(n + 255) / 256, 256>>>(a, b, c, n);
  cudaError_t err = cudaGetLastError();
  if (err != cudaSuccess) { printf("launch failed: %s\n", cudaGetErrorString(err)); return 1; }

  cudaMemcpy(hc, c, sizeof hc, cudaMemcpyDeviceToHost);
  for (int i = 0; i < n; i++)
    if (hc[i] != 3.0f * i) { printf("FAIL at %d\n", i); return 1; }
  printf("PASS\n");
  return 0;
}

Build it with the real compiler, linking the CUDA runtime shared. That one flag lets the simulator stand in for the runtime. Then run it under vgpu, choosing the card:

nvcc -cudart shared -arch=compute_75 vectoradd.cu -o vectoradd
build/vgpu run --gpu nvidia/h100 ./vectoradd

The program prints PASS, and its exit code is its own. vgpu run puts the simulator's CUDA libraries in front of the real ones and runs the program in place, so scripts, ctest and shell pipelines work unchanged. Building for compute_75 runs on a T4 and every newer card; a kernel built for compute_90 needs a Hopper profile.

A few more options you will use quickly:

build/vgpu run --gpu nvidia/t4 ./vectoradd                # a T4
build/vgpu run --gpu nvidia/h100 --count 8 ./vectoradd    # eight H100s
build/vgpu run --race --strict ./vectoradd                # look for races and undefined behaviour
build/vgpu list-gpus                                      # every card you can pick

If you would rather write the machine down once, make init writes a commented pantheonsim.yaml (gpu: nvidia/h100, count: 8 and so on), and make run APP=./vectoradd uses it. The docs cover every option, and the GPU index lists the cards.

What works

• CUDA 12 and 13 programs. Binaries built by either toolkit run, as long as the simulator was built against the same CUDA major version. vgpu run says so plainly when it was not.
• The libraries programs link. cuBLAS, cuBLASLt, cuDNN, cuFFT, cuRAND, cuSPARSE, cuSOLVER, NCCL, NVRTC, NPP and nvJPEG under their real names, each tested against NVIDIA's own library on a physical GPU.
• Python. Numba runs unmodified, and so do Triton under vgpu run and PyTorch's CUDA 13 build. See PyTorch on a simulated GPU.
• Several GPUs. Up to sixteen simulated devices, with peer copies and per-device isolation as on a real multi-GPU machine.
• Debugging a real GPU will not do for you. A simulator sees every lane of every warp, so it can name the kernel, the PTX line and the instruction of a shared-memory race, and it counts instructions and memory sectors exactly instead of sampling them.

Limits, stated plainly

• No timing. Timings from a simulated run mean nothing. There are no cycles, no stall reasons and no cache hit rates. Counters of instructions and memory traffic are exact, but they are not performance.
• Shared runtime only. A program that links the CUDA runtime statically cannot be simulated yet, which is why CuPy does not work. Build with -cudart shared, and in CMake set CMAKE_CUDA_RUNTIME_LIBRARY to Shared.
• Gaps are loud, not fatal. An unimplemented instruction or call returns the documented CUDA error and prints the instruction, the PTX line, the kernel and the GPU profile. The program carries on, so check return codes: a test that ignores them can still exit 0.
• NVIDIA's profilers are not included. Nsight Compute and Nsight Systems as NVIDIA ships them are not supported. vgpu ncu is the project's own command line, answered from the simulator's exact counters.
• Not everything is CUDA. OptiX ray tracing is a separate NVIDIA subsystem and is out of scope.

Questions

Next steps

Try a kernel in the playground, read the docs, wire it into continuous integration, or see how the simulated nvidia-smi behaves. If your program says there is no CUDA device on a real machine, start with no CUDA-capable device is detected. For AMD, the same idea is described in running ROCm and HIP without an AMD GPU.