PantheonSim / PyTorch on a simulated GPU

Run PyTorch on a simulated GPU, with no GPU

PantheonSim, the GPU simulator for CUDA and ROCm, runs PyTorch's official CUDA 13 build unmodified on a simulated NVIDIA GPU, so torch.cuda.is_available() is True on a machine that has no GPU. This guide shows the two commands that matter, what has been checked, what is not supported, and why you should treat it as a way to test code, not to train anything large.

Run it

You need PantheonSim built from its source (see how to run CUDA without a GPU), in a build that carries CUDA 13 libraries, and PyTorch's CUDA 13 wheel. Then:

pip install torch --index-url https://download.pytorch.org/whl/cu130
build/vgpu run --gpu nvidia/h100 --preload python train.py

The --gpu value picks the card, and --count gives you several. Here is a small script to try first. It runs the same computation on the simulated GPU and on the CPU and compares them:

import torch

print(torch.cuda.is_available(), torch.cuda.get_device_name(0))

torch.manual_seed(0)
x = torch.randn(64, 128)
w = torch.randn(128, 10)

def step(device):
    xd, wd = x.to(device), w.to(device).requires_grad_()
    loss = (xd @ wd).relu().sum()
    loss.backward()
    return loss.item(), wd.grad.cpu()

gpu_loss, gpu_grad = step("cuda")
cpu_loss, cpu_grad = step("cpu")
print("match" if torch.allclose(gpu_grad, cpu_grad, atol=1e-4) else "MISMATCH", gpu_loss, cpu_loss)

Why --preload

PyTorch finds NVIDIA's libraries through an RPATH into its own nvidia-* packages, and the python program itself names no CUDA library, so the simulator's usual way of getting in front of the real libraries does not apply. --preload loads every CUDA library the simulator carries first, and a library already loaded under a name is the one every later request for that name gets. Without it, NVIDIA's real runtime loads, finds no NVIDIA driver, and PyTorch reports that its integrity checks failed (error 103).

What has been checked

The repository's PyTorch test file, models.py, holds twelve whole-model checks, each compared against the CPU. They all pass on the simulated T4, A100, L4, RTX 3060, H100, B200 and RTX 5090. They cover:

• a GPT trained with a one-cycle schedule, and generating with a KV cache;
• a ResNet, an LSTM, a U-Net denoiser and a vision transformer;
• a mixture of experts and a DLRM-style recommender;
• half and bfloat16 autocast, activation checkpointing, and a checkpoint read back on the host.

Underneath, PyTorch's own kernels run as the machine code (SASS) its wheel carries for the simulated card's architecture, as on a real driver. Convolutions, batch norm, RNNs and scaled-dot-product attention go through the simulator's cuDNN, and linear layers through its cuBLAS and cuBLASLt. CUDA graphs capture and replay, and torch.compile works because Triton does: its generated kernels pass through the simulator's stand-in for ptxas. torch.distributed runs across processes, each with its own simulated GPU, over the simulator's NCCL, and a small network trained with DistributedDataParallel, tensor parallelism, pipeline parallelism and FullyShardedDataParallel ends where one process on the whole batch ends.

PyTorch built for ROCm also runs on the simulated AMD GPUs. The ROCm page has the AMD side, including the gfx target rule.

What this is good for

• Testing code paths. Device placement, mixed precision, checkpointing, graph capture and multi-process setup can all be exercised on a machine with no GPU, such as a CI runner.
• Catching bugs that depend on the card. Run the same script with --gpu nvidia/t4, nvidia/a100 and nvidia/h100 and compare.
• Checking that a simulated machine looks like yours. Pick a card, then read what torch.cuda.get_device_properties(0) reports.

Limits

• It is slow, and timings mean nothing. The kernels run on the CPU, and the simulator has no timing model. Use small batches and few steps. It checks behaviour, not speed.
• Some cuDNN graph operations are refused. Operations beyond what the simulator's cuDNN graph list computes are refused when the graph is finalized, naming the operation. Others, such as nearest and bilinear resampling, fail at plan creation with cudnn-frontend's "No valid engine configs", as they do on the RTX 3060 they were measured on, so PyTorch falls back or reports it. PyTorch's own convolutions, matmul-based linear layers and scaled-dot-product attention all run.
• Unimplemented instructions fail loudly. A kernel that uses something the simulator does not implement fails the launch and names the instruction. Do not ignore it because a check printed ok.
• CuPy does not work. It links the CUDA runtime statically, which the simulator cannot host yet.
• The playground still says PyTorch is unsupported. It prints a notice to that effect when you import it, so use a source build to run PyTorch.

Questions

Next steps

See why torch.cuda.is_available() is False on a real machine, run these tests in CI, browse the GPU index, or read the PyTorch notes in the repository, which are the source for this page.