Guide

How to Run PyTorch on a GPU Without AWS

Five practical ways to get CUDA-accelerated PyTorch running — from your own graphics card to a peer’s idle GPU — without an AWS bill. Written for students and developers who need a GPU, not a data-centre invoice.

Updated 2026 · ~9 min read

AWS works, but for a student or an indie developer it’s often the wrong tool: you set up an account, wrestle with instance types and quotas, pay by the hour whether or not your code is running, and forget to shut it down once. If all you want is torch.cuda.is_available() to return True, there are simpler, cheaper paths. Here are five, with the honest trade-offs of each.

1. Use your own NVIDIA GPU (local CUDA)

If you already have an NVIDIA card — even a modest GTX/RTX — this is the cheapest option: it’s free and fully private. The work is getting the CUDA stack lined up with PyTorch.

Step 1 — install the NVIDIA driver

On Windows, install the latest GeForce/Studio driver. On Linux, install the proprietary driver for your card (e.g. sudo apt install nvidia-driver-550). Confirm it works:

nvidia-smi   # lists your GPU, driver + CUDA version

Step 2 — install PyTorch with the matching CUDA build

You do not install the full CUDA Toolkit separately for PyTorch — the pip wheels bundle the CUDA runtime. Pick the CUDA version from pytorch.org and install into a virtual environment:

# create an isolated env first (don't pollute system Python)
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate

# CUDA 12.4 build, for example
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124

Step 3 — verify the GPU is seen

import torch
print(torch.cuda.is_available())          # True
print(torch.cuda.get_device_name(0))      # e.g. 'NVIDIA GeForce RTX 4070'
x = torch.rand(5000, 5000, device="cuda")
print((x @ x).sum().item() > 0)           # True — matmul ran on the GPU

Trade-off: free and private, but two real costs. First, version hell — driver, CUDA build and PyTorch must line up, and a mismatch gives you CUDA error: no kernel image is available. Second, while a job runs your machine is pinned; you can’t comfortably keep working. And if you’re on a Mac or a laptop with no NVIDIA GPU, this option is simply off the table.

2. Free hosted notebooks: Google Colab & Kaggle

The fastest zero-setup path. Both give you a free GPU-backed Jupyter environment in the browser — PyTorch is pre-installed and CUDA just works.

• Google Colab — free tier gives a shared GPU (often a T4). Set Runtime → Change runtime type → GPU.
• Kaggle Notebooks — free GPU/TPU hours per week, a bit more generous for longer runs.

Trade-off: genuinely free and great for learning, but built to be interrupted. Sessions time out after a few hours, disconnect if idle, and reset your environment — you re-install and re-download every time. The free GPU is whatever’s spare, and heavy users get throttled or queued. Fine for a tutorial; frustrating for a real project.

3. Attach a peer’s GPU to your own notebook (Tangible)

This is the middle ground between “buy a GPU” and “rent a cloud box.” Every cloud — AWS, and the cheaper ones below — makes you move your work into their environment: upload your files, use their notebook, configure their image. Tangible flips that. Instead of moving to the GPU, the GPU comes to you.

You keep the Jupyter you already use. One click matches a peer’s idle NVIDIA GPU, and your notebook opens in your browser with its kernel running on that card. Nothing about how you work changes:

import torch
torch.cuda.is_available()     # True — on a peer's card, through your own notebook

%pip install transformers     # installs into your session, isolated
%tangible upload              # stream your dataset in; wiped when you're done

Because the notebook is yours and the compute is remote, there’s no migration and no re-upload of your whole project — you bring only the data the run needs. You’re billed by the minute, not by the reserved hour, so an idle notebook doesn’t quietly cost you. And each session runs in an isolated sandbox with a RAM-only workspace that’s destroyed when you disconnect, so your data isn’t left sitting on a stranger’s disk.

Trade-off: it’s peer-to-peer, so a GPU in the tier you want has to be online when you need it (for big, long, reserved runs a dedicated cloud box still makes sense). But for the common case — “I need a real CUDA GPU for the next hour, from the notebook I’m already in” — it removes the whole migrate-and-reconfigure tax that AWS and the others impose.

4. Cheaper GPU clouds (Vast.ai, RunPod, Salad)

If you need a dedicated GPU for many hours or days, specialist providers are far cheaper than AWS:

• Vast.ai — a marketplace where you rent spare GPUs by bidding; usually the lowest raw hourly price.
• RunPod — developer-friendly pods and serverless GPUs, spin up in about a minute.
• Salad — distributed consumer GPUs at very low prices for batch work.

Trade-off: real savings versus AWS, and the right tool for long, dedicated jobs. The catch is the same as AWS in kind if not in price — you still move into their box: provision an instance, pick an image, SSH or use their notebook, and mind the clock. Great when you want a machine for the weekend; heavier than you need for a quick interactive session.

5. University, lab, or Colab-style institutional access

If you’re a student or researcher, check what you already have. Many universities run a shared GPU cluster (often via SLURM), and some give students cloud credits. It’s free compute if you can get on it.

Trade-off: unbeatable price (free), but you’re sharing. Expect a job scheduler, queue times when the cluster is busy, and policies about what you can run. Perfect for coursework and research; less so for a quick experiment at 2am.

So which should you use?

• You own an NVIDIA GPU → use it locally (option 1). Free and private; just budget time for the CUDA setup.
• Just learning / tiny experiments → Colab or Kaggle (option 2). Zero setup, zero cost.
• You want a real GPU from your own notebook, right now, by the minute → a peer GPU via Tangible (option 3). No migration, wiped after.
• A long, dedicated job → a niche GPU cloud (option 4). Cheaper than AWS; you rent a box.
• You’re at a university → use the cluster (option 5) if you can.

The honest summary: AWS is rarely the cheapest or the simplest way to run PyTorch on a GPU. Your own card is cheapest if you have one; free notebooks are best for learning; a dedicated GPU cloud wins for long jobs; and for the everyday “I just need CUDA in the notebook I’m already using,” attaching a peer’s GPU removes the setup and the migration entirely.