GPUs for fine-tuning
As of — QLoRA on an 8B model: from $0.15 per GPU-hour on the NVIDIA RTX A4000 (Hyperstack), 50 cards qualify; LoRA on an 8B model, 16-bit base: from $0.16 per GPU-hour on the NVIDIA V100 32GB (Geodd), 40 cards qualify; Full fine-tuning of an 8B model on one GPU: from $1.71 per GPU-hour on the AMD Instinct MI300X (TensorWave), 10 cards qualify. Rates are each card’s cheapest live on-demand rate card across the providers we track, read hourly, every one linked to the page it was read from.
Full fine-tuning with the Adam optimizer in mixed precision keeps 16 bytes per parameter: 16-bit weights and gradients (2 + 2 bytes) and 32-bit master weights, momentum and variance (4 + 4 + 4) - the accounting of the ZeRO paper. LoRA freezes the base weights and trains small adapters, so its memory is close to the weights alone; QLoRA stores those frozen weights in 4-bit, half a byte per parameter. Activations come on top in every case.
QLoRA on an 8B model
8 billion parameters × 0.5 bytes = 4 GB of frozen weights, plus adapters and activations: a card with 16 GB or more.
| Card, memory | Cheapest on-demand | Provider, read | Median rate card | $ per GB-hour | Deploy |
|---|---|---|---|---|---|
| NVIDIA RTX A4000 · 16 GB | $0.15 |  Hyperstackcaptured | $0.25 11 providers | $0.009 | Deploy ↗ |
| NVIDIA V100 32GB · 32 GB | $0.16 |  Geoddcaptured | $1.15 11 providers | $0.005 | Deploy ↗ |
| NVIDIA GeForce RTX 3090 · 24 GB | $0.18 |  OpenRelaycaptured | $0.42 10 providers | $0.007 | Deploy ↗ |
| NVIDIA A2 · 16 GB | $0.20 | Infomaniak captured | $0.23 2 providers | $0.012 | Deploy ↗ |
| NVIDIA A30 · 24 GB | $0.22 |  CloudGPUcaptured | $0.41 9 providers | $0.009 | Deploy ↗ |
| NVIDIA T4 · 16 GB | $0.24 |  immers.cloudcaptured | $0.55 19 providers | $0.015 | Deploy ↗ |
| NVIDIA RTX 2000 Ada · 16 GB | $0.24 from |  GPU.aicaptured | $0.24 2 providers | $0.015 | Deploy ↗ |
| NVIDIA V100 · 16 GB | $0.24 |  Verdacaptured | $0.83 13 providers | $0.015 | Deploy ↗ |
| NVIDIA RTX A5000 · 24 GB | $0.27 from | GPU.ai captured | $0.58 10 providers | $0.011 | Deploy ↗ |
| NVIDIA L4 · 24 GB | $0.28 GPU only |  Hikubecaptured | $0.80 23 providers | $0.012 | Deploy ↗ |
40 more cards qualify, dearer: NVIDIA RTX 4000 Ada $0.28, NVIDIA GeForce RTX 4070 Ti SUPER $0.29, NVIDIA GeForce RTX 4080 $0.31, NVIDIA Quadro RTX 6000 $0.31, NVIDIA A10 $0.34, NVIDIA RTX 4090 $0.35, NVIDIA RTX A6000 $0.35, NVIDIA GeForce RTX 5080 $0.37, NVIDIA A40 $0.39, NVIDIA RTX 4000 SFF Ada $0.44, NVIDIA GeForce RTX 3090 Ti $0.46, NVIDIA RTX 5090 $0.50, NVIDIA L40S $0.63, NVIDIA RTX 6000 Ada $0.65, NVIDIA RTX PRO 6000 Blackwell $0.65, NVIDIA A100 40GB $0.69, NVIDIA RTX PRO 4500 Blackwell $0.72, NVIDIA L40 $0.78, NVIDIA GB10 (DGX Spark) $0.79, NVIDIA RTX PRO 5000 Blackwell $0.80, NVIDIA RTX 5000 Ada Generation $0.83, NVIDIA A100 80GB $0.92, NVIDIA A10G $1.01, Intel Gaudi 2 $1.10, AMD Instinct MI300X $1.71, NVIDIA H100 80GB (form factor not stated) $1.72, NVIDIA H100 SXM $1.79, NVIDIA RTX PRO 6000 Blackwell Max-Q $1.80, NVIDIA H100 PCIe $1.98, AMD Instinct MI325X $2.25, NVIDIA GH200 $2.29, NVIDIA H200 $2.29, NVIDIA H100 NVL $2.92, AMD Instinct MI355X $2.95, NVIDIA H200 NVL $3.29, AMD Instinct MI350X $4.00, NVIDIA B200 $4.50, NVIDIA B300 $5.89, NVIDIA GB200 NVL72 $6.94, NVIDIA GB300 NVL72 $13.30.
LoRA on an 8B model, 16-bit base
8 billion parameters × 2 bytes = 16 GB of frozen weights, plus adapters and activations: a card with 24 GB or more.
| Card, memory | Cheapest on-demand | Provider, read | Median rate card | $ per GB-hour | Deploy |
|---|---|---|---|---|---|
| NVIDIA V100 32GB · 32 GB | $0.16 | Geodd captured | $1.15 11 providers | $0.005 | Deploy ↗ |
| NVIDIA GeForce RTX 3090 · 24 GB | $0.18 | OpenRelay captured | $0.42 10 providers | $0.007 | Deploy ↗ |
| NVIDIA A30 · 24 GB | $0.22 | CloudGPU captured | $0.41 9 providers | $0.009 | Deploy ↗ |
| NVIDIA RTX A5000 · 24 GB | $0.27 from | GPU.ai captured | $0.58 10 providers | $0.011 | Deploy ↗ |
| NVIDIA L4 · 24 GB | $0.28 GPU only | Hikube captured | $0.80 23 providers | $0.012 | Deploy ↗ |
| NVIDIA Quadro RTX 6000 · 24 GB | $0.31 GPU only |  centroncaptured | $0.51 4 providers | $0.013 | Deploy ↗ |
| NVIDIA A10 · 24 GB | $0.34 GPU only |  Intelion Cloudcaptured | $1.22 9 providers | $0.014 | Deploy ↗ |
| NVIDIA RTX 4090 · 24 GB | $0.35 | OpenRelay captured | $0.48 17 providers | $0.015 | Deploy ↗ |
| NVIDIA RTX A6000 · 48 GB | $0.35 |  Thunder Computecaptured | $0.59 21 providers | $0.007 | Deploy ↗ |
| NVIDIA A40 · 48 GB | $0.39 | CloudGPU captured | $0.64 8 providers | $0.008 | Deploy ↗ |
30 more cards qualify, dearer: NVIDIA GeForce RTX 3090 Ti $0.46, NVIDIA RTX 5090 $0.50, NVIDIA L40S $0.63, NVIDIA RTX 6000 Ada $0.65, NVIDIA RTX PRO 6000 Blackwell $0.65, NVIDIA A100 40GB $0.69, NVIDIA RTX PRO 4500 Blackwell $0.72, NVIDIA L40 $0.78, NVIDIA GB10 (DGX Spark) $0.79, NVIDIA RTX PRO 5000 Blackwell $0.80, NVIDIA RTX 5000 Ada Generation $0.83, NVIDIA A100 80GB $0.92, NVIDIA A10G $1.01, Intel Gaudi 2 $1.10, AMD Instinct MI300X $1.71, NVIDIA H100 80GB (form factor not stated) $1.72, NVIDIA H100 SXM $1.79, NVIDIA RTX PRO 6000 Blackwell Max-Q $1.80, NVIDIA H100 PCIe $1.98, AMD Instinct MI325X $2.25, NVIDIA GH200 $2.29, NVIDIA H200 $2.29, NVIDIA H100 NVL $2.92, AMD Instinct MI355X $2.95, NVIDIA H200 NVL $3.29, AMD Instinct MI350X $4.00, NVIDIA B200 $4.50, NVIDIA B300 $5.89, NVIDIA GB200 NVL72 $6.94, NVIDIA GB300 NVL72 $13.30.
Full fine-tuning of an 8B model on one GPU
8 billion parameters × 16 bytes = 128 GB: a card with 141 GB or more, or the states split over several GPUs, which is what ZeRO does.
| Card, memory | Cheapest on-demand | Provider, read | Median rate card | $ per GB-hour | Deploy |
|---|---|---|---|---|---|
| AMD Instinct MI300X · 192 GB | $1.71 from |  TensorWavecaptured | $2.65 7 providers | $0.009 | Deploy ↗ |
| AMD Instinct MI325X · 256 GB | $2.25 |  Bentauscaptured | $2.25 3 providers | $0.009 | Deploy ↗ |
| NVIDIA H200 · 141 GB | $2.29 |  Dataoortscaptured | $4.56 56 providers | $0.016 | Deploy ↗ |
| AMD Instinct MI355X · 288 GB | $2.95 from | TensorWave captured | $6.14 4 providers | $0.010 | Deploy ↗ |
| NVIDIA H200 NVL · 141 GB | $3.29 from | GPU.ai captured | $3.62 7 providers | $0.023 | Deploy ↗ |
| AMD Instinct MI350X · 288 GB | $4.00 |  CloudRiftcaptured | $5.08 2 providers | $0.014 | Deploy ↗ |
| NVIDIA B200 · 192 GB | $4.50 from |  Hostingercaptured | $6.94 40 providers | $0.023 | Deploy ↗ |
| NVIDIA B300 · 288 GB | $5.89 |  Deep Infracaptured | $8.99 26 providers | $0.020 | Deploy ↗ |
| NVIDIA GB200 NVL72 · 186 GB | $6.94 from |  Bitdeer AIcaptured | $13.25 6 providers | $0.037 | Deploy ↗ |
| NVIDIA GB300 NVL72 · 288 GB | $13.30 | Verda captured | $18.00 3 providers | $0.046 | Deploy ↗ |
How these figures are read
A card qualifies by its memory alone; speed, interconnect and software support are not ranked here. The rate is the card’s cheapest live on-demand rate card - each provider once, at its cheapest on-demand rate, community capacity and marketplace floors out, read within 48 hours - and the median is taken over the same rate cards, as on every card page. Memory figures are floors from the arithmetic above, not measurements: context length, batch size and the serving stack change them. Methodology.
Sources
- Rajbhandari et al., ZeRO: Memory Optimizations Toward Training Trillion Parameter Models (2019)
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (2021)
- Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs (2023)
Other workloads: GPUs for LLM inference · GPUs for training · GPUs for image generation · Cheapest GPU memory per hour.
Data updated: