Question 1

How much does a cloud GPU cost per hour?

Accepted Answer

From about $0.13/hour (L4 spot on marketplaces) to $16.11/hour (B200 on-demand at GCP). The workhorses: H100s at $1.49 to $6.98 depending on provider, A100s at $0.68 to $5.03, L4s at $0.44 to $0.80 for inference.

Question 2

Which cloud has the cheapest GPU pricing?

Accepted Answer

Marketplaces (Vast.ai) and community tiers (RunPod, Salad) list the lowest absolute rates, with H100s from $1.49 to $1.99/hour. Among hyperscalers, GCP spot at 60-91% off is cheapest. The lowest rate isn't always the lowest cost: egress fees, interruptions, and idle time routinely flip the ranking.

Question 3

Why is hyperscaler GPU pricing so much higher?

Accepted Answer

Median H100 rates run about $4.17/hour on dedicated GPU clouds versus $7.89 on hyperscalers per current market analysis. The premium buys enterprise integration: your existing security, IAM, compliance, and data locality. It doesn't buy faster silicon; the GPU is identical.

Question 4

What does GPU as a service mean?

Accepted Answer

Renting GPU compute by the hour instead of buying hardware, from hyperscalers, dedicated GPU clouds, or marketplaces. You pay for time on someone else's silicon, which fits AI's spiky workloads far better than ownership, as long as someone's watching the idle hours.

Question 5

How much is an A100 per hour in the cloud?

Accepted Answer

From $0.68/hour (spot) to $5.03 (GCP 80GB on-demand). CoreWeave anchors the dedicated tier at $2.21 to $2.70, with other dedicated clouds pricing below that anchor and hyperscalers above it. The 40GB variant runs meaningfully cheaper where offered.

Question 6

Are cloud GPU prices going up or down?

Accepted Answer

Flat, which is its own answer. GetDeploying's index puts on-demand GPU rental prices up 1.7% over the last 12 months and down 0.5% over the last four weeks, while AI API prices fell up to 80% in the same window. Blackwell demand is absorbing the supply that would otherwise have pushed rates down. GPU rental is a commodity market now; expect movement in both directions and date-check every rate table.

Question 7

What's the cheapest way to run inference in the cloud?

Accepted Answer

Often not a GPU at all: per-token APIs undercut self-hosted inference by up to 10x for many workloads. When self-hosting wins, L4s at $0.44 to $0.80/hour are the inference value pick for models up to ~13B parameters, at 3-5x less than A100 rates.

Question 8

Is it cheaper to rent a GPU or buy one?

Accepted Answer

Rent until utilization is proven. A rented H100 at $3.99/hour costs about $35,000 per GPU-year at full utilization, more than buying the card, but it converts a six-figure hardware bet into an hourly experiment you can end on Friday. A used 8x system pays back in roughly seven months at full utilization and about two years at 30%, which is the polite way of saying don't buy for a workload you haven't measured. The full break-even math is in our H100 price guide.

Question 9

How do committed use and reserved GPU discounts work?

Accepted Answer

Hyperscalers offer committed-use discounts (GCP) and capacity blocks (AWS) that cut list rates substantially; CoreWeave's reserved contracts run up to 60% off on-demand. The trade is flexibility: commitments turn volatile hourly rates into fixed costs, which is a bet on your own utilization forecast, so measure before you sign.

The AI Cost Optimization Playbook

Finding the ROI of AI: The Finance Perspective