The $6.88 hour: what a GPU actually costs on EC2, and when Bedrock tokens win

Published November 6, 2026

EC2 and Bedrock price the same NVIDIA silicon through two different accounting systems. EC2 sells you the hour whether the card produced anything or not. Bedrock sells you the tokens that came out.

An H100 hour in us-east-1 is $6.88 on-demand. That same $6.88 buys about 50,000 output tokens at the most expensive text model on Bedrock, $137.50 per million. Below some volume of generated text, renting the card is the more expensive way to buy the same work, and the line falls somewhere you can actually compute.

The ladder from T4 to B300 runs 34x, and it is not smooth

Prices are us-east-1, Linux, on-demand, from our EC2 dataset, divided by GPU count for the per-GPU rate.

The per-GPU spread is 34x from the T4 to the B300, and it steps rather than climbs: the H100 costs 2.5x the A100, the B200 costs 2.1x the H100. Monthly equivalents get absurd quickly. One p5.4xlarge is $5,022/month. A p5.48xlarge is $40,179/month. A p6-b300.48xlarge is $103,963/month. Six-figure monthly line items are ordinary now.

Before assuming the cheap end of the ladder is the bargain, price the memory.

The H100 is the worst VRAM deal in the P-series

Divide the instance price by total VRAM and the ladder reorders:

An H100 crams 80 GB into its $6.88, landing at $0.0860/GB-hr, the most expensive gigabyte of VRAM in the P-series. The B200 moves 179 GB per card and lands at $0.0796 despite a GPU-hour more than double. The B300, the most expensive hour in the dataset, is still cheaper per gigabyte than the H100. Memory capacity is where the newest silicon earns its sticker price, and on a per-gigabyte basis the best P-series deal is the A100-80GB on p4de at $0.0429.

One data quirk worth flagging, the same one the GPU selection guide calls out: the p5en.48xlarge row records 141 GB, which is the H200’s per-card spec rather than the ~1,128 GB you would expect across eight cards the way the other eight-GPU rows report theirs. We pass it through as recorded. Corrected, the H200 works out to roughly $0.056/GB-hr, between the p4de A100 and the B300.

Capability per dollar climbs just as hard

Our dataset carries gpu_compute_capability, a rough generation proxy: 7.5 for the T4, 8.0 for the A100, 8.6 for the A10G, 9.0 for the H100 and H200, 10.0 for the B200 and B300. Divide $/GPU-hr by it:

From the T4 to the B300 you pay 34x more per GPU-hour for about 1.3x the capability number. Read that as a measure of how AWS prices generation, not of throughput: capability is a catalog field, not a benchmark. The curve says the older tiers are where per-capability value lives, and the T4 and L4 sitting at the bottom has a reason beyond being old.

Eight GPUs cost exactly eight times one

You might expect bulk pricing inside a family. AWS charges 1.00x. A p5.4xlarge is one H100 at $6.88; a p5.48xlarge is eight H100s at $55.04, which is $6.88 per GPU. No discount, to the hundredth of a cent. The P-series prices linearly, the same way p3 does and the way CPU families do. If you need eight H100s to shard a model, the 48xlarge is the honest way to buy them.

The G-series goes the other way, as the GPU guide details: sizing up inside g5 or g6e often means more CPU and RAM around the same single GPU, so the jumbo sizes are routinely worse per GPU than the base xlarge.

What $6.88 buys in tokens

Take one p5.4xlarge hour at $6.88 and price it against Bedrock’s list rates for current Claude models in us-east-1. The breakeven is the output volume at which self-hosting stops losing money on a naive cost comparison:

At the top of the list, an hour of H100 has to produce about 50,000 output tokens to undercut Bedrock’s most expensive model. At the bottom, it has to produce 2.5 million, which you should treat as a warning about the Haiku tier rather than a plan. The blend column is the realistic one: three input tokens for every output token, priced at the same model’s input rate.

An open-weight model doing real inference clears 50,000 output tokens an hour comfortably. That is the case for renting. The counter-case is the idle hour: the GPU meter runs whether or not anything is generating, and the token meter only runs when the model answers. A card that sits 80% idle is $5,022 a month for nothing.

So which meter

Sustained hours, batch jobs, training, anything you can keep fed: the GPU hour wins, and the harder you run it the more decisively. Spiky, interactive, unpredictable traffic: tokens win, because you pay for no idle time and never manage a fleet. In between, the sub-hour elasticity of a token API usually beats a GPU you sized for peak load and paid for in the valleys.

Concretely:

1. Price the GPU-hour first. Take your candidate instance, divide its on-demand rate by GPU count, and keep the $/GPU-hr handy. The instance explorer filtered to accelerated instances pulls the list.
2. Check VRAM per dollar, not sticker price. The B200 and B300 are cheaper per gigabyte than the H100 despite higher hourly rates. If your model is memory-bound, $/GB-hr picks the winner, and the comparison tool lines them up.
3. Estimate real utilization before arguing. Below roughly 20% sustained, stay on the token API and let someone else eat the idle time. Above it, start pricing H100s.
4. Price the token side at your actual model. The AI models hub has every Bedrock model and rate, each with its own page for the list-price math.
5. Check the family page for quirks like the p5en memory field at /aws/families/p5en/ or /aws/families/g6e/ before committing eight cards.

The $6.88 H100 hour is a bad deal when the card idles and a good one when you can keep it fed. That single fact about utilization decides more AI bills than the choice of model.