Why the Cloud Isn't Getting More Expensive (Yet)

When Hetzner, a popular German hosting provider, recently announced price hikes of more than 30% for their services¹, this did not go unnoticed. The reasons are pretty clear: Hardware component prices jumped massively. When I accidentally bought the wrong RAM sticks in November and I needed to return them, in a two-week span the price doubled. I imagine the shop must’ve felt quite happy with my mistake – although I doubt that shops really have feelings.

Germany is not the first place I would start a hosting provider: Electricity is expensive, finding enough land for data centers is difficult, permitting is always a struggle, and personnel costs aren’t that cheap, but through fierce competition in the early 2000s a scene of reliable, low-cost providers emerged. The shrewd ones that managed to work within these thin margins have stuck around.

When AWS started building their cloud offering, their pricing structure wasn’t beholden to the same logic. SmugMug, their first enterprise customer, was happy to switch to S3 without saving any money. Passing off the responsibility to a third party at cost parity made sense to them. Of course this was 2006: All hardware was still getting dramatically cheaper. Between 2006 and 2018 hard disks dropped in price more than 90%.² Not even the floods in Thailand in 2011 could meaningfully disrupt this trend.³

In the early days, AWS passed some of these savings on, possibly to deter competitors from realizing that the cloud was a good business to be in. That deterrence, by the way, seems to have worked: All other successful entrants emerged from their own needs for cloud services. The price cuts stopped around 2014 and from then on, prices have found stable ground: Their mindshare was established, other big name providers had similar pricing, and customers wouldn’t be switching en masse to smaller providers.

AWS did launch some new products that allowed a cheaper on-ramp, like Lambda, which allowed to serve code without paying a flat rate for the substrate it’s running on, along with other “serverless” services that often start out at 0 USD, but have a sharp cost curve once used heavily.

With their AWS Graviton architecture, they also presented some cheaper EC2 options: A t4g.nano is just around 3 USD/month. Also, it only has ½ GB of memory, which makes it unusable for anything that isn’t engineered specifically to run within these confines.

The sharply rising hardware prices in recent months are attributed to the AI boom, or bubble, as some prefer. There is some circumstantial evidence: AI companies are raising insane capital, which they use to buy GPUs from a limited manufacturing pool. GPUs need memory chips, therefore GPUs and memory are getting expensive. Everybody who is not an AI unicorn needs to buy these with their real, hard earned cash, which may seem massively unfair.

But unless you plan to build a gaming rig, this is just whinging, because chances are, you have already been paying a premium. Apple is charging 400 USD for a 16 GB RAM upgrade (to 32 GB). And if you are like me and don’t find this completely outrageous yet, AWS charges for a mere 7.5 GB memory upgrade (between t4g.nano and t4g.large) over 550 USD per year!

Of course, buying instead of renting gives you the magic of depreciation: In an earnings call, AWS has reported that they run hardware for five years, meaning that even at current memory prices they have a margin of more than 15x.⁴ With a 3 year reserved instance, it shrinks into a pitiful 6x. Assuming AWS is paying retail price, which I am sure they have figured out a way not to.

For all the thin-margin hosting companies, this bears an existential risk: Their customer-facing price per GB of RAM often hovers around 10 EUR (~12 USD) per year, which only starts to break even after more than 2.5 years. Assuming the memory prices stay flat.

The market does not show any signs of relief: The number of memory manufacturers has shrunk to just three, and new contenders will find it difficult. DDR5 memory has extremely tight timing tolerances, and the big ones are utilizing EUV lithography to manufacture those chips. CXMT, a Chinese manufacturer of memory, has started a DDR5 line, but as Chinese entities are barred from importing current-generation lithography equipment, they are still struggling with engineering challenges.⁵

Chinese labs are working on their own EUV technology⁶, which may reach maturity in 2030, but everybody would prefer to be Nvidia instead of just another memory manufacturer, so it’s not clear how new capacities would be utilized. Cutting edge hardware takes years or decades to get into production while predictions are imprecise.

This gold rush sparks a lot of interest in competition for Nvidia, as nobody likes to pay 25k USD for a single H100: Google invested early into TPUs, and Cerebras offers powerful hardware for training and inference you can buy. Still, any of them need to utilize manufacturing capacity which is massively constrained and sold to the highest bidder. Fat margins insulate from icy market conditions, but attract hungry competition.

Even if AI company valuations don’t stay as high, LLMs are here to stay and will continue to devour a lot of the available hardware resources. But if capital dries up, purchase orders may disappear overnight, triggering a sell-off. OpenAI already slashed their spending plans by over 800 billion USD.⁷

The three big hyperscalers are growing massively in revenue not least because of the huge spend by AI companies, so they don’t have to worry about decreased margins in other areas – at least for the moment. The big question will be if the revenue streams from AI services will be sufficient, once the products mature.

In the tech industry we have been massively spoiled that costs for hardware only ever went down, and somebody else had to do all the hard work to get there. Sloppy software engineering could just gobble up all the resources. But what happens if the free ride is over?

The cloud revolution created generations of software developers who do not know how to run on hardware, so they have to rent it from others. The AI revolution may cause developers to not know how to build software, thus having to rent it by the token. The question isn’t really whether the cloud will get more expensive: It’s how many layers of rent you will be willing to pay.