Omdia: AI boosts server spending but unit sales still plunge

Server unit shipments for calendar year 2023 could drop by as much as 20% when compared to last year, according to projections by a market research firm. However, revenues are growing for the server vendors, so while they may be selling fewer servers in terms of units, they’re selling pricier, more decked-out hardware.

In its latest market update for cloud and data center, Omdia forecasts server unit shipments to decline by 17% to 20% this year, while revenue is expected to grow by 6% to 8%.

Omdia cites the rise of heterogeneous computing for shift in spending. Instead of buying servers with just x86 processors, customers are buying servers with GPUs, DPUs, AI processing and inferencing chips, and other silicon processors.

With so many expensive chips going into the servers, Omdia predicts CPUs and co-processors to account for 30% of data center spending by 2027, compared with less than 20% in the previous decade.

Unsurprisingly, this shift is being driven by AI. Omdia said there was a dramatic shift in data center investment priorities this year, driven by a rush to build AI capacity, which made forecasting in 2023 incredibly difficult.

Nvidia is the most popular supplier of GPUs for AI processing despite two ready competitors in Intel and AMD. Last quarter, Nvidia sold almost a half million GPUs to data center customers, and those processors go for a rumored $40,000 per card.

And while Nvidia didn’t complain about supply on its most recent earnings call, apparently it does have a problem with it. Omdia says leading server OEMs like Dell, Lenovo, and HPE are not able to fulfil GPU server orders yet, due to a lack of GPU supply from Nvidia. OEMs indicated a lead time of 36 to 52 weeks for servers configured with Nvidia H100 GPUs.

That’s in part because a few players are taking all of the supply. Omdia noted that both Microsoft and Meta are on track to receive 150,000 of Nvidia’s H100 accelerators by the end of this year – which is three times as many as Nvidia’s other major customers, Google, Amazon and Oracle.

These high-powered servers are also driving demand for better power efficiency and management. Data center operators have to get more compute power out of the same power envelope due to constraints and power supply. Omdia said rack power distribution revenue in 1H23 was up 17% over last year, while UPS revenue growth for the first half of 2023 was 7% ahead of last year.

“With a ramp of professional services for generative AI enabling broad enterprise adoption in 2024 and beyond, the only thing that can curb the current rate of AI deployment is power availability,” Omdia said in its report.

AI is also driving demand for liquid cooling, since air cooling is simply no longer efficient for the very hot processors used in AI. Cooling vendors and server OEMs tell Omdia direct-to-chip liquid cooling is ramping in line with its forecast for 80% revenue growth within the year, and it noted that server vendor Super Micro recently said that it expects 20% of the servers it ships in 4Q23 will use direct-to-chip liquid cooling.

Up through 2027, Omdia expects continued growth in rack power density, server performance improvement, and server fleet consolidation. There will be a strong focus on computing performance to enable the commercialization of AI. AI models will continue to be a research project requiring a great deal of tuning, even libraries of pre-trained models.

Because of the rate of advancements, it expects meaningful server refresh cycles to take place, where enterprises will prioritize equipment consolidation and utilization improvement during the refresh cycle. And it expects the trend for hyper-heterogeneous computing, with servers configured with 1-2 CPUs and up to 20 workload-optimized custom-build co-processors, will enable a significant consolidation in server fleets.