Data centres put their money in new chip power as GenAI needs grow

Companies are ordering graphics processing units to handle the massive computing power needed for GenAI applications

There is one technology data centres need as the use of generative artificial intelligence (GenAI) becomes common: Graphics processing unit, or GPU, a specialised chip that renders graphics and images by performing rapid mathematical calculations that are needed in AI processing.

The scale of demand is evident in the results of chipmaker Nvidia. The firm’s data centre business reported a revenue of $18.4 billion in the fourth quarter ended on January 28. Revenue was up 27 per cent sequentially and grew 409 per cent year-on-year. Growth was fuelled by Nvidia’s H100 graphic cards or GPU.

Data centres’ demand for GPU-based servers is rising. According to a report by BIS Research, the GPU server market was worth $15.4 billion in 2023 and is expected to grow at a compound annual growth rate of around 32 per cent and reach  $61.7 billion by 2028.

In India, GPU-based data centres are just coming up. Yotta Infrastructure Solutions, a data centre infrastructure provider, has placed orders worth $1 billion for Nvidia H100 chips and to create GPU-based capabilities.

“We have placed an order of 16,000 H100 GPUs from Nvidia, out of which the first lot of 4,096 GPUs is getting delivered now, and balance will be coming up around the August-September period,” said Sunil Gupta, co-founder and chief executive officer of Yotta. 

GPU-based servers are much more powerful than those using conventional central processing unit (CPU) chips. The 16,000 H100 GPUs Yotta has ordered would have a processing capability comparable to top 10 supercomputers, said Gupta. 

Such servers are expensive to set up and require additional resources in terms of infrastructure and energy – costs smaller companies can’t afford, said industry experts.

“However, those betting on AI and Gen AI are going ahead with their spending. Some of the spending is expected to happen in India too,” said Naresh Singh, senior director analyst at Gartner.

GPU-based servers would need an additional investment of up to 25 per cent, apart from money spent on traditional CPU infrastructure, said Gupta. 

Some large Cloud infrastructure providers in India have announced the first investments in GPU-based data centres. CtrlS Datacenters recently said it would invest $2 billion in such centres in six years.

“…this expansion will involve integrating advanced technologies such as liquid cooling, AI HVAC, and optimised rack layouts to cater to the burgeoning demand for AI applications and Cloud services,” said Vipin Jain, president for data centre operations at CtrlS Datacenters. (HVAC refers to various technologies to control the temperature, humidity, and purity of the air in an enclosed space. A rack layout is the arrangement of servers.)

Gartner’s Singh said: “For AI-optimised server and AI network fabric, it is very early days in India, although some of the large data centre players are now providing GPU-optimised servers for specific customer needs.”

Singh said it was unlikely that GPU technology would soon replace conventional CPU-based servers. “Demand for such GPU clusters [servers] is also dependent on how fast users take to AI and Gen AI. At this point, most of the users dabbling with these technologies are at an exploratory stage and those that have started using are mostly consuming it as a service from an AI service provider,” he said.

“I see three types of demand: One is from startups in India trying to create very large (computing) models. Second, the government trying to source lots of GPUs so that it can give it not only to larger startups and IITs, but also give it to smaller companies so that the whole ecosystem can build up,” said Gupta of Yotta, referring to the Indian Institutes of Technology.