The digital world is undergoing a massive transformation powered by the convergence of two major trends: an insatiable demand for real-time insights from data, and the rapid advancement of Generative artificial intelligence (AI). Leaders like Amazon, Microsoft, and Google are in a high-stakes race to deploy Generative AI to drive innovation. Bloomberg Intelligence predicts that the Generative AI market will grow at a staggering 42% year over year in the next decade, from $40 billion in 2022 to $1.3 trillion.

Meanwhile, this computational force is creating a massive surge in energy demand—posing serious consequences for today’s data center operators. Current power conversion and distribution technologies in the data center can’t handle the increase in demand posed by the cloud and machine learning—and certainly not from power-hungry Generative AI applications. The quest for innovative data center solutions has never been more critical.

Gallium Nitride (GaN) semiconductors emerge as a pivotal solution to data center power concerns, helping counter the impact of Generative AI challenges. We dive into how Generative AI affects data centers, the advantages GaN, and a prevailing industry perception of the Power Usage Effectiveness (PUE) metric—which is creating headwinds despite GaN’s robust adoption. With Generative AI intensifying power demands, swift measures are essential to reshape this perception and propel GaN adoption even further.

The rising impact of Generative AI on the data center

Today’s data center infrastructure, designed for conventional workloads, is already strained to its limits. Meanwhile, the volume of data across the world doubles in size every two years—and the data center servers that store this ever-expanding information require vast amounts of energy and water to operate. McKinsey projects that the U.S. alone will see 39 gigawatts of new data center demand, about 32 million homes’ worth, over the next five years.

The energy-intensive nature of generative AI is compounding the data center power predicament. According to one research article, the recent class of generative AI models requires a ten to a hundred-fold increase in computing power to train models over the previous generation. Generative AI applications create significant demand for computing power in two phases: training the large language models (LLMs) that form the core of generative AI systems, and then operating the application with these trained LLMs.

If you consider that a single Google search has the potential to power a 100W lightbulb for 11 seconds, it’s mind-boggling to think that one ChatGPT AI session consumes 50 to 100 times more energy than a similar Google search. Data centers are not prepared to handle this incredible surge in energy consumption. One CEO estimates that $1 trillion will be spent over the next four years upgrading data centers for AI.

Unfortunately, while technologies like immersion cooling, AI-driven optimizations, and waste heat utilization have emerged, they offer only partial solutions to the problem. A critical need exists for power solutions that combine high efficiency, compact form factors, and deliver substantial power outputs. Power electronics based on silicon are inefficient, requiring data centers to employ cooling systems to maintain safe temperatures.

GaN: Unparalleled performance and efficiency

GaN offers unparalleled performance and efficiency compared to traditional power supply designs, making it an ideal option for today’s data centers—particularly as Generative AI usage escalates. GaN transistors can operate at faster switching speeds and have superior input and output figures-of-merit. These features translate into system benefits including higher operating efficiency, exceeding Titanium, and increased power density.

GaN transistors enable data center power electronics to achieve higher efficiency levels—curbing energy waste and generating significantly less heat. The impact is impressive. In a typical data center environment, each cluster of ten racks powered by GaN transistors can result in a yearly profit increase of $3 million, a reduction of 100 metric tons of CO2 emissions annually, and a decrease in OPEX expenses by $13,000 per year. These benefits will only increase as the power demands of Generative AI increase and rack power density rises 2-3X.

While the benefits of GaN are profound, why aren’t even more data center operators swiftly incorporating the technology? Adoption faces headwinds from what we call the “PUE loophole”—an often-overlooked weakness within the widely accepted PUE metric.

The PUE Loophole

The PUE metric is the standard tool for assessing data center energy efficiency, calculated by dividing the total facility power consumption by the power utilized by IT equipment. The metric helps shape data center operations and guides efforts to reduce energy consumption, operational costs, and environmental impact.

Data center operators continuously strive to monitor and improve the PUE to indicate reduced energy consumption, carbon emissions, and associated costs. However, the PUE metric measures how efficiently power is delivered to servers—yet it omits power conversion efficiency within the server itself. As a result, the PUE calculation does not provide a comprehensive view of the energy efficiency within a data center—creating a blind spot for data center operators.

Consider that many servers still use AC/DC converters that are 90 percent efficient or less. While this may sound impressive—10 percent or more of all energy in a data center is lost. This not only increases costs and CO2 emissions, but it also creates extra waste heat, putting additional demands on cooling systems.

GaN is remarkably effective in addressing the PUE Loophole. For instance, the latest generation of GaN-based server AC/DC converters are 96 percent efficient or better – which means that more than 50 percent of the wasted energy can instead be used effectively. Across the entire industry, this could translate into more than 37 billion kilowatt-hours saved every year—enough to run 40 hyperscale data centers.

GaN can provide an immediately cost-effective way to close the PUE loophole and save high amounts of energy. But because the PUE doesn’t consider AC/DC conversion efficiency in the server, there is no incentive to make AC/DC converters more efficient.

This article was authored by Paul Wiener, Vice President of Strategic Marketing at GaN Systems.

Explore more

• Global GaN Power Device Market Set to Soar, Reaching $1.33 Billion by 2026
• GaN in Automotive Applications and Who’s Leading?

(Photo credit: Google)

According to TrendForce research, the global high-performance computing market reached approximately US$36.8 billion in 2021, growing 7.1% compared to 2020. The United States is still the largest market for high-performance computing in the world with an approximate 48% share, followed by China and Europe, with a combined share of approximately 35%. Segregated into application markets, high-performance computing is most widely used in scientific research, national defense/government affairs, and commercial applications, with market shares of 15%, 25%, and 50%, respectively. In terms of product type, software (including services) and hardware account for 58% and 42% of the market, respectively.

Since high-performance computing can support data analysis, machine learning (ML), network security, scientific research, etc., it plays a key role in military fields such as nuclear warhead design and missile explosion simulations. Therefore, there are relatively few players occupying key positions in the value chain. Primary suppliers are Fujitsu, HPE, Lenovo, and IBM. These four manufacturers account for a market share of approximately 73.5% globally.

In addition, the continuous development of smart cities, smart transportation, self-driving cars, the metaverse, and space exploration and travel programs launched by Space X, Blue Origin, and Virgin Galactic will increase the demand for high-performance computing focused on R&D and testing along the two major axes of simulation and big data processing and analysis. The global high-performance computing market is expected to reach US$39.7 billion in 2022, with a growth rate of 7.3%. The CAGR (Compound Annual Growth Rate) of the global high-performance computing market from 2022 to 2027 will be 7.4%.

In view of this, the global high-performance computing market is growing steadily but not by much. The reason is that many of the aforementioned commercial application terminals are still in the growth stage, so high-performance computing technologies and solutions adopted by cloud service providers are limited to local deployment This enables HPC servers to scale on-premises or in the cloud and provides dedicated storage systems and software to drive innovation, thereby accelerating the development of hybrid HPC solutions.

In terms of end-use, the high-performance computing market is segmented into BFSI (Banking, Financial Services and Insurance), manufacturing, healthcare, retail, transportation, gaming, entertainment media, education & research, and government & defense. High-performance computing’s highest revenue share was derived from the government and defense market in 2021, primarily due to related agencies actively adopting cutting-edge and advanced IT solutions to improve computing efficiency. At present, government agencies in the United States, China, Japan, South Korea, as well as European countries have successively adopted high-performance computing systems to support digitization projects and contribute to economic development. Therefore, in 2021, the global scale of the on-premise high-performance computing server market was US$14.8 billion, of which Supercomputer, Divisional, Departmental, and Workgroup accounted for 46.6%, 18.9%, 25%, and 9.5% of the market, respectively. The global on-premise high-performance computing server market in 2022 is expected to reach US$16.7 billion with Supercomputer and Divisional growing by 11.5% and 15.2% compared with 2021.

（Image credit: Pixabay）

The new normal ushered in by the pandemic will not only become the driving force of digital transformation but will also continue to drive the server market in 2022, according to TrendForce’s investigations. It is worth noting that potential unmet demand in 2021 and the risk of future server component shortages will become medium and long-term variables that influence the market. Analyzing the shipment volume of completed servers, a growth rate of approximately 4-5% in completed server shipments is expected next year with primary shipment dynamics remaining concentrated in North American data centers with an annual growth rate of approximately 13-14%. From the supply chain perspective, the ODM Direct business model has gradually replaced the business model of the traditional server market, giving cloud service providers the ability to respond quickly to market changes. However, based on the unpredictability of the market, TrendForce assumes two forecasts for server growth trends. One, the supply situation of key components is effectively improved. Two, the supply situation of key components is exacerbated.

TrendForce states, based on the current situation as materials issues ease quarter by quarter, the annual growth rate of server shipments in 2022 will reach 4~5%. There are three primary factors driving market momentum. First, the introduction of the Intel Sapphire Rapids and AMD Genoa platforms into the market may once again stimulate the replacement of enterprise client servers and infrastructure construction in data centers. Second, the market generally believes that transformational needs generated by the pandemic in 2022, such as shifts in working paradigms and the new normal, will continue to drive the cloud market. Furthermore, international tensions have led to geopolitical uncertainty, which in turn has encouraged countries to tighten their control over data sovereignty and prompting the emergence of small-scale data centers in specific geographic locations.

Actual shipment volume of completed servers in 2022 depends on improvement of supply chain issues

Based on the two aforementioned assumptions, if the pandemic is effectively controlled next year, and international logistics, satisfaction of materials demand, and other factors either return to normal or fare better than expected, server companies will be able to increase their shipping capabilities and the annual growth rate of shipments in the overall server market will be able to reach 5-6% while the annual growth rate of ODM-Direct will approach 15%, up from the original forecast 13%. However, if the pandemic intensifies next year, the overall global economy will continue under that dark cloud which will greatly affect the willingness of companies to invest. In that case, the estimated annual growth rate of server shipments will fall to only 3-4%. In addition, the growth momentum of North American data centers will also be affected leading to an annual growth rate of ODM-Direct of only 10%, approximately.

As a whole and continuing under the influence of the two-year pandemic, the business trend of flexible deployment is irreversible. Regardless of overall economic changes, TrendForce expects double-digit growth in the demand for ODM-direct servers next year while overall server demand will also maintain a positive growth trajectory. However, continued attention should be focused on issues related to server order fulfillment in the broader market, including the fulfillment rate of key PMIC and LAN chip materials. At the same time, another major market variable will be whether Intel and AMD can introduce their two new platforms as scheduled next year and inject additional momentum into equipment replacement.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com

Suppliers and clients in the server DRAM market are still having difficulty in reaching agreements on prices for 3Q21 contracts even though the quarter is well underway, according to TrendForce’s latest investigations. Hence, server DRAM contract prices are much more varied than before. Regarding the price trend in July, contract quotes for the mainstream 32GB RDIMMs rose by 5-7% MoM.

However, the price hikes have led to a reduction in demand, and there are indications that server DRAM sales bits will register some decline for 3Q21. The release of server CPUs based on the new platforms is driving the procurement of higher-density 64GB RDIMMs, but this has not resulted in a significant corresponding increase in content per unit. The general trend for buyers is to replace two 32GB modules with one 64GB module, rather than a one-to-one replacement as DRAM suppliers previously expected. Contract prices of 64GB RDIMMs rose by 5-7% MoM for July, though prices were below this range for some transactions.

TrendForce’s analysis shows that server DRAM suppliers and buyers are finding it difficult to reach a consensus on prices because DRAM suppliers expect that the demand for server DRAM modules is going to surge in 3Q21 as the third quarter is the traditional peak season for the server market. As well, suppliers also anticipate that the adoption of new server processor platforms will increase the memory content in servers.

With a more optimistic demand outlook, suppliers have adjusted their product mixes to allocate more of their production capacity to server DRAM. Hence, the supply fulfillment rate has risen significantly in the server DRAM market in 3Q21. Server DRAM buyers, on the other hand, already have a high level of inventory. Clients in the data center segment were aggressively stockpiling during the first half of this year due to worries about the impact of the COVID-19 pandemic on the supply chain. They now need some time to consume their inventories and are reluctant to procure more DRAM modules.

Contract prices will be constrained to rise further in 4Q21 as demand side has turned conservative

Currently, enterprise server OEMs in North America have finished arranging their quarterly contracts, whereas numerous cloud service providers and Chinese enterprise server OEMs are still in the midst of negotiations. TrendForce believes that, in order to reach their targets for sales and shipments, server DRAM suppliers may be willing to cut more “special deals” for server DRAM products in August. Specifically, suppliers will push for lock-in contracts that offer adjustable prices for fixed quantities.

On the whole, the general behaviors of DRAM buyers with regards to procurement have changed noticeably form the first half of this year. As the demand related to servers, PCs, and other major applications slows down, the whole DRAM market will gradually shift to the state of oversupply. Since the DRAM market is an oligopoly, the major suppliers will still have significant leverage in price negotiations. Quotes for server DRAM products could therefore rise further by 5-10% QoQ in 3Q21. However, given that prices have yet to be finalized for a substantial portion of 3Q21 contracts, the transaction volume is also very limited. This, in turn, will inevitably create a lot of uncertainties with respect to the price trend in 4Q21.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com

Thanks to their flexible pricing schemes and diverse service offerings, CSPs have been a direct, major driver of enterprise demand for cloud services, according to TrendForce’s latest investigations. As such, the rise of CSPs have in turn brought about a gradual shift in the prevailing business model of server supply chains from sales of traditional branded servers (that is, server OEMs) to ODM Direct sales instead.

Incidentally, the global public cloud market operates as an oligopoly dominated by North American companies including Microsoft Azure, Amazon Web Services (AWS), and Google Cloud Platform (GCP), which collectively possess an above-50% share in this market. More specifically, GCP and AWS are the most aggressive in their data center build-outs. Each of these two companies is expected to increase its server procurement by 25-30% YoY this year, followed closely by Azure.

TrendForce indicates that, in order to expand the presence of their respective ecosystems in the cloud services market, the aforementioned three CSPs have begun collaborating with various countries’ domestic CSPs and telecom operators in compliance with data residency and data sovereignty regulations. For instance, thanks to the accelerating data transformation efforts taking place in the APAC regions, Google is ramping up its supply chain strategies for 2021.

As part of Google’s efforts at building out and refreshing its data centers, not only is the company stocking up on more weeks’ worth of memory products, but it has also been increasing its server orders since 4Q20, in turn leading its ODM partners to expand their SMT capacities. As for AWS, the company has benefitted from activities driven by the post-pandemic new normal, including WFH and enterprise cloud migrations, both of which are major sources of data consumption for AWS’ public cloud.

Conversely, Microsoft Azure will adopt a relatively more cautious and conservative approach to server procurement, likely because the Ice Lake-based server platforms used to power Azure services have yet to enter mass production. In other words, only after these Ice Lake servers enter mass production will Microsoft likely ramp up its server procurement in 2H21, during which TrendForce expects Microsoft’s peak server demand to take place, resulting in a 10-15% YoY growth in server procurement for the entirety of 2021.

Finally, compared to its three competitors, Facebook will experience a relatively more stable growth in server procurement owing to two factors. First, the implementation of GDPR in the EU and the resultant data sovereignty implications mean that data gathered on EU residents are now subject to their respective country’s legal regulations, and therefore more servers are now required to keep up the domestic data processing and storage needs that arise from the GDPR. Secondly, most servers used by Facebook are custom spec’ed to the company’s requirements, and Facebook’s server needs are accordingly higher than its competitors’. As such, TrendForce forecasts a double-digit YoY growth in Facebook’s server procurement this year.

Chinese CSPs are limited in their pace of expansions, while Tencent stands out with a 10% YoY increase in server demand

On the other hand, Chinese CSPs are expected to be relatively weak in terms of server demand this year due to their relatively limited pace of expansion and service areas. Case in point, Alicloud is currently planning to procure the same volume of servers as it did last year, and the company will ramp up its server procurement going forward only after the Chinese government implements its new infrastructure policies. Tencent, which is the other dominant Chinese CSP, will benefit from increased commercial activities from domestic online service platforms, including JD, Meituan, and Kuaishou, and therefore experience a corresponding growth in its server colocation business.

Tencent’s demand for servers this year is expected to increase by about 10% YoY. Baidu will primarily focus on autonomous driving projects this year. There will be a slight YoY increase in Baidu’s server procurement for 2021, mostly thanks to its increased demand for roadside servers used in autonomous driving applications. Finally, with regards to Bytedance, its server procurement will undergo a 10-15% YoY decrease since it will look to adopt colocation services rather than run its own servers in the overseas markets due to its shrinking presence in those markets.

Looking ahead, TrendForce believes that as enterprise clients become more familiar with various cloud services and related technologies, the competition in the cloud market will no longer be confined within the traditional segments of computing, storage, and networking infrastructure. The major CSPs will pay greater attention to the emerging fields such as edge computing as well as the software-hardware integration for the related services.

With the commercialization of 5G services that is taking place worldwide, the concept of “cloud, edge, and device” will replace the current “cloud” framework. This means that cloud services will not be limited to software in the future because cloud service providers may also want to offer their branded hardware in order to make their solutions more comprehensive or all-encompassing. Hence, TrendForce expects hardware to be the next battleground for CSPs.

For more information on reports and market data from TrendForce’s Department of Semiconductor Research, please click here, or email Ms. Latte Chung from the Sales Department at lattechung@trendforce.com