The US Department of Commerce issued new restrictions on AI chips on October 17, 2023, with a focus on controlling the export of chips to China, including NIVIDA’s A800, H800, L40S, and RTX4090, among others. Taiwanese manufacturers primarily serve cloud service providers and brand owners in North America, with relatively fewer shipments to Chinese servers. However, Chinese manufacturers, having already faced two chip restrictions imposed by the US, recognize the significance of AI chips in server applications and are expected to accelerate their in-house chip development processes.

TrendForce’s Insights:

1. Limited Impact on Taiwanese Manufacturers in Shipping AI Servers with H100 GPUs

Major Taiwanese server manufacturering companies, including Foxconn, Quanta, Inventec, GIGABYTE, and Wiwynn, provide AI servers equipped with H100 GPUs to cloud data centers and brand owners in Europe and the United States. These Taiwanese companies have established some AI server factories outside China, in countries such as the US, the Czech Republic, Mexico, Malaysia, and Thailand, focusing on producing L10 server units and L11 cabinets in proximity to end-users. This strategy aligns with the strategic needs of US cloud providers and brand owners for global server product deployment.

On the other hand, including MiTAC, Wistron, and Inventec, also provide server assembly services for Chinese brands such as Inspur and Lenovo. Although MiTAC has a significant share in assembling Inspur’s servers, it acquired Intel DSG (Data Center Solutions Group) business in July 2023. Therefore, the focus of AI servers remains on brand manufacturers using H100 GPUs, including Twitter, Dell, AWS, and European cloud service provider OVH. It is speculated that the production ratio of brand servers will be adjusted before the new restrictions are enforced.

Wistron is a major supplier for NVIDIA’s AI server modules, DGX A100, and HGX H100. Its primary shipments are to end-users in Europe and the United States. It is expected that there will be adjustments in the proportion of shipments to Chinese servers following the implementation of the restrictions.

Compal has fewer AI server orders compared to other Taiwanese manufacturers. It has not yet manifested any noticeable changes in Lenovo server assembly proportions. The full extent of the impact will only become more apparent after the enforcement of the ban.

During the transitional period before the implementation of the chip ban in the United States, the server supply chain can still adapt shipments based on local chip demand in China to address market impacts resulting from subsequent chip controls.

2. Chinese Manufacturers Focusing on Accelerating In-House Chip Development

Chinese cloud companies had already started developing their AI chips before the first U.S. chip restrictions in 2022. This included self-developed AI chips like Alibaba Cloud’s T-HEAD, a data center AI chip, and they expanded investments in areas such as DRAM, AI chips, and semiconductors with the aim of establishing a comprehensive IoT system from chips to the cloud.

Baidu Cloud, on the other hand, accelerated the development of its third-generation self-developed Kunlun chip, designed for cloud and edge computing, with plans for an early 2024 release.

Tencent introduced three self-developed chips in 2021, including an AI inference chip called Zixiao, used for Tencent’s meeting business; a video transcoding chip called Canghai, used in cloud gaming and live streaming applications; and a smart network card chip named Xuanling, applied in network storage and computing.

ByteDance made investments in cloud AI chips through its MooreThread initiative in 2022 for applications in AI servers. Huawei released the Ascend 900 chip in 2019 and is expected to introduce the Ascend 930B AI chip in the latter half of 2024. While this chip has the same computational power as the NVIDIA A100 chip, its performance still requires product validation, and it is speculated that it may not replace the current use of NVIDIA GPUs in Chinese AI servers.

Despite the acceleration of self-developed chip development among Chinese cloud server manufacturers, the high technological threshold, lengthy development cycles, and high costs associated with GPU development often delay the introduction of new server products. Therefore, Chinese cloud companies and brand manufacturers continue to purchase NVIDIA GPUs for the production of mid to high-end servers to align with their economic scale and production efficiency.

In response to the new U.S. restrictions, Chinese cloud companies have adopted short-term measures such as increasing imports of existing NVIDIA chips and building up stockpiles before the enforcement of the new restrictions. They are also focusing on medium to long-term strategies, including accelerating resource integration and shortening development timelines to expedite GPU chip manufacturing processes, thus reducing dependency on U.S. restrictions.

In its FY2Q24 earnings report for 2023, NVIDIA disclosed that the U.S. government had imposed controls on its AI chips destined for the Middle East. However, on August 31, 2023, the U.S. Department of Commerce stated that they had “not prohibited the sale of chips to the Middle East” and declined to comment on whether new requirements had been imposed on specific U.S. companies. Both NVIDIA and AMD have not responded to this issue.

TrendForce’s analysis:

• Close ties between Middle Eastern countries and China raise U.S. concerns:

In NVIDIA’s FY2Q24 earnings report, it mentioned, “During the second quarter of fiscal year 2024, the USG informed us of an additional licensing requirement for a subset of A100 and H100 products destined to certain customers and other regions, including some countries in the Middle East.” It is speculated that the U.S. is trying to prevent high-speed AI chips from flowing into the Chinese market via the Middle East. This has led to controls on the export of AI chips to the Middle East.

Since August 2022, the U.S. has imposed controls on NVIDIA A100, H100, AMD MI100, MI200, and other AI-related GPUs, restricting the export of AI chips with bidirectional transfer rates exceeding 600GB/s to China. Saudi Arabia had already signed a strategic partnership with China in 2022 for cooperation in the digital economy sector, including AI, advanced computing, and quantum computing technologies. Additionally, the United Arab Emirates has expressed interest in AI cooperation with China. There have been recent reports of Saudi Arabia heavily acquiring NVIDIA’s AI chips, which has raised concerns in the U.S.

• Huawei is expected to release AI chips comparable to NVIDIA A100 in the second half of 2024; competition is yet to be observed:

Affected by U.S. sanctions, Chinese companies are vigorously developing AI chips. iFlytek is planning to launch a new general-purpose LLM (Large Language Model) in October 2023, and the AI chip Ascend 910B, co-developed with Huawei, is expected to hit the market in the second half of 2024, with performance claimed to rival that of NVIDIA A100. In fact, Huawei had already introduced the Ascend 910, which matched the performance of NVIDIA’s V100, in 2019. Considering Huawei’s Kirin 9000s, featured in the flagship smartphone Mate 60 Pro released in August 2023, it is highly likely that Huawei can produce products with performance comparable to A100.

However, it’s important to note that the A100 was already announced by NVIDIA in 2020. This means that even if Huawei successfully launches a new AI chip, it will already be four years behind NVIDIA. Given the expected 7nm process for Huawei’s Ascend 910B and NVIDIA’s plan to release the 3nm process-based Blackwell architecture GPU B100 in the second half of 2024, Huawei will also lag behind by two generations in chip fabrication technology. With the parameters of LLM doubling annually, the competitiveness of Huawei’s new AI chip remains to be observed.

• China remains NVIDIA’s dominion in the short term:

Despite the active development of AI chips by Chinese IC design house, NVIDIA’s AI chips remain the preferred choice for training LLM models among Chinese cloud companies. Looking at the revenue performance of the leading Chinese AI chip company, Cambricon, its revenue for the first half of 2023 was only CNY 114 million, a YoY decrease of 34%. While being added to the U.S. Entity List was a major reason for the revenue decline, NVIDIA’s dominance in the vast Chinese AI market is also a contributing factor. It is estimated that NVIDIA’s market share in the Chinese GPU market for AI training exceeded 95% in the first half of 2023. In fact, in the second quarter of 2023, the China market accounted for 20-25% of NVIDIA’s Data Center segment revenue.

The main reason for this is that the Chinese AI ecosystem is still quite fragmented and challenging to compete with NVIDIA’s CUDA ecosystem. Therefore, Chinese companies are actively engaged in software development. However, building a sufficiently attractive ecosystem to lure Chinese CSPs in the short term remains quite challenging. Consequently, it is expected that NVIDIA will continue to dominate the Chinese market for the next 2-3 years.

(Photo credit: NVIDIA)

According to the news from Chinatimes, Asus, a prominent technology company, has announced on the 30th of this month the release of AI servers equipped with NVIDIA’s L40S GPUs. These servers are now available for order. The L40S GPU was introduced by NVIDIA in August to address the shortage of H100 and A100 GPUs. Remarkably, Asus has swiftly responded to this situation by unveiling AI server products within a span of less than two weeks, showcasing their optimism in the imminent surge of AI applications and their eagerness to seize the opportunity.

Solid AI Capabilities of Asus Group

Apart from being among the first manufacturers to introduce the NVIDIA OVX server system, Asus has leveraged resources from its subsidiaries, such as TaiSmart and Asus Cloud, to establish a formidable AI infrastructure. This not only involves in-house innovation like the Large Language Model (LLM) technology but also extends to providing AI computing power and enterprise-level generative AI applications. These strengths position Asus as one of the few all-encompassing providers of generative AI solutions.

Projected Surge in Server Business

Regarding server business performance, Asus envisions a yearly compounded growth rate of at least 40% until 2027, with a goal of achieving a fivefold growth over five years. In particular, the data center server business catering primarily to Cloud Service Providers (CSPs) anticipates a tenfold growth within the same timeframe, driven by the adoption of AI server products.

Asus CEO recently emphasized that Asus’s foray into AI server development was prompt and involved collaboration with NVIDIA from the outset. While the product lineup might be more streamlined compared to other OEM/ODM manufacturers, Asus had secured numerous GPU orders ahead of the AI server demand surge. The company is optimistic about the shipping momentum and order visibility for the new generation of AI servers in the latter half of the year.

Embracing NVIDIA’s Versatile L40S GPU

The NVIDIA L40S GPU, built on the Ada Lovelace architecture, stands out as one of the most powerful general-purpose GPUs in data centers. It offers groundbreaking multi-workload computations for large language model inference, training, graphics, and image processing. Not only does it facilitate rapid hardware solution deployment, but it also holds significance due to the current scarcity of higher-tier H100 and A100 GPUs, which have reached allocation stages. Consequently, businesses seeking to repurpose idle data centers are anticipated to shift their focus toward AI servers featuring the L40S GPU.

Asus’s newly introduced L40S GPU servers include the ESC8000-E11/ESC4000-E11 models with built-in Intel Xeon processors, as well as the ESC8000A-E12/ESC4000A-E12 models utilizing AMD EPYC processors. These servers can be configured with up to 4 or a maximum of 8 NVIDIA L40S GPUs. This configuration assists enterprises in enhancing training, fine-tuning, and inference workloads, facilitating AI model creation. It also establishes Asus’s platforms as the preferred choice for multi-modal generative AI applications.

(Source: https://www.chinatimes.com/newspapers/20230831000158-260202?chdtv)

According to a report from Taiwan’s TechNews, NVIDIA has delivered impressive results in its latest financial report, coupled with an optimistic outlook for its financial projections. This demonstrates that the demand for AI remains robust for the coming quarters. Currently, NVIDIA’s H100 and A100 chips both utilize TSMC’s CoWoS advanced packaging technology, making TSMC’s production capacity a crucial factor.

Examining the core GPU market, NVIDIA holds a dominant market share of 90%, while AMD accounts for about 10%. While other companies might adopt Google’s TPU or develop customized chips, they currently lack significant operational cost advantages.

In the short term, the shortage of CoWoS has led to tight chip supplies. However, according to a recent report by Morgan Stanley Securities, NVIDIA believes that TSMC’s CoWoS capacity won’t restrict shipments of the next quarter’s H100 GPUs. The company anticipates an increase in supply for each quarter next year. Simultaneously, TSMC is raising CoWoS prices by 20% for rush orders, indicating that the anticipated CoWoS bottleneck might alleviate.

According to industry sources, NVIDIA is actively diversifying its CoWoS supply chain away from TSMC. UMC, ASE, Amkor, and SPIL are significant players in this effort. Currently, UMC is expanding its interposer production capacity, aiming to double its capacity to relieve the tight CoWoS supply situation.

According to Morgan Stanley Securities, TSMC’s monthly CoWoS capacity this year is around 11,000 wafers, projected to reach 25,000 wafers by the end of next year. Non-TSMC CoWoS supply chain’s monthly capacity can reach 3,000 wafers, with a planned increase to 5,000 wafers by the end of next year.

(Photo credit: TSMC)

ChatGPT’s debut has sparked a thrilling spec upgrade in the server market, which has breathed new life into the supply chain and unlocked unparalleled business opportunities. Amidst all this, the big winners look set to be the suppliers of ABF (Ajinomoto Build-up Film) substrates, who are poised to reap enormous benefits.

In the previous article, “AI Sparks a Revolution Up In the Cloud,” we explored how the surge in data volumes is driving the spec of AI servers as well as the cost issue that comes with it. This time around, we’ll take a closer look at the crucial GPU and CPU platforms, focusing on how they can transform the ABF substrate market.

NVIDIA’s Dual-Track AI Server Chip Strategy Fuels ABF Consumption

In response to the vast data demands of fast-evolving AI servers, NVIDIA is leading the pack in defining the industry-standard specs.

This contrasts with standard GPU servers, where one CPU backs 2 to 6 GPUs. Instead, NVIDIA’s AI servers, geared towards DL(Deep Learning) and ML(Machine Learning), typically support 2 CPUs and 4 to 8 GPUs, thus doubling the ABF substrate usage compared to conventional GPU servers.

NVIDIA has devised a dual-track chip strategy, tailoring their offerings for international and Chinese markets. The primary chip for ChatGPT is NVIDIA’s A100. However, for China, in line with U.S. export regulations, they’ve introduced the A800 chip, reducing interconnect speeds from 600GBps (as on the A100) to 400GBps.

Their latest H100 GPU chip, manufactured at TSMC’s 4nm process, boasts an AI training performance 9 times greater than its A100 predecessor and inferencing power that’s 30 times higher. To match the new H100, H800 was also released with an interconnect speed capped at 300GBps. Notably, Baidu’s pioneering AI model, Wenxin, employs the A800 chip.

To stay competitive globally in AI, Chinese manufacturers are expected to aim for the computational prowess on par with the H100 and A100 by integrating more A800 and H800 chips. This move will boost the overall ABF substrate consumption.

With the ChatBot boom, it is predicted a 38.4% YoY increase in 2023’s AI server shipments and a robust CAGR of 22% from 2022 to 2026 – significantly outpacing the typical single-digit server growth, according to TrendForce’s prediction.

AMD, Intel Server Platforms Drive ABF Substrate Demand

Meanwhile, examining AMD and Intel’s high-end server platforms, we can observe how spec upgrades are propelling ABF substrate consumption forward.

• AMD Zen 4:

Since 2019, AMD’s EPYC Zen 2 server processors have used Chiplet multi-chip packaging, which due to its higher conductivity and cooling demands, has consistently bolstered ABF substrate demand.

• Intel Eagle Stream:

Intel’s advanced Eagle Stream Sapphire Rapids platform boasts 40-50% higher computation speed than its predecessor, the Whitley, and supports PCIe5, which triggers a 20% uptick in substrate layers. This platform employs Intel’s 2.5D EMIB tech and Silicon Bridge, integrating various chips to minimize signal transmission time.

The Sapphire Rapids lineup includes SPR XCC and the more advanced SPR HBM, with the latter’s ABF substrate area being 30% larger than the previous generation’s. The incorporation of EMIB’s Silicon Bridge within the ABF substrate increases lamination complexity and reduces overall yield. Simply put, for every 1% increase in Eagle Stream’s server market penetration, ABF substrate demand is projected to rise by 2%.

As the upgrades for server-grade ABF substrates continue to advance, production complexity, layer count, and area all increase correspondingly. This implies that the average yield rate might decrease from 60-70% to 40-50%. Therefore, the actual ABF substrate capacity required for future server CPU platforms will likely be more than double that of previous generations.

ABF Substrate Suppliers Riding the Tide

By our estimates, the global ABF substrate market size is set to grow from $9.3 billion in 2023 to $15 billion in 2026 – a CAGR of 17%, underscoring the tremendous growth and ongoing investment potential in the ABF supply chain.

Currently, Taiwanese and Japanese manufacturers cover about 80% of the global ABF substrate capacity. Major players like Japan’s Ibiden, Shinko and AT&S, along with Taiwan’s Unimicron, Nan Ya, and Kinsus all consider expanding their ABF substrate production capabilities as a long-term strategy.

As we analyzed in another piece, “Chiplet Design: A Real Game-Changer for Substrates,” despite the recent economic headwinds, capacity expansion of ABF substrate can still be seen as a solid trend, which is secured by the robust growth of high-end servers. Hence, the ability to precisely forecast capacity needs and simultaneously improve production yields will be the key to competitiveness for all substrate suppliers.

Read more:

• HBM Supply Leader SK Hynix’s Market Share to Exceed 50% in 2023 Due to Demand for AI Servers, Says TrendForce
• Shipments of AI Servers Will Climb at CAGR of 10.8% from 2022 to 2026, Says TrendForce

(Photo Credit: Google)