US-based CSPs have been establishing SMT production lines in Southeast Asia since late 2022 to mitigate geopolitical risks and supply chain disruptions. TrendForce reports that Taiwan-based server ODMs, including Quanta, Foxconn, Wistron (including Wiwynn), and Inventec, have set up production bases in countries like Thailand, Vietnam, and Malaysia. It’s projected that by 2023, the production capacity from these regions will account for 23%, and by 2026, it will approach 50%.

TrendForce reveals that Quanta, due to its geographical ties, has established several production lines in its Thai facilities centered around Google and Celestica, aiming for optimal positioning to foster customer loyalty. Meanwhile, Foxconn has renovated its existing facilities in Hanoi, Vietnam, and uses its Wisconsin plant to accommodate customer needs. Both Wistron and Wiwynn are progressively establishing assembly plants and SMT production lines in Malaysia. Inventec’s current strategy mirrors that of Quanta, with plans to build SMT production lines in Thailand by 2024 and commence server production in late 2024.

CSPs aim to control the core supply chain, AI server supply chain trends toward decentralization

TrendForce suggests that changes in the supply chain aren’t just about circumventing geopolitical risks—equally vital is increased control over key high-cost components, including CPUs, GPUs, and other critical materials. With rising demand for next-generation AI and Large Language Models, supply chain stockpiling grows each quarter. Accompanied by a surge in demand in 1H23, CSPs will become especially cautious in their supply chain management.

Google, with its in-house developed TPU machines, possesses both the core R&D and supply chain leadership. Moreover, its production stronghold primarily revolves around its own manufacturing sites in Thailand. However, Google still relies on cooperative ODMs for human resource allocation and production scheduling, while managing other materials internally. To avoid disruptions in the supply chain, companies like Microsoft, Meta, and AWS are not only aiming for flexibility in supply chain management but are also integrating system integrators into ODM production. This approach allows for more dispersed and meticulous coordination and execution of projects.

Initially, Meta heavily relied on direct purchases of complete server systems, with Intel’s Habana system being one of the first to be integrated into Meta’s infrastructure. This made sense since the CPU for their web-type servers were often semi-custom versions from Intel. Based on system optimization levels, Meta found Habana to be the most direct and seamless solution. Notably, it was only last year that Meta began to delegate parts of its Metaverse project to ODMs. This year, as part of its push into generative AI, Meta has also started adopting NVIDIA’s solutions extensively.

Dell, a major server brand, placed a substantial order for AI servers just before NVIDIA’s Q2 financial report. This move is reshaping Taiwan’s supply chain dynamics, favoring companies like Wistron and Lite-On.

Dell is aggressively entering the AI server market, ordering NVIDIA’s top-tier H100 chips and components. The order’s value this year is estimated in hundreds of billions of Taiwanese dollars, projected to double in the next year. Wistron and Lite-On are poised to benefit, securing vital assembly and power supply orders. EMC and Chenbro are also joining the supply chain.

Dell’s AI server order, which includes assembly (including complete machines, motherboards, GPU boards, etc.) and power supply components, stands out with its staggering value. The competition was most intense in the assembly sector, ultimately won by Wistron. In the power supply domain, industry leaders like Delta, Lite-On, secured a notable share, with Lite-On emerging as a winner, sparking significant industry discussions.

According to Dell’s supply chain data, AI server inventory will reach 20,000 units this year and increase next year. The inventory primarily features the highest-end H100 chips from NVIDIA, with a few units integrating the A100 chips. With each H100 unit priced at $300,000 and A100 units exceeding $100,000, even with a seemingly modest 20,000 units, the total value remains in the billions of New Taiwan Dollars.

Wistron is a standout winner in Dell’s AI server assembly order, including complete machines, motherboards, and GPU boards. Wistron has existing H100 server orders and will supply new B100 baseboard orders. Their AI server baseboard plant in Hsinchu, Taiwan will expand by Q3 this year. Wistron anticipates year-round growth in the AI server business.

In just a short span of six months, AI has evolved from generating text, images, music, and code to automating tasks and producing agents, showcasing astonishing capabilities. TrendForce has issued a new report titled “Surge in AIGC Applications to Drive Long-Term Demand for AI Servers.” Beyond highlighting the latest developments in AI, the report also delves into strategies adopted by governments and industries to ensure the positive trajectory of AI’s development. It analyzes the projected timeline for the widespread implementation of AIGC applications and their impact on the demand for AI servers.

• AIGC Application Wave Expected to Arrive by 2025 Once Rules are Set

While the AIGC application frenzy in the first half of 2023 has raised concerns, it has also prompted governments and industries to actively address potential risks and issues stemming from AIGC applications, along with devising corresponding solutions. Currently, both the government and industries have strategies in place to regulate AIGC applications in terms of legal oversight, privacy protection, identity establishment, reliability enhancement, security augmentation, and copyright maintenance.

Considering the time required for governments to draft legislation and industries to enhance AI’s reliability, security, and copyright protection, it is estimated that the rules of the AIGC application will gradually solidify by late 2024 to early 2025, paving the way for the AIGC application surge around 2025.

Beyond the five major categories of AIGC applications—text generation, image generation, music generation, video generation, and code generation—AIGC technology-based applications like AI customer service, personalized AI assistants, AI search, and AI productivity tools are also gaining prominence. In the realm of gaming, whether in VR or open-world games, AIGC technology is set to significantly enhance immersion and freedom, ushering in revolutionary experiences.

• Long-Term Demand for AI Servers to Benefit Server Brands and ODMs

To secure a dominant position in the AI technology industry and embrace the upcoming AIGC application wave, application service providers, tech giants, national institutions, and startups are competing to bolster their AI computing resources. As core computing components experience increased shipments, the shipment volume of AI servers, which serve as foundational computing units, is also expected to surge.

In the proactive year of 2023, where institutions and enterprises are aggressively building computing resources, the AI server shipment volume is projected to grow substantially. Given the limited upstream semiconductor capacity, this momentum is likely to extend into 2024.

By 2025, propelled by the AIGC application frenzy, AI server shipments are poised for further stimulation. Consequently, due to institutions and businesses preemptively establishing computing resources and the projected timeline for large-scale AIGC application implementation, the AI server market is anticipated to witness a sustained demand surge. Given the intricate manufacturing of AI servers and their higher degree of customization, their profitability exceeds that of general servers. With the continual growth in AI server shipments, relevant server brands and ODM manufacturers are poised to reap significant benefits.

According to a report by Taiwan’s Commercial Times, JPMorgan’s latest analysis reveals that AI demand will remain robust in the second half of the year. Encouragingly, TSMC’s CoWoS capacity expansion progress is set to exceed expectations, with production capacity projected to reach 28,000 to 30,000 wafers per month by the end of next year.

The trajectory of CoWoS capacity expansion is anticipated to accelerate notably in the latter half of 2024. This trend isn’t limited to TSMC alone; other players outside the TSMC are also actively expanding their CoWoS-like production capabilities to meet the soaring demands of AI applications.

Gokul Hariharan, Head of Research for JPMorgan Taiwan, highlighted that industry surveys indicate strong and unabated AI demand in the latter half of the year. Shortages amounting to 20% to 30% are observed with CoWoS capacity being a key bottleneck and high-bandwidth memory (HBM) also facing supply shortages.

JPMorgan’s estimates indicate that Nvidia will account for 60% of the overall CoWoS demand in 2023. TSMC is expected to produce around 1.8 to 1.9 million sets of H100 chips, followed by significant demand from Broadcom, AWS’ Inferentia chips, and Xilinx. Looking ahead to 2024, TSMC’s continuous capacity expansion is projected to supply Nvidia with approximately 4.1 to 4.2 million sets of H100 chips.

Apart from TSMC’s proactive expansion of CoWoS capacity, Hariharan predicts that other assembly and test facilities are also accelerating their expansion of CoWoS-like capacities.

For instance, UMC is preparing to have a monthly capacity of 5,000 to 6,000 wafers for the interposer layer by the latter half of 2024. Amkor is expected to provide a certain capacity for chip-on-wafer stacking technology, and ASE Group will offer chip-on-substrate bonding capacity. However, these additional capacities might face challenges in ramping up production for the latest products like H100, potentially focusing more on older-generation products like A100 and A800.

(Photo credit: TSMC)

According to a report by Taiwan’s Economic Daily, Foxconn Group has achieved another triumph in its AI endeavors. The company has secured orders for over 50% of NVIDIA’s HGX GPU base boards, marking the first instance of such an achievement. Adding to this success, Foxconn had previously acquired an order for another NVIDIA DGX GPU base board, solidifying its pivotal role in NVIDIA’s two most crucial AI chip base board orders.

The report highlights that in terms of supply chain source, Foxconn Group stands as the exclusive provider of NVIDIA’s AI chip modules (GPU Modules). As for NVIDIA’s AI motherboards, the suppliers encompass Foxconn, Quanta, Inventec, and Super Micro.

Industry experts analyze that DGX and HGX are currently NVIDIA’s two most essential AI servers, and Foxconn Group has undertaken the monumental task of fulfilling the large order for NVIDIA’s AI chipboards through its subsidiary, Foxconn Industrial Internet (FII). Having previously secured orders for NVIDIA’s DGX base boards, Foxconn Group has now garnered additional orders from FII for the HGX base boards. This expanded supply constitutes more than half of the total, solidifying Foxconn Group’s role as a primary supplier for NVIDIA’s two critical AI chip base board orders.

Furthermore, Foxconn’s involvement doesn’t end with AI chip modules, base boards, and motherboards. The company’s engagement extends downstream to servers and server cabinets, creating a vertically integrated approach that covers the entire AI ecosystem.

(Photo credit: Nvidia)