With the skyrocketing demand for AI, cloud service providers (CSPs) are hastening the development of in-house chips. Apple, making a surprising move, is actively developing a data center-grade chip codenamed “Project ACDC,” signaling its foray into the realm of AI accelerators for servers.

As per a report from global media The Wall Street Journal, Apple is developing an AI accelerator chip for data center servers under the project name “Project ACDC.” Sources familiar with the matter revealed that Apple is closely collaborating with TSMC, but the timing of the new chip’s release remains uncertain.

Industry sources cited by the same report from Commercial Times disclosed that Apple’s AI accelerator chip will be developed using TSMC’s 3-nanometer process. Servers equipped with this chip are expected to debut next year, further enhancing the performance of its data centers and future cloud-based AI tools.

Industry sources cited in Commercial Times‘ report reveal that cloud service providers (CSPs) frequently choose TSMC’s 5 and 7-nanometer processes for their in-house chip development, capitalizing on TSMC’s mature advanced processes to enhance profit margins. Additionally, the same report also highlights that major industry players including Microsoft, AWS, Google, Meta, and Apple rely on TSMC’s advanced processes and packaging, which significantly contributes to the company’s performance.

Apple has consistently been an early adopter of TSMC’s most advanced processes, relying on their stability and technological leadership. Apple’s adoption of the 3-nanometer process and CoWoS advanced packaging next year is deemed the most reasonable solution, which will also help boost TSMC’s 3-nanometer production capacity utilization.

Generative AI models are rapidly evolving, enabling businesses and developers to address complex problems and discover new opportunities. However, large-scale models with billions or even trillions of parameters pose more stringent requirements for training, tuning, and inference.

Per Commercial Times citing industry sources, it has noted that Apple’s entry into the in-house chip arena comes as no surprise, given that giants like Google and Microsoft have long been deploying in-house chips and have successively launched iterative products.

In April, Google unveiled its next-generation AI accelerator, TPU v5p, aimed at accelerating cloud-based tasks and enhancing the efficiency of online services such as search, YouTube, Gmail, Google Maps, and Google Play Store. It also aims to improve execution efficiency by integrating cloud computing with Android devices, thereby enhancing user experience.

At the end of last year, AWS introduced two in-house chips, Graviton4 and Trainium2, to strengthen energy efficiency and computational performance to meet various innovative applications of generative AI.

Microsoft also introduced the Maia chip, designed for processing OpenAI models, Bing, GitHub Copilot, ChatGPT, and other AI services.

Meta, on the other hand, completed its second-generation in-house chip, MTIA, designed for tasks related to AI recommendation systems, such as content ranking and recommendations on Facebook and Instagram.

Read more

• [News] Breaking Away from NVIDIA Dependency, Microsoft Reportedly Developing In-House AI Server High-Speed Network Card
• [Insights] AI Market: A Battleground for Tech Giants as Six Major Companies Develop AI Chips

(Photo credit: Apple)

Four major cloud service providers (CSPs) including Google, Microsoft, Amazon, and Meta, sequentially released their first-quarter financial performance for the year 2024 (January 2024 to March 2024) at the end of April.

Each company has achieved double-digit growth of the revenue, with increased capital expenditures continuing to emphasize AI as their main development focus. The market’s current focus remains on whether AI investment projects can successfully translate into revenue from the previous quarter to date.

TrendForce’s Insights:

1. Strong Financial Performance of Top Four CSPs Driven by AI and Cloud Businesses

Alphabet, the parent company of Google, reported stellar financial results for the first quarter of 2024. Bolstered by growth in search engine, YouTube, and cloud services, revenue surpassed USD 80 billion, marking a 57% increase in profit. The company also announced its first-ever dividend payout, further boosting its stock price as all metrics exceeded market expectations, pushing its market capitalization past USD 2 trillion for the first time.For Google, the current development strategy revolves around its in-house LLM Gemini layout, aimed at strengthening its cloud services, search interaction interfaces, and dedicated hardware development.

Microsoft’s financial performance is equally impressive. This quarter, its revenue reached USD 61.9 billion, marking a year-on-year increase of 17%. Among its business segments, the Intelligent Cloud sector saw the highest growth, with a 21% increase in revenue, totaling $26.7 billion. Notably, the Azure division experienced a remarkable 31% growth, with Microsoft attributing 7% of this growth to AI demand.

In other words, the impact of AI on its performance is even more pronounced than in the previous quarter, prompting Microsoft to focus its future strategies more on the anticipated benefits from Copilot, both in software and hardware.

This quarter, Amazon achieved a remarkable revenue milestone, surpassing USD 140 billion, representing a year-on-year increase of 17%, surpassing market expectations. Furthermore, its profit reached USD 10.4 billion, far exceeding the USD 3.2 billion profit recorded in the same period in 2023.

The double-digit growth in advertising business and AWS (Amazon Web Services) drove this performance, with the latter being particularly highlighted for its AI-related opportunities. AWS achieved a record-high operating profit margin of 37.6% this quarter, with annual revenue expected to exceed $100 billion, and short-term plans to invest USD 150 billion in expanding data centers.

On the other hand, Meta reported revenue of USD 36.46 billion this quarter, marking a significant year-on-year growth of 27%, the largest growth rate since 2021. Profit also doubled compared to the same period in 2023, reaching USD 12.37 billion.

Meta’s current strategy focuses on allocating resources to areas such as smart glasses and mixed reality (MR) in the short and medium term. The company continues to leverage AI to enhance the user value of the virtual world.

2. Increased Capital Expenditure to Develop AI is a Common Consensus, Yet Profitability Remains Under Market Scrutiny

Observing the financial reports of major cloud players, the increase in capital expenditure to solidify their commitment to AI development can be seen as a continuation of last quarter’s focus.

In the first quarter of 2024, Microsoft’s capital expenditure surged by nearly 80% compared to the same period in 2023, reaching USD 14 billion. Google expects its quarterly expenditure to remain above USD 12 billion. Similarly, Meta has raised its capital expenditure guidance for 2024 to the range of USD 35 to USD 40 billion.

Amazon, considering its USD 14 billion expenditure in the first quarter as the minimum for the year, anticipates a significant increase in capital expenditure over the next year, exceeding the USD 48.4 billion spent in 2023. However, how these increased investments in AI will translate into profitability remains a subject of market scrutiny.

While the major cloud players remain steadfast in their focus on AI, market expectations may have shifted. For instance, despite impressive financial reports last quarter, both Google and Microsoft saw declines in their stock prices, unlike the significant increases seen this time. This could partly be interpreted as an expectation of short- to medium-term AI investment returns from products and services like Gemini and Copilot.

In contrast, Meta, whose financial performance is similarly impressive to other cloud giants, experienced a post-earnings stock drop of over 15%. This may be attributed partly to its conservative financial outlook and partly to the less-than-ideal investment returns from its focused areas of virtual wearable devices and AI value-added services.

Due to Meta’s relatively limited user base compared to the other three CSPs in terms of commercial end-user applications, its AI development efforts, such as the practical Llama 3 and the value-added Meta AI virtual assistant for its products, have not yielded significant benefits. While Llama 3 is free and open-source, and Meta AI has limited shipment, they evidently do not justify the development costs.

Therefore, Meta still needs to expand its ecosystem to facilitate the promotion of its AI services, aiming to create a business model that can translate technology into tangible revenue streams.

For example, Meta recently opened up the operating system Horizon OS of its VR device Quest to brands like Lenovo and Asus, allowing them to produce their own branded VR/MR devices. The primary goal is to attract developers to enrich the content database and thereby promote industry development.

Read more

• [News] NVIDIA’s H100 Chip Heat Declines, CoWoS Capacity Eases
• [News] Microsoft Reportedly Stockpiling AI Chips to Boost Hardware Capability

Meta Platform, parent company of Facebook, has announced its latest generation AI chip of its Training and Inference Accelerator (MTIA) on April 10th, fabricated using TSMC’s 5nm process. According to a report from Commercial Times, this move is expected to reduce Meta’s reliance on NVIDIA’s chips and enhance computational power for AI services.

In its shift towards AI services, Meta requires greater computational capabilities. Thus, last year, Meta introduced its AI models to compete with OpenAI’s ChatGPT. The latest AI chip, Artemis, is an upgraded version of MTIA introduced last year, assisting platforms like Facebook and Instagram with content ranking and recommendations.

Meta’s new generation AI chip will be produced by TSMC using the 5nm process. Meta reveals that Artemis offers triple the performance of the first-generation MTIA.

In October last year, Meta announced plans to invest USD 35 billion to establish infrastructure supporting AI, including data centers and hardware. CEO Mark Zuckerberg told investors, “In terms of investment priorities, AI will be our biggest investment area in 2024 for both engineering and compute resources.”

Meta’s proprietary AI chips are deployed in data centers to power AI applications. Meta has several ongoing projects aimed at expanding MTIA’s application scope, including supporting generative AI workloads.

The trend of tech giants developing their own AI chips is evident, with Meta joining competitors like Amazon, Microsoft, and Google in internal AI chip development to reduce reliance on NVIDIA. Google recently unveiled its latest data center AI chip, TPU v5p, on the 9th. Meanwhile, Intel is targeting NVIDIA’s H100 with its new AI chip, Gaudi 3.

Read more

• [News] NVIDIA’s H100 AI Chip No Longer Out of Reach, Inventory Pressure Reportedly Forces Customers to Resell

(Photo credit: Meta)

In 2023, “generative AI” was undeniably the hottest term in the tech industry.

The launch of the generative application ChatGPT by OpenAI has sparked a frenzy in the market, prompting various tech giants to join the race.

As per a report from TechNews, currently, NVIDIA dominates the market by providing AI accelerators, but this has led to a shortage of their AI accelerators in the market. Even OpenAI intends to develop its own chips to avoid being constrained by tight supply chains.

On the other hand, due to restrictions arising from the US-China tech war, while NVIDIA has offered reduced versions of its products to Chinese clients, recent reports suggest that these reduced versions are not favored by Chinese customers.

Instead, Chinese firms are turning to Huawei for assistance or simultaneously developing their own chips, expected to keep pace with the continued advancement of large-scale language models.

In the current wave of AI development, NVIDIA undoubtedly stands as the frontrunner in AI computing power. Its A100/H100 series chips have secured orders from top clients worldwide in the AI market.

As per analyst Stacy Rasgon from the Wall Street investment bank Bernstein Research, the cost of each query using ChatGPT is approximately USD 0.04. If ChatGPT queries were to scale to one-tenth of Google’s search volume, the initial deployment would require approximately USD 48.1 billion worth of GPUs for computation, with an annual requirement of about USD 16 billion worth of chips to sustain operations, along with a similar amount for related chips to execute tasks.

Therefore, whether to reduce costs, decrease overreliance on NVIDIA, or even enhance bargaining power further, global tech giants have initiated plans to develop their own AI accelerators.

Per reports by technology media The Information, citing industry sources, six global tech giants, including Microsoft, OpenAI, Tesla, Google, Amazon, and Meta, are all investing in developing their own AI accelerator chips. These companies are expected to compete with NVIDIA’s flagship H100 AI accelerator chips.

Progress of Global Companies’ In-house Chip Development

• Microsoft

Rumors surrounding Microsoft’s in-house AI chip development have never ceased.

At the annual Microsoft Ignite 2023 conference, the company finally unveiled the Azure Maia 100 AI chip for data centers and the Azure Cobalt 100 cloud computing processor. In fact, rumors of Microsoft developing an AI-specific chip have been circulating since 2019, aimed at powering large language models.

The Azure Maia 100, introduced at the conference, is an AI accelerator chip designed for tasks such as running OpenAI models, ChatGPT, Bing, GitHub Copilot, and other AI workloads.

According to Microsoft, the Azure Maia 100 is the first-generation product in the series, manufactured using a 5-nanometer process. The Azure Cobalt is an Arm-based cloud computing processor equipped with 128 computing cores, offering a 40% performance improvement compared to several generations of Azure Arm chips. It provides support for services such as Microsoft Teams and Azure SQL. Both chips are produced by TSMC, and Microsoft is already designing the second generation.

• Open AI

OpenAI is also exploring the production of in-house AI accelerator chips and has begun evaluating potential acquisition targets. According to earlier reports from Reuters citing industry sources, OpenAI has been discussing various solutions to address the shortage of AI chips since at least 2022.

Although OpenAI has not made a final decision, options to address the shortage of AI chips include developing their own AI chips or further collaborating with chip manufacturers like NVIDIA.

OpenAI has not provided an official comment on this matter at the moment.

• Tesla

Electric car manufacturer Tesla is also actively involved in the development of AI accelerator chips. Tesla primarily focuses on the demand for autonomous driving and has introduced two AI chips to date: the Full Self-Driving (FSD) chip and the Dojo D1 chip.

The FSD chip is used in Tesla vehicles’ autonomous driving systems, while the Dojo D1 chip is employed in Tesla’s supercomputers. It serves as a general-purpose CPU, constructing AI training chips to power the Dojo system.

• Google

Google began secretly developing a chip focused on AI machine learning algorithms as early as 2013 and deployed it in its internal cloud computing data centers to replace NVIDIA’s GPUs.

The custom chip, called the Tensor Processing Unit (TPU), was unveiled in 2016. It is designed to execute large-scale matrix operations for deep learning models used in natural language processing, computer vision, and recommendation systems.

In fact, Google had already constructed the TPU v4 AI chip in its data centers by 2020. However, it wasn’t until April 2023 that technical details of the chip were publicly disclosed.

• Amazon

As for Amazon Web Services (AWS), the cloud computing service provider under Amazon, it has been a pioneer in developing its own chips since the introduction of the Nitro1 chip in 2013. AWS has since developed three product lines of in-house chips, including network chips, server chips, and AI machine learning chips.

Among them, AWS’s lineup of self-developed AI chips includes the inference chip Inferentia and the training chip Trainium.

On the other hand, AWS unveiled the Inferentia 2 (Inf2) in early 2023, specifically designed for artificial intelligence. It triples computational performance while increasing accelerator total memory by a quarter.

It supports distributed inference through direct ultra-high-speed connections between chips and can handle up to 175 billion parameters, making it the most powerful in-house manufacturer in today’s AI chip market.

• Meta

Meanwhile, Meta, until 2022, continued using CPUs and custom-designed chipsets tailored for accelerating AI algorithms to execute its AI tasks.

However, due to the inefficiency of CPUs compared to GPUs in executing AI tasks, Meta scrapped its plans for a large-scale rollout of custom-designed chips in 2022. Instead, it opted to purchase NVIDIA GPUs worth billions of dollars.

Still, amidst the surge of other major players developing in-house AI accelerator chips, Meta has also ventured into internal chip development.

On May 19, 2023, Meta further unveiled its AI training and inference chip project. The chip boasts a power consumption of only 25 watts, which is 1/20th of the power consumption of comparable products from NVIDIA. It utilizes the RISC-V open-source architecture. According to market reports, the chip will also be produced using TSMC’s 7-nanometer manufacturing process.

China’s Progress on In-House Chip Development

China’s journey in developing in-house chips presents a different picture. In October last year, the United States expanded its ban on selling AI chips to China.

Although NVIDIA promptly tailored new chips for the Chinese market to comply with US export regulations, recent reports suggest that major Chinese cloud computing clients such as Alibaba and Tencent are less inclined to purchase the downgraded H20 chips. Instead, they have begun shifting their orders to domestic suppliers, including Huawei.

This shift in strategy indicates a growing reliance on domestically developed chips from Chinese companies by transferring some orders for advanced semiconductors to China.

TrendForce indicates that currently about 80% of high-end AI chips purchased by Chinese cloud operators are from NVIDIA, but this figure may decrease to 50% to 60% over the next five years.

If the United States continues to strengthen chip controls in the future, it could potentially exert additional pressure on NVIDIA’s sales in China.

Read more

• [News] NVIDIA’s Exclusive Chips for China Now Reported to be Available for Pre-Order, Priced Similar to Huawei Products

(Photo credit: NVIDIA)

MediaTek announced a collaboration with Meta to develop its next-generation smart glasses chip. Since Meta has previously used Qualcomm chips for its two generations of smart glasses products, it is speculated that Meta’s expansion of chip suppliers is aimed at maintaining supply chain flexibility and reducing costs. MediaTek, in turn, is poised to leverage smart glasses to tap into opportunities within Meta’s VR/AR devices.

 TrendForce’s Insights:

1. Meta Expands Chip Collaboration Suppliers, Maintaining Product Development Flexibility and Potential Cost Reduction

In mid-November 2023, MediaTek hosted the overseas summit, Mediatek Executive Summit 2023, where it announced a collaboration with Meta to develop the next-generation smart glasses chip.

Meta’s first smart glasses, a collaborative creation with Ray-Ban in 2021, differ from the Quest series as they are not high-end VR devices but rather feature a simpler design, focusing on additional functionalities like music playback and phone calls.

In the fall of 2023, Meta introduced a successor product with significant improvements in camera resolution, video quality, microphones, and internal storage. This new device is designed to simplify the recording and live streaming process by integrating with Meta’s social platform. Additionally, the new product aligns with the trend of generative AI and incorporates Meta’s AI voice assistant based on Llama2 LLM.

Notably, the market has shown keen interest and discussion regarding MediaTek’s announcement on the collaboration with Meta, given that Meta’s previous two generations of smart glasses used Qualcomm chips, specifically the Qualcomm Snapdragon Wear 4100 for the older version and the AR1 Gen1 for the new version.

Analysis of Meta’s Motivation: Meta’s decision to collaborate with MediaTek may be driven by considerations of risk diversification among suppliers and overall cost reduction.

Firstly, Meta has been investing in the development of in-house chips in recent years to ensure flexibility in product development. Examples include the MTIA chip, disclosed in mid-2023, designed for processing inference-related tasks, and the MSVP, the first in-house ASIC chip for video transcoding, which is expected to be used in VR and AR devices.

Given Meta’s previous attempts, including collaboration with Samsung, to independently develop chips and move towards chip autonomy, the partnership with MediaTek can be seen as a risk mitigation strategy against vendor lock-in.

Secondly, considering that smart glasses, unlike the high-priced Quest series, are currently priced at USD 299 for both models, MediaTek’s competitive pricing may also be a significant factor in Meta’s decision to collaborate with them.

2. MediaTek Eyes VR and AR Device Market Opportunities Through Smart Glasses Collaboration with Meta

From MediaTek’s perspective, their focus extends beyond smart glasses to the vast business opportunities presented by Meta’s VR and AR devices. In reality, examining Meta’s smart glasses alone reveals estimated shipments of around 300,000 pairs for the older model. Even with the new model and the anticipated successor expected to launch in 2025, there is currently no clear indication of significant market momentum.

In practical terms, this collaboration with Meta might not contribute substantially to MediaTek’s revenue. The crucial aspect of MediaTek’s collaboration with Meta lies in strategically positioning itself in Meta’s smart headwear supply chain, challenging the dominance of the original chip supplier, Qualcomm.

Looking at global VR device shipments, Meta is projected to hold over 70% market share in 2023 and 2024. There are also reports of an updated version of the Quest device expected to be available in China in late 2024. If MediaTek can expand its collaboration with Meta further, coupled with the gradual increase in the penetration rate of VR and AR devices, significant business opportunities still lie ahead.

From an overall perspective of the VR and AR industry, the current design of headwear devices no longer resembles the early models that required external computing cores due to considerations of cost, power, and heat.

The prevalent mainstream designs are now standalone devices. Given that these devices not only execute the primary application functions but also handle and consolidate a substantial amount of data from sensors to support functions like object tracking and image recognition, VR and AR devices require high-performance chips or embedded auxiliary SoCs. This market demand and profit potential are compelling enough to attract chip manufacturers, especially in the face of the gradual decline in momentum in the consumer electronics market, such as smartphones.

The VR and AR market still holds development potential, making it a strategic entry point for manufacturers. This insight is evident in MediaTek’s motivation, continuing its market cultivation efforts after developing the first VR chip for Sony PS VR2 in 2022 and collaborating with Meta.