AI


2024-10-17

[News] Liquid Cooling Set for a New Era in 2025, Favoring Taiwanese Supply Chain

According to a report by Commercial Times, as NVIDIA’s GB200 is set to officially ship by the end of this year and gradually ramping up in 2025, it is anticipated to usher in a new era of liquid cooling.

TrendForce predicts that the liquid cooling penetration will rise from 11% in 2024 to 24% in 2025. This growth is expected to be fueled by advancements in AI, which we may witness in 2025.

The expansion of liquid cooling will benefit several Taiwanese companies, including Delta Electronics, Asia Vital Components, Auras Technology, LOTES, and Fositek.

The growing global awareness of ESG principles, coupled with cloud service providers (CSPs) accelerating the deployment of AI servers and the upcoming official launch of NVIDIA’s Blackwell platform all contribute to the growth of liquid cooling technology, as the report in Commercial Times indicated.

According to the report, the market is optimistic that the significant growth in NVIDIA Blackwell’s penetration will drive a shift from air cooling to liquid cooling solutions.

Currently, the Hopper architecture remains dominant, commanding 90% of the market share this year. On the other hand, Blackwell is projected to capture only 4% of the market by the end of 2024. However, as Blackwell ramps up sales next year, it is expected to replace Hopper and establish itself as the new mainstream.

From the perspective of thermal design power (TDP), NVIDIA’s GB200 NVL72 racks, with a TDP of approximately 140 kW, will require liquid cooling solutions to manage heat dissipation, with Liquid-to-Air (L2A) technology anticipated to become the predominant method.

While immersion cooling is an option, it faces technical challenges that may prevent it from becoming mainstream in the next 3 to 5 years, as the report stated.

The report highlighted that the market price of a single NVL72 rack is 28 times that of a traditional server. As a result, an increasing number of companies are entering and competing in the liquid cooling market.

Currently, for those Taiwanese companies in the supply chain which may benefit from the market opportunities, Asia Vital Components and Cooler Master lead in providing cold plates, while Cooler Master and Auras supply manifolds, and Vertiv and Delta Electronics provide coolant distribution units.

On the other hand, quick disconnect (QD) components are critical for preventing leakage and have frequently been reported as out of stock. Taiwanese companies like LOTES and Fositek are reportedly in the validation phase, and by the first half of 2025, they are expected to join the list of QD suppliers, to compete with major companies such as CPC, Parker Hannifin, Denfoss, and Staubli.

Read more

(Photo credit: NVIDIA)

Please note that this article cites information from Commercial Times.

 

2024-10-11

[News] Despite Geopolitical Tensions, China’s AI Startups Break into the U.S. with Products like Chatbots

In light of slow revenue growth in the Chinese market, China’s AI startups are seeking to enter the U.S. market in pursuit of overseas growth opportunities, following the successful model of the short video platform TikTok, according to a report by Commercial Times.

According to a report by the Financial Times, despite U.S. restrictions on chip exports to China, the country still maintains a competitive advantage in launching products like chatbots that do not require substantial computing resources.

The report highlights that Chinese AI companies, including MiniMax, TikTok’s parent company ByteDance, and 01.AI, have introduced AI products overseas, particularly targeting the U.S. market, which boasts a larger base of high-end consumer users.

These Chinese app companies have seen significant success. For instance, the report in Commercial Times points out that the majority of MiniMax’s sales stem from its chatbot application “Talkie,” which has gained immense popularity among American teenagers.

Shanghai-based MiniMax has made major breakthroughs in the past year, and the company plans to reach a sales target of USD 70 million in 2024, Commercial Times notes.

The Financial Times report also highlights that Chinese company ByteDance has launched several AI apps internationally. ByteDance’s photo editing application “Hypic,” along with Zuoyebang’s homework assistant “Question AI,” both made it into the top 20 downloads internationally.

However, according to the Financial Times report, Chinese companies still face certain challenges. Firstly, the opportunities for revenue growth are limited due to the high costs associated with training language models, which can negatively impact the potential of these companies. Additionally, to mitigate external risks, Chinese firms need to take measures such as placing servers outside of China to avoid the potential crisis of TikTok being banned in the United States.

Please note that this article cites information from Commercial Times and the Financial Times.

2024-09-13

[News] AMD MI325X Reported to Debut in October, Claiming AI Supercycle Has Just Started

According to a report from wccftech, AMD CEO Lisa Su has indicated that she believes the AI Supercycle has just started, and the company has accelerated its product development plans to meet the substantial market demand.

In addition to NVIDIA, AMD is a significant player in the AI market as well. This is not only due to its market impact but also because the company has significantly expanded its AI product portfolio over the past few quarters, attracting attention from major clients like Microsoft and Amazon.

While AMD has not yet replicated NVIDIA’s success in the market, the company remains optimistic about the future, which is why it believes the AI boom has only just begun.

A few months ago, AMD outlined its AI chip development roadmap for the next year. The “Advancing AI” event in this October will showcase the next-generation Instinct MI325X AI chip.

The flagship Instinct MI350 AI chip is scheduled for release in 2025, followed by the Instinct MI400 AI chip in 2026. Despite AMD’s advancements, there remains a generational gap, as competitor NVIDIA is poised to launch its Blackwell architecture AI chips in the coming months.

Moreover, per a report from Yahoo Finance, Su once stated that AMD could generate USD 4.5 billion in sales from the MI300 alone in 2024, a significant increase from around USD 100 million in AI-related chip revenue last year.

The company had previously projected MI300 sales at approximately USD 4 billion for this year. Su then added that, it’s the fastest-growing product in AMD’s history.

AMD recently announced that it will merge its consumer and data center architectures into a single unit known as “UDNA,” aiming to accelerate the development and optimization of both platforms.

This move is particularly noteworthy as AMD is focusing on competing with NVIDIA’s CUDA on the software front.

Read more

(Photo credit: AMD)

Please note that this article cites information from wccftech, Yahoo Finance and AMD.

2024-09-13

[News] Latest Development on AI Models of China’s Top Techs: Alibaba, Baidu, ByteDance, Huawei and Tencent

With AI giants in the Western world, such as OpenAI, Google and Meta, stealing the spotlight by their development of generative AI, some big names in China have introduced their AI models over the past 18 months, according to a latest report by CNBC.

Though trying to keep a low profile, five tech conglomerates in China, including Alibaba, Baidu, ByteDance, Huawei and Tencent, have launched their AI models lately, adding a new dimension to the competitive landscape of the ongoing tech rivalry between China and the U.S. Here is a brief introduction of the major Chinese AI models developed by the country’s leading tech firms, based on the information compiled by CNBC and other media reports.

Alibaba: Tongyi Qianwen

In November, 2023, a report by pandaily notes that Alibaba Cloud released its AI model, Tongyi Qianwen 1.0 a few months ago, while the 2.0 version was introduced later in the same year. Another report by South China Morning Post states that as of May, Alibaba reports that its Tongyi Qianwen AI models, often referred to as Qwen, are utilized by more than 90,000 corporate clients across China.

CNBC notes that in terms of Qwen, the company has developed various versions tailored to different functions. For instance, one model specializes in generating content or solving math problems, while another handles audio inputs and provides text-based responses.

It is worth noting that as some Qwen models are open-sourced, developers are allowed to download and use them under certain restrictions, according to CNBC.

Baidu: ERNIE

As one of China’s leading internet companies, Baidu was among the first to introduce generative AI applications in the country. A report by The Verge notes that the Ernie chatbot was available for download in August, 2023, after the approval by the Chinese government.

CNBC reports that Baidu intends to compete with OpenAI’s ChatGPT with Ernie Bot, as the company claims the bot to have 300 million users.

According to CNBC, ahead of the launch of its “Turbo” version, which took place in late June, Baidu stated that its Ernie 4.0 offers capabilities comparable to OpenAI’s GPT-4. According to Baidu, this foundational model has advanced understanding and reasoning abilities.

Similar to other companies, Baidu is offering access to its AI model through its cloud computing services, CNBC says.

ByteDance: Doubao

TikTok parent company ByteDance, though entered the AI race later than competitors like Baidu and Alibaba, has surprised the market with its low-cost Doubao model, which was launched in May, 2024.

According to a report by technode, the model can process 2 million Chinese characters, equivalent to 1.25 million tokens, for just RMB 1 (USD 0.14). In comparison, OpenAI’s latest multimodal model, GPT-4o, costs USD 5 per million input tokens.

CNBC notes that Doubao has various capabilities, including voice generation and coding support for developers.

Huawei: Pangu

Introduced by Huawei in 2021 as the world’s largest pre-trained Chinese large language models (LLMs) with over 100 billion parameters, the Pangu models are now entering their fourth iteration, according to Counterpoint. In May, 2024, the latest Pangu models are said to boast 230 billion parameters.

Interesting enough, Huawei has adopted a different strategy from its competitors with its Pangu AI models, CNBC remarks. The tech giant focuses on developing industry-specific models tailored to sectors like government, finance, manufacturing, mining, and meteorology.

For instance, Huawei claims that its Pangu Meteorology Model can predict a typhoon’s trajectory 10 days in advance in just 10 seconds, a task that previously took four to five hours, according to CNBC.

Tencent: Hunyuan

Last year, Tencent introduced its foundational model, Hunyuan, which is accessible through Tencent’s cloud computing services.

According to CNBC, Tencent has highlighted Hunyuan’s strong Chinese language processing abilities and advanced logical reasoning, supporting features like image generation and text recognition. The model is designed for use across industries such as gaming, social media, and e-commerce.

As the operator of China’s largest messaging app, WeChat, Tencent launched an AI chatbot this year based on the Hunyuan model. The AI assistant, named Yuanbao, can access information and content from WeChat, setting it apart from competitors, CNBC notes.

Notably, China’s large language models, just like its rivals in the West, rely on the strong computing power of AI chips. A previous report by Reuters in November, 2023, states that Tencent is said to have stockpiled a substantial reserve of AI chips from NVIDIA, as the company prepares in advance to train its Hunyuan AI models for the following generations.

How far will the tech giants in China be able to push the boundaries of AI models? The answer may lie in the development of the country’s domestic chips, as the U.S. authority already banned the export to China of AI chips.

Read more

(Photo credit: Baidu)

Please note that this article cites information from CNBC, pandaily, South China Morning Post, tech nodeThe VergeCounterpoint and Reuters.
2024-09-10

[News] NVIDIA’s Blackwell Overcomes Delays, as GB200 Reportedly Sets for December Mass Production

According to a report from Commercial Times citing sources, it’s revealed that NVIDIA has executed changes to the Blackwell series’ 6-layer GPU mask. Therefore, the process can now proceed without re-taping out, as production delays being minimized.

The report noted that NVIDIA’s updated version of B200 is expected to be completed by late October, allowing the GB200 to enter mass production in December, with large-scale deliveries to ODMs expected in the first quarter of next year.

Previously, as per a report from The Information, NVIDIA’s GB200 was said to be experiencing a one-quarter delay in mass shipments. Another report from the Economic Daily News further suggested that the problem likely lies in the yield rates of advanced packaging, which mainly affected the non-reference-designed GB200 chips.

Industry sources cited by Commercial Times addressed that NVIDIA’s Blackwell chip used to be facing instability in metal layers during the HV process, which was then resolved by July.

In addition, since the issue reportedly occurred in the back-end-of-line process, a new tape-out was deemed unnecessary. Still, as CoWoS-L capacity remains a bottleneck, the advanced packaging for GB200 this year is expected to adopt  CoWoS-S.

Read more

(Photo credit: NVIDIA)

Please note that this article cites information from Commercial Times, The Information and Economic Daily News.

  • Page 1
  • 17 page(s)
  • 82 result(s)

Get in touch with us