Industry shifts toward heterogeneous computing as GPU dominance faces challenges
As AI tasks become more specialized, the market is shifting from a reliance on NVIDIA GPUs to "heterogeneous computing," where specific chips are selected for different stages of AI processing.
The AI industry is transitioning toward heterogeneous computing as specialized chips replace single-GPU setups for granular tasks. While NVIDIA remains dominant, companies like Google are expanding their influence by supplying TPUs to Oracle starting in 2027 to meet specific inference requirements.
The dominance of NVIDIA GPUs, which have been at the center of artificial intelligence (AI) computation, is facing cracks. As AI tasks become more granular, the industry is transitioning from an era where all computations were handled by a single GPU to an era of "heterogeneous computing," where chips are selected based on the specific nature of the task.
According to an analysis by T Times TV, the recent decision by MS to phasedly terminate server rental contracts with Oracle because Oracle's NVIDIA GPU specifications did not meet its requirements symbolically demonstrates this market shift. Customers are no longer simply purchasing "NVIDIA chips" but are beginning to demand specifications optimized for specific tasks.
AI Computation Divergence: Using "Motorcycles" Instead of "Trucks"
The core of this change lies in the differentiation of AI tasks. AI roles are divided into "training," where models are created, and "inference," where the created model provides answers. While the early market was a battle for training massive models, the emergence of AI agents is causing the proportion of inference computation to grow to be as large as, or even larger than, training.
In particular, the inference stage is divided into the "pre-fill" stage, which involves understanding the question, and the "decode" stage, which involves generating the answer. While high computational power is crucial during the question-understanding stage, the ability to quickly retrieve data from memory is the key factor during the answer-generation stage. Accordingly, AWS is dividing roles by using its proprietary Trainium chip for question understanding and Cerebras chips for answer generation. This is similar to using large trucks to move heavy cargo but deploying motorcycles for urgent deliveries.
Google has been the biggest beneficiary of this process. Google is expanding its influence in the cloud market by utilizing the TPU (Tensor Processing Unit) it developed for its own services. In particular, as Oracle has decided to equip its cloud with Google's TPU starting in 2027, evaluations suggest that Google is transforming from a mere service company into a powerful chip supplier.
Opportunities and Crises for NPU Startups: The Giant Wall of Google
This trend is a double-edged sword for startups designing Neural Processing Units (NPUs). NPUs are like "specialty chefs" that are optimized for specific AI computations and can process tasks more cheaply with less power than GPUs. While the fact that non-GPU chips are proving successful in the market is a positive sign, the fact that big tech companies like Google are preempting those positions represents a major crisis.
The highest barrier to entry is the software ecosystem. The fundamental reason NVIDIA dominates the market lies in "CUDA," the software platform it has built over nearly 20 years. Google has already established its own version of CUDA to counter this, and has recently expanded its ecosystem so that the widely used PyTorch models can run on TPUs without significant modifications. In contrast, NPU startups face the burden of building their own independent software environments from scratch.
Consequently, rather than pushing NVIDIA out, the industry is adopting a "decoupled inference" strategy, where chips perform only specific roles alongside GPUs. The US-based Cerebras has secured customers by maximizing memory capacity to outperform NVIDIA in answer generation speeds, while Groq is cooperating by entering into technology licensing agreements with NVIDIA to be integrated into NVIDIA systems.
Rebellions' "NVIDIA Alliance" vs. FuriosaAI's "Open Camp"
South Korea's representative NPU companies, Rebellions and FuriosaAI, are taking distinctly different paths to target this heterogeneous market.
Rebellions has chosen an "alliance strategy" through cooperation with the NVIDIA camp. A representative example is its recent "NVLink Fusion" cooperation with NVIDIA. In this structure, the front end is handled by GPUs while the back end is handled by Rebellions' R200 chip within the NVIDIA system. The advantage is that customers can enjoy the efficiency of Rebellions while maintaining the NVIDIA ecosystem, allowing entry into the market without directly confronting the wall of CUDA. However, the fact that NVIDIA holds the initiative remains a challenge.
On the other hand, FuriosaAI has chosen the "open camp" alongside AMD. This strategy involves combining AMD's hardware with the open-source software ecosystem "ROCm" to counter NVIDIA's monopoly. While this allows the strengths of the chips to be freely promoted because no single company monopolizes control, a variable remains that the ROCm ecosystem has not yet closed the gap with CUDA.
Currently, the combined 2025 revenue of the six major domestic NPU companies is approximately 50.4 billion won, which still shows a large gap compared to Cerebras in the US (approximately 700 billion won). The key moving forward will be whether Korean companies can achieve substantial results in overseas markets, such as data centers in Stockholm, Sweden, or Saudi Aramco, beyond supplying to the public sector.
Source: original video (YouTube)
0Comments
Comments are currently disabled.