Optical Communication Emerges as Key Solution to AI Data Center Bottlenecks

Optical communication technology is rising as a core solution for AI infrastructure as copper wiring hits physical limits in connecting massive GPU clusters. The shift toward optical solutions, including Co-Packaged Optics, aims to address increasing data transmission volumes and power consumption requirements in AI data centers.

As AI computing requirements grow exponentially, the semiconductor industry is turning its attention to a new "bottleneck" area. Following GPU memory and cooling power, the "connection" technology used to link tens of thousands of GPUs to exchange data has emerged as the decisive battlefield. With copper wiring, which has served as the pathway for data transmission, hitting its physical limits, optical communication using light is rapidly rising as a core technology for AI infrastructure.

The Limits of Copper and the Rise of Optical Communication: Replacing the "Data Highway"

Current AI data centers are structured so that tens of thousands of GPUs must operate as if they were one massive computer. However, if the data pathways connecting them are blocked, the full performance of the GPUs cannot be realized. While copper wires that transmit electrical signals have been primarily used, the requirements of the AI era are exceeding the physical limitations of copper.

According to an analysis by T Times TV, the primary reasons are the massive scale of GPUs that must be connected and the explosive increase in data transmission volumes. NVIDIA's next-generation server motherboards group 72 GPUs in a single rack for the Blackwell generation, with plans to increase this to 576 GPUs in the future Ruby Ultra model. As the distance between servers increases and the amount of data to be transmitted soars to the 1.6 terabit (Tb) level, the weaknesses of copper—where signals degrade even over short distances—are being exposed.

In contrast, optical communication holds an overwhelming advantage in three areas. First, it has low signal attenuation; while high-speed electrical signals in copper drop to 1/100 of their strength after just 15cm, optical fibers can maintain nearly half their signal strength even after traveling 15km. Second, it generates less heat and consumes less power; unlike copper, which generates heat due to resistance, optical fibers are light and thin, offering high cooling efficiency. Third, it provides wider bandwidth, creating an effect similar to adding lanes to a highway by simultaneously transmitting different colors of light.

Light Penetrating into Chips: How CPO Will Reshape the Industry Landscape

Optical communication technology is currently moving from the outside of the data center (between racks) to the inside (into the chips themselves). While the use of optical modules to connect distances of dozens of meters or more was the dominant method initially, "Co-Packaged Optics (CPO)" technology—which packages optical components directly next to switch chips—is now gaining attention.

The introduction of CPO technology will shorten the distance electrical signals must travel from dozens of centimeters to a few millimeters, bringing revolutionary changes in power efficiency and signal strength. Based on an 800Gbps port, conventional pluggable optical modules consume approximately 30 watts (W) of power, whereas the CPO method can reduce power consumption to the 9W level, cutting usage to less than 1/3. NVIDIA stated that it has already applied this technology to its Spectrum-X Photonics switch, to be unveiled in 2025, increasing power efficiency by 3.5 times.

The ultimate goal is "Optical I/O" technology, which processes data input/output using light directly from the GPU chip itself. While this is a highly difficult task that requires completely changing chip design and packaging methods, it is evaluated as a game changer that, if successful, could make numerous scattered GPUs operate like a single massive chip.

China's "Assembly Capability" vs. US and Taiwan's "Design and Packaging" Clash

The growth of the optical communication market signifies a reorganization of the entire supply chain. According to LightCounting, the global optical module market is expected to grow rapidly from $23.8 billion (approximately 33 trillion won) in 2025 to $112.3 billion (approximately 157 trillion won) by 2031, with AI data centers expected to account for more than 90% of this market.

Currently, China holds the lead in the market through its "assembly capability." The Chinese company InnoLight holds a powerful market share, supplying 40% of the 800Gbps optical modules purchased by NVIDIA and 70% of the volume purchased by Google. Because the precise assembly of core components is critical for optical modules, China's manufacturing capabilities are dominant in this field. However, the United States is responding through core original technologies such as laser chips (Rom-and-ton, Koyorant) and optical fiber (Corning).

As the industry's center of gravity shifts toward the "chip-centric" model, the landscape may change. If optical modules are integrated into a part of the chip (CPO) rather than remaining separate components, chip design capability and precision packaging technology will become the core competitive strengths rather than assembly capability. Accordingly, NVIDIA, Broadcom, and Marvell—the US leaders in chip design—and Taiwan's TSMC—the leader in packaging—are expected to emerge as key players in the future struggle for optical communication dominance.

Ultimately, the strategic tug-of-war between China, which is deploying massive volumes, and the United States and Taiwan, which are leveraging design and packaging technology, is expected to continue as they vie for the key to resolving AI infrastructure bottlenecks through optical communication technology.

Source: original video (YouTube)