"By 2040, Humanity Might Use All Its Energy"... The Solution to AI Power Shortage Lies in 'Memory Efficiency'

As artificial intelligence (AI) technology advances, projections suggest that the amount of power consumed by data centers and AI devices will explode. In particular, warnings have been raised that if current computing methods are maintained, by 2040, all the energy produced by humanity could be used for data movement. Accordingly, the 'power efficiency' of memory semiconductors is rapidly emerging as a key factor that will determine the sustainability of the AI industry.

Data center power consumption projected to surge to 945 terawatt-hours by 2030

According to a video from the 'Future Talent CLASS' on the SK hynix YouTube channel, the energy consumption occurring during the training and inference processes of AI models is predicted to increase exponentially. Professor Lee Cheol-ho from the School of Electrical and Information Engineering at Seoul National University, who appeared in the video, explained, "Approximately 50 gigawatts (GW) of energy is used to train hyper-scale AI models, which is equivalent to the amount of energy used by the City and County of San Francisco in the United States for three days."

Power demand occurs continuously not only in the training stage but also in the 'inference' stage where questions are answered. This is because the actual power consumption increases proportionally as the requested task becomes more complex, i.e., as the amount of tokens increases. The video cited statistical predictions that data center power consumption will more than double from 415 terawatt-hours (TWh) in 2024 to 945 terawatt-hours in 2030. Professor Lee emphasized the urgency of improving power efficiency, stating, "There is also a prediction that the energy used to call up data while maintaining the current computing structure will use all the energy sources produced by humanity by 2040."

These power problems are expected to become even more serious in the era of 'Physical AI', which extends into the physical world. Professor Lee said, "In 10 years, Physical AI such as humanoid robots will become commonplace, going beyond wearable devices and autonomous vehicles. The energy to operate the AI chips, which correspond to the robot's brain, and the power required for the operation of the drive system are essential, and since robots must operate with batteries anytime and anywhere, solving the power problem is a prerequisite for realizing the technology."

Solving the 'Bottleneck' is key... Next-generation technologies like HBM and PIM drawing attention

The power problem in the AI era is not limited simply to the power consumption of the memory itself. Professor Lee pointed out that the energy consumption occurring when exchanging data between the processor (CPU, GPU, etc.) and memory is very serious. This is because the current 'von Neumann architecture' separates the processor and memory, so massive power is consumed in the process of calling up information from memory for computation and then storing the results back. The phenomenon of data transmission congestion occurring during this process is called a 'Bottleneck', and solving this is the core of power efficiency.

The operating principles inside the memory are also cited as a major cause of power consumption. DRAM stores information by filling small capacitors with electric charge, and the 'Refresh' process, which refills the charge that is consumed when reading information or leaks over time, is essential. Professor Lee explained, "As the memory cell size becomes smaller, these problems are becoming larger."

To solve this, High Bandwidth Memory (HBM), which increases the density of data transmission paths, or 'PIM, CIM' technology, which minimizes data movement by performing computation and storage simultaneously inside the memory, are mentioned as major solutions. Professor Lee diagnosed, "Without the development of memory, it is difficult to develop very efficient AI."

SK hynix expands low-power lineup from LPDDR to HBM4

Semiconductor companies are developing customized low-power solutions for each product to overcome this power shortage. In the case of SK hynix, it is developing a low-power LPDDR product group optimized for battery-based 'On-device AI' such as robots or autonomous vehicles. It has also introduced the 'LPCAMM2' product, which modularizes low-power DDR for data centers.

HBM technology used in high-performance AI accelerators is also focusing on improving power efficiency. According to the video, SK hynix's next-generation model, HBM4, is known to have improved power efficiency by approximately 40% compared to the previous model, HBM3E, by increasing the density of the connection lines (I/O) and introducing ultra-fine advanced processes to the logic die.

Professor Lee defined low power not simply as lowering performance, but as "a matter of efficiency, using less power while delivering the same performance." He predicted that in the future, "the key is to develop memory elements that are high-performance yet low-power, and whether such power efficiency is secured will play a decisive role in AI advancing into the physical world."