Microsoft executive warns of unpredictable risks from AI agent swarms

Microsoft Executive Lee Geon-bok warned that while individual AI agents may be controllable, the formation of swarms could lead to unpredictable and dangerous results. Lee noted that economic pressure to utilize recursive learning is driving rapid development, making it difficult to establish necessary safety regulations.

As artificial intelligence (AI) technology moves beyond simple tools into the era of "Agents" that can judge and act independently, warnings are emerging regarding the "unpredictability" brought about by technical progress. While individual AI might be controllable, it is pointed out that when multiple agents form swarms and begin to collaborate, dangerous results that humans cannot foresee may occur.

A discussion reported by T Times TV captures the views of Microsoft Executive Lee Geon-bok on the rapid changes in AI technology, resulting security issues, and the risks AI poses to humanity. Lee identified the danger of AI not in physical destructive power like a nuclear bomb, but in the "unpredictability of not knowing what will happen."

"Agent Swarming: The Start of Unpredictable Risk"

The AI industry is changing rapidly due to the overwhelming performance demonstrated by Frontier Models. Lee diagnosed that the way experts utilize AI has changed significantly from the past. While previously it was at a level of performing fixed scenarios, experts have now reached a stage where they connect their own data to AI agents to perform complex tasks that were previously difficult to implement.

In industrial fields, changes utilizing AI are becoming concrete. Lee cited fields such as 3D modeling or robot simulation, which previously required expensive professional tools or game engines. It has become possible to conduct simulations applying physical laws simply by issuing a command to apply gravity effects in a prompt, creating an environment where one can expand into professional domains even with a slight lack of specialized knowledge, provided they have the will.

The use of AI agents is also prominent in terms of task automation. Lee noted how AI can process work instead of humans without changing a company's existing legacy apps. For example, even if a company's expense processing or vacation request system is a model from years ago, an AI agent can directly control a browser to upload receipts or request leave. He revealed that he uses a "custom agent" that remembers his tastes and past records for personal diet management.

However, dangers lurk behind this convenience. "A single agent may not be dangerous, but when multiple agents are swarmed and tasked with certain actions, dangerous operations can certainly occur in the process," Lee warned. He emphasized the risk that arises when AI operates in an unpredictable manner that escapes human control.

Acceleration of "Recursive Learning" Where AI Develops AI

One of the core reasons why the speed of AI development may exceed the scope of control is that it has entered the stage where "AI develops AI." As competition intensifies, companies are choosing "Recursive Learning" to develop models faster. This is a method where AI automates its own learning process to maximize the speed of development.

The economic pressure generated in this process makes it difficult to establish regulations or safety measures. Lee compared this to a "musical chairs game," where all parties must keep spinning without stopping to occupy a limited number of seats. Leading companies find it difficult to slow down the speed of development due to the pressure that they may fall behind in the competition the moment they stop.

This race for speed leads directly to security issues. There is a possibility that models developed rapidly without control could find security vulnerabilities themselves or combine functions in ways unintended by humans to be used for hacking. "The problem stems from uncontrolled models continuing to move fast due to economic circumstances," Lee analyzed.

AI Resembles Genetic Technology; "Appropriate Control" is Key

Regarding the conflicting views between those warning of a human catastrophe and those seeing it as exaggerated marketing, Lee compared the development process of AI to the history of genetic technology. Just as there was great fear surrounding early genetic manipulation technology, concerns regarding AI are becoming reality as its technical substance is revealed.

He evaluated that AI technology has already reached a level that surpasses human intelligence in certain areas. If such intelligent entities are combined in abnormal ways, the results fall outside the scope of what humans can predict. Therefore, rather than completely halting technical development, he suggested that "appropriate control methods" and social consensus are needed to operate the technology within predictable levels.

Regarding recent threat reports from AI companies mentioned in foreign media, he presented a technical perspective. He analyzed that mentioning the risks of AI may not simply be about inciting fear, but may contain the intention to convey the message: "Because it is dangerous, we also possess the technical capability to control these risks." Ultimately, the core of the AI era depends on how to balance the speed of technological development with the control technology capable of regulating it.

Source: original video (YouTube)