Companies adopt on-premise AI to address security and cost concerns

Businesses are increasingly shifting toward on-premise AI infrastructure to protect sensitive data and ensure predictable operating expenses. During an interview on the 3rd, GFT Ventures CEO Eum Jae-hoon noted that while cloud services face security risks in sectors like finance and healthcare, internal setups allow for unlimited use once initial hardware costs are covered.

The axis of artificial intelligence (AI) infrastructure investment is shifting from hardware to applications, and now toward internal corporate environments. Following the first AI boom led by NVIDIA's GPUs and memory semiconductors from Samsung Electronics and SK hynix, a movement is gaining momentum where companies manage their own data to simultaneously solve security and cost issues.

Eum Jae-hoon, CEO of GFT Ventures, diagnosed the latest AI investment trends in Silicon Valley during an interview conducted on the 3rd (local time). Eum identified "on-premise AI" and "vertical specialized AI" as the next stages of the AI ecosystem. His analysis suggests that after the infrastructure stage of GPUs and memory, new opportunities are opening in the application layer, which generates actual revenue.

Companies Turn to 'On-premise' Due to Security and Cost Issues

One of the biggest inflection points in the recent AI industry is data security. While cloud-based AI services provided by companies like OpenAI or Anthropic are convenient, they possess a critical weakness for corporations: the risk of sensitive data leaking externally. Eum explained, "In industries where compliance is vital, such as finance, healthcare, security, and public institutions, it is impossible for data to be uploaded to the cloud."

In practice, many employees use ChatGPT at home but find themselves unable to use AI at work due to security concerns. To resolve this, companies are paying attention to "on-premise AI," which involves building infrastructure within their own data centers or servers. By connecting AI appliances—including GPUs and memory—to an internal corporate network, companies can deploy AI into their operations immediately while resolving security issues.

Predictability of costs is also a powerful incentive. Cloud AI operates on a structure where "token" costs continue to accrue based on usage, making budget management difficult. In contrast, once the initial hardware construction costs (CAPEX) are covered, on-premise solutions allow operating expenses (OPEX) to remain constant, a method preferred by corporate CFOs. Eum added, "You can use it without limit until the computer crashes, making costs predictable."

'Specialist' AI Will Survive Over General-Purpose AI

The way AI model performance is competed is also changing. If Large Language Models (LLMs) that cover all fields are "general practitioners," AI specialized for specific industries are like "specialists." Eum predicted that "vertical specialized AI," trained on data from specific industries, will hold strong competitiveness in the future market.

While general-purpose AI possesses broad knowledge by learning from vast amounts of human data, its accuracy may drop regarding a specific company's internal regulations or professional financial and medical data. Conversely, accuracy can be dramatically increased by utilizing training datasets specialized for a specific industry. Eum forecasted that for customers where accuracy is more important than speed, specialized models will be essential, and solutions that use technologies like multi-agent systems to push accuracy above 99% will gain attention.

This trend is appearing not only in large corporations but also in mid-sized companies. While large corporations have the scale to cooperate directly with big tech firms like OpenAI, mid-sized companies desperately need customized solutions that solve both security and cost issues. Eum analyzed, "AI solutions that mid-sized companies with thousands of employees can use immediately by connecting a single Ethernet cable will target the market."

Data Center Bottlenecks and Opportunities for Korean Companies

Physical limitations occurring during the expansion of AI infrastructure are also changing the investment landscape. Currently, securing power for data center construction is cited as the biggest bottleneck worldwide. Eum pointed out, "More than half of the announced data center projects may realistically struggle to operate due to power supply issues or opposition from local communities."

As the construction of large-scale data centers is delayed by power shortages and delays in grid construction, relatively smaller on-premise setups or hybrid methods (a mix of cloud and private servers) are emerging as alternatives. In this process, observations suggest that Korean companies with manufacturing capabilities could reap windfall benefits. This is because, amidst US-China tensions, Korea holds an important position in the AI infrastructure manufacturing sector as a manufacturing powerhouse.

Eum emphasized that the weight of investment is shifting from hardware-centric to software and specialized services, stating, "Ultimately, the final winners will be the application companies that ride on top of the infrastructure to create great value."

Source: original video (YouTube)