"From 6.52 Seconds to 0.3 Seconds"... Judgment-Centric 'System 1 Model' to Change AI Software

Beyond the stage of artificial intelligence (AI) simply conversing with humans and generating long sentences, the era of the 'System 1 Model', which makes real-time judgments within software, is opening. While existing Large Language Models (LLMs) underwent long reasoning processes to complete answers, this new type of model is expected to fundamentally change software usability by returning probabilities and options that programs can utilize immediately.

"A game that waited 6.52 seconds, interpreting actions in 0.3 seconds"... Speed determines user experience

According to a demonstration video recently released on the 'Impossible Engineering - New IT Technologies' channel, the judgment speed of AI models is emerging as a key factor determining the practical utility of software. The video shows the changes brought by the difference in AI judgment speed through a case study of developing a text-based RPG game using specific numerical values.

In the existing configuration, the median time taken to interpret a user's sentence into a character's action was 6.52 seconds. However, when the 'Jev' model released by TypeSafe AI was applied, this time was drastically reduced to 0.317 seconds. The video analyzed, "A game where you have to wait more than 6 seconds every time a user moves and a game that interprets actions in 0.3 seconds will inevitably have a different user experience."

This speed innovation does not simply mean 'fast answers.' It suggests that AI can be deployed into functions that must be called frequently in real-time, such as an AI avoiding obstacles in a game, evaluating the tone and emotion of sentences during writing, or filtering SNS feeds. Functions that were difficult to implement with existing models due to cost and time issues could be integrated as basic software functions through the AI's 'fast and cheap judgment.'

Returning 'probabilities' and 'options' instead of sentence generation... Maximizing efficiency through parallel processing

The 'Jev' model developed by TypeSafe AI has a different technical direction from existing LLMs. While a typical Autoregressive model generates the next token sequentially based on previous tokens, Jev is designed to immediately calculate probability values according to the question and answer formats pre-determined by the developer. This performs a role similar to a 'judgment function' operating inside a program.

According to the video, there are three major differentiators for Jev. First, instead of generating answer sentences, it returns the probability of being 'True', a 'Choice' among predetermined options, or a 'Score' that evaluates based on an ordered scale. For example, if asked whether a specific comment aligns with a user's intention, it immediately provides the probability value of that judgment instead of a long explanation.

The second is parallel processing capability. While existing models often have to process questions one by one, Jev can pose multiple questions to a single input and produce the answers in parallel. Through this, it can evaluate at once whether a customer inquiry is a refund request, a simple complaint, or requires additional verification, thereby reducing the program's waiting time.

The third is the management of probability reliability. The company explained that they use the 'RLCD' learning method to make the predicted probability match the actual result. If the probability provided by the model is ambiguous (e.g., 52%), it is possible to perform sophisticated control, such as the program code recognizing this and inducing the collection of additional information or verification by a human.

From browser manipulation to editors... AI judgment permeates 'every moment' of an app

The application of such judgment-centric AI also appears in complex tasks such as web browser manipulation. According to a case released by the 'Browser Use' team, during the flight search process, the AI reads the buttons and input fields of a web page and converts them into multiple-choice questions. Afterward, it operates by choosing a number for what action the Jev model should take, deriving flight search results in about 7 seconds. Since not every click requires long reasoning, a division of roles is made where Jev handles tasks requiring 'fast judgment' and a separate generative model handles tasks requiring 'new sentence generation.'

As this technology advances in the future, software is expected to emerge in a form where AI judgment is attached in real-time at the moment of working, without the user having to separately pose questions to the AI. For example, an editor could point out in real-time whether a user is keeping promises while writing, or posts could be automatically collapsed in an SNS feed according to conditions set by the user.

However, the video also specified technical limitations. It predicted that the 'format guarantee' provided by the model does not necessarily mean a 'correct answer guarantee,' and that design technology to recognize the screen, actually execute, and recover from errors will be an important task in the future, even if judgment becomes faster. Currently, the price of Jev is set at approximately 0.04 dollars per 1 million tokens.