System One Models: a glossary for leaders

The words you will hear when your team or a vendor talks about System One models, explained in plain language. Each entry ends with why the term matters when you are making a decision. It goes with the four-part course System One Models for leaders.

The one test to remember

Before any AI step, ask three questions in order. Can a rule, a calculation or a lookup do it? Then it is ordinary software. Is it a one-second judgment over messy input, where the answer is one of a known set? Then it is a System One job. Does it need writing, explaining or reasoning in steps, or was the System One model unsure? Then it goes to a language model or a person.

The terms

System 1 and System 2. Daniel Kahneman's names for two modes of human thinking: fast and intuitive, and slow and deliberate (Nobel lecture). For a decision: the analogy explains the division of labour, not how the models work inside.

System One model. A vendor term for an AI model that answers fixed questions with a choice, a score or a yes or no, each with a probability, and never writes text. Examples are TypeSafe's Jev (hosted) and Convai's Laya (open). For a decision: it is a tool for judgments at volume, not a replacement for a language model.

"System 1 LLM" in research. A different meaning: academics use it for ordinary chat models, as opposed to reasoning models. For a decision: when someone says "System 1 AI", ask which one they mean.

Large language model (LLM). The kind of AI behind chat assistants, which produces text one piece at a time. For a decision: the right tool for writing and reasoning, and an expensive one for simple judgments.

Hosted and open models. A hosted model runs on the vendor's servers and you send it your data. An open model can be downloaded and run on your own infrastructure. For a decision: hosted is quicker to start; open keeps your data inside your walls but may need tuning and someone to run it.

Choice, score and yes or no. The three kinds of question a System One model answers: pick one option from a list, place something on a scale you define, or say how likely a statement is to be true. For a decision: if your question cannot be put in one of these forms, it is not a System One job.

Confidence. How sure the model is about an answer, given as a probability. For a decision: it is what lets software act on sure cases and send unsure ones to people.

Calibration. Whether the model's confidence matches reality across many cases: of all the answers it gave at 90%, were about 90% right? For a decision: a calibrated model lets you set a confidence line and predict the error rate above it. It is not a promise about any single answer.

Confidence line (threshold). The confidence above which the model's answer is used directly and below which the case goes to a person. For a decision: a business and risk choice, written as a number you can review and change.

Escalation. Sending a case the model was unsure about to a language model or a person. For a decision: the escalation rate tells you how much human work remains.

Hallucination. An AI producing something false or made up. For a decision: a System One model cannot produce an answer outside your list, but it can still choose the wrong option confidently. "Cannot hallucinate" means the format is guaranteed, not the answer.

Repeatability. Whether the model gives the same answer to the same question twice. For a decision: without it, you cannot tell whether your process improved or the AI changed its mind.

Signal value. A measure proposed in the LangChain grading experiment: agreement with people multiplied by repeatability. For a decision: a judge is only useful when it is both right and stable.

AI as a judge. Using one AI to grade the output of another. For a decision: common and useful, but check the grader against people before you trust its scores.

Shadow mode. Running a new model silently beside the current process, logging what it would have done without letting it act. For a decision: the safest way to get real evidence before switching anything on.

Tuning (fine-tuning). Training an existing model further on your own examples. For a decision: some open models need this before they are useful, so budget for it.

Version pinning. Fixing which version of a hosted model you use, so it does not change when the vendor updates it. For a decision: any judgment you rely on for compliance needs a version number.

Data residency. Rules about where data may be stored or processed. For a decision: settle it before a pilot, because it may decide between a hosted and an open model.

Prompt-and-parse. The older way of getting a judgment from AI: ask a language model to answer in a set format, then read the answer out of its text. For a decision: this is the pattern System One models replace for simple judgments.

Patterns from real projects

Quote, don't rewrite. The model picks the passages that answer a question, and software returns them word for word. Good for contracts, policies and audit evidence.

Narrow step by step. Too many options at once becomes a series of small choices: section, then page, then a final check.

Act from a menu. Software lists the actions that are allowed, the model picks one, software checks the result. Keeps AI agents bounded.

Grade at scale. "Is this good?" becomes several quick yes-or-no checks that software combines.

Watchdog. A fast model asks narrow questions about a slow, expensive AI process and raises a hand when something looks wrong.

Judgment shell. Keep creative work on a language model or a person, with quick System One checks before it and after it.