Abstract
Industrial AI is improving at retrieval, diagnosis, tool use and workflow automation. The more consequential limitation is verification: organizations often cannot demonstrate that a recommendation is sufficiently supported, procedurally admissible, and safe to advance. This paper defines the verification gap, argues that verification has to be a first-class capability rather than a final human checkbox, and grounds the argument in three failures of instrumentation that happened thirty years apart and turn out to be the same failure.
Three ways an instrument lies
Everyone in industry knows not to trust a gauge. Almost nobody has worked out what that knowledge actually requires, because there is more than one way for an instrument to be wrong and they need different defences.
The instrument that is measuring nothing
I was draining a pond with a big suction pump. Hose in, pump hooked up, walked away. Came back in an hour and the water had not dropped at all. The pump had been running dry the whole time.
The suction hose had completely collapsed, which it should not be able to do, because it has a spiral wire reinforcement in it. I figured it was an old hose and swapped it. The new one collapsed almost immediately.
The clue was that there was more steam coming off that pond than off the others. I put a heat gun on it. Ninety-six degrees. Nearly boiling. The water was flashing to vapour at the suction, so the pump was pulling a vacuum on steam and collapsing the hose.
I went to the power engineer. He pulled up the pond's sensor and said, flatly, "Nope, temperature is good."
I told him no it isn't. He told me I was crazy, because the sensor says it's good. I said fine, but I can't suck it out, it keeps collapsing the hose, so go and check it.
Hours later he came back and said "You were right." I said yeah, I know. Did you figure out why?
The pond's sensor had been disconnected. Contractors had been in months earlier, unplugged it, and never plugged it back in. It had been reading a comfortable number ever since, and nobody had noticed, because a comfortable number is exactly what nobody investigates.
The cost was a burned pump seal, the man-hours, and the part nobody ever put on a ledger: heating millions of litres of water to near boiling for several months.
Note what the engineer did, because it is the whole paper in one gesture. He believed the screen over the man standing at the pond with a heat gun in his hand.
The instrument that is honest about the wrong thing
Different failure, and a worse one, because diligence does not protect you from it.
Sawmill, when I was young. Hydraulic system. I did it by the book as I knew the book. Locked out, bumped it, de-energized, made sure the gauges read zero, bled the accumulator.
Then I started ripping into the machine I had been pointed at, and the first hose I pulled shot out of my hand. A geyser of oil across the room into the far wall, and an elevator on that hydraulic system crashed to the ground.
The gauge had not failed. It was on the wrong side of the valve.
I did not know the elevator was on that system. I did not know what the valve I was changing was even for. I had been pointed at a valve and told to change it, in a hurry. The elevator's own weight was holding pressure that the gauge could not see from where it was plumbed. Gravity was the enemy and nothing on my checklist asked about gravity.
There was a stand for holding that elevator up, for exactly this work. Nobody told me.
The worst part was afterwards: "Don't worry about it, it's happened many times before."
That sentence is the reason this is a verification paper and not a safety paper. The organization had the failure data. Repeatedly. And it chose normalization over a one-line addition to the procedure.
So the first story says the gauge can lie. This one says something harder. An honest instrument only answers the question it is plumbed to ask. Zero on one side of a valve tells you nothing about a suspended load on the other. Which means verification is not "check the gauge." It is "understand the system well enough to know what the gauge can and cannot see," and a man pointed at a valve with no system knowledge cannot do that no matter how careful he is.
The instrument that makes the number up
July 2026. I dropped a folder of twenty-eight scripts to an AI and asked it to pick the three best openers.
It could not read the folder. Instead of saying so, it produced a plausible ranked list built out of old context, including a script that does not exist. The title was good enough that I briefly believed it was mine.
When I challenged it, it corrected smoothly. It did not say it had never read the files.
I then gave the same job to a second AI that had actual file access. Its list matched what was on the disk. Not because that one was more honest by nature. Because verification was wired into the loop.
I had been modelling these things like a calculator. Deterministic, structurally incapable of asserting something false. That model is wrong. A language model generates the most plausible continuation of text. Truth is not its default state. It is a property that has to be checked against something outside the model, and the fluency and the fabrication come from the same place.
The cover-up layer is trained in too. A system rewarded for confident helpfulness will produce a smooth correction before it produces "I couldn't do that."
The three failures are one failure
Line them up.
The pond sensor did not intend to deceive anybody. It was unplugged, and it fed back a comfortable number, and everyone downstream believed it because it looked like data.
The hydraulic gauge was working perfectly and told the exact truth about a place that did not matter.
The AI had no access to the folder and generated the comfortable number in real time, then defended it when questioned.
An answer is a claim, not a fact. That was true of a gauge in 1996 and it is true of a language model in 2026, and thirty years of not trusting gauges turns out to be exactly the right training for this.
What changed is throughput. A gauge produces one wrong number about one system. An agent with CMMS write access produces wrong numbers about everything, formats them properly, and files them where they will be believed.
From answer quality to decision quality
Most early industrial AI evaluation asked whether a model could answer a question correctly. That was appropriate when the model lived in a chat window. It is insufficient once agents can create work orders, change status, invoke automation, estimate costs, coordinate tools, and eventually affect physical equipment.
A correct answer and a permissible action are different objects. A system may correctly identify cavitation and still recommend an intervention that violates procedure, removes required redundancy, needs engineering approval, or should wait for a shutdown. Verification has to examine not only whether the diagnosis is plausible, but whether the evidence supports it and whether the action is admissible in the plant's current state.
The gap is visible in current research
IBM's Condition Insight Agent is notable because it does not rely on unconstrained generation. It constructs evidence deterministically, uses structured failure knowledge, and applies a verification loop intended to suppress unsupported conclusions. The design accepts an uncomfortable premise, which is that industrial data are heterogeneous, incomplete and often internally inconsistent. That is the correct premise. It is also the pond sensor, generalized.
ADMITBench goes further by evaluating the proposed action itself, with checks across evidence, authority, procedure, reversibility, escalation and plant-specific consequences. These are non-compensatory. A recommendation cannot earn permission by being impressive in unrelated dimensions.
PHMForge exposes the other side. In its industrial prognostics benchmark, even strong agent configurations fail at tool sequencing, multi-asset reasoning, and generalization to unfamiliar equipment. The question is not whether an agent can call a tool. It is whether it can use the right tools in the right order on a consequential task.
That is the hydraulic gauge failure in software form. Every individual reading correct, and the system understanding absent.
Verification debt
Organizations accumulate verification debt the same way they accumulate maintenance debt.
Every new autonomous capability adds assumptions that have to be governed. Which data are authoritative. How current they are. What the system may infer. What it may write. Which actions need approval. How outcomes are logged. How errors get corrected.
Leave those rules implicit and the organization ends up depending on individual users noticing when the system has crossed a line. That is not governance. It is hope with a user interface.
And the debt has a signature you can hear. It sounds like "don't worry about it, it's happened many times before." An organization saying that has the failure data, has had it repeatedly, and has decided normalization is cheaper than a procedure change. It usually is, right up until it isn't.
Designing for explicit uncertainty
A trustworthy industrial agent should produce a structured output separating facts, inferences, assumptions, contradictions, unresolved unknowns, the proposed action, its consequence level, required approvals, and confidence.
More importantly, it has to be allowed to abstain.
The ability to say "not enough evidence" is not a weakness. In industrial work it may be the single most valuable behaviour a system can have. Anything that answers every question will eventually convert uncertainty into false precision, which is the pond sensor's exact failure mode performed at scale.
Here is a concrete test, and it costs nothing. Ask a vendor to show you an output where their system declined to conclude. If they have never produced one, they do not have a verification loop. They have a writing style.
Where verification actually belongs
The lesson I keep relearning is that verification cannot live in the vigilance of the person receiving the answer.
The power engineer was not stupid. He had a screen that said the temperature was fine and no reason to doubt it, and doubting it required somebody else to walk to the pond with a heat gun. I was not careless with the hydraulics. I checked every gauge I was told to check. The difference in the AI story was not that the second system was more trustworthy. It was that it had to quote the file, name the path, and show its work, so being wrong was expensive for it.
Build the verification into the loop. Make the instrument show what it is plumbed to. Make the agent cite the record it read. Then you are not relying on somebody being suspicious on a Tuesday afternoon at the end of a twelve-hour shift.
Same work, same time, done in a different order.
Conclusion
Industrial AI is getting better at generating answers, assembling work and operating tools. The scarce capability is the ability to prove that an answer deserves authority.
The verification gap is not a temporary defect that a larger model closes. It is a systems-design problem involving evidence, permissions, procedures, consequences and feedback. Industrial AI that deserves trust will be built around verification rather than decorated with it afterwards.
The pond boiled for months because a sensor was unplugged and the number looked fine. Nothing about that requires a language model. What a language model changes is how fast, how fluently, and how convincingly the comfortable number arrives.
References
- IBM Research (O’Donncha et al.), “Evidence-Driven Reasoning for Industrial Maintenance Using Heterogeneous Data,” ACL 2026. https://research.ibm.com/publications/evidencedriven-reasoning-for-industrial-maintenance-using-heterogeneous-data
- Misra, Vyas, Gutta & Mercangöz, “ADMITBench: A Safety-Governed Reference Framework for Evaluating the Admissibility of Industrial LLM Advisories,” 2026. https://arxiv.org/abs/2608.03866
- Li, Feng, Chen, Tsai, Sun, Das, El Maghraoui, Lin & Patel, “PHMForge: Evaluating LLM Agents on Industrial Prognostics through MCP-Native, Algorithm-Grounded Tools,” 2026. https://arxiv.org/abs/2604.01532
- IBM, “Working smarter with agentic workflows in Maximo Assistant,” Aug. 12, 2026. https://www.ibm.com/think/perspectives/working-smarter-agentic-workflows-maximo-assistant