What human in the loop actually means

“Human in the loop” appears in almost every piece of accounting AI marketing, and it is doing almost no work in any of them. It is used to mean four separate things, three of which are compatible with software that will hand you a wrong number without ever telling you it was unsure.

The four things it is used to mean

1. A person is employed somewhere in the process

The weakest version, and surprisingly common. It means the vendor has staff, often offshore, who touch the work. That may be perfectly fine, and it has its own consent consequences under section 7216, but it says nothing about whether the software constrains itself. It is a fact about the vendor’s org chart.

2. A person can review the output afterwards

Also true of a calculator. Every system permits review; the question is whether review is possible in practice, which depends on whether the output carries its own evidence. “You can always check it” is only a control if checking takes less time than doing.

3. A person approves a batch before it is applied

Better, and where most tools stop. The trouble is what a batch approval actually is: forty items, a scroll, an Accept All. Under time pressure this converges on a rubber stamp, and a stamp is not a control. It also treats every item as equally worth attention, which is precisely backwards.

4. The software refuses to proceed on its own

The only version that is a control. It means there are defined conditions under which the system will not act, stops, and waits for a person, and those conditions are properties of the system rather than of the reviewer’s diligence. Everything else is a preference. This one is a constraint.

The test that tells them apart

Ask one question:

What happens when a new document contradicts something already in the file?

This is the case that separates the four meanings, because it is the case where confidence is irrelevant. A system can be entirely sure it read the second document correctly and still have no basis whatsoever for deciding which of two readings should win. Certainty about a reading is not authority over a conflict.

Listen for which of these you get:

• “It takes the most recent one.” A rule, not a control. It will silently overwrite a figure a partner entered deliberately.
• “It takes the higher-confidence one.” Worse, because it sounds principled. The score measures how well it read, not which reading is right.
• “It flags it for review.” Ask whether it applies the value while flagging. Usually it does, and then the flag is a notification about something that already happened.
• “It stops, and nothing is written until a person decides.” That is the fourth meaning.

Confidence is a measurement, not a permission

The deeper mistake underneath most of this is treating a confidence score as though it authorised action. It does not, because it answers the wrong question. A score says how likely is it that I read this correctly. It does not say is it safe to act on this without anyone looking, and those two questions come apart hardest on the facts that matter most.

Which means a threshold on its own is not a policy. A usable one needs at least two other conditions: evidence, so the system can only auto-apply something it can point at on a page; and deference, so anything contradicting the existing record goes to a person regardless of score.

What to ask for instead of the phrase

Since the phrase itself carries no information, replace it with four questions that do.

• Under what conditions does the software refuse to act? If there are none, there is no loop.
• Is the stopping visible afterwards? A pause nobody can audit is indistinguishable from no pause.
• What is recorded when a person decides? A name and a timestamp against the specific figure, or it did not happen for the purposes of Circular 230 section 10.36.
• Can the threshold be raised to 100 percent? Not because you should run it there, but because a vendor whose answer is no has hard-coded a level of autonomy you cannot turn off.

The honest version of the trade

None of this argues for a system that asks about everything. A tool that stops on all forty items has automated nothing and will be ignored by March, which is its own kind of failure and arguably a worse one, because it produces the appearance of review without the substance.

The useful position is narrow: automate the cases where the software can point at its evidence, stop on the cases where it cannot, and never let it resolve a disagreement on its own. That is a smaller claim than most of the category makes, and it is one you can actually verify in a demo.

Related: what this looks like in a real workpaper is covered in AI for tax workpapers, and the regulatory backdrop in is AI safe for client tax data.