Human-in-the-loop (HITL)

Human-in-the-loop means that a person checks or approves what an AI system does before it takes effect. An AI tool may draft a reply to a customer or pre-sort incoming invoices, but an employee reads the result and decides before anything is sent or paid. It gives you a chance to catch AI mistakes before they reach anyone, and for some decisions the law requires it.

Which forms are there?

In its Ethics Guidelines for Trustworthy AI (April 2019), the European Commission's expert group on AI distinguishes three kinds of oversight:

• Human-in-the-loop: a person can intervene in every decision. The guidelines add that this is often neither possible nor desirable.
• Human-on-the-loop: a person helps design the system and monitors it while it runs.
• Human-in-command: a person oversees the system as a whole and decides when and how to use it, including overriding its decisions.

In practice the term covers several patterns:

• Approval: the AI proposes an action and a person confirms it. The specification of MCP, a standard through which AI agents connect to tools, says there "SHOULD always be a human in the loop" who can deny a tool call.
• Escalation: the AI handles routine cases and passes on the uncertain ones. Amazon Augmented AI, for example, sends low-confidence predictions to human reviewers and can send a random sample for audit.
• Training: people label data or rank answers. In RLHF their rankings teach the model, as in OpenAI's InstructGPT (2022).

What does the law say?

• Under the GDPR (Article 22), people have the right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects on them. Where such a decision is allowed because of a contract or consent, they can at least ask for human intervention.
• The European privacy regulators' guidelines on automated decision-making (Article 29 Working Party, 2018) add that human involvement only counts if it is meaningful, "rather than just a token gesture", and done by someone with the authority and competence to change the decision.
• The AI Act requires high-risk systems to be built so that people can oversee them (Article 14): understand their limits, stay aware of automation bias, override the output and stop the system. The business using such a system must assign that oversight to people with the necessary competence, training and authority (Article 26). The dates are under AI governance.

Where does it go wrong?

People tend to follow the machine, which is called automation bias. In a study in Radiology (May 2023), 27 radiologists assessed mammograms with help from what they were told was an AI system. When its suggestion was wrong, inexperienced readers rated 19.8% of the mammograms correctly, against 79.7% when it was right; very experienced readers 45.5% against 82.3%. The regulators' guidelines draw the legal line: someone who routinely applies automated results without any actual influence on the outcome does not turn it into a human decision.

Example

Suppose an accounting firm uses AI to code incoming invoices. Invoices the AI is sure about are booked directly; the rest go to a queue where a bookkeeper decides. Once a week the bookkeeper also checks a random sample of the automatically booked invoices, so mistakes in the confident cases come to light as well, and the corrections go back into the rules.