structural-intelligence

/papers/when-human-in-the-loop-fails/README.md

When Human-in-the-Loop Fails: An Answerability Test for Deployed AI Systems

Author: Vladisav Jovanović
Status: Preprint
Version: Latest archived (Apr 2026)

Abstract

This paper argues that contemporary AI safety discourse often overestimates the protective value of human-in-the-loop oversight. In many real deployments, the human role is procedural rather than consequence-bearing, while the system materially steers outcomes. The central claim is that a human in the loop is not yet an answerability structure. Deployed AI systems become structurally unsafe when they acquire steering power without inheriting proportionate consequence, and when the speed, scale, or opacity of execution exceeds the human capacity for meaningful witness and correction. The paper introduces an Answerability Test for Deployed AI Systems, focusing on substitute controllers, liability laundering, override friction, symbolic oversight, exportable error cost, and the severance point where machine execution outruns human witness.

Keywords

AI safety; human-in-the-loop; answerability; deployed AI systems; oversight; witness; substitute controller; liability laundering; override friction; structural debt; AI governance; accountability; corrigibility; Structural Intelligence