[4F5-OS-29c] OS-29

4:30 PM - 4:45 PM JST(7:30 AM - 7:45 AM UTC)

[4F5-OS-29c-05] Human-in-the-Loop Hard Attention for Privacy-Aware Active Vision

〇Pengcheng Pan¹, Shogo Yonekura¹, Yasuo Kuniyoshi¹ (1. Univ. of Tokyo)

Keywords:

Hard Attention,Privacy Preservation,Human-in-the-Loop

Hard-attention agents actively perceive through sequential glimpses, making their scanpaths an auditable record of what the model attempted to access and enabling human-in-the-loop oversight. We present a prototype of privacy-aware active vision with human-in-the-loop auditing: at each step, the agent proposes the next glimpse, and a human auditor may approve or override it under a sensitive-region constraint. We quantify leakage with requested (model-proposed) and executed (actually observed) leakage. On CIFAR-10 with a synthetic sensitive region covering 25\% of the image, an unconstrained model achieves 63.96\% accuracy with executed leakage 0.453. An automatic safety policy drives executed leakage to near-zero, while reducing accuracy to 59.44\%. Human-audited demos achieve 0 executed leakage with 72.01\% audited accuracy on 60 samples.