Contestable Agents for Data Governance: Recording Human Review History for AI-Generated Metadata Across Structured and Unstructured Systems

Problem

Industrial knowledge is scattered across structured data such as telemetry and point lists and unstructured documents such as manuals and service notes. Classifying and linking it depends on a small number of experts. LLM agents can propose metadata, but if their proposals are applied without review, errors spread to downstream systems. When approval is a single click, there is no record of what the reviewer checked. When reviewers' corrections live only in emails and tickets, agents repeat the same mistakes.

Method

The paper proposes 'Contestable Governance' as a four-step workflow. (1) An analyzer gathers candidates, related documents, a glossary and previously verified decisions (precedents). (2) Specialized agents with distinct roles produce sourced supporting and opposing arguments, and scores are computed with a QBAF (quantitative bipolar argumentation framework). (3) The steward accepts, rejects, edits or adds arguments, and the scores are recomputed. (4) The highest-scoring candidate is selected, and the rationale, arguments, intervention history and outcome are stored in a governance record. A versioned domain glossary supplies agents with abbreviation definitions per equipment type and revision. High-confidence proposals go through a Triage path that skips full argumentation, but pre-commit review is still kept. A separate gate controls which verified decisions become precedents. Records are stored as MongoDB documents, and a decision's full lineage can be retrieved with a single query using one session ID.

Results

• The pattern is illustrated with two HVAC cases: classifying points into Brick Schema classes (with an open reference implementation), and an illustrative case that links warranty claim fields and telemetry to service notes and revision notices to determine failure modes.
• In the warranty example, with α=0.3, the maximum change between consecutive iterations after 10 update iterations was 3.20×10⁻⁷. The authors report that the remaining error is at most 6.85×10⁻⁷ in raw margin, smaller than the three-decimal precision used to compare candidates.
• Changing α to 0.2 or 0.4 did not change the winner before review (Sensor) or after review (Linkage). The authors state that this shows only the numerical stability of this example and does not establish the validity or calibration of the decisions.
• Because the final class is chosen by the scoring engine rather than the Finalizer, the authors explain that auditors can recompute and verify the result from the stored arguments alone, without rerunning the model.

Limitations

This is a design-proposal paper with no quantitative evaluation of effectiveness. The warranty case is a synthetic example, and the arguments' base scores were set by the authors for illustration, without a rubric or calibration. The scoring rule (signed-sum logistic) and α=0.3 were not chosen on performance grounds, and the authors state they need review against the axioms of weighted bipolar argumentation. In the reference implementation, the agent that writes an argument also assigns its base score. Cited figures such as the 72.7% reduction in LLM calls from Triage come from other experiments and were not measured on this workflow. Whether decision quality actually improves must be verified through independent judgment.

Why it matters

It offers concrete design guidance for organizations that want to use LLM agents to generate metadata for data catalogs automatically. It shows how to keep human approval from becoming a perfunctory click and how to keep who decided what, and on what basis, as auditable data. It is a useful reference for reducing the risk, in manufacturing settings where equipment and document versions change often, that misclassification turns into remote diagnosis errors.

Key terms

Contestability

The property that qualified people can challenge an AI output's reasoning before it is finalized, and that their intervention is kept in the decision record.

QBAF(Quantitative Bipolar Argumentation Framework)

An argumentation framework in which arguments support or attack one another and each argument has a numeric strength. The final score is computed by repeatedly incorporating the strengths of neighboring arguments.

Data Steward

The person responsible for the definitions, classification and quality of a particular data domain. In this paper, it is the person who reviews agent proposals and decides whether to commit them.

Brick Schema

A shared ontology for describing building equipment, sensor points and their relationships. It is used to map point names, which differ by vendor, to standard classes.

Original

• arXiv abstract
• PDF
• HTML full text
• Hugging Face

Authors: Humza Akhtar, Brian Freeman · Posted 2026-10-22 · License CC BY 4.0 · Based on the full text

This review is an AI-written summary of the paper. Please treat the original paper as the reference.