Policy conditioned AI use detection improves academic publishing compliance
Policy-Conditioned AI-Use Detection: An Evidentiary Framework for Academic Publishing
Computation and LanguageArtificial IntelligenceComputers and Society
Summary
Academic conferences and journals now have detailed rules about how people can use AI tools, but current AI detectors only say if AI likely wrote a text, which is not enough. The authors suggest a new way to detect AI use that checks if specific rules were followed, considering different roles and tasks. Instead of just labeling text as AI-generated, this method reports how confident it is and what evidence it used. They tested this approach in peer review settings and found it can mistakenly flag more authors who follow the rules than those who break them, showing that policies and other steps must support the detection.
What this means in practice
- •For conference organizers: Use policy-conditioned AI detection to verify if authors and reviewers comply with AI use rules during paper submission and review.
- •For journal editorial teams: Implement an auditable detection process that reports evidence and uncertainty to uphold AI use policies consistently across submissions.
Authors
Jairo Diaz-Rodriguez, Mumin Jia
Abstract
Major venues now publish detailed rules about how authors, reviewers, and area chairs may use AI, and those rules differ by role, by task, and by what must be disclosed. AI detection, the instrument usually proposed to enforce them, estimates something else: whether an AI model wrote the text. We argue that this target is misaligned with the decisions conferences and journals face, and propose policy-conditioned AI-use detection, an evidentiary framework for assessing whether a human--AI workflow complied with a stated rule. Policy makes the governing rule an explicit input. Inference reports hypotheses, evidence, calibration regime, and uncertainty in place of verdicts such as "AI detected". Evaluation builds benchmarks from reproducible pipelines that generate compliant and non-compliant workflows, and reports true positive rate at a false positive rate the venue fixes in advance. We work the framework through peer review, where at plausible violation rates a detector at a strong operating point still flags more compliant authors than violating ones. The framework therefore also names what a venue must instrument: structured disclosure, approved-tool routing that respects reviewer confidentiality, and a path by which a finding can be contested. Under this framing a detector is not an authorship classifier but an auditable procedure with an error rate the venue fixes in advance and can defend.