SCOPE compiles clear natural language policies into interpretable neural programs

SCOPE: Compiling Natural-Language Policies into Neurosymbolic Predicate Programs

Artificial IntelligenceComputation and Language

Summary

Policymakers need ways to ensure AI assistants follow complex rules over many interactions. The authors introduce SCOPE, a method that turns written rules into small neural programs that work together logically to check if an AI is following those rules. SCOPE learns from examples and keeps improving by finding mistakes, making its judgments reliable and easier to understand. It works well on tricky real-world policy tasks and can run efficiently on simpler AI models.

What this means in practice

  • For ai governance teams: Deploy reliable and interpretable multi-step policy enforcement programs for AI assistants using natural language policies.
  • For chatbot developers: Run efficient policy checks with small models optimized via SCOPE to ensure AI assistant responses meet complex guidelines without heavy compute costs.
  • For customer service platforms: Automatically verify compliance of conversation histories with company policies using a modular neural-symbolic approach.$Commercial implications: Enables advanced policy monitoring features as a product add-on for service providers needing scalable compliance tools.

Authors

Arnon Mazza, Elad Levi

Abstract

Policy enforcement for LLM agents has shifted from single-turn content moderation to compound, application-specific policies evaluated over multi-turn transcripts and tool-call trajectories. Fine-tuned guardrail models are efficient but degrade on compound policies, and while LLM-as-a-judge approaches achieve higher accuracy, they incur high inference cost and tend to produce unstable predictions. Neurosymbolic methods offer a natural alternative, decomposing the policy into narrow neural decisions composed under explicit logic. However, existing symbolic layers lack the required expressiveness and are optimized against fixed data rather than the policy itself. To address this, we introduce SCOPE, which compiles a policy into an executable program of neural predicates executed by small models, deterministic extractors, and code that composes their outputs. The program is optimized jointly with its training data, using adversarial generation of counterexamples that target where the program deviates from the policy. We evaluate SCOPE on a benchmark of real-world policy tasks against prompt optimization, task decomposition, and workflow generation methods, achieving the highest average accuracy across tasks and models. Programs optimized with the weakest executor transfer to other models without further optimization. This allows them to be deployed in production on inexpensive models, where they reach accuracy comparable to frontier reasoning models. Unlike these monolithic models, SCOPE produces substantially more stable predictions, and each verdict is interpretable through the predicate outputs that produced it. Additionally, we validate the key components of our method through an ablation study. SCOPE thus enables reliable, interpretable, and efficient policy enforcement at production scale.