Policy alignment improves agent task success and runtime compliance

Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics

Machine LearningArtificial Intelligence

Summary

When AI agents operate inside specific software environments, the rules that decide which actions they can take and how the system changes are often hidden. The authors developed a method called Harness Policy Distillation (HPD) that makes these hidden rules explicit during training. By teaching the AI not just what action to take but also how those actions fit the system’s rules, the AI becomes much better at completing tasks and following runtime rules. This method improves how well the AI works inside its operating environment without needing extra information when it’s deployed.

What this means in practice

  • For software engineers: Train AI agents that reliably follow environment rules to improve task success and compliance in deployed applications.
  • For automation platform developers: Create automation systems that can better align with product-specific runtime constraints by integrating harness policy distillation techniques.

Authors

Junze Zhu, Weihao Chen, Zhiyang Zhou, Zhen Wu, Xinyu Dai

Abstract

Deploying a general-purpose model as an agent means running it inside a product-specific runtime harness that decides which actions are executable, how state advances, and when the loop may terminate. Ordinary trajectory supervision shows which action was taken but leaves the runtime semantics behind that action implicit. We introduce Harness Policy Distillation (HPD), which makes these runtime control semantics an explicit training target alongside actions to align the harness policy. HPD replays reference trajectories through the harness to recover the runtime semantics that are implicit in ordinary trajectory data, and represents them as control documents at each model-call boundary. These documents provide privileged information for a teacher to generate actions, while harness verification ensures that the resulting targets are executable under the runtime contract. We jointly train a shared model on the runtime semantics recovered from control-state privileged information and native history, thereby internalizing these semantics into the model policy. At deployment, the trained model uses the original harness interface without requiring the control document or any additional supplement. On TRAJECT-Bench with the Pi harness, Qwen3-8B trained with HPD reaches 22.98% autonomous task success and 49.23% runtime compliance, against 3.09% and 19.90% for the unadapted model. Harness-verified teacher actions substantially improve the native deployment policy, while joint control-state supervision provides further gains in task completion and runtime compliance. HPD turns executable runtime semantics into dense supervision for learning harness-compatible actions at each decision boundary.