Palmyra x6 Technical Report: An Agentic, Tool-Use Model Post-Trained via Anchored Supervised Fine-Tuning
2026-08-17 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors created a big language model called Palmyra x6, designed to help with tasks for businesses. They improved it by carefully training it on a small set of verified example actions using a special training method that combines two optimization techniques. Their approach was cautious, using only one pass over the data with a small learning rate to keep the model close to its original version. Palmyra x6 performed better than the previous model for a writing assistant and scored highest on a set of language tests, while also doing well in tests for avoiding bias and safety issues.
Large Language ModelMixture-of-ExpertsAnchored Supervised Fine-TuningTool-use TrajectoriesMuon OptimizerAdam OptimizerKL DivergenceBias EvaluationSafety EvaluationBenchmark Scores
Authors
Peng Du, Kiran Kamble, Rakshith Vasudev, Zhizhuo Yang, Rohith Nadimpally, Arjun Krishna, Waseem Alshikh, Daniel M. Bikel
Abstract
Palmyra x6 is a large language model optimized for use with enterprise-oriented agentic tasks. The model was built by post-training a Mixture-of-Experts base model with Anchored Supervised Fine-Tuning on a compact corpus of verified, synthetic tool-use trajectories, optimized with a Muon + Adam hybrid. The recipe is deliberately conservative and deliberately controlled: 626 trajectories, a single epoch, a low learning rate, and a KL anchor to the frozen base. The model shows substantial gains over the previous default model for Writer Agent, and compares favorably with several recent models on public benchmarks, scoring the highest on BFCL Core at $0.785$ and posts the highest six-benchmark mean of the cohort. Furthermore, the model has shown itself to be competitive or leading relative to comparators in our bias and safety evaluations.