Sentence specificity scores affect choosing technical document edits
Sentence Specificity Scores for Collaborative Technical Documentation: A Domain-Transfer Study
Computation and LanguageMachine Learning
Summary
When people and AI work together on technical documents, it helps to share clear and detailed context. This paper looks at ways to measure how specific sentences are in technical writing and tests if these measures can help pick better revisions made by language models. It finds that different scoring methods don’t always agree and that their usefulness depends on which scoring tool and document set are used. In some cases, using these scores improved selecting better sentence versions, but in others, the benefit was unclear.
What this means in practice
- •For technical writers: Use sentence specificity scores to help pick clearer or more precise technical document revisions generated by AI tools.
- •For ai developers: Integrate specificity scoring to improve selection among alternative text outputs from large language models in technical writing tasks.
Authors
Rocker D'Antonio, Thomas Benton Townsend, Dimitrios Michael Manias
Abstract
Collaboration depends on shared context, and technical documentation is one way that context persists across people and AI teammates. Specificity, the amount and exactness of detail expressed in language, shapes what information documentation captures and how precisely that information is communicated. This work audits sentence-specificity scoring artifacts on technical documentation and tests whether scores applied only after generation help choose among fixed LLM-generated revisions. Across Wikipedia and three technical-documentation corpora, the fixed general-domain predictor SpeciTeller and the pinned post-publication author-repository implementation of Ko et al.'s target-adapted predictor produce different corpus orders and same-sentence rank agreement from -0.066 to 0.510. Strict filtering and token-length adjustment change these patterns without reconciling them. In the Gemma set, SpeciTeller ranking raises direction-valid selection from 71.7% to 83.3% (+11.7 points; 95% source-case bootstrap interval +1.7 to +21.7); in the GPT-OSS-120B set, SpeciTeller ranking raises direction-valid selection from 51.7% to 56.7% (+5.0 points; 95% source-case bootstrap interval -6.7 to +16.7), and every primary single-score GPT-OSS-120B interval includes zero. These findings tie score interpretation and decision value to the predictor and candidate set.