Commit message generation improved by checking component and candidate consistency

CoCoRerank: Towards Conventional Commit Message Generation by Component and Candidate Consistency Reranking

Software Engineering

Summary

Commit messages explain the changes made in software and usually follow a specific format to help developers understand these changes better. The authors created a large collection of well-structured commit messages following a standard format called the Conventional Commits Specification. They built a method called CoCoRerank that improves automatic commit message creation by checking that different parts of the message align well with the actual code changes and by choosing the best candidate messages from many options. Experiments show that this approach makes generated commit messages clearer and more standardized for software developers.

What this means in practice

  • For software developers: Generate commit messages that fully follow industry format standards, with improved accuracy and consistency for better project management.
  • For software engineering teams: Integrate enhanced commit message automation tools that help maintain standardized logging in version control workflows.

Authors

Shaopeng Jia, Yali Du, Ming Li

Abstract

Commit messages are essential for understanding software changes, yet automatic commit message generation typically treats a message as an unstructured text sequence. This limits its ability to support standardized development workflows, where commit messages are often expected to follow the Conventional Commits Specification (CCS) in the form type (scope): subject. In this paper, we study conventional commit message generation under the complete CCS format. We construct a new benchmark of 86,688 high-quality commits collected from open-source GitHub repositories, with each message normalized into type, scope, and subject through structural normalization and semantic quality filtering. Based on this benchmark, we propose a two-dimensional consistency-based reranking framework named CoCoRerank for LLM-based generation. CoCoRerank exploits horizontal consistency among the code change, type, scope, and subject, as well as vertical consensus across multiple generated candidates. Experiments with representative CMG baselines, LLM generators, reranking strategies, and ablation variants show that CoCoRerank improves both structural component prediction and subject generation quality. The results demonstrate that complete CCS supervision and multidimensional consistency modeling provide an effective foundation for accurate and standardized commit message generation. The artifact is publicly released at https://github.com/bluewhalebug/CoCoRerank.
View PDFOpen arXiv