Large language models improve traffic signal control with stepwise reasoning
ProcessLight: Process Supervision for Large Language Model Based Traffic Signal Control
Artificial Intelligence
Summary
Controlling traffic lights is tricky because you need to make smart decisions to keep traffic flowing smoothly. This paper shows how using large language models (LLMs), which can explain their thinking step-by-step, helps improve traffic light control. The authors create a method called ProcessLight that breaks decisions into small steps that can be checked and improved separately. They also develop a training system that rewards good steps in the reasoning instead of just final results, which helps the model learn better. Tests on real traffic data show their approach works better than older methods.
What this means in practice
- •For traffic management teams: Improve traffic signal decision-making accuracy by optimizing reasoning steps, enhancing flow based on verifiable intermediate checks.
- •For autonomous vehicle operators: Integrate interpretable traffic signal decision support that delivers stepwise explanations for better coordination with vehicle movement strategies.
Authors
Huaitao Zhao, Tianlong Zhou, Weijie Wang, Jiasheng Shi, Weixiong Rao
Abstract
Large Language Models (LLMs) have recently been introduced into traffic signal control (TSC) as decision agents due to their strengths in human-readable reasoning generation. Yet, existing LLM TSC methods optimize only from final outcomes and fail to distinguish valid from flawed reasoning steps, causing useful or misleading steps to be jointly updated and thus impairing the model's learning of effective reasoning. To bridge this gap, we propose an LLM-based framework ProcessLight to decompose signal decisions into verifiable semantic steps. Building on ProcessLight, we further develop Step-wise Traffic Process Policy Optimization (STeP-PO), a novel reinforcement learning framework that optimizes structured reasoning processes through step-level credit assignment. Specifically, STeP-PO uses step quality scores to evaluate local reasoning quality and step importance to measure each step's influence on the final action, and then assigns step-level advantages over a semantic step tree structure. The resulting step-level advantages are propagated to reasoning tokens, enabling fine-grained policy optimization beyond outcome-only rewards. Extensive experiments over multiple real-world datasets demonstrate the superiority of our methods. Our code is available at https://github.com/wenzhaoabc/processlight.