Papers for

automated tutoring systems teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Structural supervision improves latent chain of thought reasoning accuracy

Structural Process Supervision for Latent Chain-of-Thought Reasoning

Abstract: Latent reasoning approaches enhance token-level efficiency and robustness by replacing verbose, explicit chain-of-thought (CoT) tokens with compact continuous-space embeddings. However, existing methods lack direct process supervision over these latent embeddings, which often leads to representation collapse and uneven information distribution. To address this, we propose Prototype-Mediated Process Supervision (PMPS), which introduces learnable reasoning prototypes as semantic anchors to provide structural process-level supervision for latent reasoning. PMPS projects latent embeddings and explicit CoT embeddings into a shared prototype space, achieving many-to-many soft alignment between unequal-length representations through prototype assignment. Meanwhile, we introduce a Progressive Sequential Alignment (PSA) module to further guide training: positional priors initially encourage sequential alignment structure, then gradually relax to permit adaptive matching. Experimental results show that PMPS compresses output token length to under 50% of explicit CoT on GSM8K-Aug. Compared to leading baseline SIM-CoT, our method achieves average accuracy gains of 2.08% across different model families. On GPT-2, PMPS even surpasses CoT-SFT. On larger models and a more challenging task, PMPS consistently attains the highest accuracy among all latent reasoning methods with comparable output length.

Wed 9 SeptArtificial Intelligence
The gist
Thinking steps used by AI models can be written out in full or kept hidden in compact codes. The authors found that letting the model learn by matching these hidden codes to example reasoning steps helps avoid confusion and makes the AI think better. They guide this process with special anchors that keep the hidden codes organized in the right order. This results in shorter reasoning outputs that are more accurate than earlier methods across different model sizes.
Open 2609.09928v1