Papers for

assembly line quality teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Procedural mistake recognition improves by linking video to language

Through a PRISM, Clearly: Step-Sequence Reasoning for Mistake Recognition in Procedural Videos

Abstract: Procedural mistake recognition requires identifying deviations from valid multi-step activities. We study post-completion recognition from videos with benchmark-provided step boundaries. We introduce PRISM (Procedural mistake Recognition via Interpretable Semantic Modeling), which grounds video segments in language and maps their descriptions to procedural states using a lightweight hidden Markov model trained on correct executions. Reference alignment and semantic checks identify errors in step content and order. To support evaluation across heterogeneous benchmarks, we map dataset-specific annotations to four edit-based error types: insertion, deletion, substitution, and transposition. Across three benchmarks, we compare gold and VLM-generated descriptions in raw, semantic-role, and action-object forms. Decoder controls show consistent gains from the trained local decoder over nearest-candidate matching, with additional, dataset-dependent benefits from sequence decoding. Semantic role labeling improves Any-F1 in all six matched grounded comparisons, although the gains vary across datasets and do not translate into uniform improvements under separate configuration selection. These results establish language grounding as an effective interface for explicit procedural mistake reasoning and characterize the contributions of representation and decoding.

Thu 22 OctComputer Vision and Pattern RecognitionComputation and Language
The gist
Recognizing mistakes in multi-step tasks from videos is hard because it requires checking if each step was done correctly and in order. The authors introduce PRISM, a system that describes video steps in language, then uses a model trained on correct examples to find errors. They show that careful use of language descriptions helps spot mistakes across different datasets. This approach improves detecting various types of errors like missing steps or steps done in the wrong sequence.
Open → 2610.31117v1