Multimodal system improves tobacco law retrieval by jurisdiction
PRISM-RAG: Multimodal Hypergraph Retrieval-Augmented Generation for Tobacco Product and Legislative Policy Reasoning
Computer Vision and Pattern Recognition
Summary
Different states have their own tobacco laws that can sound very similar but mean different things. This makes it hard for computer programs to find the right laws for a tobacco product in a specific state. The authors created a big collection of tobacco product images, descriptions, and laws from 13 states, plus questions with verified answers. They also made a new computer system, PRISM-RAG, that uses images and text together and carefully picks laws from the right state, improving accuracy a lot compared to older methods.
What this means in practice
- •For legal technology developers: Create tools that accurately match tobacco product images to the correct state laws for regulation and compliance checking.$Commercial implications: Enables software products that help legal and compliance teams monitor tobacco product regulations across states by linking images to precise legal text.
- •For regulatory compliance teams: Quickly verify if tobacco products meet specific state regulations by using image and text data that is guaranteed to come from the right jurisdiction.
Authors
Manuel Serna-Aguilera, Raegan Anderes, Page Dobbs, Khoa Luu
Abstract
The disambiguation of semantically similar statutory text across jurisdictions is a retrieval problem that existing methods do not solve. This inter-context conflict can steer generative models toward confidently produced answers grounded in topically relevant but jurisdictionally incorrect sources. Tobacco and nicotine regulations vary by US jurisdiction, often sharing similar language, thus, robust reasoning requires identifying which jurisdiction's law governs a given product, not merely retrieving relevant text. Emerging products (e.g., pouches) exploit ambiguous definitions to evade regulation. State-of-the-art (SOTA) document retrieval-augmented generation (RAG) methods struggle to address this inter-context conflict, and thus struggle to connect image attributes (e.g., rich attribute captions) to the set of similar legislation texts. We introduce NicoPRISM (Nicotine Product and Regulation Image-and-Text Surveillance Multimodal), comprising 161,563 images, attribute captions, a knowledge base of product, health, and legislative documents spanning 13 US jurisdictions, and 1,495 validated question-answer pairs across two tasks: policy compliance QA and product knowledge QA. We also propose PRISM-RAG, a multimodal hypergraph RAG framework built over images, captions, and entities without any LLM calls at index time, grounding every query in a product image and routes retrieval through a jurisdiction-aware context assembly mechanism guaranteeing that statutory text from the queried jurisdiction reaches the language model by construction. PRISM-RAG retrieves passages from the correct jurisdiction in 93.9% of policy compliance queries, a 48.6 percentage point advantage over standard RAG (p<0.001), using zero LLM calls at index time and one at query time, and is competitive with or outperforms SOTA RAG frameworks across keyword, semantic, jurisdiction-, and compliance-accuracy metrics.