Prism framework measures and improves AI multimodal analogies
PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies
Artificial Intelligence
Summary
AI sometimes struggles to create analogies that keep the relationships between ideas clear, especially when using images and words together. The authors developed PRISM, a method that uses math (category theory) to measure how well these relationships are kept in AI-generated analogies. PRISM can also improve image-based analogies by giving feedback to the AI and making the pictures better match the intended relationships. Tests show that PRISM’s measurement matches well with human judgment and helps create clearer visual metaphors. Still, sometimes the improvement leads to busier images instead of deeper connections.
What this means in practice
- •For multimodal ai developers: Integrate a structural measure and refinement loop to improve analogy quality in AI systems combining images and text.
- •For creative content designers: Use an interpretable feedback system to refine AI-generated visual metaphors for clearer and more appropriate analogy-based designs.
Authors
Mirella Zeisler, Ojas Shirekar, Mircea Licǎ, Chirag Raman
Abstract
Analogical reasoning involves identifying and preserving relational structures across domains. However, existing approaches to AI-driven multimodal analogy generation lack an interpretable measure of whether this structure is understood and maintained in the generated output. We address this gap with Pullback Refinement via Interpretable Structural Mapping (PRISM), a modality-agnostic framework for measuring and improving relational alignment in multimodal analogies, evaluated on visual metaphor generation. PRISM represents analogies as explicit relational mappings grounded in category theory and uses VLMs to instantiate these structures across modalities. Its first component, the pullback score, quantifies relational alignment from the resulting graph representation. On the AnaloBench benchmark, selecting the correct analogy purely by pullback score achieves 82.5% accuracy, demonstrating that the score captures meaningful relational information. PRISM's second component is an iterative refinement loop that uses the pullback score as an in-context feedback signal to iteratively revise the generated image towards greater relational depth. VLM-as-a-judge and human evaluations show that PRISM consistently improves metaphor consistency and analogy appropriateness over zero- shot generation, with human participants preferring the refined output in 57.65% of pairwise comparisons. However, a qualitative analysis reveals that refinement can favour visually crowded compositions rather than genuinely deeper relational correspondences.