ClayFlect blends clay art and AI to shape personal meanings
"I'm Very Happy for It to Start Hallucinating a Little Bit": Using ClayFlect to Negotiate Multimodal AI Representations in Material Meaning-Making
Human-Computer Interaction
Summary
People often rely on AI tools to help think about their feelings and ideas, but they want to keep control over what meanings they give things. The authors created ClayFlect, a system that mixes making things with clay, talking, and AI images to help people reflect. They found that people didn't just accept AI’s ideas but compared and changed them while working with clay, which gave them hands-on control. This way, AI suggestions were like helpful hints, not final answers, letting people explore and reflect in their own ways.
What this means in practice
- •For mental health practitioners: Use ClayFlect-style systems to combine tactile art therapy with AI-generated prompts for supporting client reflection without overriding personal interpretation.
- •For interactive experience designers: Create hands-on creative environments that integrate material making with AI conversational and visual aids to foster personal meaning development.$Commercial implications: Enables design of novel interactive art or therapy products mixing physical materials and AI, appealing to users seeking reflective creative tools.
Authors
Kellie Yu Hui Sim, Quoc-Nam Nguyen, Shuenn Yuen Han, Kenny Tsu Wei Choo
Abstract
As AI enters reflection and emotional support, understanding how it can participate in personal meaning-making while preserving users' authority over interpretation is increasingly important. We present ClayFlect, a novel MLLM-powered system integrating tactile clay-making with conversational and visual generative AI, and report a mixed-methods study with 50 participants. Reflection developed across material, conversational, and generated forms rather than through AI interaction alone. Participants treated AI representations as provisional: they compared, redirected, reinterpreted, selectively incorporated, or left them aside as their artefacts and meanings evolved. Clay provided a directly manipulable space in which participants could continue developing meaning independently of the AI, while generated representations externalised possibilities beyond what they could readily make or visualise. We show how generative AI can participate through representations that remain negotiable, and derive implications for supporting movement across representations, preserving parallel sites of control, and allowing AI support to recede or deepen as reflection unfolds.