CrowdForge shows audio visual controls boost audience engagement in live music
CrowdForge: Effects of Audio and Visual Audience Participation Modalities on Agency, Collaboration, and Engagement in AI-Mediated MR Performances
Human-Computer InteractionArtificial IntelligenceNetworking and Internet ArchitecturePerformanceSocial and Information Networks
Summary
Getting audiences to participate in live music shows is tricky because they want to feel like they really influence what happens. The authors designed CrowdForge, a system where crowds can control music and visuals together during a concert using AI and mixed reality. They found that showing clear visual feedback helps audiences feel more in control, while audio controls let them try bold changes that challenge the musicians. This back-and-forth pushed creativity and made the shows more engaging.
What this means in practice
- •For live event producers: Design performances with combined audio and visual audience controls to increase engagement and creative interaction during concerts.
- •For interactive media developers: Create mixed reality systems that provide clear visual feedback to audiences to boost perceived agency in live collaborative shows.
Authors
Torin Hopkins, Shih-Yu Ma, Zander Trafton-Hancock, Ellen Yi-Luen Do, Luca Turchet
Abstract
Technology-mediated audience participation (TMAP) enables audiences to influence live performances, yet balancing interaction with meaningful agency remains challenging. While AI-generated music and MR enable new forms of audio and visual audience participation, little is known about how these participation modalities influence audience experiences during live musical performance. In particular, it remains unclear how they shape perceived agency, collaboration, and engagement, and how they complement one another within TMAP systems. We present CrowdForge, an AI- and MR-mediated paradigm where audiences collectively control generative music and virtual environments during a live performance featuring a guitarist and DJ. Through a mixedmethods concert study evaluating three interaction conditions (audio, visual, and combined audio-visual), we demonstrate that observability is the primary driver of audience agency. While visual interaction provided easily observable feedback, audio parameters offered macro-structural control over the music. Audiences tended to use the control enabled by AI audio generation to test musical boundaries, introducing a productive adversarial tension between the crowd and the musicians. This dynamic pushed creative limits, deepened engagement, and generated moments of musical coherence when performers successfully resolved audience-induced challenges. We argue future TMAP systems should prioritize human-centered, highly observable interactions to build agency, while combining the expressiveness of music with the clarity of visual feedback.