Summary
Musical instruments can be enhanced with sensors and computer programs to make new sounds, but these changes can also limit how musicians naturally play. To explore this, the authors worked closely with a professional drummer to create a system that recognizes drum gestures in a flexible way while recording music together. Their approach balanced the musician's artistic style with the limitations of technology, leading to new insights about designing such tools. They found a type of intuition called "knowing-when" helps musicians work well with digital tools. The study raises a question about how to tell if technology truly supports musicians when we don’t understand their social environment.
digital musical instrumentsgesture mappingcontinuous gesture recognitionproductive dissonancetechnological capturesensor augmentationtacit knowledgepercussionmusical expressionpractice-based research
Authors
Jordie Shier, Teresa Pelinski, Charalampos Saitis, Andrew Robertson, Andrew McPherson
Abstract
Augmenting instruments with sensors and neural network mappings is a well-explored digital musical instrument design approach. While augmentations can create new expressive opportunities, they also exert aesthetic influence and can constrain musicians' gestural language, which, if left unchecked, can lead to technological capture. To examine this, we conducted a study with a professional percussionist, co-developing a gesture mapping toolkit and recording a ten-track album. Drawing on the concept of productive dissonance, our study aimed to hold the musician's aesthetic in tension with technological constraints. This, along with a practice-based reflective approach, supported the development of a continuous gesture recognition method for percussive mapping and surfaced insights into the design process. We identify knowing-when as a form of tacit knowledge that supported productive dissonance, and raise an open question: absent a musician's broader social context, how do we know whether a technology's influence is genuinely supporting their practice?