Postdoctoral researcher, Academy of Finland Fellow, University of the arts, Sibelius Academy, Helsinki, Finland
PhD (2010), Université Paris VIII: An approach to Instrument Augmentation: the Electric Guitar
Current Projects (2017):
« Active Acoustics – Smart Audio objects at the Interface of the Material and the Digital » Academy of Finland Key Project (Principal Investigator)
« Active Acoustic Augmented Instruments » Nordisk Kulturfond Handmade Project (Principal Investigator)
1) Structure-Borne Sound for Music and Intermedia Creation
The object of the my ongoing postdoctoral project is to investigate and tap the potential of structure-borne sound as a medium for musical expression. Structure-borne sound is defined as audio-rate vibration induced into a solid element, giving rise to air-borne sound diffusion as well as to audiotactile perception when the vibrating element is brought in direct physical contact with the body. Structural audio vibration is an everyday phenomenon. It is omnipresent in music, for instance via the vibrating soundboards of chordophones and percussions. However, the electronic music praxis has almost exclusively adopted the moving-coil loudspeaker as its tool for sound diffusion. Structure-borne sound presents numerous possibilities for transforming a range of solid objects into sound-emitting speakers and thus a radical alteration of the established sound practice in music, theatre, installation art and intermedia forms. Sound can be induced into the structures and furniture of the performance space with structure-borne sound drivers, producing a macro-scale « instrument ». Sonically active architectures can also be designed, with altered acoustics or noise-cancelling properties. Audiotactile perception can be added to a concert setting, introducing a new perceptual dimension to the musical experience.
This line of research merges issues of artistic creation, sound perception and sound driver technology in order to create a fertile research dynamic between art, science and technology. The two central research questions are the optimization of structure-borne sound quality for musical use, as well as the combination of vibrating solid elements, audiotactility, acoustic instruments and loudspeakers into « composite aural spaces », i.e. spatial instances of multiple sound-source audio.
O V A L ; Audiovisual installation using structure-borne sound on glass, Otso Lähdeoja & Lenka Novakova, 2013
This video is about research conducted at the Numediart Institute at the Mons University, Belgium. We worked on structure-borne sound, making physical elements resonate and produce sound. In this example, a glass sheet is set in vibration with a resonator (the sound is recorded directly form the glass with a mike). A kinect camera is used to implement interaction with the sound in the spirit of a « sonic cartography ».
2) Augmented Instruments & Active Acoustics
Augmenting traditional instruments with new sonic possibilities pursues the dream of combining the best of both worlds: instrumental sensitivity and virtuosity embedded in our musical traditions coupled with the vast sonic possibilities of today’s signal processing, synthesis and spatialisation techniques. Since my PhD, I have worked extensively on my main instrument – the guitar – seeking for numerous ways to integrate electronics on it and to find natural, embodied ways to control the electronic sounds.
I have developped interfaces using sensor technology as well as gesture recognition via feature extraction from the guitar signal.
In 2015 I initiated a new augmented instruments project named Acoustronics, using active acoustics (structure-borne sound) to drive electronic sounds directly into an acoustic instrument’s soundboard. The result is a seamless integration of acoustic and electronic sound into a single instrumental entity. The acoustronic intrument portrays an integrated, natural-sounding acoustic image and a strong sense of embodiment since the electronic sounds are part of the actual physical istrument. In the near future, the acoustronics project will integrate Machine Learning and Physical Computing in order to transfer the Human-Computer Interaction completely from a symbolic domain to a haptic and embodied one.
The Acoustronic Guitar: Electronic sounds driven into the guitar’s soundboard. All the sounds emanate directly from the guitar, direct recording, no external speakers, no post-processing.
This video shows some features of the augmented guitar I designed during my PhD. It uses tilt, touch and piezo sensors in order to implement new sonic and interactive possibilities for the electric guitar. I used this guitar to record my first solo album; Yonder.
The « Guitar as Controller » project, Numediart Institute. Real-time computer detection of guitar playing techniques.
3) Embodied (Music) Cognition
A transversal underlying theme to all of my research work is a profound interest in human meaning-making and cognition. I have a background of study in Music Semiotics which matured into an enthousiasm for the Embodied Cognition Theory and its numerous implications in music and Music Interactivity Design. In my understanding, music-making is an embodied phenomenon emerging from a joint and unseparable activity of the perceptive, sensori-motor and abstract cognitive systems.
The Embodied cognition paradigm creates formidable challenges for Digital Musicianship. Digital media and digital computation are symbolic operations and thus ontologically related to textuality, symbolic represeantation and abstraction. On the other hand, a great part of music-making is arguably not symbolic in nature but rather direct physical engagement into sound, understood and enacted non-verbally. I tend to think that there is a paradigm clash between computation and embodie music making and that it is presently critical to identify and address the fundamental issues hindering the development of live (digital) electronic music cultures. In my opinion, computation is yet underdevelopped to cope within such a complex, real-time, human and essentially embodied environment. Progress needs to be made and I think the response lies in the developpement of AI and Machine Learning.
4) Digital Humanities (Post-Media Studies)
Digitality has become pervasive in our society, transforming our entire culture in just as little as a quarter of a century. Within this general movement, music has undergone – and is still undergoing – huge transformations redefining all of its related practices: music composing, playing, listening, diffusion, consumption… I have a growing interest for the big picture of the dawning digital age. As the first digital native generations mature into adults we see the emergence of a digital culture in a ripened form. The new culture is astonishing and of course an endless source of controversy for us analog native dinosaurs. Nevertheless, digitaity must be studied and its critic must be done in order to advance into the next phase. Within the art world, the dominant paradign of the « electronic arts » have been the celebration of techno-scientific phenomena, producing countless artworks which have for their sole content the presentation of a technology that works « like magic ». I think it is now time to move on and infuse humanly, socially and esthetically relevant questions into the game.
I use the term « post-media » to name the current condition where technology is everywhere, pervasive, part of everyday life and does not produce reactions of awe. Technology needs to be rethought of as a general building bloc of our environment; its role in personal, social and general advancement and welfare redefined.
My research work is inherently interdisciplinary and grounded in the research-creation methodology. Research-creation is defined as creation of knowledge and innovation through artistic expression, scholarly investigation, as well as experimentation. The different disciplines are brought into a mutually fueling relation: the artistic creation stirs new scientific questions, and advances made in the scientific field open new possibilities for artistic experimentation.
I have worked in and with the following disciplines:
Arts; Music, Visual Arts, Dance, Cinema
Humanities; Semiotics, Musicology, Philosophy, Anthropology
Science/Technology; Acoustics, Psychoacoustics, Sensor Technology, Computer Sciences
Publications on google scholar