Recent sensor technologies provide a vast amount of new possibilities and developments in the field of music. But one area is nearly untouched so far: Musical instruments.
Although it might sound a little bit unromantic, playing and exercising a musical instrument and making music is a difficult task, similar to training in sports. In the latter, technology based training assistance is already common. To rise supportive technologies for musical instruments, several projects and experiments are carried out in our laboratories. With several technologies measurements are conducted to receive objective and individual data of musicians and live performances. With these, investigations regarding e.g. coordination, fatigue and general performance monitoring is possible.
Beside this, new interfaces, especially based on sensor technologies are developed and evaluated. This could lead to new musical interfaces and feedback methods or to augmented traditional musical instruments. (Contact and recent publications click here).
From a technical point of view, instrumental music making involves mainly audible and visible playing parameters. Parameters like force, pressure and fast movements, happening within milliseconds are particularly much more difficult to capture. Our research is focusing on these parameters additional to movement and audio analysis. To achieve this, several sensors are integrated unobtrusively into and fixed on the musical instrument. The evaluation of these new sensor setups is done with amateur musicians, students of the ZHDK and professionals. A further goal of the use of sensors is a more simple integration of conventional musical instruments into electronic music environments by converting the data to MIDI and OSC.
Audio-Haptic Modalities in Musical Interfaces
Sensors for Stringed Instruments
Sensor Technologies for Drums
Sensor Technologies for Keyboard Instruments
Sensor Technologies for Wind Instruments
Exercising is a daily routine in learning a musical instrument. It is one of the most important tasks but very little attention is drawn to it. But there is a high potential to increase the efficiency and to learn faster. Beside many others, this could be obtained/reached with new sensor and feedback technologies.
The following overview shows the latest of our developments in sensing different parameters for most musical instruments:
Fatigue and Malposition Detection
Musicians’ practice overly long placing themselves at the risk of performance and practice shortcomings and bodily pain. With a set of sensors consisting of 6 or 9 DOF IMUs fixed to the instrument, the orientation and movement of the violin was measured. We compared individual perceived fatigue to the motion of the body and the instrument. In short, professional musicians showed more stability regarding their posture and more endurance during a one hour practicing session. More details see at: Paper fatigue.
Synchronization (Music Groups)
Several experiments were conducted with two or more musicians playing together. In a violin duo, the timing between the bow and finger changes for each player were captured. Similar measurements are done with trios and quartets. The synchronization in professional music groups was constantly below 15ms, amateur groups showed delays up to 100ms. More info see at: Paper synchronization.
Multi-modal Real-time Feedback
Several feedback modalities are implemented to communicate with the musician while playing. Visual feedback is shown in Figure above, the background color changes, if a certain event occurs, but also haptic feedback is integrated into e.g. violin bows to indicate cramping or wrong finger positions.
Beside the pedagogical applications, the technologies enable musicians and composers to augment the traditional musical instruments. The main advantage is the low latency data capturing of the above mentioned parameters (finger/hand position and pres-sure, etc.) and sending them in real-time via osc or midi.
• New parameters for musical expression: Several parameters e.g. finger pressure, fin-ger position can be mapped to any sound re-lated event like panning, volume, effects, MIDI..., or to any other objects like light or laser shows (e.g. via DMX).
• Low-latency MIDI or OSC-Interfaces on nearly any traditional musical instrument.
• All Technologies can be integrated into electrical instruments (e.g. E-guitar) or silent
Diese Website wird in älteren Versionen von Netscape ohne graphische Elemente dargestellt. Die Funktionalität der Website ist aber trotzdem gewährleistet. Wenn Sie diese Website regelmässig benutzen, empfehlen wir Ihnen, auf Ihrem Computer einen aktuellen Browser zu installieren. Weitere Informationen finden Sie auf
The content in this site is accessible to any browser or Internet device, however, some graphics will display correctly only in the newer versions of Netscape. To get the most out of our site we suggest you upgrade to a newer browser.