In i musikens famn

In i musikens famn

Du är proffsmusiker men gravt synskadad. Framför dig spelar en symfoniorkester, men du kan inte vara med. Tills nu. Projektet utvecklar en ny teknik för synskadade musiker att kunna läsa av både rörelse och mimik på dirigenten, navet i orkestern.  En ny värld öppnas. Ellen Sundh, kreativ programmerare bakom succéprojektet The Sound of Football, Ric Wasserman filmare, och Jan Bengtson, flöjtist i Kungliga Filharmonikerna samarbetar med Wu Jing, en synskadad professionell flöjtist utvecklar tillsammans ett system för att bereda vägen för synskadade att spela tillsammans med orkester.

Projektblogg

The sensory system

The sensory system

The body has different resolution on different places of the body. Earlier technology aimed to help visually impaired “see” has used the tongue as a sensory input. The Brainport uses 400 electrodes place on the tongue. As Wu Jing is playing the flute, her tongue is highly occupied and used. The fingertips have one of the most sensitive resolutions on the body and is used for reading Braille. Wu Jings fingers are also engaged while playing as she use them on the flute. Looking at the resolution of the body’s different sensory inputs we can choose another part of the body with high resolution. Below is listed the nerves on the body and the sensitivity and resolution.

Sound as input

Sound as input

Many aids targeted to visually impaired people uses sound as an input signal as the hearing is much more developed for people who can’t see. Ellen’s earlier experience from projects with visually impaired aids have used sound like The Sound of Football. Though we could use bone conducted aids to transfer the conductors signals to the blind musician the sound intake is already stretched with sound from the orchestra, the musicians own sound and the sound of the group the musician sits with in the orchestra. Hearing the breath inhale of the fellow fluteists in the same group also communicates to the blind musician what is happening in the that sequence of the classical piece. 

The task

The task

The task to transfer the signals from the conductor to Wu Jing requires a system to register the conductors gestures and transfer these to the body of Wu Jing. As a first test to register the conductors signals we use an infrared camera that detects the depths in which the conductor moves. We aim to use a camera on the conductor so that it does not require any additional accessories to be place on the conductor. This way any conductor with a stick or not can use the technique and move their hands and arms as they usually do without any extra weight.