Music-touch Shoes: Vibrotactile Interface for Hearing Impaired Dancers

Music-touch Shoes: Vibrotactile Interface for Hearing Impaired Dancers Lining Yao, Yan Shi, Hengfeng Chi, Xiaoyu Ji, Fangtian Ying Department of Industry Design Zhejiang University Hangzhou, 310027, China {ylnhher, hzshiyan, 191744498, ghjxy1987, yingft} @gmail.com ABSTRACT

handicapped dancers cannot listen to the music and feel the rhythm when dancing. To them, the dancing world is soundless, and what they have to do for most of the time is remembering the dancing movements and speed by rote. Therefore, the process of dance learning becomes boring and difficult without hearing music.

The hearing handicapped children show a penchant for dancing and dance plays an essential part in education of deaf children. This paper introduces the Music-touch Shoes, a pair of shoes particularly designed for hearing handicapped dancers: The rhythm and tempo of music can be communicated and perceived through the vibrotactile interaction. The vibrotactile interface is applied to shoes because feet are among the body parts which are most directly involved in performing dancing rhythm. The different sequences, intensity and frequency of vibrations reflect different rhythm and tempo of music. This project sought to explore a way of making up for the shortage of hearing ability through interaction with other senses, such as vibrotactile sense, to fulfill the same dancing entertainment demand of the hearing handicapped people. Author Keywords

There is an evolutionary mechanism that allows people with the deprivation of a sense to adapt and compensate by using other senses [3]. Dean Shibata pointed out that deaf people sense vibration in the part of the brain that other people use for hearing [4]. These findings suggest that the experience deaf people have when 'feeling' music is similar to the experience other people have when hearing music. Thus, the Music-touch Shoes, embedded with vibrotactile interface, is for hearing handicapped dancers to feel the rhythm and tempo of music through variable stimulation signals in soles.

Vibrotactile interface, hearing impaired dancers, music rhythm and tempo

RELATED WORK

Geldard first introduced the notion of applying vibrotactile stimuli to the skin, and suggested that people could learn an invented tactile body language called vibratese [5].

ACM Classification Keywords

H5.2 [Information interfaces and presentation]: User Interfaces.

Gunther's SkinScape [6] used vibration devices distributed throughout the body to enhance the audio experience by immersing audience in musically synchronized tactile compositions. Further, vibrotactile interface is used for the purpose of navigation, like the ActiveBelt [7] and IDEOs concept about technojewlery called GPS Toes [8].

General Terms

Design, Human Factors INTRODUCTION

Compared with our care of their health and education, spiritual entertainment demands of the hearing handicapped are much less concerned. Karen Bond’s research shows that deaf-blind children whose access to culture is minimal show a penchant for dancing [1]. An increasing number of hearing handicapped girls fall in love with dancing and even make it their career. Also, Dance has an enormous part to play in the education of deaf children [2]. While dance and music are intimately linked [2], the hearing

CONCEPT

In this project, a pair of dancing shoes could be controlled by a software on PC via Bluetooth. Through the software system, we can choose music from computer and internet, upload the music into the shoes, and set the starting time of the music if needed. Once started by heels lifting or wireless software control, the Music-touch shoes with three vibratile points could give users vibrotactile stimulation according to the rhythm and tempo of the chosen music. While tempo is reflected by vibration’s sequence and intensity [see Table 1], rhythm is reflected by vibration’s frequency.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. TEI 2010, January 25–27, 2010, Cambridge, Massachusetts, USA. Copyright 2010 ACM 978-1-60558-841-4/10/01...$10.00.

275

Common tempo basic temp o

Vibration sequence

Vibration intensity

2 beats

a-b

Strong- Weak

3 beats

a-b-c

S- W-W

4 beats

a-b-a-b

S- W- S- W

5 beats

a-b-a-b-c

S- W- S- W-W

6 beats

a-b-c-a-b-c

S- W-W- S- W-W

Lily (dancing): “I can feel the rhythm naturally now!” Scenario 2: Begin at the Same Time with Music for Dancers on Stage

This scenario describes the interaction when a group of deaf dancers are on the stage for performance. The software can be used to control the play of coherent audio music and pairs of shoes to start work at the same time, thus group of dancers can dance at the same tempo.

Table 1. Vibration’s sequence and intensity reflect the tempo

One of the pair of shoes includes three flat micro vibrating motors. Audio decoder extracts discrete data recording the waveform of music, then MCU determines the tempo points, calculates the rhythm of music, and controls the vibrating motors in real-time. Thus, the frequency, intensity and sequence of vibration felt from the soles are related to the rhythm and tempo of the music. The calculation of rhythm and tempo is based on the characteristic of music, that is, different music has different frequency and amplitude.

CONCLUSION AND FUTURE WORK

For the accuracy of abstracting the music wave, and convenience of practicing dance without loudspeaker, we upload digital music into the Shoes’ system. After a primary user test, we find that sometimes it is possible that the hearing handicapped dancers will dance together with a normal person, when loudspeaker is used. Audiotactile transducer for generating electrical signals from outer sound waves will be added into the system at the next step.

Distance sensors are embedded in both shoes, to insure that the dancer can start the Music-touch shoes by easily lifting two heels for more than one second to a certain height.

Also, we used three flat micro vibrating motors in the soles, which makes the soles a little thicker than the normal ones. We would try electrical stimulation to see if it can make the soles thinner. After all, flexibility and lightness are important for dancers. REFERENCES

1. Karen E. Bond, Susan W. "I feel like I'm going to take off!": Young people's experiences of the super ordinary in dance stinson. Dance Research Journal 32, 2 (20002001), 52-87 2. Naomi Benari. Inner Rhythm: Dance Training for the Deaf. Harwood academic publishers (xv-xix)

Figure 1. One shoe includes three flat micro vibrating motors SCENARIO

3. http://en.wikipedia.org/wiki/Sensory_substitution

The usage is divided into two scenarios.

4. Dean K. Shibata and Edmund Kwok. Temporal lobe perfusion in the deaf: MR measurement with pulsed Arterial Spin Labeling (FAIR). Academic Radiology Volume 13, Issue 6, June 2006, Pages 738-743

Scenario 1: Slide and Feel the Rhythm!

This scenario describes the interaction with the Musictouch shoes when only one deaf dancer use them. There is no need for audio music played at the same time because the dancer cannot hear it. Different music can be uploaded from computer.

5. Geldard, EA. Some neglected possibilities of communication. Science, 131,1960, 1583- 1588. 6. Gunther, E. SkinScape tool for composition in the tactile modality. Masters of Engineering Thesis in the Department of Electrical Engineering and Computer Science, MIT, 2001 7. K. Tsukada and M. Yasumrua. Activebelt: Belt-type wearable tactile display for directional navigation. In Proc. of UbiComp2004, Springer(2004), 384–399, Berlin, Germany 8. GPS Toes Rings by IDEO. http://www.ideo.com/portfolio/re.asp?x=50165

A deaf dancer Lily: “I can start the shoes by lifting both heels at the same time for more than one second.”

276