Enabling Multimodal Communications for Enhancing the Ability of Learning for the Visually Impaired Francis Quek
David McNeill
Francisco Oliveira
Center for Human Computer Interaction Virginia Tech
Center for Gesture and Speech Research University of Chicago
Center for Human Computer Interaction Virginia Tech
[email protected]
dmcneill@uchicago. edu
[email protected]
Keywords
behavior of gesture, gaze, posture, and facial expression are brought into the service of the communicative process. The communication of mathematical concepts seems especially to engage such co-speech behavior. The extent to which one’s interlocutor is aware of such embodied behavior and utilizes it to maintain the interaction and comprehend the material conveyed is still an open question. We use the loose sense of ‘aware’ not to indicate that one is ‘consciously attentive to’, but only that one is able to derive information from the behavior,whether one is fully conscious of the behavioral carrier of the information or not. Students who are blind, in particular, have the capacity to conceptualize and access mathematical material (at their own level of competence), but have no visual awareness of a math instructor’s embodied behavior. In this paper, we present a phenomenological analysis of the communicative and cognitive aspects relevant to mathematical instruction to the blind. From this, we advance the conceptual framework, and lay the groundwork for a multimodal embodied approach to mathematics instruction for the blind. To do so, we have to establish three key principles. First, we show the inherent embodiment of mathematics conceptualization and communication. Second, we suggest that the blind have the capacity for mathematical visualization and spatialization. Third, we raise the questions of uptake of embodied communication in the understanding of mathematics instruction and discourse. Armed with these, we suggest an approach to enabling the blind to have access to critical components of embodied mathematics instruction.
Multimodal, awareness, embodiment, gestures
2. A Motivating Example
ABSTRACT Students who are blind are typically one to three years behind their seeing counterparts in mathematics and science. We posit that a key reason for this resides in the inability of such students to access multimodal embodied communicative behavior of mathematics instructors. This impedes the ability of blind students and their teachers to maintain situated communication. In this paper, we set forth the relevant phenomenological analyses to support this claim. We show that mathematical communication and instruction are inherent embodied; that the blind are able to conceptualize visuo-spatial information; and argue that uptake of embodied behavior is critical to receiving relevant mathematical information. Based on this analysis, we advance an approach to provide students who are blind with awareness of their teachers’ deictic gestural activity via a set of haptic output devices. We lay forth a set of open research question that researcher in multimodal interfaces may address.
Categories and Subject Descriptors H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6) - Haptic I/O
General Terms Multimodal interfaces, Situated discourse, embodied deictic activity, spatio-temporal cues, growth point, catchment, embodied awareness, mediating technology.
1. INTRODUCTION The foundations of multimodal interfaces lay in the facility of human users with the simultaneous diverse nteractive streams. This facility, in turn, is rooted in the fact multifacetness of human embodied sensing, perception, cognition and action. Humans are embodied beings. When we speak, our embodied Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ICMI’06, November 2--4, 2006, Banff, Alberta, Canada. Copyright 2004 ACM 1-59593-541-X/06/0011…$5.00.
To motivate our ensuing discussion, consider a teacher explaining the concept of a sine waveform to sighted high school students using the locus traced by the elevation of a circumference point of a rotating unit circle 1 . She utilizes a graphic (see Figure 1) and gestures as she discusses the function. The teacher may say (pointing gestures shown as superscripts, and the duration of the deictic is marked by brackets): “The [sine function]A {points at the sinusoid} traces the height of the end of a [rotating arm]B {points at B in the figure – Note that the italicized deictic markers do not appear 1
The first author of this paper engaged with an experienced mathematics instructor for a school of the blind and a mathematics tutor for the blind college students to derive insights that are contained in this scenario.
on the illustration used by the teacher} as it swings around [a circle]C {deictic gesture tracing the circumference of the circle in the counterclockwise direction}. When the arm is [at zero
3. About Mathematics Discourse The theories of the ‘growth point’ and the ‘catchment’ have been developed for speech production. They will be extended
90 120
60
F 150
H
30
B 180
E 0 360
G 0
30
60
90
120
150
180
210
240
270
300
330
360
330
210 300
240 270
Figure 1. A Mathematical Sine Function Illustration degrees]D {points at the zero on the circle}, the [value of the here to speech reception, via analysis of math instruction uptake sine function is zero]E {points at E}. When the arm is [at thirty by blind students. A growth point is a psycholinguistic unit in degrees]F {points at F}, the value of the sine function [at thirty which linguistic categorial content and co-expressive gestural degrees]G {points at G} [is this]H {deictic gesture traces the imagery, taken jointly, occur at the same slice of processing time. Being co-expressive and synchronous, the two forms of path shown as a grey arrow} …” The teacher continues in her information – imagery and linguistic-categorial – can combine discussion showing that the arm traces the waveform shown. into a single processing unit. A growth point is a minimal unit of Although, for simplicity, this example is a rather contrived the thought process that becomes engaged during speech, and it monologue (as opposed to a more desirable discussion with the implies that imagery is an integral part of verbal thought (a great students), it serves to illustrate that math instruction discourse is deal of evidence has been gathered in support of this theory; see a complex social interaction involving the speech of teacher and [2-5]). A catchment is a related concept applying at the level of students, artifacts employed in the instruction like our graphical discourse – the realm beyond the single sentence. A catchment illustration, and the embodied behavior of both teacher and is a discourse segment based on gesture imagery. It is a students. Much of math conceptualization is intimately tied to collection of recurring gesture features that provide a thread of spatial reasoning and visualization. The dynamics of the spatial cognition tying together possibly discontinuous elements interaction between the teacher and students requires the of the discourse stream. A catchment can be seen to embody maintenance of a situated discourse stream that is at the same thematic content, and this content provides the unifying glue for time grounded with any illustration that may be employed. thought and discourse. The concept of embodiment awareness In our example monologue, the student would find it difficult to refers to the ability of the recipients of communications to take follow her teacher without access either to the graphic or the in the full range of multimodal elements of the communication, embodied deictic activity that is co-temporal with the speech specifically the coordinated speech-gesture components of stream. One can imagine the difficulty a visually impaired growth point and catchments. By studying blind students, we student may have in acquiring the sense of the function without can start to extend these hitherto production based models to visual access to either. This is possibly a key reason that visually reception. impaired students typically lag their seeing compatriots in math In this extension, reception is conceptualized, at least initially, and science education [1]. One might imagine employing as a kind of analysis by synthesis. A recipient generates her own multimodal communication approaches to provide the student growth points and catchments, in parallel with those of the with alternate spatio-temporal cues that seeing students derive producer whose communication she is comprehending (see [6]). naturally from the visual access to the embodied expressions of Such a process leaves an observational trace of comprehension. teachers.
Figure 2: Mathematics Professional’s gesture: “so the continuous linear dual” The recipient’s own speech-gesture productions can be studied, In the ensuing sections, we shall show that math discourse is both concurrently with the teacher’s and separately during selfinherently spatio-temporal, and that this information is carried guided math speech, as reflecting comprehension. by gesticulation in conjunction with speech. Individuals who are blind do not normally have access to this embodied interaction. Gesture in mathematics discourse: In “The Emperor’s New Even when math graphics/illustrations are displayed on media Mind”, Roger Penrose, mathematician and physicist wrote: that are accessible to the blind, this lack of embodiment “almost all my mathematical thinking is done visually and in awareness severely impedes the critical need to maintain terms of nonverbal concepts, although the thoughts are quite situated communication and comprehension. often accompanied by inane and almost useless verbal
commentary, such as ‘that thing goes with that thing and that thing goes with that thing’” ([7] p. 424). Penrose emphasizes the close psychological link between imagistic and mathematical thinking. The thinking is not entirely nonverbal, since all those “that things” could play a role in pointing to the references in an intricate line of thinking, especially with written expressions. Gestures also can pack imagery with math content, and gestural pointing can perform a function as “that thing” may have for Penrose. Our earlier studies [8] and new studies by Nathaniel Smith [9] on math gestures show that gestures can be imbued with math content and this phenomenon appears on all levels of knowledge, from professional to student. The remainder of this section applies these concepts to the speech and gestures of mathematicians, both professionals and advanced students. The aim is twofold. First, to demonstrate that math content is naturally expressed in spatial form, via gestures. Second, that catchments with math content play a role in math reasoning, by providing what can be termed ‘objects for contemplation’ that can foster math reasoning. The gesture becomes the embodiment of a math concept that can be observed by the speaker herself and also by recipients. Such gestures show the close linkage of visualization and math understanding, a linkage that we propose to tap in the case of blind children exposed to math instruction, drawing on the visual cognition abilities of even the congenitally blind. Gestures by math professionals: The mathematician on the right below is saying, “so the continuous linear dual”, and performs a circular loop with his right hand (4 panels, one gesture, coinciding with the boldface): Such a gesture demonstrates a growth point in math discourse. The analysis is that the idea of a dual exists in two simultaneous forms – as a linguistic category (“a continuous linear dual”) and an image (of circling), and implies that, for this mathematician at the moment of speaking, the math concept was embodied in this image as well as being identified as a dual.
and formed a new gesture in Figure 3-middle, when the source of the difficulty and its solution become clear to her. The hold served (successfully) as an object for contemplation. This example also shows a possible case of embodiment awareness on the part of the interlocutor. The interlocutor (not shown) was engaging in conversation with the illustrated student and reacted to the first student’s catchment-hold by saying “if it contains the point zero one” (line 7); it was this which triggered the illustrated student’s insight in line 8 (R is the illustrated student, L is the interlocutor): 1 L and it’s going to be the complement … to a closed and bounded set … R’s hands drift to her left which will be compact … 2 R rotates further to the left, still staring blankly at hands 3 L plus infinity … so 4 R what about, okay … except yeah, hands shift again, still held still and stared at (Figure 3-left). 5 L You see how it’s the complement to a closed and bounded set 6 R but, except that, umm 7 L if it contains the point zero one 8 R ][oh, yeah, because it has to contain something on the other side – hands are released from hold, move around to indicate places on line on both sides of zero one, two parallel hands to indicate both sides and so everything in the middle] both hands sweep out, scoop, and then back in again (Figure 3-middle).
Figure 3: Speaker says “on both sides of zero – and so everything in the middle”. Rightmost photo shows an earlier ‘catchmenthold’ as an ‘object for contemplation’ Gestures by mathematics students: Smith [9] describes many examples of gestures with math content in videos of advanced undergraduate and early graduate student math majors at Berkeley. The gestures demonstrate growth points, comparable to the example just described, and also provide examples of gestural catchments. Many are ‘catchment-holds’ in which the hand(s) move into the gesture space before the body, adopt a pose, and then remain motionless; the extent of the hold demarcates a period of thematic cohesion around some concept. Such catchment-holds provide ‘objects for contemplation’. Figure 3 shows two examples. The two illustrations on the left occurred as speaker endured a period of confusion during which she stared at her own gesture in the form of the two hands held apart motionlessly (Figure 3-left). This hold then was animated
An even more striking example of a catchment-hold providing an object for contemplation occurred on an earlier occasion (Figure 3-right), when the same student was writing notes, looked up from the page, formed a gesture that she experienced only proprioceptively (her gaze was not at the gesture), and then returned to her writing – all without a word. It is hard to escape the impression that the gesture embodied math content. It is noteworthy that the experience of the catchment-hold was entirely actional; an experience a blind student also could access. Main point. In general, at both the professional and student levels, we see that imagery is inherent to math reasoning, and that gestures play a part in this reasoning by creating images of math concepts that can serve as ‘objects of contemplation’.
Gestures can also make (albeit simpler) math concepts accessible to visually disabled students. Blind children are capable of visual imagery, and naturally create gestures. Deixis or pointing is an important subset of natural gestural behavior [8]. Pointing in particular can be useful for keeping track of the components of a problem. Having access to both visual imagery and gesture, blind students could profit from the capacity to reason with images, if the technological issues of presenting accessible imagistic forms of math concepts and math gestures are solved.
4. Mathematics Instruction and the Blind Our second key point to motivate the role of embodiment awareness mathematics instruction for the blind is that the blind have the capacity to visualize and comprehend inherently visuospatial mathematical concepts. We also show that the impediment resides not so much in the paucity of tactile instruction material (although greater access to such material would help), but in the inability of the student to engage in situated mathematics discourse.
4.1 “Visualization” and Spatialization Numerous psychological studies show the blind to possess a surprising capacity for visual imagery and memory (e.g., [1013]) – a capacity that can be harnessed for mathematical thought. Kennedy observes even a 3D drawing capability, as in the accompanying example scanned from p. 109 of his book (it depicts three views of a table – from the top, the side and
Figure 4: Drawings of a table by Ray (late, totally blind). The top form is a table from above, the middle form is a table from the side, and the lower star-like form is a table from underneath. directly underneath: perspective is evident in the latter two views – see Figure 4). Blind children also produce gestures, including gestures to other blind children [14]. An example is tilting over a C-shaped hand in midair while describing how a liquid had been poured into a container – not a math concept but a gesture not unlike those in math discussions. Research using tactile displays even indicate that the congenitally blind is able to understand perspective
graphics [15].
4.2 Instructional interaction in Mathematics and Science Education Given the role of visual imagery and memory in math and science, the maintenance of grounded instructional communication between the instructor and student who is blind is a significant challenge. We shall illustrate this challenge by describing a characteristic scenario where an instructor discusses the sine wave in Figure 1 with a class of four in a school for the blind. The teacher begins by having the instruction material (graphic) reproduced as a tactile image. Figure 5 shows one such graphic. Typically, the graphic is first scanned and enlarged. Next, the enlarged graphic is edited on a computer to insert Braille annotations. Some systems employ a Braille extension specialized for mathematical notation know as the Nemeth Code [16]. The Braille-annotated graphic is transferred to a thermallysensitive paper and passed through a machine known as an image enhancer that heats the darkened pattern to make it rise. This produces a tactile-image as shown in Figure 5 where the darkened areas are raised. Returning to our sine wave example, the teacher might begin by first giving an overview of the graphic – that there is a x-y plot of an ‘up-and-down’ curve on the right side and a circle with radii shown at 30 degree intervals on the left side – and then giving the students some time to familiarize themselves with the illustration. The teacher may then say: “Find the zero radius on the circle on the left” [She gives students some time to do this and visits each student’s desk to see if the correct radius is found – if not, she physically positions the student’s reading hand on the appropriate portion of the graphic]. “This point corresponds to the ‘zero’ point on the x axis on the plot” [The students are expected to trace the corresponding point on the graphic]. “Now go to the 30 degree radius of the circle” [The teacher repeats the process to see if the students find the point and assisting as needed]. “Good … now the height of the end of the radius is projected to the 30 degree point of the x-y graph – find the 30 degree point and the height of the curve is the height of the 30 degree radius on the circle” [The teacher observes the motion of the reading hand of her charges and provides help as necessary]. The instruction continues with the teacher giving the students a sense of the sine wave. This example gives a sense of the communicative dynamic that must take place between the teacher and student, and difficulty in maintaining a situated communication flow with respect to the graphic that serves as the locus of the discourse. Clearly two elements are necessary. First, the communication requires the provision of the graphical material in a tactile format. Beside the tactile images described earlier, there is research in a variety of dynamic tactile display devices [17]. While there are still outstanding research questions for this in the domain of
Figure 5. A sample tactile-image used for instructing students who are blind.
disabilities/rehabilitation research, this is not the focus of our present proposal. The second necessary element is the need to provide situated-grounded discourse. This involves the instructor, student and the artifact. This area has been understudied, and is the focus of our proposal.
to the substance of the tutorial and not to the pragmatic task of trying to follow the disembodied cursor. This ‘disembodied’ cursor phenomenon is a constant reality for the visually disabled student.
5. Embodiment Awareness and the Comprehension of Mathematics Discourse
While the full range of iconic and spatial information contained in mathematically-grounded gesticulation is immense, we believe that the conveyance of a subset of such gesticulation involved with deixes around an artifact of instruction provides critical information to maintain situated mathematics instructional discourse. Given a graphical illustration to carry the iconic aspects of the discourse content, such pointing gestures represent a meaningful subset for implementation and investigation. The challenge, then, becomes the application of various tactile approaches to provide the blind student with cues for the embodied behavior of the instructor. Given the importance of the class of deictic gestures in math instruction, our approaches will focus on giving the student a sense of the deictic behavior of the instructor. The purpose of these is to facilitate discourse maintenance between the teacher and student when using a graphic for instruction. We define two foci that must be detected and represented. The first is the point of instructional focus, PIF. This is the ‘zero point’ of reference of the instructor’s discourse and may be detected by visually tracking the teacher’s pointing hand. The video stream of a camera trained at the graphic may be used to track the teacher’s deixis. In similar fashion, the student’s tactile point of access TPA, on her raised pin illustration may be tracked in video. This is the location of the student’s ‘reading hand’ on the raised pin graphic. We shall call the difference between the point on the student’s graphic corresponding to the PIF and the TPA is the focal disparity, FD. Hence, the FD is a two-dimensional vector from the student’s point of access to the current point of instruction focus. Our proposed modes of interaction are intended to inform the blind student of the FD (and hence permit her to track the PIF). We posit that the student may employ this to target her TPA, or if she has already familiar with the graphic, to mentally track the point of reference. In either case, the effect is that the common
We come now to the most problematic aspect of our conceptual argument. We know that mathematics communication and conceptualization have strong embodied components. Indeed, our basic hypothesis is that embodied communication plays a critical role in the complex spatio-temporal conceptualization of mathematics. We have shown that gesticulation and gaze play a critical role in communication between sighted individuals discussing math concepts. We have also shown the capacity of the blind for visualization and spatial reasoning. We posit that the interlocutor in such math discourse assimilates the visual information conveyed in this embodied behavior and fuses it with the information carried in the speech stream to comprehend the communication. If this is so, one might expect that deprivation of this embodied information in students who are blind contribute significantly to the difficulty such students have with math and science instruction. It is more difficult, however, to determine how much the recipient of mathematics instruction is aware of the embodied performance, and what role this awareness plays in comprehending mathematics discourse and instruction.
5.1 Embodiment as a “Visual Cue” Static media alone is not able to express the spatio-temporal communication of even the simple example given earlier. A touch-based medium necessarily requires the student to scan the document as she attempts to follow the flow of the instruction. An analogous impediment that one might place on a seeing student is that she follows the same instruction with one eye looking through a tube, trained only on the illustration being discussed (i.e. not looking at the instructor). In our sine wave discussion example for sighted students, we see that situated deixes into graphic are necessary to maintain discourse situatedness. We posit that sighted students employ the entirety of their experience in observing human multimodal communication in classroom and tutorial engagement. The student observes the location of the instructor, and the movement of her arms/hand/writing implement as a cue of where to look, and how the illustration/gesticulation is temporally and spatially situated with the instructor’s vocal utterance. Without the embodiment cues, the student would be constantly searching for the attentional focus on the visual illustration. This is no different than the difficulty a distancelearning student encounters when trying to follow an illustrated discussion on a computer monitor with access only to a disembodied flying cursor. She will lose track of it the moment her attention is not fixed on the cursor. She will then have to search for the deictic point, expending valuable cognitive and attentional resources. Having visual access to the instructor’s body will help in directing the student’s attention and cueing her to the spatial presentation of the material (we posit that access to the gestural preparation phases [8] of the instructor, for example, allows the student precue their attentional resources). This will permit the student to devote more cognitive resources
5.2 Embodiment Cues for the Blind
Figure 6. Illustration of Reverse Joystick ground of situated discourse with respect to the illustration is maintained between the student and instructor. A vector may be conveyed by means of two primitives: direction and magnitude. We posit that three tactile modes may convey the FD vector. In the first mode, the student reads the raised pin diagram with her preferred hand, and the other hand is placed on a reverse joystick. As shown in Figure 6 this device is a handle mounted on an active motorized x-y table. The displacement of
Figure 7 Haptic Glove Ring the ball from the center of the table indicates both the magnitude and direction of the FD (by the equivalent magnitude and direction deflection of the ball from center). As the instructor’s PIF and the student’s TPA change, the joystick moves to track the changes. This provides dynamic feedback to the student to
Figure 8 Haptic Glove Array remain grounded in the discourse. The second tactile mode may take the form of a ring of haptic feedback devices (such as piezoelectric transducers or off-centered vibrating mini-motors) mounted on the inside of a skin-tight cut-off (fingers exposed to permit reading of the raised line drawing) glove (see figure 7). These devices vibrate with an intensity proportional to the voltage applied to them. The glove may convey the FD to the student by vibrating the group of devices with a Gaussian intensity distribution centered in the direction of the FD, and with an intensity or repetition rate proportional to the FD magnitude. The top of the glove will be marked with fiduciary markers to permit rapid video-based tracking of the TPA. The final mode of tactile interaction involves the production of apparent motion on a grid of haptic output devices mounted on a cut-off glove similar to the second tactile mode (see figure 8). The direction of the apparent motion is motivated by activating the haptic devices in successive lines normal to the direction of the FD, and moving in the direction of the FD. The magnitude of the FD is conveyed by the speed of the apparent motion.
6. Open Questions The conceptual framework and phenomenology presented in this paper is intended to motivate practical implantations of multimodal systems to enable situated mathematics discourse between instructors and blind students. While such implementation is beyond the scope of this paper, we want to highlight a series of challenges that need to be addressed before a practicable system may be deployed. The first set of questions is about the efficacy of the proposed devices. How well do they convey spatial-temporal information? What is the correct combination of the intensity and duration of the vibration of the actuators? Which actuators to vibrate to provide the correct sense direction and distance? The same kind
of questions also applies to the joysticks. Will the angle/speed of the joysticks’ movements be enough to deliver a good quality sense of direction and distance? This is essentially a ‘perceptionand-action’ question relating how well a device can provide guidance to drive the reading action of the blind student. The second set of questions is related to the ability of the student to perform the primary task of tactile reading while also attending to the spatial-temporal cues provided by the ‘embodiment awareness device’. Do these devices interfere with the blind students’ ability to read tactile material (i.e. do they confound the finger-tip reading). Is there any difference between wearing the glove on the reading hand, and on the other hand? These questions are essentially about the confound between the haptic cues of ‘where to read’ and the content information of ‘what to read’. The third set of questions relate to whether the student is able to maintain situated communication with these devices. This is different than the second question in that it asks whether she is able to process co-temporal speech, read the raised line material, and utilize the spatial-temporal deictic cues simultaneously. These questions involve such low-level device parameters such as data rate (are the embodiment awareness cues delivered at the appropriate rate to maintain the students grounded on the teacher’s discourse?), to high level questions as to whether the cognitive load of attending to the ‘where to read’ cues and following the spoken discourse stream too taxing. How might familiarity with the system (i.e. achieving expert performance with the devices) affect the ability to maintain situated communication? One might, for example, argue that sighted individuals have ‘expert-level performance’ in interpreting embodiment cues to aid in situated discourse understanding from ‘life experience’. The final set of questions relate to how much the proposed solution helps students in understanding mathematics instruction. These are more involved question involving pedagogy and instructional approaches. The effectiveness of language embodiment cues can be judged via their impact on student and teacher gesture/speech performance in math learning environments, and is proportional to the degree to which the interaction approaches the capacity of sighted student at the same levels of math instruction. Indeed, we expect that the instructor will modify her behavior given the new mediating technology [18]. One might expect that this adaptation will alter the character of the instruction and pedagogy, and consequently the effectiveness of the instruction. Also, the use of the proposed devices will impose some restrictions on the freedom of movement of student – for example, their gesticulation may be constrained. Since gestures play very important role in embodied interaction this may have an impact on the student’s ability to assimilate the instruction. If this impact is negative, what are the possible design workarounds? In addition, movement limitation may impose restrictions on class activities that are useful for teaching math concepts to children (e.g. they may assist in the embodiment of mathematics fundamentals) [19]. These and other questions have to be addressed before, and as systems based on the concepts articulated in this paper are implemented.
7. Conclusion In this paper, we set forth a series of hypotheses regarding the provision of embodiment awareness to students who are blind.
We show through relevant phenomenological analysis the importance of embodiment to mathematics conceptualization and discourse. We show that individual who are blind have to capacity to conceptualize visuo-spatial information, and we argue that awareness of the teacher’s embodied discourse will help the student keep situated with their teacher’s instructional discourse. We propose a set of haptic devices that may assist blind students to maintain this situated communication, and we articulate a series of open research questions that this research direction presents.
8. McNeill, D., Hand and Mind: What Gestures Reveal about thought. 1992, Chicago: University of Chicago Press. 9. Smith, N., Gesture and Beyond, in Cognitive Science. 2003, University of California at: Berkeley. 10. Haber, R.N., Haber, L. R., Levin, C. A. and Hollyfield, R., Properties of spatial representations: Data from sighted and blind subjects. Perception and Psychophysics, 1993. 54: p. 1-13.
8. ACKNOWLEDGMENTS
11. Millar, S., Movement cues and body orientation in recall of locations by blind and sighted children. Quarterly Journal of Psychology, 1985(A 37): p. 257-279.
This research has been supported by the National Science Foundation, HSD program, Grant # ISS-0451843.
12. Landau, B., E. Spelke, and H. Gleitman, Spatial knowledge in a young blind child. Cognition, 1984. 16: p. 225-260.
9. REFERENCES
13. Kennedy, J.M., Drawing and the Blind. 1993, New Haven, CT: Yale Press.
1. Williams, J.M., Nationwide shortage of teachers for blind students must be corrected. National Federation of the Blind: Advocates for Equality, Canadian Blind Monitor, 2002. 14. 2. McNeill, D. and S. Duncan, Growth points in thinking-forspeaking, in Language and Gesture, D. McNeill, Editor. 2000, Cambridge University Press: Cambridge, MA. p. 141161. 3. McNeill, D., Gesture and language dialectic. Acta Linguistica Hafniensia, 2002. 34: p. 7-37. 4. Beattie, G. and H. Shovelton, Iconic hand gestures and the predictability of words in context in spontaneous speech. British Journal of Psychology, 2000. 91: p. 473-491. 5. Beattie, G., Visible Thought: The New Psychology of body Language. 2003, Hove: Routledge. 6. Furuyama, N., Gestural interaction between the instructor and the learner in origami instruction, in Language and gesture. 2000, Cambridge University Press: Cambridge. p. 99-117. 7. Penrose, R., The Emporer’s New Mind. 1989, New York: Oxford University Press.
14. Iverson, J.M. and S. Goldin-Meadow, Why people gesture as they speak. Nature, 1998. 396: p. 228. 15. Heller, M.A., Tactile picture perception in sighted and blind people. Behavioural Brain Research, 2002. 135: p. 65-68. 16. Craig, R.H., Learning the Nemeth Braille Code: A Manual for Teachers. 1980, Salt Lake City: Brigham Young University Press. 17. Eramian, M.G., et al. Talking Tactile Diagrams. in Universal Access in HCI: Inclusive Design in the Information Society, Proceedings of HCI International 2003. 2003. Hersonissos, Crete, Greece. 18. Williams, A., E. Kabish, and P. Dourish. From Interaction to Participation: Configuring Space through Embodied Interaction. in Proc. Intl. Conf. Ubiquitous Computing (Ubicomp 2005). 2005. Tokyo, Japan. 19. Tall, D., A Theory of Mathematical Growth through Embodiment, Symbolism and Proof, in International Colloquium on Mathematical Learning from Early Childhood to Adulthood. 2005: Nevellis,Belguim.
