Learning and Perceiving Colors Haptically - CiteSeerX

Learning and Perceiving Colors Haptically Kanav Kahol, Jamieson French, Laura Bratton, Sethuraman Panchanathan Center for Cognitive Ubiquitous Computing http://haptics.asu.edu Tempe, Arizona 1-480-727-7147

{kanav, jamieson.french, laura.bratton, panch}@asu.edu ABSTRACT Color is an integral part of spatial perception and there is a need to develop systems that render color information accessible to blind individuals. A novel system that allows learning, presentation and analysis of color information, designed in consultations with focus groups of individuals who are blind is proposed. Our system is based on a methodology that renders colors as textures through a haptic device. The aim of the proposed approach is to enable color perception and provide a basis for assessing color similarity. Initial testing of the system shows that both blind individuals and sighted individuals can recognize colors through our approach and further assess similarity between colors through the system. A space was obtained through multidimensional scaling performed on similarity scores between pairs of colors as presented through our system. This space obtained high congruency with the chromaticity diagram and the hue saturation color wheel which shows the validity of our system to allow color visualization. A realtime system based on the proposed mapping is designed to allow realtime color perception.

Categories and Subject Descriptors H.5.2 [User Interface]: Language Haptic I/O

General Terms Algorithms, Measurement, Performance, Design, Human Factors

Keywords Haptic user interfaces, color perception

1. INTRODUCTION In humans, color perception is an important component of spatial processing [11]. It allows efficient perception of the environment and aids in object recognition, classification, scene segmentation and other spatial tasks. Color perception also plays a significant role in social interactions. Lack of color information severely impedes spatial perception and social interactions [8]. Color blind individuals who suffer from various types of color

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ASSETS'06, October 22–25, 2006, Portland, Oregon, USA. Copyright 2006 ACM 1-59593-290-9/06/0010...$5.00.

blindness such as achromacy, tritanopia, deuteranopia, deuteranomaly etc. face several difficulties in tasks of everyday living. Individuals who are blind also suffer from lack of color information. Congenitally blind individuals do not experience color perception and need to learn about color through artificial means. Individuals, who become blind at later stages, have experience of color but the lack of sensory information pertaining to color affects daily activities. Systems that allow color perception will assist individuals with such sensory and cognitive disorders. Much of the work in universal design has focused on shape rendering, texture perception, etc. Shape and texture are features that are both visual as well as haptic in nature [9]. Humans are adept at both shape and texture perception through touch [9]. However, color as a feature is purely visual. It is not perceivable directly through the touch sensors in humans or for that matter through the auditory or olfactory sense. Hence presentation of color information to individuals who are blind represents a complex problem. It requires conversion of visual information into a form that is easily perceivable by the user and yet allows analysis of color and development of mental representations of color. A review of psychological literature on color learning for individuals who are blind was performed. While the authors could not find any scientific study on teaching the concepts of color, conversations with researchers in assistive technologies and users who are blind, yielded that in many instances color is taught through temperature analogies. For example, yellow is hot, green is cool, and so on. This analogy system is very limited but due to a lack of alternate methods, it is widely used. In our consultations with a focus group of users and assistive technology experts, the group expressed reservations in the temperature based system for color representation.

2. RELATED WORK Certain devices have been proposed that report color information to individuals who are blind. For example, a portable device that reads color information of a surface through a laser light and then reports the color through synthesized speech is commercially available (http://www.irti.net). While such devices are useful, they are not accurate and are influenced by lighting conditions and other environmental factors. However, at a fundamental level, they pose a serious constraint. It is well known that humans use a similarity based perceptual basis for color [7]. In other words, instead of representing color as discrete information, the color spectrum is perceived as a continuous signal, and colors are perceived in relation to other colors. For example, pink is perceived as being close to red and white, light blue is perceived as being in proximity of white and blue, etc.

Everyday experience confirms that sighted individuals cannot exactly name a color but perceive it based on its similarity to another color which has a name associated with it [1, 11]. Devices such as the portable color reader report a color category through their computational mechanism. However, often the reported color information does not correspond to a known color type, and additionally, the sensed color is inaccurately judged. This limits color perception in multiple ways and renders these devices unusable. A limited body of literature exists on assistive systems for color perception. Sjostrom et al. [10] developed a system for individuals who are blind that allowed painting pictures. The painting program allows the user to paint a picture using their finger with each color represented by a texture. The paper does not detail how the textures were assigned or how many colors the system incorporated. Way and Barner [12] developed a system for automatic translation from visual images to tactile images. In this system, color was converted to texture. It is however important to note that blind individuals were not required to perceive color. Cappelletti et al. [2] developed a vibrotactile system for color visualization. They represent RGB components of a color as three vibrations on three fingers of the hand through a wearable system. Initial results suggested the validity of this approach. However the study did not report the learning times required. Additionally the system presented constant vibrations to the finger that often leads to uncomfortable sensations residual sensations on the hand as reported in the paper. The study only covered 13 colors and did not actually evaluate the ability of the system to allow comparison between colors. Our consultations with focus groups of individuals who are blind, revealed that the ability to perceive color similarity is central to the success of any assistive system for color perception. Hence testing of the system should include measure of how well the users can guess the color similarity. Doel [4] designed a color visualization scheme through 3D auditory displays. The system allowed users to touch an image and receive 3D auditory feedback (continuous) derived from mapping 3 dimensional color information into 3 dimensional auditory outputs. There was no controlled study evaluating the usability of this system.

3. RESEARCH STRATEGY We designed our system in consultation with a focus group of 10 individuals who are blind. The following elements were identified as being critical to the success of an assistive system for color perception. 1.

Loss of vision makes color information inaccessible to individuals who are blind. For individuals who are late blind, experience of color information allows development of a conceptual space of color. However, congenitally blind individuals can develop this space through linguistic and temperature analogies. A system for color visualization should cater to both these communities.

2.

The system should be able to allow unobtrusive feedback that does not impede the perceptual abilities of the user.

3.

The system should allow for veridical perception of colors. Simple systems that just name a color do not actually allow for learning the concept of color. For example simply stating the color is blue does not actually allow perception of what blue actually is. Additionally this system does not allow

perceiving just noticeable differences between colors or comparing colors. There is a need to develop a mapping of color to a perceivable feature that allows these perceptual operations. 4.

The system should have a small learning curve. In order to achieve this, it was necessary to work with perceptual space of colors rather than additive or subtractive systems. In everyday life we work with a small set of colors often adding descriptions of "dark" and "light" against these basic colors. An assistive system should allow for such operations.

5.

The system should allow for addition of colors or perception of colors that have not been experienced before.

6.

It was noted that before real time perception of colors, it is important to develop a system that allows learning of color. Many of the developed systems do not have a learning phase for the system. They directly map color as audio or vibrations without a controlled experiment to allow for learning. An explicit phase that allows development of association between color and learnt stimuli was deemed necessary.

We designed a conceptual framework that is based on these recommendations. Haptics (force feedback) was chosen as the modality of information delivery as it provides unobtrusive feedback. We designed a first phase of system wherein mapping between color and its haptic equivalent were developed and tested for association of colors with haptic feedback and for judging similarity of colors. In the second phase we designed a real time cueing system that uses the learned mapping to display color images in real time. The final research methodology was approved by the focus group.

4. CONCEPTUAL FRAMEWORK Our system design is based on the PhantomTM Haptic Joystick that allows force feedback rendering. Using this device, it is possible to constrain a user's movements in virtual environments, restrict direction of movement and create virtual friction and textures. The trichromatic theory of colors [1] suggests that a color presentation scheme may benefit from presenting colors to individuals who are blind through three channels rather than simply naming the color. It is common knowledge that a wide gamut of the color spectrum can be represented as a combination of red (R), green (G) and blue (B) colors. A simple approach would essentially convert the R,G,B component of colors into a haptic representation. As an example, the user could be allowed to feel friction or texture on 3 planes that is directly proportional to the amount of reddishness, bluishness and greenishness of a color. (We use the term friction and texture interchangeably for the purposes of this paper.) As an example, when feeling the color red, the user feels three planes ordered in a pre-determined fashion. Let us suppose the first plane represents reddishness, the next plane represents greenishness and the third plane represents bluishness. So for the color red, the user feels very high friction on the first plane, no friction on the second plane and no friction on the third plane. (Red is represented as (255,0,0) in an 8 bit color coding scheme). By learning to identify friction levels, the user can perceive color and learn to compare a set of colors. This approach has the advantage that every color is represented as a

combination of three basis functions. A user hence has to learn only three basis functions or colors, and can represent any color as a combination of the quantized levels of these values.

5. EXPERIMENTAL DESIGN AND PROCEDURE

However, the problem with this approach is that as the number of quantization levels of R,G,B increase, the user has to remember all the combinations and this can limit the perception ability. This is coupled with the fact that perception of friction and differences in friction have limitations as has been shown in psychophysical evaluations [3]. In order to circumvent this problem, this paper proposes another piece of information in addition to the 3 planes that represent reddishness, bluishness and greenishness of a color. The proposed plane, called M Plane where M refers to the mixer conveys to the user whether the color is pure, or whether white color or black color is added to the color. Hence, in this scheme, a color is presented as three planes with friction directly proportional to the RGB components of the color, in addition to a fourth plane, which may add white color, black color or no color to the color. For example, the color pink is represented as a red with white color added, and the amount of white added conveys the purity of pink. This scheme does not directly map into any color scheme. However, it mimics the perceptual process that humans use for classifying color as highlighted in Berlin and Kay. Every color can be simply defined through a combination of some basic colors.

Visualization was implemented using a Phantom interface. The program was coded to allow users to feel a horizontal plane that is parallel to the base of the joystick. At the beginning of a simulation, the joystick automatically moves to the first plane. The user is instructed to move the probe from left to right to feel the friction of the plane. The user can then press a button on the probe of the device to move to the next plane. Pressing the button causes the joystick to snap to the next horizontal plane where the user can feel the friction associated with the second plane. Similar transitions allow perception of friction in the third plane. Once the fourth plane is reached, there are three possible behaviors: no white or black color added, white color is added or black added. In the fourth plane, a value of 0 means that the user can move the probe in both directions without any constraint or friction. A value of -128 means the user can move in the fourth plane only on the left side with medium friction, while a value of +128 means the user can move in the fourth plane only on right side with medium friction. Figure 2 shows a pictorial depiction of how the system works.

Table 1 gives the details of color that were coded in our system, their associated RGB levels, HSV levels (visual) and their rendered values in the RGB and M planes. The hue, saturation and brightness values are provided for the reference of the readers. Figure 1 shows the colors that were used in our experiment. Consistent with the fact that perceptual color space does not map into any known computational color space, there is no definite relation between RGB (visual) and the 4 plane levels in our system. In order to minimize learning it was imperative to keep the number of friction levels to be learnt at a minimum. We chose three levels: 0, 128 and 255. Essentially the user needed to recognize three levels of friction: no friction, medium friction and high friction. The basic idea is that a user feels three planes in succession exploring from left to right, perceiving the friction and deciphering the color associated with the friction levels. In the fourth plane, we needed to convey to the user whether white color, black color or no color is added. This is achieved by controlling the direction in which the user can move the probe of the Joystick. Here, -128 accounts to adding black to the basic color, +128 means adding white to the basic color and 0 means adding no color to the basic color. The colors can be divided into three categories based on the rendered friction in the M plane. The first category of colors consists of colors that have no friction rendered on the M plane; seven colors: red, green, blue, yellow, orange, violet and black, belong to this category. In this experiment, these colors will be called the basic colors. These colors were chosen as basic colors as they are some of the basic wavelengths that the human sensory system is tuned to pick up [11]. To each of the basic color, white or black pigment could be added to create new colors as shown in Table 1 and Figure 1.

Figure 1. Colors used in the experiments

different levels and the difference between these levels. This phase also includes familiarization with how the fourth plane functions. The next step is the learning phase. In this phase, a user is presented a color as shown in Table 1. The user is provided feedback on what color the visualization presents. This phase encompasses all the colors. In the next phase, the test phase, the user is presented with a color and requested to recognize the color. The learning and test phase are repeated 4 times to measure the color recognition accuracy over the learning trials.

5.1.1 Participants For testing, four groups of users were recruited. The first group hereafter referred to as the B group, included 5 individuals who are blind. Table 2 shows demographic information on the 5 users. 10 users who are sighted were also involved in the evaluation. Five users were trained and tested blindfolded and are referred to as the SB group. Five users were trained and tested in sighted conditions and are called the S group. The same group was tested in the blindfolded condition and formed the S/SB group. The four groups of users allowed evaluation of the scheme with different groups of population, as well as evaluation across visual abilities. Further, by using sighted individuals in the experiment, it allowed us to observe if haptic visualization of color is possible for the sighted community, and if multimodal transfer and intermodal coordination with our scheme is possible. Table 2. User information. Columns represent users. L means light perception C means complete loss

Figure 2. Experimental Protocol Two experiments were designed. The first experiment tested the ability of users to perceive colors through the proposed rendering scheme. The second experiment tested the user's ability to perceive similarity between colors. This experiment is necessary to prove that the proposed system allows development of a conceptual space of color that, as discussed before (see discussion on perceptual basis of color), is based on similarity between colors.

5.1 Experiment 1. Recognition of color through haptic visualization The following is the experimental procedure. The users were provided with an electronic copy of a handout explaining the general idea and how the color recognition scheme was implemented. The first step of the experimental procedure is to familiarize the users with the four planes and how they are presented. This is done by presenting the levels {0 0 0 0} {255 255 255 128} {255 255 255 -128}. This step is repeated and users are provided feedback on their exploration strategy. Initially no feedback is given on how to recognize different levels. In the next step, the user is presented with random combinations and requested to recognize different levels of friction while exploring the planes. The aim of this system is to enable the user to perceive

Attribute Age Group Vision Loss Age Visual Ability

Avg and Std Dev

1 1829

2 3045

3 1829

4 3045

5 4560

14

0

0

23

45

-

L

C

C

C

L

-

32.4 and 13.35

5.1. 2 Results Figure 3 shows the recognition accuracies of the three groups. The results show that all the groups can achieve 100% recognition accuracies over some learning trials. It is important to note that even congenitally blind individuals performed well in these experiments. The SB group which was trained in a unimodal condition showed the slowest learning rate. However the S/SB group showed a faster learning rate. The results clearly suggest the validity of our approach in learning to name a limited set of colors. It was encouraging to see that all the groups of users achieved 100% recognition. The SB group’s slower learning rate is an interesting result that may be explained by the relative ease of users to work in multimodal conditions. Sighted individuals are generally uncomfortable in the absence of vision and stress may cause the slower learning rates. More experiments can fully test and verify this hypothesis. While experiment 1 allows recognition of color, it does not completely prove that our system can allow the development of a conceptual space of color. In order to test the validity of the proposed approach as a perceptual mechanism, experiment 2 was designed.

Figure 3. Learning and Recognition of Color

5.2 Experiment 2. Evaluating perceptual similarity of colors as presented through the visualization scheme A multidimensional scaling (MDS) experiment was designed to evaluate the proposed visualization system's usability in assessing color similarity. MDS encompasses a collection of methods which enable insight into the underlying structure of relations between entities by providing a geometrical representation of these relations. The method essentially obtains a similarity matrix between pairs of entities through evaluation by users. The similarity matrix is scaled to plot the entities in lower dimensional spaces. The emerging clusters and patterns in the low dimensional space are representative of how the participants rate the similarity between the stimuli. This helps identify how individuals represent objects in the semantic space. More details on the MDS process can be found in [5][5]. This experiment involves users perceiving a pair of colors through the proposed visualization scheme, and rating the perceived similarity on a scale of 0 through 5: 0 being the rating for least similarity of colors, and 5 being the rating for highest similarity of colors. The Similarity matrix between the pairs of 21 colors is populated by evaluations and then scaled to 2 dimensions and 3 dimensions. Analysis of the scaled spaces can determine the basis through which similarity was assessed by the users. Visual similarity of colors can be represented through the HSV model and the color wheel. A comparison between the scaled spaces as obtained by visualization through our system, and the HSV space can provide an objective measure of how well the visualization system presents color information. This can be achieved through a program called CONGRU. This program allows congruency analysis between two configurations. CONGRU rotates a configuration while keeping the other configuration constant. The objective is to rotate a configuration until maximum overlap is obtained between the two configurations. The overlap between the configurations is evaluated as congruency. So a scaled space of similarity as obtained through our system could be compared to HSV space through CONGRU to yield how well the visualization system performed. Through the above methodology all the groups of users can be analyzed. However the above methodology has a design flaw

in that the 20 users who evaluated the system had learnt to associate friction patterns with colors. It is possible that the trained user groups would evaluate the learnt similarity between colors (as developed through experience) than the actual similarity between haptic signals. This may distort the results. Hence in order to conduct the MDS experiment, two groups of users were used. The first group of users consisted of 5 individuals who are blind and 15 sighted users (S, SB, S/SB groups each containing 5 users) who had completed experiment 1. These groups are called the trained groups. A second group of 5 users who are blind and 15 sighted individuals (S, SB, S/SB groups each containing 5 users) were recruited for the experiment. This group did not perform experiment 1. Therefore, this group of users, called the control group, did not associate the haptic signal with a color. They were trained as follows. The first step of the experimental procedure was to familiarize the users with the 4 planes and how they are presented. This is done by presenting the levels {0 0 0 0} {255 255 255 128} {255 255 255 -128}. This step is repeated and users are provided feedback on their exploration strategy. Initially no feedback is given on how to recognize different levels. In the next step the user is presented with random combinations and requested to recognize different levels of friction while exploring the planes. The aim of this system is to enable the user to perceive the different levels and the difference between these levels. This phase also includes familiarization with how the fourth plane functions. After this phase, the control group users directly proceed forward, perceiving pairs of haptic signal and rating the similarity between pairs on a scale from 0 to 5. The incorporation of the control group allows a study of the visualization methodology in isolation from memorial components of color perception. The similarity matrix of 21 x 21 colors is assumed to be symmetrical and no similarity measure between diagonal elements is obtained. Hence 210 pairs of colors needed to be evaluated for similarity. As gathering data for 210 pairs is not feasible, 109 representative pairs were identified. These pairs are identified to represent a subspace that covers the range of variations in the space while limiting the overall points in the space. This is based on the fact that sparse similarity matrices can be scaled if the data in the matrix is representative of the distribution. The users in all the groups rated the similarity measures of 109 pairs. The obtained similarity matrices were averaged for each group separately. The averaged similarity matrices were multidimensionally scaled to 2 and 3 dimensional spaces using the torgeson-young scaling [5]. The scaled spaces for B,SB, S, and S/SB group (control and trained types) were passed through CONGRU with space from HSV values to determine the congruency between the groups and the yielded color spaces. High congruency between a pair of spaces corresponds to high similarity between the spaces and points to perceptually similar basis of analysis of the two spaces. This helped determine if the visualization scheme provides sufficient basis for color similarity perception.

5.2.1 Results Figure 4 shows a hexagonal representation of the color wheel. Figure 5 shows the 2 dimensional scaled space as obtained for the B trained group. Figure 6 shows the 2 dimensional space for the B control group. Table 3 shows the congruency values between the various groups of users and the congruency between HSV scaled space and the spaces for individual spaces for some of the groups.

The results show that similarity perceived between pairs of colors as perceived through the proposed visualization scheme yields scaled spaces that are highly congruent to the visual similarity spaces modeled as the color wheel and color cylinder.

Figure6. Scaled space 2 dimensions for B control group The results for 3 dimensional scaling had even higher congruency in many of the groups which is expected in an MDS experiment[6]. Comparing figures 4 and 5 and 6 shows that our system maps the color space closely as the visual color wheel. We also found that there was no significant difference in the control groups and trained groups. This shows that our system is not biased by memorial concepts of colors and may indeed convey color and provide basis for assessing color similarity. Once again all the groups performed well which shows the generality of the approach.

Figure 4. Original color wheel

6. REAL TIME PERCEPTION OF COLOR

Figure 5. Scaled space 2 dimensions for B trained group Table 3. Congruency between user spaces Group

Congruency with HSV 2 Dimensions

Congruency with HSV 3 Dimensions

S Trained

.91

.91

SB Trained

.90

.92

B Trained

.90

.90

S/SB Trained

.90

.91

S Control

.89

.89

SB Control

.87

.88

B Control

.90

.90

S/SB Control

.89

.90

The results of the previous section showed that the developed mapping between colors and textures allowed for naming of colors and perceiving differences between colors. For real time depiction of color, we have developed a simple vibratory cue based system that works with CyberTouch gloves. In this system user can wear a glove with vibratory motors on each of the five fingers and the palm. The motor on the index finger, middle finger and ring finger are programmed to represent plane1, plane2, and plane3 intensity levels. The plane 4 information is conveyed jointly through the pinkie and the thumb. If the colors rendering has 0 in the plane 4 then no sensation is sent to the pinkie or the thumb. If the plane4 value is -128 (representing addition of black) then the pinkie finger receives a sensation while if the value is +128 then the thumb receives a vibration. In previous work with the CyberTouch glove, it was noticed that vibration intensity differences were not easily perceived by users. In order to circumvent this problem we developed a cueing mechanism with three pulses (with a gap of 2 ms between pulses) to represent low medium and high levels. Each pulse was either a haptic 'dot' (a small pulse of 3 ms) or 'dash' (a long pulse of 7ms). 'dot','dot','dot' represented low quantization, 'dot','dash'.'dot' represented medium quantization, 'dash','dash','dash' represented high quantization and no feedback meant that there was no level of the channel involved in the color. For each plane and the associated finger feedback unit, the color is communicated as three parallel signals being sent to the fingers. so for example red is a dash,dash,dash on the index finger and no feedback on the middle, ring, pinkie or thumb. Pink was represented as dash,dash,dash on the index finger, no feedback on the middle, ring and pinkie finger and a dash dash dash on the thumb

representing addition of white. It may be noticed that the real time system can handle more colors than the learning system as the quantization levels are 4 instead of 3 quantization levels. This is very useful for addition of colors which will be tested in future. The real time system was implemented and tested for perception of color. During exploration, user can press the left mouse button after which the color the basic color is conveyed. Following this, the system only conveys the change in color based on movement of the mouse. In the case of the color at present position of the mouse and the previous position of mouse being same in the segmented information only a small pulse is sent to the index finger indicating that the color is same as previous. In the case of color changing from the previous position, the color information is sent to the five channels. During any stage in exploration, the user can press the right mouse button to resend the color information to the color. In order to familiarize users with the system training was performed over 4 learning trials as detailed in experiment 1 with the difference that colors were now rendered in a glove based system. Participants described in Table 2 were involved in these experiments. All the users achieved recognition accuracy of 100% from the first trial itself. Initial testing of this system was performed with natural images segmented through k-means into 21 color bins. 5 images were chosen for training. They were told to explore the 5 images through the mouse. Feedback was provided by the experimenters on the color the users perceived while the users received the haptic feedback through the glove. Following this testing was performed through 35 images. The overall accuracy of color recognition was calculated as the number of times the user reported the correct color during exploration. An overall recognition accuracy of 98.5% was recorded. A question that arises from the development of realtime system and the results is whether it is necessary to conduct habituation experiments wherein users are familiarized with the mapping by feeling the textures or can they directly learn the cueing system of glove? To explain this, a note must be made on the difference between rendering of color through the haptic joystick and the cue based mechanism outlined above. When a texture is rendered through the haptic joystick user receives a veridical sensation of the color. On the other hand, cueing gives an artificial sensations. When learning to map color to a texture, users are learning to map an inaccessible piece of information (color) to a known concept (texture). This step was achieved in experiment 1. Once this mapping is learnt, we can develop a cueing vocabulary to associate known concept (texture) to a cue. In such a case, cueing invokes a known concept in the human brain. The learnt mapping can contextually associate color with the invoked concept. Research in psychology [7] has shown that priming through cues is more effective when the cue invokes a known concept or memory rather than an unknown concepts. On the other hand there is a possibility of mapping the inaccessible feature (color) to a cue directly. This avoids the learning phase and may offer a faster methodology for color perception. We recruited a group of 15 additional subjects (sighted) who did not participate in the learning of color through the haptic joystick experiments to test this methodology. These subjects were made to learn the color rendering system as detailed in the section on experiment 1 with the difference that they perceived color through

the glove system. 4 learning trials were performed. An average learning accuracy after 4 trials of only 78.5% was noted. The accuracy in the original group that performed the learning experiments and then the cueing experiments was noted to be 100%. This initial result tends to suggest that training that allows users to associate texture (a known concept) with a color allows for better perception of color rather than training that requires directly mapping cueing (a learnt concept) to color.

7. CONCLUSIONS AND FUTURE WORK The paper proposes and tests a methodology to render color through textures. Testing of the learning system to develop the mapping and a realtime system for color perception reveals that the proposed methodology may provide a suitable basis for color perception. The presented rendering system was tested through a vibrotactile and a force feedback system. In both cases, users can recognize colors with high accuracy following a learning phase which allows users to associate color to a realistically rendered texture. In fact it was seen that users who had learnt to map color to a texture through a realistic rendering of texture performed better than the group who learnt to map color to an artificial coded representation of texture. This result may be important in the design of learning mechanisms for color perception. In the future, this system will be extensively tested with a larger group of users. Differences between various subgroups (for example congenitally blind individuals versus late blind individuals) will be documented. Further in the future, we intend to test the robustness of the proposed mapping to adding new colors in the repertoire.

8. REFERENCES [1]

[2]

[3]

[4]

[5] [6] [7] [8]

[9] [10]

Berlin, B. and Kay, P. Basic color terms : their universality and evolution. Oxford Press, UC Berkeley, 1991. Cappelletti, L., Ferri, M. and Nicoletti, G., Vibrotactile color rendering for the visually impaired within the VIDET project. in Telemanipulator and Telepresence Technologies V, (1998), 92-96. Hatwell, Y., Streri, A. and Gentaz, E. Touching for knowing : cognitive psychology of haptic manual perception. John Benjamins Pub., Amsterdam , Philadelphia, 2003. Kees, V.D.D., Soundview: sensing color images by kinesthetic audio. in International conference on auditory display, (2003), 303-306. Kruskal, J.B. and M., W. Multidimensional Scaling. Sage Publications, Beverly Hills CA, 1977. Kruskal, J.B. and Wish, M. Multidimensional Scaling. Sage Publications, Beverly Hills CA, 1978. Marr, D. Vision. Freeman Publishers, New York, 1982. Olson, M.M. and Harris, K.R. Color Vision Deficiency and Color Blindness (Paperback). Fern Ridge Press, Portland, Oregon, 1988. Revesz, G. Psychology and art of the blind. Longmans Green, London, 1950. Sjöström, C. and Jönsson, B., The Phantasticon: To Use the Sense of Touch to Control a Computer and the World around You. in The 4th European Conference for the Advancement of Assistive Technology (AAATE'97), (Greece, 1997).

[11] Thompson, E. Colour Vision. Routledge, London, 1995. [12] Way, T.P. and Barner, K.E. Automatic visual to tactile translation-II. Evaluation of the TACTile image creation system. IEEE Transactions on Rehabilitation Engineering, 5 (1). 95.

Table 1: The color levels are depicted on an 8 bit coding system where 0 represents the minimum intensity and 255 represents the maximum intensity. The values on plane 4 are varied from -128 to +128 The negative sign means allowing movement on the left side only, the positive sign means allowing movement on the right side only and 0 means allowing movement on both sides. Color

R

G

B

H

S

B

Texture1

Texture2

Texture3

Texture4

Blue

0

0

255

240

100

100

0

0

255

0

Light Blue

128

227

255

193

50

100

0

0

255

128

Dark Blue

0

0

128

240

100

50

0

0

255

-128

Red

255

0

0

0

100

100

255

0

0

0

Pink

255

128

128

0

50

100

255

0

0

128

Maroon

128

0

0

0

100

50

255

0

0

-128

Green

0

255

0

120

100

100

0

255

0

0

Light Green

140

255

140

120

45

100

0

255

0

128

Dark Green

0

128

0

120

100

50

0

255

0

-128

Yellow

255

255

0

60

100

100

255

255

0

0

Light Yellow

255

255

128

60

50

100

255

255

0

128

Dark Yellow

154

106

2

41

99

60

255

255

0

-128

Orange

254

133

1

31

100

100

255

128

0

0

Light Orange

255

200

140

31

45

100

255

128

0

128

Dark Orange

128

66

0

31

100

50

255

128

0

-128

Violet

255

0

255

300

100

100

255

0

255

0

Light Violet

255

128

255

300

50

100

255

0

255

128

Dark Violet

128

0

128

300

100

50

255

0

255

-128

Black

0

0

0

0

100

0

0

0

0

0

Grey

128

128

128

0

0

50

0

0

0

128

White

255

255

255

0

0

100

255

255

255

0