2013 International Conference on Culture and Computing
Parallel Prototyping for Multi-Language Service Design A case study on introducing a multilingual tool into a Japanese local restaurant
Hiromichi Cho, David Kinny, and Donghui Lin Department of Social Informatics, Kyoto University Kyoto, Japan
[email protected], {dnk, lindh}@i.kyoto-u.ac.jp does not always work well in another [7]. Therefore a new tool must be specifically tailored to the target community, and the challenge is to adopt an appropriate design process for the tool.
Abstract—Multilingual tools can benefit communities in which language barriers exist. Such tools can be developed easily by combining language services provided via standard interfaces in frameworks such as the Language Grid. However, designing such tools is more challenging than implementing them since they must be customized for the target community. To optimize tool designs, designers must consider stakeholder requirements, available services and design options in an iterative process. This paper shows how a community can design such a multilingual tool by parallel prototyping using Web services, presenting a case study of an izakaya, a Japanese local restaurant. In parallel prototyping, more than one prototype is created and tested simultaneously, and this approach enables the tool users to give clearer feedback. In the case study, two different hi-fidelity prototypes are created by combining available language services, and these are then evaluated in a real life situation, resulting in effective feedback for improving the tool design. Keywords—intercultural design, prototyping
I.
collaboration,
language
There are two significant issues related to this challenge. First, the community development processes usually differ from those that are used in developer enterprises. As a community discussed here only develops a tool for its own use, there is no strict deadline for the endpoint of development, requirements may not be documented in a well-formatted way, and it may be difficult to have a formal user evaluation test. Second, as there are various possible ways to use language services, it may be difficult to determine what usage would suit the community best. Considering these issues, an iterative, informal process to optimize the software design of a tool is required. In this paper, we propose parallel hi-fidelity prototyping using Web services for multi-language service design. In this process, current problems and needs in the community are collected through interviews with the stakeholders. Based upon them, ideas for new multilingual services are gathered through brainstorming sessions. The gathered ideas are implemented by combining available language services, and multiple hi-fidelity prototypes are created. Then they are tested in a real life situation.
services,
INTRODUCTION
With the rise of international activities in both online and real-world environments, many international and intercultural communities are facing various problems that result from the differences in members' cultural and linguistic backgrounds. Multilingual tools can support such communities, and can be developed by building upon Web services providing access to machine translators, bilingual dictionaries, and parallel texts. An infrastructure framework such as the Language Grid [7, 8] makes it simple to combine language services, as it wraps such language functions as Web services and exposes them via a standard interface [13]. This allows developers to use language services via APIs so that they can more easily implement a multilingual tool even if they do not have professional skills. In this context, by developer we mean not an individual, but a group or community that needs a multilingual tool.
Operable hi-fidelity prototypes can be easily created at low cost by combining available Web services. On-site evaluation of them allows users to understand how they actually work and to discover what they like and dislike about the prototypes. Feedback obtained through such evaluations can then be used for improving the design of the multilingual tool software. The rest of this paper is organized as follows; in the next section, our proposal is described in detail. It is then illustrated with a case study of introducing a multilingual tool into a small izakaya, a Japanese local restaurant. Based on the results of the case study, our approach is compared with the conventional design methods. Finally we present our conclusions.
Hence an environment for easily developing a multilingual tool is readily available, however difficulties arise in designing a multilingual tool for application in a specific community. Requirements for a tool are very community-specific, and it is usually observed that a tool that succeeds in one community 978-0-7695-5047-3/13 $26.00 © 2013 IEEE DOI 10.1109/CultureComputing.2013.23
II.
DESIGNING MULTI-LANGUAGE SERVICES
The proposed process consists of four steps; pre-interview, brainstorming, parallel prototyping, and on-site evaluation. 86
This is, however, an iterative process in which feedback obtained through evaluation is used for redesigning the prototypes, but the starting point of the next iteration can vary depending on the evaluation results.
unexpected behaviors or potential problems. Therefore we do not recommend tasks for the evaluation should be specified too strictly. On-site evaluation can provide users with real experiences with a tool, resulting in valuable feedback from a smaller number of people.
These steps, especially brainstorming and parallel prototyping, are inspired by the design process suggested by IDEO, an American design firm famous for its design thinking [10]. Observation, brainstorming, and prototyping are particularly emphasized as effective methods for innovation. Those methods are based on rigorous studies, e.g., an ethnographic approach such as on-site observation “has gained considerable prominence as a technique for informing CSCW systems development," particularly for requirements elicitation [6].
III.
CASE STUDY
To illustrate the proposed approach, a case study of introducing a multilingual tool into an izakaya was held . In the case study, two prototypes were designed and implemented as web-application tools that operated on an iPad platform, and then evaluated in the izakaya. The prototypes utilized language services on the Language Grid [7, 8]. The target izakaya is located near a university in Kyoto, Japan. A man and his wife, who do not have any foreign language skills, manage the restaurant, and problems occur when they have foreign customers. Foreign customers are not rare because there are many foreign students studying at the university and many foreign tourists in Kyoto. Therefore we decided to design a multilingual tool using language services to support communication between the owners and foreign customers. As well as the owners, we involved two foreign students, one American and one Chinese, studying at the university. Both of them had spent about three months in Japan at that time, and their Japanese language skills were at the beginner level.
A. Pre-interview Stakeholders and target users of the tool are interviewed in this step to collect their current needs or problems. Different stakeholders such as customers and owners in a restaurant may have different needs so all types of stakeholders should be interviewed. If information from interviews seems not enough, it is also an effective way to have an on-site observation to find what causes problems and why they arise. B. Brainstorming A brainstorming session is held to gather diverse ideas for new language services addressing the stakeholder needs and to solve the problems collected in the previous step. Not only developers or designers, but also the intended tool users are expected to join the session. Each stakeholder has his/her own perspective and lacks some part of others’ knowledge, e.g., in the case study described later, Japanese engineers did not know what difficulties foreign guests really had in a Japanese restaurant, and, in contrast, the foreign students did not know what was feasible with the available Web services. Thus, the collaboration between various stakeholders promotes mutual learning, and leads to diverse ideas [1, 9].
A. Pre-interview We conducted face-to-face interviews with both of the owners and the two foreign students. First, through a interview with the owners, we understood their current situation such as what problems they had had with foreign guests and how they had dealt with them. Also they told us their values. They regarded communication with customers as important. Then we asked them for their requirements, which could be summarized in two points; knowing what a customer likes or wants to eat, and understanding responses or requests from a customer.
C. Parallel Prototyping More than one prototype is created to test ideas gathered in the previous step. The temporal and financial cost for development can be reduced by using Web services [12, 14]. Therefore, by combining Web services, hi-fidelity prototypes can be easily created, which provide users with actual interaction. Moreover, the ease and the speed of implementation even make it possible to create multiple hifidelity prototypes simultaneously. In this paper, we define parallel prototyping as the simultaneous creation of multiple alternative prototypes. Parallel prototyping allows users to express clearer preferences and feedback through evaluation, compared to just one prototype at a time [4, 5, 15]. Therefore parallel prototyping using Web services can promote design exploration aiming to optimally meet the users’ needs the best.
Second, interviews with the two foreign students were held respectively. We grasped what problems they had had in Japanese restaurants, and how they had dealt with such problems. It was found to be the most important for them to have satisfying dishes. This can be decomposed into five aspects; having dishes that match their preferences, knowing popular dishes, knowing what a dish is like, requesting and asking for customization of a dish, and making requests that are not related to dishes. B. Brainstorming Taking the gathered requirements into account, a brainstorming session has held with six people including both of the foreign students and other students who were familiar with the Language Grid. Unfortunately, the owner could not join. Sixty-six ideas were collected through the session (see Fig. 1). Another brainstorming session with the same members was held to brush up the collected ideas. After the sessions, the following eight new services were designed. Table 1 shows which requirements each of the designed services fulfills.
D. On-Site Evaluation The created prototypes are evaluated in a real environment with the real users. The aim of evaluation is to find users’ preferences on the specified new services, and discover
87
TABLE I. RELATIONSHIP BETWEEN THE FUNCTIONS AND THE REQUIREMENTS (A GRAY-COLORED CELL MEANS THE FUNCTION CONCERNS THE REQUIREMENT.) Service
Requirement Who
Owners
Customers
•
•
•
What
Menu translation
Roman alphabet translation
Comments on menu
Image search
Preferences translation
Template translation
Free-text translation
Comments translation
Knowing what the customer likes Understanding guests’ responses and requests Having dishes matching their preferences Knowing popular dishes Knowing what a dish is like Requesting and asking for customization Making requests not related to dishes
the need of the foreign students that they want to know what a dish is like, and the insight of the engineers regarding recent popular review Web sites.
Menu translation translates the Japanese menu into the language selected by the customer. Menu information such as titles and ingredients of dishes is registered in the bilingual dictionary beforehand. The function utilizes translation service combined with bilingual dictionary with longest match search, which is a composite machine translation service collaborated with bilingual dictionaries. The idea is based on the experiences of the foreign students that they could not understand the menu only written in Japanese. Roman alphabet translation translates titles of dishes into the Roman alphabet. This service utilizes MeCab, a morphological analysis service. The idea is based on the need of the foreign students that they want to know and pronounce Japanese title of the dish.
•
Image search shows the top four image search results for each dish. The search query is composed of the title of the dish and the name of the category into which the dish is classified, e.g., “fried food”. Only this service utilizes an API offered by Google, not by the Language Grid. The idea is based on the experiences of the foreign students that they could not imagine what a dish is like even if the English name of the dish is given.
•
Preferences translation translates the customer’s preferences into Japanese so that the owners can know what he/she likes. Using a template form, the customer can input his/her preferences such as his/her favorite food and their budget in the language selected by the customer, and translates the preferences into Japanese. This service utilizes translation service combined with bilingual dictionary with longest match search. The idea is based on the need of the foreign students that they want to have dishes that match their tastes, and the need of the owners that they want to offer the dishes that match guests' tastes.
•
Template translation provides templates of typical phrases used by either of the owners or the customers, e.g., “can you eat A?” Replacing “A” with a certain word such as “raw fish”, the whole sentence will be translated into the language selected by the customer. This service utilizes translation service combined with bilingual dictionary with longest match search. The idea is based on the needs of the foreign students that they want to tell their request to restaurant staffs, and the insight of the engineers that templates for typical phrases save the cost for input and improve the translation quality.
•
Free-text translation allows users to type sentences freely, which are then translated. This service utilizes translation service combined with bilingual dictionary with longest match search. The idea is based on the
Comments on menu enables the customer to see and add comments on each dish in the menu in the language selected by the customer. Customers can also add a pictogram representing their impression. This service utilizes translation service combined with bilingual dictionary with longest match search and Picton, a pictogram dictionary service that searches for pictograms related with the query. The idea is based on
Fig. 1. Post-it notes are used in the brainsotrming sessions.
88
Fig. 3. Sketches of the “deciding by oneself” prototype
need of the foreign students to communicate their request to owners, and the insight of the engineers that free text should be supported since the range of conversation that templates could cover is limited. •
a valid theoretical foundation. However, the chosen concepts promoted efficient classification of the designed services into multiple prototypes, and moreover, the two concepts were based on real situations so using them as the judgmental standard was considered reasonable. Thus, alternative approaches are also available such as classifying designed services by, e.g., a two dimensions matrix.
Comments translation provides the customer with a form to tell the owners his/her comment about the meal. The customer types his/her comment with a pictogram in the language he/she selects, and then the comment will be translated into Japanese. The difference between this and comments on menu is that this service is to provide customers’ comments directly to the owners while comments on menu can leave messages to other customers that would come afterward. This service utilizes translation service combined with bilingual dictionary with longest match search and Picton. The idea is based on the need of the owners that they want to know if guests like the offered dishes.
D. On-site Evaluation The evaluation was held in the izakaya with the participation of the owners and both foreign students (see Fig. 3). Each of the foreign students was asked to make one or more orders with each of the prototypes, so we had four sets of the “order task” in total. The owners were asked to receive the students’ orders and provide dishes based on their interaction with the customer. Before starting the tasks, we explained what each of the prototypes could do. The evaluation by each of the students proceeded as follows;
C. Parallel Prototyping Prototypes were created based on the designed services. We first build the two concepts for two prototypes, which come from the typical styles of order in a restaurant. The first one is “deciding by oneself” style, in which guests see the menu and select what to order. The second one is “leaving to the owners” style, in which guests ask owners to offer dishes that would suit the guests’ tastes. The latter style is found from the izakaya owners’ experience. They told us that foreign customers usually leave their order to the owners because they cannot read the Japanese menu. The owners had had some experiences that they had tried to understand the customer’s preferences by showing food and then served dishes selected by them. For implementation of the “deciding by oneself” prototype, menu translation, Roman alphabet translation, comments on menu, and image search were used. For implementation of the “leaving to the owners” prototype, preferences translation, template translation, free-text translation, and comments translation were used. Some sketches of the prototypes were first drawn (see Fig. 2), and, based on them, the two prototypes were finally implemented.
1.
In this case study, we first built the two concepts "deciding by oneself" and "leaving to the owners”, but this does not have
Order task 1: By using the “deciding by oneself” prototype, the student made an order as they liked. The two students conducted the task in almost same way, as follows (name of services written in parentheses stand for the services used at that moment). First, the student
Fig. 2. Evaluation conducted in the izakaya. (The Chinese student making her order with the “deciding by oneself” prototype)
89
TABLE II. # 1 2
CONVERSATION DIALOGUE IN THE ORDER TASK WITH THE “DECIDING BY ONESELF” PROTOTYPE
Speaker Student: Owner:
Utterance Excuse me. Yes. This… “sanma (saury)”…
3
Student:
4
Owner:
5
Student:
6 7 8 9 10
Owner: Student: Owner: Student: Owner:
Yes, “sanma”… fish? Yeah, it’s fish. So… this. “Sanma?” “Sanma.” All right.
11
Student:
Umm…
12
“Sanma.”
This…
13
Owner:
Beer?
14
Student:
15
Owner:
16 17
Student: Owner:
Berr… yes. All right. Your drink is beer. OK! OK!
TABLE III.
QUESTIONNAIRE RESULT: USEFULNESS EVALUATED BY THE CUSTOMER USERS AND COST EVALUATED BY THE DEVELOPER
Nonverbal action Coming to the student. Pointing to an image shown on the iPad. Pointing to the same image shown on the iPad.
Deciding by oneself Leaving to the owners
Writing on the order sheet. Moving to the page of another dish. Pointing to an image shown on the iPad. Seeing the image the student is pointing at.
3.
Writing on the order sheet.
Service
Menu translation Roman alphabet translation Comments on menu Image search Preferences translation Template translation Free-text translation Comments translation
American student
Chinese student
Ave.
4 3 1 5 5 3 3 2
4 5 3 5 5 5 4 3
4 4 2 5 5 4 3.5 2.5
Questionnaire and interview: After the two order tasks, the student answered a questionnaire, which included questions such as how useful each service was and how much each requirement was satisfied (questions were answered in 5-point scale; 1 means “not useful/satisfied at all”, and 5 means “really helpful / fully satisfied.). Based on the observations and the questionnaire, we interviewed the student. We asked the student to explain any of their behaviors that we found unnatural or unexpected. We also asked the students to criticize and suggest needed improvements to the prototypes.
After evaluation by both of the foreign students, a questionnaire and an interview with the owners were also held. The questionnaire asked if their requirements were satisfied with the prototypes (answered in the same 5-point scale as for students questionnaire). After the questionnaire, we also had an interview for explanation to their behaviors, and asked for feedback.
chose the language to use (both chose English). Then, he/she chose one category from the menu, saw the dishes in the category (menu translation, Roman alphabet translation, image search), and occasionally saw the details of a dish such as ingredients (menu translation) and comments (comments on menu). In case the student decided to order the dish, he/she showed the iPad display to the owners, pointing at the image of the dish (image search) (see Fig. 3). In case the student wanted to add another dish, he/she went back and selected other categories or dishes. Receiving the order from the student, the owner (the husband) delivered the dishes. Although the students were able to add comments on the dishes (comments on menu), this service was not used. Table 2 shows the conversation dialogue between the owner and the Chinese student for the order task with the “deciding by oneself” prototype. The conversations between the owner and the foreign students were conducted in Japanese as the foreign students used a few simple Japanese words they knew. 2.
Usefulness
Prototype Prototype
IV.
DISCUSSION
A. Summary of the Evaluation Result Table 4 shows the usefulness of each service, evaluated by the students. This can be used for determining which service should be implemented in the later artifacts. Image search, e.g., was scored the highest. This finding was interesting to us. As shown in Fig. 4, some of the search results were incorrect. While the results for “nankotsu (fried cartilage of thigh)” properly showed what the dish looks like, none of the results for “chikuwa tempura (fritter of fish paste)” were proper. Nevertheless the customer users regarded the service as quite useful. This fact implies the importance of visual information.
Order task 2: As with order task 1, each of the students made an order using the “leaving to the owners” prototype. The overview flow of the task was as follows: the student selected the language to use, and filled out the preferences form (preferences translation). He/she showed the translated result of the preferences to the owners. In addition, the student also gave further preferences in free text (free-text translation). Seeing the preferences of the student, the owner suggested some dishes with simple Japanese words and gestures. Though the owner was able to use a template “how about A?” provided by template translation, the owner did not use this service. Comments translation was not used by both of the students.
Another interesting result was a learning effect that a user adapted himself/herself to the tool. In testing the “leaving to the owners” prototype, e.g., the American student used freetext translation to request about dishes in English, but he modified his message when he saw the machine translation result. The interview with him proved that this was because he found the result inadequate with his little Japanese knowledge and made the message simpler. Moreover, he suggested the tool utilize a back translation service in the tool. Back translation enables estimation of the machine translation quality by translating the translated message back into the original language again. He could suggest this since, through the brainstorming session, he came to know this service was
90
approach of parallel prototyping using Web services enables designers to employ a trial and error process of design due to the ease and speed of development based on the Language Grid. The case study illustrated how the participants proceed, and showed that valuable feedback that might not have been obtained with conventional methods were obtained for redesign. In addition, it was also observed that the user adapted himself to the tool so as to improve the performance of the user+tool. This attitude seems to be a key success factor when a community develops a multilingual tool by itself. ACKNOWLEDGMENT This research was partially supported by Service Science, Solutions and Foundation Integrated Research Program from JST RISTEX, and a Grant-in-Aid for Scientific Research (S) (24220002) from Japan Society for the Promotion of Science.
Fig. 4. Appropriate image search results for “nankotsu” and inappropriate results for “chikuwa tempura”
We thank everyone who provided helpful comments on this paper and who cooperated us with the case study.
also offered by the Language Grid. This positive attitude to the tool seems the key for success of development within a community since a developer may not always have professional skills.
REFERENCES [1]
B. Comparison with Conventional Design Methods Lo-fidelity prototypes are preferred for design exploration [2, 3]. However, they lack navigation [13] since if one were to represent the whole navigation, a large number of sketches would be needed, which might increase the cost [11]. In contrast, while representing the full interaction, the creation cost of hi-fidelity prototypes is the biggest disadvantage.
[2]
This cost problem of hi-fidelity prototypes could be solved to a certain extent by using Web services. In the case study, it took only one and a half days to implement the two prototypes; half a day for designing the detailed specifications of the two prototypes, and one day for implementing them. The implementation was done by a student developer with no professional experience. Regarding these facts, it can be said that hi-fidelity prototypes using Web services bring the advantages of both lo-fidelity and hi-fidelity prototypes.
[5]
[3] [4]
[6]
[7] [8] [9]
A non-controlled method was taken for the evaluation step for two reasons. First, the evaluation aims at design exploration and obtain diverse and unexpected feedback rather than assessment of the detailed functions of the tool. Second, as we focus on development within a community, development and evaluation are likely to proceed in an informal way. The case study showed this non-controlled evaluation resulted in the user’s unexpected feedback; one of the foreign students pointed out the necessity of back translation. This happened because he observed a low quality message translation result. If the interaction had been controlled, this might not have happened. V.
[10] [11]
[12]
[13] [14]
CONCLUSION
[15]
As the development of a multilingual tool for a specific community has different characteristics to a usual software development project, it benefits from an iterative process for optimizing the tool design for the community. The proposed
[16]
91
Adamczyk, P. and Twidale, M. Supporting multidisciplinary collaboration: Requirements from novel HCI education. Proc. SIGCHI’07, ACM Press (2007). Buchenau, M. and Suri, J., F. Experience prototyping. Proc. DIS, ACM Press (2000), 424-433. Davis, A. Operational prototyping: a new development approach. Software, Vol. 9, Issue 5, IEEE (1992), 70-78. Dow, S., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. and Klemmer, S. Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. TOCHI, Vol. 17, Issue 4, ACM Press (2010). Dow, S., Heddleston, K. and Klemmer, S. The efficacy of prototyping under time constraints. Proc. C&C '09, ACM Press (2009). Hughes, J., King, V., Rodden, T. and Andersen, H. Moving out from the control room: ethnography in system design. Proc. CSCW '94, ACM Press (1994), 429-329. Ishida, T. Language Grid: An Infrastructure for Intercultural Collaboration. IEEE, IPSJ SAINT 06 (2006), 96–100. Ishida, T. The Language Grid Service-Oriented Collective Intelligence for Language Resources Interoperability. Springer (2011). Jobst, B., Koppen, E., Lindberg, T., Moritz, J., Rhinow, H. and Meinel, C. The Faith-Factor in Design Thinking: Creative Confidence Through Education at the Design Thinking Schools Potsdam and Stanford? Design Thinking Research, Understanding Innovation 2012 (2012), 3546. Kelley, T. The art of innovation. Profile Business (2002). Lin, J., Newman, M., Hong, J. and Landay, J. DENIM: Finding a Tighter Fit Between Tools and Practice for Web Site Design. Proc. SIGSHI '00, ACM Press (2000), 510-517. Papazoglou, M. and Van Den Heuvel, W. Service-oriented design and development methodology. International J. Web Engineering and Technology, Vol. 2, No. 4, Inderscience Enterprises Ltd (2006), 412–442. Rudd, J., Stern, K. and Isensee, S. Low vs. high-fidelity prototyping debate. Interactions, ACM Press (1996), 76-85. Sakai, S. Development support of intercultural collaboration systems based on service oriented programming (2009). Sillitti, A., Vernazza, T. and Succi, G. Service oriented programming: a new paradigm of software reuse. Lecture Notes in Computer Science 2002, Vol. 2319, Springer (2002), 39–47. Tohidi, M., Buxton, W., Baecker, R. and Sellen, A. Getting the right design and the design right. Proc. CHI 2006, ACM Press (2006), 1243– 1252.
