Formal HCI Patterns for Computer Game Development

Proceedings of the Fourth IIEEJ International Workshop        on Image Electronics and Visual Computing        Koh Samui, Thailand, October 7-10, 2014

FORMAL HCI PATTERNS FOR COMPUTER GAME DEVELOPMENT: A CONTRIBUTION TO THE IMPROVEMENT OF SOFTWARE USABILITY Roland Petrasch, Pavadee Sompagdee Department of Computer Science, Faculty of Science and Technology, Thammasat University, Bangkok, Thailand ABSTRACT Patterns for Human Computer Interaction (HCI patterns) have a history of more than ten years, but mostly they have been defined on an informal (or semi-formal) level and their practical application is still limited. This contribution describes a formal notation for HCI patterns and presents an example for the computer game domain in order to demonstrate the advantages of formal HCI patterns: For instance, they can lead to a consistent user interface that conforms with user expectations (ISO 9241-110). During the development of computer games, HCI pattern can be easily applied in many situations (like design patterns) and can also be used for code generation. Examples for HCI pattern in games are "instant replay", "slow motion", “pause / resume". These pattern are formalized and used in during game development. A new UML-based language is introduced and an example for code generation is given, so that the productivity of development teams can be increased.

1. INTRODUCTION HCI Patterns in the context of usability engineering are invented more than 15 years ago, e.g. [1], [2]. Like design patterns for software engineering, they provide well accepted solutions that can be applied in an individual and specific situation (problem)1. Borchers defines HCI pattern as follows; An HCI design pattern captures the essence of a successful solution to a recurring usability problem in interactive systems. [4] Probably every user experiences the advantages of HCI patterns when using system or application software: The user interface components that apply HCI patterns are easy to understand because the concept is already known by the user. 1 “Patterns are abstract, core solutions to problems that recur in different contexts but encounter the same ‘forces’ each time.” [3]

The feedback dialog after a “close document” action in OpenOffice is an simple example for such an HCI pattern (see fig. 1). The user has the choice weather he or she wants to save or to discard the current document or to cancel the close action. The standard layout of this modal dialog window (message, icon, push buttons etc.) contributes for a better user experience. The effect is called conformity to user expectations: The user expects to get a feedback dialog before losing any data.

Figure 1: Example for the HCI pattern “Feedback dialog” (used in OpenOffice) Most applications are using HCI patterns and in some cases UI style-guides also contain HCI patterns. However, notations and languages for describing patterns are not standardized. Various pattern languages are available [5], [6], [3], [7], [8] and a discussion is going on, what requirements patterns and pattern languages should fulfill and why they are not used in practice [9]. The integration of HCI patterns into a model driven development approach [10] [11] could lead to a better acceptance of HCI patterns in application development projects. Patterns can be the key success factor when it comes to (re)use commonly accepted solutions on the model and also code level. So far, user interface design is mainly a manual task from the development point of view. Giving the UI designer and usability engineer the ability to apply HCI patterns easily while designing user interfaces could leverage HCI patterns and model driven techniques at the same moment.

2.

MBUID – MODEL-BASED USER INTERFACE DESIGN

Interactive software systems (like any other software system) can be described by conceptual models, e.g. UML diagrams (Unified Modeling Language [12]). Using these models for transformations, e.g. generation of code or other artifacts, is the main idea of model-driven (software) engineering2. In the case of interactive software, the model driven approach is called model-based user interface design (MBUID). This means formalization, modeling, transformation and generation of main parts of an interactive software system. According to the OMG's MDA Guide [12], the abstract and (technological) platform independent model is called PIM (platform independent model). A PIM contains no information about a concrete technology or platform. The next step is the creation of a PSM (platform specific model) that describes the software system for a specific platform, so that it can be implemented [14], e.g. Android for mobile apps. With code generation tools a PSI (platform specific implementation) can be produced (see fig 1). Automatic transformation and code generation tools increase the productivity for software development.

PIM (platform independent model)

Model to model transformation

PSI (platform specific implementation)

PSM (platform specific model)

Model to code transformation (code generation)

AUI (abstract user interface) = PIM

Transformation

FUI (final user interface) = PSI

CUI (concrete user interface) = PSM

Transformation

Figure 3: Model-Based User Interface Design (MBUID: AUI, CUI and FUI 3. FORMALIZATION OF HCI PATTERNS: A UML PROFILE FOR MBUID In order to describe HCI patterns on a formal level that are human readable and can be transformed in a model-driven context, languages using the UML profile mechanism (lightweight approach) like Wisdom [16], and UWE [17] are available. A UML profile is a set of stereotypes that extend the UML without changing its meta-model, so that existing UML tools can be used for model creation. More heavyweight technologies work with their own meta-models, e.g. CTTE based approaches [18]. Based on the existing approaches a state-of-the-art UML profile was developed that can be used for user interface design in conjunction with HCI patterns. The following UML profile diagram (s. fig. 4) shows a part the UML profile for MBUID [19].

Figure 2: Model Driven Engineering: PIM, PSM and PSI Model-based user interface design (MBUID) uses the concept mentioned above: An abstract user interface (AUI) in this context is a PIM, e.g. mockups or a UI scribble. A concrete user interface (CUI) can be developed based on the AUI: It uses concrete user interface components, e.g. for a web site. The final user interface (FUI) is the implementation on the code level, so that the interactive software can be used (see fig. 3) [15].

2 The OMG (Object Managemet Group) published a Guide for Model Driven Architecture (MDA) [13]

Figure 4: UML profile for MBUID (cropped) The UML profile shows the definition of stereotypes (e.g. InteractionElement, HCI_Pattern) that can be applied to different UML elements like Class or Property, i.e. the stereotypes can be used in different UML diagrams giving the modeler flexibility in terms of the modeling language.

HCI patterns can be defined with any UML tool that is able to apply the UML profile for MBUID. Also the abstract UI model (AUI) can be created using a UML tool (s. fig. 5).

User interface concept

UML tool

HCI Pattern

AUI (abstract user interface)

In the following example, we demonstrate our MBUID approach using a formal model and a code generator: Firstly, an existing user interface is analyzed for HCI patterns. The well known computer game “Angry Birds” (Rovio, http://www.rovio.com/) is taken as an example. The navigation at the end of a level (see fig. 7) is easily understood and can be defined as a navigational HCI pattern. There are three options for user navigation: menu, replay and next level.

MBUID profile

Figure 5: Creation of the AUI using HCI patterns and the MBUID profile The next step after creating the AUI would be the generation of the concrete UI (CUI) for a specific platform, e.g. JavaFX or Android. This transformation can be done automatically with a transformation tool. The CUI can be modified manually in order to make specific adjustments or corrections, because in practice user interface builders (UIB) for particular platforms are often used (see fig. 6).

AUI (abstract user interface)

MBUID transformation tool FUI (final user interface)

CUI (concrete user interface)

UIB

Figure 7: Navigation in Angry Birds game after finishing a level The HCI pattern definition is shown as a UML diagram using the MBUID profile (see fig. 8): The dialog after finishing a level is the LevelEndDialog. From here the user can choose one out of three navigational interactions. “replay” and “next level” will navigate to a StartLevelDialog (either the current or the next one) and “menu” takes the user back in a menu area.

Figure 6: Generation of the CUI using a MBUID tool and manual modification of the CUI The following example for an HCI pattern in the game domain will demonstrate the approach. 4. HCI PATTERNS FOR COMPUTER GAMES: AN EXAMPLE Different types of HCI pattern libraries are available [20]: General purpose HCI patterns can be applied in many different contexts, e.g. breadcrumb, registration [21]. Domain specific HCI patterns focus on particular application domains, e.g. computer games. Folmer defined a set of game patterns, e.g. instant replay [22]. However, these patterns are not described formal and no code generation approach is used.

Figure 8: Definition of the navigational HCI pattern for LevelEndNavigation Now, this formalized and abstract HCI pattern can be applied to new user interface designs. The following example show the AUI for the game “FriendlyCat”: From the level 1 “FeedTheCat” the user can navigate to level 2 “PlayWithTheCat”, replay level 1 or go back to the menu.

The HCI pattern application gives the UI designer the necessary information about the navigational options (see fig. 9).

6. REFERENCES [1] T. Coram, J. Lee: Experiences -- A Pattern Language for User Interface Design. In: Pattern Languages of Program Design, Proceedings, 1996 [2] J. Tidwell: Interaction Patterns. In: Pattern Languages of Program Design,Proceedings, Monticello, IL., USA, 1998 [3] I. Graham: A pattern language for web usability. Addison-Wesley, 2003 [4] J. Borchers, J.: A pattern approach to interaction design. West Sussex, UK: Wiley & Sons, 2001, S. 179 [5] E. Bayle, R. Bellamy, G. Casaday, T. Erickson, S. Fincher, B. Grinter, B. Gross, D. Lehder, H. Marmolin, B. Moore, C. Potts, G. Skousen, J. Thomas: Putting it all together: towards a pattern language for interaction design. SGICHI Bulletin, 30(1), 1998, S. 17-23

Figure 9: Application of the HCI pattern for LevelEndNavigation for the FriendlyCat game This AUI with an applied HCI pattern shown in fig. 9 can now easily be used for the generation of code, so that main parts of the game don't have to be programmed manually. 5. SUMMARY This contribution introduced HCI patterns and their formalization with the UML profile for MBUID (modelbased user interface design). An abstract user interface (AUI) can be defined and transformed to a concrete user interface (CUI) with an MBUID tool. The UI designer is able to use HCI patterns (general purpose or domain specific patterns) during the design of the user interface, so that the interactive system can meet usability requirements like conformity to user expectations [23]. With UI generation techniques and HCI patterns the effort to create, modify, align and implement the user interface is reduced, especially the software development tasks like manual programming can be reduced significantly. A HCI pattern for computer games, the “EndLevel Navigation” HCI pattern exemplified the approach: The pattern was described as a model and applied to a new computer game.

[6] M. Bradac, B. Fletcher: A Pattern Language for Developing Form Style Windows. In: R. Martin, D. Riehle, F. Buschmann (Hrsg.): Pattern Languages of Program Design. Addison-Wesley Longman, Reading, MA, 1998, S. 347-357 [7] J. Tidwell: COMMON GROUND: A Pattern Language for Human-Computer Interface Design. http://www.mit.edu/ ~jtidwell/common_ground_onefile.html, 1999 [8] M. Van Welie, G. C. Van Der Veer: Pattern Languages. In: M. Rauterberg, M Menozzi, J Wesson (Hrsg.): Interaction Design: Structure and Organization, INTERACT 2003, . IOS Press, Zürich, Switzerland, September 2003 [9] Y. Pan and E. Stolterman: Pattern language and HCI: expectations and experiences. CHI EA '13 CHI '13 Extended Abstracts on Human Factors in Computing Systems, pages 1989-1998, 2013 [10] R. Petrasch and F. Fieber, Andreas Bednarz: Agile modellbasierte Software-Entwicklung. In: Reinhard Höhn, Roland Petrasch, Oliver Linssen (Hrsg.):Vorgehensmodelle und der Product Life-cycle – Projekt und Betrieb von IT-Lösungen, Shaker Verlag GmbH, 2008 [11] R. Petrasch: Modellbasierte Entwicklung von Web 2.0 Anwendungen mit MDA. i-com, Zeitschrift für interaktive und kooperative Medien, Oldenbourg, 2007, ISSN: 1618-162X, S. 28-32

[12] Object Management Group (OMG): UML – Unified Modeling Language (UML) Superstructure Specification. Version 2.4.1. 2011

for Developing and Analysing Task Models for Interactive System Design. In: IEEE Transactions on Software Engineering, vol. 28 (8) pp. 797 – 813, 2002

[13] Object Management Group (OMG): MDA Guide.Version. 1.0.1, omg/2003-06-01, June 2003

[19] R. Petrasch: Model Based User Interface Design: Model Driven Architecture und HCI Patterns. In: GI Softwaretechnik-Trends, Mitteilungen der Gesellschaft für Informatik, Band 27, Heft 3, Sept. 2007, S. 12-17

[14] R. Petrasch, O. Meimberg: Model Driven Architecture - Eine praxisorientierte Einführung in die MDA. dpunkt.Verlag, 2006 [15] J. M. C. Fonseca (ed.): Model-Based UI XG Final Report. W3C Incubator Group Report 04 May 2010. http://www.w3.org/2005/Incubator/model-based-ui/XGRmbui-20100504/#intro [16] N. J. Nunes: Object Modeling for User-Centered Development and User Interface Design: The Wisdom Approach, Universidade da Madeira, Diss., 2001 [17] N. Koch: Software Engineering for Adaptive Hypermedia Systems: Reference Model, Modeling Techniques and Development Process. Diss., LudwigMaximilians-University Munich, 2001 [18] G. Mori, F.. Paternò and C. Santoro: CTTE: Support

[20] Roland Petrasch: Model Based User Interface Development with HCI Patterns: Variatio Delectat. PEICS’10, June 20, 2010, Berlin, Germany, ACM, 2010 [21] Roland Petrasch: Model Based User Interface Development and HCI Patterns: The Registration-Pattern as an Example. In: S. Rongviriyapanich, R. Petrasch (Eds.): Proceedings of Software Engineering and Quality Assurance. Technical Report #1, Thammasat University, Beuth Hochschule, Logos Verlag, 2012 [22] E. Folmer: Usability Patterns in Games. Futureplay 2006 Conference, London, Ontario, Canada, 3. Oct. 2006 [23] ISO 9241-110:2006: Ergonomics of human-system interaction -- Part 110: Dialogue principles. ISO, 2006