Programming virtual interactions for gamified

Programming virtual interactions for gamified educational proposes of urban spaces Xavier Calvo1, David Fonseca1, Mónica Sánchez-Sepúlveda1, Daniel Amo1, Josep Llorca2, Ernesto Redondo2 GRETEL – Grup de Recerca en Technology Enhanced Learning, La Salle – Ramon Llull University. C/ Sant Joan de la Salle 42, 08022 Barcelona, Spain 2 AR&M, Barcelona School of Architecture, BarcelonaTech, Catalonia Polithecnic University, Av/ Diagonal 649, 08028 Barcelona, Spain {xcalvo, fonsi, monica.sanchez, damo}@salle.url.edu {josep.llorca, ernesto.redondo}@upc.edu

1

Abstract. The architectural students must acquire for their professional future the comprehension of the space and the usage of the urban projects. In this sense, the possibilities offered by interactive visualization platforms, typically used for the creation of computerized games, can be especially useful given their power and versatility (for example through the process of rendering a scene with lights, textures and materials in real time). The present article contextualizes threedimensional interactive visualization systems in architectural education, and explains the development of the main interactions for students, with elements of urban furniture, under the programming in Unreal.. Keywords: Unreal, Urban design, Virtual reality, Virtual interactions, Render, Architecture education.

1 Introduction The increasing graphic quality and ease of use of the current generation of videogame technology compels educators to rethink how architecture students learn. In the framework of urban and architecture studies, the suitability of the designs (buildings or urban environments) must be assessed before they are built. The representation technologies are used throughout the architectural design process to bring ideas into reality, allowing communication between designers, clients, contractors and collaborators [1]. Architecture students must learn to be proficient in these representation technologies (also called ICT – Information Communication Technologies) throughout their studies: 

Starting with the most basic and fundamental systems that allows 2D representation (mostly based in CAD technology - Computer Assisted Design),



Going through the BIM systems (Building Information Modeling), capable of managing both 2D and 3D information of a Project,



Finally, the new interactive systems based on platforms used in videogames, such as Unreal or Unity, capable of managing a visualization through augmented reality (AR), or virtual reality (VR) in real time, is the main advantage about CAD / BIM systems.

Therefore, it is paramount that students become skillful in multiple representation technologies, and that they are capable of incorporating the latest technologies into their design process in order to better communicate their proposals, and to facilitate the critical reasoning on the spaces they conceive [2]. The project presented in this paper consists of an investigation at the intersection between computer sciences, the education of future architects and multimedia engineers, and the urban policies in future cities [3][4]. One of the most innovative aspects is the incorporation of game strategies in a virtual and collaborative urban environment, aimed at improving the initial proposal. This approach incorporates the architecture students into the project in an active way, clearly enhancing their spatial and urban competences. Specifically, this article presents the main interactions designed and implemented in the Unreal platform for educational use. These actions must be able to select, move and place elements of previously defined urban furniture (with the possibility of incorporating textures, materials, lighting, etc.), always under the constraints of the urban project. The objective of the system is that the student can incorporate elements previously designed in CAD / BIM platforms, for a free interaction for both himself and the final users, allowing improvement in the initial process of actual urban proposals.

2 Literature Review 2.1 Architecture education Architectural education has traditionally relied on Project-Based Learning (PBL), where students are required to develop a proposal, usually over the course a semester, in a process that mimics the workflow of an architectural studio. During the development of this proposal, students learn to integrate often-conflicting aesthetic, constructive, structural, environmental, and usability requirements into a cohesive design, under the guidance of a tutor. In this scheme, the students are usually provided with the location where the design is to be developed and examples of related notable designs as reference.

Architects and urban designers (both graduate and undergraduate) learn about their discipline in a continuous and informal way, because the subject of their career surrounds them almost anywhere and anytime. Thus, explaining the important historic role of traveling in the formative years of architects. However, nowadays the world that surrounds us is increasingly digital, especially for the younger generations using mobile devices and cloud computing services [5]. In the specific framework of architectural education and professional practice, it is clear that we should incorporate this new paradigm and approaches. Representation technologies are used throughout the architectural design process to bring ideas into reality, allowing communication between designers, clients, contractors and collaborators [6]. Architecture students must learn to be proficient in these representation technologies throughout their studies, and must reach the point where drawing and representation blend together, and drawing becomes thinking [7]. Therefore, it is necessary that students become skillful in multiple representation technologies, and that they are capable of incorporating the latest technologies into their design process in order to better communicate their proposals, and to facilitate critical reasoning on the spaces they conceive. 2.2 ICT in urban and architecture data visualization Information and Communication Technologies (ICTs) are transforming citizens’ lifestyles, adding new dimensions to the concept of socialization, as well as creating new habits [8]. Other studies [9], describe the opportunities offered by these emerging technologies as “creating a new kind of reality, one in which physical and digital environments, media and interactions are woven together throughout our daily lives.” At the same time, new university students can be defined as Digital Natives [10] or Digital Residents [11], because they coexist and use all kinds of network technologies, multiple applications and all kinds of mobile devices at very early ages. Until recently, in architectural education, the use of ICT was restricted to project implementation processes, where various applications such as Computer Assisted Design (CAD) and Building Information Modeling (BIM) served merely as aids in the execution of one’s work [12]. Historically, in civil and building engineering education, visualization and understanding of 3D space was typically accomplished via the classical view (physical models and drawings), in front of 3D models and using virtual specifications. This approach is changing due to a generational change and the continuous improvement and development of technology. The new systems based on Virtual and Augmented Reality (VR/AR), Geo-Referencing, and learning gamification, will gradually reduce the control imposed on the designed tasks and scheduled presentations. Due to the potential of virtual systems, the spatial skills and abilities of students can be reinforced, while also using the essential interactive and collaborative features of these processes. Students can work with peers and teachers and participate in multi-tasking/multi-user collaborative and instant tracking [13]. Focused on urban data, it was proposed [14] a generic model to support a new way of visiting a city. In this approach, instead of understanding the city as a place for tourism, the students perceive it as a place for learning in which all necessary educational resources are available. The model has been conceived as a way to encourage learners to create their own educational tours, in which Learning Points of

Interest are set up to be discovered using two models: formal (conducted by a teacher) and informal outdoor mobile learning (where no educator is directly related to the learning experience). 2.3 Games and architectural representation Games are created by designers/teams of developers and are consumed by players [15]. They are purchased, used, and eventually cast away like most other consumable goods [16]. The difference between games and other entertainment products (such as books, music, movies and plays) is that their consumption is relatively unpredictable. The string of events that occur during gameplay and the outcome of those events are unknown at the time the product is finished [17]. The gamification in classes helps to improve the connection between the material and the student. It offers the opportunity to reflect on a topic in depth and allows positive changes in behavior [18]. In this approach, learning through gaming is achieved by aligning the game mechanics with Bloom's taxonomy of learning [19], allowing learning to be classified into three domains [20]:  Cognitive, which is taught in traditional education and implies understanding and synthesis of knowledge.  Affective (involving emotions), which reflects the attitude toward a situation.  Psychomotor (the physical), which is activated by requiring a union of mental and physical activity. To encourage the use of games in learning beyond simulations and puzzles, it is essential to develop a better understanding of the tasks, activities, skills and operations that different game types can offer, and examine how these might correspond to the desired learning outcomes [21]. Using game engines for representation is beginning to gain traction in the architectural field, which until recently had been a stronghold of 3D rendering, generally producing static images and occasionally videos (as a succession of 3D renderings with scripted camera movements). With the game industry improvements in real-time hardware-assisted 3D rendering, the quality provided by game engines is quickly approaching the levels of realism of traditional offline rendering engines, while providing additional features, at a fraction of the cost. Furthermore, real-time rendering offers one benefit that no other technology can provide: immersion, which allows the user to freely navigate the environment and interact with some of its elements (e.g. doors, lighting, avatars); this sense of presence can be enhanced when using positional audio cues and/or virtual reality (VR) headmounted displays.

3 Mechanics development 3.1 Programming system Game engines are vital for developing 3D applications and games today. However, with over 100 engines available with highly different ranges of features, performance, license, and cost, selecting an appropriate game engine for a specific objective becomes a challenging problem. In order to create a 3D game, programmers need to have various high-level skills because there are plenty of techniques within 3D games [22]. A rendering engine for a graphics intensive space can require several options and formats to work, but moreover, 3D systems are using the same optimizations and same set of core techniques [23]. Two of the most popular game engines today, Unreal Engine v4.x and Unity Game Engine v5.x have recently adopted competitive and very appealing pricing structures for individual game developers and small teams. One may lean towards one or the other game engine based on various criteria: existing familiarization/interest, steepness of learning curve, quality, richness, variability and pricing of add-on assets, initial cost of basic ownership versus subscribing to updates and acquiring needed modules, etc. [24]. Unity and Unreal are both viable options for developing a stand-alone GDC (Game Design Course). They both fit the criteria necessary for students to learn easily and use them to build games in a limited time. They have similar interfaces, and it seems that Unity is a little less complicated to learn while Unreal can create more professionallooking results more easily [25]. In our case, we have selected the Unreal Engine 4 (UE4) by Epic Games, which is a framework for developing and rendering video games. UE4 was released on 19 March 2014 and was made available for free and open-source on March 2, 2015. UE4 comes with an extensive editor, which can be used by programming in C++ or by using the visual mode and event graphs without any knowledge of the C++ programming language. In visual mode, the user can create objects that include game logic and keep them stored for further use (so called ’blueprints’) [26]. As we show in the following sections, our application allows the user to logically program through blocks in a simple and fast visual way. 3.2 Interaction mechanics In order to interact with the virtual world in the most natural way possible, a set of mechanics have been programmed to provide the user with a series of skills in the virtual world. In first place, we have analyzed what the main actions would be presented by the project. Once this previous study has been done, a prototype version of the different mechanics has been developed to be able to test with potential users and to improve the interaction based on the obtained data. Finally, we have obtained the following main mechanics (of which we can see two examples in Fig. 1): 1. Item Location: allows the user, using a parabolic beam, to select any object, put it in another place and rotate it. In the first phase of the mechanics, a

straight beam was used from the user's control device to the object. Several users were faced with problems involving objects in the middle, making the interaction ineffective. Therefore, to solve the problem, a parabolic beam was used, which allowed them to select and move objects, although there were other elements in between. 2. Movement in the location: is the mechanics that allows the user to move through the entire virtual world. In the movement mechanics, the problem is that the user is used to move through space with traditional devices such as joystick, keyboard, etc. With this new tool, we find two major changes about the traditional way: a. The user's point of view - Since we went from seeing our character from third person on the screen to seeing it inside the virtual world. b. The way in which we interact to move changes radically - The new control devices that we have in virtual reality are based on the sensors that are placed on the hands and the head of the user. To develop the mechanics, we studied the different ways in which other applications of virtual reality solved this problem and the vast majority implemented the method of teleportation. In our case, we implemented a method that consists of selecting a point on the map through a parabolic beam and by pressing a button on the device, we teleport instantly to the chosen point. This mechanics works well with users that have already tried the world of virtual reality, but it is difficult for the ones who put on the apparatus for the first time, since when they move instantaneously to the point, they are slightly disoriented. To solve this problem, a new navigation system has been designed where the user, by rotating one of the controllers, will be able to go to the direction pointed by the device. In this way, we get a smooth movement and easy to use, even if it is the first time the user moves through a virtual world. Finally, two speeds have been added to allow the user to simulate the fact of being able to walk or run in the virtual reality. 3. Object selection: the dynamics of this control device allows us to interact in a very natural way, as the movements we exact in the real world are very similar. For this mechanics, an algorithm has been programmed which allows you to touch a virtual object with the controller and by pressing a button, the object stays attached to the user's hand until it is released. This dynamic allows us to make usual actions such as consulting a map or a book in a very natural and familiar way. 4. Interaction with the menus: is the part where the user can interact with the different options that the menu offers, such as creating or deleting an object. For this mechanics, the decision taken was that when the user touches the different menu options they would be activated. The problem observed in the tests was that the users who were not yet very familiar with the navigation system, found it difficult to stand directly in front of the menu to interact. To solve this problem, it was programmed that the user by just pointing the controller at the menu and pressing a key, the actions becomes visible.

Fig. 1. Examples of the “selection object” (mechanic #3) and “item location” (mechanic #1).

3.2 Environment lighting in real time Virtual reality allows us to see in a very immersive way the changes and actions that happens in the environment in real time. Thanks to its powerful rendering engine, Unreal Engine 4 allows the calculation of lighting in a space to show a very dynamic and realistic result. By joining the tools, a system has been developed in which the user can, from within the first-person scenario, design the lighting of an urban environment. This new tool provides the students and the entire professional the possibility of seeing in real time the lighting being put on any section of the street and how it is affected by the color, intensity or type of light being used. A variety of tools has been developed so that the user can choose and precisely adjust the type of lighting required for each space (see Fig. 2).

Fig. 2. Example of real-time lighting stimulation.

3.3 Example of programming mechanics In order to understand how the different mechanics are programmed, (see Fig. 3), the programming process belonging to the dynamics of displacement through the scenario (mechanic # 2) will be explained.

Fig. 3. General scheme of the “movement in the location” mechanic.

The operation of this mechanism is quite simple. It allows us to explain with ease how programming works by blueprints. The function starts with the "Event Tick" which is executed in each frame. Next, there are two options that depending on the button that the user presses, they will instruct the object to move forward or backward ("Set World Location") (Fig. 4).

Fig. 4. “Event Tick”: system to move the object in one direction.

To know where the character has to move, first we look at the rotation of the controller with the "Get World Rotation" function. We are only interested in the rotation with respect to the Z axis, therefore we break down the variable of rotation into 3 and

then choose the axis that is needed. We do the sine and the cosine of the angle to be able to calculate the position where the player will have to move. We multiply the result by the speed at which the user will be able to go, since he can walk or run (Fig. 5).

Fig. 5. “Get World Rotation” function.

We rebuild the new position vector and add it to the player's current vector. Once we have calculated the position to which the user has to move, we call the function "Set World Location", passing this vector to him as the new position to which he has to move. For the backward shift function, we simply multiply by minus one. (Fig. 6).

Fig. 6. “Set World Location” function.

3.4 First Implementation In the academic years 2016-2017 and 2017-2018, we have selected the subject of Computer Tools II in the La Salle Architecture School (Ramon Llull University), to introduce the students to emerging technologies such as augmented and virtual reality, photogrammetry and 3D gaming. The course is focused on using videogame technology for architecture representation [27] [28], taking advantage of improvements in real-time rendering to produce interactive content. In these last editions, the students

participated in an educational experience placed at the intersection of architectural representation and urban design. Following the constructivist approach in urban planning [29] [30], a proposed urban renovation project was used as a case of study for the duration of the course. The proposal consisted in the conversion of some vehicular streets in Barcelona into pedestrian walkways, and the creation of civic squares at their intersections. The course was split into roughly two parts: at the beginning of the course, the students were divided into groups of three to four students, and each group was assigned to work on a part of the urban environment. During the following weeks, the groups modeled and textured the façades of their respective sections, following simple guidelines regarding aspects such as the polygon count or the size of the textures of their models, since they would later be used as assets in the game engine. At the end of this process, all the models were consolidated into a single environment, shared by all the groups (see Fig. 7).

Fig. 7. Some of the students’ proposals, inserted into the simulated environment and with the capacity to be interacted with (moved and rotated)

3 Conclusions In our implementation, it is critical to understand whether users (the students) are receptive and aware to adapt to this new paradigm using advanced visualization methods. In this context, we have conducted a first motivational approach in the course 2016-17, in order to investigate the perception and intrinsic motivation of higher

education students of architecture degree and master, related with the use of virtual and gamified systems to represent the 3D urban space [31]. The initial results confirm an evolution of the student, especially in terms of motivation and the perception that the systems used can have in the representation of both architectural and urban projects. While initially, the motivation could be considered medium / low, after the completion of the case study, it has increased significantly. This aspect not only reflects the usefulness of the method, but also the potential in the academic and competence improvement of the student, which previously had been referenced, is linked to the student's motivation. During this academic year, we are developing a new version with the aim to assess the system from a usability point of view. In this version we will develop a mixed method using a quantitative approach with a structured test based on ISO 9241-11, (which facilitates the evaluation of the method focusing on the main usability guidelines of effectiveness (E1), efficiency (E2), and satisfaction (S1)), and a qualitative evaluation based on the Bipolar Laddering Assessment (BLA) used and validated in previous research [32]. Acknowledgments. This research was supported by the programs BIA2016-77464C2-1-R of the National Plan for Scientific Research, Development and Technological Innovation 2013-2016, Government of Spain, titled “Gamificación para la enseñanza del diseño urbano y la integración en ella de la participación ciudadana (ArchGAME4CITY)” [Gamification for the teaching of urban design and the integration in it of citizen participation (ArchGAME4CITY)] and BIA2016-77464-C22-R of the National Plan for Scientific Research, Development and Technological Innovation 2013-2016, Government of Spain, titled “Diseño Gamificado de visualización 3D con sistemas de realidad virtual para el estudio de la mejora de competencias motivacionales, sociales y espaciales del usuario (EduGAME4CITY)” ["Gamified 3D visualization design with virtual reality systems for the study of the improvement of motivational, social and spatial competences of the user (EduGAME4CITY)"]. (AEI/FEDER, UE).

References 1. Horne, M., & Thompson, E. M. The Role of Virtual Reality in Built Environment Education. Journal for Education in the Built Environment, 3(1), 5–24. (2008). 2. Suwa, M., & Tversky, B. What do architects and students perceive in their design sketches? A protocol analysis. Design Studies, 18(4), 385-403. (1997). 3. Orland, K.: Twitch plays everything: How livestreaming is changing game design, http://arstechnica.com/gaming/2015/10/twitch-plays-everything-how-livestreaming-ischanging-game-design/. 4. Fonseca, D., Redondo, E., Valls, F., & Villagrasa, S. Technological adaptation of the student to the educational density of the course. A case study: 3D architectural visualization. Computers in Human Behavior. 72, 599-61, (2016). https://doi.org/10.1016/j.chb.2016.05.048 5. Moreira, F., & Ferreira, M. J. Teaching and Learning Requirement Engineering based on Mobile Devices and Cloud: A Case Study. In Blended Learning: Concepts, Methodologies,

6. 7. 8. 9.

10. 11. 12.

13. 14. 15.

16. 17. 18. 19. 20. 21.

22. 23. 24.

Tools, and Applications Hershey: Information Science Reference, 1190-1217, (2017). https://doi.org/10.4018/978-1-5225-0783-3.ch059 Horne, M. & Thompson, E. M. The role of virtual reality in built environment education. Journal for Education in the Built Environment 3, 5-24, (2008). Suwa, M., & Tversky, B. What Do Architects and Students Perceive in their Design Sketches? A Protocol Analysis. Design Studies, 18(4), 385–403. (1997). https://doi.org/http://dx.doi.org/10.1016/S0142-694X(97)00008-2 Oulasvirta, A., Rattenbury, T., Ma, L., & Raita, E. Habits make smartphone use more pervasive. Personal and Ubiquitous Computing, 16(1), 105–114. (2012). https://doi.org/10.1007/s00779-011-0412-2 Bower, M., Cram, A., & Groom, D. Blended reality: Issues and potentials in combining virtual worlds and face-to-face classes. ASCILITE 2010 - The Australasian Society for Computers in Learning in Tertiary Education, 129–140. (2010). Margaryan, A., Littlejohn, A., & Vojt, G. Are digital natives a myth or reality? University students’ use of digital technologies. Computers & Education, 56(2), 429–440. (2011). https://doi.org/10.1016/j.compedu.2010.09.004 White, D. S., & Cornu, A. L. Visitors and Residents: A new typology for online engagement. First Monday, 16(9). (2011). Retrieved from http://firstmonday.org/ojs/index.php/fm/article/view/3171 Navarro, I., & Fonseca, D. Nuevas Tecnologías de Visualización para mejorar la representación de arquitectura en la educación. Architecture, City, Environment - ACE, 34, 218-237, (2017). https://doi.org/10.5821/ace.12.34.5290 Calongne, C. M. Educational Frontiers: Learning in a Virtual World. Educause Review, 43, 36–48. (2008). https://doi.org/10.1007/s00371-007-0182-z Gordillo, A., Gallego, D., Barra, E., & Quemada, J. The city as a learning gamified platform. In Frontiers in Education Conference, 2013 IEEE, 372-378, (2013). https://doi.org/10.1109/FIE.2013.6684850 Villagrasa, S., Fonseca, D., & Durán, J. Teaching case: applying gamification techniques and virtual reality for learning building engineering 3D arts. In Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality. ACM, New York, NY, USA, 171-177. (2014), http://dx.doi.org/10.1145/2669711.2669896 Institute of Play: http://www.instituteofplay.org/work/projects/quest-schools/quest-to-learn/ (2013). Manrique. V., Epic Win Blog: The 35 Gamification Mechanics toolkit v2.0, 2013. http://www.epicwinblog.net/2013/10/the-35-gamification-mechanics-toolkit.html (2013). Kapp, K. M., The gamification of learning and instruction: game-based methods and strategies for training and education. John Wiley & Sons (2012). Zichermann, G., & Cunningham, C. Gamification by design: Implementing game mechanics in web and mobile apps. "O'Reilly Media, Inc." (2011). Bloom, B.S.: Thought processes in lectures and discussions. Journal of General Education, 7. (1953). Cronk, M. Using Gamification to Increase Student Engagement and Participation in Class Discussion. In T. Amiel & B. Wilson (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 311-315 (2012). Eberly, D. H. 3D Game Engine Design, Second Edition: A Practical Approach to Real-Time Computer Graphics. Morgan Kaufmann Publishers. (2006). Pattrasitidecha, A., Comparison and evaluation of 3D mobile game engines. Chalmers University of Technology, University of Gothenburg, Göteborg, Sweden, Master Thesis, févr, (2014). Pachoulakis, I., Pontikakis, G., Combining features of the Unreal and Unity Game Engines to hone development skills. arXiv preprint arXiv:1511.03640, (2015).

25. Dickson, P. E., Block, J. E., Echevarria, G. N., & Keenan, K. C. An Experience-based Comparison of Unity and Unreal for a Stand-alone 3D Game Development Course. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education, ACM, 70-75, (2017) 26. Valder, J., Comparison between Vizard VR Toolkit and Unreal Engine 4 as platforms for virtual experiments in pedestrian dynamics using the Oculus Rift. Fachhochschule Aachen Campus Jülich. Term paper Mat.Nr: 4006095, Jülich, December 15, (2015). 27. Valls, F., Redondo, E., & Fonseca, D. E-Learning and Serious Games: New Trends in Architectural and Urban Design Education. In P. Zaphiris & A. Ioannou (Eds.), Learning and Collaboration Technologies. Los Angeles, CA, USA: Springer International Publishing Switzerland 9192, 632–643, (2015). https://doi.org/10.1007/978-3-319-20609-7_59 28. Valls, F., Redondo, E., Fonseca, D., Garcia-Almirall, P., & Subirós, J., Videogame Technology in Architecture Education. In M. Kurosu (Ed.), Human-Computer Interaction. Novel User Experiences. Toronto, Canada: Springer International Publishing, 436–447, (2016). https://doi.org/10.1007/978-3-319-39513-5_41 29. Papert, S. Mindstorms: Children, Computers, and Powerful Ideas. New York, NY, USA: Basic Books, Inc. (1980). 30. Hjorth, A., & Wilensky, U. Redesigning Your City – A Constructionist Environment for Urban Planning Education. Informatics in Education, 13(2), 197–208. (2014). https://doi.org/10.15388/infedu.2014.02 31. Fonseca, D., Villagrasa, S., Navarro, I., Redondo, E., Valls, F., Llorca, J., Gómez-Zevallos, M., Ferrer, A., & Calvo, X., Student motivation assessment using and learning virtual and gamified urban environments. In Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM 2017), Juan Manuel Dodero, María Soledad Ibarra Sáiz, and Iván Ruiz Rube (Eds.). ACM, New York, NY, USA, Article 72, 7 pages. (2017), https://doi.org/10.1145/3144826.3145422 32. Fonseca, D., Redondo, E., & Villagrasa, S. Mixed-methods research: a new approach to evaluating the motivation and satisfaction of university students using advanced visual technologies. Univers. Access Inf. Soc. 14, 311–332 (2015) https://doi.org/10.1007/s10209014-0361-4