Participatory Design of the Virtual Camera for Deep Space Exploration Don Platt Micro Aerospace Solutions, Inc. 907 East Strawbridge Ave Suite 103 Human-Centered Design Institute Florida Institute of Technology Melbourne, FL 32901 USA +1 321 243 4633
[email protected] ABSTRACT
In this paper, a new Virtual Camera (VC) system is described that has been developed to assist astronauts in deep space exploration. Participatory design, involving users and stakeholders in all aspects of the design was used to make a system that provides a mediation environment. This environment, implemented on a tablet computer system, allows explorers in the field to share data and knowledge with those in remote command and control centers. There are other applications for this system design such as disaster response, law enforcement and aviation cockpits.
Guy A. Boy Human-Centered Design Institute Florida Institute of Technology Melbourne, FL 32901 USA +1 321 674 7631
[email protected]
for the astronaut’s exploration of the remote environment. They use the VC by annotating areas of interest or of safety concern. This information is then viewed as astronauts prepare to explore as well as in real-time while they are exploring. They can call up annotated data from ground personnel which will help them determine what to explore as well as what areas may be dangerous and should be avoided. The interaction model is shown in Figure 1.
Keywords
Situation Awareness (SA);Augmented Reality; HumanComputer Interaction (HCI); Tablet Computing; Usability Testing; Space Exploration. INTRODUCTION
The Virtual Camera (VC) is an interactive database that enabled mediation of collaboration for exploring unknown environments. Its original goal was to assist in the exploration of deep space planetary bodies by astronauts. As astronauts move further from Earth than low-earth orbit (LEO) they no longer will be able to rely on real-time interactive communication with ground controllers and scientists. They will still need the expertise of these controllers so this expertise needs to be captured in the remote environment being explored. A database of what is expected to be explored can be developed and then tools provided to navigate that database, annotating and adding information and data as it becomes apparent. The VC does this using a tablet-based interaction system. Using the VC, goals; such as areas to be explored, are shared among team members so that everyone is aware of what should be explored, what team members are currently exploring and also what areas should be avoided. In Human Deep Space Exploration autonomy of the local explorers will be more important. They need a way to call up knowledge and trigger previous training as they explore. The VC provides a way to capture knowledge from training, as well as previous exploration. Interaction with the VC is then conducted by ground personnel as they plan
Figure 1. Interaction with the VC is off-line (training and planning) for ground personnel and real-time during exploration for astronauts. The virtual camera (VC) concept for human space exploration was first coined and described by Boy et. al. [1]. The VC integrates and manages mission data and areas of interest for both exploration and safety. This information will include system health and status, caution and warning, safety, traverse execution and mission timeline parameters. This paper examines the background and potential uses of a VC and presents the results of a small-scale field investigation of a VC-like implementation. Testing and evaluation explore how the VC influences interaction and cooperation between actors involved in human spaceflight. The VC encourages user interaction within its database. According to Grudin and Poltrock [3], determining how to route, organize, and present contextual information to
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions
[email protected]. HCI-Aero '14, July 30 - August 01 2014, Santa Clara, CA, USA Copyright 2014 ACM 978-1-4503-2560-8/14/07…$15.00 http://dx.doi.org/10.1145/2669592.2669653
facilitate collaboration is a pressing challenge. They also suggested that qualitative field research is an area with unlimited potential. New technology deployment always provides phenomena of interest. Qualitative research can find patterns that are then followed up with qualitative research to discover what the patterns mean. This testing was meant to determine these initial patterns. PARTICIPATORY DESIGN
Human-centered participatory design approaches were used throughout the development of the VC system. Rather than developing technology and finding a use for it, participatory design looks at goals and then finds technology to assist in accomplishing those goals. It should be pointed out that the process is actually iterative, involving potential users as often and much as possible. This way a two-way feedback is established so that potential solutions are presented and evaluated with lowlevel prototypes before it gets too late in the design process to change. The full iterative process is illustrated in Figure 2.
Brief scenarios and interaction diagrams as well as storyboards and prototypes were also developed. These were presented to users to show the basic concept of the VC and to get feedback of what would be a beneficial interface and what may also not be of benefit to a user. Scenarios and use cases give users examples of how the VC can be applied. Next horizontal prototypes were developed that captured the basic interaction requirements for the VC and were used to demonstrate an operational scenario. Taking the feedback into account a final functional prototype was developed. This was developed using the Android tablet platform, see Figure 3. A main goal of developing the functional prototype was to have a tool to test in the field to determine the usability of a tablet for interacting with an exploration database as well as what emergent properties would be observed during its use. This is also a key element of human-centered participatory design. When technology is applied in the field users and operators find unique ways to interact with and use the technology. These can never be fully anticipated without placing the technology in the hands of users in the field and giving them a chance to use the technology in a real-world setting. In fact, several rounds of testing and evaluation should be completed using each one to better inform further testing and allowing for design enhancements. This was the case for the VC system.
Figure 3. The Functional Prototype Showing Icons and Representations of Areas of Interest. IMPLEMENTATION
Figure 2. The Human-centered Participatory Design Process Used to Develop the VC for Human Deep Space Exploration. The design process started by analyzing what were important aspects of exploration which then led to a cognitive function analysis of exploration. A survey of potential users and stakeholders (astronauts, mission operations personnel, and scientists) for the VC system was conducted by the author at the 2011 NASA DesertRATS analog exploration testbed. The main questions during potential user interviews involved their background, general system uses, interface type and data display parameter formats desired.
A tablet computer interface provides a small, portable platform with a very common user interface that is gaining wide acceptance. Hutchins [2] suggested that humans are quite good at pattern recognition, modeling simple dynamics of the world and manipulating objects in the environment. The VC takes these considerations into account in its development and implementation. This is done through the use of easily recognizable icons as well as breadcrumb trails that correlate with traverses and the ubiquitous interactions made possible with a tablet. Icons have been developed that symbolically represent their intended purpose. A science beaker represents places in the terrain database where scientifically important areas of interest are located. A water droplet is used to represent
areas of potential in-situ resource use. A warning yellow triangle is used to represent an area of safety concern. An information cartouche is used to summarize consumable concerns such as power and propulsive capability. Communications connectivity is summarized by an antenna icon and potential collision dangers were also summarized in icon form, see Figure 3. Colors change from gray for nominal to yellow for a warning to red for a danger in the information cartouche. Initial testing was completed in terrestrial settings to evaluate the technology and implementation possible for future space exploration applications. To demonstrate the ability to navigate and annotate through a database which could be useful for terrestrial exploration testing, the Google Earth and Google Map databases were selected. Initial design thought was to use the Google Earth application which takes satellite imagery and maps it onto a 3-D globe of the Earth. This would also have the advantage of leveraging the Google Moon and Google Mars databases. However, it was discovered that no API existed on a tablet platform. Another consideration was to import one of the Asteroid Itokawa STL databases and create an interactive tablet application. Issues related to coordinate systems and tablet memory limitations made this less than ideal for near-term proof of concept. It also would not have allowed field testing to take place with a variety of users. Google Maps version 1 was selected to develop the vertical prototype since an API was available for tablet PCs and it then made a convenient test platform. Google services were used for storage and navigation (GPS coordinates). Google Maps also provides a familiar interface with a large user base. The Google maps API is also available for Apple iOS, allowing for cross-platform development in the future. A web interface was developed that adds the ability to monitor mobile application (VC) users in both current and past locations. It also creates a central hub for communication while in the field. Data collected and annotated in the field can be called up and viewed at any time in the future with data and location annotations tagged. The web application uses Google Maps Web API for easy integration with the mobile VC application. Viewing of VC data is then possible from any browser that can access the internet. EVALUATION
Early testing was completed with users who were familiar with tablets but not necessarily explorers. The intention of this testing was to evaluate the usability of the system, evaluate test techniques and observe emergent uses and phenomena as the device was used in the field. For the first round of testing three teams were involved with 8 different subjects in each who all had tablet experience. The goal was not to gather a large statistical sample but rather to gather feedback and collect emergent behaviors and uses of the system. The subject team used the VC to navigate a
course encountering AOIs of various point values to simulate mission importance scale. The order of AOIs was defined by a Mission Control (MC) team ahead of time in consultation with the VC subject team. A control team was also on the same course using a non-interactive tablet-based map of the area. The control team used a feature limited version of Virtual Camera which can display static AOIs and report data to MC. The MC team had access to the same database of AOIs as the Subject and Control teams, although communication between the teams in the field and MC team involved a simulated lightspeed delay. The communication delay was simulated by having a phone in the possession of the observer of each of the two field teams who did NOT answer the phone. The MC team left voice mail responses. The observer then waited an agreed upon time delay, then played back the voicemail. Thus, half duplex, time delayed messaging was simulated. This simulated the changing environment the VC is expected to operate in. Pictures were also taken by the field teams to confirm that each resource site was visited correctly. Observations were made of the test subjects in the field. They were asked to speak aloud and give their impressions of the use of the VC or map-only interface to assist them in determining what to explore. Questionnaires were used to evaluate navigation situation awareness during VC testing and to compare it to teams in the field who had the map only for navigation. For navigation situation awareness, at two points in each 30 minute traverse both the VC and map only subjects were asked to answer a series of questions about their ability to perceive their situation, comprehend it and project into the future. The observer then determined whether the answer was appropriate or not for the given situation. Questionnaires and debriefings were given to the participants after the testing to elicit feedback and general impressions. They were asked to rate their confidence (trust) in navigation, location finding and identifying exact position of themselves and AOIs/hazards in the field with either the map only or the VC. They then selected a value from 1 to 5, with 1 being no confidence and 5 being complete confidence. Using feedback from these debriefings, it was determined that using the map alone with none of the interactive capability of the VC, the control team found they had high confidence in where in the course they were but lower confidence on what exact AOIs they were looking for, see Table 1. They did not have the ability to see the AOIs marked off at a higher zoom-in level and had no access to the annotation information placed by the simulated ground experts in the database. They found themselves surveying actual landmarks often to get their bearings. This indicates a reliance on the map for navigation but more unknowns about the actual AOIs and what the team was actually looking for. Since time is precious during space-based exploration, efficiency and precision are very important. VC users were operating in a much more “heads-down” mode relying more heavily on the technology. The
breadcrumb display was useful for finding where the team was and the direction they were headed, much more useful than the large heading arrow. The VC teams felt with a high level of confidence where they were headed and where the AOIs were. This indicates the VC is quite useful for defining, identifying and locating the AOIs. When questions about the planned traverse did come up, they were “radioed” back to simulated mission control through the text message time delay interface. Test subjects often found themselves feeling the time crunch and not waiting for a response. They would use the VC tool and their own knowledge of the situation to make decisions. Similar issues will arise in deep space exploration with limited consumables forcing astronauts to make decisions and not wait for responses from ground control.
exploration for these scientists and in no evaluation point did it rate worse than current systems.
Table 1. Confidence ratings (scale 1 to 5) for navigation and AOI identification for both VC and map only (averages and standard deviation). VC Navigation
VC AOI Identification
Map Only Navigation
Map Only AOI Identification
Average
4.4
4.0
3.4
3.0
Std Dev
0.79
0.58
0.55
0.71
This round of testing then led to subsequent testing with a team of astrobiologists who used the VC during a science expedition to Craters of the Moon National Park in Idaho. The scientists used the device in a different way than in the initial testing. They used it to collect data and collaborate while exploring in the field, see Figure 4. This testing then further demonstrated an observation of the ability for a tablet interactive technology to assist in collaboration while exploration is being conducted. This observation was initially made while doing the first round of testing previously described. This illustrates an advantage of the participatory design approach. Emergent behaviors can be observed and then further testing and evaluation tailored to further explore the behavior observed. Part of the goal of the expedition was to use a hand-held xray fluorescent spectrometer to map the concentration of elements in the rocks around lichen outcroppings. This device produced a read-out of the elemental concentrations. The VC was then used to capture the element compositions and these were then combined with a picture taken of the site showing where the data was collected and other scientific notes about the sample. The VC then provided a geo-located scientific record of the measurement including observations and other documentation. For the field tests, where it should be noted the scientists were evaluating the device on their own, they rated the VC about the same as current systems for interaction and facilitating scientific discovery. The encouraging results are that overall the VC rates very favorably for scientific
Figure 4. Scientists Collaborating while Conducting Field Exploration with the VC. Conflict resolution and finding points to explore faster were the two areas with the highest comparative rating for the VC. This is encouraging as well for a device intended to be used as a decision support aid and improvisation tool when outside expertise may not be available. The VC was also found to be useful even when cell phone coverage was nonexistent in the field. Points could still be dropped onto the map and geo-located with the GPS. Then when a link-up was again possible with cell phone coverage the Google Map would update and the points would be uplinked to the server so that others could see them. IMPLICATIONS EXPLORATION
FOR
HUMAN
DEEP
SPACE
Testing of the VC tablet technology has shown patterns that should be evaluated further for use of annotated database technology and tools for collaboration in deep space exploration. Testing of the simulated communications delay with the Earth showed the need for on-board decision support technology such as the VC to assist astronauts with the exploration decision-making process. The confidence shown by test subjects in navigating with the VC should be further evaluated as well as the ability to collaborate
together remotely. The VC connectivity allows the capture and sharing of real-time exploration knowledge which can be used to determine the next step in a traverse. It can also be a safety aid to help in determining where your fellow explorers currently are and if they need assistance.
of deep space exploration vehicles and to aid surface explorers. Risks abound in this environment so new technology needs to be applied in a human-centered way to assist this exploration. One such technological tool is the Virtual Camera for Human Deep Space Exploration.
The VC has been developed during this research for surface exploration. This development can be continued with enhanced hardware intended for the harsh surface environments encountered in space. Hardware-wise, a tablet device would need to be made more robust for use in space-surface exploration. Dust, radiation, vacuum and battery safety concerns would all have to be addressed.
The VC has been developed using a human-centered participatory design approach showing the usefulness of this technique for risk-intensive systems. The humancentered design tenets of early stakeholder involvement and iterative design, development and testing were applied. They allow a flexible design process which can consider elements of risk-critical systems and design decisions to be made before they are used in the actual safety-critical environment. This work is a first point design demonstrating the basic capabilities of the VC. It provided a preliminary evaluation of the tool and demonstrated some emergent behaviors and uses for the VC. It also demonstrated that many domains beyond deep space exploration can be served by a VC tool. The emergent features discovered during this research for the VC, such as the utility of collaboration in the field and sharing new knowledge quickly can be used in the next level of VC systems. This would allow an agent-based architecture that is flexible and easily modifiable.
We now realize that the VC will assist in situation awareness for exploring the unknown but the other emerging area observed was the collaboration possibilities. The evaluations completed so far are very preliminary and really just designed to determine if a VC is useful for assisting exploring in the unknown. Further testing needs to be done to determine more quantitatively how well the VC assists in situation awareness and collaboration for exploring unknown environments. Testing should be done with field teams of experts operating in scenarios specifically designed to evaluate the VC. One possible location and scenario for testing would be Devon Island Haughton-Mars Project. Each summer researchers use the site in the arctic for Mars analog studies [4 Lee, et al, 2013]. Scenarios such as multiple explorers in the field, sharing points of interest and conducting complementary exploration could collaborate through the VC. A mission control home base would communicate with the explorers with a simulated communications delay. Off nominal situations could also be explored such as loss of communication with a simulated mission control for one crew member with assistance then provided through the VC. If possible, expert users such as astronauts, mission operations personnel and planetary geology scientists should also be involved for planning and execution of each simulated traverse. In this proposed test, surveys consisting of subjective ratings and comments can be used to evaluate performance. The number of activities completed (sites explored for instance) with both the VC and standard procedures and delayed communications with “ground control” would be collected. Execution logs and all voice communications would be collected for evaluation. Workload rating scales such as NASA TLX or Bedford could be used as well. CONCLUSION
Human space exploration will change noticeably as we move further into the solar system. Roles will change and astronauts will have more responsibility for local decision making. Ground-based expert knowledge will still be very important but will not always be available in real-time. Tools are needed to bring this knowledge into the cockpits
ACKNOWLEDGMENTS
We would like to thank Darrell Boyer, Luke Cremerius and Matthew Long who have provided software development support for this project. This research was supported, in part, by the FrenchAmerican Project “Risk Management in Life Critical Systems” funded by the Partner University Fund. I also appreciate the help of Dr. Chris McKay and all of the Spaceward Bound 2013 participants, led by Angela Farnham-Banks as well as the evaluation participants. REFERENCES
1. Boy, G., et al. The Virtual Camera: A Third Person View, Third International Conference on Applied Human Factors and Ergonomics, July 2010. 2. Hutchins, E. Cognition in the Wild. MIT Press, Cambridge MA, 1995. 3. Grudin, J. and Poltrock, S. In S.W. Kozlowski (Ed.), Handbook of Organizational Psychology. Oxford University Press, 2012, 1323-1348. 4. Lee, P., et. al. Haughton Impact Crater and Surrounding Terrain, Devon Island, High Arctic: A Multi-Mission Mars Analog Science Site, Analog Sites for Mars Missions II (2013), Washington, D.C.. 5. Mackay, W.E. Ethics, lies and videotape, in Proceedings of CHI '95 (Denver CO, May 1995), ACM Press, 138-145. 6. Schwartz, M., and Task Force on Bias-Free Language. Guidelines for Bias-Free Writing. Indiana University Press, Bloomington IN, 1995.
