iSPAM: Using Handheld Technology to Assess Situation Awareness in ...

1

SHORT COMMUNICATION

iSPAM: Using Handheld Technology to Assess Situation Awareness in Simulation Cliff Shelton1 & Adrian Molyneux2 1

North Western Deanery, Manchester, UK (work undertaken at School of Medicine, Keele University, UK) School of Medicine, Keele University, UK

2

Abstract Situation awareness is of vital importance to patient safety in the complex environment of healthcare. In this short communication we report the adaptation of the situation present assessment method (SPAM) for use in simulated healthcare practice. This was achieved through the development of a mobile application (app), which can be used by educators and researchers to assess situation awareness. We present a three-stage approach for using the app to facilitate debrief in the education setting. Keywords: simulation, learning technology, human factors, situation awareness

Corresponding author: Cliff Shelton, Department of Anaesthesia, University Hospital South Manchester, Southmoor Road, Wythenshawe, M23 9LT, Manchester, UK Email: [email protected], Phone: +44 (0) 7806 771 901 © 2014 S.P. Forrest, The Higher Education Academy

Figure 1 A situation awareness query, delivered by the iSPAM app.

HSCE, Vol 0, Issue 0 (Month 2014) doi:10.11120/hsce.2014.00046

2

Handheld technology in situation awareness simulation

Situation awareness Situation awareness (SA) can be defined as ‘the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future’ (Endsley 1995). The World Health Organisation has identified SA as a human factor of critical importance to patient safety

(Flin et al. 2009). Although whole-patient simulation often aims to examine and debrief human factors, SA is cognitive in nature and therefore cannot be observed directly. This has necessitated the development of a variety of measurement tools which operationalise SA in terms of outcomes, biometric measurements, or the response to queries about the situation (Salmon et al. 2006). Query-based tools

Figure 2 Example situation awareness debrief dialogue. © 2014 S.P. Forrest, The Higher Education Academy


Shelton & Molyneux

include ‘freeze and probe’ techniques which require the subject to answer queries during a pause in the simulation throughout which the clinical information is concealed, or ‘real-time’ in which queries are presented as the situation unfolds. One such realtime technique is the Situation Present Assessment Method (SPAM) (Durso & Dattel 2004). Situation awareness measurement tools have been used in high-risk industries for some years, however their implementation in healthcare is in its infancy. Thus far, SA assessment in healthcare simulation has relied on outcome measures which do not necessarily represent SA, and ‘freeze and probe’ techniques. Keele University’s adaptation of SPAM to critical illness simulation represents the first use of a SA measurement technique in healthcare which quantitatively measures SA without the need to interrupt the simulation (Shelton et al. 2013). This mitigates the influence of recall, integrates with exiting facilities and permits the learning activity to proceed at normal pace. SPAM presents the subject with queries regarding their situation, which must be answered as quickly and accurately as possible. The latency between query and answer is a proxy measure of SA, with a faster response indicating superior SA. Latency, more so than accuracy, is theorised to represent SA in the real-time context as the information of interest remains accessible throughout the assessment and an accurate response can therefore be expected. In a complex situation such as the management of an unwell patient a practitioner is unlikely to be able to maintain equal awareness of all of the elements within their environment. Therefore, queries relating to information that is

References Durso, F.T. and Dattel, A. (2004) SPAM: The real-time assessment of SA. In A Cognitive Approach to Situation Awareness: Theory and Application (eds. S. Banbury and S. Tremblay). Farnham: Ashgate. Endsley, M.R. (1995) Toward a theory of situation awareness in dynamic systems. Human Factors 37, 32–64.

known already can be answered more quickly than those relating to information which needs to be sought out (Durso & Dattel 2004). Two SPAM procedures were described in the literature; query delivery by telephone or in-person. These procedures were qualitatively assessed in the simulation setting and this led us to adapt the method for healthcare. To achieve this we developed an iOS app called iSPAM, which delivers queries via an iPod Touch, accurately measuring the latency data, and presenting it to the instructor for use in debrief (Figure 1). This use of learning technology presented significant advantages over the existing procedures, which tended to distract the participant’s attention from the scenario or be misinterpreted as hints (Shelton et al. 2013). The Keele University simulation faculty is developing experience integrating SA data into debrief, and evaluation of the simulation sessions has indicated that students find the app easy to use and intend to modify their practice in the future as a result of the SA feedback they receive. The typical SA debrief dialogue uses a three-stage approach (Figure 2). The same debrief structure is used when discussing poor SA, but this example dialogue illustrates one of the advantages of SPAM: using latency data instead of accuracy alone allows the instructor to observe increments of success in SA and deliver credible positive feedback to learners, reinforcing good practice. Keele University has published a ‘lite’ version of iSPAM to the Apple App Store, which is available free of charge for download. We plan to release the full version in the near future.

Salmon, P., Stanton, N., Walker, G. and Green, D. (2006) Situation awareness measurement: a review of applicability for C4i environments. Applied Ergonomics 37, 225–238. Shelton, C.L., Kinston, R., Molyneux, A.J. and Ambrose, L.J. (2013) Real-time situation awareness assessment in critical illness management: adapting the situation present assessment method to clinical simulation. BMJ Quality and Safety 22, 163–167.

Flin, R., Winter, J., Sarac, C. and Raduma, M. (2009) Human factors in patient safety: a review of topics and tools. Geneva: World Health Organisation.

© 2014 S.P. Forrest, The Higher Education Academy


3