Taking lessons from teleconference to improve same time, same place ...

Taking lessons from teleconference to improve same time, same place interaction Bridget Kane and Saturnino Luz Department of Computer Science O’Reilly Institute Trinity College, Dublin, Ireland [email protected] [email protected]

Abstract Performance on an information gathering task is shown to be superior in teleconference. Analysis of errors in an exercise revealed the data sources used in co-located and teleconference scenarios. The use of a visual display for text data, in addition to the audio source, is demonstrated in both co-located and teleconference discussions. Audio was used as a source of information more frequently in teleconference which resulted in an overall improvement in task performance. The lesson learned from the higher performance in teleconference, can be applied to improve performance at colocated meetings. Providing appropriate visual data together within audio enhanced spaces can be expected to improve the communication event and reduce medical errors. Results support proposals for the incorporation of physical spaces to improve communication in everyday work in cooperative workplaces, such as hospitals.

1. Introduction Interaction in teleconference tends to be compared with face-to-face settings and effort has traditionally focussed on improving the technology to meet face-to-face standards, or overcome the challenges of distance [2, 13]. In this paper, a situation is identified in which performance is improved in teleconference, compared to co-located setting. The lesson learned from behaviour in teleconference can be used to apply technology to improve the ‘natural’ setting of the interaction. The need for both audio and visual sources of information to improve communication and the exchange of information is reinforced in the results reported here. The setting examined is a multidisciplinary team meeting (MDTM) and this work is part of an on-going study analysing the work of multidisciplinary teams (MDT) at their meetings. MDTMs are fora where medical specialists come together to discuss patient cases, agree the definitive diagno-

sis and plan treatment. Collaboration is achieved through speech interaction among the senior team members. Some roles use images to supplement their contribution, such as radiology and pathology, but the principle mode of communication is through speech interaction, as in other areas of healthcare work e.g. shift handover and ward rounds. In all of these situations, inter-professional and inter-specialist communication can be observed. Information is presented, evaluated and exchanged in discussion among individual clinical staff involved in patient care. Problems in the exchange and sharing of information are among the root causes of medical errors (65% [9]) and can give rise to substantial clinical mordbidity and mortality [6]. Communication failures can be expected to result in patient death for approximately 70% of incidents [11]. Communication breakdowns within teams, where complex social processes can interfere with the exchange of information, contribute significantly to errors [10], particularly with trainees (70%) [14] and have the potential to affect patient care [6]. For MDTMs, that have become the central hub for patient management, it is critical that communication among team members is supported and medical errors are reduced. Information communication technologies (ICT) have an important role to play in advancing the quality of health services through such fora as MDTMs. In order to improve the communication process in healthcare, and reduce the incidence of medical errors, the media and modes of communication by which communication is achieved need to be examined to inform the design of the technology appropriate for the setting. This paper reports on an exercise conducted among observer participants at MDTMs to measure the ability to gather specific items of data from discussion, and the accuracy with which those data are noted. Teleconference and co-located discussion scenarios are examined as well as the use of a text display during proceedings.

Kane, B., and Luz, S. Taking lessons from teleconference to improve same time, same place interaction. In Proceedings of the 21st IEEE International Symposium on Computer-Based Medical Systems (2008), IEEE Computer Society, pp. 494– 499.

2. Background Research on providing technological support for collaboration within teams has tended to be dominated with studies on virtual teams engaged in physical tasks with the focus on task performance [4]. Resolving the special communication difficulties caused by time and distance has engaged much research work [8], sometimes at the expense of providing improvements for same time, same place collaboration on non-physical tasks. Those studies that have examined colocated team work have tended to examine workers engaged in visible tasks such as architectural design, simulation or computer programming groups [12]. Although a need for dedicated rooms in co-operative workplaces is long recognised [1], and architectural solutions are proposed [15], these tend to concentrate on interactive walls / white boards, tables or furniture with integrated interactive devices for drawing, etc. but have not yet incorporated audio enhancement for co-located interaction. In healthcare there are many situations where collaboration among staff, including inter-specialist collaboration, is achieved through talk and the use of artefacts, such as radiological images. Text based reports and summaries are used in record keeping and filed text reports traditionally serve as the definitive record. In the course of day-to-day management of patients, however, much of the work is conducted through talk and speech interaction among health professionals. Meetings are commonplace in teamwork, for such purposes as such disease staging and treatment planning. Although communication in medicine is an established research area, little attention has been given to those situations, such as team meetings, where different roles and clinical specialities interact through talk.

3. The meeting room The layout of the meeting room is shown in Figure 1. MDTM attendees sit facing the plasma screen display on which the radiology and pathology are demonstrated, and through which the remote party is made visible in teleconference. A template with patient details is projected onto a SmartBoardTM on the adjacent wall, at right angles to the main display and marked as ‘Large Display’ in Figure 1. R , in The videoconferencing system, named Telesynergy use is a HIPAA-compliant video conferencing system with the components necessary for clinicians to collaborate with one another [5], represented in Figure 1. Multi-imaging capabilities integrate images to be used together during a video conference. When connected in teleconference, audio from remote sites is broadcast in the meeting room over a centrally placed loudspeaker, shown in Figure 1. The same technology is used for all MDTMs, i.e. for both co-located and teleconference patient case discussions.

Sign-in sheet

Speaker

Plasma screen

Radiologist

Pathologist

Operator

Lead Clinician

Most active participants

Nurse

Exercise participants

Data manager

MDT Coordinator

Large Display

Server

MDT members

Figure 1. The meeting room layout.

3.1

The participants

For MDTMs, a number of hierarchical teams come together. Senior members of the MDT are the main contributors. Other attendees can be classified as ‘observer participants’ as they are present for educational benefit or to fulfil other functions, such as data gathering for on-going audit purposes. Some observer members of the team take notes for tasks to be conducted post-discussion. Appointments may need to be made or a patient may need to be scheduled for special procedures, for example.

3.2. Conduct of proceedings An agenda with a list of patient names and identifiers is circulated beforehand so that individual team members will prepare the necessary information for discussion. Each individual patient is discussed separately and the patient’s name and age are articulated at the outset of the case presentation by a member of the multidisciplinary team (MDT). The opening of a case discussion typically provides important items of information that will be necessary in making the diagnosis and deciding on the most appropriate course of treatment for the patient, such as: “Jane Doe is a 40 year old woman who presented to A+E with shortness of breath. She reported a 3 month history of cough with dark sputum. She has weight loss and loss of appetite in the past month. On examination she was found to have clubbing. Her performance status is 2. ” For the duration of a patient case discussion, a page of text is projected onto a SmartBoardTM on the side wall of the meeting room. (Figures 2 or 3. Figure 3 is the current version.) The page on display presents the patient’s name and date of birth and makes available certain patient facts that are known, and considered critical, to the discussion in which the clinical specialists are engaged. Significant


Figure 2. Original template used to display patient details during MDT discussion. (Fictitious information for illustration.) Figure 3. Revised template in current use. (Fictitious information for illustration.) symptoms or clinical findings are normally included in the page on display as well as being articulated verbally in the case discussion (but not always). In addition to providing a means of having facts available throughout the discussion, the display of the page with the patient’s details is an awareness mechanism, or prompt, of the progress of the meeting and of the patient being currently discussed. The original template shown in Figure 2 was considered difficult to read, particularly with respect to the clinical information and was revised to the version in Figure 3. The original template was in landscape view using Arial 18 and bolded text. The revised template contains most of the same information, but has differences. The page is designed to be included in the patient’s paper chart after the meeting, when a summary of the information presented at the meeting and the decision made is inserted in the lower half of the page. The upper half of the page is displayed during the discussion. Information is ‘grouped’ and the font style is Book Antiqua size 10 plain text. Instead of a single line to insert patient symptoms (as in the original template), a free text box. 2.25cm x 16.83cm is incorporated, for ‘clinical summary’. (The revised template is currently undergoing further revision.)

4. Method Weekly meetings of the respiratory multidisciplinary team (MDTMs) were observed for an extended period. Over a set of six, non-consequetive, meetings over a three month period, some of which included a video-conference link to another hospital, an exercise was conducted among the observer participants. The use of observer participants serves as a proxy measure for communication effectiveness among the active participants, some of whose task is to take notes during the proceedings. Video recordings were taken, in conjunction with the information gathering exercise at

the meeting. The results reported here are confined to a set of questions in an exercise that has particular relevance to the audio and visual support provided at the meeting. Analysis of the errors made in exercise questions revealed the information source from which respondents gathered the data required to answer the questions. The questions relevant to this paper, that observer participants were requested to answer, were the name and age of the patient being discussed and the symptom that the respondent felt was most significant in arriving at the patient’s diagnosis. For two out of six MDTMs at which the exercise was undertaken, the template used for the display of information on the secondary display was changed. The original template is shown in Figure 2 and the revised template in Figure 3. Both versions provided the patient’s name and date of birth, the main symptoms and clinical findings to date of the discussion, see Figures 2 and 3.

5. Results and Discussion 345 response sheets were returned from 115 patient case discussions, that took place over the six meetings. Of the 345 responses, 260 were completed at co-located discussions and 85 were obtained at teleconference discussions. The majority, 266, were completed with the original template in use and the remainder, 79, were completed during discussion that utilised the revised template. 24 responses were completed in teleconference when the revised template was on display. These responses are summarised in Table 1. In the following subsections, it will be demonstrated how both audio and visual sources are shown to have been used for gathering the data to complete the questionnaire.


Table 1. Responses available for analysis. Condition Co-located Teleconference Total Old Template 205 61 266 New Template 55 24 79 Total

5.1

260

85

345

Patient’s Name

In response to patient’s name, respondents were generally able to provide the correct patient’s name and there was a non-response rate of 1.4%. Evidence that the text source was used was provided in the ability of respondents to correctly spell some unusual names (or common names with unusual spelling). Evidence that audio sources were also used was provided in the submission of names mis-spelled, or spelled correct phonetically. For example, ‘Valdemer’ vs. ‘Waldermar’, ‘Annmarie’ vs. ‘Anne-Marie’ or ‘Bridget’ vs. ‘Brigid’.

5.2

Patient’s Age

The use of the secondary screen display is also illustrated in the error responses to questions on the patient’s ‘Age’. In order for the respondent to submit the correct age for the patient, either a calculation could be made from the screen display of the date of birth presented, or the respondent could listen for the age in years being articulated during the presentation of the patient details in the opening of the case. There were some examples of the incorrect age being verbally articulated in the case presentation with the correct date of birth in the text display and another example where the text date of birth was not available but the age was articulated during verbal presentation. For most cases, when the correct age was given, having been articulated and available in text display, it was not possible to identify the source used for information. However, the response submitted often yielded information from where the data were gathered. A summary of responses is given in Table 2.

Table 2. Result summary to question on patient’s ‘Age’, as % of questionnaires. Discussion No Correct Incorrect Scenario Answer Age response Overall 10.7 64.6 24.7 Teleconf. Co-located

2.4 13.5

67.1 63.8

30.5 22.7

Examination of the incorrect submissions revealed the source used to gather the information. For example, there were several errors in calculation, where the patient age submitted was in difference by 1, 10, or other number that was explained when the month, year and/or day was taken into account, rather than the year alone. The patient’s age in Table 3 was 45 in October 2007, and the patient described in Table 2 was 65 in September 2007. For these examples ages were submitted as 44, 46 and 48 and 66 and 68 respectively. In some cases the difference between the submitted and the actual age was 20 and 40 years in difference, which is likely due to a subtraction error. There were also occasions when the age was wrongly articulated in the opening of the case by the presenter, the date of birth was not on display, and the verbalised age was submitted in the questionnaire. Thus, comparing the submitted response with the text displayed and the recording from the meeting, a number of responses could be traced to the audio or text source for information. A breakdown of the identifiable sources for ‘Age’ data is given in Table 3. The ambiguous results are those where there was one year difference in the ‘Age’ submitted, but this number was articulated as the ‘Age’ in discussion. Thus, it is not possible to differentiate with certainty between calculation errors of one year, or if the audio source is being accessed. The incorrect ‘Age’ was articulated more frequently in teleconference.

Table 3. Summary of the information source used to access the patient’s ‘Age’, as % of questionnaires. Discussion Audio Visual text Ambiguous Scenario Source display Overall 2.9 16.5 5.2 Teleconference 3.5 13 14.1 † Co-located 2.7 17.8 2.3 †Pearson’s Chi-Square = 40.76, df = 14, p < 0.01

When submissions from the co-located and teleconference case discussions are compared, a couple of important differences are observed. In the first instance the reponse rate is significantly higher in teleconference discussions, shown in Table 2. The response rate in teleconference discussions for ‘Age’ is 97.4% compared to 86.5% for colocated discussions. There is less ambiguity in many cases. Assuming that the level of incorrect submission correlates with utilisation of that source, results show that the audio source is accessed more frequently in teleconference. Table 2(b) demonstrates the differences between teleconference


and co-located discussions. Although there is some ambiguity on the extent of the errors in calculation and those that accessed ‘Age’ via audio, the combination of results reveals that the audio data source was used more than the text data source in teleconference, when loudspeakers are being used to broadcast speech from the participants on the remote side of the interface.

5.3

Use of text template

Overall, the format of the template appeared to have an effect on the response and correct rates in submission, shown in Table 4. Table 4. Result summary to question on patient’s ‘Age’, as % of questionnaires. Discussion No Correct Incorrect Scenario Answer Age Response Overall 10.7 64.6 24.7 Old template New template

3.8 34.2

70.7 44.3

25.5 21.5

Fewer responses were submitted following introduction of the revised template, but when a response was submitted, it was more likely to be correct. When incorrect responses were examined with respect to the template being used at the time of the test, the importance of clear text display is highlighted. Result is shown in Table 5.

Table 5. Breakdown of incorrect submissions for ‘Age’ when different templates were in use, as % of questionnaires. Discussion Audio Visual text Ambiguous Scenario Source display Overall 2.9 16.5 5.2 Old template 3.8 18.9 3.0 † New template 8.9 12.7 †Pearson’s Chi-Square = 88.71, df = 14, p < 0.01

5.4

Significant symptoms

Respondents were reluctant to submit a response to the question “What sign or symptom do you think was probably most significant in arriving at this patient’s diagnosis?”. Table 6 shows that less than half of respondents attempted to answer the question (46.7%), even though there were clinical particulars articulated and in the text on display. When

submissions were reviewed for those in teleconference and co-located discussions, more responses were submitted in teleconference cases (56.5% vs. 43.5%) and the symptoms given tended to be symptoms articulated in conversation across the interface, i.e. broadcast over the loudspeaker.

Table 6. Responses ‘Significant Symptoms’ available for analysis, as % within group. Template Used Teleconference Co-located Overall Old 59 42 45.9 New 50 49 49.4 Total

56.5

43.5

46.7

Examination of the submissions identified the following behaviours with respect to the pattern of use of the text display: a. Words submitted were used that were visible on display, in both co-located and teleconference settings; b. Terms were submitted that were not on display, but were articulated in discussion in both co-located and teleconference settings; c. Blank responses were submitted in both co-located and teleconference settings; d. For some co-located discussions that used the old template and had no symptoms available on display, no submission was made from any respondent for that case; e. For any discussions in teleconference for which symptoms were not available for display, submissions were made from some respondents; f. When a lot of text was presented on the screen display, particularly when the revised template was used, respondents did not offer any symptom in response to the question. Generally, the introduction of the revised text did not result in symptoms being offered more often, presumably because the clinical details in the revised template were less clear than in the old template (45.9% response when old template was used compared to 49.4% submissions when the revised template was used.) It can be concluded from the pattern of results that i) both the audio and visual sources were accessed to obtain the information necessary to respond to the question asked regarding the patient’s symptoms for both co-located and teleconference discussion; ii) in teleconference it was easier to access the patient’s symptoms from the audio source, iii) having limited amount of text on display made it more likely for that data to be captured and iv) that too much text on display made information more difficult to access on the text source.


6. Conclusion It is important that information can be accurately exchanged in clinical settings. If information cannot be shared and exchanged without the generation of error, there are serious consequences. Instances of patients being wrongly identified for treatment or intervention increases the risk of error and it is a worry for patients [3]. Up to now, effort to provide technological support for group work has tended to concentrate on different place and time issues for physical tasks [8]. Designers of collaborative tools for group work, have tended to engage in the development of interactive displays or asynchronous communication tools [12]. Furthermore, developers tend to look towards behaviour in the natural settings when designing technology. Here, we use the experience in teleconference to gain insight into how co-located working might be improved. The analysis of use of the different modes of communication provides evidence for the usefulness of dual sources of data in the communication of important patient details in interactive settings. In addition to demonstrating the usefulness of dual data sources (audio and text display), the exercise also demonstrates the importance of appropriate amounts and format of data. Too little text on display, yielded low rates of success; too much text on display made the relevant data inaccessible. The results in this study support the case for ‘roomware’ and proposals of those who argue for architectural spaces combined with digital information spaces, such as [15]. The incorporation of audio enhancement is argued here to be of prime importance, however. The importance of audio is often underestimated and it is easy to forget that one in four adults can be expected to have a unilateral or bilateral hearing impairment [7]. In a work situation such as healthcare, where workers collaborate, share and exchange information through talk, having clear audio is vital. Textual display of appropriate information is also shown to have a high value and can reduce error rates. The development of architectural spaces, integrated with tools for collaboration that include text displays and enhanced audio, has the potential to bring enormous benefits for patients and services. As well as contributing to reduced error rates and improved service, such ‘rooms’ could also be equipped with recording capabilities to capture data for incorporation into the electronic patient record.

7. Acknowledgements We gratefully acknowledge the on-going co-operation and support of Dr. Finbarr O’Connell and all the members of the respiratory MDT at St. James’s Hospital, Dublin. We also thank Professor Donal Hollywood and the School of

Radiation Therapy, Trinity College Dublin. This work is funded by a grant from the Science Foundation, Ireland under the National Development Plan.

References [1] S. A. Bly, S. R. Harrison, and S. Irwin. Media spaces: Bringing people together in a video, audio, and computing environment. Communications of the ACM, 36(1):28–46, 1993. [2] P. Bordia. Face-to-face versus computer-mediated communication: A synthesis of the experimental literature. Journal of Business Communication, 34(1):99–118, 1997. [3] T. E. Burroughs, A. D. Waterman, T. H. Gallagher, B. Waterman, D. B. Jeffe, W. C. Dunagan, J. Garbutt, M. M. Cohen, J. Cira, and V. J. Fraser. Patients concerns about medical errors during hospitalization. Journal of Quality and Patient Safety, 33(1):5–14, 2007. [4] L. Carles. Benefits and limits of video channel on mediated interactions. Technical Report 1, Geneva Interaction Lab, 2001. [5] Center for Information Technology and National Institutes for Health. Telesynergy - a telemedicine partnership, April 17 2007 2007. [6] E. Coiera. When conversation is better than computation. Journal of the American Infrormatics Association, 7(3):277 – 286, 2000. [7] A. C. Davis. The prevalence of hearing impairment and reported hearing disability among adults in great britain. International Journal of Epidemiology, 18(4):911–917, 1989. [8] K. E. Finn, A. J. Sellen, and S. B. Wilbur. Video-Mediated Communication. Lawrence Erlbaum Associates, Inc., 1997. [9] J. C. for Accreditation of Healthcare Organizations. Root causes of sentinel events, 2008. [10] A. A. Gawande, M. J. Zinner, D. M. Studdert, and T. A. Brennan. Analysis of errors reported by surgeons at three teaching hospitals*. Surgery, 133:614–621, 2003. [11] M. Leonard, S. Graham, and D. Bonacum. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care, 13(suppl 1):i85–90, 2004. [12] G. Mark. Extreme collaboration. Communications of the ACM, 45(6):89–93, 2002. [13] S. E. Poltrock and J. Grudin. Videoconferencing: Recent experiments and reassessment. In HICSS ’05: Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS’05), page 104.1. IEEE Computer Society, 2005. [14] H. Singh, E. J. Thomas, L. A. Petersen, and D. M. Studdert. Medical errors involving trainees: A study of closed malpractice claims from 5 insurers. Archives of Internal Medicine, 167(19):2030–2036, 2007. [15] N. A. Streitz, J. Geibler, and T. Holmer. Roomware for cooperative buildings: Integrated design of architectural spaces and information spaces. In Lecture Notes in Computer Science, volume 1370/1998 of Lecture Notes in Computer Science. Springer, Berlin, 2007.
