Collaborative Workspace for Multimedia Medical Conferencing Han-joon Kim, Eun-Soo Park, Sang-goo Lee, Yeong-Gil Shin Department of Computer Science, Seoul National University, Seoul, Korea Abstract We propose an approach for collaborative workspace management in medical conferencing. A collaborative workspace is a virtual data space shared between medical experts for working out solutions collaboratively while conferencing. Our approach provides medical users with an integrated view of various kinds of multimedia patient data and a unified control over the workspace. For data navigation and conferencing, a tree-like navigation tool, which we named the patient record tree, is provided. And we classify patient data, which is the object of medical collaborative works, into six basic types, and provide a view template for displaying each of these types. Keywords: patient record, conferencing system, shared workspace
Introduction Recent medical trends towards specialization, geographical distribution of medical experts, and uneven distribution of expensive medical equipments elevated the need for medical experts in remote locations to be able to collaborate. Improvements of network resources and multimedia technologies have made it possible to satisfy these needs through medical teleconferencing, or telemedicine [1]. It has been argued that to perform remote activities through conferencing in medical environment is more helpful and more effective than in any other domain [2]. The conferencing systems enable medical experts (or expert groups) to perform remote medical activities, such as diagnosis for remote patients, exchange of advanced medical knowledge, quick attainment of new treatment from remote experts, and remote medical education, etc. A medical conferencing system needs to have a shared workspace of a collection of patient records. The patient record includes various kinds of media, such as graphic material with X-ray image and 3D magnetic resonance image, audio/video records, and written notes. Also, a patient record has a complex structure and has a wide variety of representations. So, it is not easy for medical experts to quickly locate and understand a certain parts of patient data to work out solutions. Furthermore, if a patient record is governed by more than one hospital, it may become unmanageable. We
assume that our system is level 31 of electronic patient record system defined in [3]. The shared workspace in medical conferencing system must be equipped with its own special facilities so that medical professionals can view patient data in a coherent and consistent way and communicate with others through a convenient graphical user interface. The workspace must be more than a mere interconnection of different applications. We shall term the shared space ‘collaborative workspace’ to emphasize medical ‘collaboration’ among medical experts. The collaborative workspace in medical conferencing system is a virtual shared space among geographically distributed medical experts for jointly observing and interpreting the images and other information of interest, discuss about a diagnosis, and present solutions or treatments for medical care. We propose an approach for collaborative workspace management in medical conferencing that allows effective patient data navigation. This collaborative workspace provides an integrated view of multimedia patient record and includes both data navigation and conferencing facilities. We have developed a prototype conferencing system implementing this approach. Our system supports satisfactory facilities for collaborative work on multimedia patient data stored in a central medical database. A tree-based navigation menu, which is the functional core of our collaborative workspace, is provided. We call the menu patient record tree (PRT). For multimedia data display, different types of view templates are supported. And for the image and video data display, pop-up viewer is provided to tackle the problem of limited display space. As for conferencing facility, the medical conference sequence can be adjusted through manipulating the conference plan. Also, synchronization control for gesture support is performed by recognizing control information embedded in patient data. In this paper, we briefly outline medical conferencing in our system and describe the components and functionality of 1
Medical Records Institute defines 5 distinct levels of computerization for patient information system. The patient record of level 3 is governed by one enterprise, and is completely automated record in all its aspects, which is called ‘The Electronic Medical Record’.
medical collaborative workspace. In the next section, we explain how collaborative workspace is implemented, and evaluate the result of our works. And finally conclusions and future works are presented.
Medical Conferencing In general, conference initiator who takes charge of supervising a conference prepares materials before establishing a conference session. Thus our system provides two different modes; Search/Preparation mode and Conference mode. In the Search/Preparation mode, the initiator prepares materials for medical tele-collaboration. He/she navigates through the relevant patient data asynchronously at his/her local site, selects the patient data for the collaborative workspace, and composes the conference scenario. Through this process, a conference plan is generated, which contains metadata about the conference schedule.
The collaborative workspace should be flexible enough to meet various conference contexts for different medical communities [4]. Another reason why flexibility is required is that not all patients have the same set of patient data because medical examinations depend on the type of illness the patient has. Components In this section, we elaborate the components for medical collaborative workspace that meets the requirements described above. Figure 1 shows the internal structure of the collaborative workspace. It consists of Patient Data, Conference Plan, and Workspace Control. The Patient Data is classified into 6 categories according to the characteristics of the data. The Workspace Control consists of the patient record tree , sequence control, and synchronization control. It loads patient data from the database into the workspace and provides a suitable view for each data type. M edical Collaborative W orkSpace
In the Conference mode, the initiator directs the proceeding of the conference. This is synchronized with other participants based on the conference plan in the prepared collaborative workspace. All the participants in the conference interact with each other through the graphically represented workspace while sharing the conference material.
Mem o
WorkSpace C ontrol
Patient Data
G rid Table Sprea dsheet V ideo
Patient Record Tree
Im age Sound D erived Data
Synchronization Control
Medical collaborative workspace
Sequence Control
Profile
Conference Plan
View Sequence Derived Data Info
In this section, we describe the collaborative workspace for medical conferencing. We begin by introducing requirements of the collaborative workspace in the medical domain. Then we present different parts composed of the workspace while explaining their functionality. Requirements It is essential to provide the collaborative workspace in a way that the user is allowed to organize the various forms of conference-related data, and to present the materials systematically for better medical conferencing. As mentioned earlier in a medical environment, various complex types of data are generated and accumulated for each patient, i.e., medical history, examination, medication records, notes, and different diagnosis data, etc. Therefore, an integrated view of these patient data must be provided for the medical collaborative workspace. Using this view, medical experts are able to locate and organize the necessary data easily as well as to systematically communicate with other members. In this context, the user must have a unified control over the medical collaborative workspace. As a result, simple user-friendly graphical user interface should be provided. The critical nature of the patient information requires highquality display technology, and so, needs support for comprehensive multimedia capability. Efficient use of the limited display space is necessary in order to observe a large amount of disparate data without confusion.
annotation
Figure 1- Components of Medical Collaborative Workspace The patient record tree, which forms the core of the collaborative workspace, can put the workspace under a unified control. The Workspace Control also regulates the flow of medical conference by manipulating the Conference Plan. All the control information in the workspace is stored in the database. Patient Data The patient data is classified into six types by the characteristics of each data type; Memo, Grid Table, Spreadsheet, Image, Audio, and Video. •
Memo is textual data appearing in chief complaints, system reviews, notes, history, and examination notes, etc. It is also used for descriptions of other data.
•
Grid Table is used to accumulate examination results of CT, MRI, X-ray, or admission history. Its row is associated with the related multimedia (image, video, or audio) data. Generally, each row is composed of several fields including examination dates, departments, simple notes, etc.
•
Spreadsheet is provided to display the numeric values of each examination in the form of tables. For example, most of the examination data out of clinical pathology can be represented by this type. Although its external shape is similar to that of grid table, it differs in the characteristic of row data. A row of spreadsheet contains the numeric value of examination such as WBC, HGB, and MPV in blood examination category, not the linkage information of related multimedia data. These row values can be used to generate graphs for the transition comparison of each detailed examination. So, the view template, which will be explained later, for this type must support multiple selection capability on columns and rows.
•
Image is used for scanned images from X-ray, CT, MRI, and SPECT, etc. This type of data usually occupies very large portion of the patient record, and it plays a major role for a high-quality medical conference.
•
Audio is sound records in WAVE format.
•
Video is animated images in MPEG-II format generated from angiography, ultrasonography, and others.
Patient Record Tree (PRT) There are several approaches toward navigation through a large number of data, i.e., hierarchical view, fish-eye lens view, 3D graph, etc [4]. We adopt the hierarchical view considering the feature of patient data. Our system provides a data menu in the form of tree hierarchy. We named this tree based navigation menu the Patient Record Tree (PRT) since it can represent the whole patient record set. Using the PRT, users can access the patient data easily from the systematical viewpoint. That is, the PRT enables users to quickly navigate to any portion of the patient records in just a few clicks of the mouse. It gives participants an integrated view of many types of patient data. The basic idea of PRT is to partition patient data by departments and their examinations. Because one department can perform several kinds of examinations, so the patient data are divided hierarchically. And the leaf nodes of PRT are then linked to corresponding view instances. And, for the sake of easy adaptation to general medical environment, some facilities that allow medical communities to customize tree structure to their own preference are offered. In other words, users can add new node, change the order of nodes, or modify the structure of PRT.
Conference profile is a kind of summary information to identify the specific conference session presented to other participants. It includes author, title, case, and conference date, remarks, etc. View sequence is a step-by-step procedural definition for the conference session. The conference initiator creates this view sequence via two separate phases. In the first phase, the preliminary view sequence is generated by navigating through patient record tree and marking leaf nodes to be used in the conference (see Figure 2). In the second phase, the initiator modifies the preliminary view sequence while taking the conference scenario into consideration. For this, user can use Sequence Editor to refine the path information by deleting, copying, and moving view entries (see Figure 3).
Figure 2 – Patient Record Tree
Figure 3 – Sequence Editor
The derived data information is defined by the conference initiator to create the Grouping View and Transition Graph View. Grouping View is a special grid table view to provide the integrated view of a group of CT, MRI, or X-ray images and angiography videos as one episode. Transition Graph View is used to analyze the trends over time about the values selected from the spreadsheet data as seen in Figure 4. To display the trend, it is necessary to determine several parameters such as the examination name, the examination date or time, the range and scale, etc. That is, the supplementary parameters for displaying these special views belong to the derived data information. Grouping view can become an entry of view sequence. That is, it is a leaf node below the relevant examination node in PRT, and can be handled in the same way as other leaf nodes. But, Transition Graph View is only auxiliary view which is a subpart of spreadsheet data.
Conference Plan The conference plan contains all the information necessary to control the conference session in the collaborative workspace and it is generated in the Search/Preparation mode. It consists of the conference profile, the view sequence, annotated information, and derived data information.
Sequence Control To control the sequence of conference, the system creates and displays view instances in accordance to the view sequence. Four buttons are provided; next, previous, first, last.
Synchronization Control This control is for synchronization of gesture that enables users to observe other participant’s mouse movement.
The view template for grid table and spreadsheet type is implemented using two different OLE (Object Linking and Embedding) libraries. In particular, for the spreadsheet type, we make use of a special OLE, VSFlex2 that supports function of multiple selection. For the view templates of image, audio, and video type, we had to consider the problem of limited display space. Thus, when displaying image or video data, we chose to use the pop-up window. That is, only on demand, its viewer pops up to display the image (or video) data in an active manner, and when the user clicks on other regions, it disappears down to the bottom.
Figure 4 – Transition Graph
Implementation
And, the viewer of image data type supports simple image processing such as zooming and fully supports annotation facility as seen in other conferencing application. The viewer of video data type supports repeating function; that is, setting a certain range and playing the range repeatedly. Patient Record Tree
We present in this section the implementation of the collaborative workspace for multimedia conferencing system. The components of Workspace control have a unified control over the collaborative workspace. They provide users with an integrated view of various kinds of patient data. This is
A Patient Record Tree has a corresponding internal file, the PRT file. On entering the collaborative workspace, the PRT file is loaded into the workspace. For adjusting its structure to specific needs, system administrator can change its structure of PRT by editing the PRT file directly. Each leaf node of PRT has a query information that includes a few parameters in SQL, that is, table name, and column name. Thus, on a click of mouse on the node, the data corresponding to the query is retrieved, and then according to the type of retrieved data, the appropriate view template is displayed. If the internal patient data of a certain node is empty, the node will not be shown.
Figure 5 – Example of collaborative workspace achieved by combining data navigation facility with conferencing facility based on the PRT. Figure 5 shows that when user arrives at a leaf node by clicking nodes to browse nuclear medical examination images, view template of grid table is displayed, and then pop-up viewer is activated on double-clicking each row. View template As discussed above, we classify patient data into six basic types. We provide view templates for displaying data of each type. View templates are implemented using classes in C++, which improves productivity of implementation. Additional data types can easily be added to the collaborative workspace.
The structure of PRT reflects the procedure of examination and the sequence of medical conferencing. On PRT, hospital name, patient identifier, admission date, examination category, and examination name are hierarchically placed in this order. The number of nodes of Admission data can be more than two. The nodes of history, physical examination, and clinical examination are maintained respectively for each admission date. Control of Synchronization and Sequence The synchronization of mouse gesture, which is called gesture support, incurs such a large number of message exchanges in whole space of graphical user interface that it could degrade the system performance due to network delay. So our system provides gesture support only in those regions in which the partner’s action have to be observed. This is accomplished by placing flag information in each patient data, which indicates whether its region is for mouse pointer synchronization, or not. With this information, the 2
VSFlex OLE, a product of VideoSoft corp., is a control library that presents functions to spreadsheet or grid control. We used 1.0 version of VSFlex.
synchronization control part of workspace control performs synchronous gesture with the other party’s mouse. For all view instances except memo type, this gesture synchronization is supported at present. And in dealing with the view sequence, we found it convenient to use a circular linear list of references to view instances. The sequence editor for manipulating this list is implemented, in which view sequence can be defined and edited according to user’s preferences.
Conclusions The proposed work satisfies most requirements for collaborative workspace for medical conferencing system. We have continuously tested our system in actual medical sites to practically verify the effective value of this system. This system has been mainly used to perform remote caseconference between medical experts. Among them, the PRT has been accepted as a convenient tool for multimedia medical conferencing. With the PRT, they could observe complex patient data at a glance, and take advantage of the conferencing facilities without any problem. Previous works [5-7] have not been concerned with the view of data, but it is necessary to provide tools for catching patient data easily in perspective of medical experts. In order to customize the structure of PRT for a specific medical community, editing the PRT file through text editor is required. But such operation is open to an accidental mistake. Thus, there needs to be an optional interface to easily change the structure of PRT. As for view templates, their major advantage lies in the fact that we could reduce the size and complexity of system and consistently extend the view of data. But we are deliberately not concerned with some parts of view template. In case of grid table type, user still cannot catch meaningful and sufficient information involved with examination image data only with textual information in each row. Thus, the view template needs to have additional column with the thumbnail of image data. And in the view template for image data, there are also a few functions necessary to enhance the quality of collaborative conferencing; for example, enlargement of some parts from image, contrast and intensity adjustment, etc. A key aspect of our research is an integrated view of patient data in medical collaborative workspace, which is accomplished by the Patient Record Tree. The tree menu helps medical users to easily navigate the patient data and to neatly conference with their partners. The view templates contribute to achieve both simplification and extendibility in implementing the system. And we tried to tackle the problem of limited display space by providing pop-up viewer for image or video data. The proposed approach for collaborative workspace management was initiated to build the prototype system for mul-
timedia medical conferencing. To make it a successful system, more efforts must be made into collaborative workspace management, i.e. in terms of the view of patient data. Our future works cover other issues of the collaborative workspace, for example, ownership, confidentiality, and security.
References [1] Rosen E. Personal Videoconferencing. Manning Publication, 1996. [2] Kleinholz L, and Ohly M. Multimedia Medical Conferencing: Design and Experience in the BERMED project. IEEE conferencing, 1994; pp. 255-264. [3] Medical Records Institute. What Is An Electronic Patient Record ?. http://www.medrecinst.com/levels.html, 1995. [4] Chakravathy S, Krishnaprasad V, Tamizuddin Z, and Lambay F. A Federated Multi-media DBMS for Medical Research: Architecture and Functionality. Dept. of Computer and Information Sciences, Univ. of Florida, Gainesville. Tech. Report UF-CIS-TR-93-006, 1993. [5] Shortliffe EH, Barnett GO, Cimino JJ, Greenes RA, Huff SM, and Patel VL. Collaborative Medical Informatics Research Using the Internet and the World Wide Web. Proceedings of 1996 AMIA Annual Fall Symposium, 1996; pp. 125-129. [6] Los Alamos National Laboratory. TeleMed Virtual Patient Record System. http://www.acl.lanl.gov/TeleMed/, 1996. [7] Seo YG, Choi SH, Oh HS, Jung MR. Medinet: Remote Medical Diagnostic Support System. Proceedings of the 23rd KISS Spring Conference, 1996; pp. 743-746. Address for correspondence Department of Computer Science, Seoul National University, San 56-1 Shillim-dong Kwanak-ku, Seoul, Korea
[email protected]
