develop and evaluate a system of computer-based guidance to support decision making in UK General. Practice. The guidance presents clinical advice and.
The PRODIGY Knowledge Architecture for Chronic Disease Management in Primary Care Bob Sugden1 MBCS, Ian N. Purves1 MB BS, MRCGP, MD, Nick Booth1 MA MB BS MRCGP DCH, Peter Johnson1 MB BS, Samson Tu2 MS 1
Sowerby Centre for Health Informatics at Newcastle, Newcastle University, UK 2 Stanford Medical Informatics, Stanford University, California, USA
This paper describes the requirements derived from evaluation of the first two phases of the PRODIGY project, and how this has driven development of the PRODIGY architecture and guidance model in Phase III of the project. PRODIGY provides decision support to a General Practitioner in the UK by presenting clinical advice and therapeutic recommendations. Use of PRODIGY to date suggests that at present the most useful guidance is that for acute prescribing. The project has now developed a novel architecture, guidance model, and clinical knowledge base that will shortly be evaluated to test its ability to provide improved support for chronic disease management.
INTRODUCTION In 1995 the National Health Service Executive in the United Kingdom commissioned the Sowerby Centre for Health Informatics at Newcastle University to develop and evaluate a system of computer-based guidance to support decision making in UK General Practice. The guidance presents clinical advice and therapeutic recommendations, which may be drug prescriptions, non-drug treatments, or patient information leaflets. The PRODIGY∗ project was conceived of as a three phase iterative study, with the evaluation findings of each phase informing development in the next phase. The research team developed the system specifications and knowledge base. The five largest UK General Practice system suppliers developed software to these specifications and integrated it into their clinical systems. This paper discusses the requirements derived from evaluation of the first two phases of the PRODIGY trials and the implications for the development in Phase III of a decision support architecture and guidance model for chronic disease management in Primary Care. Work on Phase III of the project is ongoing, in collaboration with Stanford Medical Informatics, and a third prototype system has now been developed.
∗
PRODIGY stands for (Prescribing RatiOnally with Decision Support In General Practice StudY
EVALUATION OF PHASES I AND II In the first two phases of the project, trials to evaluate the system were conducted using 137 practices and 183 practices respectively. Each practice had approximately four General Practitioners. Descriptions of the evaluation methods can be found in the project reports on the PRODIGY web site1. In summary, the methods used were: • National Questionnaire (to a 10% sample of all GPs in England) • Questionnaires (sent to all study practices) • Patient questionnaires (to a selection of PRODIGY practices) • Video observation of actual doctor-patient consultations • Computerised log file of guidance usage • Computerised log file of guidance editing • Prescribing Analysis (using PACT system2) • Comment Forms from participating GPs • Monthly technical meetings with Suppliers • Evaluation of Patient Information leaflets • Monthly facilitated groups and workshops for GPs • Laboratory evaluation of supplier systems The Phase II results confirmed the findings of the Phase I study that suggested that PRODIGY is a concept worth developing3. Although 65% of GP users believed some further work needed to be done on the system, 95% indicated a desire to continue to use PRODIGY. Requirements Derived from Evaluation Despite the success indicated by these results, more detailed analysis of the response data indicated that a number of problems remain to be solved before the project can claim to be fully successful in achieving its objective of providing effective decision support for the majority of GP consultations. Monitoring usage of PRODIGY to date suggests that at present the most useful guidance is that for acute prescribing, and user comments confirm that the guidance for chronic disease management is difficult
to use in the current form. Constraints of the current software model result in the advice offered being too generalised to be of use to clinicians during consultations. These problems were abstracted into a set of requirements to be met by the next phase of the project: 1
2
3
4
5
Chronic disease management guidance must take account of successive consultations, which has implications for both the way knowledge is represented and for the way the Electronic Patient Record (EPR) represents temporal issues The guidance model should not be totally dependent on data in the EPR since this is often missing or incorrect It must always be possible to revise the guidance offered in response to changed circumstances, as not all patient events can be anticipated, nor when they occur are they necessarily recorded on the EPR Standardised coding of drugs and clinical concepts is needed, to facilitate authoring and distribution of a consistent set of guidance It must be possible to rapidly develop and modify KA tools to support the authoring process in line with the developing guidance model, as knowledge authoring is a difficult problem, and the difficulty is compounded when the guidance model is unstable
In addition, evaluation had confirmed a set of requirements that had guided development of PRODIGY Phase II: 6
The basis of the user interaction model should be the successful cognitive model identified in PRODIGY Phase I 7 The system must minimise user interaction, as consultations are of short duration 8 Guidance should be structured in such a way that although the system can be used with minimal user interaction when required, greater depth of information can be obtained when the user requires it 9 The system should make suggestions, not demands, and it should be possible to override those suggestions if individual clinical judgement indicates that this was appropriate in specific cases 10 The guidance offered should as far as possible be tailored to individual patient circumstances 11 The number of options presented to the user should be five or fewer 12 It should not be necessary to enter patient data into the system in order to access the guidance provided by PRODIGY
DESIGN FOR PHASE III Initially, in order to understand the problems involved, clinical domain experts began modelling a small number of chronic diseases. The initial objective was to explore current clinical practice with regard to chronic disease management. Eventually, through modelling disease states and treatment options as state transition diagrams, these were mapped to an adaptation of the current cognitive model. A detailed description of the guidance model developed to support this is also available4. Knowledge Representation Two of our requirements - regarding the unreliability of the EPR and the potential for unanticipated change in the actual events relating to the patient - make previous knowledge representation approaches (procedural or algorithmic approaches, and a declarative approach such as PRESTIGE5) unacceptable as solutions. The procedural approach is dependent on defining a predetermined sequence of possible events, with alternative paths determined according to branching algorithms. These may or may not take account of individual patient factors. The level of user input required to fuel such algorithms has proved a barrier to user adoption. Additionally, if unanticipated events occur, it is difficult to reconcile this with the expected progression through the protocol. The declarative approach seeks to overcome this latter problem by creating a set of independent actions or sub-protocols, which may be declared relevant to the circumstances of a particular patient by evaluation of pre-conditions (sometimes called eligibility criteria). Once again, the level of user input required to fuel the evaluation algorithms may prove a barrier to user adoption. When scaling the approach to complex disease management problems the inability to relate these components to an overall procedural structure makes knowledge authoring exceedingly difficult. Finally, evaluation of many criteria imposes a significant performance overhead at execution time. The approach taken in PRODIGY is to augment the declarative approach with a user/machine interaction architecture that specifies a procedural structure to assist in determining a precedence order for interpretation of the declarative components of the model. The precedence order should not be confused with a control flow structure. It is simply an efficient means of representing the probability of one event following another, whilst allowing the declarative rules to override this flow if the actual order of events shifts into a different flow pattern. This augmented
declarative approach gains the advantages of both previous approaches - a comprehensible model based on precedence rules, together with the flexibility of the declarative approach in handling unanticipated events. This approach, together with other features described below, also avoids penalties of execution performance and excessive demands for user input. Knowledge Architecture The user/machine interaction architecture has two components: (1) a longitudinal view that links successive consultations (Figure 1), and (2) a snapshot view that uses a cognitive model to structure the interaction between PRODIGY and its user.
2XWFRPH
&RQVXOWDWLRQ $FWLRQV 0JW $FWLRQ
/DVW 7LPH
2XWFRPH 1H[W 7LPH
6FHQDULR 2XWFRPH
0JW $FWLRQ
/RJLFDO(35
Figure 1: PRODIGY model of clinician actions in the management of patients with chronic disease In chronic-disease management, at each consultation patients present GPs with clinical scenarios. These are easily recognisable patient states, e.g. ‘hypertensive on non-pharmacological treatment’. These scenarios are usually, but not always, a combination of diagnosis and current therapy. After reviewing all of the information available at a consultation, clinicians make decisions to carry out management actions. The outcomes following the actions become the basis for defining scenarios that may be presented at future consultations. In any consultation, the guidance available is structured according to a three-layer model. This satisfies the requirement that ‘Guidance should be structured in such a way that although the system can be used with minimal user interaction when required, greater depth of information can be obtained when the user requires it’. The top layer contains the snapshot cognitive model that is automatically presented to the user when PRODIGY is entered. This is dynamically tailored to a specific condition/patient combination. The guidance offered at this level is intended to be sufficient to support an experienced clinician in decisions on
managing a patient readily falling within typical disease and treatment scenarios. The middle layer of guidance is designed to offer assistance where there is a lack of familiarity with the treatment choices offered, or in resolving uncertainty as to the precise category into which this patient may be placed. The knowledge at this level is presented as text screens navigated via hyperlinks, and is written in a brief, bullet-point form in order to be easy to absorb when used during a consultation. The bottom layer is intended for user information and education outside the consultation, presented through text screens and hyperlinks in a more in-depth manner, and containing citations for further reading. This layer has the potential to contain graphics, moving images, etc. as appropriate. Cognitive Model The overriding requirement in structuring user/machine interaction is that ‘The basis of the user interaction model should be the successful cognitive model identified in PRODIGY Phase I’. On first entering the consultation mode in the system, the user is presented with information designed to answer the question ‘What am I dealing with?’ Current conditions known for this patient are displayed, together with reference to any previous use of PRODIGY guidance for a condition. Alternatively, a new condition can be entered, using a standard classification system that is linked to the PRODIGY knowledge base using declarative rules to ensure presentation of relevant guidance. If the consultation covers a condition for which the patient has been previously treated using PRODIGY guidance, then stored guidance-positioning information is used to answer the question ‘Where am I in the guidance?’ The knowledge in this layer is structured into distinct pattern-recognisable clinical scenarios, for example ‘asthma on a short-acting beta2 agonist and low-dose steroids’. This is the most visible fulfilment of the requirement that ‘Chronic disease management guidance must take account of successive consultations, which has implications for both the way knowledge is represented and for the way the EPR represents temporal issues’. In most consultations, the clinician would accept the scenario indicated by the stored guidance-positioning information, proceeding to assess the current state of the patient and decide on appropriate management actions. There is a requirement that ‘It must always be possible to revise the guidance offered in response to changed circumstances, as not all patient events can be anticipated, nor when they occur are they necessarily recorded on the EPR’. It is therefore
possible for the clinician to reject the proposed scenario and re-position the patient in terms of other possible scenarios. Declarative rules are invoked to propose likely alternative scenario(s). In order to satisfy the requirement that ‘The guidance model should not be totally dependent on data in the EPR since this is often missing or incorrect’, it is possible for the clinician to select any of the scenarios available within the guidance for this condition. The final layer of the cognitive model addresses the question ‘What can I do?’ This layer of guidance is divided into two parallel sections: consultation guidance and management guidance. Consultation guidance recommends actions that can be completed during the consultation, and is concerned with patient history and clinical findings. Using this, the clinician may be required to input data to the system, and if so this is recorded on the EPR and may be used by the system to provide guidance on therapeutic options. In order to satisfy the requirement that ‘It should not be necessary to enter patient data into the system in order to access the guidance provided by PRODIGY’ there is no mandate to enter or complete any consultation actions. The second section of guidance at this third layer is concerned with actions which may be initiated within the consultation, but which can only be completed at a later time. These actions are presented in terms of therapeutic options, such as prescribe, refer, advise and investigate. They are offered subject to individual patient characteristics and the current trend in the disease within this patient , meeting the requirement that ‘The guidance offered should as far as possible be tailored to individual patient circumstances’. Based on declarative rules, options are presented in ‘preferred’, ‘neutral’ or ‘ruled out’ categories. To satisfy the requirement that ‘The number of options presented to the user should be five or fewer’, guidance is authored such that the ‘preferred’ category would not normally contain more than five options. To assist in the decision-making process, specially written information can be shared with the patient in reaching a joint decision on treatment. Finally, the clinician will select appropriate actions, either from amongst those offered or not, in accordance with the requirement that ‘The system should make suggestions, not demands, and it should be possible to override those suggestions if individual clinical judgement indicates that this is appropriate in specific cases’. System Architecture and Interface Specification The PRODIGY decision support system is embedded into existing clinical systems, and in previous phases
this has been achieved by developing a Software Requirement Specification, which was then independently implemented by each software supplier. In Phase III of the project, it has been possible to offer a common ‘execution module’. This software interprets the guidance, evaluates criteria via calls to the host system EPR, and passes appropriate information to the host system user interface. It also requires some information to be stored on the host system EPR (Figure 2). The execution module does not interact directly with the user, which is entirely under the control of the user interface provided by the system supplier. Neither does it directly interact with the host system EPR, but makes request for data from the EPR, and passes data to the host system for storage in the EPR. All data passed in this way is defined in a PRODIGY standard format. The execution module is available as an ActiveX control, and the C++ source code is also made available to authorised suppliers.
Guideline
KB
Execution Module
Recommends, Requests
UI
Choices, Data Queries, Updates
Drug Ontology
Host System
Logical EPR Actions
Host System Physical EPR Prodigy EPR Extension
Figure 2: PRODIGY System Architecture Guidance Modelling and Authoring Environment Experience in previous phases of the project produced a requirement stated thus: ‘It must be possible to rapidly develop and modify KA tools to support the authoring process in line with the developing guidance model, as knowledge authoring is a difficult problem, and the difficulty is compounded when the guidance model is unstable’. When reviewing the work of other project teams working with clinical decision support protocols, the work of the Knowledge Modelling Group at Stanford Medical Informatics was of especial interest. The Protégé knowledge acquisition tool6 offers the ability to support the definition of a domain-specific ontology, from which an interactive knowledge
acquisition tool could be automatically generated. Moreover, it is possible to rapidly redefine the underlying ontology and regenerate another version of the knowledge acquisition tool. This approach offers considerable advantages in the prototyping approach adopted in PRODIGY. An additional advantage of Protégé is that the design of a specific protocol is primarily via a graphical representation. This readily supports a procedural view of the knowledge via the state-transition diagrams which had already been used in PRODIGY to model protocols for chronic disease management. Declarative knowledge is then added as preconditions attached to the primary objects. A problem that has added to the difficulty of delivering PRODIGY guidance nationally is the number of different clinical term and code sets in use at the present time. Hence the requirement that ‘Standardised coding of drugs and clinical concepts is needed, to facilitate authoring and distribution of a consistent set of guidance’. Diversity is accommodated using translation tables and links. When supported by a detailed knowledge and understanding of the coding systems, this suffices to deliver a working system. It is to be hoped that increasing standardisation of coding systems, via development of a PRODIGY-sponsored ontology and knowledge base using GALEN7 technology, will eventually eliminate the need for such efforts.
IMPLEMENTATION STATUS AND EVALUATION PLAN Work on defining a suitable knowledge architecture and guidance model for Phase III began in spring 1998, and two prototype systems have been demonstrated at the time of writing (February 1999). The project methodology allows a flexible, iterative development process, facilitating a process of continuous evaluation and improvement based on feedback from early adopters. After two or three cycles of improvement, enough clinicians should be using the software to commence a full intervention study of the type carried out in previous project phases. By their nature, the study of chronic conditions requires a time period long enough to observe a number of consultations with a patient for the same condition. The initial intervention period will be six months, and it is hoped that it will be possible to follow this with a longer term study at some stage in the future.
CONCLUSIONS Deploying guidance-based decision-support systems and gaining user acceptance is a difficult task. The
PRODIGY project has taken an iterative development and evaluation approach to ensure that the system meets the needs of primary-care clinicians. After evaluating the first two phases of the system, we have derived a set of requirements for Phase III of the project. Using the knowledge architecture and cognitive model described above, we developed a guidance model and user/machine interaction style for managing patients with chronic diseases. It is the hypothesis of the authors that all the requirements are thus more than adequately met, including the key user requirement for acceptance that: ‘The system must minimise user interaction, as consultations are of short duration’. This hypothesis will be tested in future evaluation cycles of the project.
Acknowledgements Funding provided by NHS Executive Primary Care and project management by Mike Sowerby. Thanks to SMI for enabling Samson Tu’s sabbatical at Newcastle, the five software suppliers and about 900 GP’s who evaluated the system. We thank them all.
References 1
Available from URL: http://www.schin.ncl.ac.uk/prodigy/ 2
Information on the PACT system is available from URL: http://www.ppa.org.uk/
3
Purves, I.N., Sugden, R.C., Booth, N.S., Sowerby, M., The PRODIGY Project - the Iterative Development of the Release One Model. Submitted to: Proceedings of the 1999 AMIA Fall Symposium. Washington, DC, 1999 4
Johnson, P, Tu, S.W., Booth, N., Sugden, R.C., Purves, I.N., A Guideline Model for Chronic Disease Management in Primary Care. Submitted to: Proceedings of the 1999 AMIA Fall Symposium. Washington, DC, 1999
5
C Gordon and M Veloso, The PRESTIGE Project: Implementing Guidelines in Healthcare. Medical Informatics Europe '96, IOS Press 1996, 887-891
6
Musen, M.A., Gennari, J.H., Eriksson, H., Tu, S.W., Puerta, A. R., PROTÉGÉ-II: Computer Support For Development Of Intelligent Systems From Libraries of Components. Proceedings of MEDINFO '95, Eighth World Congress on Medical Informatics, Vancouver BC. 1995; pp. 766-770
7
Solomon, W.D., Wroe, C.J., Rector, A.L. et al., A Reference Terminology for Drugs. Submitted to: Journal of the American Medical Informatics Association 1999; Fall Symposium Special Issue
