development and process for electronic documentation of health services received by a patient (e.g., HL7. CDA, Continuity of Care Document (CCD)). The NIST ...
Data Quality and Interoperability Challenges for eHealth Exchange Participants: Observations from the VA Virtual Lifetime Electronic Record Health Pilot Phase Nathan Botts, PhD 1, Omar Bouhaddou, PhD 2, Jamie Bennett2, Eric Pan, MD, MSc 1, Colene Byrne, PhD 1, Karl Banty, BS 1, Lauren Mercincavage, MHS 1, Lois Olinger, MA1, Tim Cromwell, RN, PhD, PMP2 1 Westat, Rockville, MD; 2US Department of Veterans Affairs, Washington, DC Abstract We examine the state of the Department of Veterans Affairs’ (VA) Virtual Lifetime Electronic Record (VLER) Health pilot phase relative to the adoption of eHealth Exchange data standards, health information exchange (HIE) data quality measurement, and clinical content exchanged. The VLER Health pilot phase was one of the first efforts to test the implementation of eHealth Exchange standards and technology. We evaluated exchange data from the VLER Health pilot sites partners - the VA, the Department of Defense and private sector health care organizations (exchange partners), and assessed 1) conformance with data standards related to the underlying structure of the C32 documents; and 2) the types of C32 clinical content being exchanged. Several standards non-conformance issues in sample C32 files were identified whose roots are likely due to the optionality in interpretation and implementation of the standards. We highlight the need to carefully monitor HIE exchange partner data content and standards conformance. Introduction Through the Virtual Lifetime Electronic Record (VLER) Health Exchange initiative, the United States (U.S.) Department of Veteran Affairs (VA) can electronically share parts of Veterans’ health records with providers at the U.S. Department of Defense (DoD), and participating private sector health care organizations (exchange partners). VLER Health Exchange leverages the policies and technical standards of the eHealth Exchange (formerly the Nationwide Health Information Network) (NwHIN)) supported by the non-profit, public-private collaborative called Healtheway (Healtheway, 2014) (Bouhaddou, et al., 2012). Goals include better informed care providers, improved continuity and timeliness of care, enhanced awareness among all parties, and elimination of gaps in a patient's health record. VLER Health serves as a national catalyst for interoperable health systems and a model for future health information exchange (HIE). VA engaged in a pilot of VLER Health Exchange to serve as a national catalyst for the development and adoption of interoperable health systems and as a model for future HIE, to identify scalable implementation strategies; and to assess any early impact of VLER Health Exchange. VA selected 12 sites with a strong business case for HIE and sought a diversity of characteristics, such as geographic factors (e.g., rural, urban), populations served, the maturity of the HIE organization, and their sustainability models. Four pilot sites participated in a three-way exchange between VA, DoD, and the private sector, and eight sites participated in two-way exchange between VA and the private sector. The pilot period concluded in October 2012. The VLER Health pilot was one of the first national efforts to comprehensively test and improve health data standards to support effective exchange of Veteran health information across the eHealth Exchange. (Saef, 2013) (Ross, et al., 2013)
As the policies and technical systems that serve as the foundation for nationwide HIE continue to mature, and the amount of health data and documents being shared across the eHealth Exchange increases, greater attention is now focused on the quality of data exchanged (Kuperman, et al., 2013). With increased interoperable exchange between health systems, challenges and data issues are also exposed. These data issues impact the way in which the data reach the intended recipients and are realized in the user interface and, for many user systems, incorporated by the Electronic Health Record (EHR). Health data standards are continually evolving to address these data quality issues. Better health information supports better health care. The essential value of HIE is the degree to which it can provide useful data to providers to support care coordination and transitions of care. HIE has the potential to improve patient quality of care including patient safety, increase efficiencies in care provision, and provide opportunities for cost savings (Ross, et al., 2013) (Saef, 2013). Exchanging health information between the numerous products and versions of electronic EHR systems requires a standardized method for communicating data that is agreed upon and adopted by entities seeking to share data. The eHealth Exchange, maintained by a public-private partnership called “Healtheway” (Healtheway, 2014), is composed of a set of standards, services and policies agreed upon by federal agencies and non -federal organizations with an interest in safely and securely exchanging health information. To date, there are 40 eHealth Exchange participants, and more than 85 currently engaged in the onboarding process towards participation in eHealth Exchange. These eHealth Exchange participants are national or regional organizations that represent more than 700 hospitals and 5000 physician practices, covering all 50 states, and includes over 100 million patients. During the VLER Health pilot, eHealth Exchange Partners shared data through the eHealth Exchange (referred to as the Nationwide Health Information Network at the time of the pilot) using a suite of data transport tools and services called CONNECT. The primary health data standards used to structure the data as it was exchanged included the HITSP C32 and C62 standards (Bouhaddou, et al., 2012). C32 summary documents use the HL7 Continuity of Care Document (CCD) component to describe the content of a medical document to be stored or exchanged, and to summarize a patient’s medical status. The content may include administrative (e.g., demographics, insurance) and clinical (e.g., problem l ist, medication list, allergies, test results) information (Healthcare Information Technology Standards Panel, 2009). C32 content standards are comprised of 18 content modules that represent the underlying clinical data in both narrative and structured forms. C62 documents incorporate unstructured clinical notes, and scanned documents (e.g., text file, PDFs, or images such as a scanned image of an ECG report) (Healthcare Information Technology Standards Panel, 2009). The technical hurdles of matching patient records, exchanging data from different EHR systems and geographies, and then properly rendering the data in a manner that can be effectively used by health care providers are significant. Within the data standards themselves, including the C32, issues of optionality and interpretation create differences in the way the standards are implemented and the way the data are mapped across systems. Consequently, even the most diligent HIE development and implementation can produce challenges that hinder effective health data exchange. The VLER Health pilots established an important baseline of understanding as it relates to the adoption and implementation of eHealth Exchange health data standards, the availability and quality of clinical content of shared Veterans within pilot regions.
Methods Effective sharing of Veteran health information across the eHealth Exchange requires export and import of a validated C32 document with health information. We established a baseline for: 1) compliance of exchange partner’s C32 to current data standards; and 2) 2) the clinical content being provided within VLER Health documents exchanged. Validation of C32 Data Standards Compliance Potential eHealth Exchange partners must comply with the current eHealth Exchange standards. The eHealth Exchange onboarding process required exchange partners to be able to produce a well-structured C32, but did not include formal compliance and content testing. VA uses the National Institutes of Standards and Technology (NIST) Clinical Document Architecture (CDA) validator to validate eHealth Exchange partner C32’s for conformance to the HITSP/C32 v2.5 standard. The NIST CDA validator tests the underlying XML found in the C32 to determine whether the schema and data provided conform to the requirements established by the HITSP/C32 v2.5 specification. Should a part of the C32 not conform, a report is provided that outlines where and the type of non-conformance that occurred. The NIST CDA validation application was downloaded from the NIST website, installed locally and configured with the libraries necessary to check the validity of C32 documents per the v2.5 specification. The HITSP C32 specification is described by a hierarchy of standards that inform its ever-evolving development and process for electronic documentation of health services received by a patient (e.g., HL7 CDA, Continuity of Care Document (CCD)). The NIST CDA validator reports the types of non conformances found in each related section and classifies non-conformance alerts with the current C32 standard into levels of severity that include errors (items of non-conformance), warnings (items that technically conform, but could be better constructed), and notes (general co mments and suggestions on implementation). A sample of fourteen populated C32s provided between October 2011 and July 2012, were tested for their conformance to the current C32 standard - one from each of the 12 pilot site private exchange partner, and one from the VA and DoD. Test results recorded when non-conformance to the specification was found and where (e.g., CDA, CCD), the types of non-conformance identified (e.g., missing data element or attribute), and the sections in which the non-conformances were found (e.g., header, problems/conditions, medications). This assessment only considered those classified by the NIST CDA validator as potential errors in the C32 structure. Evaluation of C32 Data Content Availability The clinical content of the C32 was also assessed to better understand the type and amount of clinical data available to and from VA providers during the VLER Health pilot. In July 2012, tests of VLER Health pilot clinical content retrieved were conducted by the VA as a part of an operations and quality assurance initiative. Partner C32s analyzed were based on data available between October 2011 and June 2012. Based on a random sample of 250 shared Veterans per VLER Health pilot site private partners, plus the VA, C32s were assessed as to whether any clinical content could be identified and if so, the types of content that were found (e.g., medications, laboratory results, procedures). Eight of the twelve pilot site exchange partners, plus the VA were included in the study, providing a total of 2,250 Veteran records for analyses. Partners that were not included in this sub-study were either not in production or lacked a sufficient number of shared patients at the time the study was conducted.
VA staff conducting the tests of C32 retrieval used VA’s VistAWeb EHR application to query for data from the eHealth Exchange Partner system, and recorded whether a C32 was returned for each Veteran, whether the Veteran had eHealth Exchange data available and if so, the types of clinical modules retrieved. Testers only examined whether each C32 clinical module (e.g., medications) contained data, but did not review the C32 for other attributes of the data content such as completeness, display issues or data quality. Results VLER Health Exchange Partner C32 Validation Validation analysis of eHealth Exchange partner C32s indicated that six of the VLER Health exchange partners produced conformant validation results. The other eight C32s tested, however, resulted in some level of error being reported when run through the NIST CDA validator. Two of the C32s tested produced over 10 unique errors of non-conformance to the standard. (Often the same error is repeated multiple times depending on the content present in the C32.) Issues encountered were primarily related to undefined attributes or XML pattern errors, problems found within the document header, and missing data elements or required values as defined by the C32 standard. The majority of issues identified would not necessarily impact the way in which clinical content was reported, many issues were due to administrative attributes that might not have been a factor prior to becoming an eHealth Exchange partner (e.g., proper inclusion of a country code). A person viewing this information in an EHR may not even perceive the impact of these types of errors, but as established by the HITSP C32, these specifications are deemed important and mandatory for proper inclusion, and consequently may result in incompatibilities and errors when shared among C32-compliant software. Outlined in Table 1 are the issue categories, definitions, percentage of total issues found and distribution of error types for 103 issues identified across eight of the fourteen C32s analyzed. As noted previously, depending on how many records are present in the C32 these issues can then be reproduced many times, but represent only one main issue in how the C32 is constructed. The frequency of how of issues is important in terms of the user experience when pulling up the document on their screen. Table 1. Validation Issues Identified in VLER Health Partner C32 Samples Issue Categories
Definitions
Non-conformant Attribute or Pattern
An XML attribute or pattern that is unrecognizable by the rules and requirements provided by the C32 schema
General Header Constraints
Important details regarding the origins and author of the record may be missing or improperly described
Missing Elements or Required Values
Required data elements and/or coding missing or improperly represented in the C32
Percent of total issues (n=103)
Prevalence
10%
Found at least once in 5 partner C32s, with upwards of 200 different instances found
58%
Found at least once in 8 partner C32s, with upwards of 300 different instances found
32%
Found at least once in 8 partner C32s, with upwards of 200 different instances found
Source: Based on 14 sample C32s, one from each of the VLER Health exchange partners provided between October 2011 and July 2012.
Non-conformance issues were most commonly found in the medication, insurance, laboratory result, allergy, and problem modules. This fact may only be because these were the modules reported most frequently for the C32s tested or because information for some clinical modules are not sent at all by a particular eHealth Exchange partner at the time of the study. Types of Clinical Content eHealth Exchange Partners are Capable of Sharing Of the nine eHealth Exchange partners whose C32s were analyzed for clinical content (8 private partners and VA), the majority (88%) of C32s included clinical content in addition to basic demographic data. Table 2 provides a breakdown of clinical content data availability among VLER Health exchange partners by C32 module, the average of C32s that were populated with data for that module and the range of results across exchange partners. Given the patient matching processes needed to exchange clinical data, basic demographics information was available in 100% of the C32s retrieved. Provider information was available in 86% of the records. Problem list, allergy list, encounters, medications, and vital signs are also available in more than half of the C32s tested. Even when analyzing the C32s by VLER Health Partners, problem list, allergy list, and medications remains the most commonly available data module; three VLER Health Partners send these data for more than 75% of their Veterans, and another four for at least half of their Veterans. Table 2. C32 Data Availability for VLER Health Partners by Module
C32 Module
Percent of C32s Populated
Range Populated by Partner
Demographics
100%
100%
Providers
86%
71% ~ 99%
Problems
84%
52% ~ 99%
Allergies
74%
13% ~ 99%
Encounters
74%
35% ~ 94%
Medications
63%
