Were they same, combining the numerator to compute the index should have been simpler. â¢To add to the complexity, the rates for these PSIs differ greatly.
LOGO
Methodology for and Issues in Constructing a Composite Index of the Individual Patient Safety Indicators (PSIs) Gulzar H. Shah, MSat, MSS, PhD Director of Research, National Association of Health Data Organizations (NAHDO)
INTRODUCTION AND BACKGROUND What is an Index or a Composite Measure A composite index is based on a large number (typically 5 or more) of individual measures, indicators, or factors which are combined together to form a single representative measure. Background Standard measures of quality and safety of hospital care have been developed and disseminated by the Agency for Healthcare Research and Quality (AHRQ). A total of 23 Patient Safety Indicators (PSIs) have been developed by the AHRQ. These individual measures provide a detailed picture of the quality of care a facility provides, but consumers may find varied performance of a hospital across multiple indicators, a bit confusing when trying to make comparisons between providers, as they may not have the medical knowledge to prioritize between indicators, or analytical capacity to determine an overall performance. To address this issue, appropriate methodology for a summary Index is needed. Challenges •The denominators for individual AHRQ PSIs are not same. Were they same, combining the numerator to compute the index should have been simpler. •To add to the complexity, the rates for these PSIs differ greatly ranging from 0.005 per 1000 population at risk for “Transfusion Reaction (PSI 6) to 237.81 per 1000 population at risk for Obstetric Trauma with 3rd Degree Lacerations―Vaginal Delivery with Instrument (PSI 27). •Even when denominators are same, simply combining numerator events leads to domination of the Index by more prevalent PSIs
PURPOSE This poster proposes a methodology for combining AHRQ’s individual patient safety indicators (PSIs) into a single composite or index. The summary index methodology developed for PSIs can provide a unified summary indicator of provider or area level performance, enabling a report-card like ranking. When the measurement of a phenomenon is based on multiple items, creating a summary scale or index improves simplicity, understanding, and interpretability. Indexing multiple measures of performance may also help simplify data presentations mitigate small cell size issues, as small cell size is a potential deterrent to reliability in healthcare data
MATERIALS AND METHODS
Critical Decisions Combining Individual measures: Composites measures can be additive or multiplicative. Multiplicative composites are complex and present difficulty in interpretation Directionality of an indicator: What is the meaning of “high” and “ low”? The rank score should be computed with this in mind. Weighting: Should each indicator be given an equal weight, or a weighting scheme be developed to represent the inportance of individual PSIs? Population size: How should population size be incorporated? Index Construction Principles • Individual components of Index should be related to the same indicator • Some or all indicators can be used in an Index • It is recommended that at least 5 indicators should be used. • Create one or more summary measures, or justify why indicators will be used separately. E.g. AHRQ’s proposed methodology of Indexing PQIs involve creation of three separate Indexes (1) Overall Index (11 selected) PQIs; (2) Index for Chronic conditions; (2) Index for Acute conditions. • Translate the indicators into a resource allocation formula. • Discuss the results Proposed Methodology This methodology was proposed by the author for the “Quality Counts” report of the Wisconsin Employer Health Care Alliance Cooperative. The index created for this report was additive; that is, it consisted of average of Standardized Incidence Ratios (SIRs) for the individual PSIs. The SIRs were used instead of the rates because there was a great variation in rates of individual PSIs. SIR is a simple standardized measure computed as observed rate (or number) divided by the expected rate (or number), which takes care of the individual PSI variation. The following steps were followed: • For each AHRQ PSI, SIR was computed using the following formula: SIR = observed rate / expected rate; • The SIRs were added and were divided by the number of PSIs included in the Index. Index or Summary Score = ∑SIRi / K Index = (SIR1+ SIR2 + SIR3+ SIR4 + SIR5 … SIRk)/K Where K is number of PSIs included in the index. Statistical Method for the Confidence Interval: The lower limit of the confidence interval (LLCI), and the upper limit were computed by performing the following computation. LLCIi = SIRi -1.96 SQRT(SIRi / Ei) ULCIi = SIRi + 1.96 SQRT(SIRi / Ei) (continues)
LOGO
Where: ‘Ei’ represent the expected number of numerator events across AHRQ PSIs for a given hospital, and ‘i’ ranges from 1 to K (the number of PSIs included in the Index). To compute the confidence interval for the Summary Score (Index), the averages of lower limits for SIRs (LLCIi) and the upper limit of the SIRs (ULCIi) were taken.
DECISION RULE When the average ratio of observed to expected expressed by the Index is 1, this means observed numbers/rates are exactly at the level of expected numbers/rates. To examine this, confidence bounds for the Index were examined to determine if the confidence interval contained the ratio 1. When the confidence interval on the Index for a hospital J contained 1, the hospital J was marked to have the complication rates “As Expected”, i.e., not different than the expected. On the other hand, when the confidence bounds for a hospital did not contain 1, that hospital was marked to have the rate significantly different than the expected. In that case, if the upper bound of the confidence interval was smaller than 1, the performance of the hospital on this indicator was marked as “Better than Expected” or smaller number of complications than expected. Otherwise, if the lower bound of the confidence interval was greater than 1, the performance of the hospital on this indicator was marked as “Poorer than Expected” or higher number of complications than expected.
CONCLUSIONS Summary performance measures, when used to supplement individual measures of performance, can improve consumers’ understanding of public health reports. There are several methodological crossroads in construction of an Index measure. The proposed methodology prevents a typical problem in Index construction -- more prevalent individual measures tend dominating the Index.
BIBLIOGRAPHY Liddell FD. Simple exact analysis of the standardised mortality ratio. Journal of Epidemiology and Community Health 1984;38:85-88 [1] Rosenberg G. Methods for Summarizing Data, In: Handler A, Rosenberg D, Monahan C, Kenny J (eds). Analytical Methods in Maternal and Child Health. Maternal and Child Health Bureau, HRSA, DHHS, 1998.(Page 183).
