CE Update Submitted 1.15.10 | Revisions Received 2.23.10 and 3.20.10 | Accepted 3.29.10
Cost of Quality at a Clinical Laboratory in a Resource-Limited Country
Ali Elbireer, MBA, MT(ASCP), CPHQ,1,2 Alicia R. Gable, MPH,2 J. Brooks Jackson, MD, MBA1,2 (1Makerere University-Johns Hopkins University Core Laboratory, Kampala, Uganda, 2Department of Pathology, School of Medicine, Johns Hopkins University, Baltimore, MD) DOI: 10.1309/LMCZ0ZFR80QWIBEM
Abstract
Health care decision makers often have a limited understanding of the costs associated with quality improvement efforts. However, such information is vital for managers to make appropriate operational and financial decisions to ensure high quality health care services. We used the Cost of Quality (COQ) model to estimate the cost of quality-related activities at the Makerere University-Johns Hopkins University (MU-JHU) Laboratory in Uganda. Using data collected from January 1, 2007–December 31, 2007, we allocated the direct costs associated
with quality-related activities in the laboratory across COQ’s cost categories: prevention, appraisal, internal failures, and external failures. We found that approximately 94% of total COQ was spent on costs of “good quality” (prevention and appraisal), while 6% was spent on costs of “poor quality” (internal and external failures). The high percentage of COQ spent on prevention and appraisal activities is consistent with efforts to ensure high quality laboratory results. Future studies should try to link COQ to health outcomes and quantify “hidden” failure costs.
After reading this article, readers should be able to describe the COQ concept and its 4 major cost categories, understand how it can be applied in a clinical laboratory setting, identify specific clinical laboratory costs that fall under each cost category, and discuss some of the advantages and limitations of this model.
In recent years, health care providers, patients, and payers have placed a growing emphasis on improving the quality of health care services. The Institute of Medicine (IOM) defines quality in health care as “the degree to which health care services for individuals and populations increase the likelihood of Corresponding Author J. Brooks Jackson, MD, MBA
[email protected]
Abbreviations COQ, Cost of Quality; MU-JHU, Makerere University-Johns Hopkins University; IOM, Institute of Medicine; P-A-F, prevention-appraisalfailure; CAP, College of American Pathology; PCR, polymerase chain reaction; QA, quality assurance; LIS, laboratory Information System; EQA, External Quality Assurance; CSC, Customer Service Concern; PM, preventive maintenance; QC, Quality Control; PIR, Process Improvements Reports; GLP, good laboratory practices; GAAP, generally accepted accounting principles
labmedicine.com
This CE update introduces the Cost of Quality (COQ) model and illustrates how it can be applied to a clinical laboratory setting to help managers and executives make more informed decisions about spending on quality-related activities. We begin with an overview of the COQ concept and its 4 main cost categories, followed by an illustration of how the model can be applied in a medical laboratory setting. We then discuss some of the advantages and limitations of this methodology and provide suggestions for future research.
Management exam 91001 questions and the corresponding answer form are located after this CE Update on page 434 Keywords: cost of quality, calculation of quality costs, clinical laboratory, resource-limited setting, Uganda
desired health outcomes and are consistent with current professional knowledge.”1 Health care decision makers often have a limited understanding of the costs associated with such quality improvement efforts. Such information, however, is vital for managers to make appropriate operational and financial decisions to ensure high quality health care services. This paper introduces the Cost of Quality (COQ) model as a tool for organizations to use in assessing the resources spent on qualityrelated activities and to improve decisions about investment in quality improvement activities.
The COQ Model The COQ concept was first developed in the 1950s2 as a tool for managers to use in assessing quality improvement and profit contributions. Although there are several different variations of the COQ model, most published COQ analyses are based on the classical prevention-appraisal-failure (P-A-F) model, which classifies quality-related costs into 4 primary categories: prevention, appraisal, internal failure, and external failure costs (Figure 1).3 July 2010 ■ Volume 41 Number 7 ■ LABMEDICINE
429
CE Update Prevention and appraisal costs represent the costs of “good quality.” Prevention costs are the costs of activities designed to prevent poor quality, and maintain good quality in products and services. Appraisal costs are the costs associated with measuring, evaluating, or auditing products or services to assure conformance to quality standards and performance requirements. Failure costs (both internal and external) comprise the costs of “poor quality.” Failure costs result from products or services not conforming to established standards or customer needs; these can be detected either prior to (internal failures) or after (external failures) delivery of a service or product to the customer. Figure 1_Cost of quality overview diagram. The underlying premise of the COQ model is that investing in prevention and appraisal activities leads to a reduction of failures (both internal and external), and that inPrevention costs included the sum of the following costs: vestment in prevention leads to reductions in appraisal costs • Personnel time: This includes a laboratory quality and improvements in overall quality.4 assurance (QA) team of 3 employees at 100% effort; Although the theory and methods of COQ have been writlaboratory administration (including the administraten about extensively in the literature, there are few published tive director, manager, and human resource coordinapractical applications of the methodology outside of the high-tech tor) at an estimated 10% effort; laboratory technical industrial and engineering fields. Although there are a handful supervision (5 technical team leaders) at 40% effort; of applications of COQ in health care settings, we found only 1 and all other laboratory staff at an estimated 5% effort. published paper applying COQ to a clinical laboratory setting.5 Personnel costs include only direct costs of salary and The author used COQ analysis to calculate the cost of non-conbenefits. To estimate personnel costs, we measured the formance (ie, cost of poor quality) at a laboratory in the United time of each quality-related activity performed during States and to identify opportunities for improvement by reducing the study period and multiplied it by the average perthe number of cancelled tobramycin tests due to inadequate samsonnel cost per hour for the relevant staff. For example-collection information. This analysis led to quality improveple, the cost of daily QA final review of all lab results ment programs that reduced the costs of non-conformance by for any given work day was calculated by multiplying 83% in a period of 9 months. Other industries also cited examples the cost per hour of a QA employee by the average of success with COQ systems, with significant reductions in total number of hours spent on the review of test results on quality costs and improvements in quality of products over time.4 an average day; Despite these successes, COQ remains an underappreciated con• Preventive maintenance (PM) contracts for laboracept by many organizations.4 tory instruments (eg, chemistry, hematology, flow, and In this paper, we apply the COQ model to estimate COQ PCR testing) and equipment (eg, centrifuges, freezers, at the Makerere University-Johns Hopkins University (MUrefrigerators, and incubators); JHU) Laboratory, a not-for-profit, College of American Pathol• Office supplies/copying costs associated with prevention ogy (CAP) accredited clinical laboratory located in Kampala, quality activities; Uganda. The example provided in this paper will illustrate how • Annual competency assessment, continuing education, managers can apply the COQ model in their own laboratories. and other training for staff; and • Laboratory information system (LIS) based on an estimate that 15% of the total annual LIS usage costs are QA related. Application of the COQ Model in a Laboratory Setting Appraisal costs included the sum of the following costs: The MU-JHU Laboratory supports several clinical trial • Quality control (QC) and calibration reagents and networks sponsored by the United States’ National Institutes analysis, based on usage frequency and price per test; of Health and offers laboratory testing services to more than • Annual CAP accreditation, including inspection costs 80 research studies, clinics, and programs. For the calendar year and annual accreditation fees; 2007, the laboratory billed 141,359 tests primarily in the areas • External QA (EQA) for proficiency testing from CAP of routine hematology, chemistry, immunology, flow cytometry, and other EQA-certified agencies; and HIV polymerase chain reaction (PCR) testing. • Running all EQA surveys, based on usage frequency This study analyzed cost data from January 1, 2007– and actual price per test; December 31, 2007 to estimate the COQ at the MU-JHU labora• Process improvements and quality activities, including tory. Costs were allocated across COQ’s 4 major cost categories: secondary results review, daily temperature monitoring, and CAP self inspections, based on time spent by prevention, appraisal, internal failures, and external failures. Most personnel on these activities; and costs are based on actual costs of the items purchased and services • Biannual internal and external method comparison procured. However, some costs, such as personnel costs, had to be testing, based on actual price per test. estimated due to limitations in the cost accounting system. 430
LABMEDICINE ■ Volume 41 Number 7 ■ July 2010
labmedicine.com
CE Update Internal failure costs (detected prior to delivery of the service) were based on the sum of the following costs: • Expired reagents, due to poor inventory management; • Repeated QC and calibration tests; • Correction of data entry errors, specimen processing, and accessioning errors, and • Standard operation procedures violations and cost of reeducating staff. External failure costs (detected after the delivery of the service) were based on the sum of the following costs: • Correction of pre-analytical errors including laboratory errors committed before analyzing the samples, such as requisition data entry errors and processing errors; • Correction of analytical errors occurring at the testing stage, such as performing the wrong test or doing the test incorrectly; and • Correction of post-analytical errors occurring after testing, such as transcribing the wrong results, sending patient results to the wrong client, or a delay in reporting test results to clients. For the purpose of this analysis, we limited external failure costs to the cost of identifying and correcting errors based on Customers’ Services Concern (CSC) forms received from clients (Table 1). To be conservative, we estimated that only onethird of all actual customer problems were reported via CSC forms. We therefore multiplied the number of reported CSCs during the study period by 3. We estimated it took an average of 3 hours to investigate, rectify, and prepare a response for a CSC. The cost per error was calculated by multiplying 3 hours by the average hourly rate of a laboratory employee.
Figure 2_MU-JHU core lab quality vs non-quality related expenses, Jan-Dec, 2007.
COQ Analysis Results The total COQ expenses for the MU-JHU laboratory during the 12-month study period were $545,428, approximately 32% of the total direct laboratory expenses ($1,722,618) (Figure 2). Approximately 94% of total COQ was spent on costs of good quality (24% prevention costs and 70% appraisal costs), and 6% of COQ was spent on costs of poor quality (4% internal failure costs and 2% external failure costs) (Figure 3). Total costs in the prevention category were estimated to be $132,280, approximately 8% of the total direct laboratory
Table 1_Customer Services Concerns (CSC) Reports by Category (January 1, 2007–December 31, 2007) Category Item
Jan 07
Feb 07
Mar 07
Apr 07
May Jun 07 07
Jul 07
Aug 07
Sep 07
Oct 07
Nov 07
Dec 07
Total % of 2007 Total Total 80
53%
Pre-analytical Delay in specimen 1 1 1 errors pick-up/processing Data entry error 3 3 4 7 16 10 2 9 6 2 2 Specimen processing error 2 2 1 Unacceptable/rejected specimen 1 1 not called to clinic Pre-other 1 1 3 1
3
Analytical errors
2 25 17 2 4
16%
47
31%
Wrong tests performed 1 1 Results error suspected 3 2 2 1 3 1 2 1 2 Tests not performed 2 Analytical-other 1 2 1
64 5 2 6
Post-analytical Critical results not called 1 errors Results not delivered within 1 2 2 2 2 1 expected TAT Results transcription errors 2 1 1 1 Data download error Lab requisitions—yellow 1 1 1 form missing Storage status error 1 3 6 7 2 1 2 Post-other 1 1 2 1 1
5 0 3
Total per month
152
labmedicine.com
6
16
18
21
28
18
10
11
11
6
5
2
1 10
22 6 152
100%
July 2010 ■ Volume 41 Number 7 ■ LABMEDICINE
431
CE Update expenses in 2007. Prevention costs were comprised mostly of personnel salary costs ($71,796), which are relatively high due to the lack of an automated data interface LIS. As a result, many of the critical end points are entered manually and require multiple verifications before sending results to the client. The second highest prevention costs were the expenses associated with the maintenance of the laboratory instruments and equipment ($42,391). Preventive maintenance contract costs are relatively high in Uganda and other resource-limited settings, and turnaround time on repairs can take up to 30 days or more due to the lack of local trained repair engineers. To ensure uninterrupted laboratory services, the laboratory purchased 2 of each instrument, which in turn increased the overall cost of PM contracts. Appraisal costs of quality in the MU-JHU laboratory were estimated to be $380,472, or 22% of the total direct laboratory expenses in 2007. Almost 89% of the costs in the appraisal category are associated with QC analysis and calibrations, which are necessary to confirm the accuracy and reliability of test results. The second highest appraisal cost is for CAP accreditation and biannual inspections, including the inspectors’ travel and accommodation expenses. The third highest appraisal costs are the proficiency test surveys and the cost of performing the surveys. Costs of poor quality, including both internal and external failures, were $32,677, or 2% of total direct laboratory expenses in 2007. Internal failure costs were estimated to be $23,877, or 73% of the total costs of poor quality. We identified many of the internal failures using Process Improvements Reports (PIR), a tool used by the staff for internal reporting of errors and problems (Figure 4). In addition, $11,328 in reagents had expired or were wasted. However, procuring reagents in Uganda takes a relatively long time; thus, having reagents in stock at all times to ensure uninterrupted services could be considered a preventive quality measure and prevention cost. External failure costs were estimated to be $8800, or 0.5% of the total direct laboratory expenses in 2007. Customer service concern forms, which laboratory customers can complete to voluntarily report problems, were used to identify external failures. We classified these errors according to the following 3 stages of the testing process: pre-analytical, analytical, and post-analytical (Table 1). The costs of correcting errors in the pre-analytical phase (eg, data entry and processing errors) were estimated to be $4506, or 51% of the cost of all external failures. The costs of errors in the analytical phase (eg, failure to perform a requested test) were estimated to be $1495, or 19% of all external failure costs. Costs of post-analytical phase errors (eg, failure to deliver results or delayed turnaround time) were estimated to be $2600, or 30% of external failure costs (Table 2).
Implications of COQ for Clinical Laboratories This is the first study to quantify COQ in a clinical laboratory in a resource-limited setting. Our analysis indicates that 32% ($512,751) of total laboratory expenses and 94% of total COQ were spent on creating and following processes that ensure good laboratory practices (GLP) and contribute to high-quality outcomes. This finding was validated to some extent when the CAP inspected the laboratory in June 2007 for re-accreditation and had 0 citations and only 2 minor recommendations. The optimal level of COQ has been the source of much debate.4 Under the traditional P-A-F view, optimal COQ is reached at the point when the costs of improved quality exceed 432
LABMEDICINE ■ Volume 41 Number 7 ■ July 2010
Figure 3_MU-JHU core lab cost of quality, January 1, 2007-December 31, 2007.
Figure 4_MU-JHU core lab monthly process improvement reports (PIR), January 1, 2007-December 31, 2007.
the benefits. More recent quality economic models argue that spending on prevention can be justified until the point where there are 0 defects or deficiencies. Some economists have attempted to reconcile these 2 disparate views arguing both models are relevant, but they assume different time horizons for decision making.4 Despite disagreement on the optimal level of COQ, the high proportion of spending on costs of good quality relative to poor quality at the MU-JHU laboratory (94% costs of good quality versus 6% costs poor quality) is consistent with the goal of ensuring high quality outcomes. Although it would appear from this analysis that maintaining good quality is more expensive than the cost of failures, until one can truly assess the costs of hidden failures, it is difficult to know which is more expensive. Erring on the side of investing too much in prevention and appraisal activities is generally more acceptable than investing less if patient outcomes are poorer. On the other hand, it is possible that decreasing certain prevention and/or appraisal costs might not lead to an increase in failure costs so as to be more cost effective without sacrificing good patient care. In any case, we will examine ways to invest in prevention to hopefully decrease overall appraisal and failure costs. This model would allow one to measure the outcome of making such changes. In terms of external comparisons, the costs of good quality (poor quality) at the MU-JHU laboratory appear to be higher (lower) than other comparable studies. For example, the COQ study at a hospital clinical laboratory in the United States found that the overall COQ was about 35% of the operating expenses, the cost of good quality was 10%, and the cost of poor quality was 25% of the operating expenses.5 Similarly, Zimak labmedicine.com
CE Update reported that prevention costs genTable 2_Cost of Quality Expenses by Category (January 1, 2007– December 31, 2007) erally constitute about 5% of an organization’s total COQ, while Cost of Good Quality in MU-JHU Core Laboratory appraisal costs are approximately 20%–25%, and failure (poor qualPrevention Costs of Quality Total ity) costs are about 65%–70% of company’s quality costs.6 However, (a) Salaries of QA team and laboratory management for QA-related activities $83,785 $132,280 (b) Preventive maintenance of laboratory equipment and instruments $42,391 the value of external comparisons (c) Office supply costs for quality-related preventive activities $525.15 may be limited by variations in (d) Total QA and competence training, and continuing education for laboratory staff $2236 the costs included in the COQ (e) Laboratory information system $3342 calculations or the denominators Appraisal Costs of Quality Total used (eg, use of operating expenses versus expenditures as the denomi(a) Cost of quality control and calibration reagents $337,786 $380,472 nator). In addition, caution must (b) Annual CAP accreditation and inspection costs $15,000 be used when comparing COQ (c) External Quality Assurance (EQA) proficiency testing $9052 results from organizations in dif(d) Costs of running all EQA surveys $11,204 (e) Other process improvements and quality activities $6342 ferent settings. For example, the (f) Costs of internal and external method comparison testing $1018 magnitude of costs in each cate gory (and therefore the breakdown by cost category) may differ sigCost of Poor Quality in MU-JHU Core Laboratory nificantly depending on whether Internal Costs of Poor Quality Total an institution is located in a resource-limited versus a resource(a) Costs of poor inventory management (wasted reagents) $11,328 $23,877 advantaged setting. (b) Costs of QC/calibration failures/repeats for all analyses $9538 The COQ methodology has (c) Estimated data entry errors and rework costs $926 (d) Processing and accessioning errors and rework costs $116 a number of advantages. First, it (e) SOP violations/corrections/rework costs $1969 can be a useful management tool for identifying and focusing on External Costs of Poor Quality Total areas of poor performance and can (a) Estimated costs of pre-analytical errors $4506 $8780 provide a basis for benchmarking (b) Estimated costs of analytical errors $1695 progress and overall spending on (c) Estimated costs of post-analytical errors $2600 quality-related activities over time. Another key advantage of COQ is that it translates quality-related activities (eg, QA activities, process that can more accurately capture quality costs, conducting root improvements, corrective actions, etc) into monetary terms that cause analyses, attempting to quantify hidden failure costs, and can be easily understood by executives making overall financial relating COQ to patient outcomes. Ultimately, more widedecisions. COQ can be a powerful tool by illustrating the true spread use of COQ and other similar quality costing models can costs of quality-related activities to upper management, which help health care organizations identify quality deficiencies and can in turn help them in the budgetary process and making debenchmark their quality costs over time. LM cisions about implementation of cost-control measures. However, COQ also has some important limitations. First, traditional cost accounting methods, which are based on generally accepted accounting principles (GAAP) and driven by 1. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the external reporting requirements, do not always capture relevant, 21st Century. Washington, DC: National Academies Press; 2001. accurate, or complete quality cost data, such as personnel costs 2. Juran JM. Juran’s Quality Handbook. 1st ed. New York, NY: McGraw-Hill; associated with quality-related activities.7 Second, they cannot 1951. always capture the intangible benefits or opportunity costs of 3. Westgard JO. Six Sigma Quality Design and Control. Madison, WI: Westgard QC; 2001:225. quality improvement.4 In addition, it is difficult to identify and 4. Schiffauerova A, Thomson V. A review of research on cost of quality models and quantify “hidden” failure costs associated with poor quality,8 best practices. IJQRM. 2006;23:647–669. such as adverse events resulting from inaccurate laboratory re 5. Menichino T. A cost-of-quality model for hospital laboratory. MLO Med sults, loss in customer confidence, deterioration in reputation, Lab Obs. 1992;24:47–48,50. Available at http://findarticles.com/p/articles/ and poor employee or customer retention rates. Although the mi_m3230/is_n1_v24/ai_11815914/. Accessed March 10, 2010. cost of poor quality (internal and external failures) at MU-JHU 6. Zimak G. Cost of Quality (COQ): Which Collection System Should Be Used? laboratory is very low (6% of total COQ expenses and
