quality assurance audits on physicians" ordering behav/or and to learn whether doctors ..... factors such as revenue from or costs of additional mammograms or ...
Should Physicians Perform Their Own Quality Assurance Audits? WILLIAM I. B R A D Y , MD, D E B O R A H C. HISSA, MSN, M A R K McCONNELL, MD, R O B E R T G. WONES, MD Ouality assurance is required of all hospital outpatient there is little ev/dence documenting its value. The purposes of thisstudy were to assess the/mpact of quality assurance audits on physicians" ordering behav/or a n d to learn whether doctors who actual/y performed audits b e h a v e d differently from physicians who p a s s i v e l y received audit results. Baseline influenza vacc/natJon a n d screening m o m , - o g r a p h y ordering rates were established for the authors" residents" clin/c in 1985. In 1986, residents were a s s / g n e d random/y to three groups. Res/dents in o n e group audited their own charts for 1985 inBuenza vacc/nat/on ordering; the second group audited its own charts for 1985 screening mo'mmograp~y ordering; and the third group performed no audit but received the other groups" results. Passive receipt of results improved ordering of vacc/nat/on from 40% to 59% a n d ordering of m a m m o g r a p h y from 8% to 16%. Actual performance of audits/reproved ordering of m a m m o g r a p b ¥ from 16% to 26% but did not improve vacc/nat/on ordering, l"bese qual/ty assurance audits were effective in/reproving the performance of sel e c t e d p r e v e n t i v e health m e a s u r e s / n a residents" c/fific. Key words: quality assurance, health care; ambalatory care; graduate medical education," inlluenza vaccine; mo'mmog. raphy; cosf-benefft analysis. ] GEN ~ MED 1988;
clinics although
3:560- 565.
THEPASTTWODECADESh a v e witnessed a n increasing interest in health care quality a s s u r a n c e (QA) on the part of government, accrediting bodies, a n d thirdparty payers. Indeed, the integration of QA into both hospital a n d ambulatory settings is m a n d a t e d by two federal laws L ~ a n d is required by a n y ambultory care clinic or HMO seeking accreditation or federal certification,s-5 During this t/me, m a n y authors h a v e attempted to define methods for quality assurance, including w h a t it should m e a s u r e a n d how it should be m e a sured, e-~°A few studies h a v e b e g u n to explore such methodologies in detail, n-14 but the educational a n d f e e d b a c k techniques u s e d in these models require further critical r e s e a r c h to document their efficacy. One a p p r o a c h that h a s received particularly little attention is the use of physicians themselves to audit charts for compliance with QA standards. It h a s b e e n a s s u m e d that asking physicians to review charts would be too costly7 or that physicians would not be objective w h e n auditing their own records. However, ff such a n a p p r o a c h were more effective in Received from the Department of Internal Medicine (W.J.B., M.McC., R.G.W.) arid the University of Cincinnati Hospital (D.C.H.), University of Cincinnati, Cincinnati, Ohio. Presented at the 1986 annual meeting of the Society for Research and Education in Primary Care Internal Medicine (SREPCIM). Address correspondenceand reprint requests to Dr. Wones: 231 Bethesda Avenue, ML 535, Cincinnati, OH 45267.
$60
motivating desired behavioral c h a n g e s t h a n other QA methods, objectivity would be irrelevant a n d the cost-effectiveness of the program might be superior to that of more traditional approaches. The goal of this study, then, w a s to e v a l u a t e the effectiveness of a model quality a s s u r a n c e program in a n ambulatory care clinic in which resident physicians audited their own charts. Specifically, the study protocol a d d r e s s e d three questions: 1) Do residents provided QA audit results g e n e r a t e d by others improve their compliance with QA standards; 2) Do residents who actually perform QA audits improve their compliance with s t a n d a r d s more t h a n residents who just receive results; and, 3) If residents who perform audits comply better than those who just receive results, at w h a t cost is this a d d e d improvement achieved? METHODS
The University of Cincinnati Hospital is a 680bed u r b a n teaching hospital. The G e n e r a l Medical Clinic of the hospital h a s a census of 5,000 patients with approximately 14,000 patient visits annually. The clinic is staffed by 46 internal medicine residents, e a c h of whom h a s a p a n e l of about 100 patients that he or she sees in the clinic one afternoon per week. Each clinic session is supervised by a general medicine faculty person, although residents h a v e a great d e a l of i n d e p e n d e n c e in m a n a g i n g patients. Figure 1 illustrates the study design. The protocol used a before - after design to learn whether passively receiving QA results w a s effective in c h a n g ing residents' behavior. While a simultaneous control group would h a v e b e e n helpful in this part of the study, creating such a group w a s impossible practically b e c a u s e residents talk with e a c h other a n d there would be no w a y to restrict the intervention to the intervention group. A randomized, controlled design w a s used to learn whether residents who actually audited charts performed better t h a n those who just received results. A controlled design w a s feasible for this part of the study since this intervention could be restricted to a specific group of residents. Ordering of intluerma vaccination a n d ordering of screening m a m m o g r a p h y were chosen as the quality a s s u r a n c e problems for study, for several reasons. First, s t a n d a r d s a n d criteria for these procedures h a v e b e e n well defined a n d a r e well ac-
561
JOURNALOFGENERALINTERNALMEDICINE, Volume 3 (Nov/Dec), 1988
cepted.lS, ]e Second, determining c o m p l i a n c e with s t a n d a r d s is unambiguous, since audits n e e d not rely on the residents' r e c o r d - k e e p i n g habits but could utilize x - r a y reports a n d nurses' records of injections. Third, these p r o c e d u r e s h a v e b e e n utilized in studies in other clinics, so that c o m p a r i s o n of this intervention with others would b e possible. A didactic e d u c a t i o n a l program, b e g u n in 1985, w a s active in the clinic throughout the t w o - y e a r study.17 E a c h week, o n e topic f u n d a m e n t a l to a m b u latory medicine w a s r e v i e w e d by a faculty p e r s o n at a daily c o n f e r e n c e during the first half hour of clinic. In S e p t e m b e r 1985, o n e of the w eekl y topics w a s the CDC r e c o m m e n d a t i o n s for influenza vaccination. ~s Another of the topics w a s the A m e r i c a n C a n c e r Society's r e c o m m e n d a t i o n s for s cr e e ni ng m a m m o g r a phy, including a critical review of the HIP study of b r e a s t c a n c e r s c r e e n i n g methods. 16, is All residents w e r e p r o v id ed with written materials a n d h a d access to the pre-clinic discussion. The three months following these didactic presentations in 1985 repres e n ted the period from which the b a s e l i n e d a t a w e r e later derived. In July 1986, 45 clinic residents w e r e stratified b y training y e a r (16 Rls, 15 R2s, 14 R3s) a n d then alloc a t e d to o n e of three groups (n = 15 e a c h) using a r a n d o m - n u m b e r table a n d the m e t hod of r a n d o m permutations. In July a n d August 1986, residents in o n e of the groups (FLUVAX) spent o n e af t e r noon in lieu of their n o r ma l clinic activity auditing their 1985
Baseline
influenza vacci nat i on perform ances, using 40 to 60 of their own outpatient charts. New R 1s aud ite d the charts of patients a s s i g n e d to them at the be g in n in g of the a c a d e m i c y e a r . Explicit criteria for this task stated that influenza vacci nat i on w a s indicated for a n y patient w ho w a s 65 y e a r s old or older or w h o h a d chronic lung or h e a r t d i s e a s e of sufficient severity to h a v e n e c e s s i t a t e d clinic or hospital c a r e in the past y e a r . In a similar fashion, residents in a s e c o n d group (SCREENMAM) audi t ed their ow n charts for performa n c e of s c r e e n i n g m a m m o g r a p h y in 1985. Explicit criteria for this task stated that s c r e e n i n g m a m m o g r a p h y w a s indicated for a n y w o m a n 50 y e a r s of a g e or older w ho h a d not h a d o n e in the past 12 months. Residents in a third group (CONTROL)p e r f o r m e d no audit. In S e p t e m b e r 1986, the didactic sessions given in S e p t e m b e r 1985 w e r e r e p e a t e d for all 45 residents. In addition, all residents (FLUVAX,SCREENMAM, C O B O L ) w e r e provi ded with the cumulative group results of the 1985 vacci nat i on a n d s c r e e n i n g m a m m o g r a p h y audits. Efficacy of the QA intervention w a s e v a l u a t e d using two i n d e p e n d e n t audits p e r f o r m e d b y the authors. The first w a s a retrospective audit of a r a n d o m sam pl e of a third of all the patients s e e n in the medical clinic b e t w e e n S e p t e m b e r a n d D e c e m b e r 1985, to establish the baseline. These charts w e r e audited to det erm i ne w h e t h e r influenza vacci na tio n or
(1985)
Study Period
(1986)
D i d a c t i c Education QA Results Presented
Didactic Education
July A : s .
i
|
o
N
I
l
o I
F I
N i
A !
N t
J !
J
CONTROL Group In=iS}
.-
ij u l y A
s I
0 I
N I
O I
J I
F I
M I
A I
I
M J 1
,
g FLUVAX Group I I !
(n-t5}
1 Retrospective Review of a 33Z Random sample of Charts for Flu Vaccination (n-255) end Maimgrea Orders (n-222)
'
1I
SCREENMAM Group (n-t5J
Prospective Review of All Charts for Flu vaccination (n-8it) and Mommooram Orders (n-728) FIGURE 1. Study protocol. Analysis: 2 X 2 chi-square with Yates' correction. * Self-audits of 1985 flu vaccine ordering: ** self-audits of 1985 mammogram ordering; ® 45 residents stratified by level of training and then randomly assigned to one of the groups.
$62
Brady et aL, QUALm'ASSURANCEAUDITS TABLE 1
Baseline and Post-intervention Ordering Rates for Influenza Vaccine and Screening Mammography Influenza Vaccine Ordered/indicated* Group
(%)
Baseline(1985) CONTROL(1986) FLUVAX(1986) SCREENMAM(1986)
102/255 (40%) 149/252 (59%)t 173/280 (62%)t 175/279 (63%)t
Screening Mammography Ordered/Indicated*
(~)
18/222 39/199 47/268 58/222
(8%) (16%)t ( 18%)t (26%)t~
* Patient refusals were counted as indicated but not ordered (see text). t p < 0.02 for these groups vs. baseline. p < 0.05 for SCREENMAMVS. CONTROLor FLUVAX.
s c r e e n i n g m a m m o g r a p h y w a s indicated, using the explicit criteria d e s c r i b e d a b o v e . If o n e or both w e r e indicated, the chart w a s r e v i e w e d to d e t e r m i n e w h e t h e r i t / t h e y h a d b e e n or de r ed. Patient refusals of r e c o m m e n d e d p r o c e d u r e s w e r e c o u n t e d a s "not o r d er ed ." From this, a n overall b a s e l i n e or deri ng ra t e for influenza va c c i na t i on a n d s c r e e n i n g m a m m o g r a p h y w a s derived. Upon completion of the QA intervention in 1986, a second, prospective audit of influenza v a c c i n a t i o n and screening mammography was performed by the authors. Using the s a m e explicit criteria described ab o v e, the charts of all :patients s e e n b y all 45 residents from S e p t e m b e r to D e c e m b e r 1986 w e r e r e v i e w e d a n d p e r f o r m a n c e of influenza v a c c i n a t i o n or s c r e e n i n g m a m m o g r a p h y w a s r e c o r d e d w h e n indicated. As w a s the c a s e for ! 985, patient refusals d o c u m e n t e d in the char t w e r e c o u n t e d as "not ord e r e d . " Residents w e r e u n a w a r e of the existence of this ongoing audit, a n d the investigators w e r e generally u n a w a r e of e a c h resident's group assignment. Counting patient refusals a s "not o r d e r e d " w a s n e c e s s a r y to eliminate r e c o r d - k e e p i n g bias. For example, a resident w ho h a d recently c o m p l e t e d a chart audit of influenza vaccinations might b e m o r e likely to m a k e the effort to r e c or d a refusal t h a n a resident wh o h a d n e v e r d o n e a n audit a n d h a d no a p p r e c i a t i o n of the n e e d to do this. This policy resulted in underestimation of the true or de r i ng b e h a v ior of the residents but w a s n e c e s s a r y to m ai nt ai n the validity of the m e a s u r e m e n t . The principal endpoints w e r e group rates of influenza v accin atio n a n d s c r e e n i n g m a m m o g r a p h y . Thus, effectiveness of just pr e s ent i ng QA results w a s d e t e r m i n e d b y c o m p a r i n g 1986 CONTROLgroup rat es with 1985 b a s e l i n e rates. Effectiveness of actual resident participation in the audit process w a s determ i n ed b y c o m p a r i n g 1986 SCREV.N~tAMa n d FLUVAX g r o up rates with CONTROL group rates. All differe n c e s w e r e e v a l u a t e d b y 2 × 2 chi-square anal ysi s with Yates' correction.
C o s t - e f f e c t i v e n e s s w a s est i m at ed b y a s s e s s i n g the cost to the clinic for resident time spent in the QA p r o g r a m versus the p e r c e n t a g e i m prove m e n t in p r o c e d u r e ordering b y e a c h study group. This rudim e n t a r y analysis w a s p e r f o r m e d from the hospital's perspective a n d is strictly limited to the cost a n d effectiveness of the p r o g r a m itself, ignoring s e c o n d a r y factors such as r e v e n u e from or costs of additional m a m m o g r a m s or v a c c i n e a n d costs incurred b y further testing or visits resulting from the initial procedure. Assumptions underlying the c o s t - effectiveness analysis w e r e as follows. Residents in the clinic s e e a n a v e r a g e of 2.4 patients per hour, billing $25 p e r patient with a n a v e r a g e collection r a t e of 64%. From this, the residents' time in clinic is v a l u e d at $38.40 p e r hour ($0.64 per minute) to the hospital. In the audit groups, the a v e r a g e time spent b y e a c h resident reviewing charts w a s 90 minutes. The didactic session lasted 30 minutes, so that the total time spent b y auditing residents w a s 120 minutes, for a cost to the clinic of $76.80 per resident. In the control group, the residents spent no time auditing a n d 30 minutes in the didactic session, for a total of 30 minutes. Assuming the cost of g e n e r a t i n g audit results for the CONTROL group w a s 0, the cost to the clinic of passively providing QA results to residents w a s $19.20 per resident.
RESULTS The retrospective audit to establish the 1985 b a s e l i n e identified a s a m p l e of 255 patients for w h o m influenza v a c c i n a t i o n w a s indicated (Table 1). Vaccination w a s actually o r d e r e d for 102 of th e se patients (40%). A s a m p l e of 222 patients w a s identified for w hom s c r e e n i n g m a m m o g r a p h y w a s indicated. M a m m o g r a p h y w a s o r d e r e d for only 18 (8%) of these patients. The results of the investigators' prospective post-QA audit of all patients s e e n in a u t u m n 1986 a r e show n in Table 1. Influenza vacci nat i on w a s performed significantly m ore often in 1986 b y the CONTROLgroup (and the other groups) c o m p a r e d with the basel i ne in 1985. However, there w a s no difference in vacci nat i on ordering a m o n g the groups in 1986. Screeni ng m a m r n o g r a p h y also w a s p e r f o r m e d significantly m ore often in 1986 by the CONTROLgroup (and the other groups) c o m p a r e d with baseline. Additionally, the residents in the SCREENMAMgr o u p ord e r e d m a m m o g r a p h y significantly m o r e often t h a n residents in either the FLUVAXor the CONTROLgroup. Counting patient refusals as " o r d e r e d " ra th e r t h a n "not o r d e r e d " would not h a v e materially c h a n g e d these results. Only five instances of p a tients refusing m a m m o g r a p h y w e r e r e c o r d e d in the charts (one in the FLUVAXgroup a n d four in the CONTROLgroup), a n d counting these as " o r d e r e d " would
JOURNALOFGENERALINTERNALMEDICINE,Volume 3 (Nov/Dec), 1:988
not h a v e alter ed the n u m b e r s significantly. In contrast, 30 instances of patients refusing influenza v a c cine (11%) w e r e r e c o r d e d b y the FLUVAXgroup, 21 instances (7%) b y the SCREENMAMgroup, a n d 19 ins t a n ces (8%) b y the CONTROLgroup. H a d t he s e b e e n c o un ted as "ordered," then the c o m p l i a n c e of the FLUVAX group with the s t a n d a r d would h a v e b e e n 73%, c o m p a r e d with 70% a n d 67% for the S C n E V . N ~ a n d CONTROL groups, respectively (p = 0.17). However, b e c a u s e these results could be b i a s e d b y dift00 I,I
Z {_) o
90 B0 70
Z LU
50 40
Z LU {_) cr LU n
•
/
/
/
/
/ /
/
////I .....
Q_ F-
" •
F-
30
/
/
/
Baseline
Contro I
/
/
i
i
i
J
F ." ." ." ."
"
•
I///,
i
i
i
•
i
i
•
////, i
/
•
/
/
"
/
~
'
/
/
/ /
/
/
/
/
•
/ /
,
/ / / / , i
i
i
i
/////
¢///.
0
/
i
/Y/l,
t0
/
////~
,
•
•
20
63 ~(
B2
59 ~(
60
ScreenNam
Fluvax
1986
1985
FIGURE 2. Influenza vaccination performance. % vaccinated =
no. vaccinated X 100% no. indicated (includes pt. refusals)
* p < 0.001 for 1986 (all groups) vs, 1985; p = NS for intergroup comparisons 1986.
50
@J t@J
4O
3O
26 ww
t.,-e
Q_
u L
F
2O
10
18 w
/
•
B.!
/
/
i
/
/
i
/
/
.~ _.~
r _/ ./_i
/
/
"////
Baseline
1985
Control
Fluvax
ScreenMam
ferential r e c o r d - k e e p i n g a m o n g the groups, it is m ore valid to use the act ual orders for t hes e p ra c tices as endpoints b e c a u s e t hey a r e recordindependent. T hese c h a n g e s could not b e e x p l a i n e d b y differential a t t e n d a n c e at the didactic preclinic conferences. A t t e n d a n c e at the ~ u v A x didactic session w a s 75% (34/45) overall, with 73% a t t e n d a n c e (11/15) b y the FLUVAXself-audit group a n d 87% att e n d a n c e (13/15) b y the CONTROL g r o u p (p = NS). A t t e n d a n c e at the SCREENMAMdidactic session w a s 82% (37/45) overall, with 73% (11/15) a t t e n d a n c e b y the SCREENMA~tgroup a n d 100% (15/15) b y the CONTROL group (p = N S ) . Written audit results w e r e given to all residents, including those w ho did not a t t e n d the didactic sessions. C o s t - e f f e c t i v e n e s s of resident participation in audits w a s not e v a l u a t e d for influenza v a c c i n a t i o n since t here w a s no difference in effectiveness. In the SCREENMAM group, a n audit cost of $76.80 p e r resid e n t w a s a s s o c i a t e d with a n i n c r e m e n t a l rise of m a m m o g r a p h y ordering of 18 o r d e r e d p e r 100 indic a t e d (8/100 basel i ne to 26/100 post-QA), giving a c o s t - effectiveness ratio of $4.27 per 1% i n c r e a s e in m a m m o g r a p h y ordering. In the CONTROLgroup, the cost p e r resident of the p r o g r a m w a s $19.20 for a n i ncrem ent al rise in m a m m o g r a p h y orderi ng of 8 p e r 100 indicated (8/100 b a s e l i n e to 16/100 post-QA), giving a c o s t - e f f e c t i v e n e s s ratio of $2.40 p e r 1% inc r e a s e in m a m m o g r a p h y ordering. Thus, as su m in g the audit cost for controls w a s 0, the self-audit prog r a m a c h i e v e d better c o m p l i a n c e with s c r e e n i n g m a m m o g r a p h y s t a n d a r d s but at a cost roughly twice that of a traditional program. T hese cost - effectiveness ratios a s s u m e that the cost of the audit for the control group is zero, which would b e possible~ for example, in a clinic with a computerized m edi cal record. However, for most clinics, the cost of doing QA audits is likely not to b e zero, since a nurse or other skilled p e r s o n must do the audit. The n o n - p h y s i c i a n audit cost at which the c o s t - e f f e c t i v e n e s s of the SCREENMAMgroup e q u a ls the c o s t - e f f e c t i v e n e s s of the CONTROL group iS $14.96 per resident audited. Since the audit required about 1,350 minutes (22.5 hours) overall p e r group, if the p e r s o n a s s i g n e d to do the audit cost the clinic m o r e t h a n $9.97 per hour, the physi ci an audit a p p r o a c h would b e m ore cost-effective t h a n the m o re traditional a p p r o a c h .
1986
FIGURE 3. Screening mammography performance. %screened=
563
no. screened X 100% no. indicated (includes pt. refusals)
p < 0.001 for 1986 (groups combined) vs. 1985; * p < 0.02 for each group (1986) vs. baseline (1985); ** p < 0.05 for ScreenMarn vs. Fluvax or control.
DISCUSSION Providing residents in a t e a c h i n g hospital general medical clinic with QA audit results significantly a n d consistently i m proved c o m p l i a n c e with QA s t a n d a r d s for both influenza vacci nat i o n a n d s c r e e n i n g m a m m o g r a p h y . Residents w h o actually audi t ed charts for c o m p l i a n c e with s c r e e n i n g m a m -
$64
Bradyetal., QUALF'/'Y ASSURANCEAUDFrS
m o g r a p h y s t a n d a r d s d e m o n s t r a t e d m u c h better c o m p l i a n c e with those s t a n d a r d s t h a n did residents w h o passively r e c e i v e d audit results. This important finding w a s not consistent, h o w e v e r , since residents w h o a u d i t e d their charts for c o m p l i a n c e with influe n z a v a c c i n a t i o n s t a n d a r d s did not perform a n y better t h a n those w h o just r e c e i v e d audit results. T h e r e a r e two likely e x p l a n a t i o n s for the disc r e p a n c y b e t w e e n the results for m a m m o g r a p h y a n d influenza vaccination. The first is that practical limitations m a k e a 60% influenza v a c c i n a t i o n ordering r a t e n e a r the m a x i m u m attainable. S o m e p a tients will refuse v a c c i n a t i o n (at least 11% did in this study), s o m e m a y b e allergic to eggs, others might h a v e a n a c u t e illness at the time of the visit which p r e c l u d e s vaccination. Indeed, t h r e e other studies of motivational modalities failed to p r o d u c e o r d e r i n g rates e v e n c o m p a r a b l e to this one,~°. ~4. ~9 a n d only s t a n d i n g nursing orders for v a c c i n a t i o n h a v e prod u c e d a better p e r f o r m a n c e t h a n o b s e r v e d in this study. ~ Thus, w e m a y h a v e failed to o b s e r v e a b e n efit from self-audit in this c a s e b e c a u s e no such a d ditional benefit w a s possible. The s e c o n d e x p l a n a tion is that motivational a n d e d u c a t i o n a l techniques m a y not b e e q u a l l y effective for all p r o c e d u r e s . Influe n z a v a c c i n a t i o n is well a c c e p t e d b y patients a n d easily performed, while s c r e e n i n g m a m r n o g r a p h y is not as readily a c c e p t e d or a s easily performed, a s e v i d e n c e d b y the v e r y different b a s e l i n e ordering rates. For this reason, p a s s i v e receipt of QA audit results m a y b e all that is n e c e s s a r y to i n c r e a s e the influenza v a c c i n a t i o n ordering r a t e to the maximum, while a higher d o s e of motivation (active participation in the QA audit) is n e e d e d to h a v e a n impact on the s c r e e n i n g m a m m o g r a p h y rate. An alternative interpretation of the s c r e e n i n g m a r n m o g r a p h y findings is that sCrtEENMAM group residents o r d e r e d m o r e m a m m o g r a p h y b e c a u s e t h e y k n e w their o w n ordering rates, w h e r e a s the CONTHOL group residents k n e w only a n overall g r o u p rate. In other words, w a s it the f e e d b a c k of specific, p e r s o n a l results a n d not the p r o c e s s of auditing charts that w a s important? We c a n n o t e x c l u d e this possibility, but such a n e x p l a n a t i o n for our findings is ur~]~ely. While CONTROL g r o u p residents w e r e n ' t given their o w n ordering rates, the group r a t e for m a m m o g r a p h y w a s so low a n d so uniform at baseline, a n d residents' a c c e p t a n c e of audit results a s true w a s so quick a n d complete, that w e believe that CONTROL group residents a c c e p t e d the group r a t e as their o w n rate. Thus, the most likely e x p l a n a tion of the findings is that the audit effort itself w a s responsible for the g r e a t e r p e r f o r m a n c e of s c r e e n Lug m a m m o g r a p h y b y the SCREENMAMgroup. The FLUVAXg~oup o r d e r e d m a m m o g r a p h y a t the s a m e r a t e that the CONTrtOLg r o u p did, a n d the SCREENMAM group o r d e r e d influenza v a c c i n a t i o n at
the s a m e r a t e a s the other groups. Thus, t h e r e did not a p p e a r to b e a n y spillover effect of the auditing exp e r i e n c e to p r o c e d u r e s other t h a n the o n e b e i n g audited. Thus, the impact of this t y p e of QA intervention a p p e a r s to b e specific for the p r o c e d u r e u n d e r consideration. The results of this brief a n d easily a p p l i e d intervention c o m p a r e v e r y f a v o r a b l y with other e d u c a tional a n d motivational modalities such a s c h a r t reminders a n d c o m p u t e r - b a s e d prompting, which h a v e a c h i e v e d influenza v a c c i n a t i o n o r d e r i n g rates of 30% to 46% l°' ~4. m a n d s c r e e n i n g m a m m o g r a p h y ordering rates of 8% to 32%. 14' 19Moreover, since this QA strategy internally motivates the physician, it m a y continue to influence b e h a v i o r long-term o n c e the resident h a s left the clinic a n d no longer h a s the benefit of special strategies such a s c o m p u t e r reminders. The c o s t - e f f e c t i v e n e s s calculations indicate that significant improvements c a n b e m a d e at r e a s o n a b l e cost. E v e n though the c o s t - e f f e c t i v e n e s s ratio for the SCREENMAMg r o u p w a s h i g h e r t h a n that for the c o ~ r t o r , group [assuming a control audit cost of zero), m a n y clinics might find the a d d e d effectiveness worth the a d d e d cost. 2~ If the cost of a non-physician auditor (to allow p a s s i v e QA audit f e e d b a c k to physicians) w a s m o r e t h a n $9.97/hour g i v e n these results, t h e n the resident self-audit m a y b e both more effective a n d m o r e cost-effective for selected procedures. This study is limited s o m e w h a t b y the fact that only two p r o c e d u r e s w e r e studied a n d the subjects of the study w e r e resident physicians. W h e t h e r the results c a n b e g e n e r a l i z e d to other p r o c e d u r e s or physicians in practice n e e d s to b e d e m o n s t r a t e d . Additionally, the lack of additional i m p r o v e m e n t in influenza v a c c i n a t i o n ordering in the FLUVAX selfaudit group suggests that self-audits m a y not b e effective for all procedures. R e s e a r c h should further e v a l u a t e audit with f e e d b a c k , self-audit, a n d other methods of quality a s s u r a n c e in different settings a n d with different p r o c e d u r e s . While th/s discussion h a s focused on the quality a s s u r a n c e a s p e c t s of resident physicians' selfaudits, the potential e d u c a t i o n a l v a l u e of such prog r a m s should not b e ignored. O n e of the principal difficulties in a m b u l a t o r y m e d i c a l e d u c a t i o n is the n e e d for intensive, o n e - o n - o n e faculty involvement. ~ The p r o g r a m d e s c r i b e d in this p a p e r requires v e r y little faculty input a s i d e from the d a t a collection n e e d e d for r e s e a r c h purposes. Thus, b u s y clinic directors could a s s e m b l e r e a d i n g materials a n d explicit QA s t a n d a r d s a n d use resident physic i a n self-audits a s a n innovative w a y to effectively deliver a g i v e n m e s s a g e while s p a r i n g e x p e n s i v e faculty time. In summary, resident physicians w h o passively
JOURNALOFGENERALINTERNALMEDICINE,Volume3 (Nov~Dec), 1988
receive QA audit results achieve greater complia n c e with defined QA standards. Active participation in the QA audit by the physician results in e v e n better compliance for some procedures and may be acceptably cost-effective. Closer involvement of physicians in the quality assurance auditing process deserves further research and consideration. REFERENCES 1. Public Law 92-603, Social Security Amendments of 1972: 92nd Congress, Second Session, United States Code. Congressionaland Administrative News 1972; 1:1548-747. 2. Public Law 93-222, Health MaintenanceOrganization ACt of 1973: 93rd Congress,First Session, United States Code.Congressionaland Administrative News 1974; 1:1015-42. 3. Joint Commission on Accreditation of Hospitals. Accreditation manLlal for ambulatory health care, 1982 edition. Chicago: Joint Commission on Accreditation of Hospitals, 1982. 4. 0t~ce of Health MaintenanceOrganizations. Quality assurancestrategy for HMOs. Washington, DC: Public Health Service, U.S. Department of Health, Education, and Welfare, September 1979. 5. Accreditation Association for Ambulatory Health Care, Inc. Accreditation handbook for ambulatory health care, 1982-83 edition. Skokie, IL: Accreditation Association for Ambulatory Health Care, Inc., 1982. 6. Palmer RH, Nesson HR. A review of methods for ambulatory medical care evaluations. Med Care 1982; 20:758-79. 7. Hill RK. Quality assurance in ambulatory care. Primary Care 1980; 7:713-21. 8. Brook RH, Williams KN, Avery AD. Quality assurancetoday and to-
565
morrow: forecast for the future. Ann Intern Med 1976; 85:809-17. 9. PayneBC. The medical record as a basisfor assessingphysiciancompetence. Ann Intern Med 1979; 91:623-9. 10. Kosecoff J, Fink A, Brook RH, et al. General medical care and the educationof internists in university hospitals. Ann Intern Med 1985; 102:250-7. 11. Brook RH, Williams KN. Efl:ect of medical care review on the use of injections. Ann Intern Med 1976; 85:509-15. 12. Beloff JS. Improving staff performance in an ambulatory health care system. QRB 1983; 9:209-12. 13. Barnett GO, Winickoff R, Dorsey JL, Morgan MM, Lurie RS. Quality assurance through automated monitoring and concurrent feedback using a computer-based medical information system. Med Care 1978; 16:962-70. 14. McDonaldCJ, Hui SL, Smith DM, et al. Remindersto physiciansfrom an introspective computer medical record: a two-year randomized trial. Ann Intern Med 1984; 100:130-8. 15. Centers for DiseaseControl. ACIP: prevention and control of influenza. MMWR 1985; 34:261-74. 16. American Cancer Society. ACS report on the cancer-related health checkup. CA 1980; 30:225-9. 17. Wones RG, Rouan GW, Brody TL, Bode RB, RadackKL. An ambulatory medical education program for internal medicine residents. J Med Educ 1987; 62:470-6. 18. Shapiro S. Evidenceon screeningfor breast cancerfrom a randomized trial. Cancer 1977; 39:2772-82. 19. Cohen DI, Littenberg B, Wetzel C, NeuhauserDV. Improving physician compliancewith preventive medicineguidelines.Med Care 1982; 22:1040-5. 20. Margolis KL, Lofgren RP, Korn JE. Organizational strategies to improve influenzavaccinedelivery: effect of a standing order in a general medicine clinic. Clin Res 1987; 35:848A. 21. Doubilet P, Weinstein MC, McNeil BJ. Use and misuse of the term "cost effective" in medicine. N Engl J Med 1986; 314:253-5.
