Construct Validation of a Physician Satisfaction ... - Wiley Online Library

10

I

Journal for Healthcare Quality

Construct Validation of a Physician Satisfaction Survey Robert J. Wolosin, Sabina B. Gesell, Brian Taber, Gladys J. Epting

Abstract: Hospitals continuously look for ways t o improve patient care and retain high-quality physicians. Previous research indicates that physicians’ satisfaction with where they practice i s a crucial part of addressing these issues. A reliable and valid method t o assess physician satisfaction i s needed i n order t o identify potential areas of discontent. The purpose of the present study was t o develop and validate a self-administered medical staff satisfaction survey. The survey contains 13 Likert-type items divided into three reliable subscales: Quality of Patient Care ( a = .84), Ease of Practice (a= .76), and Relationship with Leadership (a= .92). Results from both exploratory and confirmatory factor analyses supported the survey’s structure and robustness across three independent samples. Key Words physicians psychometrics satisfaction validation studies

/ournn//or Healfhcare Qualify Vol. 28, No.4, pp. 10-21 02006 National Association for HealthcareQuality

Physicians’ satisfaction with their careers is an ongoing concern both within and outside of the profession. Within the medical profession, physician satisfaction is a concern because it is related to quality of patient care. Because physician satisfaction affects the size of the physician workforce, it is a social issue as well. Although physicians’ satisfaction seems to have declined over time, a recent overview of the topic concluded that discontent is neither new nor limited to physicians (Zuger, 2004). Moreover, physician dissatisfaction may be a reaction to current conditions when compared to those of the ”Golden Age” of the medical profession in the 1950s and 1960s. The cure may be found in correcting students‘ expectations. “The key to restoring a sense of contentment to the medical profession may lie in the hands of educators who encourage students to have more accurate expectations of a medical career than did the generations trained during the tumultuous past 50 years” (Zuger, p.74). Another commentator suggested that reports of rampant physician dissatisfaction have been greatly exaggerated.Increased patient consumerism and demands for preventive

services consume physicians’ time and cause disappointment when patient expectations are not met. Moreover, the rise of managed care and its demands for documentation, the increased complexity of the regulatory context, and the pressure to practice ”defensive medicine” have contributed to perceptions of loss of professional autonomy. Such perceptions, combined with turmoil in the U.S. healthcare system at large, may contribute to a sense of dissatisfaction. Nonetheless, physicians themselves report good overall career satisfaction and enjoy trust and loyalty from patients (Mechanic, 2003). The authors of a large-scale physician satisfaction survey came to a similar conclusion, although they noted that level of satisfaction varied by practice site (Landon, Reschovsky, & Blumenthal, 2003). The literature suggests that, at the level of the individual physician, career satisfaction (or more precisely, career dissatisfaction) is implicated in a number of important outcomes. These include outcomes for physicians, patients, and nurses and their employers and retention within specific practices (Williams & Skinner, 2003).

Physician Outcomes Investigators working with the Society of General Internal Medicine used structural equation modeling to determine the relationships between characteristics of physicians, their practices and patients, and their perceived physical and mental health (Williams et al., 2002). Survey responses from 2,325 primary care physicians showed that practice characteristics (size, culture, workflow) influenced perceived stress and job satisfaction, which in turn influenced self-rated physical and mental health. Similarly, a group of Dutch investigators (Visser, Smets, Oort, & de Haes, 2003) found that Dutch medical specialists’ professional satisfaction was affected by their perceptions of working conditions (e.g., work

Vol. 28 No. 4 July/August 2006

initial report stated that nurse-physician interactions were directly related to nurse job satisfaction and retention; moreover, disruptive physician behavior such as disrespect, beratt‘atient Outcomes fi number of studies have investigated the ing of colleagues, use of abusive language, or Felationship between physician satisfaction condescension was considered a key issue by and various aspects of patient care and out- all three types of respondents. Respondents comes. They have generally found that higher estimated that 2%-3% of the medical staff professional satisfaction for a physician cor- displayed such behavior (Rosenstein, 2002). relates with better outcomes for his or her Rosenstein, Russell, and Lauve later specuatients. A longitudinal study reported in 1993 lated that the ”increasing external pressures’’ vestigated physician satisfaction in relation to patients’ adherence to medical regimens (DMatteo et al., 1993). Patients with chronic conditions (e.g., diabetes, hypertension, congestive heart failure) were asked to rate their kdherence to their physicians’ recommenIdations. The relation of patients’ adherence ratings to their treating physicians’ ratings of professional job satisfaction was statistically significant, albeit small (Y = .25, p < .05). The researchers concluded that ”physicians who facing physicians result in “demoralization” were happier in their work appeared to have and an increased ”predilection for disruptive done something . . . to make adherence easier behavior” (Rosenstein et al., 2002, p. 10). for their patients” (DiMatteo et al.). Another study investigated whether general internists’ Practice Retention Outcomes satisfaction was related to two kinds of patient A number of studies examined the link satisfaction: satisfaction with overall health- between physician satisfaction and retention care and satisfaction with the most recent visit in a practice. One cross-sectional study found (Haas et al., 2000). Physicians rated their over- that physicians’ satisfaction with their jobs all work satisfaction, which was associated was an outcome of perceived job stress and with both measures of patient satisfaction. in turn influenced their stated intentions of However, mean differences in patient satis- leaving their current practices (Williams et faction as a function of physician satisfaction al., 2001). Similar results were found among were relatively small, in part because of the general practitioners in the United Kingdom concentration of patient satisfaction scores (Simoens, Scott, & Sibbald, 2002). A second near the high end of the scale. More recently, U.S. cross-sectional study examined job satisa survey of family physicians found that faction among generalist and specialist phythose who expressed dissatisfaction with their sicians. A questionnaire returned by 1,939 careers were more likely to report difficulties physicians assessed multiple areas of satisfacwith patient care, such as maintaining relation with their work and asked their intent tionships that promote high-quality patient to leave their current workplace within 2 care (DeVoe, Fryer, Hargraves, Phillips, & years. Physicians who rated their job satGreen, 2002). isfaction lowest were most likely to anticipate a job move. The authors wrote, “The Nurse and Nurse Employer Outcomes most obvious consequence of a dissatisfying The Voluntary Hospitals of America (VHA) job. . . is that physicians will quit to work West Coast conducted a survey of hospi- elsewhere, disrupting patient-physician contal nurses, physicians, and administrators tinuity and creating organizational instability designed to assess how physicians’ disruptive and replacement costs, estimated at nearly behavior, nurse-physician relationships, and $250,000 per physician lost” (Pathman et al., administrative response influenced nurses’ 2002, p. 593). A longitudinal study examined job satisfaction and retention (Rosenstein, the association of self-rated job satisfaction 2002; Rosenstein, Russell, & Lauve, 2002). The with actual turnover among primary care intruding into private life, difficulty living up ti0 one’s professional standards).

L

They have generally found t h a t higher professional satisfaction for a physician correlates with better outcomes for his or her patients.

I

11

12

I


physicians 4 years later and found that dissatisfied physicians were more than twice as likely as their satisfied counterparts to leave their practice in the intervening years (Buchbinder, Wilson, Melick, & Powe, 2001). The same group of investigators had previously estimated replacement costs at $236,383 for general and family practitioners; $245,128 for general internists; and $264,645 for general pediatricians. In the aggregate, turnover costs for the three specialties exceeded $65 million (Buchbinder, Wilson, Melick, & Powe, 1999). Hospital work affects physician career satisfaction. Hospitals traditionally provide independent local physicians with a setting in whch to treat their sickest patients (Grifith & White, 2002). For a physician, hospital practice provides income, fellowship, medical education, and prestige. Hospital admitting privileges validate the physician’s training and serve as a mark of status; one of the first tasks for a new physician is to establish admitting privileges at one or more local hospitals.Although the traditional model has changed recently (more physicians are becoming direct employees of hospitals and health systems, and hospitalists are assuming more of the care of local physicians’ patients), in many parts of the United States it remains intact; in others, it is a viable alternative. Hospitals have much at stake, depending on the satisfaction levels of their medical staff (Reece, 2003). In many locations, where the traditional fee-forservice model is strong, hospital chief executiveofficersare increasinglyconcerned about the number of physicians abandoning hospitals and establishing their own for-profit enterprises. Cardiologists, cardiac surgeons, and orthopedic surgeons, whose work contributes substantially to the hospital‘s revenue, can cause financial harm by establishing a freestanding ambulatory center that skims the most profitable patients from the hospital’s rolls. Physicians‘ motives for such moves include the desire for higher incomes and frustration over the scheduling problems and inefficient, outdated technology that are common in some hospitals. Physician satisfaction is related to issues that affect both patients and hospitals. Coherent and accurate assessment of this factor is therefore crucial if sound decisions are to be made about conditions of physicians’ hospital practice. The purpose of the current study was to idenhfy the dimensions that underlie physician satisfaction as it relates specifically to hospital practice.

Instrument History The instrument used in this study is a revision of a survey originally developed in 1997-1999 by Parkside Associates, Inc. Parkside initiated efforts to develop a survey that would assess attending physicians’ opinions about hospital services to provide general and specialty acute care hospitals with direction for planning. An initial item pool was created after review of the relevant literature. Next, a focus group of hospital administrators and medical staff members evaluated the items’ importance, clarity, and applicability. Questions were arranged into 13 scales, such as Quality of Staff and Services and Medical Technology and Equipment. Two response scales were used for different items: poor to excellent and low to high. In 1998an initial pilot study conducted at 25 acute care hospitals confirmed the validity and reliability of the scales; in 1999 a larger data set (3,451 responses from 38 acute care community hospitals) was used to test the previously obtained factors, assess criterion-related validity, and create a comparative benchmarking database. Factor analysis of the larger data set eliminated one of the original scales; the remaining 12 scales accounted for 70% of the common variance. This survey, consisting of 95 questions to rate aspects of hospital quality, as well as 10 demographic items, was marketed in 1999;in 2001 rights to the survey were transferred to its current owner. The survey was revised in 2002. An expert panel of hospital administrators representing 18 University Healthsystem Consortium (UHC) members was formed to discuss refining the instrument and to provide feedback during a new testing period. In addition, UHC provided an existing organizational structure that was able to facilitate data collection. The panel was asked to consider items from an attending physician’s viewpoint and to review questions for face and content validity. Items were reworded until agreement was reached on a set of items that were expressed in clear, concise language and that covered the most salient characteristics of hospitals. Extensive efforts were made to eliminate sources of instrument bias, such as question wording and ambiguity, and to ensure that the items addressed the focal concepts and the range of meanings included within each dimension. The Medical Staff Survey uses a 5-point Likert


response scale with the following categories: 1 = very poor, 2 = poor, 3 =fair, 4 = good, 5 = very good. This rating scale has proven to be effective in the measurement of satisfaction across a variety of healthcare settings (Clark, 2003; Drain, 2001; Gesell, 2001; Gregory & Kaldenberg, 2000; Hall & Press, 1996). From autumn 2002 through spring 2003 the survey was field-tested at 18UHC-member academic medical centers. Each center supplied a list of its medical staff, and a three-wave, mailout, mail-back survey, including a cover letter signed by a center administrator, was sent to all physicians on the lists. A total of 4,577 completed surveys was returned, for an average response rate of 39%per facility (range 22%49%).At the conclusion of the test, the instrument underwent psychometric analysis and was shortened from 82 questions in 13 subscales to 68 questions in 9 subscales (Patient Care, Services, Strategic Planning, Medical Technology and Equipment, Confidence in Leadership, Quality of Communication/Responsiveness, Scheduling, Emergency Room/After Hours Services, Overall Assessment) that measure specific dimensions of a physician’s experience as a member of a healthcare facility’s medical staff. However, it was understood from the outset that the sites in the initial test that led to the revised questionnaire represented a highly selected group of institutions-academic medical centers-and that physician satisfaction in community hospitals could be based on different hospital experiences. In 2004 the existing survey was enhanced by using a theoretical structure for physician satisfaction to provide a framework for survey questions. Physicians’professionalsatisfactionas members of hospital medical staffs reflects how well the hospital allows physicians to fulfill altruistic and self-interested motives (Jonsen, 1983). Altruism and self-interest form the moral basis of medical practice. The altruistic basis of medicine is institutionalized in medical education, when up until very recently interns learned that they must answer the call of patient service regardless of the cost to themselves. The self-interest of the profession can be seen in its struggles against regulation and against other healing professions that threaten its territory (e.g., nursing). Hospitals allow physicians to serve others (altruism) by providing high-quality patient

care; they allow physicians to serve themselves (self-interest) by making it easy for them to practice medicine within an institution. Questions based on this formulation of physician satisfactionwere used in the present study. Experience in the marketplace suggested that physicians are more satisfied when they have good relationships with hospital administrators; for that reason, a number of items assessing such relationships were included in the study. In addition to the 13 items reflecting the above content domains, the survey contained 7 items to assess global ratings such as overall satisfaction with the hospital.

Physicians’ professional satisfaction reflects how well the hospital allows physicians t o fulfill altruistic and self-interested motives. Methods Sample Satisfaction data for this study were drawn from the medical staff database maintained by an independent research firm specializing in satisfaction measurement and improvement within the healthcare industry. Hospitals voluntarily enter into a contract with the research firm and pay a fee for the collection, storage, analysis, and benchmarking of their satisfaction data for the purposes of internal quality improvement. Each client hospital is asked to invite all physician members of its medical staff to complete the survey. Client hospitals understand that data from their facilities may be deidentified, aggregated, and used for research purposes, as was done for the present article. As of March 2005 the database contained the completed surveys of 17,422 physicians at 124 hospitals across the United States. Data from physicians who were mailed blank surveys between September 2004 and February 2005 and who returned completed surveys between October 2004 and March 2005 were extracted from the database, resulting in a sample of 3,781 physicians at 35 hospitals. These data were randomly split into three independent samples comprising physicians from different hospitals. For the purposes of

I

13

14

I


the present study, IZ will refer to the number of physicians in each sample and N will refer to the number of hospitals from which the samples of physicians were drawn. The resulting sample sizes were as follows: n = 1,686, N = 12 (Sample 1);n = 1,015, N = 11 (Sample 2); n = 1,080, N = 12 (Sample 3). Responding physicians represented over 50 specialties. Sample demographic information is show in Table 1. Four percent of respondents completed the survey online.

interaction that leads respondents to acquiescd and to give socially desirable responses. In order to achieve a strong response rate, the following three-wave mailing method was used: prenotification letter, cover letter with survey (Wave 1);reminder letter with survey, if the first survey was not returned within 2 weeks (Wave 2); second reminder letter with survey if second survey was not returned within 2 weeks (Wave 3). The cover and reminder letters were signed by a member of senior management such as the hospital CEO. The letters informed physicians that the hospital - Table 1. Respondent Demographics for Each Sample administration was interested in improving the hospital as a place to practice medicine and Physician Age Physician Gender % desired honest feedback. The sentence “No Mean SD Min. Sample Max. Male Female member of the medical staff, administration, 1 48.0 10.0 or board of trustees (or nanie of other appropriate 27 84 81 19 group) will see your individual questionnaire” 2 49.3 30 10.4 90 82 18 was included in the cover letter to assure 3 48.0 10.4 26 79 21 90 physicians that they could express themselves freely on the survey. Procedure Hospital response rates averaged 35.5%. A self-administered survey was provided to Because of the relevance of its content, the physicians either through the mail or on the satisfaction survey tends to achieve a higher Internet. Invitations to complete a survey response rate than other direct mail surveys. were mailed by the research firm to eligible In general, surveys that are mailed ”cold” (i.e., physicians identified by participating hos- without any previous contact between mailer pitals. Physicians could either complete a and receiver) tend to achieve a 20% response survey and return it by mail or log on to a rate depending on length and content of the Web site and take the identical survey on the question set (Kelley, Clark, Brown, & Sitzia, Internet. Those choosing the Internet option 2003), and response rates of 5%-10% are comwere given a password that could be used mon (Alreck & Settle, 1995). Completed suronly once to ensure that users would not veys were returned to the research firm for complete more than one survey. Eligibility coding and analysis. was determined by status as a member of the medical staff with either admitting privileges Data Analysis or a hospital-based practice (e.g., emergency Exploratory Factor Analysis: Sample 1 medicine, hospitalist, anesthesiology). House To detect structure in the relationships among staff were excluded because their relationship the 13survey variables, a principal components with their institution is different from that analysis (PCA) was performed on Sample 1. of attending physicians. Some institutions PCA is a statistical technique commonly used included courtesy staff as well as regular to analyze the correlations among a group of variables and explain the variables in terms of attending physicians. Preaddressed stamped envelopes were any common underlying dimensions (factors). A PCA, or any other exploratory factor analincluded with the survey packet so that physicians could return their mail survey at no cost. ysis, shows the order and structure in a data set The Internet surveys could be completed on containing multiple variables. It is a method any computer with Internet access. This pro- for understanding the underlying dimensions cedure allowed physicians the opportunity to and dynamics that give rise to a psychological complete the survey in private, at their own phenomenon-in this case physician satisfacconvenience, and using a mode of their choos- tion. Individual physician satisfaction varies, ing. It further allowed for more complete cog- yet the internal attribute is not directly meanitive processing of the survey questions. Both surable. What can be measured are surface survey methods eliminate the person-to-person attributes that are systematically influenced by

Val. 28 No. 4 July/August 2006

the internal attribute or underlying dimension of physician satisfaction. The surface attributes are assessed by individual survey questions. A physician’s responses to the survey questions are influenced to some degree by his or her overall level of satisfaction. Although one cannot directly measure a physician’s level of satisfaction, one can measure the extent to which he or she thinks staff reliably report changes in a patient’s condition, how timely the follow-through is on written orders, and so on. The PCA shows how well individual survey questions measure the underlying dimensions of hospital practice satisfaction, how best to combine multiple questions to capture hospital practice satisfaction, and how much linear combinations of variables can explain differences between individual physicians. In PCA and other exploratory factor analyses, the researcher explores the underlying order and structure of the data rather than imposing structure onto the data. Each identified factor constitutes specific questions that are a facet of the broader evaluative dimension (in this case, medical staff satisfaction). In other words, a factor analysis is the means of dividing a multiquestion scale into meaningful subscales. Global ratings of the hospital as a place to practice medicine were excluded from the PCA because of their high correlations with the independent variables of interest. The factor loading reflects the role each variable plays in defining each factor. A factor loading is the correlation between a variable and a factor, Loadings indicate the degree of correspondence between the variable and the factor, such that hgher loadings make the variable representative of the factor. According to conservative guidelines, factor loadings greater than .30 are statistically significant in a sample size of 350 or larger (Hair, Anderson, Tatham, & Black, 1995). Cronbach’s alpha was computed to evaluate the reliability of the factors identified in the PCA. A set of questions with no internal consistency has an alpha of 0.0, indicating that the questions within the scale may not be measuring the same issue. A set of questions with perfect internal consistency has an alpha value of 1.0. Data were missing either because respondents chose not to answer a question that applied to them (i.e., they had experience with and had made a judgment on a particular

I

aspect of their work situation but chose not to reveal their judgment on the survey) or because respondents were following survey instructions to skip questions that did not apply to them. The pattern of incomplete data indicated that most missing data were because of the items’ not being applicable rather than to the respondents’ refusals to answer. Items that were considered universally applicable to physicians because of the nature of their work environment (e.g., ”Staff’s concern for and interest in your patients”) had missing data within traditionally acceptable limits (7%), and items that were not expected to apply to all physicians (e.g., “Medical technology and equipment available in the ICU/CCU”) exhibited the most missing data (47%).

I A factor analysis i s the means of divid ng

I a multiquestion scale into rneani 1gful 1 subscales.

In the PCA, a missing data (pairwise) correlation matrix was analyzed (i.e., the correlations between each pair of variables were calculated from all cases having valid data for those two variables), because (a) respondents were instructed to skip questions that did not apply to them, and most missing data were deemed appropriately missing, and (b) using only full-information cases may have resulted in a unique subsample of physicians that was not representative of the full group under examination. The sample size was greater than 1,000, which is generally considered excellent for the purposes of obtaining reliable results with factor analysis (Tabachnik & Fidell, 1996). SPSS 12.0.1 was used to compute the exploratory factor analysis. Confirmatory Factor Analysis: Sample 2 Confirmatory factor analysis (CFA) was used to test whether the multidimensional model of physician satisfaction obtained in the PCA fit a second set of data. In confirmatory factor analysis, the researcher imposes a hypothesized structure onto the data-here we imposed the structure revealed through the PCA-to determine how well the hypothesized model corresponds with the data‘s actual structure. If the underlying (covariance) structure of Sample 2 is similar to the structure implied by the PCA, the results are evidence that the model

15

16

I


does indeed explain the underlying dynamics of medical staff satisfaction. If the covariance structure of Sample 2 and the covariance structure implied from the PCA are dissimilar, then the results suggest that the hypothesized model fails to accurately reflect the dynamics of medical staff satisfaction. The instrument’s proposed factor structure conskts of three factors, each of which is measured by multiple items (Figure l).Items contribute information to only one factor. AU factors were allowed to correlate with the other factors. The variance of each factor was set to 1.0. Unique factor loadings, or error terms, were not correlated. Maximum likelihood (ML) estimation was used to estimate the parameters of the models.

Figure 1. Hypothesized Three-Factor CFA Model of MedicaL Staff Satisfaction

Vote. Variable effects are signaled by arrows in the model; null effects are

signaled by an absence of an arrow. In a structural equation model, each unobserved latent variable must be explicitly assigned a metric (a measurement range) by setting the path from the latent variable to one of its indicator variables to the value of 1.0. Given this constraint, the remaining paths :an then be estimated. Similarly, error terms are set to 1.0.

Assessment of the CFA model fit was based on multiple goodness-of-fit measures including the normed fit index (NFI) (Benter & Bonnett, 1980), the relative fit index (RFI)(Bollen, 1986), the incremental fit index (IFI) (Bollen, 1989), the Tucker-Lewis index (TLI) (Tucker & Lewis, 1973), the comparative fit index (CFI) (Bentler, 1990), and the root mean square error of approximation (RMSEA) (Browne & Cudeck, 1993).The NFI, RFI, IFI, TLI, and CFI yield values ranging from 0 to 1.00. In general for these incremental fit indexes, values above .95 indicate a good model fit. RMSEA values below .08 signify reasonable fit, and values below .05 signify good model fit. A 90% confidence interval (CI) around the RMSEA demonstrates the precision of the estimate. The RMSEA p value is for testing the null hypothesis that the population RMSEA is no greater than .05. AMOS 5 was used to compute the CFA. The program can estimate ML parameters with incomplete data, yielding more consistent, efficient, and unbiased results compared to using listwise deletion, pairwise deletion, or imputation. The ratio of cases to parameters to be estimated was 24:1, generally considered an excellent ratio (Tabachnik & Fidell, 1996).

Cross-Validation Analyses: Sample 3 Evaluation of the model’s robustness was judged on the basis of two cross-validation analyses: (1) invariance of the factor structure across two independent samples and (2) convergence of the factor loadings obtained by ML with the factor loadings obtained from 2,000 bootstrap samples. To test for equivalency of the factor structure across multiple samples, cross-group constraints were imposed on all measurement weights and structural covariances between Samples 2 and 3, and the data from both samples were analyzed simultaneously. The difference in model fit between the two nested models provides the basis for determining whether the hypothesized equality constraints are tenable. A nonsignificant change in x2 indicates invariance, or robustness. The ratio of cases to parameters was 24:1, generally considered an excellent ratio (Tabachnick& Fidell, 1996). Two thousand bootstrap samples were simulated from Sample 3 to determine whether estimate bias existed in the measurement weights obtained by ML. Bootstrapping is a way of testing the reliability of a data set. It is a way of


placing confidence limits on a set of observations without making too many assumptions about the data. This statistical method creates new samples through repeated random sampling with replacement from an original sample. The original sample is used as a population from which samples are drawn to study the variability of estimators. Every resample has the same number of data points as the on@ sample. Bootstrapping is a recommended way of estimating standard error and sigruficancebased on empirical resampling with replacement of the data. Taking a large number of random samples from the data set generates information on the variability of parameter estimates based on the empirical samples rather than on assumptions about probability theory of normal distributions. Using AMOS,bootstrapping can be employed only with complete data. Consequently, all fullmformation cases from Sample 3 were included in the analysis (n = 85). The ratio of cases to parameters was 20:1, generally considered an excellent ratio (Tabachnick & Fidell, 1996). Minimal parameter estimate bias confirmed the accuracy of the original ML estimates and lends further support for the hypothesized model. .

Results Exploratory Factor Analysis: Sample 1 A principal component factor analysis using a Promax (oblique) rotation was performed on the 13 independent variables from the survey. A three-factor solution provided a clean fit to the data. The factor loadings lined up with predictions and paralleled the structure of subscales on the questionnaire. Items loaded highly onto the same factor on which the other items constructed for a given dimension loaded and, in general, did not load on the other two dimensions. All loadings exceeded the minimum cutoff of .30 (ranging between .41 and .95), indicating that they were indeed representative of the underlying factors. The factors were labeled Relationship with Leadership, Quality of Patient Care, and Ease of Practice. Table 2 shows the complete item wording, factors, and loadings. The factor solution explained 66% of the total variance in the data set. The Relationship with Leadership factor accounted for 45% of the variance, the Quality of Patient Care factor accounted for 13% of the variance, and the Ease of Practice factor accounted for 8% of the

Table 2. Item Wording and Factor .oadings (Sample 1) Factors and Loadings

2uestionnaire Item A2. Staff’s concern and interest in your patients A4. Quality of nursing staff

A8. Timeliness of follow-through on written orders A9. Staff’s reliability in recognizing and reporting changes in patients’ conditions B1. Turnaround for lab results D1. Ease of admitting patients G1. Medical technology and equipment available in

ICUlCCU L2. Ease of scheduling inpatient tests/therapy M5. Overall rating of the Emergency Department F1. Level of information you receive about the Strategic Plan for the facility as a whole 15. Your confidence in the Hospital Administration to carry out its duties and responsibilities J5. Responsiveness of the Hospital Administration to ideas and needs of medical staff members JlO. Communication between yourself and the Hospital Administration

Relationship with Leadership -.01

Quality of Care .84

.02

-8.5

-.07

-.05

.74

-.oo

.87

.08 -.03

.01 -.08 .05

.06 -.11 .26

.04 .09

-.04

Ease of Practice

.oo

-66

-89 .45 .82 .40 .07

-8.5

.22 -.07

.87

.08

-.02

-92

.02

-.01

.95

-.04

Vote. Boldface figures represent factor loadings for items predicted to define their respective factors.

-.03

I

17

18

I


variance. Factor intercorrelationsare shown in Table 3. All subscales exceeded the stringent .70 standard for reliable measures (see Table 3). The Cronbach‘s alpha for the entire survey was .90, confirming the instrument’s high internal consistency and reliability.

Confirmatory Factor Analysis: Sample 2 The 3-correlated-factor model provided a good fit to the data: x2 (62) = 212.998, p < .001; NFI = .966; RFI = .951; IF1 = .976; TLI = .965; CFI = .976; RMSEA = .049 (90% CI = .042, .056; p > .576). These indices converge in meeting the evaluation criteria, reflecting the magnitude of congruence between the observed and model-implied covariance structures. Figure 2 shows the standardized parameter estimates for the model.

Intercorrelations (Sample 1) Alpha

Factor

Factor 1 Factor 2 Factor 3

1. Relationship with Leadership

.92

2. Quality of Patient Care

.84

,477

3. Ease of Practice

.76

522

1 1

.571

1

Figure 2. Standardized Parameter Estimates for Hypothesized CFA Model (Sample 2)

.67

Note. Significant pathways are indicated with boldface type (p < ,001).Pathways set to 1.0 to assign a metric d o not yield p values.

Cross-Validation Analyses: Sample 3 Comparing a three-factor model in which measurement weights between Samples 2 and 3 were free to vary with a three-factor model in which measurement weights between the samples were constrained to be equal yielded a nonsignificant change in x2 (A x2 (10) = 7.314, p = .696), indicating that the measurement weights are invariant across the two samples. Comparing a three-factor model where measurement weights between the samples were constrained to be equal and structural covaria n c e ~between samples were either free to vary or constrained to be equal yielded a non(Ax2 (6) = 11.619, p = significant change in .071), indicating that the structural covariances are invariant across the two samples when measurement weights are constrained to be equal. These results provided strong evidence of cross-validation of the three-factor model across an independent sample of physicians. The measurement weights converged with their bootstrap estimates, reflecting negligible parameter-estimate bias, in further support of the three-factor model’s robustness (Table 4). Together these results provide evidence of cross-validation of an independent sample of physicians, thereby substantiating the factor structure of the Medical Staff Survey instrument illustrated in Figure 1.

x2

Discussion Prior research suggests that physician dissatisfaction has negative rippling effects for patients, healthcare organizations, and physicians. Therefore, assessment of physicians’ satisfaction within the context of where they practice seems essential to hospital qualityimprovement initiatives. The purpose of the current study was to establish initial construct validity evidence of the Medical Staff Survey to achieve that goal. We hypothesized that the structure of the survey would match the altruism and self-interest dimensions proffered by Jonsen (1983). However, results of the PCA showed that a three-factor structure-labeled Relationship with Leadership, Quality of Patient Care, and Ease of Practicebest explained the data. We were able to cross-validate these results by (1) obtaining model-confirming results from a CFA on an independent sample, (2) establishing factorstructure invariance across two independent samples, and (3) obtaining convergence of


-

Table 4. Standardized Measurement Wei hts with Bootstrap Means, Standard Errors (SE), and 90% Confidence Intervals 9CI) from 2,000 Maximum Likelihood Bootstrap Samples (Sample 3)

Measurement Weights

ML Estimate

Bootstrap Bootstrap Mean SE

-

Bootstrap Bootstrap CI Lower CI Upper Bias

90%

90%

P

Quality of Care A2 A4 A8 A9

0.780 0.821 0.725 0.838

0.780 0.820 0.726 0.839

0.036 0.019 0.031 0.018

0 0 0.001 0

0.704 0.787 0.666 0.807

0.826 0.851 0.771 0.868

.002 .001 ,002

Relationship with Leadership F7 15 J5 JlO

0.769 0.903 0.924 0.895

0.769 0.903 0.924 0.895

0.024 0.011 0.01 0.011

0 0 0 0

0.726 0.884 0.905 0.875

0.805 0.921 0.94 0.912

.001 .001 ,001 .001

Ease of Practice Bl D1 G1 L2 M5

0.679 0.702 0.662 0.723 0.593

0.679 0.702 0.661 0.722 0.593

0.037 0.031 0.031 0.031 0.037

0 0

0.611 0.645 0.614 0.662 0.529

0.736 0.748 0.717 0.768 0.65

,001 .001 ,001 .002 ,001

-0.002

0 0

.001

Note. Table Interpretation. Row 1:The loading of Item A2 (”Staff’s concern and interest in your patients”) on the factor of Quality of Care was ,780 in the CFA using Sample 3. Subsequent 2,000 bootstrap samples (simulated with Sample 3 data using replacement) yielded an average loading of ,780 (SE of those 2,000 samples was .036), converging with the original factor loading (i.e., bias equals zero). The loading of A2 has a 90%)confidence interval ranging from ,704 to ,826. Because this range does not include zero, the hypothesis that A2 regression weight is equal to zero in the population can be rejected. This information is also reflected in the p values, which show how small the confidence level must be to yield a confidence interval that would include zero. With respect to the A2 parameter, a p value of ,002 implies that the confidence interval would have to be at the 99.8% level before the lower bound value would be zero.

the factor loadings with the factor loadings from 2,000 bootstrap samples. The factors are robust across multiple independent samples and show a high degree of internal consistency. Therefore, it can be concluded that the survey items used in the present investigation sufficiently tap three facets important to physician satisfaction within the context of hospital practice. Although the results do not coincide precisely with Jonsen’s (1983) formulation of medical practice, inspection of the item loadings indicate conceptual overlaps between what was hypothesized and what was observed. Jonsen postulated that physicians enter the medical profession with altruistic motivations to care for patients. We found that they want to affiliate with a hospital that provides high-q!iality care to patients. Jonsen also postulated :hat physicians are motivated by self-interest. LVe found that they prefer to practice medicine in an organization that facilitates their ability to care for patients. Thus, ease of practice becomes a crucial factor. In order to achieve the

goals of ease of practice and provision of highquality care, physicians may rely heavily on having good relations with the hospital’s leadership. Doing so may enable them to influence decisions that ultimately affect whether their altruistic and self-interestneeds are met. The present results make clear that physician satisfaction specific to hospital practice can be parsimoniously conceptualized and measured. The brevity and context specificity of the Medical Staff Survey provides a distinct advantage over longer global measures of physician satisfaction (Williams et al., 1999). Now that the internal structure of the Medical Staff Survey has been established and confirmed, future research should examine links between scores and physicians‘ work behavior. For example, physician turnover rates within hospitals could be explored using the Medical Staff Survey. Indeed, other investigators have found evidence indicating that physician dissatisfaction has been linked to leaving an outpatient practice (Buchbinder, Wilson, Melick, & Powe, 2001). In addition, the present measure should

I

19

20

I


be used to examine physician satisfaction with hospital practice and its relations to the quality of care patients receive and hospital efficiency. Survey users may obtain usable data b o u g h exploration of relationships among the three facets Of physician and various patient and hospital 0Ut~C)nIes (e.& patient Satisfadon and hospital financia] performance). Often spend sigruficant amounts of time and money measuring and working to improve patient and employee satisfaction, but he fact that their oMTn physicians are key p o n a b in achieving positive outcomes sought within the organkation is sometimesoverlooked. to impmve patient and tied Outcomes that do not take into account the medical staffs satisfaction with and influence over such issues may be without value. Formally measuring physician satisfaction is important for a variety of reasons. The act alone indicates that the organization has recognized the importance of physician satisfaction and is willing to commit time and resources to the process. Furthermore, the organization is displaying a commitment to improving its relationship with its physicians. By and the fattors that contribute to physician satisfaction among their medical staff, hospitals can then take ’ystematic actions to improve any ’pecific issues that are identified as a priority to their medical staff. The items on the Medical Staff not just for monitoring Survey are faction levels but also for formulating quality improvement actions. The survey process also provides physicians a forum for submitting feedback and provides the organization with an assessment of the true state of affairsfrom all groups the medical staff,not just the physicians they may hear from or see every day. Measuring physician satisfaction with their medim can provide cal staff’s “loyalty factor”--the percentage of physicians who are loyal to the organization. The development of a valid and reliable physician satisfaction survey is an important step in the ongoing process of improving the quality of care and adequate levels of a physician workforce.

Acknowledgments

The authors thank three anonymous reviewers for their thoughtful feedback on an earlier version of this article.

References Alreck, P. L., 81 Settle, R. B. (1995).The sirrvey research handbook: Guidelines and strntc