Taking the PACIC back to basics: the structure of the

Journal of Evaluation in Clinical Practice ISSN 1356-1294

Taking the PACIC back to basics: the structure of the Patient Assessment of Chronic Illness Care jep_1568

1..6

John Spicer PhD,1 Claire Budge PhD2 and Jenny Carryer RGON PhD FCNA (NZ) MNZM3 1

Associate Professor, School of Psychology, Massey University, Palmerston North, New Zealand Research Associate, MidCentral DHB, Palmerston North, New Zealand 3 Professor of Nursing, MidCentral DHB & Massey University, Palmerston North, New Zealand 2

Keywords chronic care model, factorial validity, PACIC, reflective and formative measures Correspondence Professor John Spicer School of Psychology Massey University PB 11-222 Palmerston North New Zealand E-mail: [email protected] Research was carried out at MidCentral DHB. Accepted for publication: 10 August 2010 doi:10.1111/j.1365-2753.2010.01568.x

Abstract Rationale, aims and objectives The Patient Assessment of Chronic Illness Care (PACIC) is a widely used 20-item measure consisting of five subscales. Published factor analyses of PACIC scores have produced conflicting results on the measure’s factorial validity, and therefore some confusion as to the utility of its subscales. We aim to reduce this confusion by reviewing the evidence on the PACIC’s factorial validity, exploring the statistical issues it raises, and considering more broadly what such analyses can reveal about the validity of the PACIC. Methods To achieve these aims we review six published studies on the PACIC’s factorial validity, present confirmatory factor analyses of our own PACIC data from 251 chronic care patients, and assess the PACIC with respect to its status as a reflective or a formative measure. Results Our statistical analyses support the view that a 5-factor model does not fit the structure of the PACIC, and highlight a variety of technical issues that confront researchers who wish to factor analyse the measure. However, we argue that, as the PACIC is more accurately seen as a formative measure, such analyses do not provide information that should be used to assess the PACIC’s validity. Conclusions We conclude that, while it is important to continue examining the reliability and validity of the PACIC in a variety of ways, traditional analyses of its factorial validity (and internal consistency) are inappropriate. Meanwhile, use of the subscales is defensible as long as they continue to meet other types of reliability and validity requirements.

Introduction The Patient Assessment of Chronic Illness Care (PACIC) is a 20-item, self-report measure devised by Russell Glasgow and his colleagues [1]. As the name indicates, the PACIC offers patients with chronic illnesses the opportunity to assess the extent to which they are receiving appropriate care. Appropriateness in this context is defined with reference to the Chronic Care Model, which stresses the need for chronic care to be planned, proactive, population-based and patient-centred [2]. The 20 PACIC items are divided into five subscales: Patient Activation (three items), Delivery System Design/Decision Support (three items), Goal Setting/ Tailoring (5 items), Problem-Solving Contextual (four items), and Follow-up/Coordination (five items). These groupings are based on conceptual categories derived from the Chronic Care Model, and from the parallel Assessment of Chronic Illness Care measure (ACIC), which allows health care providers to assess the care they are providing [3]. Over the last 5 years, various research groups have used factor analyses of PACIC data from a variety of patient groups to examine the question of whether the pattern of correlations among

© 2010 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice

the items can be decomposed into five factors that map onto the five conceptual categories. This is a highly abstract analytic exercise, but one which has practical consequences for would-be users of the PACIC. Within the traditional psychometric framework, the subscales of a measure are judged acceptable if the items within each subscale are well correlated with each other but less so with items in other subscales, and if their content reflects the content of the appropriate sub-construct, such as Patient Activation. This type of evidence of factorial validity is a major part of the justification for forming and interpreting subscale scores. It also provides a rationale for evaluating the internal consistency reliability of subscales using indices such as Cronbach’s alpha. In this article, we address four issues bearing on the PACIC’s psychometric attributes. First, we summarize the conclusions that researchers have reached about the factorial validity of the PACIC. Second, we provide factor analyses of our own PACIC data to highlight the concerns that these researchers recommend should be addressed in future assessments of the measure. Third, we argue that this research agenda is misconceived, and propose an alternative perspective on the structure of the PACIC. Fourth, we explore the practical implications of this alternative perspective for 1

Taking the PACIC back to basics

would-be users of the instrument. We conclude that, when it is viewed from this perspective, the PACIC is already a useful instrument that need not and should not be evaluated in traditional terms of factorial validity and internal consistency. More generally, we suggest that the alternative perspective we recommend may well also apply to other health-related measures.

Existing evidence on the structure of the PACIC The six published studies which we have located have examined the structure of the PACIC using either exploratory factor analysis (EFA) or confirmatory factor analysis (CFA), or both in one case. In EFA, the factors emerge from the data, whereas in CFA, factors and their contents are pre-specified, and this structure is tested as a set of formal hypotheses [4]. For present purposes it is also noteworthy that CFA, but not EFA, provides a number of indices that quantify how well a hypothesized factor structure fits the data. Both techniques, but especially CFA, raise a bewildering array of technical issues [5]. Throughout this article we attempt to minimize technical details in order to highlight the key conceptual issues and their practical implications. Rosemann et al. [6] conducted EFA on data obtained from 236 osteoarthritic patients using a version of the PACIC that was translated into German and adjusted in minor ways to reflect the local patient-care context. Little detail is provided on the analytic strategy or on the results, but the authors conclude unequivocally that the PACIC exhibited a 5-factor structure that matched the five categories represented by the subscales. Aragones et al. [7] used EFA (although they erroneously refer to it as CFA) to analyse data from 100 diabetic patients obtained with a Spanish translation of the measure. They also interpret their results as supporting an appropriate 5-factor structure. Wensing et al. [8] conducted EFA on data obtained from 114 patients with either diabetes or chronic obstructive pulmonary disease, using a Dutch translation of the PACIC. The statement in their Abstract that their analysis ‘identified the previously defined five domains reasonably well’, paints a rather rosy picture of their results. In fact the EFA results presented in their table 3 show that none of the factors correspond very well to the subscale domains. Turning to the CFA studies, Glasgow et al. [1] analysed PACIC data provided by 255 adults with one or more chronic illnesses who were being treated in a large-managed care organization. They found some evidence for the hypothesized 5-factor structure, but reported only a moderate overall fit to the data and substantial correlations among the factors (ranging between 0.49 and 0.80). Accordingly they concluded that ‘they are most confident recommending use of the entire PACIC and the total score to represent CCM congruent care’. As part of the Canadian Survey of Experiences with Primary Health Care, McIntosh [9] analysed PACIC data from nearly 1000 participants with chronic conditions. In these analyses, he used data on all but the first three items (the Patient Activation subscale), and accordingly tested a 4-factor model corresponding to the remaining four subscales. This model did not fit the data well, and further analyses led him to the conclusion that the 4-category PACIC contains two factors which he labelled Whole Person Care and Coordination of Care. Subsequently, he further concluded that these two factors could reasonably be aggregated. Finally, Gugiu et al. [10] conducted EFA and 2

J. Spicer et al.

CFA on data from 529 diabetic patients. These analyses failed to support a 5-factor model (or models with two or three factors), and led the authors to accept a 1-factor model. In a related article [11], this research group also report EFA and CFA analyses which they used to develop a brief 11-item, single factor version of the PACIC. In summary, the published results on the factorial validity of the PACIC present a confusing picture for the would-be user of the subscales. Only two of the six studies report unequivocal support for a 5-factor structure that maps onto the subscale domains, and the remainder suggest different ways of proceeding, though with a tendency to gravitate towards a single factor interpretation. Given the relatively small number of studies, it is not surprising that many of the authors call for more psychometric research on the PACIC. In the following section, we explore the sorts of issues that are recommended for inclusion in such a research agenda, using some of our own PACIC data as a vehicle.

Issues raised by factor analyses of the PACIC As we noted earlier, discussions of factor analysis can quickly become mired in swamps of technical complexity. In this section, we highlight issues raised by factor analyses of the PACIC as far as possible in conceptual rather than in statistical terms, and use our own data set in order to make the issues more concrete. These PACIC data were obtained from patients who had attended ambulatory chronic illness clinics during the previous 18 months in the central North Island of New Zealand. Patients were asked to use the PACIC to assess the care provided by a salient member of their health care team rather than by the team as a whole. The data used in the following analyses were collected from the 307 patients who chose to assess their general practitioner. This group comprised 44% female patients, aged 23–93 years (mean = 68.4), and suffered from diabetes (39.5%), cardiac problems (64.1%), respiratory problems (28%), and pain (28.6%). Nearly two-thirds of the sample was being treated for multiple conditions. Any patient who did not complete all of the PACIC items was excluded, resulting in a sample size for analysis of 251. The PACIC administered to these patients was in the original form provided by Glasgow et al. [1], except that the 6-month time frame was extended to 12 months to allow patients to base their responses on a more extended period of care. The main analysis reported here is an attempt to replicate Glasgow et al.’s [1] CFA of a 5-factor model of the PACIC data, which allowed for correlations among the factors. These authors do not report the specific technique they adopted to estimate parameters, but it seems fairly safe to assume that it was the generally used maximum likelihood estimation (MLE) strategy. Accordingly we tested a 5-correlated factor model using MLE with the AMOS 17 package [12]. To test the overall fit of this model to their data, Glasgow et al. used two indices: the non-normed fit index and the comparative fit index, and reported values of 0.87 and 0.89, respectively. The fit index values for our analyses were strikingly similar: 0.84 and 0.87, respectively. Glasgow et al. [1] describe this level of fit as ‘moderate’, but recent work on fit thresholds for these indices suggests that fit becomes clearly acceptable at 0.95, and values in the range 0.90–0.95 are marginal [5]. It appears then that the overall fit of the 5-factor model is not acceptable in either analysis. Overall fit is a relatively crude approach to model evaluation, so a more detailed analysis of the specific parameter values in the

© 2010 Blackwell Publishing Ltd

J. Spicer et al.

model is important to reveal specific areas of fit and misfit. Of particular interest are the magnitude and statistical significance of the factor loadings that quantify the relationships between each item score and the factor that represents its subscale domain. Again, our results tend to parallel those of Glasgow et al. In both analyses all 20 loadings were statistically significant (P < 0.001); about half of the standardized loadings had values of at least 0.80; and none fell below about 0.50. The remaining notable result reported by Glasgow et al. [1] from their CFA is that the five factors were correlated in the range 0.49–0.80 with a median r of 0.65. The corresponding range in our results was 0.56–0.93 with a median r of 0.73. On the basis of these results it appears that the 5-factor model is reasonably accurate in the sense that items load on the factors that represent their subscales, and that the correlated factors confirm the expectation that subscale scores are likely to be correlated. Turning to areas of misfit, two issues typically deserve attention: the possibilities of cross-loadings and of correlated errors. The model under scrutiny here is specified in such a way that a given item is allowed to load on the factor that corresponds to the item’s subscale, but is constrained to have a zero loading on any other factor. One way to explore the possibility that items cross-load on inappropriate factors is to examine modification indices that highlight particular ways in which a model might be mis-specified. In broad terms, modification indices estimate what the value of a parameter such as a loading would be if the zero constraint was removed [5]. This type of analysis is not reported by Glasgow et al. [1], but our results suggested that items 16 and 17 were problematic in this respect. Item 16 (I was contacted after a visit to see how things were going) loaded not only on the appropriate Follow-up/Coordination factor, but also on the Goal setting/ Tailoring factor. Item 17 (I was encouraged to attend programmes in the community that could help me) loaded not only on the appropriate Follow-up/Coordination factor, but also on the Problem-solving/Contextual, Goal-setting/Tailoring and the Delivery system design/Decision support factors. A second aspect of the present specification that can be evaluated with modification indices is the constraint that the measurement errors for the items are totally uncorrelated. This implies that any correlations among the items of a subscale are caused entirely by the underlying factor for that subscale. In practice, there are a number of ways in which this assumption may be undermined, most notably by similarities in the way in which the items are measured, that is, method biases or shared method variance [13]. Inspection of the modification indices for error correlations in our analysis suggested that the error correlations between items 4 and 9, 5 and 12, 10 and 17, and 16 and 17 were highly unlikely to be zero. Resonances across the content of the first three item pairs immediately suggest the possibility that they will be correlated despite representing different factors. Thus, the content of item 4 (I was given a written list of things I should do to improve my health) resonates with the content of item 9 (I was given a copy of my treatment plan) in referring to documentation received. Item 5 (I was satisfied my care was well organized) resonates with item 12 (I was sure that the health professional thought about my values and my traditions . . .) in referring to attitudes rather than reported behaviours. Item 10 (I was encouraged to go to a specific group or class to help me cope with my chronic illness) clearly resonates with item 17 (I was encouraged to attend programmes in the



community that could help me) in terms of content. Our results on cross-loadings and correlated errors suggest the need for more careful specification of CFA models of the PACIC, which in turn could lead to improved fit of the models overall. Further critical issues about PACIC factor analyses have been raised by McIntosh [9] and by Gugiu et al. [10] concerning the assumptions that PACIC scores lie on an interval as opposed to an ordinal scale, and that the item score distributions are multivariate normal. These authors rightly point out that both assumptions are required for accurate MLE estimation of CFA models, and argue that neither can be justified. Accordingly, they both adopted an alternative strategy of analysing polychoric correlations using robust weighted least squares as ways of dealing with ordinal data and non-normality, respectively. Given our primary objective in the present article, it is not appropriate to pursue these complex issues here. Instead, two general comments are in order. First, parallel analyses of a data set conducted under different scaling and distributional assumptions do not necessarily produce different results, especially when large samples are available. These authors did not report parallel analyses under different assumptions, but proceeded on the assumptions that their data deviated sufficiently from interval scaling and normality to require special treatment. As well as the MLE analysis of our data reported above, we also conducted a Bayesian analysis that did not require the assumption of multivariate normality and treated the scores as ordinal [12]. These parallel analyses under different assumptions produced strikingly similar results. Second, as noted earlier, the analyses provided by McIntosh [9] and by Gugiu et al. [10], still found that the 5-factor model of the PACIC was clearly untenable, leading them to argue for a simpler structure of one or two factors. This cursory review of factor analysis issues should at least give some sense of the range of concerns that future analysts of the PACIC’s factorial validity could take into account. The list is not comprehensive as it excludes, for example, concerns about how EFA analysts have chosen the number of emergent factors, and about their inappropriate strategy of forcing such factors to be uncorrelated [11]. (It is pertinent to note that the studies so criticized are the only ones to claim support for the 5-factor model.) More generally, there are further concerns about how best to deal with missing data, whether by exclusion of cases or by imputation of missing values [14]. For researchers interested in the factorial validity of the PACIC there clearly seems to be much work to be done in terms of specifying the number of factors and addressing a range of technical issues in so doing. However, we wish to argue in the following section that this research agenda is misconceived as it stands, in that it rests on an erroneous view of the type of measurement instrument that the PACIC exemplifies.

Is the PACIC a reflective or a formative measure? The discussion in this section focuses on the distinction between reflective and formative measures, and draws on arguments put forward by Edwards and Bagozzi [15] in their influential article: On the nature and direction of relationships between constructs and measures, and by Fayers and Hand [16] and Streiner [17]. A reflective measure is so called because its item scores are assumed to be caused by, or to reflect, the underlying construct. Conversely, in a formative measure, scores on the items cause or form the 3


respondent’s status with respect to the construct. This causal formulation is particularly relevant in the present context because CFA is a form of causal modelling where correct specification of the direction of causal flow is crucial. All of the PACIC analyses discussed in this article assume that variation in the subscale scores is produced by the construct that underlies that subscale, in other words that the PACIC is a reflective measure. It is this assumption of a common cause that leads to the expectation that PACIC scores will exhibit patterns of co-variation that reflect the subscale structure, which in turn reflects the underlying construct structure. Most psychological measures have been assumed to be reflective, and accordingly their factorial validity and internal consistency have been assessed in conventional ways, as exemplified above. But is the PACIC actually a reflective measure? To address this question it is easier to focus on an illustrative PACIC subscale, such as Patient Activation, than on the whole measure. The definition of this sub-construct is ‘Actions that solicit patient input and involvement in decision-making’ [1] and the relevant PACIC items are: (1) I was asked for my ideas when we made a treatment plan; (2) I was given choices about treatment to think about; and (3) I was asked to talk about any problems with my medicines or their effects. We find it hard to view patient activation as a pre-existing common cause that somehow generates the patient’s responses on these items. Instead, it seems more accurate to say that the patient’s activation status emerges or is formed as a result of their responses. It is true that of necessity their experiences predate their responses. But these are the particular experiences described in the items rather than some underlying state or structure. We accept that this is not an easy question to resolve in this abstract way, so a more concrete approach may provide further support for our contention that the PACIC is a formative rather than a reflective measure. This approach starts with the general question: why would we expect the activation subscale items to be correlated at all? It seems perfectly consistent and credible that patients could give any combination of responses to these three items depending on their particular experiences. Pushing this idea further leads to a fundamental confusion between patients’ care experiences and the quality of the instrument used to assess them. Patients who report the presence of all of these experiences will enhance the correlations among the scores, as will patients who experience none of them. But patients who report only some of the experiences will dilute the correlations among the item scores. Consequently, measures of patients receiving consistently good or consistently poor care will appear to be performing well, whereas measures of those whose care is more ‘spotty’ will appear to be performing poorly. The key point to note is that this confusion of the quality of care with the quality of the measure only appears if the measure is taken to be reflective of some common cause. From a formative perspective, the patient’s activation status emerges from their pattern of responses, and there is no expectation that the relevant items are consistently correlated. If it is true that the PACIC is a formative rather than a reflective measure, what are the implications for assessing its quality? In the present context the main implication is that it is inappropriate to continue factor analytic assessments that specify PACIC item scores as consequences of underlying constructs, as is conventionally done. It is also inappropriate to assess the internal consistency of the PACIC scale or its subscales with correlation-based statis4

J. Spicer et al.

tics such as Cronbach’s alpha. Alphas may look reassuring but they reflect a ragbag of correlations representing a variety of processes rather than an underlying construct. Any approach to evaluating measures that rests on the requirement that the item scores should be correlated in patterns reflective of underlying constructs is misconceived if the PACIC is a formative measure. (These arguments apply of course to the PACIC measure as a whole and to all of its subscales, not just to the Patient Activation subscale.) The consequences of ignoring these implications are twofold. First, assessments of the PACIC’s factorial validity and reliability are likely to be erroneous and misleading. Second, as Law and Wong [18] have demonstrated, treating a formative measure as if it were reflective can distort estimates of relationships between the target construct and other constructs when structural equation models are used. If the PACIC is a formative measure, it seems that a logical strategy would be to continue CFA assessments of its factorial validity with models that correctly specified the PACIC items as causes of the underlying constructs. Unfortunately, as Brown [5] has pointed out, a measurement model that consists only of formative relationships is not identified, that is, there is insufficient information for the model to be tested. There are ways to address this problem by introducing additional reflective measures and additional constructs that are related to the target construct. But, given the attendant technical complexities and need for strong arguments to support the introduction of more measures and constructs, these strategies are not for the faint-hearted. In the following section, we adopt a more positive tone and suggest that the PACIC is nonetheless a valuable instrument whose qualities can be assessed in other ways.

How should the PACIC be evaluated? Recognizing the PACIC as a formative rather than a reflective measure does not lessen the need to ensure its reliability and validity. Two psychometric criteria – internal consistency reliability and factorial validity – become inappropriate, but other conventional criteria remain. The PACIC’s test–retest reliability is of interest, and evidence reported in four of the studies reviewed earlier provides an encouraging picture of the consistency over time of the full-scale and the subscales. Content validity remains a key issue, and again the extensive item development work put in by Glasgow and his colleagues, with the assistance of a national pool of experts, is testimony to the relationship between the items and the chronic care constructs, at both the scale and subscale levels [1]. Concerns remain about the relationships between scores on the PACIC and on other relevant measures (its criterion or construct validity), and here the evidence is less reassuring [10]. Finally, translation validity continues to receive attention with the PACIC’s translation into German, Dutch and Spanish. It is important that the reliability and validity of the PACIC continue to be evaluated in appropriate ways. These include consideration of ways in which any aspect of the measure might reduce its reliability and validity [19]. Thus, items may need to be excluded or modified to make them congruent with the local context. Further, the original response scale may need to be modified to suit the patient population and the use to which the PACIC is being put. In the original PACIC, patients are asked to indicate


J. Spicer et al.

how often something happened during the last 6 months on a 5-point scale ranging from ‘almost never’ to ‘almost always’. Gugiu et al. [10] argued that this scale is ordinal and attempted to replace it with an 11-point percentage scale ranging from 0% to 100%. Patients were asked ‘Over the past 6 months, when I received care for my chronic conditions, what percentage of the time was I . . .’ But, in the event, this scale polarized patients’ responses and the researchers were obliged to treat it as a 3-point ordinal scale. They suggested that this outcome may have been due to the 6 months timeframe, but it is also possible that the percentage scale was not easily used by the patients in their sample. The most appropriate scale and instructions are not necessarily the most complex and sophisticated. Often a simple dichotomy – whether an event occurred or not in a specified time period – may produce data that are sufficient for the chosen purpose. More generally, psychometric evaluations of the PACIC should also take account of the use to which it is being put. It may be used to assess individuals or groups at one or more time points for clinical purposes, or it may be used for research purposes. For example, if the purpose is to assess change over time in a clinical context, any measure should not only be reliable and valid, but should also be sensitive to change [20]. To our knowledge, this issue has not yet been addressed for the PACIC. In a research context, there is nothing in the arguments we have made that might discourage researchers from analysing PACIC scale and subscale scores on their own or in relation to other scores, as long as the statistics used are suitable for the scaling properties and frequency distributions of those scores. The reliability and validity of the scores remain as important prerequisites for sound analyses; only the internal consistency and factorial validity criteria disappear from the picture. The exception is where a decision is made to use structural equation modelling, when specifying the correct measurement model for the PACIC becomes critical.

Conclusion In summary, we suggest that the PACIC is a very useful instrument that deserves wide usage. As a recently developed measure, its psychometric qualities should continue to be investigated. However, if our argument that it is a formative rather than a reflective measure is accepted, these investigations should not include assessments of its internal consistency and factorial validity, at least in the ways that have been adopted to this point. More generally, we suggest that the issue of formative versus reflective measures is one that is worthy of wider consideration in the health field. It receives a passing mention in a recent review of quality criteria for health status measures [21], but we have been unable to find more extensive discussions or applications. Podsakoff et al. [13] concluded that in the area of occupational psychology: ‘Researchers . . . are now beginning to recognize that many of the most widely used constructs in the field (e.g. job satisfaction, role ambiguity, role conflict, task characteristics) are more accurately represented as formative indicator constructs than they are as reflective-indicator constructs’. Given the many links between psychological and health constructs and their measures, it is possible that a similar conclusion might well result from conceptual analyses of a range of health measures.



References 1. Glasgow, R. E., Wagner, E. H., Schaefer, J., Mahoney, L. D., Reid, R. J. & Greene, S. M. (2005) Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Medical Care, 43 (5), 436–444. 2. Wagner, E. H. (1998) Chronic disease management: what will it take to improve care for chronic illness? Effective Clinical Practice, 1 (1), 2–4. 3. Bonomi, A. E., Glasgow, R. E., Wagner, E. H. & Von Korf, M. (2001) Assessment of chronic illness care (ACIC): a practical tool for quality improvement. Health Services Research, 37, 791–820. 4. Spicer, J. (2005) Making Sense of Multivariate Data Analysis. pp. 42–55. Thousand Oaks, CA: Sage. 5. Brown, T. A. (2006) Confirmatory Factor Analysis for Applied Research. New York: The Guilford Press. 6. Rosemann, T., Laux, G., Droesemeyer, S., Gensichen, J. & Szencenyi, J. (2007) Evaluation of a culturally adapted German version of the Patient Assessment of Chronic Illness Care (PACIC 5A) questionnaires in a sample of osteoarthritis patients. Journal of Evaluation in Clinical Practice, 13, 806–813. 7. Aragones, A., Schaefer, E. W., Stevens, D., Gourevitch, M. N., Glasgow, R. E. & Shah, N. R. (2008) Validation of the Spanish translation of the Patient Assessment of Chronic Illness Care (PACIC) survey. Preventing Chronic Disease, 5 (4), A113. Available at: http:// www.ncbi.nlm.nih.gov/pmc/articles/PMC2578783/?tool=pubmed (last accessed 30 July 2010). 8. Wensing, M., Van Lieshout, J., Jung, H. P., Hermsen, J. & Rosemann, T. (2008) The Patients Assessment of Chronic Illness Care (PACIC) questionnaire in The Netherlands: a validation study in rural general practice. BMC Health Services Research, 8, 182. Available at: http://www.biomedcentral.com/content/pdf/1472-6963-8-182.pdf (last accessed 20 July 2010). 9. McIntosh, C. N. (2008) Examining the Factorial Validity of Selected Modules from the Canadian Survey of Experiences with Primary Health Care. Ottawa, ON: Health Information and Research Division, Statistics Canada. ISBN 978-0-662-48102-7. 10. Gugiu, P. C., Coryn, C. L. S. & Applegate, E. B. (2010) Structure and measurement properties of the Patient Assessment of Chronic Illness Care instrument. Journal of Evaluation in Clinical Practice, 16 (3), 509–516. 11. Gugiu, P. C., Coryn, C., Clark, R. & Kuehn, A. (2009) Development and evaluation of the short version of the Patient Assessment of Chronic Illness Care instrument. Chronic Illness, 5 (4), 268– 276. 12. Arbuckle, J. L. (2008) Amos 17.0 User’s Guide. Crawfordville, FL: Amos Development Corporation. 13. Podsakoff, P. M., MacKenzie, S. M., Lee, J. & Podsakoff, N. P. (2003) Common method variance in behavioral research: a critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903. 14. McKnight, P. E., McKnight, K. M., Sidani, S. & Figueredo, A. J. (2007) Missing Data: A Gentle Introduction. New York: The Guilford Press. 15. Edwards, J. R. & Bagozzi, R. P. (2000) On the nature and direction of relationships between constructs and measures. Psychological Methods, 5 (2), 155–174. 16. Fayers, P. & Hand, D. (2002) Causal variables, indicator variables and measurement scales: an example from quality of life. Journal of the Royal Statistical Society – Series A: Statistics in Society, 165 (2), 233–253. ISSN/ISBN: 0964-1998. 17. Streiner, D. L. (2003) Being inconsistent about consistency: when coefficient alpha does and doesn’t matter. Journal of Personality Assessment, 80, 217–222.

5


18. Law, K. & Wong, C. (1999) Multidimensional constructs in structural equation analysis: an illustration using the job perception and job satisfaction constructs. Journal of Management, 25 (2), 143–160. 19. Schwarz, N. (1999) Self-reports: how the questions shape the answers. American Psychologist, 54, 93–105. 20. Testa, M. A. & Simonson, D. C. (1996) Current concepts: assessment

6

J. Spicer et al.

of quality-of-life outcomes. The New England Journal of Medicine, 334, 835–840. 21. Terwee, C. B., Bot, S. D. M., de Boer, M. R., van der Windt, D. A. W. M., Knol, D. L., Dekker, J., Bouter, L. M. & de Vet, H. C. W. (2006) Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60, 34–42.
