Exploring the conceptualization of program theories in

Health Promotion International Advance Access published August 12, 2010 Health Promotion International doi:10.1093/heapro/daq045

# The Author (2010). Published by Oxford University Press. All rights reserved. For Permissions, please email: [email protected]

JANNEKE HARTING1* and PATRICIA VAN ASSEMA2 1

Social Medicine, AMC/UvA, PO Box 22660, Amsterdam 1100 DD, The Netherlands and 2Health Promotion, Maastricht University, Maastricht, The Netherlands *Corresponding author. E-mail: [email protected]

SUMMARY Our objective was to evaluate whether the limited effectiveness of most community programs intended to prevent disease and promote health should be attributed to the quality of the conceptualization of their program theories. In a retrospective multiple case study we assessed the program theories of 16 community programs (cases) in the Netherlands (1990–2004). Methods were a document analysis, supplemented with member checks (insider information from representatives). We developed a community approach reference framework to guide us in reconstructing and evaluating the program theories. On the whole,

programs did not clearly spell out the process theories (enabling the implementation of effective interventions), the program components (interventions) and/or the impact theories (describing pathways from interventions to ultimate effects). Program theories usually turned out to be neither specific nor entirely plausible (complete and valid). The limited effectiveness of most community programs should most probably be attributed to the limited conceptualization of program theories to begin with. Such a failure generally also precludes a thorough examination of the effectiveness of the community approach as such.

Key words: community health promotion; evaluation; logic model

INTRODUCTION The development of the new public health movement was accompanied by a growing interest in the community as a setting for health promotion (Green et al., 2000). This development represents a change in emphasis from individually focused educational approaches to programs that also incorporate social and environmental influences (Richard et al., 1996; Thompson and Kinne, 1999). This shift is reflected in ecological models of health (McLeroy et al., 1988; Green et al., 1996). Such models are based on the assertion that an individual’s health status and health-related behaviors are shaped by a dynamic interaction with the physical and social environment, which includes influences at the interpersonal,

organizational, community and policy levels (McLeroy et al., 1988; Stokols, 1996). Community-wide health promotion programs, whether their orientation is community-based or based on community development (Boutilier et al., 2000), are therefore assumed to be more effective than individually focused health education strategies (Green et al., 2000). Review studies indicate, however, that with some positive exceptions, community programs to promote health or to prevent disease generally do not yield the expected results or merely lead to limited changes in health behavior and health status outcomes (Sorensen et al., 1998; Roussos and Fawcett, 2000; Wandersman and Florin, 2000; Merzel and D’Afflitti, 2003; Harting and Van Assema, 2007). A number of explanations have been offered for the lack of Page 1 of 14

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

Exploring the conceptualization of program theories in Dutch community programs: a multiple case study

Page 2 of 14

J. Harting and P. van Assema

METHODS We performed a retrospective multiple case study of program theories of community programs in the Netherlands. A case study is a qualitative social research strategy defined as an empirical inquiry that investigates a contemporary phenomenon (case) within its real-life context (Yin, 2003). A multiple case study includes several single case descriptions to enable a cross-case analysis in order to create more robust evidence than can be provided by a single case study (Yin, 2003). Consistent with the research methodology, our study consisted

of (i) developing a reference framework; (ii) selecting cases (i.e. community programs) and formulating a research protocol; (iii) generating thick-description single case reports (i.e. comprehensive accounts); (iv) performing a crosscase analysis and producing a comparative report. Reference framework The frame of reference (Figure 1) was based on a general model of program theory (Weiss, 1998; Rossi et al., 1999). This consists of five causally related elements: the distal outcomes (outcomes), the proximal outcomes (impact), the program components (interventions), and an organization and delivery process (inputs and activities). These elements and their causal linkages form a general theory expressing how a program is expected to work. Such a program theory can be divided into a process theory and an impact theory, and the latter can in turn be subdivided into an action hypothesis (linking program components to proximal outcomes) and a conceptual hypothesis (linking proximal outcomes to distal outcomes). This general model was further specified by including three hypothesized requirements for effective community programs as integral causal components. The first was a community organization model, or any other model reflecting the stages of program development, which could serve as a program planning or implementation tool (Goodman, 1998; Bracht, 1999). The second encompassed three well-known working principles, that is, citizen participation, crosssectoral collaboration and a social network approach, which could operate as building blocks for the community organization process (Davies and Macdowell, 2006). The third consisted of social science theories and intervention development methods that could support the design of effective program components at the various ecological levels of the community (Stokols, 1996; McLeroy et al., 2003). To do further justice to this ecological perspective, the program components and proximal outcomes were subdivided into four domains that are commonly distinguished: physical/social environment issues, individual/behavioral aspects, organization/policy matters and healthcare concerns (Lalonde, 1974; Goodman, 1998). This specified model served as the frame of

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

more convincing results: methodological issues, the influence of secular trends, limitations of interventions and limitations of theory (Sorensen et al., 1998; Mittelmark, 1999; Wandersman and Florin, 2000; Merzel and D’Afflitti, 2003). One of the possible limitations of theory is the absence of a wellconceptualized program theory, that is, the causal mechanism by which a program brings about the intended changes (Koepsell et al., 1992; Goodman et al., 1996; Baranowski et al., 1997; Weiss, 1997). Although program theories may offer the greatest promise to further document the effectiveness of and improvements in community approaches in health promotion and disease prevention, they have so far been the least explored (McLeroy et al., 2003). A qualitative cross-case analysis articulating and comparing such program theories may therefore offer a valuable starting point for further exploration (McLeroy et al., 2003). To investigate whether the generally limited effectiveness of community programs should be attributed to the way their program theories have been conceptualized, this article presents a multiple case study (Yin, 2003) of Dutch community and neighborhood initiatives to prevent disease and/or promote health. As an evaluation in it own right (Rossi et al., 1999), we describe the program theories underlying the initiatives and assess them in terms of their specificity and plausibility (i.e. completeness and validity). Part of the assessment procedure involves a comparison with a generic frame of reference for the community approach developed for the purpose of this study, which also enables a final cross-case analysis.

Conceptualization of program theories in Dutch community programs

reference to describe and assess the program theories of the community programs included in our study.

of information consisted of documents describing the community programs from the inside and/or the outside. All available documents for each of the included programs were identified and retrieved. The collection process started with documents with a high level of information aggregation, such as grant applications, project plans, work plans, contracts or agreements, mission statements, annual reports and research reports. If they were thought to be useful, additional documents with lower levels of aggregation were also gathered, such as informal proceedings, minutes of meetings and letters or e-mails. On the basis of the documents available, a single case report was drafted for each of the community interventions (see next section). As a secondary source of information, the document analysis was supplemented by member checks with one or more representatives who could provide insider information. During interviews, one for each of the community programs included, these representatives were invited to share their experiences with the program in which they were or had been involved and encouraged to express their opinion about the single case report that we had prepared from the document analysis. On the basis of verbatim transcriptions of the interviews, the single case reports were revised and presented again to the community program representatives for their agreement.

Fig. 1: Reference framework to assess community programs for disease prevention and health promotion, based on Rossi et al. (1999).

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

Selecting cases and formulating a research protocol We tried to identify eligible Dutch community programs by consulting various national databases and experts. Eligibility criteria were primarily based on two important principles that are required to arrive at collective conclusions in multiple case studies: replication (repetition of findings) and contrast (conflicting findings) (Yin, 2003). To permit the application of both, the selected community projects had to have enough in common as well as enough variability. Within these broad boundaries, it was decided that the programs had to (i) have started during or after 1990; (ii) describe themselves as community or neighborhood initiatives; (iii) cover a particular geographic unit (e.g. region, municipality, district or neighborhood); (iv) explicitly aim at health promotion or disease prevention and (v) be in an advanced stage of development or have been finalized by the end of 2004. An additional requirement was the availability of enough documents to enable us to study the case. As recommended in the literature, two types of data sources were combined (Seale, 1999; Lincoln and Guba, 2000). The primary source

Page 3 of 14

Page 4 of 14

J. Harting and P. van Assema

Cross-case analysis and comparative report All single case reports were entered into NVivo 2 for a further structured qualitative analysis (Gibbs, 2002). To arrive at a comparative case report, the most important aspects of each single case report were coded using a coding structure that reflected the frame of reference. The coded materials were then summarized in various comparative matrices (Miles and Huberman, 1994), depicting general information, the process theories of the programs (especially whether these reflected community organization models, working principles, social science theories and intervention development models) and the impact theories of the programs. This information was summarized in a final comparative report (Harting and Van Assema, 2007). For the purpose of this article,

we present two final comparative matrices. More detailed information is presented in Supplementary Appendix SI.

RESULTS Selection of community programs Initially, 22 eligible community programs were identified. On further consideration, eight of these were excluded, reasons for exclusion being: having a primary focus on health care, working in a setting rather than a geographic community, still being in an early stage of development or lack of available archival materials. The 16 community programs that were included are presented in the matrices by the Dutch names of their geographic location, in alphabetical order, with discriminating numbers for multiple programs at the same location. An overview of the most informative materials is presented in Supplementary Appendix SII.

General characteristics The community programs originated from various parts of the Netherlands. They covered a whole range of ‘communities’, from small subgroups in a neighborhood to the entire population of a region. The programs, including any preparatory and/or continuation periods, took or intended to take between 30 months and 11 years. The available budgets were difficult to ascertain, for instance because of different but interwoven flows of money or insufficient bookkeeping, but most programs were rather underfunded given their ambitions (e.g. a 2-year program budget of E30 000 to improve lifestyle and environmental factors in a deprived neighborhood). A similar shortage of resources also characterized the staffing of most programs (e.g. a 0.1 FTE project leader position to achieve substantial changes in local policies). Nine programs explicitly aimed to reduce health inequalities and one aimed at health promotion in general (Matrix 1). Seven of these programs adopted a community development approach, indicating a bottom-up working procedure, whereas two of them used a combined community-based and community development approach. Six programs aimed at disease prevention, five of which also adopted a combined

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

Single case reports The single case reports summarized some general characteristics of the program, such as its name, geographic area, number of inhabitants, duration, resources, goals or mission and the intended community approach. This was supplemented with a reconstruction of the program theory in terms of the reference framework (see also Matrices 1 and 2). With regard to the process theory, we qualitatively assessed the application of the various models, principles and theories. As regards the social science theories and intervention development methods, four different levels of application were distinguished (Painter et al., 2008): not applied, merely mentioned, informed by/partially applied and specified/actually applied. With regard to the application of working principles (aims, elaboration, methods and strategies), we used a similar categorization: not, limitedly, fairly or extensively. Regarding the impact theory, we assessed the specificity and plausibility of the action hypothesis and conceptual hypothesis, as well as those of the impact theory as a whole. Relevant questions were whether the program’s goals and objectives were welldefined and feasible, and whether the cause-and-effect chain was complete and valid (Rossi et al., 1999). All assessments were informed by the social science background of both authors as well as by their empirical knowledge of health education and health promotion strategies and their effectiveness.

Conceptualization of program theories in Dutch community programs

Page 5 of 14

Matrix 1: Community programs: general information and process theories

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

Continued

J. Harting and P. van Assema Page 6 of 14

Matrix 1: Continued

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

Continued

Conceptualization of program theories in Dutch community programs

Page 7 of 14

Matrix 1: Continued

b

approach. The sixth disease prevention program explicitly stated that it used a top-down approach. All but one of the programs intended to reach their ultimate goals through changes in lifestyle. Seven of these also intended to address the social or physical environment. Six programs explicitly aimed at changes at the organizational or policy level and one at modifications of the healthcare system.

Organization and delivery process Community organization All programs claimed to apply a particular community organization, planning or implementation model (Matrix 1). These models typically involved the stages of community analysis (collection of information), initiation (creating commitment and designing the program),

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

Program mission: , decrease; , increase; CHD, coronary heart disease; HI health inequalities. Community approach of program: , community-based (top-down) rather than community development , community development rather than community-based; , mixed community approach (bottom-up); combining community development and community-based elements. c Community organization models: McG, McGuire’s communication-persuasion matrix; COM, community organization model by Bracht and Rissel; HC, healthy cities model; PP, precede-proceed planning model; PD, policy development model; #, other models not clearly specified. d Aspects of working principles: A, defined aims and expected results; S/A, further substantiation and argumentation of principle of program (or its role); M/S, methods and strategies to be used. e Application of aspects of working principles: empty cell means not; limited: aspect of principle merely mentioned/ described to limited extent; fairly: aspect of principle moderately described/partially applied; extensively: aspect of principle extensively described and actually applied; circles in gray indicate that the application of the principle was not explicitly intended. f Levels at which theories or intervention development methods were applied: Ph/S E, physical and/or social environment level (e.g. social network theory or environmental determinants); I/B, level of individual and/or behavior (e.g. theory of planned behavior, social learning theory, and health belief model); O/P, organizational and/or policy level (e.g. policy development theories); Com, community level (e.g. socio-ecological theory). g Application of social science theories and intervention development methods: empty cell means not applied: no theory or model mentioned at this level; merely mentioned: theory or model only mentioned or applied to limited extent; informed by: theory or model fairly well-specified and partially applied; applied: theory or model extensively specified and actually applied; squares in gray indicate the application of theory or model in retrospect; examples of intervention development methods: model of behavior change through education, intervention mapping, precede-proceed model, communication persuasion matrix, outcome mapping, action research, and a variant of methodological approach. a

Page 8 of 14

J. Harting and P. van Assema

Matrix 2: Community programs: specification and plausibility of impact theoriesa

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010 Continued

Conceptualization of program theories in Dutch community programs

Page 9 of 14

Matrix 2: Continued

Specification and plausibility: not very plausible, weakly specified; probably plausible, but modestly specified; reasonably plausible, relatively well-specified;  causal chain not further specified. b Domains: Ph/S E, physical and/or social environment issues; I/B, individual/behavioral aspects; O/P, organization and/or policy matters; HC, healthcare concerns. c Description of program components, immediate change and secondary effects: black plus symbols, output or effect described; gray plus symbols, output or effect not described but to be expected given information on previous step(s) in program theory; 3, effect described but not to be expected given information on previous step(s) in program theory. d Intermediate results and ultimate outcomes: , increase; , decrease; CHD, coronary heart disease; HI, health inequalities. e Healthcare effects intended through individual prevention program accompanying the community program.

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

a

Page 10 of 14

J. Harting and P. van Assema

Working principles Most programs intended to apply at least two of the three working principles, but on the whole their actual application remained rather incomplete (Matrix 1). Initially, all but one program intended to achieve citizen participation. Three merely mentioned this working principle, whereas the application of participation ranged from limited to fair in seven programs, and the remaining four programs addressed the entire principle more comprehensively. All but one of the programs involved cross-sectoral collaboration, though three of them applied this working principle to a limited extent only. For eight of the programs, the specification of the various aspects of this principle was on average fair, whereas four programs applied the principle of cross-sectoral collaboration more extensively. Five programs did not initially adopt the social network approach as a working principle. Two others just made mention of it, whereas six programs applied it to a limited extent only. The application of the social network approach was fair in three of the programs, whereas the remaining four applied it more extensively. Social theories None of the programs specified social science theories at all four levels of the community, and only half of them could be classified as being informed by at least one social science theory (Matrix 1). All but three programs mentioned certain theories or theoretical concepts related to individual behavioral changes. Of the three which did not, one aimed at parental support, one concentrated on policy changes and one was based on ‘trial-and-error’. Two programs had partially applied the theoretical notions to

develop their own program, whereas three other programs had applied the theories more extensively in this individual domain. Four programs mentioned theories related to changes in the social or physical environment. These theories were partially applied in one program and more extensively in two. Theories related to policy development were merely mentioned by one of the programs and partially applied in three of them (albeit in retrospect in two cases). Of the programs that referred to community or system theories, three described their approach as being system-oriented. One partially applied a socio-ecological model, but in retrospect and for evaluative purposes only. Intervention development methods Intervention development methods were mostly used to inform the programs at individual level (Matrix 1). Thirteen programs implicitly or explicitly used such a tool. In one case, the intervention development method was merely mentioned, whereas it was partially applied in nine other programs, albeit in retrospect in one case. Three programs made more extensive use of such a tool, albeit in retrospect in one case. Impact theories General findings None of the programs explicitly described its action and/or conceptual hypotheses. The impact theories presented here are thus a result of interpretations by the authors. Impact theories were easier to reconstruct if the programs had formulated less ambitious goals or had planned any kind of evaluation effort. The more specified, the easier it was to assess the level of (or lack of ) plausibility of the program theories. Specification of impact theories Most ‘action hypotheses’ focused on program components in the individual/behavioral domain and were least specific with regard to the immediate and secondary effects in the organizational/policy and healthcare domains (Matrix 2). All programs specified certain program components in the behavioral domain, and more or less specified immediate and secondary effects, like changes in behavioral determinants and lifestyle. Although less specified, 13 programs also proposed program

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

implementation (carrying out interventions), consolidation (embedding program elements or operational methods) and reorientation (evaluation and adjustment). The programs, however, differed considerably in their degree of specification of the various developmental stages. Only a few described the process of community organization in more detail, while usually devoting more attention to the early rather than to later stages. Most programs only roughly outlined the process of community organization, in part because planning models were seen as incompatible with a community development approach.

Conceptualization of program theories in Dutch community programs

Plausibility of impact theories All impact theories could be regarded as at least somewhat plausible; no entirely nonplausible linkages were found (Matrix 2). Since we considered the plausibility of the impact theory as a whole to be as strong as its weakest link, the impact theory of three programs could be regarded as plausible or possibly plausible. Two of these had gradually developed their program theory over the years, whereas the action hypothesis of one other program could be regarded as plausible or possible plausible. The same could be said for the conceptual hypothesis of seven other programs.

Cross-case analysis The working principles mostly tended to be rather isolated components of the program theories. As far as they were an integral part, they informed the community organization process rather than the process of intervention development and the specification of the impact theory. The extent to which the working principles were actually applied was not clearly related to the nature of the community approach that had been adopted by the program. A community development approach was associated with less extensive application of community organization models and with less specified and coherent interventions. More detailed specifications of immediate and secondary effects were associated with having done a determinant study or a literature review and with applying an intervention development method. Important causes of insufficient plausibility of impact theories were insufficient or vaguely described program components, or an incomplete specification of immediate changes, secondary effects and/or intermediate results, either overall or within certain domains. A higher degree of specificity, and sometimes also plausibility, of impact theories was associated with less ambitious goals and the extensive application of community organization models and social science theories.

DISCUSSION Our multiple case study of Dutch community programs for health promotion and disease prevention revealed that most program theories were poorly conceptualized. Although this finding was not unexpected (Weiss, 1997; Rossi et al., 1999; Donaldson, 2003; Merzel and D’Afflitti, 2003), our systematic analysis, which used a reference framework of community programs, further specifies the difficulties. With regard to the process theories underlying the programs, the poor conceptualization especially related to the limited application of planning models, community organization principles and social science theories. The impact theories underlying the programs suffered from vaguely described interventions with a limited ecological spectrum, incomplete specification of intended changes and ill-founded causal relationships between interventions, impacts and outcomes. These results may explain why Dutch community

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

components in the social or physical environment. Nine programs described immediate effects in this domain, like offering certain facilities or a change in social norms, and nine programs ( partly overlapping with the other nine) described secondary effects, such as a safe and healthy environment. Three of these specified secondary effects without having described program elements or immediate effects. Nine programs had planned (mostly unspecified) interventions aimed at organizational or policy change. Five described immediate as well as secondary effects, including administrative involvement in health and a healthy public policy. The other four did not specify direct effects, with two of them nevertheless describing secondary effects. Three programs intended a well-specified individual high-risk intervention in addition to the community program. Five others anticipated rather undetermined program components in the healthcare domain. Four of these specified immediate as well as secondary effects, for instance more attention to disease prevention and a more accessible healthcare system. Two programs described such effects without, however, having described interventions in this domain. Most ‘conceptual hypotheses’ were phrased in rather general terms (Matrix 2). All programs formulated one or more expected intermediate results, such as a decrease in coronary heart disease or cancer or an increase in psychological health. For six programs, the causal chain of the program theory ended there, whereas the remaining 10 programs also described ultimate results, mostly expecting a decrease in health inequalities.

Page 11 of 14

Page 12 of 14

J. Harting and P. van Assema

Although member checks were used to compensate for this, the rigidity of our analyses could have further benefited from additionally involving a panel of reviewers and a more thorough comparison of program theories with the social science literature (Rossi et al., 1999). One could, however, doubt whether the quality of most of the program theories included in the present study would have justified such substantial research efforts, especially in the absence of explicit quality criteria. A second limitation is that our analysis of the process theories underlying the programs focused on certain characteristics of community programs indicated by the literature to be essential. This is inherent to the way we conceptualized the community approach. Since we took a community organization model as a starting point and extracted three commonly identified working principles, our method can be viewed as not doing enough justice to the more comprehensive community capacity and social capital building processes that the community approach has been associated with (Kreuter and Lezin, 2002; Norton et al., 2002). A related implication is that we gave less attention to aspects like the available resources, leadership issues and community history, which are other essential ingredients of community programs. Future studies on program theories could therefore benefit from a more holistic conceptualization of the community approach. On the other hand, it is seldom possible or useful to individually appraise each distinct assumption and expectation presented in a program theory (Rossi et al., 1999). From this perspective, the somewhat reductionist reference framework that guided our work may be seen as a major strength of our study in the sense that it allowed us to compare program theories of highly divergent community programs. We conclude that the generally limited effectiveness of most community programs should be attributed to the weak conceptualization of their program theories to begin with, and that, on the whole, such a failure does not permit a thorough examination of the effectiveness of the community approach as such. SUPPLEMENTARY DATA Supplementary data are available at Health Promotion International online.

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

programs, as well as community programs in general (Sorensen et al., 1998; Mittelmark, 1999; Wandersman and Florin, 2000; Merzel and D’Afflitti, 2003), generally have little success (Rossi et al., 1999) and indicate that a first step to increase their effectiveness might be a conscientious conceptualization of program theories. In other words, more specified and plausible program theories would strengthen the evaluability of community programs (Wholey, 1987) as well as considerably increase ‘design sensitivity’ and thus the opportunity to detect favorable program findings when they do exist (Lipsey, 1993). Articulating program theories for complicated and complex community programs may however be a challenging task (Bickman, 1989; Weiss, 1997; Barnes et al., 2003; Rogers, 2008). Such programs typically consist of a range of components at various levels and include not just linear but also recursive relationships. It may therefore be useful to design an umbrella program theory (e.g. starting from an ecological perspective on health) as well as to articulate various subtheories (which further specify the various components and explain their specific relationships; McLaughlin and Jordan, 1999). The literature furthermore provides some useful suggestions to strengthen the conceptualization of program theories, including the requirement for community initiatives to include program theories as part of their grant proposal (Kaplan and Garrett, 2005) and manuals on how to facilitate their construction (Moyer et al., 1997; Gugiu and Rodrı´guez-Campos, 2007). A useful procedure may, for instance, be a form of ‘backward mapping’, which starts with a set of welldefined program objectives and successively unravels all means –ends relations to ensure that the program theory encompasses all the necessary activities, events and resources (Rossi et al., 1999; Abbema et al., 2004). Before starting such a procedure, it may however be necessary to overcome a commonly observed resistance against working with program theories (Donaldson et al., 2002). One possible way to bridge the gap between theory and practice is by intensifying the collaboration between researchers and practitioners, for instance by creating ‘technical assistance systems’ (Wandersman and Florin, 2003). One limitation of the present study is its heavy reliance on ‘logical analysis and armchair reviews’ by both authors (Rossi et al., 1999).

Conceptualization of program theories in Dutch community programs

ACKNOWLEDGEMENTS Thanks are due to all representatives of community programs who contributed to the collection of documents and who were so kind as to take part in the member checks. We would like to express our gratitude to Irene van Valkengoed and Karien Stronks for their helpful comments on previous drafts of this manuscript.

The study on community programs in the Netherlands was funded by the Dutch Organization for Health Research and Development (ZonMw; 4010.0031). REFERENCES Abbema, E. A., Van Assema, P., Kok, G. J., De Leeuw, E. and De Vries, N. K. (2004) Effect evaluation of a comprehensive community intervention aimed at reducing socioeconomic health inequalities in the Netherlands. Health Promotion International, 19, 141–156. Baranowski, T., Lin, L. S., Wetter, D. W., Resnicow, K. and Hearn, M. D. (1997) Theory as mediating variables: why aren’t community interventions working as desired? Annals of Epidemiology, S7, 89–95. Barnes, M., Matka, E. and Sullivan, H. (2003) Evidence, understanding and complexity. Evaluating in non-linear systems. Evaluation, 9, 265– 284. Bickman, L. (1989) Barriers to the use of program theory. Evaluation and Program Planning, 12, 387 –390. Boutilier, M., Cleverly, S. and Labonte, R. (2000) Community as a setting for health promotion. In Poland, B. D., Green, L. W. and Rootman, I. (eds), Settings for Health Promotion. Linking Theory and Practice. Sage, Thousand Oaks, pp. 250–279. Bracht, N. (1999) Health Promotion at the Community Level 2—New Advances. Sage, Thousand Oaks. Davies, M. and Macdowell, W. (2006) Health Promotion Theory. Open University Press, Maidenhead. Donaldson, S. I. (2003) Theory-driven program evaluation in the new millennium. In Donaldson, S. I. and Scriven, M. (eds), Evaluating Social Programs and Problems. Lauwrence Erlbaum, Mahwah, pp. 109–141. Donaldson, S. I., Gooler, L. E. and Scriven, M. (2002) Strategies for managing evaluation anxiety: toward a psychology of program evaluation. American Journal of Evaluation, 23, 261– 273. Gibbs, G. R. (2002) Qualitative Data Analysis. Explorations with NVivo. Open University Press, Maidenhead. Goodman, R. M. (1998) Principles and tools for evaluating community-based prevention and health promotion programs. Journal of Public Health Management Practice, 4, 37–47. Goodman, R. M., Wandersman, A., Chinman, M., Imm, P. and Morrissey, E. (1996) An ecological assessment of

community-based interventions for prevention and health promotion: approaches to measuring community coalitions. American Journal of Community Psychology, 24, 33– 61. Green, L. W., Richard, L. and Potvin, L. (1996) Ecological foundations of health promotion. American Journal of Health Promotion, 10, 270–281. Green, L. W., Poland, B. D. and Rootman, I. (2000) The settings approach to health promotion. In Poland, B. D., Green, L. W. and Rootman, I. (eds), Settings for Health Promotion. Linking Theory and Practice. Sage, Thousand Oaks, pp. 1– 43. Gugiu, P. C. and Rodrı´guez-Campos, L. (2007) Semi-structured interview protocol for constructing logic models. Evaluation and Program Planning, 30, 339– 350. Harting, J. and Van Assema, P. (2007) Communityprojecten in Nederland: de eeuwige belofte? [Community Projects in the Netherlands: Forever Promising?]. Maastricht University, Maastricht. Kaplan, S. A. and Garrett, K. E. (2005) The use of logic models by community-based initiatives. Evaluation and Program Planning, 28, 167–172. Koepsell, T. D., Wagner, E. H., Cheadle, A., Patrick, D., Martin, D., Diehr, P. H. et al. (1992) Selected methodological issues in evaluating community-based health promotion and disease prevention programs. Annual Review of Public Health, 13, 31–57. Kreuter, M. W. and Lezin, N. (2002) Social capital theory. Implications for community-based health promotion. In DiClemente, R. J., Crosby, R. A. and Kegler, M. C. (eds), Emerging Theories in Health Promotion Practice and Research. Strategies for Improving Public Health. Jossey-Bass, San Francisco. Lalonde, M. (1974) A New Perspective on the Health of Canadians. A Working Document. Government of Canada, Ottawa. Lincoln, Y. S. and Guba, E. G. (2000) Paradigmatic controversies, contradictions, and emerging confluences. In Denzin, N. K. and Lincoln, Y. S. (eds), The Handbook of Qualitative Research. Sage, Thousand Oaks, pp. 163– 188. Lipsey, M. W. (1993) Theory as method: small theories of treatments. New Directions for Program Evaluation, 57, 5 –38. McLaughlin, J. A. and Jordan, G. B. (1999) Logic models: a tool for telling your program’s performance story. Evaluation and Program Planning, 22, 65–72. McLeroy, K. R., Bibeau, D., Steckler, A. and Glanz, K. (1988) An ecological perspective on health promotion programs. Health Education Quarterly, 15, 351–377. McLeroy, K. R., Norton, B. L., Kegler, M. C., Burdine, J. N. and Sumaya, C. V. (2003) Community-based interventions. American Journal of Public Health, 93, 529– 533. Merzel, C. and D’Afflitti, J. (2003) Reconsidering community-based health promotion: promise, performance, and potential. American Journal of Public Health, 93, 557 –574. Miles, M. B. and Huberman, A. M. (1994) Qualitative Data Analysis: An Expanded Sourcebook, 2nd edition. Sage, Thousand Oaks. Mittelmark, M. B. (1999) Health promotion at the communitywide level. Lessons from diverse perspectives. In Bracht, N. (ed.), Health Promotion at the Community

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

FUNDING

Page 13 of 14

Page 14 of 14

J. Harting and P. van Assema Seale, C. (1999) The Quality of Qualitative Research. Sage, London. Sorensen, G., Emmons, K., Hunt, M. K. and Johnston, D. (1998) Implications of the results of community intervention trials. Annual Review of Public Health, 19, 379–416. Stokols, D. (1996) Translating social ecological theory into guidelines for community health promotion. American Journal of Health Promotion, 10, 282 –298. Thompson, B. and Kinne, S. (1999) Social change theory. Applications to community health. In Bracht, N. (ed.), Health Promotion at the Community Level 2—New Advances. Sage, Thousand Oaks, pp. 29– 58. Wandersman, A. and Florin, P. (2000) Citizen participation and community organizations. In Rappaport, J. and Seidman, E. (eds), Handbook of Community Psychology. Kluwer Academic/Plenum Publishers, New York, pp. 247– 272. Wandersman, A. and Florin, P. (2003) Community interventions and effective prevention. American Psychologist, 58, 441– 448. Weiss, C. H. (1997) How can theory-based evaluation make greater headway? Evaluation Review, 21, 501–524. Weiss, C. H. (1998) Evaluation. Methods for Studying Programs and Policies. Prentice Hall, Upper Saddle River. Wholey, J. S. (1987) Evaluability assessment: developing program theory. In Bickman, L. (ed.), Using Program Theory in Evaluation. New Directions for Program Evaluation. Jossey-Bass, San Francisco. Yin, R. K. (2003) Case Study Research: Design and Methods, 3rd edition. Sage, Thousand Oaks.

Downloaded from http://heapro.oxfordjournals.org at Universiteit van Amsterdam on August 15, 2010

Level 2—New Advances. Sage, Thousand Oaks, pp. 3 –27. Moyer, A., Verhovsek, H. and Wilson, V. L. (1997) Facilitating the shift to population based public health programs: innovation through the use of framework and logic model tools. Canadian Journal of Public Health, 88, 95– 98. Norton, B. L., McLeroy, K. R., Burdine, J. N., Felix, M. R. J. and Dorsey, A. M. (2002) Community capacity. Concept, theory, and methods. In DiClemente, R. J., Crosby, R. A. and Kegler, M. C. (eds), Emerging Theories in Health Promotion Practice and Research. Strategies for Improving Public Health. Jossey-Bass, San Francisco. Painter, J. E., Borba, C. P. C., Hynes, M., Mays, D. and Glanz, K. (2008) The use of theory in health behavior research from 2000 to 2005: a systematic review. Annals of Behavioral Medicine, 35, 358– 362. Richard, L., Potvin, L., Kishchluk, N., Prlic, H. and Green, L. W. (1996) Assessment of the integration of the ecological approach in health promotion programs. American Journal of Health Promotion, 10, 318– 328. Rogers, P. J. (2008) Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation, 14, 29–48. Rossi, P. H., Freeman, H. E. and Lipsey, M. W. (1999) Evaluation: A Systematic Approach, 6th edition. Sage, Thousand Oaks. Roussos, S. T. and Fawcett, S. B. (2000) A review of collaborative partnerships as a strategy for improving community health. Annual Review of Public Health, 21, 369 –402.