and Evaluating Physical Activity Promotion. Programs in Curitiba, Brazil. Isabela C. Ribeiro, Andrea Torres, Diana C. Parra, Rodrigo Reis, Christine Hoehner,.
Journal of Physical Activity and Health, 2010, 7(Suppl 2), S155-S162 © 2010 Human Kinetics, Inc.
Using Logic Models as Iterative Tools for Planning and Evaluating Physical Activity Promotion Programs in Curitiba, Brazil Isabela C. Ribeiro, Andrea Torres, Diana C. Parra, Rodrigo Reis, Christine Hoehner, Thomas L. Schmid, Michael Pratt, Luiz R. Ramos, Eduardo J. Simões, and Ross C. Brownson Background: The Guide for Useful Interventions for Activity in Brazil and Latin America (GUIA), a systematic review of community-based physical activity (PA) interventions in Latin American literature, selected the CuritibAtiva program for a comprehensive evaluation. We describe the process of developing logic models (LM) of PA community interventions from Curitiba, Brazil, and discuss influential factors. Methods: The year-long process included engaging stakeholders involved in the promotion of PA in Curitiba, working with stakeholders to describe the programs and their goals, and developing LMs for the 2 main secretaries promoting PA in the city. Results & Conclusions: As a result of stakeholder interviews and discussion and the development of the LMs, local officials are coordinating programming efforts and considering ways the programs can be more complementary. The process has prompted program managers to identify overlapping programs, refine program goals, and identify gaps in programming. It also helped to frame evaluation questions, identify data sources, describe realistic outcomes, and reinforce the importance of intersectoral alliances for public health impact. Developing LMs proved to be feasible in the Latin American context, therefore adaptable and useful for other PA promotion programs in the region. Keywords: GUIA, community-based research, program evaluation, health promotion, Latin America Regular physical activity (PA) reduces the risk of many adverse health outcomes and can improve chronic health conditions.1,2 Despite the recognized public health benefits of regular PA, a significant proportion of the population remains sedentary or insufficiently active, both in developed3 and developing4 countries. The World Health Organization’s Global Strategy on Diet and Physical Activity was a fundamental advocacy tool for PA as it recognizes the need for public health action to increase Ribeiro is with the Air Pollution and Respiratory Health Branch, National Center for Environmental Health, Centers for Disease Control and Prevention, Atlanta, GA. Torres, Schmid, and Pratt are with the Physical Activity and Health Branch, Centers for Disease Control and Prevention, Atlanta, GA. Parra is with the Prevention Research Center in St. Louis, George Warren Brown School of Social Work, Washington University in St. Louis. Reis is with the Dept of Physical Education, Pontific Catholic University of Paraná, Curitiba, Brazil. Hoehner and Brownson are with the Siteman Cancer Center, Washington University School of Medicine, St. Louis, MO. Ramos is with the Dept of Preventive Medicine, Federal University of São Paulo, São Paulo, Brazil. Simões is with the Prevention Research Centers Program, Centers for Disease Control and Prevention, Atlanta, GA.
PA and improve nutrition habits worldwide.5 Even though there is evidence of a large number of PA interventions being carried out in Latin America, 6,7 relatively few have been carefully evaluated.8,9 Even fewer have practical information available via scientific publications or public reports on issues for effective implementation and replication (eg, design, needed resources, evaluation strategies).10 The Guide for Useful Interventions for Activity in Latin America (GUIA) was initiated in 2005 to examine and promote evidence-based strategies, with the intention of increasing PA in Latin America.10 The first phase of GUIA involved a systematic review of community-based PA interventions in Latin American literature.10 This review followed the U.S. Guide to Community Preventive Services (the Community Guide) methods.10–12 The GUIA search identified 3 categories of community interventions in Brazil that were not identified in the U.S. (English language) review: PA classes in community settings, community-wide PA policies and planning, and short physical activity-related messages. Two specific programs were selected for a comprehensive evaluation within these 2 domains—1 in Recife9 and 1 in Curitiba. The CuritibAtiva program, classified as a community-wide PA policy and planning strategy in the GUIA systematic review,10 is one of these interventions. Although this initial classification S155
S156 Ribeiro et al
fits Curitiba’s broad city-wide recreational initiatives and infrastructure, the CuritibAtiva program is best classified under the community-wide campaigns category based on what we learned after completing the logic framework. Located on a plateau at 932 m (3060 ft) above sea level, Curitiba has approximately 1.8 million inhabitants (7th largest city in Brazil) and is the largest economy in Southern Brazil. The CuritibAtiva program was created and implemented by Curitiba’s Municipal Secretary of Sports and Leisure (MSSL) in 1998, in response to the burden of chronic diseases among the local population, to inform and educate the residents about the potential for PA to improve quality of life.13 The activities delivered by the CuritibAtiva program include distribution of printed educational materials addressing PA, assessment of lifestyle practices, and physical fitness examination. These activities occur in parallel with others carried out by the MSSL and by the Municipal Secretary of Health (MSH), which were compiled into 2 logic models and will be further described in this article. Secretaries in Brazil are departments of state (State Secretary) or municipality (Municipal Secretary) dedicated to different administrative areas such as health, education, and transportation. Curitiba has been nationally and internationally recognized for its creativity in facing severe problems associated with dense urban development, such as traffic congestion, loss of public space, and adverse environmental impacts, similar to those found in other large urban agglomerations across Latin America.14 Despite these major challenges that came with rapid growth, there have been significant efforts to improve citizens’ quality of life through the improvement and creation of alternative public transportation, preservation of the city’s cultural heritage, expansion of parks and green areas, and social and environmental programs.15 The city of Curitiba is divided into 9 regional governments, which manage the 75 neighborhoods of the municipality. This decentralized organization of public management created opportunities for partnerships and collaborations with formal and informal organizations in the city, as well as within the public sector. While the decentralized model favors the development of strategic programs and activities, it also requires multi institutional action and mobilization of the community to be more effective.16 Understanding this characteristic of Curitiba’s administration was a key factor in the development of the logic model as it helps to explain the complexity and intricacy of the process to identify the many elements that went into the overall picture of PA promotion in Curitiba. With sight of Curitiba’s large number of PA programs and specific public administration characteristics the GUIA team decided to undertake an initial step to inform the evaluation planning. This consisted of a thorough description of the programs including the development of a logic model, with an intense involvement of municipality staff. The Centers for Disease Control and Prevention’s (CDC) Handbook for Evaluation of Physical Activity Programs in Latin America17 outlines 6 steps that relate to PA program evaluation: engage stakeholders, describe
the program, focus the evaluation design, gather credible evidence, justify conclusions, and ensure use and share lessons learned. Describing the program is one essential piece in the evaluation process that can be developed using the logic model tool. Logic models provide a common language among stakeholders and help to identify important variables that ought to be measured when conducting evaluations.17 This article focuses on 2 steps of the handbook: engaging stakeholders and describing the program, including the development of the logic model. Logic models should depict the core components of a program, illustrate the connections between program components and expected outcomes, and include pertinent information about contextual factors that could influence the program.17 Logic models and the processes used to develop them help clarify the implicit and explicit “theory of action” for a program (ie, Why do you expect an input such as an annual run to increase overall community levels of physical activity?). The aims of this article are to 1) describe the process of developing logic models of PA programs in the city of Curitiba, Brazil; 2) portray, in a visual and logical manner, the various ongoing PA community interventions in Curitiba, so that a comprehensive evaluation can account for all of the factors that may be contributing to population levels of PA; and 3) discuss factors that had an important influence on the process of creating the logic models.
Methods Developing the Logic Models Although there is often a distinction between programs and policies, in the context of this article, “program” is being defined in a very broad way; as any organized public health activity, such as health care service interventions, community mobilization, policy development, communication campaign, and infrastructure projects.18 Figure 1 summarizes the steps used in the process of developing the logic models of PA interventions in Curitiba. The first was to engage individuals or organizations involved in some way with the promotion of PA in Curitiba, known as stakeholders. Identifying and engaging stakeholders from different institutions or groups, and with various functions within the program (eg, implementers, partners, decision makers, community members) is critical for the success of the evaluation.17,18 Two main questions were used to identify program stakeholders: “What are the organizational structures within the municipality (such as secretaries)?”, and “What types of employees are responsible for planning, implementing, and coordinating the PA programs in Curitiba?”. Since the CuritibAtiva program was identified in the GUIA literature review, we started contacting its coordinator and staff in January and February of 2007, who then indicated other individuals and institutions that were somehow involved in the promotion of PA in Curitiba. We brought in representatives from various municipal agencies including MSH, Education, Urban Planning,
Logic Model for PA Programs S157
Figure 1 — Steps in developing the logic models of PA interventions in Curitiba.
Transportation, Social Action, and Environment, to form the initial stakeholders’ group. The next task was to develop a clear description of all PA promotion programs, taking into account stakeholders’ responses. This description included a definition of the problem that the programs addressed, the program activities, resources, expected outcomes, and the context (eg, political, environmental, cultural).17 To initiate and guide discussion, prompting questions (eg, “What are the inputs and activities of the program?” and “What are the desired outcomes of the program?”) were developed. In addition, to have a uniform understanding of these terms among stakeholders, we initiated the process with a brief discussion about concepts such as resources and program outcomes. Stakeholders’ responses to these general questions were collected to help define the many program elements and their intended goals and to identify appropriate elements for the logic models. Initially, this was done in large meetings with the participation of various institutions’ staff. After the major activities were identified, the discussions happened in smaller groups or individual in-person, telephone, or e-mail contacts. Overall, this process of engaging stakeholders and developing the logic models took about a year and involved the participation of at least 1 representative from 7 secretaries from the municipality of Curitiba at some point of the process, most intensively from MSSL and MSH, and members of the GUIA team. The first product in this process was the identification of a vast number of activities and projects coordinated and developed within MSSL, with different levels of shared
roles with the CuritibAtiva program. In addition, upon the description of MSSL’s programs, the staff unveiled numerous links, either through partnership or collaboration, between their secretary and other secretaries, such as Environment, Transportation, Social Action, and Health. As the information exchange moved forward and the stakeholders’ inputs were considered, we realized that there were 2 key players: MSSL and MSH. Therefore, the second product of this process was the shift in the focus of the GUIA evaluation planning to all PA community interventions promoted either by MSSL and by MSH, and not only the CuritibAtiva program. With that in mind, an initial logic model was drafted, taking into account the inputs from all stakeholders, especially program coordinators and staff from MSSL and MSH, regarding the programs’ descriptions. The large amount of information supplying the required components of the logic model (resources, activities, outputs, short, mid, and long-term outcomes) resulted in an unclear figure, which did not achieve the purpose of logic models; among them clarifying programs components and connections. As a result, in agreement with the stakeholders, we opted to describe the interventions in 2 separate logic models (Figure 2 and Figure 3). The models presented do not detail a single program, but rather a number of less detailed activities identified by each secretary, MSSL and MSH, as the core components of the PA promotion in Curitiba. While programs’ resources, activities, and outputs were rapidly identified by MSSL and MSH staff, defining the main problem, the short-, mid, and long-term outcomes, and the factors
S158
Figure 2 — Prefeitura Municipal de Curitiba—Municipal Secretary of Sports and Leisure (MSSL); problem: high prevalence of sedentary behavior among the population of Curitiba.
S159
Figure 3 — Prefeitura Municipal de Curitiba—Municipal Secretary of Health (MHS); problem: high incidence and prevalence of chronic diseases among the population of Curitiba.
S160 Ribeiro et al
influencing the programs was a less easy task. In the end, no individual arrows connecting activities to outcomes are shown, because the outcomes were seen as expected considering all programs and activities collectively.
Logic Model 1: Municipal Secretary of Sports and Leisure (MSSL) The logic model developed for MSSL (Figure 2) shows programs and activities in which MSSL has a stronger role, such as coordination, training, and organization. The CuritibAtiva program clearly encompasses a large amount of activities promoted by MSSL, connecting in various ways with the other activities within the Secretary. A number of activities were offered on a short-term basis, such as the Rustic Race, the marathon, and Dance Curitiba (Figure 2), and other activities were offered on an on-going basis, like the PA classes, Night Bikers, and the walking circuit. Sports-related activities, such as the School Games, CATES (sports initiation program), and Regional Sports Meetings, target the younger population, especially children and adolescents. In addition, informational and training activities were also carried out. This was consistent with the knowledge and awareness expected as short term outcomes. One recurring characteristic of MSSL PA programs is the use of public parks and squares, as well as the sports and leisure centers, located in public spaces called “Rua da Cidadania” (Citizenship Street), a branch of the municipality existent in each of the 9 regional governments where various public services are offered with no cost to the population.
Logic Model 2: Municipal Secretary of Health (MSH) Figure 3 presents the MSH programs and activities for promoting PA in the city of Curitiba. With the exception of the Health on the Move program, the listed activities and programs have a much broader scope than only PA, including healthy eating, tobacco control, and traffic safety issues. Two of the programs, Health on the Move and Healthy Environments, have narrower target populations for their activities, which are essentially defined by the settings where these activities are delivered; for instance, health care units for Health on the Move and companies, universities, or schools for Healthy Environments. Information on the number of program participants is not always available, particularly when a program occurs irregularly, such as the Healthy People program, or when the program was recently implemented (at the time of this study), like the Health on the Move program (Figure 3).
Discussion Several enablers and barriers for program implementation were identified in the process of developing the logic models (Figure 4). Engaging various stakeholders in this process provided it with enabler factors such as conveying
different perceptions about the programs, activities, and even resources. The diversity of the group also helped to identify the different roles played by a vast number of partners that the others were not aware of, especially those within the public sector. Although a great number of valuable and long-standing partnerships are in place, occasionally, these proved to be ineffective when it came to sharing information and collaborating in the planning and execution of new or existing programs and activities. Through the engagement of stakeholders, a large number of PA promotion programs, being carried out in various settings across the city, were identified. This process, however, also exposed the disconnection between the PA promotion agendas of MSSL and MSH and the other agendas within the municipality, such as the urban planning and the public transportation secretaries. One strong component of the PA programs in Curitiba is the promotion of social interaction, such as in the Walking Campaign program, and the Civic Gathering activities. On the other hand, some of the activities were found to focus on sports practice rather than PA for the general population. Lastly, although many of the programs described by the stakeholders have been carried out for a long time (the CuritibAtiva program is over 10 years old), the description of goals and expected outcomes was initially very fuzzy and disconnected from the activities implemented. One year of collaborative work involving key staff from MSSL and MSH in the development of the logic models helped stakeholders within the 2 secretaries to improve their understanding of the programs and revealed a number of overlapping activities. The process of developing the logic models pointed out a large number of strategies including distribution of informative material, capacity building activities, PA classes, and the creation and renovation of spaces dedicated to PA. It also revealed that these strategies were lead originally by MSSL, but have been supported by a number of partners inside and outside of the public sector. In addition, it emphasized the recent engagement of MSH in promoting and supporting programs with specific focus on PA, independently and also partnering with MSSL and other secretaries. The process of developing these logic models provided stakeholders with a clearer picture of the resources used to sustain the activities, such as personal, infrastructure, funding, and partnerships. This information was recognized as very useful in assessing the needs for sustaining their programs. Stakeholders used the logic models process to clarify the products and outcomes expected with the implementation of the activities, being able to organize this information in the context of the city and connect it all with the secretaries’ (MSSL and MSH) main goals. An interesting example of the significance of this process was the inclusion of MSSL’s logic model in the recent publication from the CuritibAtiva program13 showing that this tool has being recognized as being important in the planning process and has been incorporated in the implementation of the PA programs by coordinators and staff.The logic model development process brought to
Logic Model for PA Programs S161
Figure 4 — Enablers and barriers from the process of developing the Logic Models.
light the unique ways these 2 sectors address the same issue of increased prevalence of chronic disease in the city. Although both logic models show similar short-, mid, and long-term outcomes, MSSL’s and MSH’s stakeholders identified different inputs and activities pertaining to the programs from their institutions. With a clear focus on PA and sports programs centered in providing everyday opportunities for the population to be physically active, MSSL’s logic model allowed stakeholders to visualize how the CuritibAtiva program depends on and supports other programs within MSSL.13 Particularly among the CuritibAtiva program stakeholders, this process encouraged the reorganization of some activities, such as the proposal of a follow-up telephone survey with individuals who received the printed educational materials on PA. In addition, this process compelled the CuritibAtiva program coordinators to look for ways to strengthen the program’s role in promoting PA in the city, such as turning it into a comprehensive PA policy for Curitiba. On the other hand, with a focus on building participation, sharing responsibilities, and enhancing institutional recognition on health promotion leadership in the city, MSH has strengthened its alliances through the development of programs and activities in the context
of universities, schools, and private companies.19 This strong health promotion component became visible in the process of development of the logic model and is exposed in the comprehensiveness of the interventions, in which many times PA is just one of the many program components. The process enabled stakeholders and the GUIA members to better understand the characteristics of the PA interventions in the context of MSH, where numerous intervention activities are underway (eg, PA, nutrition, tobacco, and hypertension). Thus, the difficulty in isolating the PA piece in MSH’s programs prompted the stakeholders to present all components together, in the same way they are delivered by MSH to the community. The decision of developing 2 and not 1 model favored clarity and usefulness of the final products. However, the way these logic models were developed do not necessarily match the more rigid approach outlined in the handbook,17 but this flexibility in illustrating the program components and relationships is one of the great features of using logic models.7 During this process of developing the logic models, what became clear to the stakeholders is the importance of developing a logic model, not only before a program is developed and implemented but even if the program has had many years in place, to help
S162 Ribeiro et al
guide evaluations and carry out program improvements. Paramount in this process was the comprehension that it helped stakeholders from MSSL and MSH to recognize the need for strong and strategic alliances to sustain their programs.
Conclusion The process of developing a detailed description of the program using the logic models tool was essential in showing the contribution of each partner in promoting PA in Curitiba and how progress depended on many sectors. It was an opportunity for those involved to discuss and articulate strategies and efforts with current and potentially new partners. The better understanding of the program’s elements enhanced team work and may have had a positive effect in the way the staff planned and managed the programs. The presence of a long-standing and credible staff, and the equally long-standing support for a series of programs and activities, helped to gather credible information about the PA interventions and the context in which these interventions were proposed and implemented. Evaluation of the PA programs in Curitiba is ongoing and has been enhanced greatly by the process of developing logic models. It has helped in framing evaluation questions, focusing resources, identifying sources of data, describing realistic outcomes, and establishing goals and objectives. The process proved to be feasible in the Latin American context and could be very useful for other PA promotion programs in Latin America. Acknowledgments This study was funded through the Centers for Disease Control and Prevention contract U48/DP000060-01 (Prevention Research Centers Program). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention. The authors are thankful for the contribution of staff from Curitiba’s Municipal Secretaries of Sport & Leisure, Health, Education, Urban Planning, Transportation, Social Action, and Environment, particularly the program coordinators Marcia Krempel (MSH) and Dalton Grande (MSSL). We are also grateful for the administrative support of Madalena Soares and Diva Brunieri.
References 1. World Health Organization. World Health Report 2002. Reducing Risks, Promoting Healthy Life. Geneva: WHO; 2002. 2. Physical Activity Guidelines Advisory Committee. Physical Activity Guidelines Advisory Committee Report, 2008. Washington, DC: U.S. Department of Health and Human Services; 2008. 3. Centers for Disease Control and Prevention. Adult participation in recommended levels of physical activity—United States, 2001 and 2003. MMWR Morb Mortal Wkly Rep. 2005;54(47):1208–1212.
4. BRASIL. VIGITEL Brasil 2007. Vigilância de fatores de risco e proteção para doenças crônicas por inquérito telefônico. Brasília, DF 2008. pg 70–79. http://portal. saude.gov.br/portal/arquivos/pdf/vigitel2007_final_web. pdf. Accessed February 2009. 5. World Health Organization. WHO resolution. Global strategy on Diet, Physical activity and health. 2004. http:// www.who.int/dietphysicalactivity/en/. Accessed February 2009. 6. Centro de Estudos do Laboratório de Aptidão Física de São Caetano de Sul, Centers for Disease Control and Prevention. Best Practices for Physical Activity Promotion Around the World. Atlanta, GA: CDC; 2006. 7. Schmid TL, Librett J, Neiman A, Pratt M, Salmon A. A framework for evaluating community-based physical activity promotion programmes in Latin America. Promot Educ. 2006;13(2):112–118. 8. Pratt M, Jacoby ER, Neiman A. Promoting physical activity in the Americas. Food Nutr Bull. 2004;25(2):183–192. 9. Simões EJ, Hallal P, Pratt M, et al. Effects of a communitybased, professionally supervised intervention on physical activity levels among residents of Recife, Brazil. Am J Public Health. 2009;99(1):68–75. 10. Hoehner CM, Soares J, Parra DP, et al. Physical activity interventions in Latin America: a systematic review. Am J Prev Med. 2008;34(3):224–233. 11. Zaza S, Wright-De Aguero LK, Briss PA, et al. Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community Preventive Services. Am J Prev Med. 2000;18(suppl 1):44–74. 12. Briss PA, Zaza S, Pappaioanou M, et al. Developing an evidence-based Guide to Community Preventive Services–methods. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(1, Suppl):35–43. 13. Grande D, Siqueira JE, Cassou JC, Rauchbach R, Kruchelski S. Curitibativa: Política pública de Atividade Física e Qualidade de Vida de uma Cidade. Ed. Venezuela. 2008. 14. Tapia Granados JA. Reducing automobile traffic: an urgent policy for health promotion. Rev Panam Salud Publica. 1998;3(4):227–241. 15. Instituto de Pesquisas e Planejamento Urbano de Curitiba. IPPUC. http://www.ippuc.org.br/ippuc/index. Accessed February 2008. 16. Moysés SJ, Moysés ST, Krempel MC. Assessing the building process of health promotion public policies: the experience of Curitiba. Cien Saude Colet. 2004;9(3):627–641. 17. U.S. Department of Health and Human Services. Handbook for Evaluation of Physical Activity Programs in Latin America. Atlanta GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention; 2005. http://www.cdc.gov/nccdphp/dnpa. 18. Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR Morb Mortal Wkly Rep. 1999;48(No. RR-11). 19. Krempel MC, Moysés ST. Ambientes saudáveis; uma estratégia de promoção de saúde. Projeto Técnico. SMSCuritiba; 2002.
