PEER REVIEW HISTORY BMJ Open publishes all

PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (http://bmjopen.bmj.com/site/about/resources/checklist.pdf) and are provided with free text boxes to elaborate on their assessment. These free text comments are reproduced below.

ARTICLE DETAILS TITLE (PROVISIONAL)

AUTHORS

Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review McFadyen, Tameka; Chai, Li Kheng; Wyse, Rebecca; Kingsland, Melanie; Yoong, Sze Lin; Clinton-McHarg, Tara; Bauman, Adrian; Wiggers, John; Rissel, Chris; Williams, Christopher; Wolfenden, Luke

VERSION 1 – REVIEW REVIEWER REVIEW RETURNED

Susanna Geidne, Phd Public Health, Senior lecturer Sports Örebro University, Sweden 11-Sep-2017

GENERAL COMMENTS

Dear authors, Implementation of health promoting initiatives is important and relevant for changing health outcomes. Health promotion studies within the sports clubs/sports organizations setting is still a quite new research area and it is always interesting to read some new papers. I think you have done a very thorough, information rich and technically advanced systematic review. In that sense I have nothing to complain about. However when it comes to the relevance of this systematic review I would like to reflect on some issues. The sports clubs setting are in many way different from for example schools/childcare settings because of its informal and voluntary nature. To implement HP initiatives in sports clubs can therefor differ from implementation in other settings. You have referred to some literature in this area, but I think you miss a dissertation (with three included papers) on implementation in Norwegian sports clubs that was published in the spring 2017: Sövik, Margaret L. (2017) Evaluating the implementation of the Empowering CoachingTM program in Norway. Dissertation for PhD at University of Bergen. My most serious concern is however if three included trials is enough to do a systematic review? Is there enough knowledge to get out of a study “that we need more knowledge”? Connected to this is the quite hard criteria for study designs, with only controlled studies being considered. An interesting perspectives could be to compare implementation strategies of your included studies with what studies with another design says about implementation strategies. This would also broader your “knowledge base”, because describing

1

three trials does not produce that much new knowledge. REVIEWER REVIEW RETURNED

Matthew Philpott European healthy Stadia Network, UK 12-Sep-2017

GENERAL COMMENTS

Undoubtedly a well written paper on a pressing subject for both sports and public health policy. However, I would argue that the overly stringent inclusion criteria restrict the scope of the paper and its findings. There are a number of papers within the literature which would seem to fall within the inclusion criteria that are not included in the evidence base, some of which are actually referenced by the authors (e.g. Hunt et al. 2014). More could be made within the Discussion section concerning the relatively large number of papers that have been published in this area of research, but currently sit outside of the very narrow inclusion parameters et by the authors, and the inherent tensions between pragmatic implementation of interventions in sports settings and the scientific rigor of RCTs. This, I feel, is the nub of the issue here.

REVIEWER

Linda Ooms Mulier Institute, The Netherlands 13-Sep-2017

REVIEW RETURNED GENERAL COMMENTS

Manuscript ID: bmjopen-2017-019151 Title: Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review General remarks: - It is important that the aim of the review is clear. Do you focus on strategies to improve implementation (EPOC parts) or is the focus on to what extent interventions are implemented (and their effects)? - The article needs some more background/discussion from previous studies in this setting. - The discussion does not answer the research question. It should be more discussed. Abstract: - p1. Line24 ‘sporting venue’: Do you mean sports club? Introduction: - p.2 Are you focussing on the sports club or on organized sports more broadly (sport venues, sport clubs, sport stadia)? - p2., lines 25-26: Two relevant articles about sporting programs aimed at improving physical activity levels of less active people:  Ooms L, Veenhof C, Bakker DH de. Effectiveness of Start to Run, a 6-week training program for novice runners, on increasing health-enhancing physical activity: a controlled study. BMC Public Health 2013, 13:697. DOI: 10.1186/14712458-13-697.  Ooms L, Veenhof C, de Bakker D. The Start2Bike program is effective in increasing health-enhancing physical activity: a controlled study. BMC Public Health 2017, 17:606. DOI: 10.1186/s12889-017-4523-1.

2

- p.2, lines 29-35: Are you focusing about implementation of strategies in de sports club setting? Because this focuses on sports stadia. This article specifically focuses on health promotion policies in the sports club setting:  Dobbinson SJ, Hayman JA, Livingston PM: Prevalence of health promotion policies in sports clubs in Victoria, Australia. Health Promot Int 2006, 21:121-129. - p.2, lines 37-48: There is some evidence about factors influencing implementation in the sports club setting, for instance:  Eime RM, Payne WR, Harvey JT: Making sporting clubs healthy and welcoming environments: a strategy to increase participation. J Sci Med Sport 2008, 11:146-154.  Ooms L, Veenhof C, Schipper-van Veldhoven N, de Bakker D. Sporting programs for inactive population groups: factors influencing implementation in the organized sports setting. BMC Sports Sci Med Rehabil 2015, 7:12.  Casey MM, Payne WR, Eime RM: Partnership and capacitybuilding strategies in community sports and recreation programs. Managing Leisure 2009, 14:167-176.  Casey MM, Payne WR, Eime RM: Organisational readiness and capacity building strategies of sporting organisations to promote health. Sport Management Review 2012, 15:109124.  Crisp BR, Swerissen H: Critical processes for creating health-promoting sporting environments in Australia. Health Promot Int 2003, 18:145-152.  Kokko S: Guidelines for youth sports clubs to develop, implement, and assess health promotion within its activities. Health Promot Pract 2014, 15:373-382.  Dobbinson SJ, Hayman JA, Livingston PM: Prevalence of health promotion policies in sports clubs in Victoria, Australia. Health Promot Int 2006, 21:121-129. These are no controlled studies, but they should be mentioned/discussed in the background, or otherwise in the discussion section. - p.2, lines 50-54, objectives: Do you want to determine the effectiveness of strategies (EPOC parts are used to improve implementation of the different intervention parts) to improve implementation? Or do you want to know how effectively intervention parts are implemented? This is a little bit confusing. Methods: - p.3, lines 37-43: Types of participants: The review is not only about sports clubs, but broader. This should be clear throughout the article (see also remark introduction). - p.4, lines 32-53: I have looked at the search strategy. Did you also look directly for ‘sports club’, ‘organised sports’, ‘health-enhancing physical activity’? This may also be relevant terms. - p.6, line 41: EPOC taxonomy: add reference to appendix 2 here. -p.7, line 41: what do you mean with the study inclusion criteria was ‘relaxed’? Results: - p. 16, lines 9-16: risk of bias: blinding of participants and personnel is not possible with these kinds of strategies/programs in the sports

3

setting. This should be taken into account when interpreting these biases. - p. 17, lines 29-57, primary outcomes: ‘effectiveness of strategies to improve implementation’: I think this part describes to what extent different parts of the intervention are implemented. I think the EPOC parts are the strategies to improve implementation. This is a little bit confusing. This is also in other parts of the article, like the aims. Discussion: - p.19, lines 14-16: See previous remark. What is the aim of the study? - p.19, lines 44-53: Is this the answer to your question? Which strategies are effective in improving implementation of different implementation parts: educational materials, educational outreach visits and monitoring performance? Is this only based on the fact that these strategies were used in all three interventions? Or are there other things pointing out that these strategies improve implementation? What does the research literature (for instance, implementation literature, qualitative studies) say about these strategies? This should be more discussed. REVIEWER REVIEW RETURNED GENERAL COMMENTS

Dr Louise Mansfield Brunel University London, UK. 17-Sep-2017 This is a timely and important review given the increasing priority being given towards the public health role of community and sport clubs. The design, conduct and reporting of the review follow established guidelines. I would accept for publication but have some comments for very minor revisions that would strengthen the clarity of the paper further. 1.The secondary aims suggest the work will describe cost and cost effectiveness in the review but this is not mentioned in the abstract. Might it be prudent to do so given that cost is an important aspect of implementation strategies? 2. It is listed elsewhere but I think the PROSPERO registration number should appear in the methods section at the end of the introductory paragraph where it is mentioned again. 3. In the section on ' Search methods for identification of studies' (page 4, from line 27) there is some confusion about your search strategies for grey literature. In the first sentence of this section it reads as if you have searched for grey literature in the same data bases as the published literature. This seems to contradict the definition of grey literature as unpublished. I think some clarity about the grey literature searches is needed in this section. 5. I also think a short clarification statement is needed in the paper about what 'companion papers' are, why they are part of a systematic review method, how they were searched for, what the inclusion/exclusion criteria were for them, and what they bring to the review. 4.In my copy it appears a full stop is missing from page 1, line 31/32 after the words 'control trials'.

REVIEWER REVIEW RETURNED

Cindy Gray University of Glasgow, UK 17-Sep-2017

4

GENERAL COMMENTS

Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review This is a very timely review, as high-quality implementation of health promotion interventions is essential if the potential of community sports settings to improve health outcomes it to be maximised. The abstract needs re-written on BMJ Open house style. Also, could it say what implementation strategies were used? Method p5 ln 41 – could the authors provide some more detail of information to allow assessment of bias. Why is the risk of bias results provided in the methods (as well as in the results)? Results – it is not clear what happened to the 6 companion papers – were they included in data extraction– more detail is required at this point (p7,ln51). More detail is also required about how the Wolfenden et al study was nested within the Kingsland et al study. Figure 2 – could there be a legend to show what the colours mean. What does Figure 3 add, it is not clear? Discussion: p19 why are the below different? Could the authors provide more explanation? “In this review, absolutes effect sizes range from 1% - 38% for policy or practice implementation.” Vs “Trials included in this review reported improvements of between 1066% in policy or practice implementation for intervention groups within the sporting club context, which is somewhat great compared to these other implementation reviews [48].”

P20 lin4-6 participating clubs may have differed in relation to what? Some of the references seem old (e.g. 1-3). Are there more up-todate ones available? Check tense and typos throughout REVIEWER REVIEW RETURNED GENERAL COMMENTS

Rebecca Armstrong University of Melbourne, Australia 19-Sep-2017 This is a well constructed and useful paper particularly given the role sporting organisations now play in health promotion. There are some minor issues which should be clarified: 1. The section on data synthesis is very brief and would be challenging to replicate. Can the authors provide any additional information about their narrative synthesis? 2. The authors of this review are also authors of 2/3 included studies. Whilst this is ok - more should be included in the paper about how any potential conflict of interest was managed. There is a brief section under competing interests but more should be said

5

about lack of involvement in other stages of review (e.g. extraction and analysis) not just risk of bias assessment. 3. The section on quality of the evidence also appears to be brief. The authors note that the overall body of the evidence is very low but no further context is provided. 4. There are a couple of minor typos remaining in the paper. A careful edit should be conducted. 5. Paragraph 3 in the discussion should be re-considered given the findings presented in the review. The statement around the effectiveness of multi-strategic interventions appears to be overstated. 6. One of the criticisms of the paper may be the study design inclusions. These types of strategies are likely to have been conducted without rigorous study designs. I think it may be worth addressing this issue and identifying how this may be addressed. Do you have a sense of the body of evidence beyond your included studies? 7. Appendix 3 is really useful but it is hard to assess the authors judgement. i.e. what does high risk mean and how was this assessed? You might consider including additional info in the methods section or use a legend. There are also some formatting issues with these tables (Naylor) REVIEWER REVIEW RETURNED GENERAL COMMENTS

Dr Shaun Scholes University College London, United Kingdom 19-Oct-2017 The authors of this study undertook a systematic review to examine the effectiveness of strategies to improve the implementation of policies, practices or programmes (focusing on health risks) in sporting organisations. I accepted the invitation to review this manuscript due to my interests in physical activity / sports and I cannot really comment on the methods of the systematic review itself. Only finding three studies that met the inclusion criteria (out of 5926 citations) must be a limitation: although I did note that at some point the authors did relax the inclusion criteria. The paper is very well written and the authors are clearly experienced in the conduct of systematic reviews. The limitations of the study are clearly set out in the discussion and I agree with the concluding statement that “our ability to identify effective strategies or provide clear directions for future sports based interventions is limited”. Therefore I encourage this paper’s publication to help progress things further. In what follows I simply make some comments the authors may wish to consider. Abstract The authors mention that “club revenue was a measure of adverse effect” (line 37) but made no mention of this in the first paragraph. This sentence was very unclear at first reading. This result ties in with the secondary aim of the study: and so it would help readers if those secondary aims are outlined in the abstract as well. Results Figure 1 should be placed after its mention in the text. Should the main reasons for exclusion (n=5913) at the screening stage be mentioned: either in the figure or in the text? I found this a surprising omission.

6

The authors state that the quality of evidence in the included studies was very low. Should this point be included in the abstract? I am confused by the ORs in Table 1: how do they relate (if at all) to the percentages quoted? If their meaning is not immediately clear then should this table include the odds ratios at all: especially if the quality of evidence is low? Going back to the abstract, the authors say that “club revenue was a measure of adverse effect for two studies”. But only one p-value is provided (line 10, page 19). Discussion I recommend that the authors refer (briefly) to the quality of the evidence before summarising the effect sizes of the included studies. Appendix Please shorten the length of Appendix 1. REVIEWER REVIEW RETURNED

Töres Theorell Stress Research Institute, Stockholm University 05-Nov-2017

GENERAL COMMENTS

BMJ Open 2017-019151 I was asked to make a judgement regarding the statistical approach only. The authors have followed a Cochrane-inspired strategy. The most striking finding is that there are very few rigorous published studies on the effects of programs targeting poor diet, physical activity, obesity and risky use of alcohol and tobacco. They end up with only three studies, two of them Australian studies that are linked to one another and one Canadian. It is impossible on the basis of so few studies to make any systematic conclusions regarding evidence level. Publication bias (funnel plots), homogeneity, consistency – everything is impossible. However the authors make some important points in their discussion and they have an interesting discussion regarding different types of bias that is illuminating for the field.

REVIEWER REVIEW RETURNED GENERAL COMMENTS

Cara Booker University of Essex, UK 20-Nov-2017 Review of Manuscript “Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review” Manuscript: bmjopen-2017-019151 This article reviews studies that have evaluated policies, programs or practices of sporting organisations in addressing different health behaviours. Studies included in the review required a RCT or pseudo-RCT implementation of the policy, program or practice implemented. Only three studies were included in the review, with some improvement in the health behaviour addressed. There were mixed findings with respect to cost-effectiveness.

7

General comments: 1. Was the lack of papers addressing this issue due to lack of academic involvement in the implementation of certain policies, practices and programs in sporting organisations? Could it be that these organisations implement these without the aid of academics, evaluate them on their own and then decide whether to continue or discontinue the implemented policy/program/practice on their own? 2. You looked exclusively at sporting organisations, could you have included other venues such as community centres? REVIEWER REVIEW RETURNED GENERAL COMMENTS

Asad Khan The University of Queensland, Australia 01-Dec-2017 Thanks for giving me the opportunity to review the manuscript titled “Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review”. This is a well written report where the authors have tried to determine the effectiveness of strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use. To me it’s a very broad area which involves considerable complexities as implementation involves various stakeholders with different context and interest. The authors have identified only three studies without much commonalities between them, which is understandable given the broad focus of the review. I think this is merely a description of three different studies with different focuses. I’m not sure how the findings of this review can help the fellow researchers. Given this, I’m afraid I won’t be able to comments on the suitability of the manuscript.

VERSION 1 – AUTHOR RESPONSE Reviewer(s)' Comments to Author: Reviewer: 1 Implementation of health promoting initiatives is important and relevant for changing health outcomes. Health promotion studies within the sports clubs/sports organizations setting is still a quite new research area and it is always interesting to read some new papers. I think you have done a very thorough, information rich and technically advanced systematic review. In that sense I have nothing to complain about. However when it comes to the relevance of this systematic review I would like to reflect on some issues. The sports clubs setting are in many way different from for example schools/childcare settings because of its informal and voluntary nature. To implement HP initiatives in sports clubs can therefor differ from implementation in other settings. You have referred to some literature in this area, but I think you miss a dissertation (with three included papers) on implementation in Norwegian sports clubs that was published in the spring 2017: Sövik, Margaret L. (2017) Evaluating the implementation of the Empowering CoachingTM program in Norway. Dissertation for PhD at University of Bergen. My most serious concern is however if three included trials is enough to do a systematic review? Is there enough knowledge to get out of a study “that we need more knowledge”?

8

Connected to this is the quite hard criteria for study designs, with only controlled studies being considered. An interesting perspectives could be to compare implementation strategies of your included studies with what studies with another design says about implementation strategies. This would also broader your “knowledge base”, because describing three trials does not produce that much new knowledge. Response to reviewer 1: Thank you for your comment. The authors feel that the inclusion criteria is quite liberal with the range of health risk factors and desired outcomes. The inclusion around the study design – “any study with a parallel control group” is required to answer the proposed research question. The value which this review adds to the current evidence is that there is a need for more research in the area on this important topic. Cochrane reviews can be published without any included studies (Priest 2008 “policy interventions implemented through sporting organisations for promoting health behaviour change”) as this identifies a gap in the evidence, which supports the development of future research in this area. Thank you for the suggestions of the additional papers (are these the correct ones below?), they are interesting and informative. However they don’t meet the inclusion criteria for this review. Paper from the dissertation Paper I: Søvik ML, Larsen T, Tjomsland HE, Samdal O. (2016). Evaluating the implementation of the Empowering Coaching™ programme: balancing fidelity and adaptation. Health Education,116(3), 238-258. doi: 10.1108/HE- 07-2014-0077. Paper II: Søvik ML, van Hoye A, Larsen T, Tjomsland HE, Samdal O. Implementation research: a mixed-method study exploring delivery quality of and participant responsiveness to the Empowering Coaching™ workshop. To be resubmitted to Research for All. Paper III: Søvik ML, Larsen T, Tjomsland HE, Samdal O, Wold B (in press). Barriers in implementing coach education in grassroots youth football: a convergent parallel mixed methods study. International Sport Coaching Journal.

Reviewer: 2 Undoubtedly a well written paper on a pressing subject for both sports and public health policy. However, I would argue that the overly stringent inclusion criteria restrict the scope of the paper and its findings. There are a number of papers within the literature which would seem to fall within the inclusion criteria that are not included in the evidence base, some of which are actually referenced by the authors (e.g. Hunt et al. 2014). More could be made within the Discussion section concerning the relatively large number of papers that have been published in this area of research, but currently sit outside of the very narrow inclusion parameters et by the authors, and the inherent tensions between pragmatic implementation of interventions in sports settings and the scientific rigor of RCTs. This, I feel, is the nub of the issue here. Response to reviewer 2: Thanks for the comments. Please refer to response to reviewer 1 above. The suggest addition to the discussion has been made, see final paragraph.

9

Additionally, Hunt 2014 was not included as the study doesn’t address the main aim of determining what strategies support the implementation of policies, practices or programs in sporting organisations, it is a program with a primary aim of weight reduction. Please also not that the text does state –“ Given the limited number of included trials, the study inclusion criteria was relaxed and citations re-examined by two authors (TM and RW) to identify prepost trials without a parallel control group, however no additional trials were found to be eligible.” Reviewer: 3 This is a very relevant article. The focus (aim) of the article does not combine properly with the research results. Do you focus on strategies to improve implementation (EPOC parts) or is the focus on to what extent interventions are implemented (and their effects)? Furthermore, the article should be more embedded/discussed in existing research about the topic. The methods are appropriate. I think it is mostly in the way it is written down and this can be adjusted. Response to reviewer 3: Thanks for the comments. Yes – For the primary aim we are interested in the strategies to improve implementation which is measured by levels of implementation. Methods have been reviewed and adjusted where possible. Reviewer: 4 This is a timely and important review given the increasing priority being given towards the public health role of community and sport clubs. The design, conduct and reporting of the review follow established guidelines. I would accept for publication but have some comments for very minor revisions that would strengthen the clarity of the paper further. 1. The secondary aims suggest the work will describe cost and cost effectiveness in the review but this is not mentioned in the abstract. Might it be prudent to do so given that cost is an important aspect of implementation strategies? Response to reviewer 4 Response 1: Thanks for the comments. Changes have been made to reflect this comments pg 1 “A review was undertaken to determine the effectiveness of strategies to improve the implementation of policies, practices or programs in sporting organisations. This review further sought to describe the cost or cost-effectiveness and adverse effects of such strategies and to examine the effects of those implementation strategies on individual’s diet, physical activity, obesity, alcohol use or tobacco use.” 2. It is listed elsewhere but I think the PROSPERO registration number should appear in the methods section at the end of the introductory paragraph where it is mentioned again. Response 2: The PROSPERO registrations number has been placed at the end of the abstract as per BMJ Open authorship guidelines under abstract section “where applicable, trial registration: registry and number (for clinical trials and, if available, for observational studies and systematic reviews”. But happy to move if the journal supports. 3. In the section on ' Search methods for identification of studies' (page 4, from line 27) there is some confusion about your search strategies for grey literature. In the first sentence of this section it reads as if you have searched for grey literature in the same data bases as the published literature. This seems to contradict the definition of grey literature as unpublished. I think some clarity about the grey literature searches is needed in this section.

10

Response 3: Suggested changes have been made. Pg 4 “Searches for grey literature was also conducted in the same way, with a focus on web-based search engines and government web-sites.” We searched for grey literature in the same way as peer reviewed as we have previously found that some of these databases have kept records of some unpublished works, as well as reference lists and web searches. 4. I also think a short clarification statement is needed in the paper about what 'companion papers' are, why they are part of a systematic review method, how they were searched for, what the inclusion/exclusion criteria were for them, and what they bring to the review. Response 4: Suggested changes have been made. Pg 7 “These papers further inform the research questions of this review by providing information on secondary aims and methods used. The companion papers included study protocols, secondary outcomes, prevalence studies and an economic report.” Companion papers are used when they further inform the information on an included trial or study. The companion papers in this instance are included as they provide more information about methods used as well and the data needed to answer secondary aims. 5. In my copy it appears a full stop is missing from page 1, line 31/32 after the words 'control trials'. Response 5: Thanks, changes made. Reviewer: 5 Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review This is a very timely review, as high-quality implementation of health promotion interventions is essential if the potential of community sports settings to improve health outcomes it to be maximised. 1. The abstract needs re-written on BMJ Open house style. Also, could it say what implementation strategies were used? Response to reviewer 5: Response 1: Some changes have been made to the abstract. 2. Method p5 ln 41 – could the authors provide some more detail of information to allow assessment of bias. Why is the risk of bias results provided in the methods (as well as in the results)? Results – it is not clear what happened to the 6 companion papers – were they included in data extraction– more detail is required at this point (p7,ln51). More detail is also required about how the Wolfenden et al study was nested within the Kingsland et al study. Response 2: Suggested changes have been made. Pg 7 “These papers further inform the research questions of this review by providing information on secondary aims and methods used. The companion papers included study protocols, secondary outcomes, prevalence studies and an economic report.” 3. Figure 2 – could there be a legend to show what the colours mean. What does Figure 3 add, it is not clear? Response 3: Figure 2 has been removed.

11

4. Discussion: p19 why are the below different? Could the authors provide more explanation? “In this review, absolutes effect sizes range from 1% - 38% for policy or practice implementation.” Vs “Trials included in this review reported improvements of between 10-66% in policy or practice implementation for intervention groups within the sporting club context, which is somewhat great compared to these other implementation reviews [48].” Response 4: Suggested changes have been made to the discussion. 5. P20 lin4-6 participating clubs may have differed in relation to what? Response 5: We are not sure where this is referring too. Is it “Club participation rate for both Kingsland et al and Wolfenden et al were relatively low, suggesting that participating clubs may have differed in terms of their readiness to change, potentially limiting the external validity of the findings.”? 6. Some of the references seem old (e.g. 1-3). Are there more up-to-date ones available? Response 6: updates have been made where applicable. 7. Check tense and typos throughout Reviewer: 6 This is a well constructed and useful paper particularly given the role sporting organisations now play in health promotion. There are some minor issues which should be clarified: 1. The section on data synthesis is very brief and would be challenging to replicate. Can the authors provide any additional information about their narrative synthesis? Response 1: Thanks for the comments, data has been synthesised narratively, suggested changes have been made. Pg 6”Study characteristics were grouped as types of studies, participants and implementation strategies. Implementation strategies were classified using the EPOC taxonomy to classify implementation strategies [25]. As trial heterogeneity precluded meta-analysis the trial findings were described and synthesised narratively. The primary outcome (effectiveness of strategies to improve implementation) and secondary outcomes (cost or cost-effectiveness of intervention strategies, effects on poor diet, physical inactivity, obesity, risky alcohol use or tobacco use, reported adverse consequences) for the review reported all available and applicable statistical and descriptive data of the included studies. Two review authors (TM, LW) gave an overall quality of evidence for each outcome using the GRADE system [36].

2. The authors of this review are also authors of 2/3 included studies. Whilst this is ok - more should be included in the paper about how any potential conflict of interest was managed. There is a brief section under competing interests but more should be said about lack of involvement in other stages of review (e.g. extraction and analysis) not just risk of bias assessment. Response 2: suggested changes have been made, addition of other areas of conflict. 3. The section on quality of the evidence also appears to be brief. The authors note that the overall body of the evidence is very low but no further context is provided. Response 3: suggested changes have been made pg 6 and 16 “The GRADE system [36] was used by two reviewers (TM, LW) to assess the quality of the body of evidence through consideration of

12

study limitations, consistency of effect, imprecision, indirectness and publication bias.” With the addition of Cochrane reference. 4. There are a couple of minor typos remaining in the paper. A careful edit should be conducted. 5. Paragraph 3 in the discussion should be re-considered given the findings presented in the review. The statement around the effectiveness of multi-strategic interventions appears to be overstated. Response 5: Changes made, pg 19 “The findings of this review suggest that multi-strategic interventions may have the potential to improve the implementation of policies and practices targeting health behaviours.” 6. One of the criticisms of the paper may be the study design inclusions. These types of strategies are likely to have been conducted without rigorous study designs. I think it may be worth addressing this issue and identifying how this may be addressed. Do you have a sense of the body of evidence beyond your included studies? Response 6: Please refer to response to reviewer 1 7. Appendix 3 is really useful but it is hard to assess the authors judgement. i.e. what does high risk mean and how was this assessed? You might consider including additional info in the methods section or use a legend. There are also some formatting issues with these tables (Naylor) Response 7: changes have been made, links have been provided to the Cochrane site.

Reviewer: 7 The authors of this study undertook a systematic review to examine the effectiveness of strategies to improve the implementation of policies, practices or programmes (focusing on health risks) in sporting organisations. I accepted the invitation to review this manuscript due to my interests in physical activity / sports and I cannot really comment on the methods of the systematic review itself. Only finding three studies that met the inclusion criteria (out of 5926 citations) must be a limitation: although I did note that at some point the authors did relax the inclusion criteria. The paper is very well written and the authors are clearly experienced in the conduct of systematic reviews. The limitations of the study are clearly set out in the discussion and I agree with the concluding statement that “our ability to identify effective strategies or provide clear directions for future sports based interventions is limited”. Therefore I encourage this paper’s publication to help progress things further. In what follows I simply make some comments the authors may wish to consider. 1. Abstract The authors mention that “club revenue was a measure of adverse effect” (line 37) but made no mention of this in the first paragraph. This sentence was very unclear at first reading. This result ties in with the secondary aim of the study: and so it would help readers if those secondary aims are outlined in the abstract as well. Response 1: suggested changes have been made: “This review further sought to describe the cost or cost-effectiveness and adverse effects of such strategies and to examine the effects of those implementation strategies on individual’s diet, physical activity, obesity, alcohol use or tobacco use.” 2. Results

13

Figure 1 should be placed after its mention in the text. Should the main reasons for exclusion (n=5913) at the screening stage be mentioned: either in the figure or in the text? I found this a surprising omission. Response 2: suggested changes made. 3. The authors state that the quality of evidence in the included studies was very low. Should this point be included in the abstract? Response 3: Thanks for the comment. We didn’t include this in the abstract as it can be hard to interpret without the consideration of the Risk of Bias result alos 4. I am confused by the ORs in Table 1: how do they relate (if at all) to the percentages quoted? If their meaning is not immediately clear then should this table include the odds ratios at all: especially if the quality of evidence is low? Response 4: Both ORs and percentages were used as they portray different statistical representations of the effect of implementation strategies reported in the included studies. The percentages were included in the body of the paper as they are more easily interpreted and for completeness of reporting ORs were included also. 5. Going back to the abstract, the authors say that “club revenue was a measure of adverse effect for two studies”. But only one p-value is provided (line 10, page 19). Response 5: Not exactly sure where you are referring too, in the results section of the paper there are two p vaules included - p=.910 and p=.378? 6. Discussion I recommend that the authors refer (briefly) to the quality of the evidence before summarising the effect sizes of the included studies. Response 6: suggested changes made to the discussion 7. Appendix Please shorten the length of Appendix 1. Response 7: Unfortunately appendix one cannot be shortened without removing search terms. These terms are required to allow those potentially interested in replicating the search strategy.

Reviewer: 8 I was asked to make a judgement regarding the statistical approach only. The authors have followed a Cochrane-inspired strategy. The most striking finding is that there are very few rigorous published studies on the effects of programs targeting poor diet, physical activity, obesity and risky use of alcohol and tobacco. They end up with only three studies, two of them Australian studies that are linked to one another and one Canadian. It is impossible on the basis of so few studies to make any systematic conclusions regarding evidence level. Publication bias (funnel plots), homogeneity, consistency – everything is impossible. However the authors make some important points in their discussion and they have an interesting discussion regarding different types of bias that is illuminating for the field. Response: Thank you for your comments

14

Reviewer: 9 Review of Manuscript “Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review” Manuscript: bmjopen-2017-019151 This article reviews studies that have evaluated policies, programs or practices of sporting organisations in addressing different health behaviours. Studies included in the review required a RCT or pseudo-RCT implementation of the policy, program or practice implemented. Only three studies were included in the review, with some improvement in the health behaviour addressed. There were mixed findings with respect to cost-effectiveness. General comments: 1. Was the lack of papers addressing this issue due to lack of academic involvement in the implementation of certain policies, practices and programs in sporting organisations? Could it be that these organisations implement these without the aid of academics, evaluate them on their own and then decide whether to continue or discontinue the implemented policy/program/practice on their own? Response 1: Thanks for the comment. Yes this could be true. We have added a statement in the discussion. 2. You looked exclusively at sporting organisations, could you have included other venues such as community centres? Response 2: This review included “Studies conducted in any organisation that is a venue to undertake organised sport were included. This could include non-elite community sports clubs, recreational centres, as well as professional/elite sporting clubs and stadia.” We included places that had an organisational structure as this is important when implementing policies and practices aimed at creating change. One study was conducted in sports recreation setting.

Reviewer: 10 Thanks for giving me the opportunity to review the manuscript titled “Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review”. This is a well written report where the authors have tried to determine the effectiveness of strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use. To me it’s a very broad area which involves considerable complexities as implementation involves various stakeholders with different context and interest. The authors have identified only three studies without much commonalities between them, which is understandable given the broad focus of the review. I think this is merely a description of three different studies with different focuses. I’m not sure how the findings of this review can help the fellow researchers. Given this, I’m afraid I won’t be able to comments on the suitability of the manuscript. Response: Thanks for the comment. VERSION 2 – REVIEW

15


Töres Theorell Karolinska Institutet, Stockholm, Dep of Neuroscience 14-Jan-2018

GENERAL COMMENTS

They have adequately dealt with my comments

REVIEWER

Susanna Geidne Orebro University, Sweden 15-Jan-2018


I am pleased with your changes. It has been interesting too read your paper.

REVIEWER

Matthew Philpott European Healthy Stadia Network, UK 18-Jan-2018


I think the paper has improved significantly since the first draft and now ready for publication

REVIEWER

Dr Louise Mansfield Brunel University London 18-Jan-2018


Many thanks to the authors for a detailed consideration and response to my first review. This paper is ready for publication and I would recommend it for BMJ Open.


Dr Shaun Scholes University College London, United Kingdom 25-Jan-2018

GENERAL COMMENTS

The authors have responded appropriately to my comments.

REVIEWER

Rebecca Armstrong University of Melbourne, Australia 05-Feb-2018


I think the authors have done a great job incorporating the necessary changes.

REVIEWER

Cara Booker University of Essex, UK 10-Feb-2018


The authors responded to the reviewers comments adequately and I have no further comments.

REVIEWER

Cindy Gray University of Glasgow, UK 17-Feb-2018


This is a very thorough review that reveals the lack of research in this important area of implementing healthy lifestyle interventions in sports club settings. I think it could be published in BMJ Open, but would recommend some redrafting of the manuscript before it is

16

accepted – including some attention to commas, typos, tense and grammar throughout. Specific points for consideration are as follows: Abstract; p1 we need some context in which the study was conducted and a statement of the study design; we also need some indication of the databases that were searched; should the authors define parallel control group, either here or elsewhere in the manuscript; ln 39-41, the last sentence here is hard to understand – it could be expressed more simply; in the conclusion, it would be good to summarize what strategies were used, even if the results are not generalizable. Strengths and limitations; this should be written in sentences, and limitations included. Introduction; p2, ln 19; I don’t think we need sporting clubs (also this is an odd way of describing them); keep number of decimal points are consistent as possible throughout the manuscript; ln 34, why is jurisdictions relevant?; ln 44 should you repeat that it is sport club settings here? Methods: p3, ln 19 perhaps explain parallel control group here?; watch capitalisation of bullet points; p4, ln 22 – should you specify that this is individual level? p6, ln 31-35 – ‘was unable to be performed’ is clumsy; ln 43-44 repetition in sentence starting same order throughout the manuscript. les of reasons for exclusion; p8, ln 10-11 please specify if data were extracted from the companion papers; ln 17-18 – I don’t understand the description of the sample sizes – could they be made clearer?; ln 36 – I don’t understand the last bit of the last sentence – could it be written in a simpler fashion?; p16 I think Figure 2 provides more information than Figure 3 and should be retained instead of Figure 3; p18, ln 11 – is this heading needed; report secondary outcomes in same order as in the methods; ln 20-22 the sentence beginning ‘the intervention is…’ is clumsy, as is; ln 30-33 – please rewrite these sentences more clearly to aid the reader’s understanding of the points being made; ln 24 could you explain what a cost ration of 4.2 means in practice?; ln 41, is 25% to 24% really a reduction?; ln 45, please explain a CATI survey the first time it is used; p 19 ln 9-11, please supply numbers of players, senior teams and spectators for intervention and control clubs. Discussion: p19 ln42-49 is this paragraph in the wrong place, should it go before the previous paragraph?; ln 38-41 – I think you still need to comment on how the current study fits into the context of previous work; ln 48-50 is misleading – there is some evidence of cost effectiveness; what about adverse events and individual impact – you should discuss these outcomes too; p20, ln 8-10 club participation rate is a new result- it should not be discussed unless it is in the results section – the same goes for study design (i.e. selfselection of clubs); ln 24-26 the sentence beginning ‘Therefore, consideration…’ is clumsy; ln 39 ‘ alternate’ is a strange word to use; ln 40 don’t is not formal writing, and could you also give an example of the types of studies that would be included in a different review?

17

VERSION 2 – AUTHOR RESPONSE Thank you for the opportunity to revise and resubmit our manuscript “Strategies to improve the implementation of policies, practices or programs in sporting organisations targeting poor diet, physical inactivity, obesity, risky alcohol use or tobacco use – A systematic review”. The authors would like to thank the journal and the reviewer for their through feedback. We believe the changes made in response to this feedback have improved the manuscript.

Please find below a response to each comment from the reviewer. Changes within the manuscript have been made using track changes.

Systematic review comments

Specific points for consideration are as follows:

Reviewer comment: Abstract; p1 we need some context in which the study was conducted and a statement of the study design; we also need some indication of the databases that were searched; should the authors define parallel control group, either here or elsewhere in the manuscript; ln 39-41, the last sentence here is hard to understand – it could be expressed more simply; in the conclusion, it would be good to summarize what strategies were used, even if the results are not generalizable.

Author response: We have attempted to include this additional information and adhere to the limit of 300 words for the abstract. We will include a definition of parallel control group in the manuscript as requested. Changes have been made to the last sentence include: “We conducted searches of academic data-bases (e.g. MEDLINE, EMBASE and CENTRAL) and trial registers and hand searches of selected journals.” “Two studies assessed club revenue as a potential adverse effect, neither reported significant between group differences on these measures.”

Reviewer comment: Strengths and limitations; this should be written in sentences, and limitations included.

Author response: First point re-worded to make a sentence. “This is the first synthesis of implementation studies targeting multiple health risk behaviours in the sports setting.” Final points are limitations “The review identified a limited number of studies within this area which met the inclusion criteria.” “As there is considerable heterogeneity in terms used to describe implementation, the search terms may not have identified all potentially eligible trials “

18

Reviewer comment: Introduction; p2, ln 19; I don’t think we need sporting clubs (also this is an odd way of describing them); keep number of decimal points are consistent as possible throughout the manuscript; ln 34, why is jurisdictions relevant?; ln 44 should you repeat that it is sport club settings here?

Author response: We have retained the term sporting club in this sentence as it describes sporting contexts which are the focus of the review. The term is used frequently in the current international literature. Changes have been made to decimal points to make consistent Jurisdictions has been changed to setting for clarity

Reviewer comment: Methods: p3, ln 19 perhaps explain parallel control group here?; watch capitalisation of bullet points; p4, ln 22 – should you specify that this is individual level? p6, ln 31-35 – ‘was unable to be performed’ is clumsy; ln 43-44 repetition in sentence starting ‘Implementation same order throughout the manuscript.

Author response: Changes have been made to address comments Parallel group defined as “a group participating in study at the same time as the intervention group, but receiving no or a modified form of intervention” Line ‘31-35’ was changed to “was not performed” Line ‘43-44’ has been amended to remove repetition removed All reporting of secondary outcomes have changed to the same order throughout manuscript

ul to have examples of reasons for exclusion; Author response: Consistent with PRISMA reporting guidelines we have provided reasons for exclusion following full text screening in. We have now also included the following sentence to the results section pg 8 “The more common reasons for exclusion included: study design, setting and study participant.”

Reviewer comment: p8, ln 10-11 please specify if data were extracted from the companion papers; Author response: Changes made to the manuscript “All companion papers were reviewed for information relevant to the review, and data extracted from such texts when appropriate”

Reviewer comment: ln 17-18 – I don’t understand the description of the sample sizes – could they be made clearer?; Author response: Changes made to the manuscript “The sample sizes ranged from 85 sports clubs [37] to 106 recreation and sports facilities [39].”

19

Reviewer comment: ln 36 – I don’t understand the last bit of the last sentence – could it be written in a simpler fashion?; Author response: The line numbers on the document we have to refer to don’t seem to line up correctly with those reported by the reviewer. As such we are unsure what paragraph this is referring to. But we did make the following changes to the manuscript for the sentence we think the comment refers to. “The majority of communities were cities with only a few towns or villages as classified by Naylor et al.” was removed as the sample size description was changed and this sentence is no longer relevant.

Reviewer comment: p16 I think Figure 2 provides more information than Figure 3 and should be retained instead of Figure 3; Author response: Figure 2 has been retained and a legend has been added for more information

Reviewer comment: p18, ln 11 – is this heading needed; report secondary outcomes in same order as in the methods; Author response: Heading “effects of intervention” has been removed from page 18

Reviewer comment: ln 20-22 the sentence beginning ‘the intervention is…’ is clumsy, as is; Author response: Changes made to the manuscript were “It was estimated that the intervention resulted in a cost reduction associated with short term risky alcohol behaviour valued at approximately AUD$13.8 million, or about AUD$3,823 for a typical club.”

Reviewer comment: ln 30-33 – please rewrite these sentences more clearly to aid the reader’s understanding of the points being made; Author response: Changes made to the manuscript were “The proportion of club members who reported purchasing fruit and vegetable items from sports club canteens was presented by Wolfenden et al [38]. This study found a significant increase in fruit and vegetable items being purchased among members from intervention clubs compared to members from control clubs (OR: 2.58; 95%CI 1.086.18; p=.033).”

Reviewer comment: ln 24 could you explain what a cost ration of 4.2 means in practice?; Author response: Changes made to the manuscript include: “Meaning, for every $1 spent on implementing the program, $4.20 would be expected to be returned to the Australian economy.”

Reviewer comment: ln 41, is 25% to 24% really a reduction?;

20

Author response: Changes made to the manuscript include “compared to almost little change (25% to 24%), among control club members over this period (OR: 0.63; 95%CI 0.40 -1.00).”

Reviewer comment: ln 45, please explain a CATI survey the first time it is used; Author response: Manuscript changed to include definition of CATI for the first time it was used in the text of the document (page 19) not just the table.

Reviewer comment: p 19 ln 9-11, please supply numbers of players, senior teams and spectators for intervention and control clubs. Author response: All requested information has been added to Table 1

Reviewer comment: Discussion: p19 ln42-49 is this paragraph in the wrong place, should it go before the previous paragraph?; Author response: The line numbers on the document we have to refer to don’t seem to line up correctly with those reported by the reviewer, so we are unsure what paragraph this is referring to.

Reviewer comment: ln 38-41 – I think you still need to comment on how the current study fits into the context of previous work; Author response: The line numbers on the document we have to refer to don’t seem to line up correctly with those reported by the reviewer, so we are unsure what paragraph this is referring to. We believe, however that the second paragraph of the discussion elaborates in detail the findings of reviews of implementation studies in other settings, to contextualise the effects reported in this review. As such, we have made no further edits to the discussion.

Reviewer comment: ln 48-50 is misleading – there is some evidence of cost effectiveness; what about adverse events and individual impact – you should discuss these outcomes too; Author response: Changes made to the manuscript: “Similar to other implementation reviews [21,47], this review found little evidence of the assessment or reporting of cost or cost effectiveness of health promoting policies or practices, with only one study including this as an outcome. Further, while club revenue was assessed as a potential adverse event in two trials, little consideration was given to a range of potential unintended adverse consequences to sporting organisations, their staff or players among included trials. As policy makers and practitioners weigh the beneficial effects of interventions (or implementation strategies) with their costs and risk of adverse effects, research to address this evidence gap is warranted.”

Reviewer comment: p20, ln 8-10 club participation rate is a new result- it should not be discussed unless it is in the results section – the same goes for study design (i.e. self-selection of clubs);

21

Author response: Section was removed from manuscript

Reviewer comment: ln 24-26 the sentence beginning ‘Therefore, consideration…’ is clumsy; Author response: Changes made to the manuscript “However unlikely, the potential of eligible studies being missed through the use of this search filter needs to be considered.”

Reviewer comment: ln 39 ‘ alternate’ is a strange word to use; Author response: ‘alternate’ as removed from the manuscript

Reviewer comment: ln 40 don’t is not formal writing, and could you also give an example of the types of studies that would be included in a different review? Author response: Changes to the manuscript have been made – changed don’t to “do not” added “(observational studies).”

22