Connected Health Symposium 2015 www.iproc.org/2015/1/e11. DOI: http://doi.org/10.2196/iproc.4695. Abstract. Background: Since 2008, GHDonline.org has ...
Professional Virtual Communities for Health Care Implementers: Impact of Participation on Practice Marie Connelly, Aaron Beals, Aaron VanDerlip, Koundinya Singaraju, Rebecca L. Weintraub Connected Health Symposium 2015 www.iproc.org/2015/1/e11 DOI: http://doi.org/10.2196/iproc.4695
Abstract Background: Since 2008, GHDonline.org has provided a platform of professional virtual communities (PVCs) for health care implementers around the world to connect and discuss delivery challenges. Initially focused on low-resource settings internationally, GHDonline received funding from the Agency for Healthcare Research and Quality (AHRQ) in 2013 to expand the platform and launch the US Communities Initiative (USCI), creating PVCs for US-based health care professionals working with underserved patient populations. Objective: Over the course of the three-year funding period, we established four PVCs focused on population health, quality and safety, costs of care, and delivery innovations. We aim to facilitate the dissemination and translation of evidencebased resources and novel approaches to delivering care. We seek to understand the impact that participation in these PVCs has on the implementation and integration of best practices in care delivery around the country. Methods: Each PVC is supported by a team of expert moderators who guide and shape community goals, content, and programming. GHDonline works closely with these moderators to organize virtual Expert Panels (week-long, asynchronous online conferences), which facilitate the spread of evidence-based resources and, through dialogue with experts, educate members on strategies for adapting these tools for a range of delivery settings. Our impact evaluation includes three methods: analysis of site data, member surveys, and phone interviews. Site data shows the scope of and engagement in these PVCs. Surveys, fielded before and after each Expert Panel, assess members’ knowledge of and ability to implement relevant best practices. Individual interviews identify examples of PVC participation impacting practice, as well as opportunities to improve the PVCs themselves. Results: While evaluation efforts are ongoing, current survey data shows a majority of respondents, 91% (149/163), found information shared in Expert Panels relevant to the populations they serve. A strong majority, 73% (127/175), report an intention to make changes in their practice, and 47% (81/172) report implementing changes based on knowledge gained through PVC participation. We randomly selected active members to participate in interviews and have completed 50 interviews to date. A significant majority of interviewees, 82% (41/50), recommended GHDonline to colleagues, and 60% (30/50), indicated they are making changes in their practice based on information gained through PVC participation.
Conclusions: Recognizing the limitations of self-reported surveys and interview responses, and the preliminary nature of our current findings, we believe these results show strong potential for PVCs to facilitate dissemination and translation of evidence-based practices and improve care delivery in the US.
Introduction Background
The challenge of translating health care research findings into practice is well documented [1]. In 2000, Balas and Boren found it took an estimated 17 years [2] for clinical evidence to be fully integrated in practice. While there is likely some variation in the length of “lag time”[3] for different interventions, the World Health Organization (WHO) has identified access to information and knowledge generation as critical steps to bridge the disconnect between evidence and practice in health care—also called the “know-do gap” [4,5]. This disparity is not new, but effective dissemination strategies are urgently needed to accelerate improvements in population health [6]. Groups like the Agency for Healthcare Research and Quality (AHRQ), the Institute of Medicine (IOM), and the US Preventative Services Task Force have built comprehensive, online resource libraries in response. To ensure that the availability of evidence-based knowledge translates into practice, robust knowledge management systems and collaboration among health researchers and practitioners is critical. Business strategists define knowledge management as a sequence of processes to generate, disseminate and utilize knowledge [7]. Practitioners who understand each other’s stories, difficulties, and insights are the direct “carriers” of knowledge who give it value by sharing it with patients, explaining its applicability to others, and expanding upon it. Without the opportunity for rich discussions to occur between professionals from disparate settings and sectors—promoting knowledge creation, categorization, and transfer; individual accountability; and feedback loops—the potential of information to have real-life impact is diminished. Professional virtual communities (PVCs)—online professional networks—create these opportunities for knowledge management and exchange across multiple organizations, industries, and geographies. Professional networks are vital to diffusing information, identifying new research questions, and translating and testing new tools. PVCs for Health Care Delivery
In 2008, the Global Health Delivery Project (GHD) at Harvard University built GHDonline to systematize the translation and dissemination of evidence-based health information into practice. Considering the lengthy process of conducting, reviewing, and implementing research, GHD aimed to diffuse knowledge by enabling researchers and practitioners to share all forms of data, expertise and resources widely and quickly. The GHDonline PVCs address common challenges in health care delivery— integrating new treatment protocols, quality improvement measures, and emerging
research—within an open-access online platform. Built on this foundation of creating public goods, GHDonline has developed an approach to community management focused on fostering high-quality, dynamic conversations amongst a group of diverse health care professionals. Initially developed for professionals in low-resource settings internationally, GHDonline received funding from AHRQ in 2013 to launch the US Communities Initiative (USCI), a series of PVCs for US-based health care professionals working predominantly with underserved patient populations. Since then, GHDonline has established PVCs focused on population health, quality and safety, costs of care, and delivery innovations (see Figure 1). Today, nearly 16,000 health care professionals use these virtual communities to share advice and disseminate proven practices and tools. Target users of the GHDonline PVCs work in a wide variety of roles as clinicians, researchers, program officers, engineers, technical specialists, community health workers, and policy-makers.
Figure 1. Example of a GHDonline PVC focused on quality and safety. Through this work, we seek to understand the impact of PVCs on the implementation and integration of best practices in care delivery around the country.
Methods Each GHDonline community is built in close collaboration with a group of expert moderators who guide and shape community discussions, programming, and events. Moderators ensure interactions in the PVCs are productive, high quality, and relevant to the diverse needs of members living and working in the US and around the world.
GHDonline encourages leaders in the field to share their expertise during online Expert Panel discussions—virtual, week-long events that address key issues in the field (see Figure 2). Panelists share their experiences and answer member questions in a text-based, asynchronous discussion. This “low tech” format accommodates a wider audience and allows members and panelists greater time to reflect on their questions and comments, leading to a richer dialogue between colleagues. Like GHDonline PVCs, these Expert Panels are publicly available and accessible at no cost.
Figure 2. Example of a recent Expert Panel on GHDonline. After the week of the event, key points from the conversation are summarized in short, peer-reviewed Discussion Briefs. In addition to highlighting best practices, Discussion Briefs feature vital references and identify areas for further research and exploration. These documents are reviewed by experts prior to publication and made available exclusively to GHDonline members for reference and dissemination. Intervention
As part of USCI, GHDonline has organized 14 Expert Panels (see Table 1). These events are focused on evidence-based resources from AHRQ, the IOM, the Commonwealth Fund, and other organizations publishing high-quality references for health care professionals in the United States. Panelists are invited based on their expertise as researchers, implementers, and policy advocates. Discussions address the evidence available to support these best practices, and panelists share experiences adapting recommendations for a wide range of patient populations. To understand the impact of individual Expert Panels on practice, GHDonline conducts a series of surveys around each event. Surveys are fielded one week prior
to the event to identify members’ familiarity with relevant evidence-based resources before discussion. A second survey is conducted during the week following the Expert Panel to assess the immediate impact on members’ understanding of the topic and their perceived ability to implement best practices, as well as their likelihood of making changes in their practice. A final survey is conducted three months following the Expert Panel to understand whether members have begun making changes in their practice. (See Appendix 1 for examples of each survey.) To gain a more complete understanding of PVC participation on member practice, GHDonline also conducts one-on-one phone interviews with a randomly selected cohort of active community members. Phone interviews are conducted following a brief, 10-question discussion guide, and typically last 7-25 minutes. Members who complete a phone interview receive a $20 Amazon.com gift card. This evaluation approach was reviewed by the Brigham & Women’s Hospital Institutional Review Board and approved as exempt. Participants and Recruitment
Participation in GHDonline PVCs and Expert Panels is open to all, but PVC and Expert Panel topics have been selected specifically to meet the needs of US-based health care professionals working with underserved patient populations. Survey participation in each pre-, post-, and follow-up Expert Panel survey is also open to all, but to increase responses from our core audience, those members who have contributed to, or actively followed, an Expert Panel are contacted individually with an opportunity to take surveys following these events. Those who have participated in one-on-one interviews are also emailed directly and invited to complete each survey. GHDonline recruited 500 active USCI members for one-on-one phone interviews. Members must work in the United States and have joined and actively followed, or contributed to, discussions or Expert Panels related to USCI. Members who decline to participate in phone interviews, or who fail to respond to invitations to participate on multiple occasions, are removed from the pool of interview subjects.
Results While evaluation efforts are ongoing (GHDonline continues to host Expert Panels as part of the US Communities Initiative), preliminary findings show that members’ participation in GHDonline PVCs and Expert Panels has positively impacted their knowledge of, and ability to implement, best practices. Preliminary data from pre-, post-, and three-month follow-up Expert Panel surveys, combined with responses from 50 one-on-one phone interviews, show that members are typically eager to make changes in their practice based on knowledge shared during PVC discussions and Expert Panels, though it is unclear whether significant changes have been implemented during the evaluation period. Survey Findings
Table 1 shows the number of respondents for each Expert Panel survey fielded to date. Survey data shows a majority of respondents reported an increase in their overall knowledge of best practices following these Expert Panels. While members
typically reported a need for some additional assistance to implement relevant best practices, a strong majority, 73% (127/175), reports an intention to make changes in their practice, and 47% (81/172) report implementing changes based on knowledge gained through PVC participation. Table 1. Total number of responses to Expert Panel surveys. Expert Panel PrePostFollow-up surveys surveys surveys Principles and Challenges of Health 42 12 NAa Innovation Connected Health for Providers 19 32 55 Connected Health for Patients 26 5 2 Integrating Mental Health in Primary Care 85 22 27 Settings: US & Abroad Opportunities for Impact: Community Health 19 15 8 Workers in the US Care Coordination & the Iora Health Model 78 8 11 Improving Health Equity through Health IT 141 28 16 in the US and Internationally Primary Care Integrated, Complex Care 86 13 5 Management Launching a Health Literacy Initiative: 100 18 7 Implementation Strategies Reducing Readmissions, Improving 44 3 12 Transitions in Care Quality Improvement Strategies for 29 6 13 Ambulatory Care Settings Community Health Initiatives to Improve 17 3 1 Health Insurance Enrollment Health Literacy to Patient Engagement 31 11 12 Diagnostic Error Reduction 47 4 4 GHDonline continues to organize and host Expert Panels as part of the USCI and will share results from future surveys at the Connected Health Symposium in October. Interview Findings
The analysis to date shows a majority of interviewees, 60% (30/50), are making changes in their practice based on information gained through participation in the PVCs. Members frequently cite interaction with experts and the high-quality nature of discussions as reasons for recommending GHDonline to colleagues and continuing to interact with the PVCs. As we conduct further interviews, we will Final survey not conducted, as immediate implementation of innovative principles was not expected. a
collect specific examples of changes in practice that have occurred as a result of participation in GHDonline PVCs and Expert Panels.
Discussion Principal Results These findings, while preliminary, show that an overwhelming majority of GHDonline Expert Panels have positively impacted members’ knowledge of evidence-based resources, and increased their likelihood to implement changes in their practice. Limitations
While survey respondents and interview participants have little incentive to overstate their ability and likelihood of making changes in their practice, we recognize the self-reported nature of this data may impact the reliability of our findings. Additionally, while the overall sample pool (1117 total pre-, post-, and follow-up survey responses, 50 phone interview subjects at present) is substantial, responses to individual surveys were occasionally quite low. In some cases, this low response rate may limit our ability to fully assess the impact of a specific Expert Panel, but we believe these data are still relevant to determining the overall impact of PVC engagement on changes in practice. Future Work
In the coming months, we will be hosting additional Expert Panels, fielding associated surveys, and conducting a significant number of phone interviews. We intend to update our findings to include analysis of these new events for attendees at the Connected Health Symposium and also incorporate these data into a final paper for submission to the Journal of Medical Internet Research. To better assess changes in care delivery and whether such changes positively impact health outcomes for patients, future studies should explore the impact of health care professionals’ PVC participation on select patient health metrics. To our knowledge, such studies have not been conducted to date. Conclusions
Our findings show that health care professionals who participate in PVCs make positive shifts in practice and further the dissemination and translation of evidencebased resources, thereby reducing the “know-do gap,” which remains a significant challenge in health care delivery. Through discussion with colleagues and experts with diverse implementation experience, GHDonline members are able to learn about best practices, identify strategies for translating these practices to target populations, and understand what is necessary to begin integrating these recommendations into their own care delivery systems.
Acknowledgements
This work has been supported by a grant from AHRQ, a division of the US Department of Health and Human Services, to study the role of professional virtual communities in disseminating best practices. The authors thank the many members of GHDonline for their contributions to the community, as well as the team of GHDonline moderators who support and guide the GHDonline PVCs. The authors would also like to thank Keri Wachter and Jessica Ludvigsen for their help reviewing and proofing this manuscript. Conflicts of Interest
The authors have no conflicts to declare. Abbreviations
AHRQ: Agency for Healthcare Research and Quality GHD: Global Health Delivery Project at Harvard University IOM: Institute of Medicine PVC: Professional virtual community USCI: US Communities Initiative WHO: World Health Organization
References 1. García C, Green L, Hiatt R, Ottoson J. Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health; 2009 (30):151-174. PMID:19705558 2. Balas EA, Boren SA. Managing clinical knowledge for healthcare improvement. In: Bethesda MD, editor. Yearbook of Medical Informatics. National Library of Medicine; 2000. p. 65-70. 3. Grant J, Morris Z, Wooding S. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med; 2011 (104): 510-520. PMID:22179294. 4. Chunharas S, Lansang MA, Pablos-Mendez A, Shademani R, Tugwell P. Knowledge translation in global health. Bull World Health Organ; 2005 (10): 723. PMID:16283043 5. Chen L, Evans D, Evans T, Sadana R, Stilwell B, et al. World Health Report 2006: Working together for health. Geneva, WHO; 2006. ISBN:9241563176. 6. Yarney G. Excluding the poor from accessing biomedical literature: a rights violation that impedes global health. Health Hum Rights; 2008 (1): 21-42. PMID:20845828. 7. Wenger E. Knowledge management as a doughnut: Shaping your knowledge strategy through communities of practice. IBJ Online; 2004 (68): 1. http://iveybusinessjournal.com/publication/knowledge-management-as-adoughnut/. Archived at: http://www.webcitation.org/6YWTI3vOL
