Stakeholder engagement in patient-centered

Expert Review of Pharmacoeconomics & Outcomes Research Downloaded from informahealthcare.com by 86.6.202.5 on 03/24/14 For personal use only.

Perspective

Stakeholder engagement in patient-centered outcomes research: high-touch or high-tech? Expert Rev. Pharmacoecon. Outcomes Res. Early online, 1–10 (2014)

Danielle C Lavallee1, Paul Wicks2, Rafael Alfonso Cristancho1 and C Daniel Mullins*3 1 University of Washington Department of Surgery, 1107 NE 45th St, Suite 502, Seattle, WA 98105, USA 2 PatientsLikeMe, 155 Second Street, Cambridge, MA 02141, USA 3 University of Maryland School of Pharmacy, Baltimore, MD 21201, USA *Author for correspondence: Tel.: +1 410 706 0879 [email protected]

Patient and stakeholder engagement enhances the meaningfulness of patient-centered outcomes research. Continuous engagement of diverse patients helps to achieve representativeness and to avoid tokenism, but is perceived as challenging due to resource and time constraints. The widespread availability of the internet, mobile phones, and electronic devices makes ‘high-tech’ solutions appealing, but such approaches may trade-off larger sample sizes for shallower engagement and/or skewed perspectives if most participants reflect users of technology. More traditional ‘high-touch’ solutions such as in-person interviews, focus groups, and town hall meetings can provide qualitative and sociological context and potentially more in-depth insights from small numbers of patients, but such approaches are also prone to selection bias as well. We compare and contrast high-tech and high-touch approaches to engaging stakeholders and suggest hybrid processes. KEYWORDS: eHealth • innovation • patient-centered outcomes research • patient engagement • stakeholders

Over the past two decades, increasing attention has been paid to the misalignment between healthcare research and the informational needs of decision-makers [1–4]. Of greatest concern is the paucity, and at times the absence, of scientific evidence to support medical decisions made by patients and the clinicians who treat them. In response, health policy organizations and patient advocacy groups are promoting greater partnerships between researchers and the broader healthcare community in the development of research agendas as well as the subsequent implementation and dissemination of research findings [4–6]. Enhanced involvement of clinicians, policymakers, patients and other healthcare stakeholders is a growing trend in research in the USA [7–9]. When effectively carried out, such engagement results in the development of shared understanding of key issues as viewed from different perspectives [10]. The merits of engagement include improving the relevancy of research, creating transparency and identifying opportunities to translate research findings into practice in a timely and useful manner [10,11]. There is also the opportunity to informahealthcare.com

10.1586/14737167.2014.901890

avoid repeating previous harms committed by the research establishment against the population of interest such as the Tuskegee syphilis experiment or the use of Henrietta Lack’s cells without her knowledge [12,13]. Recognizing the potential benefits, funding agencies are including stakeholders in work conducted as well as making engagement with stakeholders a requirement for funding. In the USA, the Agency for Healthcare Research and Quality (AHRQ) established the first Comparative Effectiveness Research program under authorization provided in the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 [14]. One tenet of this program is inclusion of stakeholder involvement to ensure that evidence collected aligns with the needs of those applying such evidence to real-world decisions [9,14]. To highlight this focus, AHRQ funded the Community Forum in 2009, an initiative aimed to improve and expand public and stakeholder engagement in all aspects of the AHRQs Effective Health Care program [15,16]. The funding and subsequent establishment of the Patient Centered Outcomes Research Institute (PCORI) as part

2014 Informa UK Ltd

ISSN 1473-7167

1

Perspective

Lavallee, Wicks, Alfonso Cristancho & Mullins


Table 1. Examples of high-touch and high-tech approaches to stakeholder engagement. High-touch approach

High-tech approach

• Convene real-world multi-stakeholder advisory groups to discuss research governance • Host a ‘town forum’ to obtain input and discuss key issues related to a research topic • Convene real-world meetings to rank priorities for research through discussion or card-sorting • Individual meetings with relevant organizations to discuss research findings and approaches for disseminating information • Dedicated note-taker summarizes contributions from stakeholder and circulates minutes • Set up 1:1 in-person interviews with participants to get feedback on consent forms, questionnaires, research hypotheses • Invite stakeholders to help recruit from within their community • Key informant interviews to select study design and outcomes parameters • Present results of research back in town hall meeting or through paper newsletters • Engage minorities and individuals from groups who are underrepresented in high-touch engagement activities to assist with translating results

• Launch an online campaign to crowd source topics for research and a virtual advisory group meeting through telepresence (e.g., Google Hangout, GoToMeeting, WebEx) • Use telepresence to conduct a video conference with stakeholders to present and discuss research projects • Use social media platforms (e.g., Facebook), to allow voting on research priorities • Collaborate with eHealth communities (e.g., PatientsLikeMe, HealthTalk Online, Inspire communities) to develop studies using patient-generated data • Digital video/audio capture used to web-stream or archive discussions, collaborative notes taken with Google Docs • Post consent forms, questionnaires, research hypotheses on a collaborative website, for example, Google Docs and allow commenting or even editing by community • Invite stakeholders to join social media campaign (e.g., Thunderclap) • Continually update community on research progress through email newsletters and text messaging, summarize results in videos, publish open access and share links

of the Affordable Care Act in 2010, further heightened the national focus on stakeholder-informed, specifically patientinformed research. Embedded in both the definition of patientcentered research and criteria for funding is the direct involvement and engagement of patients, caregivers, clinicians and other stakeholders in the development, design, conduct and dissemination of research [17,18]. Similar initiatives in other developed nations such as the UK’s National Institute for Health Research refer to such activities as patient and public involvement and involves members of the public in research funding decisions, training for researchers and even empowering patients to lead their own research studies [19–21]. Funding organizations including PCORI and AHRQ require applicants to clearly demonstrate meaningful stakeholder engagement in the development and implementation of research as a criterion for funding. The breadth of such engagement in research is only expected to increase. During the November 2013 PCORI Board Meeting, the Board of Governors approved a resolution for committing up to US$1.03 billlion in research funding for the next 2 fiscal years [22]. While this offers the hope that more impactful research will be developed resulting in better evidence and ultimately better healthcare, there also exists a great risk for superficial engagement that fails in developing more informed research. The evidence on meaningful involvement and engagement of stakeholders in PCOR is evolving both in experience as well as opportunities to access and connect with diverse audiences [9,17,23]. Traditional approaches to stakeholder engagement are rooted in methods involving direct contact with participants through meetings, focus groups and one-on-one discussions, or ‘high-touch’. While this approach provides the advantage of

relationship building, it is off-set by the often demanding timelines in research as well as limitations in available resources for coordinating such activities. Emerging approaches are connecting with virtual communities and leveraging advances in technology to conduct engagement through web-based platforms, social media and video conferencing, or ‘high-tech’. This allows for rapid engagement across geographically diverse populations yet limits the in-person connection and risks exclusion of certain populations. To inform researchers actively engaging stakeholders in their work, we discuss different approaches, both high-touch and high-tech, weighing the pros and cons of each and suggest hybrid processes.

doi: 10.1586/14737167.2014.901890

High-touch

Successful research partnerships are built on trust, respect, accountability and transparency [24,25]. To achieve this, one might argue that a ‘high-touch’ approach, or one in which stakeholders and research teams directly engage through person-to-person interaction, is necessary. Examples of high-touch activities are included in TABLE 1. A high-touch approach allows for direct, facilitated discussion. In addition, it provides a forum where relationships and networks are developed and fostered over time. A parallel to stakeholder engagement in PCOR can be made to community engagement in Community-Based Participatory Research (CBPR). CBPR emphasizes the active involvement and partnership of non-academic researchers, specifically members of the community, in all aspects of the research process to more effectively reduce health disparities [26,27]. CBPR harbors a number of examples of high-touch approaches for integrating research and practice to advance public health [11,26,28]. For

Expert Rev. Pharmacoecon. Outcomes Res.


Stakeholder engagement in patient-centered outcomes research

example, many partnerships ensure the inclusion of community members as part of governing and advisory boards to inform and guide decisions on research activities pursued, to participate in developing research protocols and to assist in bridging communication between research activities and the broader community [29]. Another approach described is employing members of the community as part of the research team to carry out research activities, such as leading focus groups, conducting surveys or conducting education as part of study interventions [29–31]. An example of such engagement is the Zuni Health Initiative developed to address health disparities and promote disease prevention for chronic disease affecting members of the Zuni Indians [30]. The initiative is based on a relationship cultivated between researchers at the University of New Mexico and the Zuni community. Research activities such as one-on-one interviews with community members, conduct of focus groups and integration of health programs directly involves community health representatives. The success of the program is largely based on the trusted relationship built overtime and adherence to key principles of engagement including direct involvement of the community early in the research process and respecting the importance of partnership in the conduct of research [30]. Involving stakeholders as part of workgroups and committees in the development of research guidance is another example of high-touch engagement. The Center for Medical Technology and Policy involves stakeholders as an integral part in developing effectiveness guidance documents (EGDs) [32]. EGDs are meant to provide clinical researchers and product developers with guidance on how to design clinical studies that will produce the evidence needed for decision-making by the healthcare community [33,32]. The process for developing EGDs includes direct collaboration with product developers, payers, clinicians, patient representatives, researchers and other relevant experts. Engagement occurs in three distinct ways: the development of a Technical Working Group comprising 8–12 individuals to lead the activity, convening a symposium of experts, stakeholders and decision-makers to discuss key issues on the specific topic and finally, requesting input from the public on the draft document prior to finalizing recommendations [27]. This tiered approach allows for iterative discussion and multiple opportunities for input as the document develops and is finalized. The involvement of skilled facilitators neutral to the given topic assists with ensuring open and balanced discussion across the different voices represented. Benefits of high-touch approaches

High-touch promotes trust, a critical component for motivating participation. The importance of relationship building through personal interaction is most notable, where mistrust of research and academia exists [30,34,35]. This is especially notable in minority populations such as with American Indian and African American communities, where past mistreatment under the context of research occurred and in conditions where concerns of social stigmas may prohibit participation [30,34,36]. Making a concerted and genuine effort to understand the needs of the informahealthcare.com

Perspective

patient population through engagement and collaboration with patient advocacy organizations, support groups, civic leaders and community organizations will assist in building trust in the research team and intent of the research activities. Further, it will help identify topics of greatest importance to study as well as elucidate opportunities to disseminate research findings effectively [30,31]. High-touch engagement provides opportunities to obtain and delineate more nuanced input, such as sarcasm or body language that cannot easily be obtained using more high-tech means. Through focus groups or one-on-one interviews, researchers can learn directly and indirectly from patients based on the nonverbal cues and facial expressions of participants. Moreover, participants are able to clarify in real time what the other is attempting to communicate so as to avoid misinterpretation. Finally, high-touch engagement encourages cultural competence. It is not uncommon for people to go into research with preconceived ideas or biases. Culture, including customs, beliefs and values, influences how individuals access healthcare, when and how they seek treatment and how information is obtained and interpreted [35,37]. High-touch engagement provides an opportunity for researchers to learn more about cultural dynamics including how to more effectively communicate, interact and support people from ethnic and socioeconomically diverse backgrounds. Barriers to high-touch approaches

The barriers of a high-touch approach to engagement are equally important to consider. Most notably, this approach is hindered by the resources necessary to plan and conduct activities on behalf of both the organizer as well as the participants [9,23,29]. Meetings are time intensive for research teams both in planning and coordinating. This can impose challenges, especially when the timeline of research is short. In an evaluation of stakeholder engagement for prioritizing Comparative Effectiveness Research studies in genomic tests, participating stakeholders noted the concern that while the process of deliberation and discussion was important and valued, in-person meetings slowed progress leading to the recommendation that efficient practices including web-based engagement be used in order to keep pace with the changing healthcare environment [23]. In addition to time, there are financial implications including costs for travel, meeting space and honoraria for participants, if applicable. Such costs can be significant, thus impeding their feasibility. Finally, in-person activities can have financial implications for participants, particularly those outside of the traditional healthcare arena, if participation requires them to take time away from work [9,21]. High-touch activities may also limit participation. Participation may purposefully be limited to ensure the environment allows for robust discussion. By limiting numbers of participants to facilitate discussion, it is possible that not all perspectives will be adequately represented. Furthermore, the requirement to attend meetings or events in-person may limit doi: 10.1586/14737167.2014.901890

Perspective


the ability of some individuals to fully participate due to prior commitments, work or for some health conditions, which make travel or participation difficult.


High-tech

For many, technology is an integral part of life. Over the past decade, the web has evolved from static unidirectional publishing to a more interactive environment, where consumers increasingly become producers, or ‘prosumers’ [38]. With advances in mobile technology (smart phones and tablets) and Internet access becoming more ubiquitous, people are readily able to connect to each other and to expansive amounts of information. In medicine, the patient community is increasingly leveraging social media platforms, eHealth communities and personal health tracking tools and devices to better engage in their own health resulting in ‘Health 2.0’, or participatory health using web-based technology [39]. This presents the research community with unique ‘high-tech’ opportunities to engage stakeholders in research activities including study development, research prioritization, data collection and dissemination efforts. The rapid evolution in web-based technology has the potential to creatively disrupt research by opening new approaches for collecting patient-generated data; however, it remains in nascent stages of development. For instance, while it is estimated that over 40,000 health applications, or apps, for mobile devices are currently in use [40], these tend to suffer a high rate of attrition and relatively few patients use them for a sustained period [41]. The Internet has enabled simple group formation around shared interests and supports these with powerful tools that can support research [42]. For instance, the patient-powered research network PatientsLikeMe (PLM) supports patients to track quantified medical data that is important to them, share information with others and benchmark their experience against other people with similar conditions and characteristics [43]. There is even some evidence to suggest that participation in the platform might improve health outcomes, particularly among those who form many social ties with other patients like them [44,45]. This interest and involvement in tracking personal data over time provides a unique opportunity for patient-research collaborations to emerge. For example in 2009, a patient with amyotrophic lateral sclerosis (ALS) complained that the standard rating scale in her disease, the ALS functional rating scale (Revised, ALSFRS-R), had insufficient resolution at lower levels of functioning to measure her progress over time, she had been rated as a ‘zero’ at the bottom of the scale despite still having some remaining function enabled by technology [46]. Through the PLM message-board and a survey she was able to partner with researchers to elicit new rating scale items from other patients like her who would normally be considered too profoundly disabled to travel to hospital or take part in research, as their means of communication was limited to eye tracking machines or other sensors, which required painstaking hours to communicate even brief sentences. These new items, known as the ALSFRS extension items, reflected subtle functions such as using facial expression to communicate rather than speech or doi: 10.1586/14737167.2014.901890

writing, or the use of assistive technology such as digital switches [47]. The new instrument included the patient herself as a co-author, has been translated into other languages [48] and is the preferred instrument of the Department of Veterans Affairs’ ALS biorepository brain bank [49]. With support from the Robert Wood Johnson Foundation, a new platform called the open research exchange was launched in 2013. This initiative seeks to change the traditional paradigm where health outcomes are determined and developed by researchers to one that uses an open platform for developing, validating and sharing health outcome measures that better reflect patients’ experiences [50]. Researchers are able to draft patient-reported outcome items and questionnaires and get rapid qualitative feedback from patients themselves about the language used, set of responses available or whether the domains are relevant to them. In addition, the availability of a large digital community also allows for quantitative feedback such as the psychometric characteristics of the scale or how it compares against other similar measures. Patients themselves can increasingly take advantage of the means of health knowledge production to answer their own research questions. When a small Phase II study was published suggesting that the drug lithium carbonate halted the progression of ALS, over 160 patients obtained the drug off-label and tracked their data on PLM using the ALSFRS-R scale. Collaboration between the patient community and PLM researchers led to the design and conduct of an observational case-matched study, which noted no slowing of disease progression associated with lithium, a finding later corroborated through large randomized controlled trials conducted several years after the close of the PLM study [51]. More recently, patients have built on these techniques to try and unblind themselves during formal randomized controlled trials, underlining that such technologies may act as a double-edged sword [52]. Online collaborative platforms allow researchers opportunities to engage with stakeholders to obtain input and generate discussion [16]. IdeaScale is one example of a platform used by research funders and organizations [16,53]. IdeaScale allows for interactive conversations between and among community members, contains voting options allowing members to indicate preference and offers reporting and analysis tools to clearly inform the ranking and prioritization of ideas. It also allows for asynchronous participation giving members the ability to log-in at a time and place of their choosing. The National Cancer Institute is leveraging a collaborative platform as part of the Provocative Questions Campaign inciting those interested to propose topics that should be pursued in the fight to cure cancer. The comparative effectiveness research translational network (CERTAIN), a program of the University of Washington, is utilizing this approach to obtain input from their stakeholders to determine future research efforts in spine and vascular care [54,53]. Not only does this allow input from diverse audiences, but it also instills transparency in how the information is generated and discussed. Newer platforms are emerging that leverage crowd-sourcing for research development purposes. In January 2012, Transparency Expert Rev. Pharmacoecon. Outcomes Res.



Life Sciences, LLC launched a platform that allows patients, clinicians and other stakeholders the opportunity to contribute to the design of clinical trials [55]. The web-based tool allows participants to take part in selecting key aspects for the trial design (comparators, outcomes) and also comment and discuss topics relevant to the design such as inclusion/exclusion criteria and length of data collection. The goal for this platform is to increase transparency in clinical trial design, while harnessing input from a diverse group to improve the relevancy and strength of the study design [55]. Finally, social media platforms (Twitter, Facebook, Google Hangout) allow targeted outreach to key audiences. Posting messages or using hashtags are growing trends for generating discussion among groups. TweetChats, for example, allow for real-time moderated discussion on a particular topic [56]. This allows an unlimited number of participants to follow or participate in conversations of most interest. Further, since chats are conducted via the Internet, participation can reach audiences across geographic locations, local, regional, national and international. Increasingly, patient-centered medical conferences such as ‘Doctors 2.0 & You’ attract a massive online digital footprint relative to their on-site attendees by inviting online patient thought leaders to speak, to take questions from their online audience during session and to live-stream their content. In 2013, the conference had 400 in-person attendees, but 7464 tweets in a 2-day period, becoming the second most tweeted European healthcare conference of all time behind the European Society of Cardiology Congress, which had 30,000 in-person attendees [57]. Benefits of high-tech approaches

The pace in which technology continues to advance is astounding. Not only is it generating new ways for people to become involved in their own health, it is also allowing a broader community to discuss and exchange ideas about key topics. Technology is becoming more accessible and utilized and this trend is anticipated to continue. Changes in healthcare policy will likely continue to expand individual involvement in tracking and managing personal health data. The Health Information Technology for Economic and Clinical Health Act aims to improve patient safety, quality and efficiency of care through the advancement of health IT [58]. Expectations for Meaningful Use standards established under the Health Information Technology for Economic and Clinical Health Act aim to expand patient involvement in their care by requiring providers to make electronic copies of medical record securely available to patients over the Internet [58]. High-tech approaches breed transparency. Open platforms for engagement allow for communities to not only discuss key topics, but also to view comments and discussion threads. With respect to data gathering and sharing activities, patients are able to maintain control of their data allowing how and with whom data are shared. Finally, high-tech approaches can be rapidly deployed. With the ability to reach a large and diverse number of people, information and feedback can quickly be generated informahealthcare.com

Perspective

supporting demanding timelines for research. This is particularly advantageous for research activities such as topic generation and prioritization, where broad and diverse input is helpful. Barriers to high-tech approaches

High-tech approaches are not without limitations. While the use of technology is increasing across many demographics, it remains inaccessible or underutilized by others. Not all audiences have access or choose to access technology. A recent Pew Research Center study reported 15% of adults (9% of whom are 65 and older) do not access the Internet [59]. Socio-economic status, age and cultural factors may limit use of technology. Lower rates of adoption of technology among older adults are reported citing reasons such as complexity and lack of interest [60,61]. Among hard to reach populations, including minorities, those who do not speak English well or at all, those with low socio-economic status and those with physical (e.g., hearing, vision or mobility) or cognitive impairments, report preferences for personal interaction versus high-tech approaches [35]. The result of such limitations is the potential exclusion of populations. Online communities such as PLM have been found to involve participants that are younger, better educated and more likely to be female than a corresponding clinical population [62]. However, the differences were relatively minor (a few years younger, a few % points more likely to be female) and these biases can be counteracted by weighting or stratified sampling, difficult through real-word methods. Another challenge to high-tech engagement is that online communities often comprise like-minded individuals [39]. As a result, there is considerable risk that such engagement activities may be biased and not reflect a full array of opinions or perspectives. Further, without skilled facilitation, the most active participants may dominate discussion threads or activities. Resources for sustaining continued engagement using hightech solutions should also be considered. Web-based collaborative platforms and survey services are relatively userfriendly, but often are accompanied with subscription fees. Further, for online discussion forums and social media platforms to succeed, dedicated personnel are needed to maintain engagement and interaction with the community as well as troubleshoot when technical issues arise [16]. Failure to connect with the community at regular intervals with important or useful information may result in decreased participation over time. Finally, an important limitation to consider is that validating patient-reported diagnoses and related data (i.e., treatments, procedures, lab values) via online systems is not currently a simple process. This limitation can be addressed through incorporating screening surveys and/or physician verification to increase the level of confidence to approach that of traditional studies when utilizing online systems for patient recruitment [63]. Looking to the future, patients might be able to grant permission to researchers to access their electronic medical records for doi: 10.1586/14737167.2014.901890

Perspective



Table 2. Benefits of high-touch and high-tech throughout the research continuum. High-touch pros

High-tech pros

• High-touch approaches build trust through proximity (e.g., body language, contact, casual conversations) • In-person deliberation allows for intense discussion, especially around topics that may be too sensitive or contentious to be broadcast in a manner that is attributable (e.g., ‘Chatham House Rule’) • Presenting findings in person at meetings (local, regional, national) allows research teams to directly engage with audience • Personal relationships built through networking with organizations allow for targeted dissemination efforts to key audiences • High-touch approaches reduce exclusion of low literate and others who might find technology (computers, smart phones, etc.) challenging • Participating in community events such as health fairs brings results directly into the community

• High-tech approaches build trust through transparency (e.g., audit trail of who said what and when) and equality (Internet values content over traditional signals of status) • Online deliberation allows for distributed discussion over the course of days, weeks, months and can encourage discussion of sensitive topics through anonymity • Web-based platforms allows broad outreach, for example, those who can’t travel, speak or are shy/introverted • Allows for recruitment of larger sample sizes • Allows for rapid collection of data and input plus metadata such as demographics allows for sample stratification and more complex analytics • Discussing research findings through Twitter (Live Tweet Chat) allows outreach to a broader audience through retweets and hashtags • Generally cheaper and easier to organize than in-person meetings • Potential for archiving, for example, video archive, documents, Tweetstream

High-touch cons

High-tech cons

• Difficult to find appropriate meeting space (e.g., transport links, wheelchair accessible, affordable) • Meetings limited by geographic location • Difficult to schedule simultaneous meeting of disparate people • Potential for social conventions and prejudices (e.g., gender, accent, race, dress, age) to influence perceptions • Can only hear from one person at a time • May consolidate existing power-base rather than introducing new voices or disruption • Social conventions against voicing strong criticism in public, organizers may feel more ‘buy-in’ than was actually achieved

• Likely to overrepresent younger, wealthier, better educated individuals with English as first language • Greater potential for abuse by vested interests, vocal minority or ‘trolls’ • Technical complexities may be unsatisfying or off-putting • Requires organizers and participants to have technical expertise, software, equipment • May not foster the strength and intimacy of relationships generated through in-person meetings • Less of a ‘filter’ in interactions and permanent record may invite unexpectedly harsh criticism and visibility • May be limited in the ability to validate patient-reported diagnoses

the same purpose; however, such approaches might have downsides themselves, such as reducing patient comfort sharing experiences they haven’t disclosed to their medical team or that report directly on the quality of the care they experienced. Considerations for selecting high-touch, high-tech or both

Expectations that research activities build from partnerships between researchers and other healthcare stakeholders is increasingly becoming a requirement for funding. Since this deviates from the traditional paradigm for research, it is expected that research teams will seek to build new relationships to support current and future research initiatives. To ensure that engagement is both meaningful and successful, it is important that considerable attention is paid to techniques and approaches selected. Matching approach with the goals of engagement

A continuum of engagement exists varying in both the flow of information between stakeholders and research teams as well as doi: 10.1586/14737167.2014.901890

intensity of involvement [19,64]. At one end is consultation, where the flow of information is unilateral with stakeholders providing input or information about a specific topic. This is often conducted through interviews, focus groups or surveybased methods and occurs at a single time point. Collaboration, the goal of engagement for PCOR, encourages active partnership between stakeholders and research teams and requires a more dynamic and interactive approach to sustain ongoing relationships. When effectively applied, both hightouch and high-tech approaches present unique benefits (TABLE 2) and allow researchers to develop and sustain partnerships with stakeholders. Technology does not preclude the ability to establish and foster relationships often associated with personal interaction. Continued advances in technology are increasing the level and intensity in which people can interact. One might argue that across the continuum, high-tech approaches may equal the personal experience obtained via direct interaction. Online video conferencing (Skype, Facetime, Google Hangout, etc.) is one such example. Through such tools, it Expert Rev. Pharmacoecon. Outcomes Res.


Maximizing engagement & participatory experiences

Whether a high-touch or high-tech approach is selected, research teams need to plan appropriately to ensure a successful experience. TABLE 3 highlights key points for consideration to make engagement activities, both high-touch and hightech, successful. High-tech approaches to engagement require appropriate design and testing prior to launching. High-touch approaches are maximized when adequate time and resources are invested into planning and execution [65]. Clearly articulating the goals of engagement including expectations for involvement and time commitment are important. This includes informing participants how their involvement informed and shaped research, providing regular updates on the research process (e.g., history of the project, current status, future plans), and reporting results of research back to those individuals who took part and other communities who are interested or benefit from the information.

Recruitment Project kick-off

Many research studies span over time. Further, the intensity of engagement through the duration of a project is likely to ebb and flow [23]. For example, a high level of engagement is required during the initial development or implementation of a research study, whereas the intensity of engagement will lessen as the research takes shape and data collection or analysis is underway. For key points in the research process, high-touch approaches may be warranted, especially when contentious issues warrant discussion. FIGURE 1 depicts an example of a hybrid approach to sustain engagement over the life of a research study.

Conduct of research


Sustaining long-term engagement using a hybrid approach

High-touch approach

Dissemination

is now possible to directly engage in discussion with one or even multiple individuals at one time. Similar to in-person discussion, it provides the ability to view body language, important non-verbal cues that add to the dynamics of a conversation.

Perspective

High-tech approach

Initial stakeholders identified through personal networks, referrals, recommendations Broader group of stakeholders identified through social media using organization Facebook page, Twitter hashtag, YouTube channel Researchers meet with stakeholders in person individually or in small groups to introduce themselves, establish trust, and gain candid feedback Stakeholders are kept informed of progress regularly via email and quarterly tele-meetings using GoToMeeting® or WebEx®; provide input through Google Docs or chat rooms In-person annual meeting to discuss progress, conduct workgroup sessions, obtain input on key study decisions Live-stream presentations via the internet with dedicated live-Twitter feed and hashtag Present findings through hosted community meetings and events or through professional meetings

Figure 1. Example of a hybrid approach to stakeholder engagement.

Table 3. Tips to enhance engagement success. High-touch engagement

High-tech engagement

• Identify and involve stakeholders early in the research process • Create clear expectations for involvement • Allow networking opportunities where participants can learn more about each other • Conduct activities in areas that are easily accessible to all, and when possible, at times that won’t preclude participation • Establish procedures for stakeholders to provide input directly to research staff • Ensure skilled facilitators lead discussions • Identify and include key decision-makers and thought-leaders • Provide regular updates through meetings, newsletters • Work directly with stakeholders to ensure patient-facing messaging is clear and culturally sensitive

• Find your audience where they are, don’t assume the biggest consumer networks (e.g., Facebook) are what your community considers their space • Provide clear expectations on how to participate in the manner you need • Dedicate resources for maintaining communication with stakeholders in the appropriate manner for the platform, for example, Twitter – rapid turnaround time and at unusual hours • Make a ‘crisis communication’ plan for dealing with heated arguments (flame wars), spam, trolling, hacked accounts • Use multiple methods to ensure representation, for example, anonymous voting not just threaded discussions • Examine tradeoffs between anonymity and personal identification

informahealthcare.com

doi: 10.1586/14737167.2014.901890

Perspective



Expert commentary

Currently, most researchers utilize either high-tech or hightouch approaches, often based upon their scientific training without complete consideration for the alternative approaches to patient and stakeholder engagement. Moreover, many individuals do not take full advantage of advances in technology or methods for engagement, regardless of whether high-tech or high-touch approaches are employed. Given the high number of ongoing research studies aimed at developing enhanced methods for stakeholder engagement, it is imperative for PCOR scientists to think creatively about how to use evidencebased approaches that draw upon best practices from both high-tech and high-touch solutions.

the diversity of individuals sought for engagement and tailor activities accordingly. Understanding and respecting values and preferences for communication and participation is key to ensuring meaningful and productive relationships. Through bidirectional learning, PCOR scientists with experiences in high-tech and high-touch approaches to stakeholder engagement can utilize the best of the best types of strategies. Disclaimer

The findings and conclusions in this manuscript are those of the authors, who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this manuscript should be construed as an official position of AHRQ, LSDF or of the US Department of Health and Human Services.

Five-year view

Continued advances in and integration of technology into our personal lives and healthcare will result in the expansion of high-tech approaches for engaging stakeholders in research. Online access to medical records, growth of patient ecommunities and advances in personal health devices allowing individuals to track their health will further integrate technology into healthcare. Further, as patient involvement in research grows, we anticipate that initiatives promoting patient-driven data sharing to support research initiatives will grow. We anticipate this increase to be most significant among younger populations, as the integration of computers into early education is resulting in better proficiency and wider use of technology in everyday activities. Despite the advances in technology discussed, it is important for researchers to remain cognizant of

Financial & competing interests disclosure

R Alfonso and D Lavallee supported by the AHRQ grant number R01HS022959 and LSDF Grant 4593311. C Mullins supported by AHRQ PCOR R24 grant number 1R24HS022135–01. P Wicks is employed by PatientsLikeMe. The PatientsLikeMe R&D team has received research support from Accorda, Abbott, the AKU Society, Avanir, Biogen, Boehringer Ingelheim, Genzyme, Janssen, Johnson & Johnson, Kaiser Permanente, Merck, Novartis, Otsuka, the Robert Wood Johnson Foundation, Sanofi, University of Maryland, University of Michigan and UCB. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No writing assistance was utilized in the production of this manuscript.

Key issues • It is anticipated that technology will continue to become more integrated into healthcare and healthcare research knowledge exchange within the research community about best practices is critical. • No one-size-fits-all approach exists. The approach taken needs to match the goals of the project’s topic and scope. • Factors such as cost and timelines may drive the selection of approach. Explaining rationale including potential limitations incurred is important. • Engagement of diverse stakeholders is required regardless of the approach in order to assure representativeness of the population with the health condition. • Standard practices for reporting approach(es) used in research activities including challenges and successes as part of work to inform others should be developed.

References

3.

Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;12(290):1624-32

4.

IOM (Institute of Medicine). Initial national priorities for comparative effectiveness research. The National Academies Press; Washington DC, USA: 2009

Papers of special note have been highlighted as: • of interest •• of considerable interest 1.

Brook RH. Health Services Research: Is it good for you and me? Acad Med 1989; 64(3):124-30

2.

Eddy DM, Billings J. The quality of medical evidence: implications for quality of care. Health Aff 1988;7(1):19-32

doi: 10.1586/14737167.2014.901890

5.

National Health Council. Letter to the Honoroable Henry A. Waxman. US House of Representatives, Washington DC, USA;

2009 July 30. Available from: www. nationalhealthcouncil.org/NHC_Files/ Pdf_Files/CERproposal_07-31-09.pdf [Last accessed 18 November 2013] 6.

Tunis SR. A clinical research strategy to support shared decision making. Health Aff 2005;24(1):180-4

7.

Viswanathan M, Ammerman A, Eng E, et al. Community-based participatory research: assessing the evidence. summary, evidence report/technology assessment: number 99. Agency for Healthcare Research

Expert Rev. Pharmacoecon. Outcomes Res.



and Quality Publication Number 04-E022-1. August 2004.AHRQ, Rockville, MD, USA. Available from: www.ahrq.gov/ clinic/epcsums/cbprsum.htm 8.

Zerhouni E. Medicine. The NIH Roadmap. Science 2003;302:63-72

9.

O’Haire C, McPheeters M, Nakamoto EK, et al. Methods for Engaging Stakeholders To Identify and Prioritize Future Research Needs. Methods Future Research Needs Report No. 4. (Prepared by the Oregon Evidence-based Practice Center and the Vanderbilt Evidence-based Practice Center under Contract No. 290-2007-10057-I.) AHRQ Publication No. 11-EHC044-EF. Agency for Healthcare Research and Quality, Rockville, MD, USA; 2011. Available from: www.effectivehealthcare. ahrq.gov/reports/final.cfm [Last accessed 10 November 2013]

•

Presents recommendations for successful practices for stakeholder engagement.

10.

Deverka PA, Lavallee DC, Desai PJ, et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J Comp Eff Res 2012;1(2):181-94

11.

12.

13.

14.

15.

16.

Jagosh J, Macaulay AC, Pluye P, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q 2012;90(2):311-46 Gray FD. The Tuskegee Syphilis Study: An insiders’ account of the shocking medical experiment conducted by government doctors against African American Men. NewSouth Books; Montgomery, Alabama: 2013 Greely HT, Cho MK. The Henrietta Lacks legacy grows. EMBO Rep 2013;14(10): 849-9 Slutsky JR, Clancy CM. AHRQ’s Effective Health Care Program: why comparative effectiveness research matters. Am J Med Qual 2009;24(1):67-70 Carman KL, Maurer M, Mallery C, et al. Community forum deliberative methods demonstration: evaluating effectiveness and eliciting public views on use of evidence. Executive Summary. (Prepared by the American Institutes for Research Under Contract No. 290-2010-00005.) AHRQ Publication No. 14-EHC007-1-EF. Agency for Healthcare Research and Quality, Rockville, MD, USA; 2013.Available from: www.effectivehealthcare.ahrq.gov/reports/ final.cfm Mallery C, Ganachari D, Fernandez J, et al. Innovative methods in stakeholder

informahealthcare.com

engagement: an environmental scan. prepared by the American Institutes for Research under contract No. HHSA 290 2010 0005 C. AHRQ Publication NO. 12-EHC097-EF. Agency for Healthcare Research and Quality, Rockville, MD, USA; May 2012. Available from: www. effectivehealthcare.ahrq.gov/index.cfm/toolsand-resources/how-to-get-involved-in-theeffective-health-care-program/ •

This report presents innovative approaches for engaging stakeholders building from efforts outside of healthcare.

17.

Mullins CD, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA 2012;307(15):1587-8

18.

Fleurence R, Selby JV, Odom-Walker K, et al. How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Aff 2013;32(2):393-400

19.

••

20.

Nass P, Levine S, Yancy C. Methods for involving patients in topic generation for patient-centered comparative effectiveness research: an international perspective. PCORI 2013. Available from: www.pcori. org [Last accessed 13 December 2013] Presents an in-depth review of different approaches, both high-tech and high-touch for engaging patients in topic generation for research. National Institute for Health Research. Patient and public involvement plan: purpose, objectives and work (April 2013 to March 2015). July 2013. Available from: www.ccf.nihr.ac.uk/PPI/Documents/PPI% 20Plan%20Progress%20Report%2001.pdf [Last accessed 16 January 2013]

21.

INVOLVE. Available from: www.invo.org. uk [Last accessed 16 January 2013]

••

This website provides useful tools for patients, public and researchers for developing effective research collaborations.

22.

PCORI. PCORI board adopts strategic plan, approves 2014 budget and advisory panel charters, accepts revised methodology release. 2013

23.

Deverka PA, Lavallee DC, Desai PJ, et al. Facilitating comparative effectiveness research in cancer genomics: evaluating stakeholder perceptions of the engagement process. J Comp Eff Res 2012;1(4): 359-70

24.

Lavallee DC, Williams CJ, Tambor ES, Deverka PA. Stakeholder Engagement in

Perspective

Comparative Effectiveness Research: how will we measure success? J Comp Eff Res 2012;1(5):397-407 25.

Abelson J, Forest PG, Eyles J. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Soc Sci Med 2003; 57(2):239-51

26.

Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health 1998;19:173-202

27.

Collaboration with Community-Based Organizations and Agencies: a Quick Guide for UCSF Researchers. Available from: http://researchtoolkit.org.isomedia.net/ media/content/UCSF%20GuideCollaborating%20with%20Communities.pdf [Last accessed 11 December 2013]

28.

Tendulkar SA, Chu J, Opp J, et al. A funding initiative for Community-Based Participatory Research: Lessons from the Harvard catalyst seed grants. Prog Community Health Partnersh 2011;5(1): 35-44

29.

Hood NE, Brewer T, Jackson R, Wewers ME. Survey of community engagement in NIH-funded research. Clin Transl Sci 2010;3(1):19-22

30.

Shah VO, Ghahate DM, Bobelu J, et al. Identifying barriers to healthcare to reduce health disparity in Zuni Indians using focus group conducted by community health workers. Clin Transl Sci 2013. [Epub ahead of print]

31.

Shaya FT, Chirikov VV, Mullins CD, et al. Social networks help control hypertension. J Clin Hypertens 2013;15(1):34-40

32.

The Center for Medical Technology Policy. Effectiveness Guidance Documents General Overview. 2012. Available from: http:// www.cmtpnet.org/wp-content/uploads/ downloads/2012/05/EGD-Overview.pdf [Last accessed 11 December 2013]

33.

Basch E, Abernethy AP, Mullins CD, et al. Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. JCO 2012;30(34): 4249-55

34.

Christoper S, Watts V, McCormick AK, Young S. Building and maintaining trust in a community-based participatory research partnership. Am J Public Health 2008; 98(8):1398-406

35.

Mullins CD. Integrating patients’ voices in study design elements with a focus on

doi: 10.1586/14737167.2014.901890

Perspective



hard-to-reach populations. Available from: www.pcori.org [Last 9 December 2013] ••

This report provides insight on meaningful engagement of hard to reach patients.

36.

Uebelacker LA, Marootian BA, Pirraglia PA, et al. Barriers and facilitators of treatment for depression in a latino community: a focus group study. Community Ment Health J 2012;48(1):114-26

37.

The Office of Minority Health. What is cultural competency. Available from: http:// minorityhealth.hhs.gov/templates/browse. aspx?lvl=2&lvlID=11 [Last accessed 10 December 2013]

48.

Abdulla S, Vielhaber S, Ko¨rner S, et al. Validation of the German version of the extended ALS functional rating scale as a patient-reported outcome measure. J Neurol 2013;260(9):2242-55

49.

Brady CB, Trevor KT, Stein TD, et al. The department of veterans affairs biorepository brain bank: a national resource for amyotrophic lateral sclerosis research. Amyotroph Lateral Scler Frontotemporal Degener 2013;14(7-8):591-7

50.

38.

Toffler A. The third wave. Bantam books; NY, USA: 1981

39.

Hesse BW, O’Connell M, Augustson EM, et al. Realizing the promise of Web 2.0: engaging community intelligence. J Health Commun 2011;16(Suppl 1):10-31

40.

Silow-Carrol S, Smith B. Clinical management apps: creating partnerships between providers and patients. Issue Brief (Commonw Fund) 2013;30:1-10

••

Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11

52.

41. 42.

43.

Swan M. Crowdsourced health research studies: an important emerging complement to clinical trials in the public health research ecosystem. J Med Internet Res 2012;14(2): e46 PatientsLikeMe. Available from: www. patientslikeme.com [Last accessed 5 December 2013]

44.

Wicks P, Massagli MP, Frost J, et al. Sharing health data for better outcomes on PatientsLikeMe. J Med Internet Res 2010; 12:2

45.

Wicks P, Keininger DL, Massagli MP, et al. Perceived benefits of sharing health data between people with epilepsy on an online platform. Epilepsy Behav 2012;23(1):16-23

46.

Cedarbaum JM, Stambler N, Malta E, et al. The ALSFRS-R: a revised ALS functional rating scale that incorporates assessments of respiratory function. J Neurol Sci 1999; 169(1):13-21

47.

Wicks P, Massagli MP, Wolf C, Heywood J . Measuring function in advanced ALS:

doi: 10.1586/14737167.2014.901890

chatting-101-how-to-participate [Last accessed 12 December 12]

validation of ALSFRS-EX extension items. Eur J Neurol 2009;16(3):353-9

51.

Wicks P, Little M. The virtuous circle of the quantified self: a human computational approach to improved health outcomes. In: Handbook of human computation. Springer New York; NY, USA: 2013. p. 105-29 Wicks P, Vaughan TE, Massagli MP, et al. Accelerated clinical discovery using self-reported patient data collected online and a patient-matching algorithm. Nat Biotechnol 2011;29(5):411-14 This study demonstrates how eHealth communities are advancing research through data-sharing. Wicks P, Heywood J, Vaughan T. 2012; Waiting for p