BJUI
Partial nephrectomy online: a preliminary evaluation of the quality of health information on the Internet
BJU INTERNATIONAL
Angela M. Fast*, Christopher M. Deibert*, Celia Boyer‡, Gregory W. Hruby*† and James M. McKiernan* *Department of Urology, †Department of Biomedical Informatics, Columbia University Medical Center, New York, NY, USA, and ‡Health On the Net Foundation, Geneva, Switzerland Accepted for publication 31 July 2012
Study Type – Therapy (evaluation of information materials) Level of Evidence 3b
OBJECTIVE • To further evaluate the quality of information available on the Internet with regard to the management of localized renal cancer, we evaluated websites providing information on ‘partial nephrectomy’ in conjunction with the Health On the Internet (HON) Foundation. Many patients now utilize the Internet as a resource to provide further information on disease, treatments and outcomes, and health information on the Internet is largely unregulated. Inaccurate information may contribute to unrealistic expectations and dissatisfied patients. PATIENTS AND METHODS • A google.com search identified the top 30 websites for the search term ‘partial nephrectomy’. • The HON Foundation evaluated each website according to the eight principles for Health on the Internet code of conduct
INTRODUCTION Traditionally, medical information has been limited to professional publications and the doctor–patient direct interaction in the physician’s office. However, many patients now utilize the Internet as a resource to provide further information on disease, treatments and outcomes. Patients now
©
What’s known on the subject? and What does the study add? Patients are highly likely to access the Internet for health information, and studies have reported that inaccurate or low-quality information may alter patients’ expectations and negatively impact informed decision-making. In a unique collaboration with the Health On the Internet (HON) Foundation, we evaluated the top 20 search results for the urology search term ‘partial nephrectomy,’ and identified the highest and lowest scoring criteria to increase awareness of areas of concern and improvement.
(HONcode) certification and reported the overall frequency of certification, as well as individual website compliance with each of the principles. RESULTS • Overall, seven (23.3%) of 30 websites met the requirements of HONcode certification and an additional two (6.7%) websites were under review to maintain their certification based on updating their resources. • The remaining 21 (70%) websites did not meet the standards for certification. • The lowest performing criteria included proper citation of medical information and a clear distinction of advertising from editorial content.
show an increasing tendency to access the Internet specifically for health information and education, with 78% of North American adults accessing the Internet, up to 86% of those adults utilizing the Internet to obtain health information, and 5% of all Internet searches involving a health-related topic [1–4]. At the same time, the incidence of small renal masses has risen and the
CONCLUSIONS • The low rate of HONcode compliance for these websites illustrates the poor quality of information that patients may encounter when researching options for nephronsparing surgery, which may have a significant impact on patient decisionmaking and treatment choices. • Physicians should be aware of the quality of Internet resources and how to best use these tools to help guide patients to websites with valid information.
KEYWORDS HONcode, Internet quality, partial nephrectomy
management of these masses has evolved as both nephron-sparing and minimallyinvasive surgery have gained favour over the past decade [5–8]. Because most small renal masses are incidental findings on abdominal imaging, patients may turn to the Internet and encounter an overwhelming array of information, treatment options and outcomes before
2012 THE AUTHORS
BJU INTERNATIONAL
©
2 0 1 2 B J U I N T E R N A T I O N A L | 11 0 , E 7 6 5 – E 7 6 9 | doi:10.1111/j.1464-410X.2012.11626.x
E765
FAST ET AL.
they first meet with a urologist to discuss their disease [9]. Of those adults who access the Internet for health information, a recent survey found that 86% consider this information to be reliable [4]. Previous studies have shown that inaccurate or low-quality information may alter patients’ expectations and negatively impact upon informed decisionmaking [10–13]. Physicians now find themselves in situations where patients have already informed themselves using this information before the office consultation. To help guide patients to reliable health information, several validated tools have been developed, such as the Health on the Internet code of conduct (HONcode) certification from the Health on the Internet (HON) Foundation, and utilization of the Internet to access high-quality information has been shown to aid informed decisionmaking and potentially improve health outcomes [13,14]. To better understand the quality of information and guide patients to these Internet resources, we collaborated with the HON Foundation to assess webpages according to HONcode quality criteria for the search term ‘partial nephrectomy’. We hypothesized that most websites would not meet the criteria for certification and generally be of mediocre quality. Websites were further stratified by affiliation and we hypothesized that commercial websites would be of poorer quality compared to academic websites.
PATIENTS AND METHODS The most commonly used Internet search engine, Google, was selected to best represent the most probable process of patient-generated Internet searching [15]. Before using the search engine, the Google browser location was set to ‘USA’ and cookies and the cache were cleared to prevent Google from using the computer’s geographical location or browsing history to inform the search results. Because previous studies have shown that most Internet web searches do not proceed past the first page of search results (10 websites), the first 30 websites were identified for the search term ‘partial nephrectomy’ [2,16]. Only websites written in English were included. Google sponsored links, video results and news
E766
TABLE 1 Brief description of Health on the Internet code of conduct (HONcode) principles HONcode principle 1 Authoritative 2 Complementarity (a) Complements (b) Mission (c) Audience 3 Privacy 4 Attribution (a) Date (b) References 5 Justifiability 6 Transparency 7 Financial disclosure 8 Advertising policy
Description Indicates the qualifications of the authors Information should support, not replace, the doctor–physician relationship Statement that the information is not meant to replace physician advice Statement of the website’s mission, its purpose and the organization behind the website Statement of the intended audience and purpose for that audience Respects the privacy and confidentiality of personal data submitted to the site by the visitor Documentation, references and dating Cites a date when a clinical page was last modified Cites the sources of published information Backs up claims relating to benefits and performance Accessible presentation and accurate e-mail contacts Identifies funding sources Clearly distinguishes advertising from editorial content
Adapted from the HONcode website [17].
articles were excluded from the search results. In a unique collaboration, we partnered with the HON Foundation (http://www.hon.ch/) to directly evaluate each of these websites containing information regarding partial nephrectomy. The HON Foundation is a non-profit, non-governmental agency based in Geneva, Switzerland, accredited by the Economic and Social Council of the United Nations and supported by the World Health Organization. The HON Foundation evaluates websites for authoritative, ethical and trustworthy Internet medical information [17]. In general practice, individual websites can apply to this organization to review and evaluate the website for HONcode certification, which requires the website to meet all eight HONcode principles listed in Table 1. The HONcode certification is free of charge. Patients can download a toolbar from the HONcode website and the HONcode symbol will be illuminated on the toolbar when a patient accesses a certified website, indicating that the website contains trustworthy health information. For the purposes of the present study, the HON Foundation collaborated by individually reviewing each of the 30 websites detected using our Google search process. The HONcode team comprised individuals with varying types of expertise, including a physician, pharmacist and legal advisor, as well as health sciences and medical
informatics specialists, to evaluate whether the website achieved the standards for HONcode certification. The HON Foundation reported certification status and website compliance with each principle for the individual websites. Websites were categorized as either academic, non-profit, commercial, physician or other, and HONcode certification (Yes vs No) was compared using Fisher’s exact test. The mean number of HONcode principles met by each category was compared by ANOVA. Academic websites were those affiliated with a university, medical society or medical journal. Non-profit websites were those operating from government funding or donations only. Websites that had industry funding or those involving the sale of medical devices or services were considered commercial, whereas websites belonging to individual physician or physician groups without university affiliation were considered as physician websites. Websites not meeting the aforementioned categories were categorized as other. Statistical analysis was performed using Stata, version 11.0 (StatCorp, College Station, TX, USA). P < 0.05 (two tailed) was considered statistically significant. No patient data were incorporated, allowing the present study to be exempt from institutional review board approval.
© BJU INTERNATIONAL
©
2012 THE AUTHORS
2012 BJU INTERNATIONAL
PARTIAL NEPHRECTOMY ONLINE
FIG. 1. Percentage of websites meeting each Health on the Internet code of conduct (HONcode) principle as defined in Table 1. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
1 2a 2b 2c 3 4a 4b 5
6
7
8
FIG. 2. Distribution of website by type. 6.7%
6.7% 36.7%
13.3%
Academic (n = 11) Commercial (n = 11) Non-profit (n = 4) Physician (n = 2) Other (n = 2)
36.7%
RESULTS A total of 30 websites were identified, with 28 websites originating from the USA, one website from New Zealand and one website from the UK, and 29 were reviewed by the HON Foundation. The website not reviewed was medical tourism, which is beyond the scope of HONcode certification. Overall, seven (23.3%) of 30 websites met criteria for HONcode certification and all seven websites had been HONcode certified before reviewing their content for the present study. An additional two (6.7%) websites were under review to maintain their HONcode certification but were not certified at the time of the present study. Both websites under review had been previously HONcode certified and were being audited for updated resources and citations. The remaining 21 (70%) out of 30 websites did not meet the standards for HONcode certification and included both websites that had been previously reviewed and websites reviewed for the purpose of the present study. Figure 1 illustrates the proportion of websites that met the individual HONcode principles of the 29 websites reviewed. The two highest scoring domains involved the complementary and transparency principles. Twenty-seven (93.1%) of 29 websites had a
©
statement identifying the intended audience and purpose (HONcode principle 2c) and 27 (93.1%) of 29 websites complied with website transparency (HONcode principle 6), in which information is accessible, clearly presented and website editors can be easily contacted. Of the lowest scoring criteria, reference citation (principle 4b) and date of last modification (principle 4a) were only attained by 16 (55.2%) and 17 (58.6%) of 29 websites, respectively. Another low scoring domain involved clearly distinguishing advertising content from editorial content, which was met by 17 (58.6%) of 29 websites. Most websites were either academic (n = 11; 36.7%) or commercial (n = 11; 36.7%) in nature (Fig. 2). Of 11 academic websites, three were HONcode certified, whereas two of 11 of commercial websites met the criteria for certification. Although only four websites were non-profit, two of these websites were HONcode certified. There were no significant differences between website classification and HONcode certification status (P = 0.21). There were two physician websites and two other websites, none of which were HONcode certified. When looking at the mean number of HONcode principles each category of websites met, academic websites met 75.2% of criteria, commercial websites met 78.2% of the criteria and non-profit websites met 86.4% of the criteria (P = 0.52).
DISCUSSION Over the past few decades, Internet usage has rapidly increased, with most adults reporting that they have used Internet to identify health information [1,18]. Although teenagers and young adults show an increasing tendency to access the Internet, usage is growing most rapidly in adults aged > 50 years, with recent studies showing that up to 67% of adults > 50 years access the Internet and that 76% of these adults search online for health information [19]. Because most urological oncology patients fall within this age group, knowledge concerning how to obtain high-quality information about their disease and treatment options has become increasingly important for these patients and could potentially empower patients to improve informed decision-making and overall satisfaction with their care [13,14].
The HON Foundation found that most websites did not meet their principles for certification and only 23.3% of partial nephrectomy websites were determined to have reliable, high-quality health information. Previous studies focusing specifically on urological oncology Internet health resources, including robotic prostatectomy prostate, kidney, testicular and bladder cancer, have reported similar certification rates (26% to 29% of websites) [1,20]. Unfortunately, these results show that there has been no improvement in the quality of health information pertaining to urological oncology on the Internet since these previous studies were published. These results illustrate the poor quality of information that patients may encounter, which could potentially influence treatment decisions and expectations, indicating the importance of utilizing resources such as the HONcode. Additionally, one report examining Internet use in patients with prostate cancer found that, although well-educated patients aged 50–59 years showed a greater probability of utilizing the Internet to obtain information on their disease, they did not show a greater probability of identifying high-quality resources compared to older, less educated men, which further shows the need for patient education on tools such as the HONcode in the urological oncology patient population [21]. Websites were consistently low in specific HONcode principles. The lowest scoring domain involved documentation, referencing and updating of material. Because content found on the Internet is largely unregulated and uncontrolled, health resources must include citations and references to ensure the integrity, accuracy and reliability of information. Another low scoring criterion involved clearly distinguishing editorial content from advertising material. Online advertisements often use the browser history and cache to target the consumer, which may make it particularly confusing for patients to distinguish editorial content from advertisements and may lead the patient away from the original website to commercial and potentially biased information [22]. Because almost half of all patients report discussing materials found online with their healthcare providers, physicians should be mindful of these particular weaknesses and educate patients to specifically look for citations, references
2012 THE AUTHORS
BJU INTERNATIONAL
©
2012 BJU INTERNATIONAL
E767
FAST ET AL.
and regularly updated resources, as well as increase their awareness of targeted advertising that patients may confuse as website content [23]. Despite these weaknesses, most websites performed well on certain HONcode criteria, including complementarity and transparency. The complementarity principle requires websites to specifically identify their intended audience and include a statement of purpose. Moreover, the website must state that the information is not meant to replace a physician’s advice but rather facilitate the doctor–patient relationship. Previous studies have shown that patients who talk to their physician about medical information found online are significantly more satisfied with their care, showing the importance of the complementarity principle [24]. Website transparency ensures website accessibility and clear presentation, as well as providing contact information. Most websites were either commercial or academic in nature. There were no differences in the rate of HONcode certification or the mean number of HONcode principles met between the different categories of websites. Although this is probably influenced by only a few websites being included in the present study, there is probably an overall low quality of information content on all websites, which is supported by the overall low certification rate found in several studies evaluating HONcode certification on various healthrelated topics [6,18,20,25,26]. Furthermore, another report on orthopaedic sports medicine Internet information found a similar rate of HONcode certification between commercial and academic websites in a significantly larger series [26]. Although various additional criteria were unfulfilled, all uncertified academic websites failed to comply with the attribution principle, which requires the proper citation, referencing and updating of website content. Website developers within academic settings should pay particular attention to the proper citation of information, updating material regularly and providing additional resources and references for patients when designing their content. There are several organizations that provide guidelines regarding the quality of Internet health information outside of the HONcode to assist with this process. For example, DISCERN, a health information quality tool originating from the National
E768
Health Service in the UK, provides a full handbook of criteria aiming to help website developers identify weaknesses and improve the quality of health information [27]. A potential way of improving the quality of health information on these academic and physician-related websites is for academic institutions or physician societies, such as the AUA or American Medical Association, to request their constituents meet the principles for HONcode certification. There are several limitations of the present study that are worthy of note. First, the Internet is dynamic and constantly changing and websites are continually being developed and updated. Additionally, search results may change on a daily basis and vary depending on user geography and browser history. Although we attempted to emulate patient behaviour in our methodology, the computer cache and cookies on the patient’s computer would inform the search results based on the patient’s browsing history and could potentially influence the websites that they access. Google requires a browser location for the search engine and we broadened the search location from a city to the country of the USA to better generalize the results. However, because the search engine was set to the USA, only English language websites were identified and reviewed with most originating from the USA, which may not be generalizable to patients outside of the USA. Second, we had the HON Foundation individually review each website in the search results. In practice, the website must apply to the HON Foundation to be considered for certification. Interestingly, all websites that achieved certification in the present study had already been certified in the past, which suggests that high-quality websites show a greater tendency to apply for HONcode certification. However, further studies are needed to substantiate this statement. Additionally, the HON Foundation reviewed only 30 websites (the first three pages of search results). Although previous studies have shown that 90% of web searches do not proceed past the first page of results (10 websites), this low number of websites may be underpowered for the statistical analysis comparing certification based on the type of website [2,16]. Another limitation is patient and physician knowledge of these tools. The HON Foundation is one of many organizations that evaluates online medical information. As Internet use for identifying
health information and educational resources becomes more widespread, physicians should educate themselves about these tools and the Internet resources associated with their specialty and the common diseases that they treat to better guide patients to appropriate resources. Assisting patients in identifying quality health information provides an opportunity for the physician to help their patients better understand their disease, as well as empower the patient. However, it remains to be determined whether directing patients to high-quality resources affects the doctor– patient relationship or treatment outcomes. In conclusion, patients are increasingly turning to the Internet for health information and education on their disease and treatment options, and having access to high-quality information may help improve informed decision-making and overall patient satisfaction. Most websites for ‘partial nephrectomy’ failed to meet the criteria for HONcode certification. These results are similar to previous studies evaluating urological oncology Internet resources and show a lack of improvement despite the increased utilization of these resources by patients. The lowest scoring criteria involved proper citations, references and updating of material, in addition to clearly distinguishing editorial from advertising content, and academic websites were no more HONcode compliant than commercial websites. Physicians should be aware of Internet information and resources, such as the HONcode, so that they can help guide their patients to high-quality websites and potentially improve overall patient care. ACKNOWLEDGEMENTS Angela M. Fast is funded by the Doris Duke Charitable Foundation. We would like to acknowledge the generous assistance from the team of reviewers at the Health on the Internet Foundation who evaluated the individual websites. CONFLICT OF INTEREST None declared. REFERENCES 1
Lawrentschuk N, Abouassaly R, Hackett N, Groll R, Fleshner NE. Health information quality on the
© BJU INTERNATIONAL
©
2012 THE AUTHORS
2012 BJU INTERNATIONAL
PARTIAL NEPHRECTOMY ONLINE
2
3
4
5
6
7
8
9
10
11
©
internet in urological oncology: a multilingual longitudinal evaluation. Urology 2009; 74: 1058–63 Routh JC, Gong EM, Nelson CP. Pediatric urology and the internet – does an uncommon topic decrease content quality? J Urol 2009; 182: 1569–74 Fox S. The Engaged E-Patient Population. Washington DC, USA: Pew Internet and American Life Project, 2008 Taylor H. Number of ‘Cyberchondriacs’ – Adults Going Online for Health Information – Has Plateaued Or Declined. New York, NY, USA: Harris Interactive, 2008 Chow WH, Devesa SS, Warren JL, Fraumeni JF Jr. Rising incidence of renal cell cancer in the United States. JAMA 1999; 281: 1628–31 Jemal A, Murray T, Ward E et al. Cancer statistics, 2005. CA Cancer J Clin 2005; 55: 10–30 Lee CT, Katz J, Shi W, Thaler HT, Reuter VE, Russo P. Surgical management of renal tumors 4 cm or less in a contemporary cohort. J Urol 2000; 163: 730–6 McKiernan J, Simmons R, Katz J, Russo P. Natural history of chronic renal insufficiency after partial and radical nephrectomy. Urology 2002; 59: 816–20 Janzen NK, Kim HL, Figlin RA, Belldegrun AS. Surveillance after radical or partial nephrectomy for localized renal cell carcinoma and management of recurrent disease. Urol Clin North Am 2003; 30: 843–52 Risk A, Petersen C. Health information on the internet: quality issues and international initiatives. JAMA 2002; 287: 2713–5 Eysenbach G, Powell J, Kuss O, Sa ER.
12
13
14
15
16
17
18
19
20
Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 2002; 287: 2691–700 Menon M, Bhandari M. Unhappy patients: musings of two surgical nihilists. Eur Urol 2008; 54: 723–5 Woolf SH, Chan EC, Harris R et al. Promoting informed choice: transforming health care to dispense knowledge for decision making. Ann Intern Med 2005; 143: 293–300 Finkle-Perazzo D, Jetha N. Online resources to enhance decision-making in public health. Chronic Dis Inj Can 2011; 31: 172–5 Burns E. Almost 12 Billion US Searches Conducted in July. New York, NY, USA: Search Engine Watch, 2008 Colleen Reed. Blended Search Results Study. Watertown, MA and San Francisco, CA, USA: iProspect, 2008 Health on the Net Foundation. HON code of conduct (HONcode) for medical and health. 2011. Available at: http:// www.hon.ch/HONcode/. Accessed August 2011 Ahmed OH, Sullivan SJ, Schneiders AG, McCrory PR. Concussion information online: evaluation of information quality, content and readability of concussion-related websites. Br J Sports Med 2012; 46: 675–83 Zulman DM, Kirch M, Zheng K, An LC. Trust in the internet as a health resource among older adults: analysis of data from a nationally representative survey. J Med Internet Res 2011; 13: e19 Alkhateeb S, Lawrentschuk N. Consumerism and its impact on robotic-assisted radical prostatectomy. BJU Int 2011; 108: 1874–8
21 Buntrock S, Hopfgarten T, Adolfsson J, Onelov E, Steineck G. The Internet and prostate cancer patients: searching for and finding information. Scand J Urol Nephrol 2007; 41: 367–74 22 Evans DS. The online advertising industry: economics, evolution, and privacy. J Econ Perspect 2009; 23: 37–60 23 Madden M, Fox S. Finding Answers Online in Sickness and in Health. New York, NY, USA: Pew Internet and American Life Project, 2006 24 Himmel W, Meyer J, Kochen MM, Michelmann HW. Information needs and visitors′ experience of an Internet expert forum on infertility. J Med Internet Res 2005; 7: e20 25 Fast AM, Deibert CM, Hruby GW, Glassberg KI. Evaluating the quality of Internet health resources in pediatric urology. J Pediatr Urol 2012; [Epub ahead of print] 26 Starman JS, Gettys FK, Capo JA, Fleischli JE, Norton HJ, Karunakar MA. Quality and content of Internetbased information for ten common orthopaedic sports medicine diagnoses. J Bone Joint Surg Am 2010; 92: 1612–8 27 Charnock D. The DISCERN Handbook: Quality Criteria for Consumer Health Information on Treatment Choices. Abingdon: Radcliffe Medical Press Ltd, 1998 Correspondence: James M. McKiernan, Department of Urology, Herbert Irving Pavilion, 11th Floor, 161 Fort Washington Avenue, New York, NY 10032, USA. e-mail:
[email protected] Abbreviations: HON, Health on the Internet; HONcode, Health on the Internet code of conduct.
2012 THE AUTHORS
BJU INTERNATIONAL
©
2012 BJU INTERNATIONAL
E769
