Contribution of university departments of rural health to rural health ...

bs_bs_banner

Aust. J. Rural Health (2015) 23, 101–106

Original Research Contribution of university departments of rural health to rural health research: An analysis of outputs Kaniz Gausia, MBBS, PhD,1 Sandra C. Thompson, FAFPHM, PhD,1 Melissa A. Lindeman, BA, PhD,2 Leanne J. Brown, BHSc, PhD3 and David Perkins, BA (Hons), PhD4 1

Western Australian Centre for Rural Health (WACRH), University of Western Australia, Geraldton, Western Australia, 2Centre for Remote Health, Flinders University, Alice Springs, Northern Territory, 3 Department of Rural Health, University of Newcastle, Tamworth and 4Centre for Rural and Remote Mental Health, Faculty of Health and Medicine, University of Newcastle, Orange, New South Wales, Australia

Abstract Objective: To assess the research contribution of eleven University Departments of Rural Health (UDRH) which were established as a rural health workforce program in the late 1990s through analysis of peerreviewed journal output. Design and settings: Descriptive study based on validated publications from publication output reported in annual key performance indicator (KPI) reports to the Commonwealth Department of Health, Australia. Main outcome measures: In addition to counts and the type of publications, articles were examined to assess fields of research, evidence of research collaboration, and potential for influencing policy. Funding acknowledgement was examined to provide insight into funding sources and research consultancies. Results: Of the 182 peer-reviewed articles, UDRH staff members were the first and corresponding author for 45% (n = 82); most (69%, n = 126) were original research. Most publications examined included Australian data only (80%, n = 101). Over half (56%; n = 102) of the articles addressed rural health issues; Aboriginal health was the main subject in 14% (n = 26). Thirty-three articles (18%) discussed the policy implications of the research and only half (51%, n = 93) of the articles listed sources of funding. Number of authors per article ranged from 1–19, with a mean of 5 (SD = 3.2) authors per article, two-thirds of articles included authors from 2–5 universities/organisations

but only 5% of articles included an author from more than one UDRH. Conclusions: Staff from UDRHs are regularly publishing peer-reviewed articles, and research productivity demonstrated cooperation with external partners. Better collaboration between UDRH staff and others may help increase the quality and value of Australian rural health research. KEY WORDS: Australia, collaboration, peerreviewed journal article, research contribution, rural health, university department of rural health (UDRH).

Introduction

Accepted for publication 22 August 2014.

As a response to continuing rural workforce shortages and maldistribution, the university departments of rural health (UDRH) were established as part of the Australian Government Rural Health Workforce Strategy under the 1996/1997 Budget. The first seven UDRHs were established by 2000 (based in Broken Hill, Mt Isa, Shepparton, Launceston, Whyalla, Alice Springs and Geraldton,1 and four more followed (Lismore, Tamworth, Moe and Warrnambool). The UDRH program focuses on rural health education and training, and on improving access to appropriate health services for rural and remote communities. To achieve this, UDRHs provide support, education and training for health professionals in their regions, as well as assistance for urban-based health science students to undertake placements in rural and remote sites.2,3 They adopt a population health focus, contribute to the development of innovative service delivery models and advance the rural health agenda. Production of relevant research outputs is a performance indicator in their funding agreements. As

© 2015 National Rural Health Alliance Inc.

doi: 10.1111/ajr.12142

Correspondence: Dr Kaniz Gausia, University of Western Australia, Western Australian Centre for Rural Health (WACRH), 167 Fitzgerald Street, Geraldton, Western Australia, 6530, Australia. Email: [email protected]

102

What is already known on this subject: • The UDRH program focuses on rural health education, training and research on improving access to appropriate health services for rural and remote communities. • Although research and research capacity building have been core objectives of the UDRHs for over a decade, little is known about the publications of research findings by UDRHs in the peer-reviewed journals.

university-based departments, UDRHs are expected to seek additional funds through research grants and consultancies, and share new knowledge through published reports and research papers. All UDRHs received funding for 10 years under the Primary Health Care Research, Evaluation and Development (PHCRED) program for research capacity building to support rural health practitioners in gaining research skills and undertaking primary health care research.4–6 These funds ceased in December 2011. All UDRHs report on the same key performance indicators (KPIs) that include publications in peer-reviewed journals.7 Examining one KPI, in this case peer-reviewed journal publications, is only one aspect of the differences in UDRH programs. Despite methodological limitations, an earlier evaluation of the PHCRED program reported increasing numbers of UDRH publications, with the mean number of publications increasing from 2.4 in 2000 to 7.6 in 2004.8 A recent report confirms this pattern, with publications in peer-reviewed journals growing from 113 in 2004 to 310 in 2007.3 These findings indicate that performance has improved, starting from a low base. Most UDRHs employ a combination of teaching and research staff, some of whom are undertaking higher degrees by research.9 The Australian Rural Health Education Network (ARHEN) was established in 2001 as a peak body linking and representing the 11 UDRHs.9 ARHEN facilitates a number of staff networks to facilitate the sharing of knowledge and information across different rural health settings. The AHREN Research Leaders’ Network proposed an analysis of UDRH research outputs to better understand UDRH research activities. Since research and research capacity building have been core UDRH objectives for over a decade, this analysis of peer-reviewed publications provides a lens through which UDRH contribution to research can be assessed.

Methods All 11 UDRHs provided extracts from their annual KPI reports to the Department of Health and Ageing

K. GAUSIA ET AL.

What this study adds: • This analysis of peer-reviewed publications shows that the UDRH staff members were the first author and corresponding author for 45% of the articles. • UDRH staff are contributing to the evidencebase in rural, Aboriginal, health services and other areas by publishing articles in peerreviewed journals. Collaboration with external departments and organisations enhances research outputs.

(DoHA) for the years 2008–2010, which included published research articles, conference presentations and book chapters. In total, 563 peer-reviewed journal articles, 496 conference presentations, symposia and seminars, and 236 reports and book chapters were reported for the period (Fig. 1). Journal articles were verified using electronic publication databases, and an electronic bibliographic database was prepared using the Endnote software. While 2008–2010 KPI data were reviewed, after careful validation of their status as being articles in peer-reviewed journals, articles for 2010 were subjected to detailed analysis. Editorials or commentaries published in peer-reviewed journals that might or might not have been peer-reviewed were included. Editorials and commentaries in the peer-reviewed journals often present an in-depth analysis of a current issue of importance for the journal’s readership or the wider community. Full articles were downloaded using the University of Western Australia library, journal websites and Google Scholar. If articles were not accessible from subscription databases, the appropriate UDRH was asked to provide a copy. Five ‘research-active’ staff with a health background and postgraduate qualifications in health/social science from four UDRHs contributed to a working group. A pre-coded data extraction form (DEF) was developed, piloted and used to extract relevant information from the articles. In the initial pilot of the DEF, five articles were randomly selected from the list of publications and independently coded by the five academics. Challenges identified in this initial coding included distinguishing research categories, a phenomenon noted by other researchers,10 and poor inter-rater consistency on items such as whether the journal was an Australian or international journal, whether the journal was open access and whether the article included recommendations for policy. Items with poor coding reliability that were not germane to the study purpose were excluded from the revised DEF. Although policy implications were © 2015 National Rural Health Alliance Inc.

103

RURAL HEALTH RESEARCHERS’ PUBLICATION OUTPUT

FIGURE 1: Research outputs of 11 university departments of rural health as provided in key performance indicator reports, 2008–2010. (( ) Total; ( ) Articles in peer-reviewed journal; ( ) Conference/symposium/seminar; ( ) Book chapters/report/newsletter)

challenging to code with most articles lacking explicit policy implications, it was retained in the DEF because of its importance and its link to UDRH aims. Second and third rounds of pilot testing of the DEF led to further refinements. An example of items in the DEF is given below: • Rural health issue: articles that dealt with both specific (e.g. farms, forestry) and non-specific (e.g. study participants recruited in rural areas) rural health issues11 • Aboriginal health issue: articles that dealt with Aboriginal culture, health problems and health promotion/education program in Indigenous populations • Rural health workforce: articles that dealt with delivering, training and testing different models of education in rural areas • Policy implications: articles that make recommendation(s) with clear policy relevance. After finalising the DEF, peer-reviewed articles, short report, editorials and commentaries listed in the 2010 KPI report were coded by a trained research assistant with a health sciences background. All the papers included at least one UDRH staff member among the authors and quality checks were undertaken (K.G.). Meetings to discuss and resolve data extraction problems were held as necessary.

Data analysis Descriptive analysis using simple frequencies and percentages was undertaken using SPSS (IBM Corp. Released 2013; IBM SPSS Statistics for Windows, Version 22.0; Armonk, NY: IBM Corp.). The articles were examined to assess fields of research, evidence of research collaboration and potential for influencing policy. Funding acknowledgement was examined to © 2015 National Rural Health Alliance Inc.

provide insight into funding sources and success in competitive grants and research consultancies.

Results Articles in peer-reviewed journals Of 213 articles reported in 2010, 196 were published in peer-reviewed journals. In total, 182 full articles were included in the study. Sixteen articles were excluded due to duplicate reporting or because no copy could be obtained. All UDRHs published articles in 2010, with numbers per UDRH ranging from 3 to 36. UDRH staff were the first and corresponding author for 45% (n = 82) articles. Of these, 38 staff published one article, nine published two, six published three and two staff published four articles in 2010. For 48% (n = 87) of the articles, the corresponding author did not have a UDRH affiliation, and for 7% (n = 13) of articles the corresponding author’s affiliation was not available. The majority of the published articles (69%, n = 126) were original research (Table 1). Overall, 30% (n = 56) of articles were reviews, short reports, editorials, or others such as commentaries or descriptions of education programs. The proportion of articles that used qualitative data collection methods (28%, n = 50) was slightly higher than those using quantitative methods (23%, n = 41). Around a fifth (19%, n = 35) employed mixed methods approaches, and these distinctions did not apply for 30% (n = 55), including review articles or editorials. Of the 69% (n = 126) of publications based on primary data, the majority came from Australian populations (80%, n = 101). A quarter (25%, n = 32) of the articles used rural data only and 15% (n = 19) urban areas only. In a fifth of articles (20%, n = 25), the data came from overseas.

104

K. GAUSIA ET AL.

TABLE 1: Distribution of type of article, networking and funding source of university departments of rural health (UDRHs) publications

Publication type Original research Review article/meta-analysis Short report Editorial Other† Networking of UDRHs UDRH alone UDRH + university UDRH + industry partner UDRH + international partner UDRH + multisite partner (university, industry, international) Other Acknowledging funding sources (n = 93) DoHA Local/state University NGO PHCRED Non-Australian/international Scholarship

n = 182

%

126 25 16 13 2

69.2 13.7 8.8 7.1 1.1

38 51 10 17 65

20.9 28 5.5 9.3 35.7

1

0.5

45 14 12 7 6 5 4

48.4 15.1 12.9 7.5 6.5 5.4 4.3

†Commentaries and description of education work. DoHA, Department of Health and Ageing; NGO, non-government organisation; PHCRED, Primary Health Care Research, Evaluation and Development; UDRH, university department of rural health.

Collaboration and networking The number of authors ranged from 1 to 19, with a mean of five (Standard Deviation, SD = 3.2) authors per article (Table 1). Most articles (95%; n = 173) included an author from only one UDRH, although for nine articles (5%) there were authors from two to four UDRHs. Around two thirds (n = 123) of articles included authors from two to five universities and organisations. Of the 182 articles, 17% (n = 31) included some international data, reflecting UDRH collaborations with overseas researchers. One fifth (21%; n = 38) of articles had UDRH staff only as authors, and more than a third (36%; n = 65) had co-authors from the local industry (such as government and non-government organisations (NGOs)), Australian universities, overseas universities, and international organisations.

Content of publications Over half (56%; n = 102) of the articles addressed rural health issues, and Aboriginal health was the main

subject in 14% (n = 26) of papers. Twelve per cent (12%; n = 22) of articles addressed rural health workforce issues, including the education and training of rural health workers and strategies to increase the number of rural health workers. Thirty-three (18%; n = 33) articles discussed policy implications of the research, but for only 18 (10%; n = 18) were policy recommendations explicit.

Funding acknowledgement Only half (51%, n = 93) of the articles acknowledged funding sources for the research and financial support of research staff. Of those that did, 48% (n = 45) mentioned DoHA support. Some reported UDRH staff salary support from other agencies. PHCRED and NGOs were acknowledged in 7% (n = 6) and 8% (n = 7) of articles, respectively. All articles acknowledging overseas funding included international data (Table 1).

Discussion This analysis of publications provides insight into the research contribution of UDRHs. While the numbers of articles in peer-reviewed journals varied widely, all UDRHs reported these publications. Over two thirds were classified as original research and the majority used Australian data. The results show that UDRH staff were collaborative and co-authored articles, with authors from local organisations, industry partners and international researchers in more than a third of the total articles. Interestingly, collaboration between different UDRHs as assessed by co-authorship was relatively uncommon (5%, 9 articles), and can reflect the isolation and geographical distance between UDRHs, their location in different states, affiliation with different universities, their small numbers of staff, and the variety of research interests. However, there are clearly opportunities to foster more research collaboration. Since UDRH core funding relates to health workforce development, many staff are selected for their teaching skills, and focus on education and training of health students in preference to research and publishing.3,12–14 Successful publication in a peer-reviewed journal requires time, expertise and enthusiasm to negotiate ethics, data collection, analysis and writing, on top of publication complexities.15–17 UDRH staff do not have the same ready access to research skills development as their peers working on large university campuses. Publication bias can also exist, and has been widely described in terms of journals favouring articles reporting positive rather than neutral and negative results,15 reflecting reviewers’ familiarity with known topics, and institutional connections with journal editors.16,17 © 2015 National Rural Health Alliance Inc.

105

RURAL HEALTH RESEARCHERS’ PUBLICATION OUTPUT

Rurally based researchers could be disadvantaged in this. UDRH researchers were the first author in 45% of articles in this study, reflecting their skills in conducting research, disseminating new knowledge and publishing. The variation in peer-reviewed publication output among UDRHs during the study period can be associated with different UDRH strategies, the experience of the director and senior academics, staff qualifications, expertise and turnover, staff numbers, work commitments (full-time, part-time), and connections with industry partners, local organisations and international researchers. Some UDRHs might focus more heavily on research than on student support (as both are KPIs), and parent universities might have different expectations about research productivity. Furthermore, some UDRHs face difficulties recruiting experienced staff to remote areas and have few senior researchers; it might take some years for clinicians who take up an academic position to develop their research skills.9 In addition, research takes considerable time before there are outputs such as a peer-reviewed journal publication, so some outputs are products of earlier roles or are not attributed to the UDRH when a researcher has moved to a new job. Furthermore, it takes time for new academic staff to become research-active. A recent systematic review found that central to the success of the transition from clinician to academic is an identity shift that can take up to 3 years to occur.18 The review emphasised the need for ‘collegial contact and support’ to assist the transition from clinician to academic, and this might be lacking in the locations of some UDRHs given the small size of centres and their distance from the university campus and other research forums. Despite constraints on rural academics imposed by distance, clinical and teaching loads, this analysis shows that UDRH staff are contributing to and disseminating new knowledge about rural health. There is a strong contribution to rural and Aboriginal health issues, as well as rural health workforce development. Recent data suggest that Australia’s contribution to the international rural health literature is increasing both in numbers and in the prominence of selected Australian journals that include rural health topics.19 This study reveals collaboration between UDRHs staff and local industry partners (government and NGOs), universities and overseas universities/ organisations. UDRHs have contributed to increasing rural Australian health research from a low base, and this research has often occurred through collaborations with other university researchers. UDRH staff with external links authored more publications. It was also evident that PHCRED funding had supported rural research development.5 However, overall, the reporting of rural health research could be more consistent and thoughtful in terms of clear policy recommendations. © 2015 National Rural Health Alliance Inc.

We recommend that rural researchers consider framing specific policy and practice recommendations wherever appropriate. Considerable effort was put into developing a robust coding form, and one individual coded all articles with quality checks to ensure consistency. Given the timeconsuming nature of in-depth examination and coding, the study is limited by detailed analysis of only 1 year publication data, given that numbers vary by year. Also, the turnover of academic staff means that the affiliation of a staff member does not always reflect the institution where the bulk of the work was undertaken. Our analysis did not explore factors that might support higher publication rates, such as the numbers of staff with research higher degrees (MPH, PhD), research-only staff, full-time versus part-time staff, or the numbers of academic staff employed; however, all UDRHs receive the same core grant funding. Lessons were learnt in this project. The entry of journal articles into the KPI reports is often undertaken by administrative staff who might be unaware of the research publication paradigm. Effort and considerable time were necessary to ensure that all articles published in the peer-reviewed journals were included. To minimise these challenges and to enable consistent, quality reporting on KPI from all UDRHs, appropriate training is needed for staff responsible for compiling KPI reports.

Conclusions This study describes the contribution of UDRH staff in publishing articles in peer-reviewed journals, suggests that collaboration with external organisations can enhance research productivity, and indicates that there is scope for further collaboration between UDRH staff and with others to help increase the quality and significance of rural health research across Australia.

References 1 Humphreys JS, Lyle D, Wakerman J et al. Roles and activities of the commonwealth government university departments of rural health. The Australian Journal of Rural Health 2000; 8: 120–133. 2 Walker JH, Dewitt DE, Pallant JF, Cunningham CE. Rural origin plus a rural clinical school placement is a significant predictor of medical students’ intentions to practice rurally: a multi-university study. Rural and Remote Health 2012; 12: 1908–1917. 3 Department of Health and Ageing. Evaluation of the university departments of rural health program and the rural clinical school program – final report. Canberra, ACT 2601; 2008. [Cited 8 Oct 2013]. Available from URL: http://www.health.gov.au/internet/main/publishing.nsf/ Content/A3760E61F341B7F5CA257BF0001D73AB/ $File/udrheval.pdf

106 4 Brown LJ, McIntyre EL. The contribution of Primary Health Care Research, Evaluation and Developmentsupported research to primary health care policy and practice. Australian Journal of Primary Health 2014; 20: 47–55. 5 PHCRIS infonet. PHCRED strategy: research capacity building initiative, 2011; 15(6). [Cited 12 Aug 2014]. Available from URL: http://www.phcris.org.au/phplib/ filedownload.php?file=/elib/lib/downloaded_files/ publications/pdfs/news_8353.pdf 6 School of Rural Health University of Melbourne. Primary Health Care Research, Evaluation and Development (PHCRED) strategy, research capacity building initiative – 2009 business plan, 2009. [Cited 12 Aug 2014]. Available from URL: http://www.ruralhealth.unimelb.edu .au/pdf/RESEARCH/PHCRED/SRH%20PHCRED%20 Business%20Plan%202009.pdf 7 Commonwealth Government Department of Health and Ageing. University department of rural health (UDRH) program 2011–14, 2011. [Cited 16 Apr 2014]. Available from URL: http://www.health.gov.au/internet/main/ publishing.nsf/Content/CCCE659E3BE838F5CA2578 C40016DB9E/$File/Guide.pdf 8 Oceania Health Consulting. Summary report of the evaluation of the primary health care research, education and development strategy. Australia; 2005. [Cited 22 Apr 2014]. Available from URL: http://www.phcris.org.au/ phcred/evaluation.php 9 ARHEN: Australian Rural Health Education Network. Linking and supporting the university department of rural health across Australia, 2012. [Cited 12 Aug 2014]. Available from URL: http://www.arhen.org.au/about-us/what -is-arhen.html 10 Ward AM, Lopez DG, Kamien M. General practice research in Australia, 1980–1999. The Medical Journal of Australia 2000; 173: 608–611. 11 McGrail MR, Jones R, Robinson A, Rickard CM, Burley M, Drysdale M. The planning of rural health research:

K. GAUSIA ET AL.

12

13

14

15

16

17

18

19

rurality and rural population issues. Rural and Remote Health 2005; 5: 426–433. Mason J Review of Australian government health workforce programs, 2013. [Cited 8 Oct 2013]. Available from URL: http://www.health.gov.au/internet/main/ publishing.nsf/Content/D26858F4B68834EACA257BF 0001A8DDC/$File/Review%20of%20Health%20 Workforce%20programs.pdf Siggins M. Rural and remote health workforce innovation and reform strategy: draft background paper, 2011. [Cited 12 Aug 2014]. Available from URL: https://www.hwa .gov.au/sites/uploads/hwa-rural-and-remote-consultation -draft-background-paper-20110829c.pdf David L. How can education and training programs make an impact on rural and remote health workforce. In: University department of rural health symposium. 7th National Rural health conference, 2003. [Cited 8 Oct 2013]. Available from: http://www.ruralhealth.org.au/7thNRHC/ Papers/general%20papers/ARHEN%20symposium.pdf Hasenboehler EA, Choudhry IK, Newman JT, Smith WR, Ziran BH, Stahel PF. Bias towards publishing positive results in orthopedic and general surgery: a patient safety issue? Patient Safety in Surgery 2007; 1: 4. Campanario JM. Peer review for journals as it stands today – part 2. Science Communication 1998; 19: 277– 306. Michael M. Publication prejudices: an experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research 1977; V1: 161–175. Murray C, Stanley M, Wright S. The transition from clinician to academic in nursing and allied health: a qualitative meta-synthesis. Nurse Education Today 2014; 34: 389– 395. McLean R, Mendis K, Harris B, Canalese J. Retrospective bibliometric review of rural health research: Australia’s contribution and other trends. Rural and Remote Health 2007; 7: 767–780.

© 2015 National Rural Health Alliance Inc.