Rating Pakistani medical journals using the ... - Wiley Online Library

DOI: 10.1111/j.1471-1842.2007.00727.x

Rating Pakistani medical journals using the principles of citation analysis Blackwell Publishing Ltd

Midrar Ullah* & Idrees Farooq Butt†, *Librarian and Literature Search Officer and †Associate Professor, Army Medical College, Rawalpindi, Pakistan

Abstract Objective: To quantify the impact of Pakistani Medical Journals using the principles of citation analysis. Methods: References of articles published in 2006 in three selected Pakistani medical journals were collected and examined. The number of citations for each Pakistani medical journal was totalled. The first ranking of journals was based on the total number of citations; second ranking was based on impact factor 2006 and third ranking was based on the 5-year impact factor. Selfcitations were excluded in all the three ratings. Results: A total of 9079 citations in 567 articles were examined. Forty-nine separate Pakistani medical journals were cited. The Journal of the Pakistan Medical Association remains on the top in all three rankings, while Journal of College of Physicians and Surgeons—Pakistan attains second position in the ranking based on the total number of citations. The Pakistan Journal of Medical Sciences moves to second position in the ranking based on the impact factor 2006. The Journal of Ayub Medical College, Abbottabad moves to second position in the ranking based on the 5-year impact factor. Conclusion: This study examined the citation pattern of Pakistani medical journals. The impact factor, despite its limitations, is a valid indicator of quality for journals.

Introduction The impact of journals has been evaluated on the basis of frequency with which its articles are cited in research publications for the last 50 years.1 Citation analysis as a measure of impact of journals was first suggested by Eugene Garfield in 1955.2 The Journal Citation Report (JCR) began publication in 1975 by the Institute for Scientific Information (ISI), currently Thomson Scientific. The JCR gives Journal Impact Factor (JIF) as a qualitative tool for ranking, evaluating and comparing journals as a supplement, and not as a replacement to traditional measures of quality such as peer review. The Journal Citations Report Correspondence: Midrar Ullah, Army Medical College (NUST), Abid Majeed Road, Rawalpindi, Pakistan. E-mail: [email protected]

50

2005 covers 6088 science and technology journals in the JCR Science edition, and 1747 journals in the JCR Social Science edition.3 The impact factor is considered to be an important tool for the ranking of journals. Thomson Scientific defines the impact factor for a given year as ‘the total number of citations received in that year to articles published in the previous 2 years divided by the total number of citable items published in those 2 years’. It helps to direct authors to those outlets for their work which have a track record of publishing data that is frequently cited.4,5 Librarians also select and evaluate journals for their library by using the impact factor. The aim of this study was to quantify the impact and rankings of Pakistani medical journals by using the principles of citation analysis.

© 2008 The authors Journal compilation © 2008 Health Libraries Group. Health Information and Libraries Journal, 25, pp.50–54

Rating Pakistani medical journals, Midrar Ullah & Idrees Farooq Butt

Materials and methods The first step in the citation analysis was to select base journals from which the citations were collected. Pakistani medical journals selected were: • Journal of the Pakistan Medical Association; • Journal of the College of Physicians and Surgeons— Pakistan; • Journal of Ayub Medical College, Abbottabad. These were selected on the basis of their indexation in  (PubMed), and also the regularity of publication in printed form as well as online. References of articles published during 2006 in the three above mentioned journals were collected and examined. The articles cited from Pakistani medical journals were enlisted and the number of citations for each Pakistani medical journal was totalled. The first ranking of the Pakistani medical journals was based on the total number of citations of all dates. Ranking of the journals based solely on the total number of citations may introduce some bias, as there may be a tendency or deliberate attempt for articles published in a particular journal to cite other articles in that same journal. This tendency may bias the results in favour of the three base journals used in this study. This problem was sorted out by eliminating all self citations (citations to each of the three base journals generated by articles from the same journal). A minimum of ten citations was applied to eliminate the journals that were seldom cited. A second source of prejudice may be the difference in the number of articles published by various journals and the length of time the journal has been publishing. One would expect that journals which publish a great number of articles each year would have increased likelihood of citations because of the larger pool of citable articles. This bias was eliminated by calculating the impact factor 2006 and the 5-year impact factor. The second ranking of the journals was based on the impact factor 2006. The impact factor 2006 was calculated by dividing citations to the 2004 and 2005 published items of a particular journal by the total number of published citable items in that journal from 2004 and 2005. For example, in 2006, journal X was cited 50 times for articles published in 2004 and 2005. In 2004 and 2005, a

total of 280 citable items were published in journal X. The impact factor 2006 of journal X = citation received in 2006 to articles published in 2004 and 2005/number of source items published in 2004 and 2005 = 50/280 = 0.179. A minimum of five citations was applied to eliminate the journals that had a negligible impact factor. The third ranking of the journals was based on the 5-year impact factor. The 5-year impact factor was derived by dividing the total number of citations in 2006 to the published items from 2001 to 2005 by the total number of citable items published in 2001 to 2005—similar to the procedure used by Thomson Scientific. A minimum of 10 citations was applied to eliminate the journals that had negligible impact factor. The JCR counts citations of all published items including; editorials, letters, short communications, articles and reviews, etc., in the numerator of its impact factor calculations, while restricting the denominator to include only substantive and citable items like original research articles, review articles, case reports and proceedings. The same procedure was adapted for this study. The impact factor for 2006 and the 5-year impact factors were adjusted by excluding self citations. Results A total of 9079 citations in 567 articles was examined. Forty-nine separate Pakistani medical journals were cited. Table 1 shows the ranking of Pakistani medical journals on the basis of the total number of citations (excluding self citations) of all dates. The top-ranked journal is the Journal of the Pakistan Medical Association, with 110 citations (excluding 96 self citations). Included among the top five journals are the Journal of the College of Physicians and Surgeons—Pakistan (JCPSP), the Pakistan Journal of Medical Sciences, the Journal of Ayub Medical College, Abbottabad (JAMC) and Annals of King Edward Medical College. Table 2 shows the rankings based on the impact factor 2006. The Journal of the Pakistan Medical Association remains unaffected. However, others change places in the ranking. The Pakistan Journal


51

52

Rating Pakistani medical journals, Midrar Ullah & Idrees Farooq Butt Table 1 Ranking of Pakistani medical journals based on total citations, excluding self citations Ranking Journal name 1 2 3 4 5 6 7 7 8 8 9 10 11

2006 Total citations (excluding self citations)

Journal of the Pakistan Medical Association (Centre; monthly) 110 (excluding 96 self citations) Journal of the College of Physicians and Surgeons—Pakistan (monthly) 51 (excluding 125 self citations) Pakistan Journal of Medical Sciences (quarterly) 26 Journal of Ayub Medical College, Abbottabad (quarterly) 25 (excluding nine self citations) Annals of King Edward Medical College (quarterly) 24 Journal of Postgraduate Medical Institute (quarterly) 21 Pakistan Armed Forces Medical Journal (quarterly) 16 Pakistan Journal of Medical Research (quarterly) 16 Pakistan Journal of Pathology (quarterly) 15 Pakistan Paediatric Journal (3/year) 15 Pakistan Journal of Ophthalmology (quarterly) 13 Pakistan Postgraduate Medical Journal (quarterly) 11 The Professional Medical Journal (quarterly) 10

Table 2 Ranking of Pakistani medical journals based on impact factor 2006

Ranking

Journal name

1

Journal of the Pakistan Medical Association (centre; monthly) Pakistan Journal of Medical Sciences (quarterly) Journal of Ayub Medical College, Abbottabad (quarterly) Journal of Postgraduate Medical Institute (quarterly) Journal of the College of Physicians and Surgeons—Pakistan (monthly)

2 3 4 5

Total citations to 2004 and 2005 articles (excluding self citations)

Total citable articles published in 2004 and 2005

Journal impact factor 2006

30 (excluding 25 self citations)

343

0.087

11

173

0.064

11 (excluding eight self citations)

182

0.060

10

192

0.052


464

0.043

of Medical Sciences and the Journal of Ayub Medical College, Abbottabad (JAMC) move to second and third position, respectively, while the Journal of Postgraduate Medical Institute (JPMI) moves to fourth position and the Journal of the College of Physicians and Surgeons—Pakistan (JCPSP) drops to fifth position. Table 3 shows the ranking based on the 5-year impact factor. The ranking of the top journal Journal of the Pakistan Medical Association (JPMA) remains unchanged. However, this ranking does alter the position of other journals. The Journal of Ayub Medical College, Abbottabad (JAMC) moves to second position and the Pakistan Journal of Medical Sciences drops to third position. While the Journal of Postgraduate Medical Institute (JPMI),

the Journal of the College of Physicians and Surgeons Pakistan (JCPSP) and Annals of King Edward Medical College attain the fourth, fifth and sixth positions, respectively. Discussion The ranking of Pakistani medical journals with a bibliometric method may give librarians, information scientists and policy makers another approach to evaluate medical journals. Many citation studies have been published in the West. A 1998 study entitled ‘The relation between journal use in a medical library and citation use’ documented that the Proceedings of the National Academy of Sciences was the most


Rating Pakistani medical journals, Midrar Ullah & Idrees Farooq Butt Table 3 Ranking of Pakistani medical journals based on 5-year impact factor

Ranking

Journal name

1

Journal of the Pakistan Medical Association (centre; monthly) Journal of Ayub Medical College, Abbottabad (quarterly) Pakistan Journal of Medical Sciences (quarterly) Journal of Postgraduate Medical Institute (quarterly) Journal of the College of Physicians and Surgeons—Pakistan (monthly) Annals of King Edward Medical College (quarterly)

2 3 4 5 6

Total citations (excluding self citations) to 2001–2005 articles

Total citable articles published in 2001–2005

Five-year impact factor


712

0.088

22 (excluding nine self citations)

342

0.064

23

384

0.06

18

326

0.055

1067

0.033

670

0.021

35 (excluding 101 self citations) 14

frequently cited journal and Clinical Research had the highest impact factor.4 A 2002 study entitled ‘Self citation and impact factors in otolaryngology journals’ concluded that self citation of a journal may affect its impact factor. This study also revealed that Acta Otolaryngologica and Laryngoscope had the highest self-citing rates (11.9 and 10.02%).6 A 2002 analysis entitled ‘Citation analysis of otorhinolaryngology journals’ revealed that Head and Neck has the highest adjusted impact factor, and Archives of Otolaryngology—Head and Neck Surgery has the highest 5-year impact factor.7 A 2003 study entitled ‘Citation analysis and journal impact factors in ophthalmology and vision science journals’ reported that Investigative Ophthalmology and Visual Science is the most cited journal (a total of 17 330 citations in 2001).8 However, we could not find a single study on the citation of Pakistani medical journals, and therefore our article is the first this subject. Our results show that the Journal of the Pakistan Medical Association remains top in all the three rankings. The Journal of College of Physicians and Surgeons—Pakistan and the Pakistan Journal of Medical Sciences attain second and third positions in the ranking, based on the total number of citations. The Pakistan Journal of Medical Sciences and the Journal of Ayub Medical College, Abbottabad move to second and third positions, respectively, in the ranking based on impact factor 2006. The Journal of College of Physicians and Surgeons—

Pakistan drops to fifth position. The Journal of Ayub Medical College, Abbottabad moves to second position, while the Pakistan Journal of Medical Sciences drops to third position in the ranking, based on the 5-year impact factor. There are many contradictory opinions regarding the use of the impact factor to assess the quality of journals. Therefore, it should not be considered as the only tool to assess the quality of any journal.9 Additional measures include: the average number of times each article is viewed online, and the total number of online accesses the journal receives in a given period. Thomson Scientific includes very few journals from the developing countries and it also covers primarily scientific literature in the English language. In Pakistan, there is no tool available to assess the quality, such as the impact factor of medical journals.10 Conclusion In this study we examined the citation pattern of Pakistani medical journals. Citation analysis is a useful tool to be used for quality control and performance enhancement. It deals with the assessment of research performance of individual scholars, institutions and scholarly journals. The Journal Impact Factor, despite its limitation, is a valid indicator of quality for journals, as there is no other single ‘perfect’ indicator of journal performance. As the JCR Science edition covers


53

54

Rating Pakistani medical journals, Midrar Ullah & Idrees Farooq Butt

no medical journal from Pakistan, there is a need for an instrument/database in Pakistan to quantify the impact of Pakistani medical journals and to evaluate journals and their contribution to the research community.

3

4

Key Messages Implications for Policy • Ranking of the journals based solely on the total number of citations may introduce some bias. • The impact factor should not be considered as the only tool to assess the quality of any journal. • Additional measures include: the average number of times each article is viewed online, and the total number of online accesses the journal receives in a given period.

5

6

7

Implications for Practice • In Pakistan, there is no tool available to assess the quality of medical journals, such as the impact factor. • There is a need for an instrument/database in Pakistan to quantify the impact of Pakistani medical journals.

8

9

References 1 Garfield, E. Use of journal citation reports and journal performance indicators in measurement of short and long term journal impact. Croat Medical Journal 2000, 41, 368– 74. Available from: http://www.vms.hr/cmj (accessed 12 December 2006). 2 Garfield, E. How can impact factors be improved? British

10

Medical Journal 1996, 313, 411–3. Available from: http:// www.garfield.library.upenn.edu (accessed 6 December 2006). Thomson Scientific. Journal Citation Report, web portal, updated 19 January 2007. Philadelphia: The Thomson Corporation, 2007. Available from: http://portal.isiknowledge.com (accessed 2 February 2007). Tsay, M. Y. The relationship between journal use in a medical library and citation use. Bulletin of the Medical Libraries Association 1998, 86, 31–9. Available from: http://www.mlanet.org/publications/jmla/jmlainfo.html (accessed 6 December 2006). Saha, S., Saint, S. & Christakis, D. A. Impact factor: a valid measure of journal quality. Journal of the Medical Libraries Association. 2003, 91. Available from: http://www.pubmedcentral.nih.gov (accessed 6 December 2006). Motamed, M, Mehta, D, Basavaraj, S & Fuad, F. Self citations and impact factors in otolaryngology journals. Clinical Otolaryngology and Allied Sciences 2002, 27, 318 – 20. Available from: http://www.blackwell-synergy.com (accessed 6 December 2006). Roy, D., Hughes, J. P., Jones, A. S. & Fenton, J. E. Citation analysis of otorhinolaryngology journals. The Journal of Laryngology and Otology. 2002, 116, 363–6. Available from: http://www.journals.cambridge.org/ (accessed 6 December 2006). Sims, J. L. & McGhee, C. N. Citation analysis and journal impact factors in ophthalmology and vision science journals. Clinical and Experimental Ophthalmology. 2003, 31, 14 –22. Available from: http://www.blackwellsynergy.com (accessed 6 December 2006). Tsai, Y. L., Lee, C. C., Chen, S. C. & Yen, Z. S. Top-cited articles in emergency medicine. American Journal of Emergency Medicine 2006, 24, 647–54. Available from: www.elsevier.com/locate/ajem (accessed 6 January 2007). Shiwani, M. H. Quality of a medical journals: where do we stand? Journal of the Pakistan Medical Association 2006, 56, 570–2. Available from: http://www.jpma.org.pk (accessed 6 January 2007).

Received 13 April 2007; Accepted 25 May 2007
