IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 21, NO. 1, JANUARY 2017
41
Automatic Detection and Classification of Colorectal Polyps by Transferring Low-Level CNN Features From Nonmedical Domain Ruikai Zhang, Yali Zheng, Tony Wing Chung Mak, Ruoxi Yu, Sunny H. Wong, James Y. W. Lau, and Carmen C. Y. Poon, Senior Member, IEEE
Abstract—Colorectal cancer (CRC) is a leading cause of cancer deaths worldwide. Although polypectomy at early stage reduces CRC incidence, 90% of the polyps are small and diminutive, where removal of them poses risks to patients that may outweigh the benefits. Correctly detecting and predicting polyp type during colonoscopy allows endoscopists to resect and discard the tissue without submitting it for histology, saving time, and costs. Nevertheless, human visual observation of early stage polyps varies. Therefore, this paper aims at developing a fully automatic algorithm to detect and classify hyperplastic and adenomatous colorectal polyps. Adenomatous polyps should be removed, whereas distal diminutive hyperplastic polyps are considered clinically insignificant and may be left in situ. A novel transfer learning application is proposed utilizing features learned from big nonmedical datasets with 1.4–2.5 million images using deep convolutional neural network. The endoscopic images we collected for experiment were taken under random lighting conditions, zooming and optical magnification, including 1104 endoscopic nonpolyp images taken under both white-light and narrowband imaging (NBI) endoscopy and 826 NBI endoscopic polyp images, of which 263 images were hyperplasia and 563 were adenoma as confirmed by histology. The proposed method identified polyp images from nonpolyp images in the beginning followed by predicting the polyp histology. When compared with visual inspection by endoscopists, the results of this study show that the proposed method has similar precision (87.3% versus 86.4%) but a higher recall rate (87.6% versus 77.0%) and a higher accuracy (85.9% versus 74.3%). In conclusion, automatic algorithms can assist endoscopists in identifying polyps that are adenomatous but have been incorrectly judged as hyperplasia
Manuscript received March 21, 2016; revised September 10, 2016 and October 25, 2016; accepted November 17, 2016. Date of publication December 5, 2016; date of current version January 31, 2017. This work was supported by the Hong Kong Innovation and Technology Fund, Shaw Endoscopy Center and the Chow Yuk Ho Technology Centre for Innovative Medicine. R. Zhang, Y. Zheng, T. W. C. Mak, R. Yu, J. Y. W. Lau, and C. C. Y. Poon are with the Department of Surgery, The Chinese University of Hong Kong, Shatin, Hong Kong SAR (e-mail:
[email protected]. edu.hk;
[email protected];
[email protected];
[email protected];
[email protected]; cpoon@ surgery.cuhk.edu.hk). S. H. Wong is with the Department of Medicine & Therapeutics, Li Ka Shing Institute of Health Sciences, Institute of Digestive Diseases, State Key Laboratory of Digestive Diseases, The Chinese University of Hong Kong, Shatin, Hong Kong SAR (e-mail:
[email protected]). Digital Object Identifier 10.1109/JBHI.2016.2635662
and, therefore, enable timely resection of these polyps at an early stage before they develop into invasive cancer. Index Terms—Colorectal cancer, deep learning, health informatics, polyp diagnosis.
I. INTRODUCTION OLORECTAL cancer (CRC) is one of the leading causes of death worldwide with about estimated 700 000 deaths in 2012 [1]. Long-term follow-up studies confirmed that removal of adenomatous polyps reduces CRC mortality [2]. Nevertheless, only around half of small polyps (
