International Journal of Applied Science and Engineering
Published by Chaoyang University of Technology

Zhe Guoa, Xin Zhua*, Qin Lia, Daiki Nemotob, Daisuke Takayanagib, Masato Aizawab, Noriyuki Isohatab, Kenichi Utanob, Kensuke Kumamotob, Shungo Endoand Kazutomo Togashib

aBiomedical Information Engineering Lab, The University of Aizu, Aizu-Wakamatsu, Fukushima, Japan
bDivision of Proctology, Aizu Medical Center, Fukushima Medical University, Aizu-Wakamatsu, Fukushima, Japan


Download Citation: |
Download PDF


Colorectal cancer (CRC) is one of the most popular cancer in the world. Adenoma and sessile serrated polyp precursor lesions claim over 95% of CRC. The incidence of CRC is reduced 76-90% through the early diagnosis and removal of colorectal polyps. Colonoscopy is the golden standard for the detection of colorectal polyps but about 25% of polyps were missed during colonoscopy examinations. In this study, we proposed a novel method to recognize polyps from colonoscopy images based on bag-of-visual-words (BoW) with extracted regions of interest. The proposed method generates a histogram of visual word occurrences to represent an image, and uses support vector machine (SVM) with error correcting output codes (ECOC) for the detection of polyps. A dataset composed of 131 cases’ clinical data were used to train and test the proposed method. Validation demonstrates an average specificity of 97.8±1.5%, an average sensitivity of 97.2±1.7%, and an average accuracy of 97.5±1.0%.

Keywords: Bag of visual words; colorectal cancer; colonoscopy; region of interest.

Share this article with your colleagues



  1. [1] Mesejo, P., Pizarro, D., Abergel, A., Rouquette, O., Beorchia, S., Poincloux, L. and Bartoli, A. (2016). Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE transactions on medical imaging, 35(9), 2051-2063.

  2. [2] Siegel, R., DeSantis, C. and Jemal, A. (2014). Colorectal cancer statistics, 2014. CA: a cancer journal for clinicians, 64(2), 104-117.

  3. [3] Chen, W., Zheng, R., Baade, P. D., Zhang, S., Zeng, H., Bray, F. and He, J. (2016). Cancer statistics in China, 2015. CA: a cancer journal for clinicians, 66(2), 115-132.

  4. [4] Pickhardt, P. J. (2016). Emerging stool-based and blood-based non-invasive DNA tests for colorectal cancer screening: the importance of cancer prevention in addition to cancer detection. Abdominal Radiology, 41(8), 1441-1444.

  5. [5] Cancer statistic of Japan. [Online]. Available:

  6. [6] Meyer, J. E., Narang, T., Schnoll‐Sussman, F. H., Pochapin, M. B., Christos, P. J. and Sherr, D. L. (2010). Increasing incidence of rectal cancer in patients aged younger than 40 years. Cancer, 116(18), 4354-4359.

  7. [7] Gado, A., Ebeid, B., Abdelmohsen, A. and Axon, A. (2014). Colorectal cancer in Egypt is commoner in young people: Is this cause for alarm?. Alexandria Journal of Medicine, 50(3), 197-201.

  8. [8] Gloor, F. J. (1986). The adenoma-carcinoma sequence of the colon and rectum. Sozial-und Präventivmedizin, 31(2), 74-75.

  9. [9] Sovich, J. L., Sartor, Z. and Misra, S. (2015). Developments in screening tests and strategies for colorectal cancer. BioMed research international, 2015.

  10. [10], Mar. 9, 2018.

  11. [11] Sharma, P., Gupta, N., Kuipers, E. J., Repici, A. and Wallace, M. (2014). Advanced imaging in colonoscopy and its impact on quality. Gastrointestinal endoscopy, 79(1), 28-36.

  12. [12] Pullens, H. J. and Siersema, P. D. (2014). Quality indicators for colonoscopy: Current insights and caveats. World journal of gastrointestinal endoscopy, 6(12), 571.

  13. [13] Rabeneck, L., El-Serag, H. B., Davila, J. A. and Sandler, R. S. (2003). Outcomes of Colorectal Cancer in the United States: No Change in Survival (1986–1997). The American journal of gastroenterology, 98(2), 471.

  14. [14] Tamaki, T., Yoshimuta, J., Kawakami, M., Raytchev, B., Kaneda, K., Yoshida, S. and Tanaka, S. (2013). Computer-aided colorectal tumor classification in NBI endoscopy using local features. Medical image analysis, 17(1), 78-100.

  15. [15] Hazewinkel, Y. and Dekker, E. (2011). Colonoscopy: basic principles and novel techniques. Nature Reviews Gastroenterology and Hepatology, 8(10), 554.

  16. [16] Karkanis, S. A., Iakovidis, D. K., Maroulis, D. E., Karras, D. A. and Tzivras, M. (2003). Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE transactions on information technology in biomedicine, 7(3), 141-152.

  17. [17] Iakovidis, D. K., Maroulis, D. E., Karkanis, S. A. and Brokos, A. (2005, June). A comparative study of texture features for the discrimination of gastric polyps in endoscopic video. In Computer-Based Medical Systems, 2005. Proceedings. 18th IEEE Symposium on (pp. 575-580). IEEE.

  18. [18] Alexandre, L. A., Nobre, N. and Casteleiro, J. (2008, May). Color and position versus texture features for endoscopic polyp detection. In BioMedical Engineering and Informatics, 2008. BMEI 2008. International Conference on (Vol. 2, pp. 38-42). IEEE.

  19. [19] Ameling, S., Wirth, S., Paulus, D., Lacey, G. and Vilarino, F. (2009). Texture-based polyp detection in colonoscopy. In Bildverarbeitung für die Medizin 2009 (pp. 346-350). Springer, Berlin, Heidelberg.

  20. [20] Tajbakhsh, N., Gurudu, S. R. and Liang, J. (2016). Automated polyp detection in colonoscopy videos using shape and context information. IEEE transactions on medical imaging, 35(2), 630-644.

  21. [21] Mori, Y., Kudo, S. E., Wakamura, K., Misawa, M., Ogawa, Y., Kutsukawa, M. and Inoue, H. (2015). Novel computer-aided diagnostic system for colorectal lesions by using endocytoscopy (with videos). Gastrointestinal endoscopy, 81(3), 621-629.

  22. [22] Takeda, K., Kudo, S. E., Mori, Y., Misawa, M., Kudo, T., Wakamura, K. and Inoue, H. (2017). Accuracy of diagnosing invasive colorectal cancer using computer-aided endocytoscopy. Endoscopy, 49(08), 798-802.

  23. [23] Kominami, Y., Yoshida, S., Tanaka, S., Sanomura, Y., Hirakawa, T., Raytchev, B. and Chayama, K. (2016). Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy. Gastrointestinal endoscopy, 83(3), 643-649.

  24. [24] Takemura, Y., Yoshida, S., Tanaka, S., Kawase, R., Onji, K., Oka, S. and Chayama, K. (2012). Computer-aided system for predicting the histology of colorectal tumors by using narrow-band imaging magnifying colonoscopy (with video). Gastrointestinal endoscopy, 75(1), 179-185.

  25. [25] Yu, L., Chen, H., Dou, Q., Qin, J. and Heng, P. A. (2017). Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE journal of biomedical and health informatics, 21(1), 65-75.

  26. [26] Sivic, J. and Zisserman, A. (2003). Video Google: a text retrieval approach to object matching in videos. Proceedings Ninth IEEE International Conference on Computer Vision, (ICCV), 2, 1470–1477.

  27. [27] Nowak, E., Jurie, F. and Triggs, B. (2006, May). Sampling strategies for bag-of-features image classification. In European conference on computer vision (pp. 490-503). Springer, Berlin, Heidelberg.

  28. [28] Tong, S. and Koller, D. (2001). Support vector machine active learning with applications to text classification. Journal of machine learning research, 2(Nov), 45-66.

  29. [29] Csurka, G., Dance, C., Fan, L., Willamowski, J. and Bray, C. (2004, May). Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV(Vol. 1, No. 1-22, pp. 1-2).

  30. [30] Bay, H., Ess, A., Tuytelaars, T. and Van Gool, L. (2008). Speeded-up robust features (SURF). Computer vision and image understanding, 110(3), 346-359.

  31. [31] Hackbusch, W. (2013). Multi-grid methods and applications(Vol. 4). Springer Science & Business Media.

  32. [32] James, G. and Hastie, T. (1998). The error coding method and PICTs. Journal of Computational and Graphical Statistics, 7(3), 377-387.

  33. [33] Bernal, J., Sánchez, J. and Vilarino, F. (2012). Towards automatic polyp detection with a polyp appearance model. Pattern Recognition, 45(9), 3166-3182.


Received: 2018-05-10
Revised: 2019-06-12
Accepted: 2019-06-30
Available Online: 2019-06-01

Cite this article:

Guo, Z., Zhu, X., Li, Q., Nemoto, D., Takayanagi, D., Aizawa, M., Isohata, N., Utano, K., Kumamoto, K., EndoS., Togashi, K. 2019. Automatic polyp recognition from colonoscopy images based on bag of visual words. International Journal of Applied Science and Engineering, 16, 69-81.

We use cookies on this website to improve your user experience. By using this site you agree to its use of cookies.