Comparative Analysis of Convolutional Neural Network (ResNet-50) and Vision Transformer (ViT-B/16) for Histopathological Image Classification of Colorectal Cancer
DOI:
10.29303/jppipa.v12i7.14880Published:
2026-07-25Downloads
Abstract
The diagnosis of colorectal cancer (CRC) through histopathological images requires high accuracy to support appropriate clinical decisions. Although Convolutional Neural Networks (CNN) have become the gold standard in medical image analysis, the emergence of Vision Transformer (ViT) architecture offers a new paradigm based on global attention mechanisms (self-attention) that is claimed to be superior on large-scale datasets. However, the effectiveness of ViT-B/16 on medical datasets with limited sample sizes and high texture variation remains debatable. This study aims to comprehensively evaluate the performance of the ViT-B/16 architecture compared to ResNet-50 on the NCT-CRC-HE-100K histopathology dataset, which consists of 9 network classes. The performance of both models was tested using equivalent training scenarios. The evaluation was conducted multidimensionally, covering classification metrics (Accuracy, F1-Score), training stability, feature space separability (t-SNE), visual interpretability (Grad-CAM), and computational efficiency. The experimental results show that ResNet-50 significantly outperforms ViT-B/16 with a test accuracy of 93.24%, compared to ViT-B/16 which only achieves 57.11%. The t-SNE analysis revealed that ViT-B/16 failed to form well-separated feature clusters due to a lack of inductive bias to recognize local features such as cell membrane edges. Failure analysis shows that ViT-B/16 often misclassifies adipose cells as mucus and smooth muscle as tumors. In terms of efficiency, ResNet-50 is 5.8 times lighter in storage size and has lower inference latency. This study concludes that CNN-based architecture (ResNet-50) is still far superior, more stable, and more feasible for clinical implementation than ViT-B/16 in the context of medium-scale histopathological image classification.
Keywords:
Colorectal cancer Computational efficiency Convolutional neural network Histopathology ResNet-50 Vision transformerReferences
Araújo, A. L. D., Sperandio, M., Calabrese, G., Faria, S. S., Cardenas, D. A. C., Martins, M. D., Vargas, P. A., Lopes, M. A., Santos-Silva, A. R., Kowalski, L. P., & Moraes, M. C. (2025). Artificial intelligence in healthcare applications targeting cancer diagnosis—part II: interpreting the model outputs and spotlighting the performance metrics. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 140(1), 89–99. https://doi.org/10.1016/j.oooo.2025.01.002 DOI: https://doi.org/10.1016/j.oooo.2025.01.002
Balasubramanian, A. A., Al-Heejawi, S. M. A., Singh, A., Breggia, A., Ahmad, B., Christman, R., Ryan, S. T., & Amal, S. (2024). Ensemble Deep Learning-Based Image Classification for Breast Cancer Subtype and Invasiveness Diagnosis from Whole Slide Image Histopathology. Cancers, 16(12), 2222. https://doi.org/10.3390/cancers16122222 DOI: https://doi.org/10.3390/cancers16122222
Dabass, M., Vashisth, S., & Vig, R. (2022). A convolution neural network with multi-level convolutional and attention learning for classification of cancer grades and tissue structures in colon histopathological images. Computers in Biology and Medicine, 147, 105680. https://doi.org/10.1016/j.compbiomed.2022.105680 DOI: https://doi.org/10.1016/j.compbiomed.2022.105680
De, A., Mishra, N., & Chang, H.-T. (2024). An approach to the dermatological classification of histopathological skin images using a hybridized CNN-DenseNet model. PeerJ Computer Science, 10, e1884. https://doi.org/10.7717/peerj-cs.1884 DOI: https://doi.org/10.7717/peerj-cs.1884
Desai, A., & Mahto, R. (2025). Multi-Class Classification of Breast Cancer Subtypes Using ResNet Architectures on Histopathological Images. Journal of Imaging, 11(8), 284. https://doi.org/10.3390/jimaging11080284 DOI: https://doi.org/10.3390/jimaging11080284
Dunn, C., Brettle, D., Cockroft, M., Keating, E., Revie, C., & Treanor, D. (2024). Quantitative assessment of H&E staining for pathology: development and clinical evaluation of a novel system. Diagnostic Pathology, 19(1), 42. https://doi.org/10.1186/s13000-024-01461-w DOI: https://doi.org/10.1186/s13000-024-01461-w
Gamage, L., Isuranga, U., Meedeniya, D., De Silva, S., & Yogarajah, P. (2024). Melanoma Skin Cancer Identification with Explainability Utilizing Mask Guided Technique. Electronics, 13(4), 680. https://doi.org/10.3390/electronics13040680 DOI: https://doi.org/10.3390/electronics13040680
Ghosh, S., Bandyopadhyay, A., Sahay, S., Ghosh, R., Kundu, I., & Santosh, K. C. (2021). Colorectal Histology Tumor Detection Using Ensemble Deep Neural Network. Engineering Applications of Artificial Intelligence, 100, 104202. https://doi.org/10.1016/j.engappai.2021.104202 DOI: https://doi.org/10.1016/j.engappai.2021.104202
Hossain, M. M., Hossain, M. A., Musa Miah, A. S., Okuyama, Y., Tomioka, Y., & Shin, J. (2023). Stochastic Neighbor Embedding Feature-Based Hyperspectral Image Classification Using 3D Convolutional Neural Network. Electronics, 12(9), 2082. https://doi.org/10.3390/electronics12092082 DOI: https://doi.org/10.3390/electronics12092082
Howell, L., Ingram, N., Lapham, R., Morrell, A., & McLaughlan, J. R. (2024). Deep learning for real-time multi-class segmentation of artefacts in lung ultrasound. Ultrasonics, 140, 107251. https://doi.org/10.1016/j.ultras.2024.107251 DOI: https://doi.org/10.1016/j.ultras.2024.107251
Jeddah, Y. M., Hassan Abdalla Hashim, A., Omran Khalifa, O., & Ouhada, K. (2025). Enhancing Anomaly Detection Performance: Deep Learning Models Evaluation. IIUM Engineering Journal, 26(2), 96–108. https://doi.org/10.31436/iiumej.v26i2.3287 DOI: https://doi.org/10.31436/iiumej.v26i2.3287
Kather, J. N., Halama, N., & Marx, A. (2021). 100,000 Histological Images of Human Colorectal Cancer and Healthy Tissue (V0.1). Zenodo. Retrieved from https://zenodo.org/records/1214456
Kumar, A., Vishwakarma, A., & Bajaj, V. (2023). CRCCN-Net: Automated framework for classification of colorectal tissue using histopathological images. Biomedical Signal Processing and Control, 79, 104172. https://doi.org/10.1016/j.bspc.2022.104172 DOI: https://doi.org/10.1016/j.bspc.2022.104172
Lan, X., Guo, G., Wang, X., Yan, Q., Xue, R., Li, Y., Zhu, J., Dong, Z., Wang, F., Li, G., Wang, X., Xu, J., & Jiang, Y. (2024). Differentiation and risk stratification of basal cell carcinoma with deep learning on histopathologic images and measuring nuclei and tumor microenvironment features. Skin Research and Technology, 30(1). https://doi.org/10.1111/srt.13571 DOI: https://doi.org/10.1111/srt.13571
Le, T. T., Nguyen-Truong, V.-T., Van Nhat Duong, Q., Le Phan, N. T., Thien Dao, P. N., Mavuso, M. F., Nguyen, H. N. A., Mai, T. T., & Quang, K. T. (2025). Deep learning-based classification of colorectal cancer in histopathology images for category detection. Biology Methods and Protocols, 10(1). https://doi.org/10.1093/biomethods/bpaf077 DOI: https://doi.org/10.1093/biomethods/bpaf077
Ma, J., Yang, H., Chou, Y., Yoon, J., Allison, T., Komandur, R., McDunn, J., Tasneem, A., Do, R. K., Schwartz, L. H., & Zhao, B. (2025). Generalizability of lesion detection and segmentation when ScaleNAS is trained on a large multi‐organ dataset and validated in the liver. Medical Physics, 52(2), 1005–1018. https://doi.org/10.1002/mp.17504 DOI: https://doi.org/10.1002/mp.17504
Martínez-Fernandez, E., Rojas-Valenzuela, I., Valenzuela, O., & Rojas, I. (2023). Computer Aided Classifier of Colorectal Cancer on Histopatological Whole Slide Images Analyzing Deep Learning Architecture Parameters. Applied Sciences, 13(7), 4594. https://doi.org/10.3390/app13074594 DOI: https://doi.org/10.3390/app13074594
Musthafa, M. M., Mahesh, T. R., Kumar, V., & Guluwadi, S. (2024). Enhancing brain tumor detection in MRI images through explainable AI using Grad-CAM with Resnet 50. BMC Medical Imaging, 24(1), 107. https://doi.org/10.1186/s12880-024-01292-7 DOI: https://doi.org/10.1186/s12880-024-01292-7
Oh, S., Kim, N., & Ryu, J. (2024). Analyzing to discover origins of CNNs and ViT architectures in medical images. Scientific Reports, 14(1), 8755. https://doi.org/10.1038/s41598-024-58382-3 DOI: https://doi.org/10.1038/s41598-024-58382-3
Shi, S., Xu, Y., Xu, X., Mo, X., & Ding, J. (2023). A Preprocessing Manifold Learning Strategy Based on t-Distributed Stochastic Neighbor Embedding. Entropy, 25(7), 1065. https://doi.org/10.3390/e25071065 DOI: https://doi.org/10.3390/e25071065
Snead, D. R., Azam, A. S., Thirlwall, J., Kimani, P., Hiller, L., Bickers, A., Boyd, C., Boyle, D., Clark, D., Ellis, I., Gopalakrishnan, K., Ilyas, M., Kelly, P., Loughrey, M., Neil, D., Rakha, E., Roberts, I. S., Sah, S., Soares, M., … Dunn, J. (2025). Variation within and between digital pathology and light microscopy for the diagnosis of histopathology slides: blinded crossover comparison study. Health Technology Assessment, 1–75. https://doi.org/10.3310/SPLK4325 DOI: https://doi.org/10.3310/SPLK4325
Wang, Y.-L., Gao, S., Xiao, Q., Li, C., Grzegorzek, M., Zhang, Y.-Y., Li, X.-H., Kang, Y., Liu, F.-H., Huang, D.-H., Gong, T.-T., & Wu, Q.-J. (2024). Role of artificial intelligence in digital pathology for gynecological cancers. Computational and Structural Biotechnology Journal, 24, 205–212. https://doi.org/10.1016/j.csbj.2024.03.007 DOI: https://doi.org/10.1016/j.csbj.2024.03.007
Wang, Y., Deng, Y., Zheng, Y., Chattopadhyay, P., & Wang, L. (2025). Vision Transformers for Image Classification: A Comparative Survey. Technologies, 13(1), 32. https://doi.org/10.3390/technologies13010032 DOI: https://doi.org/10.3390/technologies13010032
Wen, Z., Wang, S., Yang, D. M., Xie, Y., Chen, M., Bishop, J., & Xiao, G. (2023). Deep learning in digital pathology for personalized treatment plans of cancer patients. Seminars in Diagnostic Pathology, 40(2), 109–119. https://doi.org/10.1053/j.semdp.2023.02.003 DOI: https://doi.org/10.1053/j.semdp.2023.02.003
Zaman, F. H., Ng, K. M., & Abdullah, S. A. C. (2025). Comparative Analysis of Vision Transformers and CNN Models for Driver Fatigue Classification. IIUM Engineering Journal, 26(2), 169–186. https://doi.org/10.31436/iiumej.v26i2.3488 DOI: https://doi.org/10.31436/iiumej.v26i2.3488
Zhang, C., Aamir, M., Guan, Y., Al-Razgan, M., Awwad, E. M., Ullah, R., Bhatti, U. A., & Ghadi, Y. Y. (2024). Enhancing lung cancer diagnosis with data fusion and mobile edge computing using DenseNet and CNN. Journal of Cloud Computing, 13(1), 91. https://doi.org/10.1186/s13677-024-00597-w DOI: https://doi.org/10.1186/s13677-024-00597-w
Zhang, T., Xu, W., Luo, B., & Wang, G. (2025). Depth-Wise Convolutions in Vision Transformers for efficient training on small datasets. Neurocomputing, 617, 128998. https://doi.org/10.1016/j.neucom.2024.128998 DOI: https://doi.org/10.1016/j.neucom.2024.128998
Zhu, M., Zhai, Z., Wang, Y., Chen, F., Liu, R., Yang, X., & Zhao, G. (2025). Advancements in the application of artificial intelligence in the field of colorectal cancer. Frontiers in Oncology, 15. https://doi.org/10.3389/fonc.2025.1499223 DOI: https://doi.org/10.3389/fonc.2025.1499223
License
Copyright (c) 2026 Muhammad Fazly Qusyairy, Habibullah Akbar, Wahyu Purnama Magribi, Khusnul Fajri Rhomadon
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with Jurnal Penelitian Pendidikan IPA, agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution 4.0 International License (CC-BY License). This license allows authors to use all articles, data sets, graphics, and appendices in data mining applications, search engines, web sites, blogs, and other platforms by providing an appropriate reference. The journal allows the author(s) to hold the copyright without restrictions and will retain publishing rights without restrictions.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in Jurnal Penelitian Pendidikan IPA.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
