Challenges and Practical Pathways for the Clinical Translation of Deep Learning-Based Nuclei Segmentation in Breast Histopathology

DOI: https://doi.org/10.62517/jmhs.202605232

Author(s)

Tianqi Lu

Affiliation(s)

ZJE, Zhejiang University, Hangzhou, China

Abstract

Histopathological diagnosis of the breast highly depends on the characteristics of the cell nuclei. Thus, accurate segmentation of nuclei is regarded as the critical foundation in computational pathology analysis of breast tissue. During the past few years, deep learning techniques have made great breakthroughs in the research of nucleus segmentation in breast histopathology. In particular, they can obviously demonstrate strong advantages for complex feature extraction and high-performance pixel-wise prediction. Moreover, deep learning has been proved to show remarkable potential in quantitative pathology, lesion recognition, and computer-assisted diagnosis. However, previous investigations based on publicly available datasets and controlled experiments encounter numerous limitations during the process of clinical practice. The present review mainly discusses the development of deep learning techniques applied to the segmentation of cell nuclei in breast histopathology. It comprehensively introduces the theoretical background and research mechanism of this field. Besides, it summarizes the current status of the research from the perspective of deep learning models, supervision method, and clinical direction. Based on this, this paper makes an analysis of the main problems associated with nucleus segmentation in breast histopathology. Such problems include data heterogeneity and inadequate standardization, substantial workload of manual annotation and poor consistency of the ground truth, insufficient model interpretability, and lack of workflow integration and external validation. In order to overcome the mentioned problems, this paper outlines a number of possible improvement directions. These directions include multicenter construction of data, stain normalization, weakly and semi-supervised learning, structured interpretation of prediction, external validation, and human-computer collaboration. According to this review, a change of the research focus on nucleus segmentation in breast histopathology could be observed. In addition to the traditional pursuit of improved accuracy, generalizability, interpretability, and clinical practicability will gradually attract attention in this field.

Keywords

Breast Histopathology; Nucleus Segmentation; Deep Learning; Digital Pathology; Clinical Translation

References

[1] Al-Kofahi, Y., Lassoued, W., Lee, W., & Roysam, B. (2009). Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Transactions on Biomedical Engineering, 57(4), 841-852. [2] Bai, W., Oktay, O., Sinclair, M., Suzuki, H., Rajchl, M., Tarroni, G., ... & Rueckert, D. (2017, September). Semi-supervised learning for network-based cardiac MR image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 253-260). Cham: Springer International Publishing. [3] Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V., & Madabhushi, A. (2019). Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nature reviews Clinical oncology, 16(11), 703-715. [4] Caicedo, J. C., Roth, J., Goodman, A., Becker, T., Karhohs, K. W., Broisin, M., ... & Carpenter, A. E. (2019). Evaluation of deep learning strategies for nucleus segmentation in fluorescence images. Cytometry Part A, 95(9), 952-965. [5] Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015, August). Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1721-1730). [6] Chen, H., Qi, X., Yu, L., & Heng, P. A. (2016). DCAN: deep contour-aware networks for accurate gland segmentation. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (pp. 2487-2496). [7]Chen, L. C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2017). Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4), 834-848. [8]Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., ... & Dean, J. (2019). A guide to deep learning in healthcare. Nature medicine, 25(1), 24-29. [9]Graham, S., Vu, Q. D., Raza, S. E. A., Azam, A., Tsang, Y. W., Kwak, J. T., & Rajpoot, N. (2019). Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Medical image analysis, 58, 101563. [10]He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (pp. 2961-2969). [11]Holzinger, A., Carrington, A., & Müller, H. (2020). Measuring the quality of explanations: the system causability scale (SCS) comparing human and machine explanations. KI-Künstliche Intelligenz, 34(2), 193-198. [12]Kamnitsas, K., Baumgartner, C., Ledig, C., Newcombe, V., Simpson, J., Kane, A., ... & Glocker, B. (2017, May). Unsupervised domain adaptation in brain lesion segmentation with adversarial networks. In International conference on information processing in medical imaging (pp. 597-609). Cham: Springer International Publishing. [13]Kirillov, A., He, K., Girshick, R., Rother, C., & Dollár, P. (2019). Panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9404-9413). [14]Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3431-3440). [15]Macenko, M., Niethammer, M., Marron, J. S., Borland, D., Woosley, J. T., Guan, X., ... & Thomas, N. E. (2009, June). A method for normalizing histology slides for quantitative analysis. In 2009 IEEE international symposium on biomedical imaging: from nano to macro (pp. 1107-1110). IEEE. [16]Naylor, P., Laé, M., Reyal, F., & Walter, T. (2018). Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE transactions on medical imaging, 38(2), 448-459. [17]Qu, H., Wu, P., Huang, Q., Yi, J., Yan, Z., Li, K., ... & Metaxas, D. N. (2020). Weakly supervised deep nuclei segmentation using partial points annotation in histopathology images. IEEE transactions on medical imaging, 39(11), 3655-3666. [18]Rajchl, M., Lee, M. C., Oktay, O., Kamnitsas, K., Passerat-Palmbach, J., Bai, W., ... & Rueckert, D. (2016). Deepcut: Object segmentation from bounding box annotations using convolutional neural networks. IEEE transactions on medical imaging, 36(2), 674-683. [19]Reinhard, E., Adhikhmin, M., Gooch, B., & Shirley, P. (2002). Color transfer between images. IEEE Computer graphics and applications, 21(5), 34-41. [20]Ronneberger, O., Fischer, P., & Brox, T. (2015, October). U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention (pp. 234-241). Cham: Springer international publishing. [21]Samek, W., Wiegand, T., & Müller, K. R. (2017). Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296. [22]Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision (pp. 618-626). [23]Stacke, K., Eilertsen, G., Unger, J., & Lundström, C. (2020). Measuring domain shift for deep learning in histopathology. IEEE journal of biomedical and health informatics, 25(2), 325-336. [24]Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z., & Ding, X. (2020). Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Medical image analysis, 63, 101693. [25]Tizhoosh, H. R., & Pantanowitz, L. (2018). Artificial intelligence and digital pathology: challenges and opportunities. Journal of pathology informatics, 9(1), 38. [26]Topol, E. J. (2019). High-performance medicine: the convergence of human and artificial intelligence. Nature medicine, 25(1), 44-56. [27]Veeling, B. S., Linmans, J., Winkens, J., Cohen, T., & Welling, M. (2018, September). Rotation equivariant CNNs for digital pathology. In International Conference on Medical image computing and computer-assisted intervention (pp. 210-218). Cham: Springer International Publishing. [28]Veta, M., Huisman, A., Viergever, M. A., van Diest, P. J., & Pluim, J. P. (2011, March). Marker-controlled watershed segmentation of nuclei in H&E stained breast cancer biopsy images. In 2011 IEEE international symposium on biomedical imaging: from nano to macro (pp. 618-621). IEEE. [29]Veta, M., Pluim, J. P., Van Diest, P. J., & Viergever, M. A. (2014). Breast cancer histopathology image analysis: A review. IEEE transactions on biomedical engineering, 61(5), 1400-1411. [30]Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N., & Liang, J. (2018, September). Unet++: A nested u-net architecture for medical image segmentation. In International workshop on deep learning in medical image analysis (pp. 3-11). Cham: Springer International Publishing.