Automated segmentation of nuclei in Hematoxylin and Eosin (H&E) stained histopathology images plays a vital role in accurate cancer diagnosis and prognosis. These techniques enable detailed analysis of numerous nuclei in H&E images, providing both qualitative and quantitative insights. However, challenges arise when segmenting nuclei of varying sizes and with indistinct boundaries, which can undermine the reliability of segmentation outcomes. To address these issues, we propose a novel approach that integrates edge information, extracted from the input data, into the UNET architecture a well-established model for image segmentation. Our approach involves modifying the Attention Gate (AG) mechanism within the UNET to emphasize edge features during segmentation. This modification improves the precision of nucleus boundary delineation, particularly in cases with vague or overlapping boundaries, reducing segmentation errors and boosting overall accuracy.

[1] Bhukya R., “Encoding Gene Expression Using Deep Autoencoders for Expression Inference,” The International Arab Journal of Information Technology, vol. 18, no. 5, pp. 625-633, 2021. https://doi.org/10.34028/iajit/18/5/1

[2] Bhukya R., Kumari A., Dasari C., and Amilpur S., “An Attention-Based Hybrid Deep Neural Networks for Accurate Identification of Transcription Factor Binding Sites,” Neural Computing and Applications, vol. 34, no. 21, pp. 19051-19060, 2022. https://doi.org/10.1007/s00521-022-07502-z

[3] Dasari C. and Bhukya R., “Explainable Deep Neural Networks for Novel Viral Genome Prediction,” Applied Intelligence, vol. 52, no. 3, pp. 3002-3017, 2022. https://link.springer.com/article/10.1007/s10489- 021-02572-3

[4] Gugulothu P. and Bhukya R., “Coot-Lion Enhanced Nucleus Segmentation with Sobel Edge Detection and Attention Gate with ... 729 Optimized Deep Learning Algorithm for COVID- 19 Point Mutation Rate Prediction Using Genome Sequences,” Computer Methods in Biomechanics and Biomedical Engineering, vol. 27, no. 11, pp. 1410-1429, 2023. https://doi.org/10.1080/10255842.2023.2244109

[5] He K., Zhang X., Ren S., and Sun J., “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, pp. 770-778, 2016. DOI:10.1109/CVPR.2016.90

[6] Kartheek M., Prasad M., and Bhukya R., “Local Triangular Patterns: Novel Handcrafted Feature Descriptors for Facial Expression Recognition,” International Journal of Biometrics, vol. 15, no. 2, pp. 194-211, 2023. https://doi.org/10.1504/ijbm.2023.129224

[7] Kartheek M., Prasad M., and Bhukya R., “Texture Based Feature Extraction Using Symbol Patterns for Facial Expression Recognition,” Cognitive Neurodynamics, vol. 18, pp. 317-335, 2024. https://doi.org/10.1007/s11571-022-09824-z

[8] Lal S., Das D., Alabhya K., Kanfade A., Kumar A., and Kini J., “NucleiSegNet: Robust Deep Learning Architecture for the Nuclei Segmentation of Liver Cancer Histopathology Images,” Computers in Biology and Medicine, vol. 128, pp. 104075, 2021. https://doi.org/10.1016/j.compbiomed.2020.1040 75

[9] Long J., Shelhamer E., and Darrell T., “Fully Convolutional Networks for Semantic Segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, pp. 3431-3440, 2015. DOI:10.1109/CVPR.2015.7298965

[10] Milletari F., Navab N., and Ahmadi S., “V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation,” in Proceedings of the 4th International Conference on 3D Vision, Stanford, pp. 565-571, 2016. DOI:10.1109/3DV.2016.79

[11] MoNuSeg Dataset, MoNuSeg-Grand Challenge, https://monuseg.grand-challenge.org/Data/, Last Visited, 2024.

[12] Oktay O., Schlemper J., Le Folgoc L., Lee M., et al., “Attention U-Net: Learning Where to Look for the Pancreas,” arXiv Preprint, vol. arXiv:1804.03999v3, pp. 1-10, 2018. https://arxiv.org/abs/1804.03999

[13] Ronneberger O., Fischer P., and Brox T., “UNET: Convolutional Networks for Biomedical Image Segmentation,” in Proceeding of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, pp. 234-241, 2015. https://link.springer.com/chapter/10.1007/978-3- 319-24574-4_28

[14] Szegedy C., Liu W., Jia Y., and Sermanet P., et al., “Going Deeper with Convolutions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, pp. 1-9, 2015. DOI:10.1109/CVPR.2015.7298594

[15] Yu C., Wang J., Peng C., Gao C., Yu G., and Sang N., “Learning a Discriminative Feature Network for Semantic Segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Utah, pp. 1857-1866, 2018. DOI:10.1109/CVPR.2018.00199

[16] Zhang Z., Liu Q., and Wang Y., “Road Extraction by Deep Residual U-Net,” IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 5, pp. 749- 753, 2018. DOI:10.1109/LGRS.2018.2802944