Boundary-Aware Attention Guided 3D Residual Network for Accurate Cell Segmentation in Volumetric Microscopy Images

Abstract

Cell segmentation in volumetric microscopy images is a challenging problem that is crucial for quantitative biomedical analysis, as it is affected by intensity variation, low resolution of cell boundaries, irregular shapes, sparse and densely overlapping cells, and the presence of noise in the images. Standard 3D segmentation networks used for a long time tend to lose some spatial detail in the down sampling process, leading to incomplete small-cell segmentation and separation of neighbouring structures. In this regard, we propose a Boundary-Aware Residual Attention Network for 3D Segmentation (BAR-Net3D), which is an improvement over these drawbacks. It proposes a framework that leverages the following elements in a 3D encoder–decoder structure: residual feature extraction, attention-guided skip connections, multiscale feature fusion, and a boundary refinement module. A hybrid loss function that optimizes regional segmentation accuracy and contour consistency is used. Four publicly available volumetric microscopy datasets (VGG, MBM, ADI and DCC) are used to evaluate the model in terms of Dice similarity coefficient (DSC), intersection over union (IoU), precision, recall, F1-score, boundary accuracy, small-cell detection ratio (SDR), and overlapping-cell separation accuracy (OSAA). On the reported VGG evaluation, BAR-Net3D achieved a Dice score of 95.82%, an IoU of 91.74%, a precision of 95.11%, a recall of 94.86%, and an F1-score of 94.98%. It also achieved 93.44% boundary accuracy and 94.62% small-cell detection ratio and 93.85% overlapping-cell separation accuracy. The ablation results show that all the above-mentioned components contribute to the overall results, and the improvement in contour localization and cell separation by boundary refinement is significant. While the proposed model demands higher computational resources than a traditional 3D U-Net, it offers a good trade-off between accuracy and inference speed. Results indicate that BAR-Net3D is a potential useful tool for high quality segmentation of cells in complex volumetric microscopy images. Volumetric microscopy, cell segmentation, 3D convolutional neural network, Residual learning, attention mechanism, boundary refinement, multiscale feature fusion, Biomedical image analysis.