Semantic Segmentation

Semantic Segmentation模型及损失函数总结

Posted by JY on April 12, 2020

UNet-family

U-Net

[paper] : U-Net: Convolutional Networks for Biomedical Image Segmentation

根据 UNet的结构,它能够结合底层和高层的信息。

底层(深层)信息:经过多次下采样后的低分辨率信息。能够提供分割目标在整个图像中上下文语义信息,可理解为反应目标和它的环境之间关系的特征。这个特征有助于物体的类别判断(所以分类问题通常只需要低分辨率/深层信息,不涉及多尺度融合)

高层(浅层)信息:经过concatenate操作从encoder直接传递到同高度decoder上的高分辨率信息。能够为分割提供更加精细的特征,如梯度等。

img

V-Net

[paper]:V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

相当于3D的unet

img

img

H-DenseUNet

[paper]: H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes

网络可以很好的提取混合特征:片内特征和片间的融合特征

网络很好的解决了用2D训练缺少体积上下文信息以及3D训练需要消耗高额计算内存的问题

img

每一个3D输入,通过变换处理函数F,将3D的体积快变成2D的相邻的slices;然后将这些2D slices送进2D DenseUNet,用来提取片内特征;3D的原始输入和2D DenseUNet(用于有效的片内2D特征提取)转换之后的预测结果concat在一起送进3D网络3D DenseUNet(用于上下文信息探测),用来提取片间特征;然后将两者特征融合并经由HFF层(用于联合优化片内特征和片间特征的混合特征融合层)预测最终结果。

原始3D volume上下各填充一层slice 按照 012、 123、 234、 345 … 的方式,将3D volume变换成 2D slices。

UNet++

[paper]: UNet++: A Nested U-Net Architecture for Medical Image Segmentation

https://zhuanlan.zhihu.com/p/44958351

img

MDU-Net

[paper]: MDU-Net: Multi-scale Densely Connected U-Net for biomedical image segmentation

img

DANet

[paper]:

img

img

DUNet

[paper]: DUNet: A deformable network for retinal vessel segmentation

Deformable U-Net

img

Deformable Convolution

img

Structure

img

Multi-path U-Net

[paper]: Dense Multi-path U-Net for Ischemic Stroke Lesion Segmentation in Multiple Image Modalities

img

Bridged U-net

[paper]: Prostate Segmentation using 2D Bridged U-net

img

SUNet

[paper]: SUNet: a deep learning architecture for acute stroke lesion segmentation and outcome prediction in multimodal MRI

img

LadderNet

LADDERNET: Multi-Path Networks Based on U-Net for Medical Image Segmentation [paper]

img

Attention U-Net

[paper]: Attention U-Net: Learning Where to Look for the Pancreas

img

img

M-Net

[paper]: M-NET: A Convolutional Neural Network for Deep Brain Structure Segmentation

img

img

Improved Attention M-Net

[paper]: A Novel Focal Tversky Loss Function with Improved Attention U-Net for Lesion Segmentation

img

R2U-Net

[paper]: Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation

img

img

SE-Net

[paper]: Concurrent Spatial and Channel ‘Squeeze & Excitation’ in Fully Convolutional Networks

img

Y-Net

[paper]: Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images

(input: 2D)

img

U-Net vs Y-Net

img

[paper]: Detection and Delineation of Acute Cerebral Infarct on DWI Using Weakly Supervised Machine Learning (Y-Net)

(input: 3D)

img

MultiResUNet

[paper]: MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation

img

img

img

CE-Net

[paper]: CE-Net: Context Encoder Network for 2D Medical Image Segmentation

img

Graph U-Net

[paper]: Graph U-Net

img

CSA U-NET

[paper]: Connection Sensitive Attention U-NET for Accurate Retinal Vessel Segmentation

img

img

CIA-Net

[paper]: CIA-Net: Robust Nuclei Instance Segmentation with Contour-aware Information Aggregation

img

W-Net

[paper]: W-Net: A Deep Model for Fully Unsupervised Image Segmentation

img

[paper]: W-Net: Reinforced U-Net for Density Map Estimation

img

ScleraSegNet

[paper]: ScleraSegNet: an Improved U-Net Model with Attention for Accurate Sclera Segmentation

img

img

AHCNet

[paper]: AHCNet: An Application of Attention Mechanism and Hybrid Connection for Liver Tumor Segmentation in CT Volumes

img

img

Probabilistic U-Net

[paper]: A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities

img

Partially Reversible U-Net

[paper]: A Partially Reversible U-Net for Memory-Efficient Volumetric Image Segmentation

img

RA-UNet

[paper]: RA-UNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments

img

Unet-GAN

[paper]: The Domain Shift Problem of Medical Image Segmentation and Vendor-Adaptation by Unet-GAN

img

img

Siamese U-Net

[paper]:Siamese U-Net with Healthy Template for Accurate Segmentation of Intracranial Hemorrhage

img

U-Det:

img


Losses

img

https://blog.csdn.net/m0_37477175/article/details/83004746

Cross Entropy Loss

  • binary cross entropy 二分类交叉熵
  • categorical cross entropy 多分类交叉熵
  • weighted cross entropy 加权交叉熵

缺点:交叉熵的损失函数单独评估每个像素矢量的类预测,然后对所有像素求平均值。但是图像中常出现类别不均衡(class imbalance)的问题,由此导致训练会被像素较多的类主导,对于较小的物体很难学习到其特征,从而降低网络的有效性

Dice Coefficient Loss

\[\begin{equation} \text {Dice}=\frac{2|A \cap B|}{|A\|+|B|} \end{equation}\]

该指标范围从0到1,其中1表示完整的重叠,loss=1-dice

dice loss比较适用于样本极度不均的情况

IOU Loss

\[I O U=\frac{|A \cap B|}{|A|+|B|-|A \cap B|}\]

loss=1-IOU

Focal Loss

可以改善目标不均衡的现象

Generalized Dice loss

在使用DICE loss时,对小目标是十分不利的,因为在只有前景和背景的情况下,小目标一旦有部分像素预测错误,那么就会导致Dice大幅度的变动,从而导致梯度变化剧烈,训练不稳定。

the generalized Dice loss在dice loss基础上给每个类别加权,起到平衡各类(包括背景类)目标区域对loss的贡献。

Tversky loss

\[T(A, B)=\frac{|A \cap B|}{|A \cap B|+\alpha|A-B|+\beta|B-A|}\]

是dice系数和jaccard系数的一种广义系数