CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration

Posted 2022-01-06 Paul-Huang

CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration

• 摘要
• • KeyWords
• 1. Introduction
• 2. RELATED WORKS
• • 2. 1 Diffeomorphic Image Registration
  • 2.2 Deep-learning-based Image Registration
  • 2.3 Consistent Image Registration(一致性的图像配准算法)
• 3. 原理(Theory)
• • 3.1. 整体理解
  • 3.2 Loss Function
  • • 3.2.1 配准损失
    • 3.2.2 循环一致性损失
    • 3.2.3 映射损失
  • 3.3 Spatial Transformation Layer
  • 3.4 多尺度图像配准
  • • 3.4.1 训练阶段的多尺度
    • 3.4.2 测试阶段的多尺度
• 4.实验
• • 4.1 实验数据
  • 4.2 评估
  • • 4.2.1 Analysis of Face Expression Image Registration
    • 4.2.2 Analysis of Brain MRI Registration
    • 4.2.3 Analysis of Liver CT Registration
  • 4.3 实验结果
  • • 4.3.1 Face的实验结果
    • 4.3.2 Brain MR Image Registration
    • 4.3.3 Multiphase Liver CT Image Registration
• 5.讨论
• 参考

摘要

1. 现有的深度学习方法在配准向量场变形过程中保留原始拓扑结构方面仍然存在局限性。因此提出$\text{a cycle-consistent }$ $\text{deformable image registration}$.
2. 周期一致性通过提供隐式正则化($\text{implicit regularization}$)来保持变形期间的拓扑结构，提高了图像配准性能。
3. 可以很容易地扩展到$\text{多尺度的应用}$，以处理$\text{large volume registration}$中的$\text{内存问题}$。
4. $\text{Qualitative and quantitative evaluations}$ on deformation fields also verify the effectiveness of the cycle consistency of the proposed method.

KeyWords

• explicitly enforce the criterion to guarantee topology preservation.
• providing an implicit regularization to preserve topology during the deformation.

---

1. Introduction

• 文章名称：CycleMorph: Cycle consistent unsupervised deformable image registration
• 作者：Boah Kim, Dong Hwan Kim, Seong Ho Park, Jieun Kim, June-Goo Lee
• Code: https://github.com/boahK/MEDIA_CycleMorph

1. 这项工作最重要的贡献之一是通过简单地对图像施加循环一致性($\text{cylce consistency}$)来演示拓扑保存。
2. 该方法的另一个重要创新是将图像配准技术扩展到多尺度(($\text{multi-scale}$))实现，解决了大体积图像配准问题。
3. 为了提高配准精度，我们提出了一种基于下采样体的大变形粗三维配准方法(($\text{a coarse 3D registration using }$$\text{the subsampled volume for large defromation}$))，然后进行局部变形估计。

---

2. RELATED WORKS

2. 1 Diffeomorphic Image Registration

1. 在经典的变分图像配准方法中，能量函数通常由两项组成:
   $$\mathcal{L}(X,Y,\varphi )={\mathcal{L}}_{sim}(\mathcal{T}(X,\varphi ),Y)+{\mathcal{L}}_{reg}(\varphi )\text{\hspace{1em}(2.1)}$$

   • $X$ and $Y$ denote the moving image and fixed image, respectively;
   • $\varphi$ represents the displacement vector field(位移向量场),
   • $\mathcal{T}$ is the transformation function which warps X to Y using the deformation vector field $\varphi$ .

2. 微分同胚形变确保了两个图像体积之间的某些令人满意的性质，如$\text{连续的}$、$\text{可微分的}$和$\text{保持拓扑的}$。常用的方法是：

   • 大形变微分同态匹配( $\text{Large Deformation Diffeomorphic Metric Matching}$(LDDMM))
   • 对称图像归一化($\text{Symmetric image Normalization method}$)

2.2 Deep-learning-based Image Registration

1. 监督学习网络

   • 监督学习主要是patch-wise prediction.

2. 非监督学习网络

   • 非监督学习主要是：deformed image vs fixed target image.
   • 出现过的方法有：
     • KL散度
     • STL和cross correlation function
     • 仿射和非刚性配准
     • global and local 配准网络

   这些方法通常不会对一致性施加约束，因此会由于映射的简并性而导致$\text{折叠问题(folding problem)}$。

2.3 Consistent Image Registration(一致性的图像配准算法)

1. 虽然经典的微分同形变形配准算法已经被提出来保证双射，但是变形通常是以$\text{有限数目的参数离散表示的}$，因此可能会有一些小的违背。

   Although the classical diffeomorphic deformable registration algorithms have been proposed to ensure the one-to-one correspondence, deformations are generally represented discretely with a finite number of parameters, so there may be some small violations.

2. $\text{the estimated deformation}$ $F:X\mapsto Y$ is $\text{not equal}$ to $\text{the inverse of the estimated deformation}$ from $R:Y\mapsto X$ . 在一致性的图像配准算法, 这个问题通过方向一致性来缓解：
   $$R\simeq F^{-1}\text{\hspace{1em}(2.2)}$$

---

3. 原理(Theory)

3.1. 整体理解

相关定义

• moving source and fixed target images, $X$ and $Y$ , in different shapes or contrast,
• two registration networks as $G_X:(X,Y)\to {\varphi }_{XY}$ and $G_Y:(Y,X)\to {\varphi }_{YX}$ ,

  where ${\varphi }_{XY}$ (resp. ${\varphi }_{YX}$ ) denotes the deformation field from $X$ to $Y$ (resp. $Y$ to $X$ ).

• 利用网络中的一个空间变换层$\mathcal{T}$，对运动图像进行扭曲，从而训练配准网络，使变形图像与固定图像之间的差异最小。
• 这两张变形的图像通过切换它们的顺序再次作为网络的输入，以在图像的像素级别上施加循环一致性。

下图是Cycle-Morph 的框架：

3.2 Loss Function

1. 整体的Loss Function 可以表示为：
   $$\underset{G_X,G_Y}{\min }\mathcal{L}\left(X,Y,G_X,G_Y\right),\text{\hspace{1em}(3.1)}$$
   其中
   L ( X , Y , G X , G Y ) = L regist  ( X , Y , G X ) + L regist  ( Y , X , G Y ) + α L cycle  ( X , Y , G X , G Y ) + β L identity  ( X , Y , G X , G Y ) (3.2) \\colorred \\beginaligned \\mathcalL\\left(X, Y, G_X, G_Y\\right)=& \\mathcalL_\\text regist \\left(X, Y, G_X\\right) +\\mathcalL_\\text regist \\left(Y, X, G_Y\\right) \\\\ &+\\alpha \\mathcalL_\\text cycle \\left(X, Y, G_X, G_Y\\right) +\\beta \\mathcalL_\\text identity \\left(X, Y, G_X, G_Y\\right) \\endaligned\\tag3.2 L(X,Y,GX​,GY​)=​Lregist ​(X,Y,GX​)+Lregist ​(Y,X,G以上是关于CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration的主要内容，如果未能解决你的问题，请参考以下文章

   Cron expression must consist of 6 fields (found 7 in "0 * * * * ? *")

   Cause: org.xml.sax.SAXParseException: The content of elements must consist of well-formed character

   Mybatis 异常: The content of elements must consist of well-formed character data or markup

   ibatis动态sql配置启动时提示：The content of elements must consist of well-formed character data...

   a DNS-1123 label must consist of lower case alphanumeric characters or ‘-‘, and must start and end w

   a DNS-1123 label must consist of lower case alphanumeric characters or ‘-‘, and must start and end w