Literature Review: ICRA 2020: Beyond Photometric Consistency: Gradient-based Dissimiliarity for Impr

Posted 2020-12-13 tweed

Abstract

我们调查了基于光度误差的图像注册的新metric.

我们的方法结合了一个梯度的基于旋转的metric with a magnitude-dependent scaling term.

我们囊括了立体估计和视觉里程计, 展示说对于典型的视差和直接图像注册任务是有益的.

我们的实验显示说有更鲁邦/更准确的位姿深度和相机轨迹.

1. Introduction

gradient orientation-based metric, 通过 magnitude depending scaling term来提升.

我们在四个估计系统里囊括了我们的metric (OpenCV, MeshStereo, DSO和Basalt).

2. Related Work

LSD-SLAM优化了variance-weighted光度误差.

[18]用了NID (Normalized Information Distance) metric for 直接单目SLAM.

3. Our Method

这个metric衡量了图像梯度的orientation, 同时考虑了大小.

basic误差函数:

[e_{ ext {photo}}left(mathbf{u}_{i}, mathbf{u}_{j} ight)=I_{i}left(mathbf{u}_{i} ight)-I_{j}left(mathbf{u}_{j} ight) ]

更鲁邦的版本:

[egin{array}{l} e_{g m}left(mathbf{u}_{i}, mathbf{u}_{j} ight)=left(left| abla I_{i}left(mathbf{u}_{i} ight) ight|-left| abla I_{j}left(mathbf{u}_{j} ight) ight| ight) \mathbf{e}_{g n}left(mathbf{u}_{i}, mathbf{u}_{j} ight)= abla I_{i}left(mathbf{u}_{i} ight)- abla I_{j}left(mathbf{u}_{j} ight) end{array} ]

(e_{gn}) 包括梯度的大小和旋转的不同. PatchMatch Stereo算法[20] 经典的组合了这俩误差:

[e_{p m}left(mathbf{u}_{i}, mathbf{u}_{j} ight)=(1-alpha)left|e_{p h o t o}left(mathbf{u}_{i}, mathbf{u}_{j} ight) ight|+alphaleft|e_{g n}left(mathbf{u}_{i}, mathbf{u}_{j} ight) ight|_{ell_{1}} ]

A. Normalized Gradient-based Direct Image Alignment

一个补足的方法是align梯度方向. 一个天真的方法是使用很费计算力的atan计算来获得旋转 ( heta). 我们按照 [9, 12]的方法使用点乘和它的跟cos的关系作为衡量指标. 如果两个向量 (a, b) 是单位长度, 那么他们的点乘就是角度的 cosine, 0 是垂直, 1 是一样, -1 是相反. 简单的归一化用梯度的magnitude来归一化是我们不需要的, 因为在低梯度的地方, 噪声会统治旋转. 所以[21]通过点乘一个窗的magnitude来归一化:

[e_{g o m}left(mathbf{u}_{i}, mathbf{u}_{j} ight)=1-frac{sum_{u in W}left| abla I_{i}left(mathbf{u}_{i} ight) cdot abla I_{j}left(mathbf{u}_{j} ight) ight|}{sum_{u in W}left| abla I_{i}left(mathbf{u}_{i} ight) ight|left| abla I_{j}left(mathbf{u}_{j} ight) ight|} ]

它有效的降低了低梯度区域的梯度magnitude, 因为 (|| abla_{epsilon}I||) 会接近 0.

在[9]的multi-modal图像注册里, 它最小化每个像素的误差(e_{ngf}).

[e_{n g f}left(mathbf{u}_{i}, mathbf{u}_{j} ight)=1-left[ abla_{varepsilon} I_{i}left(mathbf{u}_{i} ight) cdot abla_{vartheta} I_{j}left(mathbf{u}_{j} ight) ight]^{2} ]

平方点乘, 或者取绝对值, 会导致相同或者相反方向符合. 这个对注册 CT 到 MRT数据或者反之是很重要的, 因为图像可能有相反的方向. 这个误差有一个重要的缺点是低梯度像素倾向于和高magnitude匹配而不是和类似的梯度.

如果最大梯度边缘总是被匹配, 我们会获得不稳定的深度估计, 有大的重投影误差.

因为我们想要使用一样传感器类型的图像, 我们可以忽略平方, 只使用下述的残差:

[e_{u g f}left(mathbf{u}_{i}, mathbf{u}_{j} ight)=1- abla_{vartheta} I_{j}left(mathbf{u}_{j} ight) cdot abla_{varepsilon} I_{i}left(mathbf{u}_{i} ight) ]

误差 (e_{ngf}) 和 (e_{ugf}) 在[0, 2]. 为了保证正确的行为, 我们用最大值scale这个点乘结果

[e_{s g f}left(mathbf{u}_{i}, mathbf{u}_{j} ight)=1-frac{ abla_{vartheta} I_{j}left(mathbf{u}_{j} ight) cdot abla_{varepsilon} I_{i}left(mathbf{u}_{i} ight)}{max left(left| abla_{varepsilon} I_{i}left(mathbf{u}_{i} ight) ight|^{2},left| abla_{vartheta} I_{j}left(mathbf{u}_{j} ight) ight|^{2}, au ight)} ]

这个 scaling term 会提升在半稠密深度估计的正确估计点数. 这里( au) 是一个防止被0除的最小值.

为了减少数学操作, 我们推导了两个旋转和magnitude的组合:

[egin{aligned} nleft(mathbf{u}_{i}, mathbf{u}_{j} ight) &= abla I_{j}left(mathbf{u}_{j} ight) cdot abla I_{i}left(mathbf{u}_{i} ight) i jleft(mathbf{u}_{i}, mathbf{u}_{j} ight) &=frac{left| abla_{vartheta} I_{j}left(mathbf{u}_{j} ight) ight|}{left| abla_{varepsilon} I_{i}left(mathbf{u}_{i} ight) ight|}left| abla I_{i}left(mathbf{u}_{i} ight) ight|^{2} j ileft(mathbf{u}_{i}, mathbf{u}_{j} ight) &=frac{left| abla_{varepsilon} I_{i}left(mathbf{u}_{i} ight) ight|}{left| abla_{vartheta} I_{j}left(mathbf{u}_{j} ight) ight|}left| abla I_{j}left(mathbf{u}_{j} ight) ight|^{2} e_{s g f 2}left(mathbf{u}_{i}, mathbf{u}_{j} ight) &=max (n i j, n j i)-nleft(mathbf{u}_{i}, mathbf{u}_{j} ight) e_{s g f 3}left(mathbf{u}_{i}, mathbf{u}_{j} ight) &=left| abla I_{i}left(mathbf{u}_{i} ight) ight|left| abla I_{j}left(mathbf{u}_{j} ight) ight|-nleft(mathbf{u}_{i}, mathbf{u}_{j} ight) end{aligned} ]

**做立体匹配: **

[egin{aligned} d_{mathbf{u}}^{*} &=underset{d in mathcal{R}}{arg min } sum_{mathbf{u}_{l} in W} eleft(mathbf{u}_{l}, mathbf{u}_{r}(d) ight) \mathbf{u}_{r}(d) &=mathbf{u}_{l}-(d, 0)^{ op} end{aligned} ]

这里视差 (d) 被定义为立体rectified左右图对 x轴的距离. 为了鲁邦, 误差函数 (e) 是在一个patch (W_u) (大小是 (w)) 上计算的.

**做直接图像匹配: **

[T_{c r}=arg min sum_{mathbf{p}_{r} in mathcal{M}} sum_{mathbf{p}_{k} in mathcal{N}_{mathbf{p}_{r}}} holeft(left|eleft(mathbf{p}_{i} ight) ight|^{2} ight) ]

这里( ho) 是huber norm. 雅克比:

[egin{aligned} n n &= abla_{vartheta} I_{j}left(mathbf{u}_{j} ight) cdot abla_{varepsilon} I_{i}left(mathbf{u}_{i} ight) s_{1} &=n nleft{egin{array}{ll} -1, & ext { if }left| abla_{vartheta} I_{j} ight|^{2}>left| abla_{varepsilon} I_{i} ight|^{2} 1-frac{2}{left| abla_{varepsilon} I_{i} ight|}, & ext { otherwise } end{array} ight.\frac{partial e_{s g f}}{partial mathbf{u}_{i}} &=-frac{left( abla_{vartheta} I_{j}+s_{1} abla_{varepsilon} I_{i} ight)^{ op}}{max left(left| abla_{varepsilon} I_{i} ight|,left| abla_{vartheta} I_{j} ight| ight)} frac{left( abla_{2} ight) I_{i}}{left| abla I_{i} ight|_{varepsilon}} s_{2} &=left{egin{array}{ll} frac{left| abla_{vartheta} I_{j} ight|}{left| abla_{varepsilon} I_{i} ight|}left(2-frac{left| abla I_{i} ight|^{2}}{left| abla I_{i} ight|^{2}+varepsilon} ight), & ext { if } n i j>n j i \frac{left| abla_{e} I_{i} ight|}{left| abla_{vartheta} I_{j} ight|} frac{left| abla I_{j} ight|^{2}}{left(left| abla I_{i} ight|^{2}+varepsilon ight)}, & ext { otherwise } end{array} ight.\frac{partial e_{s g f 2}}{partial mathbf{u}_{i}}=&left(s_{2} abla I_{i}- abla I_{j} ight)left( abla_{2} ight) I_{i} \frac{partial e_{s g f 3}}{partial mathbf{u}_{i}} &=left(frac{1}{2} frac{left| abla I_{j} ight|}{left| abla I_{i} ight|} abla I_{i}- abla I_{j} ight)left( abla_{2} ight) I_{i} end{aligned} ]

这里(( abla_2)I_i) 表示在像素 (u_i) 的光度的hessian.

4. Evaluation

我们用ICL-NUIM[10]的图, 改变曝光时间, 加入一个光晕对于帧120和808.

...

5. Conclusion

在这里个文章, 我们提出了一个新的metric做直接的图像alignment.