Abstract
我们调查了基于光度误差的图像注册的新metric.
我们的方法结合了一个梯度的基于旋转的metric with a magnitude-dependent scaling term.
我们囊括了立体估计和视觉里程计, 展示说对于典型的视差和直接图像注册任务是有益的.
我们的实验显示说有更鲁邦/更准确的位姿深度和相机轨迹.
1. Introduction
gradient orientation-based metric, 通过 magnitude depending scaling term来提升.
我们在四个估计系统里囊括了我们的metric (OpenCV, MeshStereo, DSO和Basalt).
LSD-SLAM优化了variance-weighted光度误差.
[18]用了NID (Normalized Information Distance) metric for 直接单目SLAM.
3. Our Method
这个metric衡量了图像梯度的orientation, 同时考虑了大小.
basic误差函数:
[e_{ ext {photo}}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=I_{i}left(mathbf{u}_{i}
ight)-I_{j}left(mathbf{u}_{j}
ight)
]
更鲁邦的版本:
[egin{array}{l}
e_{g m}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=left(left|
abla I_{i}left(mathbf{u}_{i}
ight)
ight|-left|
abla I_{j}left(mathbf{u}_{j}
ight)
ight|
ight) \
mathbf{e}_{g n}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=
abla I_{i}left(mathbf{u}_{i}
ight)-
abla I_{j}left(mathbf{u}_{j}
ight)
end{array}
]
(e_{gn}) 包括梯度的大小和旋转的不同. PatchMatch Stereo算法[20] 经典的组合了这俩误差:
[e_{p m}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=(1-alpha)left|e_{p h o t o}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)
ight|+alphaleft|e_{g n}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)
ight|_{ell_{1}}
]
A. Normalized Gradient-based Direct Image Alignment
一个补足的方法是align梯度方向. 一个天真的方法是使用很费计算力的atan计算来获得旋转 ( heta). 我们按照 [9, 12]的方法使用点乘和它的跟cos的关系作为衡量指标. 如果两个向量 (a, b) 是单位长度, 那么他们的点乘就是角度的 cosine, 0 是垂直, 1 是一样, -1 是相反. 简单的归一化用梯度的magnitude来归一化是我们不需要的, 因为在低梯度的地方, 噪声会统治旋转. 所以[21]通过点乘一个窗的magnitude来归一化:
[e_{g o m}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=1-frac{sum_{u in W}left|
abla I_{i}left(mathbf{u}_{i}
ight) cdot
abla I_{j}left(mathbf{u}_{j}
ight)
ight|}{sum_{u in W}left|
abla I_{i}left(mathbf{u}_{i}
ight)
ight|left|
abla I_{j}left(mathbf{u}_{j}
ight)
ight|}
]
它有效的降低了低梯度区域的梯度magnitude, 因为 (||
abla_{epsilon}I||) 会接近 0.
在[9]的multi-modal图像注册里, 它最小化每个像素的误差(e_{ngf}).
[e_{n g f}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=1-left[
abla_{varepsilon} I_{i}left(mathbf{u}_{i}
ight) cdot
abla_{vartheta} I_{j}left(mathbf{u}_{j}
ight)
ight]^{2}
]
平方点乘, 或者取绝对值, 会导致相同或者相反方向符合. 这个对注册 CT 到 MRT数据或者反之是很重要的, 因为图像可能有相反的方向. 这个误差有一个重要的缺点是低梯度像素倾向于和高magnitude匹配而不是和类似的梯度.
如果最大梯度边缘总是被匹配, 我们会获得不稳定的深度估计, 有大的重投影误差.
因为我们想要使用一样传感器类型的图像, 我们可以忽略平方, 只使用下述的残差:
[e_{u g f}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=1-
abla_{vartheta} I_{j}left(mathbf{u}_{j}
ight) cdot
abla_{varepsilon} I_{i}left(mathbf{u}_{i}
ight)
]
误差 (e_{ngf}) 和 (e_{ugf}) 在[0, 2]. 为了保证正确的行为, 我们用最大值scale这个点乘结果
[e_{s g f}left(mathbf{u}_{i}, mathbf{u}_{j}
ight)=1-frac{
abla_{vartheta} I_{j}left(mathbf{u}_{j}
ight) cdot
abla_{varepsilon} I_{i}left(mathbf{u}_{i}
ight)}{max left(left|
abla_{varepsilon} I_{i}left(mathbf{u}_{i}
ight)
ight|^{2},left|
abla_{vartheta} I_{j}left(mathbf{u}_{j}
ight)
ight|^{2}, au
ight)}
]
这个 scaling term 会提升在半稠密深度估计的正确估计点数. 这里( au) 是一个防止被0除的最小值.
为了减少数学操作, 我们推导了两个旋转和magnitude的组合:
[egin{aligned}
nleft(mathbf{u}_{i}, mathbf{u}_{j}
ight) &=
abla I_{j}left(mathbf{u}_{j}
ight) cdot
abla I_{i}left(mathbf{u}_{i}
ight) \
n i jleft(mathbf{u}_{i}, mathbf{u}_{j}
ight) &=frac{left|
abla_{vartheta} I_{j}left(mathbf{u}_{j}
ight)
ight|}{left|
abla_{varepsilon} I_{i}left(mathbf{u}_{i}
ight)
ight|}left|
abla I_{i}left(mathbf{u}_{i}
ight)
ight|^{2} \
n j ileft(mathbf{u}_{i}, mathbf{u}_{j}
ight) &=frac{left|
abla_{varepsilon} I_{i}left(mathbf{u}_{i}
ight)
ight|}{left|
abla_{vartheta} I_{j}left(mathbf{u}_{j}
ight)
ight|}left|
abla I_{j}left(mathbf{u}_{j}
ight)
ight|^{2} \
e_{s g f 2}left(mathbf{u}_{i}, mathbf{u}_{j}
ight) &=max (n i j, n j i)-nleft(mathbf{u}_{i}, mathbf{u}_{j}
ight) \
e_{s g f 3}left(mathbf{u}_{i}, mathbf{u}_{j}
ight) &=left|
abla I_{i}left(mathbf{u}_{i}
ight)
ight|left|
abla I_{j}left(mathbf{u}_{j}
ight)
ight|-nleft(mathbf{u}_{i}, mathbf{u}_{j}
ight)
end{aligned}
]
**做立体匹配: **
[egin{aligned}
d_{mathbf{u}}^{*} &=underset{d in mathcal{R}}{arg min } sum_{mathbf{u}_{l} in W} eleft(mathbf{u}_{l}, mathbf{u}_{r}(d)
ight) \
mathbf{u}_{r}(d) &=mathbf{u}_{l}-(d, 0)^{ op}
end{aligned}
]
这里视差 (d) 被定义为立体rectified左右图对 x轴的距离. 为了鲁邦, 误差函数 (e) 是在一个patch (W_u) (大小是 (w)) 上计算的.
**做直接图像匹配: **
[T_{c r}=arg min sum_{mathbf{p}_{r} in mathcal{M}} sum_{mathbf{p}_{k} in mathcal{N}_{mathbf{p}_{r}}}
holeft(left|eleft(mathbf{p}_{i}
ight)
ight|^{2}
ight)
]
这里(
ho) 是huber norm. 雅克比:
[egin{aligned}
n n &=
abla_{vartheta} I_{j}left(mathbf{u}_{j}
ight) cdot
abla_{varepsilon} I_{i}left(mathbf{u}_{i}
ight) \
s_{1} &=n nleft{egin{array}{ll}
-1, & ext { if }left|
abla_{vartheta} I_{j}
ight|^{2}>left|
abla_{varepsilon} I_{i}
ight|^{2} \
1-frac{2}{left|
abla_{varepsilon} I_{i}
ight|}, & ext { otherwise }
end{array}
ight.\
frac{partial e_{s g f}}{partial mathbf{u}_{i}} &=-frac{left(
abla_{vartheta} I_{j}+s_{1}
abla_{varepsilon} I_{i}
ight)^{ op}}{max left(left|
abla_{varepsilon} I_{i}
ight|,left|
abla_{vartheta} I_{j}
ight|
ight)} frac{left(
abla_{2}
ight) I_{i}}{left|
abla I_{i}
ight|_{varepsilon}} \
s_{2} &=left{egin{array}{ll}
frac{left|
abla_{vartheta} I_{j}
ight|}{left|
abla_{varepsilon} I_{i}
ight|}left(2-frac{left|
abla I_{i}
ight|^{2}}{left|
abla I_{i}
ight|^{2}+varepsilon}
ight), & ext { if } n i j>n j i \
frac{left|
abla_{e} I_{i}
ight|}{left|
abla_{vartheta} I_{j}
ight|} frac{left|
abla I_{j}
ight|^{2}}{left(left|
abla I_{i}
ight|^{2}+varepsilon
ight)}, & ext { otherwise }
end{array}
ight.\
frac{partial e_{s g f 2}}{partial mathbf{u}_{i}}=&left(s_{2}
abla I_{i}-
abla I_{j}
ight)left(
abla_{2}
ight) I_{i} \
frac{partial e_{s g f 3}}{partial mathbf{u}_{i}} &=left(frac{1}{2} frac{left|
abla I_{j}
ight|}{left|
abla I_{i}
ight|}
abla I_{i}-
abla I_{j}
ight)left(
abla_{2}
ight) I_{i}
end{aligned}
]
这里((
abla_2)I_i) 表示在像素 (u_i) 的光度的hessian.
4. Evaluation
我们用ICL-NUIM[10]的图, 改变曝光时间, 加入一个光晕对于帧120和808.
...
5. Conclusion
在这里个文章, 我们提出了一个新的metric做直接的图像alignment.