cv2 Farneback 光流值太低

Posted 2023-02-21

【中文标题】cv2 Farneback 光流值太低

【英文标题】：cv2 Farneback Optical FLow values are too low

【发布时间】：2020-05-07 00:46:29

【问题描述】：

我正在尝试计算两帧之间的光流，然后使用计算的光流扭曲前一帧。我发现 cv2 有 Farneback Optical FLow，所以我用它来计算 Flow。我从cv2 tutorial 中获取默认参数，并使用this answer 中给出的代码扭曲框架。但是当我看到扭曲的帧时，它与前一帧完全一样，没有变化（数组相等）。

经过进一步调试，我发现计算出的流量值太低了。为什么会这样？我是不是做错了什么？

代码：

def get_optical_flow(prev_frame: numpy.ndarray, next_frame: numpy.ndarray) -> numpy.ndarray:
    prev_gray = skimage.color.rgb2gray(prev_frame)
    next_gray = skimage.color.rgb2gray(next_frame)
    flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    return flow


def warp_frame(prev_frame: numpy.ndarray, flow: numpy.ndarray):
    h, w = flow.shape[:2]
    flow = -flow
    flow[:,:,0] += numpy.arange(w)
    flow[:,:,1] += numpy.arange(h)[:,numpy.newaxis]
    # res = cv2.remap(img, flow, None, cv2.INTER_LINEAR)
    next_frame = cv2.remap(prev_frame, flow, None, cv2.INTER_LINEAR)
    return next_frame


def demo1():
    prev_frame_path = Path('./frame025.png')
    next_frame_path = Path('./frame027.png')
    prev_frame = skimage.io.imread(prev_frame_path.as_posix())
    next_frame = skimage.io.imread(next_frame_path.as_posix())
    flow = get_optical_flow(prev_frame, next_frame)
    print(f'Flow: max:flow.max(), min:flow.min(), mean:flow.__abs__().mean()')
    warped_frame = warp_frame(prev_frame, flow)

    print(numpy.array_equal(prev_frame, warped_frame))

    pyplot.subplot(1,3,1)
    pyplot.imshow(prev_frame)
    pyplot.subplot(1,3,2)
    pyplot.imshow(next_frame)
    pyplot.subplot(1,3,3)
    pyplot.imshow(warped_frame)
    pyplot.show()
    return

输入图像：

输出： Warped Image 与 prev image 完全相同，而它应该看起来像 next image。

感谢任何帮助！

【参考方案1】：

问题在于将 rgb 帧转换为灰色。 skimage.color.rgb2gray() 将强度范围从 [0,255] 更改为 [0,1]。将其改回[0,255] 有效！

def get_optical_flow(prev_frame: numpy.ndarray, next_frame: numpy.ndarray) -> numpy.ndarray:
    prev_gray = (skimage.color.rgb2gray(prev_frame) * 255).astype('uint8')
    next_gray = (skimage.color.rgb2gray(next_frame) * 255).astype('uint8')
    flow = cv2.calcOpticalFlowFarneback(prev_gray, next_gray, None, 0.5, 3, 15, 3, 5, 1.2, 0)
    return flow