dlib库包的介绍与使用，opencv+dlib检测人脸框opencv+dlib进行人脸68关键点检测，opencv+dlib实现人脸识别，dlib进行人脸特征聚类dlib视频目标跟踪

Posted 2023-04-07 点亮～黑夜

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了dlib库包的介绍与使用，opencv+dlib检测人脸框opencv+dlib进行人脸68关键点检测，opencv+dlib实现人脸识别，dlib进行人脸特征聚类dlib视频目标跟踪相关的知识，希望对你有一定的参考价值。

文章目录：

1 dlib库介绍
2 dlib人脸检测：绘制出人脸检测框
3 dlib人脸关键点检测：并绘制检测框、关键点、不同区域关键点连线
- 3.1 dlib人脸关键点检测源码
- 3.2 opencv + dlib 进行人脸关键点检测
4 dlib人脸识别
5 dlib人脸聚类
6 dlib视频目标跟踪

1 dlib库介绍

dlib官网：http://dlib.net/
dlib模型文件和源码下载：http://dlib.net/files/

dlib介绍

dlib人脸检测与人脸识别

2 dlib人脸检测：绘制出人脸检测框

2.1 dlib人脸检测源码

1、人脸检测，dlib官方例子face_detector.py

face detector

这个人脸检测器是使用现在经典的直方图定向梯度(HOG)特征，结合线性分类器，图像金字塔和滑动窗口检测方案。这种类型的物体检测器是相当普遍的，除了人脸之外，还能够检测许多类型的半刚性物体。因此，如果您对制作自己的对象检测器感兴趣，那么请阅读train_object _detector.py示例程序。

# coding=utf-8
"""
Copyright (c) 2018-2022. All Rights Reserved.

@author: shliang
@email: shliang0603@gmail.com
"""

import dlib


def dlib_detector_face(img_path):
    print("Processing file: ".format(img_path))
    img = dlib.load_rgb_image(img_path)
    # dlib正面人脸检测器
    detector = dlib.get_frontal_face_detector()
    win = dlib.image_window()
    # 1 表示图像向上采样一次，图像将被放大一倍，这样可以检测更多的人脸
    dets = detector(img, 1)
    print("Number of faces detected: ".format(len(dets)))
    for i, d in enumerate(dets):
        # 人脸的左上和右下角坐标
        print("Detection : Left:  Top:  Right:  Bottom: ".format(i, d.left(), d.top(), d.right(), d.bottom()))

    win.clear_overlay()
    win.set_image(img)
    win.add_overlay(dets)
    dlib.hit_enter_to_continue()

    # 让检测器获取人脸得分score，约有把握的检测，得分越高。
    # 第三个参数调整检测的阈值，负值将返回更多的检测结果，而正值将返回较少的检测结果
    # 同时idx会告诉你那个面部对应的检测结果，用来广泛识别不同方向的面孔，idx的具体含义我还没有弄清楚
    # 如果第三个参数设置位-2就会输出更多的检测结果，如果你设置为1可能就没有检测结果
    dets, scores, idx = detector.run(img, 1, -1)   # 把-1以上的score的检测结果全部输出
    for i, d in enumerate(dets):
        print("Detection , score: , face_type:".format(d, scores[i], idx[i]))


if __name__ == '__main__':
    dlib_detector_face('./images/single_face.jpg')
    dlib_detector_face('./images/multi_face.jpg')

输出结果：

Processing file: ./images/single_face.jpg
Number of faces detected: 1
Detection 0: Left: 118 Top: 139 Right: 304 Bottom: 325
Hit enter to continue
Detection [(118, 139) (304, 325)], score: 1.6567257084382887, face_type:0
Detection [(55, 218) (98, 262)], score: -0.6576982940533731, face_type:2

Processing file: ./images/multi_face.jpg
Number of faces detected: 3
Detection 0: Left: 339 Top: 82 Right: 468 Bottom: 211
Detection 1: Left: 163 Top: 140 Right: 270 Bottom: 247
Detection 2: Left: 265 Top: 136 Right: 354 Bottom: 226
Hit enter to continue
Detection [(339, 82) (468, 211)], score: 1.9180631791420404, face_type:1
Detection [(163, 140) (270, 247)], score: 1.5263808407319899, face_type:0
Detection [(265, 136) (354, 226)], score: 1.153857532931697, face_type:0

如下是检测结果：

2.2 opencv + dlib 人脸检测

1、把上面的代码改用opencv读取与显示

# coding=utf-8
"""
Copyright (c) 2018-2022. All Rights Reserved.

@author: shliang
@email: shliang0603@gmail.com
"""

import dlib
import cv2


def dlib_detector_face(img_path):
    print("Processing file: ".format(img_path))
    img = cv2.imread(img_path)
    # dlib正面人脸检测器
    detector = dlib.get_frontal_face_detector()
    # 1 表示图像向上采样一次，图像将被放大一倍，这样可以检测更多的人脸
    dets = detector(img, 1)
    print('dets:', dets)  # dets: rectangles[[(118, 139) (304, 325)]]
    print("Number of faces detected: ".format(len(dets)))
    for i, d in enumerate(dets):
        # 人脸的左上和右下角坐标
        left = d.left()
        top = d.top()
        right = d.right()
        bottom = d.bottom()
        print("Detection : Left:  Top:  Right:  Bottom: ".format(i, left, top, right, bottom))
        cv2.rectangle(img, (left, top), (right, bottom), color=(0, 255, 255), thickness=1, lineType=cv2.LINE_AA)
    cv2.imshow('detect result', img)
    cv2.waitKey(0)


if __name__ == '__main__':
    dlib_detector_face('./images/single_face.jpg')
    dlib_detector_face('./images/multi_face.jpg')

2.3 dlib人脸检测总结

1、dlib人脸检测效果

dlib人脸检测并不是很准确，如果人脸比较小，有大角度偏转，很有可能就检测不到人脸，我以为dlib版本更新了，人脸检测的模型也会更新，其实不是，dlib是一个机器学习库，里面有很多算法的！

2、使用dlib的场景

最好是正脸
图像分辨率不能太小
图像清晰

3 dlib人脸关键点检测：并绘制检测框、关键点、不同区域关键点连线

3.1 dlib人脸关键点检测源码

1、人脸关键点检测，dlib官方例子face_detector.py

face detector

# coding=utf-8
"""
Copyright (c) 2018-2022. All Rights Reserved.

@author: shliang
@email: shliang0603@gmail.com
"""



import dlib

# 下载人脸关键点检测模型： http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
predictor_path = './model/shape_predictor_68_face_landmarks.dat'


detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(predictor_path)
win = dlib.image_window()

# face_image_path = './images/single_face.jpg'
face_image_path = './images/multi_face.jpg'
print("Processing file: ".format(face_image_path))
img = dlib.load_rgb_image(face_image_path)

win.clear_overlay()
win.set_image(img)

# Ask the detector to find the bounding boxes of each face. The 1 in the
# second argument indicates that we should upsample the image 1 time. This
# will make everything bigger and allow us to detect more faces.
dets = detector(img, 1)
print("Number of faces detected: ".format(len(dets)))
for k, d in enumerate(dets):
    print("Detection : Left:  Top:  Right:  Bottom: ".format(
        k, d.left(), d.top(), d.right(), d.bottom()))
    # Get the landmarks/parts for the face in box d.
    shape = predictor(img, d)
    print("Part 0: , Part 1:  ...".format(shape.part(0),
                                              shape.part(1)))
    # Draw the face landmarks on the screen.
    win.add_overlay(shape)

win.add_overlay(dets)
dlib.hit_enter_to_continue()

3.2 opencv + dlib 进行人脸关键点检测

# coding=utf-8
"""
Copyright (c) 2018-2022. All Rights Reserved.

@author: shliang
@email: shliang0603@gmail.com
"""
import os
import cv2
import dlib
import numpy as np


pred_types = 'face': ((0, 17), (0.682, 0.780, 0.909, 0.5)),
              'eyebrow1': ((17, 22), (1.0, 0.498, 0.055, 0.4)),
              'eyebrow2': ((22, 27), (1.0, 0.498, 0.055, 0.4)),
              'nose': ((27, 31), (0.345, 0.239, 0.443, 0.4)),
              'nostril': ((31, 36), (0.345, 0.239, 0.443, 0.4)),
              'eye1': ((36, 42), (0.596, 0.875, 0.541, 0.3)),
              'eye2': ((42, 48), (0.596, 0.875, 0.541, 0.3)),
              'lips': ((48, 60), (0.596, 0.875, 0.541, 0.3)),
              'teeth': ((60, 68), (0.596, 0.875, 0.541, 0.4))
              

# 绘制直线和关键点
def draw_line(img, shape, i):
    cv2.line(img, pt1=(shape.part(i).x, shape.part(i).y), pt2=(shape.part(i+1).x, shape.part(i+1).y),
             color=(0, 255, 0), thickness=1, lineType=cv2.LINE_AA)

# 连成一圈
def draw_line_circle(img, shape, i, start, end):
    cv2.line(img, pt1=(shape.part(i).x, shape.part(i).y), pt2=(shape.part(i + 1).x, shape.part(i + 1).y),
             color=(0, 255, 0), thickness=1, lineType=cv2.LINE_AA)

    cv2.line(img, pt1=(shape.part(start).x, shape.part(start).y), pt2=(shape.part(end).x, shape.part(end).y),
             color=(0, 255, 0), thickness=1, lineType=cv2.LINE_AA)



# 使用训练好的模型shape_predictor_68_face_landmarks.dat检测出人脸上的68个关键点
def dlib_face_keypoint_detector(img_path, save_result=True):

    # 检测人脸框
    detector = dlib.get_frontal_face_detector()

    # 下载人脸关键点检测模型： http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2
    predictor_path = './model/shape_predictor_68_face_landmarks.dat'
    # 检测人脸关键点
    predictor = dlib.shape_predictor(predictor_path)

    img = cv2.imread(img_path)
    print("Processing file: ".format(img_path))

    # # 1 表示图像向上采样一次，图像将被放大一倍，这样可以检测更多的人脸
    dets = detector(img, 1)
    print("Number of faces detected: ".format(len(dets)))
    for k, d in enumerate(dets):
        x1 = d.left()
        y1 = d.top()
        x2 = d.right()
        y2 = d.bottom()
        print("Detection : Left:  Top:  Right:  Bottom: ".format(k, x1, y1, x2, y2))
        cv2.rectangle(img, (x1, y1), (x2, y2), color=(0, 255, 255), thickness=1, lineType=cv2.LINE_AA)
        # Get the landmarks/parts for the face in box d.
        shape = predictor(img, d)
        # print(dir(shape))  # 'num_parts', 'part', 'parts', 'rect'
        print(shape.num_parts)  # 68   打印出关键点的个数
        print(shape.rect)  # 检测到每个面部的矩形框 [(118, 139) (304, 325)]
        print(shape.parts())  # points[(147, 182), (150, 197), (154, 211), (160, 225),...,(222, 227), (215, 228)]   # 68个关键点坐标
        # print(type(shape.part(0)))  # <class 'dlib.point'>
        # 打印出第一个关键点和第2个关键点的坐标
        print("Part 0: , Part 1:  ...".format(shape.part(0), shape.part(1)))

        # 可以把关键点转换成shape为(68,2)的矩阵
        landmarks = np.matrix([[p.x, p.y] for p in shape.parts()])
        # 绘制所有的关键点
        for i, point in enumerate(shape.parts()):
            x = point.x
            y = point.y
            cv2.circle(img, (x, y), 1, color=(255, 0, 255), thickness=1, lineType=cv2.LINE_AA)
            cv2.putText(img, str(i+1), (x, y), fontFace=cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,
                        fontScale=0.3, color=(255, 255, 0))

            # 连接关键点
            if i + 1 < 17:  # face
                draw_line(img, shape, i)
            elif 17 < i + 1 < 22:  # eyebrow1
                draw_line(img, shape, i)
            elif 22 < i + 1 < 27:  # eyebrow2
                draw_line(img, shape, i)
            elif 27 < i + 1 < 31:  # nose
                draw_line(img, shape, i)
            elif 31 < i + 1 < 36:  # nostril
                draw_line(img, shape, i)
            elif 36 < i + 1 < 42:  # eye1
                draw_line_circle(img, shape, i, 36, 42 - 1)
            elif 42 < i + 1 < 48:  # eye2
                draw_line_circle(img, shape, i, 42, 48 - 1)
            elif 48 < i + 1 < 60:  # lips
                draw_line_circle(img, shape, i, 48, 60 - 1)
            elif 60 < i + 1 < 68:  # teeth
                draw_line_circle(img, shape, i, 60, 68 - 1)

    cv2.imshow('detect keypoints', img)
    if save_result:
        dir, filename = os.path.split(img_path)
        save_filename = os.path.join(dir, filename.split('.')[0] + '_keypoint' + '.' + filename.split('.')[1])
        cv2.imwrite(save_filename, img)
    cv2.waitKey(0)


if __name__ == '__main__':
    dlib_face_keypoint_detector('./images/single_face.jpg')
    # dlib_face_keypoint_detector('./images/face.jpg')
    dlib_face_keypoint_detector('./images/multi_face.jpg')

注意：

    # 检测人脸框
    detector opencv基础opencv和dlib库中rectangle类型之间的转换

前言
最近使用dlib库的同时也会用到opencv，特别是由于对dlib库的画图函数不熟悉，都想着转换到opencv进行show。本文介绍一下两种开源库中rectangle类型之间的转换。
类型说明
opencv中cv::Rect     以及opencv中的rectangle函数：

void cv::rectangle( InputOutputArray img, Point pt1, Point pt2, const Scalar & color, int thickness = 1, int lineType = LINE_8, int shift = 0)     

或者

void cv::rectangle(Mat & img, Rect rec, const Scalar & color, int thickness = 1, int lineType = LINE_8, int shift = 0)         

dlib中的rectangle类型：

rectangle ( long left_, long top_, long right_, long bottom_ );

或者

template <typename T> rectangle ( const vector<T,2>& p1, const vector<T,2>& p2);

如何转换

static cv::Rect dlibRectangleToOpenCV(dlib::rectangle r)
{
    return cv::Rect(cv::Point2i(r.left(), r.top()), cv::Point2i(r.right() + 1, r.bottom() + 1));
}

或者

static dlib::rectangle openCVRectToDlib(cv::Rect r)
{
    return dlib::rectangle((long)r.tl().x, (long)r.tl().y, (long)r.br().x - 1, (long)r.br().y - 1);
}

或者

dets.l = detect_rect.x;
dets.t = detect_rect.y;
dets.r = detect_rect.x + detect_rect.width;
dets.b = detect_rect.y + detect_rect.height;

其中detect_rect是opencv类型，dets是dlib类型；
参考
1.opencv中的Rect类型；
2.dlib中的rectangle类型；
3.stackoverflow-convert-opencvs-rect-to-dlibs-rectangle；
4.dlib和opencv之间类型格式的转换；
完
以上是关于dlib库包的介绍与使用，opencv+dlib检测人脸框opencv+dlib进行人脸68关键点检测，opencv+dlib实现人脸识别，dlib进行人脸特征聚类dlib视频目标跟踪的主要内容，如果未能解决你的问题，请参考以下文章 
 教你快速使用OpenCV/Python/dlib进行眨眼检测识别！
 Anaconda+linux +opencv+dlib安装
 opencv基础opencv和dlib库中rectangle类型之间的转换
 Python+OpenCV+dlib实现人脸检测与表情识别
 安装/使用 dlib 以与 Eclipse CDT、MinGW 和 OpenCV 一起使用时出错
 图片人脸检测——Dlib版