DeepFace 用于提取图像的矢量信息

Posted 2023-03-16

【中文标题】DeepFace 用于提取图像的矢量信息

【英文标题】：DeepFace for extracting vector information of an image

【发布时间】：2021-03-08 01:04:29

【问题描述】：

为了使用 Deepface 进行人脸识别，我正在尝试提取图像的矢量信息以存储在数据库中。所以下次为了匹配，我将提取新图像的矢量信息并查看 db 。如果搜索有结果，那么它是匹配的。我使用了 DeepFace 的验证方法，但它比较了 2 张图像并返回：

from deepface import DeepFace
import os

detected_face = DeepFace.detectFace("sly.jpg")
print (detected_face)

这是上面的输出：

result  = DeepFace.verify("sly1.jpg","sly2.jpg");

为此我得到：

Using VGG-Face model backend and cosine distance.
'verified': True, 'distance': 1.1920928955078125e-07, 'max_threshold_to_verify': 0.4, 'model': 'VGG-Face', 'similarity_metric': 'cosine'

这是比较结果，但我只需要一张图像的信息而不进行比较，因为当测试一张新面孔时，我将有很多记录要搜索（用于矢量信息）。任何帮助将不胜感激。

【问题讨论】：

repo 链接和预期输出是什么？

预期输出是图像的唯一向量表示（DeepFace 可以），我可以保存它。

【参考方案1】：

我假设它是this repo 以及其他同名并使用setup.py 或pip install deepface 安装。

我在 google colab 上对此进行了测试。对于本地使用，请使用 cv2.imshow(...) 而不是 cv2_imshow(...)。

下载测试图片

!wget "http://*.jpg" -O "1.jpg"

!wget "https://*.jpg" -O "2.jpg"

检查图片

import cv2
from google.colab.patches import cv2_imshow
im1 = cv2.imread("1.jpg")
#cv2.imshow("img", im1)
cv2_imshow(im1)

人脸检测

DeepFace.detectFace 的输出返回标准化的裁剪面。对于mtcnn，我得到了形状(224, 224, 3) 的图像。您可以验证和查看图像，

from deepface import DeepFace
import cv2
from google.colab.patches import cv2_imshow

#backends = ['opencv', 'ssd', 'dlib', 'mtcnn']
backends = ['mtcnn']
for backend in backends:
  #face detection and alignment
  detected_face = DeepFace.detectFace("1.jpg", detector_backend = backend)
  
  print(detected_face)
  print(detected_face.shape)

  im = cv2.cvtColor(detected_face * 255, cv2.COLOR_BGR2RGB)
  #cv2.imshow("image", im)
  cv2_imshow(im)

输出

[[[0.12156863 0.05882353 0.02352941]
  [0.2901961  0.18039216 0.1254902 ]
  [0.3137255  0.20392157 0.14901961]
  ...
  [0.06666667 0.01176471 0.01176471]
  [0.05882353 0.01176471 0.00784314]
  [0.03921569 0.00784314 0.00392157]]

 [[0.26666668 0.2        0.16470589]
  [0.19215687 0.08235294 0.02745098]
  [0.33333334 0.22352941 0.16862746]
  ...
  [0.03921569 0.00392157 0.00392157]
  [0.04313726 0.00784314 0.00784314]
  [0.04313726 0.         0.00392157]]

 [[0.11764706 0.05098039 0.01568628]
  [0.21176471 0.10588235 0.05882353]
  [0.44313726 0.3372549  0.27058825]
  ...
  [0.02352941 0.00392157 0.        ]
  [0.02352941 0.00392157 0.        ]
  [0.02745098 0.         0.        ]]

 ...

 [[0.24313726 0.1882353  0.13725491]
  [0.24313726 0.18431373 0.13725491]
  [0.22745098 0.16470589 0.11372549]
  ...
  [0.654902   0.69803923 0.78431374]
  [0.62352943 0.67058825 0.7529412 ]
  [0.38431373 0.4117647  0.45882353]]

 [[0.23529412 0.18039216 0.12941177]
  [0.22352941 0.16862746 0.11764706]
  [0.22745098 0.16470589 0.11764706]
  ...
  [0.6392157  0.69803923 0.78039217]
  [0.6156863  0.6745098  0.75686276]
  [0.36862746 0.40392157 0.4627451 ]]

 [[0.21568628 0.16862746 0.10980392]
  [0.2        0.15294118 0.09803922]
  [0.20784314 0.14901961 0.10196079]
  ...
  [0.6313726  0.6901961  0.77254903]
  [0.6039216  0.6627451  0.74509805]
  [0.36078432 0.39607844 0.4509804 ]]]
(224, 224, 3)

人脸嵌入

既然您正在寻找嵌入向量，您可以在下面获得它。它是verify 函数的修改版本。我保留了两个图像的选项，距离计算，验证，但你可以修改它，只为单个人脸生成人脸嵌入。我没有删除任何未使用的导入。

"""
Modified verify function for face embedding generation
backends = ['opencv', 'ssd', 'dlib', 'mtcnn']
"""

from keras.preprocessing import image
import warnings
warnings.filterwarnings("ignore")
import time
import os
from os import path
from pathlib import Path
import gdown
import numpy as np
import pandas as pd
from tqdm import tqdm
import json
import cv2
from keras import backend as K
import keras
import tensorflow as tf
import pickle

from deepface import DeepFace
from deepface.basemodels import VGGFace, OpenFace, Facenet, FbDeepFace, DeepID
from deepface.extendedmodels import Age, Gender, Race, Emotion
from deepface.commons import functions, realtime, distance as dst


def FaceEmbeddingAndDistance(img1_path, img2_path = '', model_name ='Facenet', distance_metric = 'cosine', model = None, enforce_detection = True, detector_backend = 'mtcnn'):

  #--------------------------------
  #ensemble learning disabled.
  
  if model == None:
    if model_name == 'VGG-Face':
      print("Using VGG-Face model backend and", distance_metric,"distance.")
      model = VGGFace.loadModel()

    elif model_name == 'OpenFace':
      print("Using OpenFace model backend", distance_metric,"distance.")
      model = OpenFace.loadModel()

    elif model_name == 'Facenet':
      print("Using Facenet model backend", distance_metric,"distance.")
      model = Facenet.loadModel()

    elif model_name == 'DeepFace':
      print("Using FB DeepFace model backend", distance_metric,"distance.")
      model = FbDeepFace.loadModel()
    
    elif model_name == 'DeepID':
      print("Using DeepID2 model backend", distance_metric,"distance.")
      model = DeepID.loadModel()
    
    elif model_name == 'Dlib':
      print("Using Dlib ResNet model backend", distance_metric,"distance.")
      from deepface.basemodels.DlibResNet import DlibResNet #this is not a must because it is very huge.
      model = DlibResNet()

    else:
      raise ValueError("Invalid model_name passed - ", model_name)
  else: #model != None
    print("Already built model is passed")

  #------------------------------
  #face recognition models have different size of inputs
  #my environment returns (None, 224, 224, 3) but some people mentioned that they got [(None, 224, 224, 3)]. I think this is because of version issue.
    
  if model_name == 'Dlib': #this is not a regular keras model
    input_shape = (150, 150, 3)
  
  else: #keras based models
    input_shape = model.layers[0].input_shape
    
    if type(input_shape) == list:
      input_shape = input_shape[0][1:3]
    else:
      input_shape = input_shape[1:3]
    
  input_shape_x = input_shape[0]
  input_shape_y = input_shape[1]

  #------------------------------

  #tuned thresholds for model and metric pair
  threshold = functions.findThreshold(model_name, distance_metric)

  #------------------------------
  

  #----------------------
  #crop and align faces

  img1 = functions.preprocess_face(img=img1_path, target_size=(input_shape_y, input_shape_x), enforce_detection = enforce_detection, detector_backend = detector_backend)
  img2 = functions.preprocess_face(img=img2_path, target_size=(input_shape_y, input_shape_x), enforce_detection = enforce_detection, detector_backend = detector_backend)

  #----------------------
  #find embeddings

  img1_representation = model.predict(img1)[0,:]
  img2_representation = model.predict(img2)[0,:]

  print("FACE 1 Embedding:")
  print(img1_representation)

  print("FACE 2 Embedding:")
  print(img2_representation)

  #----------------------
  #find distances between embeddings

  if distance_metric == 'cosine':
    distance = dst.findCosineDistance(img1_representation, img2_representation)
  elif distance_metric == 'euclidean':
    distance = dst.findEuclideanDistance(img1_representation, img2_representation)
  elif distance_metric == 'euclidean_l2':
    distance = dst.findEuclideanDistance(dst.l2_normalize(img1_representation), dst.l2_normalize(img2_representation))
  else:
    raise ValueError("Invalid distance_metric passed - ", distance_metric)

  print("DISTANCE")
  print(distance)

  #----------------------
  #decision

  if distance <= threshold:
    identified =  "true"
  else:
    identified =  "false"

  print("IDENTIFIED")
  print(identified)

上面的函数是通过调用的，

FaceEmbeddingAndDistance("1.jpg", "2.jpg", model_name='Facenet', detector_backend = 'mtcnn')

输出

FACE 1 Embedding:
[-0.7229302  -1.766835   -1.5399052   0.59634393  1.203212   -1.693247
 -0.90845925  0.5264039   2.148173   -0.9786542  -0.00369854 -1.2710322
 -1.5515596  -0.4111185  -0.36896533 -0.30051672  0.35091963  0.5073533
 -1.7270111  -0.5230838   0.3376239  -1.0811361   1.5242224  -0.6137103
 -1.3100258   0.80050004 -0.7087368  -0.64483845  1.0830203   2.6056807
 -0.76527536 -0.83047277 -0.7335422  -0.01964059 -0.86749244  2.9645889
 -2.426583   -0.11157394 -2.3535717  -0.65058017  0.30864614 -0.77746457
 -0.6233895   0.44898677  2.5578005  -0.583796    0.8406945   1.1105415
 -1.652044   -0.6351479   0.07651432 -1.0454555  -1.8752071   0.50948805
 -1.6050931  -1.1769634  -0.02965304  1.5107706   0.83292925 -0.5382068
 -1.5981512  -0.6405941   0.5521577   0.22957848  0.506649    0.24680384
 -0.91464925 -0.18441322 -0.6801975  -1.0448433   0.52288735 -0.79405725
  0.5974493  -0.40668172 -0.00640235 -0.742475    0.1928863   0.31236258
 -0.37383577 -1.5883486  -1.5336255  -0.74254227 -0.8524561  -1.4625055
 -2.718953   -0.7180952  -1.2140683  -0.5232462   1.2576898  -1.1097553
  2.3971314   0.8855096  -0.16556528 -0.07307663 -1.8778017   0.8690948
 -0.39043528 -0.5494097  -2.2382076   0.7101087   0.15859437  0.2959841
  0.8605075  -0.2040207   0.77952844  0.04542177  0.92514265 -1.988945
  0.9418363   1.6509243  -0.20324889  0.2974357   0.37681833  1.095943
  1.6308782  -1.2553837  -0.10246387 -1.4697052  -0.5832107  -0.34192032
 -1.1347024   1.5154309  -0.00527111 -1.165709   -0.7296148  -0.20767921
  1.2530949  -0.9487353 ]
FACE 2 Embedding:
[ 0.9399996   1.3996615  -1.2931366   0.6869738  -0.03219241  0.96111965
  0.7378809  -0.24804354 -0.8128112   0.19901593  0.48911542 -0.91603553
 -1.1671298   0.88576627  0.25427592  1.1395477   0.45400882 -1.4845027
 -0.90582514 -1.1371222   0.47669724  1.2933927   1.4533392  -0.46943524
  0.10245587 -1.4916894  -2.3223586  -0.10979578  1.7803721   1.0051152
 -0.09164213 -0.64848715 -1.4191641   1.811776    0.73174113  0.2582223
 -0.26430857  1.7021953  -1.0571098  -1.1215096   0.3606074   1.5136883
 -0.30045512  0.26225814 -0.19101554  1.269355    1.0674374  -0.2550623
 -1.0582973   1.7474637  -1.7739134  -0.67914337 -0.1877765   1.1581128
 -2.281225    1.3955555  -1.2690883  -0.16299461  1.337664   -0.8831901
 -0.6862674   2.0526903  -0.6325836   1.333468   -0.10851342 -0.64831966
 -1.0277263   1.4572504  -0.29905424 -0.33187118 -0.54727656  1.1528811
  0.12454037 -1.5835186  -0.2271783   1.3911225   1.0170195   0.5741334
 -1.3088373  -0.5950714  -0.6856393  -0.910367   -2.0136826  -0.73777384
  0.319223   -2.1968741   0.9673934  -0.604423   -0.08049382 -1.948634
  1.88159     0.20169139  0.7295723  -1.0224706   1.2995481  -0.3402595
  1.1711328  -0.64862376  0.42063504 -0.01502114 -0.7048841   1.4360497
 -1.2988033   0.31773448  1.534014    0.98858756  1.3450235  -0.9417385
  0.26414695 -0.01988658  0.7418235  -0.04945141 -0.44838902  1.5288658
 -1.1905407   0.13961646 -0.17101136 -0.18599203 -1.9648114   0.66071814
 -0.07431012  1.5870664   1.5989372  -0.21751085  0.78908855 -1.5576671
  0.02266342  0.20999858]
DISTANCE
0.807837575674057
IDENTIFIED
false

【讨论】：

谢谢，它成功了.....我正在尝试从用户那里获取图像作为输入并将其作为参数传递给 Python

您可以传递 numpy 数组或 base64 编码图像作为输入，而不是 deepface 中的确切路径。

【参考方案2】：

在 DeepFace 0.0.41 中变得更容易了。

from deepface import DeepFace
from deepface.commons import functions

models = ['VGG-Face', 'Facenet', 'OpenFace', 'DeepFace', 'DeepID', 'Dlib']
model = DeepFace.build_model(models[0])
target_size = model.layers[0].input_shape

img1_path = "img1.jpg"
img2_path = "img2.jpg"

#detect and align 
img1 = functions.preprocess_face(img1_path, target_size = target_size)
img2 = functions.preprocess_face(img2_path, target_size = target_size)

#find vector embeddings
img1_embedding = model.predict(img1)
img2_embedding = model.predict(img2)