yolov3 darknet 转 TVM 推理输出一文读懂

Posted 墨理学AI

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了yolov3 darknet 转 TVM 推理输出一文读懂相关的知识，希望对你有一定的参考价值。

🥇 版权: 本文由【墨理学AI】原创、首发、各位大佬、敬请查阅
🎉 声明: 作为全网 AI 领域 干货最多的博主之一，❤️ 不负光阴不负卿 ❤️

  • 🍊 计算机视觉： Yolo专栏、一文读懂

  • 🍊 Yolo 系列推荐：yolov3 darknet 转 TVM Python 推理

  • 📆 最近更新：2022年1月10日

  • 🍊 点赞 👍 收藏 ⭐留言 📝 都是博主坚持写作、更新高质量博文的最大动力！

文章目录

📕 tvm 源码安装

tvm 源码安装，可参考博文

【初识TVM】| LLVM编译 | tvm 源码安装 | deploy ONNX models with Relay 测试【一文读懂】

git clone --recursive https://github.com/apache/tvm.git

cd tvm

mkdir build

cp cmake/config.cmake build

cmake ..

make -j8
  • 直接 cmake … 输出如下
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/moli/project/project21/modelTrans/tvm/build
  • make -j8 输出如下
[100%] Built target tvm_objs
[100%] Linking CXX shared library libtvm.so
[100%] Built target tvm

📕 此次运行Python代码如下

该代码支持 YOLO-V2 and YOLO-V3 DarkNet Models 转 TVM 推理输出

  1. 模型下载【代码自动下载、网速不佳、也可手动下载】
  2. 模型转换【DarkNet Models 转 TVM 】
  3. 模型推理【TVM 推理示例】

"""
Compile YOLO-V2 and YOLO-V3 in DarkNet Models
=============================================
**Author**: `Siju Samuel <https://siju-samuel.github.io/>`_

This article is an introductory tutorial to deploy darknet models with TVM.
All the required models and libraries will be downloaded from the internet by the script.
This script runs the YOLO-V2 and YOLO-V3 Model with the bounding boxes
Darknet parsing have dependancy with CFFI and CV2 library
Please install CFFI and CV2 before executing this script

.. code-block:: bash

  pip install cffi
  pip install opencv-python
"""

"""
**Second release Author**: 墨理学AI 
=============================================
CSDN 博客主页
<https://positive.blog.csdn.net/>

计算机视觉各领域交流群
<https://gitee.com/bravePatch/datasets/blob/master/jindachang.md>
"""

# numpy and matplotlib
import numpy as np
import matplotlib.pyplot as plt
import sys

# tvm, relay
import tvm
from tvm import te
from tvm import relay
from ctypes import *
from tvm.contrib.download import download_testdata
from tvm.relay.testing.darknet import __darknetffi__
import tvm.relay.testing.yolo_detection
import tvm.relay.testing.darknet

######################################################################
# Choose the model
# -----------------------
# Models are: 'yolov2', 'yolov3' or 'yolov3-tiny'

# Model name
MODEL_NAME = "yolov3"

######################################################################
# Download required files
# -----------------------
# Download cfg and weights file if first time.
CFG_NAME = MODEL_NAME + ".cfg"
WEIGHTS_NAME = MODEL_NAME + ".weights"
REPO_URL = "https://github.com/dmlc/web-data/blob/main/darknet/"
CFG_URL = REPO_URL + "cfg/" + CFG_NAME + "?raw=true"
WEIGHTS_URL = "https://pjreddie.com/media/files/" + WEIGHTS_NAME

cfg_path = download_testdata(CFG_URL, CFG_NAME, module="darknet")
weights_path = download_testdata(WEIGHTS_URL, WEIGHTS_NAME, module="darknet")

# Download and Load darknet library
if sys.platform in ["linux", "linux2"]:
    DARKNET_LIB = "libdarknet2.0.so"
    DARKNET_URL = REPO_URL + "lib/" + DARKNET_LIB + "?raw=true"
elif sys.platform == "darwin":
    DARKNET_LIB = "libdarknet_mac2.0.so"
    DARKNET_URL = REPO_URL + "lib_osx/" + DARKNET_LIB + "?raw=true"
else:
    err = "Darknet lib is not supported on  platform".format(sys.platform)
    raise NotImplementedError(err)

lib_path = download_testdata(DARKNET_URL, DARKNET_LIB, module="darknet")

DARKNET_LIB = __darknetffi__.dlopen(lib_path)
net = DARKNET_LIB.load_network(cfg_path.encode("utf-8"), weights_path.encode("utf-8"), 0)
dtype = "float32"
batch_size = 1

data = np.empty([batch_size, net.c, net.h, net.w], dtype)
shape_dict = "data": data.shape
print("Converting darknet to relay functions...")
mod, params = relay.frontend.from_darknet(net, dtype=dtype, shape=data.shape)

######################################################################
# Import the graph to Relay
# -------------------------
# compile the model
target = tvm.target.Target("llvm", host="llvm")
dev = tvm.cpu(0)
data = np.empty([batch_size, net.c, net.h, net.w], dtype)
shape = "data": data.shape
print("Compiling the model...")
with tvm.transform.PassContext(opt_level=3):
    lib = relay.build(mod, target=target, params=params)

[neth, netw] = shape["data"][2:]  # Current image shape is 608x608
######################################################################
# Load a test image
# -----------------
test_image = "dog.jpg"
print("Loading the test image...")
img_url = REPO_URL + "data/" + test_image + "?raw=true"
img_path = download_testdata(img_url, test_image, "data")

data = tvm.relay.testing.darknet.load_image(img_path, netw, neth)
######################################################################
# Execute on TVM Runtime
# ----------------------
# The process is no different from other examples.
from tvm.contrib import graph_executor

m = graph_executor.GraphModule(lib["default"](dev))

# set inputs
m.set_input("data", tvm.nd.array(data.astype(dtype)))
# execute
print("Running the test image...")

# detection
# thresholds
thresh = 0.5
nms_thresh = 0.45

m.run()
# get outputs
tvm_out = []
if MODEL_NAME == "yolov2":
    layer_out = 
    layer_out["type"] = "Region"
    # Get the region layer attributes (n, out_c, out_h, out_w, classes, coords, background)
    layer_attr = m.get_output(2).numpy()
    layer_out["biases"] = m.get_output(1).numpy()
    out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
    layer_out["output"] = m.get_output(0).numpy().reshape(out_shape)
    layer_out["classes"] = layer_attr[4]
    layer_out["coords"] = layer_attr[5]
    layer_out["background"] = layer_attr[6]
    tvm_out.append(layer_out)

elif MODEL_NAME == "yolov3":
    for i in range(3):
        layer_out = 
        layer_out["type"] = "Yolo"
        # Get the yolo layer attributes (n, out_c, out_h, out_w, classes, total)
        layer_attr = m.get_output(i * 4 + 3).numpy()
        layer_out["biases"] = m.get_output(i * 4 + 2).numpy()
        layer_out["mask"] = m.get_output(i * 4 + 1).numpy()
        out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
        layer_out["output"] = m.get_output(i * 4).numpy().reshape(out_shape)
        layer_out["classes"] = layer_attr[4]
        tvm_out.append(layer_out)

elif MODEL_NAME == "yolov3-tiny":
    for i in range(2):
        layer_out = 
        layer_out["type"] = "Yolo"
        # Get the yolo layer attributes (n, out_c, out_h, out_w, classes, total)
        layer_attr = m.get_output(i * 4 + 3).numpy()
        layer_out["biases"] = m.get_output(i * 4 + 2).numpy()
        layer_out["mask"] = m.get_output(i * 4 + 1).numpy()
        out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
        layer_out["output"] = m.get_output(i * 4).numpy().reshape(out_shape)
        layer_out["classes"] = layer_attr[4]
        tvm_out.append(layer_out)
        thresh = 0.560

# do the detection and bring up the bounding boxes
img = tvm.relay.testing.darknet.load_image_color(img_path)
_, im_h, im_w = img.shape
dets = tvm.relay.testing.yolo_detection.fill_network_boxes(
    (netw, neth), (im_w, im_h), thresh, 1, tvm_out
)
last_layer = net.layers[net.n - 1]
tvm.relay.testing.yolo_detection.do_nms_sort(dets, last_layer.classes, nms_thresh)

coco_name = "coco.names"
coco_url = REPO_URL + "data/" + coco_name + "?raw=true"
font_name = "arial.ttf"
font_url = REPO_URL + "data/" + font_name + "?raw=true"
coco_path = download_testdata(coco_url, coco_name, module="data")
font_path = download_testdata(font_url, font_name, module="data")

with open(coco_path) as f:
    content = f.readlines()

names = [x.strip() for x in content]

tvm.relay.testing.yolo_detection.show_detections(img, dets, thresh, names, last_layer.classes)
tvm.relay.testing.yolo_detection.draw_detections(
    font_path, img, dets, thresh, names, last_layer.classes
)

plt.imshow(img.transpose(1, 2, 0))
plt.show()
plt.savefig("yolov3_infer.png")

"""
# 代码运行输出如下：

python yolov3_darknet_infer.py

File /home/moli/.tvm_test_data/darknet/yolov3.cfg exists, skip.
File /home/moli/.tvm_test_data/darknet/yolov3.weights exists, skip.
File /home/moli/.tvm_test_data/darknet/libdarknet2.0.so exists, skip.
Converting darknet to relay functions...
Compiling the model...
One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.
Loading the test image...
File /home/moli/.tvm_test_data/data/dog.jpg exists, skip.
Running the test image...
File /home/moli/.tvm_test_data/data/coco.names exists, skip.
File /home/moli/.tvm_test_data/data/arial.ttf exists, skip.
class:['dog 0.994'] left:127 right:227 top:316 bottom:533
class:['truck 0.9266'] left:471 right:83 top:689 bottom:169
class:['bicycle 0.9984'] left:111 right:113 top:577 bottom:447

"""

📕 yolov3 darknet 转 TVM 推理输出

依赖库

环境搭建比较常规、主要是顺利安装 tvm 即可

代码会自动下载相关模型和 dog.jpg 测试图片，到如下路径

  • /home/moli/.tvm_test_data

推理运行输出效果如下

📗 此次源码仓库地址

📙 预祝各位 2022 前途似锦、可摘星辰

  • 🎉 作为全网 AI 领域 干货最多的博主之一，❤️ 不负光阴不负卿 ❤️
  • ❤️ 如果文章对你有帮助、点赞、评论鼓励博主的每一分认真创作

❤️ 比寻找温暖更重要的是，让自己成为一盏灯火 ❤️

以上是关于yolov3 darknet 转 TVM 推理输出一文读懂的主要内容，如果未能解决你的问题，请参考以下文章

模型推理谈谈 darknet yolo 的 route 算子

yolov3 MNN框架部署C++版

基于Atlas 200 DK的原版YOLOv3（基于Darknet-53）实现（Python版本）

YOLOv3目标检测：原理与Darknet源码解析

在Windows下Darknet.exe的命令参数

YOLO---多个版本的认识