56训练PaddleDetection的tinypose128x96关键点模型部署ncnn框架mnn框架openvino框架
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了56训练PaddleDetection的tinypose128x96关键点模型部署ncnn框架mnn框架openvino框架相关的知识，希望对你有一定的参考价值。
基本思想：需要移植一个自训练的关键点模型到某开发板上,看到社区已经有了，记录一下流程ncnn/mnn框架流程，然后在移植开发板上，以备自己以后自训练使用
实验数据
链接: https://pan.baidu.com/s/1Bd8lLtmWLyX64Jzc5743EQ?pwd=yhgh 提取码: yhgh
一、配置环境
ubuntu@ubuntu:~$ pip install paddlepaddle-gpu==2.2.2.post101 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html -i https://mirror.baidu.com/pypi/simple
下载源码
ubuntu@ubuntu:~$ git clone https://github.com/PaddlePaddle/PaddleDetection.git
Cloning into PaddleDetection...
remote: Enumerating objects: 255570, done.
remote: Counting objects: 100% (63/63), done.
remote: Compressing objects: 100% (47/47), done.
remote: Total 255570 (delta 22), reused 37 (delta 16), pack-reused 255507
Receiving objects: 100% (255570/255570), 414.49 MiB | 856.00 KiB/s, done.
Resolving deltas: 100% (208587/208587), done
下载模型
https://github.com/PaddlePaddle/PaddleDetection/tree/release/2.4/configs/keypoint/tiny_pose
测试检测+识别
ubuntu@ubuntu:~/PaddleDetection$ python3 deploy/python/det_keypoint_unite_infer.py --det_model_dir=/home/ubuntu/PaddleDetection/picodet_v2_s_192_pedestrian --keypoint_model_dir=/home/ubuntu/PaddleDetection/tinypose_128x96 --image_file=/home/ubuntu/PaddleDetection/demo/hrnet_demo.jpg --device=cpu
keypoint visualize image saved to: output/hrnet_demo_vis.jpg
------------------ Inference Time Info ----------------------
total_time(ms): 55.599999999999994, img_num: 1
average latency time(ms): 55.60, QPS: 17.985612
preprocess_time(ms): 2.40, inference_time(ms): 53.20, postprocess_time(ms): 0.00
------------------ Inference Time Info ----------------------
total_time(ms): 23.8, img_num: 1
average latency time(ms): 23.80, QPS: 42.016807
preprocess_time(ms): 1.90, inference_time(ms): 15.10, postprocess_time(ms): 6.8
测试识别
ubuntu@ubuntu:~/PaddleDetection$ python3 deploy/python/keypoint_infer.py --model_dir=/home/ubuntu/PaddleDetection/tinypose_128x96 --image_file=./demo/hrnet_demo.jpg --device=CPU --threshold=0.5
二、先转一下官方的模型，自己训练的就很简单了
ubuntu@ubuntu:~/PaddleDetection$ pip install paddle2onnx -i https://pypi.mirrors.ustc.edu.cn/simple/
然后转模型
ubuntu@ubuntu:~/PaddleDetection$ paddle2onnx --model_dir /home/ubuntu/PaddleDetection/tinypose_128x96/infer_cfg.yml --model_filename /home/ubuntu/PaddleDetection/tinypose_128x96/model.pdmodel --params_filename /home/ubuntu/PaddleDetection/tinypose_128x96/model.pdiparams --opset_version 11 --save_file paddlepose.onnx
[Paddle2ONNX] Start to parse PaddlePaddle model...
[Paddle2ONNX] Model file path: /home/ubuntu/PaddleDetection/tinypose_128x96/model.pdmodel
[Paddle2ONNX] Paramters file path: /home/ubuntu/PaddleDetection/tinypose_128x96/model.pdiparams
[Paddle2ONNX] Start to parsing Paddle model...
[Paddle2ONNX] Use opset_version = 11 for ONNX export.
[Paddle2ONNX] PaddlePaddle model is exported as ONNX format now.
2023-03-07 13:43:40 [INFO] ===============Make PaddlePaddle Better!================
2023-03-07 13:43:40 [INFO] A little survey: https://iwenjuan.baidu.com/?code=r8hu2s
三、先转ncnn模型，发现不支持op存在
ubuntu@ubuntu:~/ncnn/build/install/bin$ python3 -m onnxsim ~/PaddleDetection/paddlepose.onnx ~/PaddleDetection/paddlepose_sim.onnx
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add │ 79 │ 79 │
│ ArgMax │ 1 │ 1 │
│ BatchNormalization │ 268 │ 0 │
│ Concat │ 52 │ 52 │
│ Constant │ 1511 │ 0 │
│ Conv │ 269 │ 269 │
│ Relu │ 144 │ 144 │
│ Reshape │ 99 │ 99 │
│ Resize │ 29 │ 29 │
│ Shape │ 3 │ 3 │
│ Slice │ 3 │ 3 │
│ Split │ 49 │ 49 │
│ Transpose │ 49 │ 49 │
│ Model Size │ 5.4MiB │ 5.1MiB │
└────────────────────┴────────────────┴──────────────────┘
ubuntu@ubuntu:~/ncnn/build/install/bin$ ./onnx2ncnn ~/PaddleDetection/paddlepose.onnx ~/PaddleDetection/paddlepose.param ~/PaddleDetection/paddlepose.bin
Shape not supported yet!
Unknown data type 0
Unsupported Resize scales and sizes are all empty!
Shape not supported yet!
Unknown data type 0
Unsupported Resize scales and sizes are all empty!
Shape not supported yet!
Unknown data type 0
Unsupported Resize scales and sizes are all empty!
ArgMax not supported yet!
# axis=-1
# keepdims=0
四、修正一下固定的输入大小
ubuntu@ubuntu:~$ git clone https://github.com/jiangjiajun/PaddleUtils.git
设置固定输入大小
ubuntu@ubuntu:~/PaddleUtils/paddle$ python3 paddle_infer_shape.py --model_dir /home/ubuntu/PaddleDetection/tinypose_128x96 --model_filename /home/ubuntu/PaddleDetection/tinypose_128x96/model.pdmodel --params_filename /home/ubuntu/PaddleDetection/tinypose_128x96/model.pdiparams --save_dir /home/ubuntu/PaddleDetection/tinypose_128x96/new_model --input_shape_dict="image:[1,3,128,96]"
ubuntu@ubuntu:~/PaddleUtils/paddle$ paddle2onnx --model_dir /home/ubuntu/PaddleDetection/tinypose_128x96/infer_cfg.yml --model_filename /home/ubuntu/PaddleDetection/tinypose_128x96/new_model/model.pdmodel --params_filename /home/ubuntu/PaddleDetection/tinypose_128x96/new_model/model.pdiparams --opset_version 11 --save_file /home/ubuntu/PaddleDetection/tinypose_128x96/new_model/paddlepose.onnx
然后sim一下，就可以在转ncnn试一下
ubuntu@ubuntu:~/PaddleUtils/paddle$ python3 -m onnxsim /home/ubuntu/PaddleDetection/tinypose_128x96/new_model/paddlepose.onnx /home/ubuntu/PaddleDetection/tinypose_128x96/new_model/paddlepose_sim.onnx
Simplifying...
Finish! Here is the difference:
┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ ┃ Original Model ┃ Simplified Model ┃
┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ Add │ 79 │ 79 │
│ ArgMax │ 1 │ 1 │
│ BatchNormalization │ 268 │ 0 │
│ Concat │ 52 │ 49 │
│ Constant │ 1511 │ 0 │
│ Conv │ 269 │ 269 │
│ Relu │ 144 │ 144 │
│ Reshape │ 99 │ 99 │
│ Resize │ 29 │ 29 │
│ Shape │ 3 │ 0 │
│ Slice │ 3 │ 0 │
│ Split │ 49 │ 49 │
│ Transpose │ 49 │ 49 │
│ Model Size │ 5.4MiB │ 5.1MiB │
└────────────────────┴────────────────┴──────────────────┘
问题就很少了，只剩下一个函数不支持，就很容易解决了
ubuntu@ubuntu:~/ncnn/build/install/bin$ ./onnx2ncnn /home/ubuntu/Downloads/tinypose_128x96/a/paddlepose_sim.onnx /home/ubuntu/Downloads/tinypose_128x96/a/example.param /home/ubuntu/Downloads/tinypose_128x96/a/example.bin
ArgMax not supported yet!
# axis=-1
# keepdims=0
提取一下可用的onnx,重新转ncnn
提取有效
import onnx
input_path = "/home/ubuntu/Downloads/tinypose_128x96/a/paddlepose_sim.onnx"
output_path = "/home/ubuntu/Downloads/tinypose_128x96/a/paddlepose_sim_remove_argmax.onnx"
input_names = ["image"]
output_names = ["reshape2_98.tmp_0"]
onnx.utils.extract_model(input_path, output_path, input_names, output_names)
其实我只要conv2d_441/temp_1的数据，他是关键点数据，另一个个输出是heatmap数据，我并不需要～
然后在转ncnn模型
ubuntu@ubuntu:~/ncnn/build/install/bin$ ./onnx2ncnn /home/ubuntu/Downloads/tinypose_128x96/a/paddlepose_sim_remove_argmax.onnx /home/ubuntu/Downloads/tinypose_128x96/a/paddlepose_sim_remove_argmax.param /home/ubuntu/Downloads/tinypose_128x96/a/paddlepose_sim_remove_argmax.bin
五、写一段ncnn的逻辑呗
cmakelists.txt
cmake_minimum_required(VERSION 3.16)
project(prj)
set(CMAKE_C_FLAGS "$CMAKE_C_FLAGS -o3")
set(CMAKE_CXX_FLAGS "$CMAKE_CXX_FLAGS -o3")
set(CMAKE_C_FLAGS "$CMAKE_C_FLAGS -fopenmp ")
set(CMAKE_CXX_FLAGS "$CMAKE_CXX_FLAGS -fopenmp ")
set(CMAKE_CXX_STANDARD 11)
include_directories($CMAKE_SOURCE_DIR/include)
include_directories($CMAKE_SOURCE_DIR/include/ncnn)
find_package(OpenCV REQUIRED)
include_directories($OpenCV_INCLUDE_DIRS)
#导入ncnn
add_library(libncnn STATIC IMPORTED)
set_target_properties(libncnn PROPERTIES IMPORTED_LOCATION $CMAKE_SOURCE_DIR/lib/libncnn.a)
add_executable(prj main.cpp)
target_link_libraries(prj $OpenCV_LIBS libncnn)
代码
#include <iostream>
#include <algorithm>
#include <vector>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/opencv.hpp>
#include "platform.h"
#include "net.h"
#include "omp.h"
#include "chrono"
#if NCNN_VULKAN
#include "gpu.h"
#endif // NCNN_VULKAN
using namespace std;
using namespace chrono;
struct Keypoints
float x;
float y;
float score;
Keypoints() : x(0), y(0), score(0)
Keypoints(float x, float y, float score) : x(x), y(y), score(score)
;
void rotate_point(float *pt, float angle_rad, float *rotated_pt)
float sn = sin(angle_rad);
float cs = cos(angle_rad);
float new_x = pt[0] * cs - pt[1] * sn;
float new_y = pt[0] * sn + pt[1] * cs;
rotated_pt[0] = new_x;
rotated_pt[1] = new_y;
void _get_3rd_point(cv::Point2f a, cv::Point2f b, float *direction)
float direction_0 = a.x - b.x;
float direction_1 = a.y - b.y;
direction[0] = b.x - direction_1;
direction[1] = b.y + direction_0;
void get_affine_transform(float *center, float *scale, float rot, float *output_size, float *shift, bool inv,
cv::Mat &trans)
float scale_tmp[] = 0, 0;
//scale_tmp[0] = scale[0] * 200.0;
//scale_tmp[1] = scale[1] * 200.0;
float src_w = scale[0];
float dst_w = output_size[0];
float dst_h = output_size[1];
float rot_rad = M_PI * rot / 180;
float pt[] = 0, 0;
pt[0] = 0;
pt[1] = src_w * (-0.5);
float src_dir[] = 0, 0;
rotate_point(pt, rot_rad, src_dir);
float dst_dir[] = 0, 0;
dst_dir[0] = 0;
dst_dir[1] = dst_w * (-0.5);
cv::Point2f src[3] = cv::Point2f(0, 0), cv::Point2f(0, 0), cv::Point2f(0, 0);
src[0] = cv::Point2f(center[0] + scale_tmp[0] * shift[0], center[1] + scale_tmp[1] * shift[1]);
src[1] = cv::Point2f(center[0] + src_dir[0] + scale_tmp[0] * shift[0],
center[1] + src_dir[1] + scale_tmp[1] * shift[1]);
float direction_src[] = 0, 0;
_get_3rd_point(src[0], src[1], direction_src);
src[2] = cv::Point2f(direction_src[0], direction_src[1]);
cv::Point2f dst[3] = cv::Point2f(0, 0), cv::Point2f(0, 0), cv::Point2f(0, 0);
dst[0] = cv::Point2f(dst_w * 0.5, dst_h * 0.5);
dst[1] = cv::Point2f(dst_w * 0.5 + dst_dir[0], dst_h * 0.5 + dst_dir[1]);
float direction_dst[] = 0, 0;
_get_3rd_point(dst[0], dst[1], direction_dst);
dst[2] = cv::Point2f(direction_dst[0], direction_dst[1]);
if (inv)
trans = cv::getAffineTransform(dst, src);
else
trans = cv::getAffineTransform(src, dst);
void pretty_print(const ncnn::Mat &m, std::vector<float> &vec_heap)
for (int q = 0; q < m.c; q++)
const float *ptr = m.channel(q);
for (int z = 0; z < m.d; z++)
for (int y = 0; y < m.h; y++)
for (int x = 0; x < m.w; x++)
// printf("%f ", ptr[x]);
vec_heap.emplace_back(ptr[x]);
ptr += m.w;
//printf("\\n");
//printf("\\n");
void
transform_preds(std::vector<cv::Point2f> coords, std::vector<Keypoints> &target_coords, float *center, float *scale,
int w, int h)
float temp_scale[] = scale[0] * 200, scale[1] * 200;
float rot=0;
bool inv=true;
float shift[2]=0;
cv::Mat trans;
float heap_wh[2]=(float)w,(float)h;
get_affine_transform(center, temp_scale, rot, heap_wh, shift, inv, trans);
//std::cout<<trans<<std::endl;
double mat_00=trans.at<double>(0,0);
double mat_01=trans.at<double>(0,1);
double mat_02=trans.at<double>(0,2);
double mat_10=trans.at<double>(1,0);
double mat_11=trans.at<double>(1,1);
double mat_12=trans.at<double>(1,2);
for (int i = 0; i < coords.size(); i++)
target_coords[i].x = coords[i].x * mat_00 + coords[i].y *mat_01+1*mat_02;
target_coords[i].y = coords[i].x * mat_10 + coords[i].y *mat_11+1*mat_12;
int main(int argc, char **argv)
float keypoint_score=0.3f;
cv::Mat bgr = cv::imread("/home/ubuntu/PaddleDetection/demo/hrnet_demo.jpg");
//cv::Mat rgb;
//cv::cvtColor(bgr, rgb, cv::COLOR_BGR2RGB);
float center_x=bgr.cols/2.0;
float center_y=bgr.rows/2.0;
float scale_x=bgr.cols;
float scale_y=bgr.rows;
float image_target_w = 128;
float image_target_h = 96;
float center[2] = 0, 0;
float scale[2] = 0, 0;
center[0]=center_x;
center[1]=center_y;
scale[0]=scale_x;
scale[1]=scale_y;
float rot = 0;
float shift[] = 0, 0;
bool inv = false;
float output_size[] = image_target_h, image_target_w;
cv::Mat trans;
get_affine_transform(center, scale, rot, output_size, shift, inv, trans);
//std::cout << trans << std::endl;
cv::Mat detect_image;//= cv::Mat::zeros(image_target_w ,image_target_h, CV_8UC3);
cv::warpAffine(bgr, detect_image, trans, cv::Size(image_target_h, image_target_w), cv::INTER_LINEAR);
//std::cout << detect_image.cols << " " << detect_image.rows << std::endl;
//std::cout << detect_image<<std::endl;
ncnn::Net harnet;
harnet.load_param("/home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/paddlepose_sim_remove_argmax.param");
harnet.load_model("/home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/paddlepose_sim_remove_argmax.bin");
ncnn::Mat in = ncnn::Mat::from_pixels(detect_image.data, ncnn::Mat::PIXEL_BGR2RGB, detect_image.cols,
detect_image.rows);
// transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
const float mean_vals[3] = 0.485f * 255.f, 0.456f * 255.f, 0.406f * 255.f;
const float norm_255[3] = (1 / 0.229f / 255.f), (1 / 0.224f / 255.f), (1 / 0.225f / 255.f);
in.substract_mean_normalize(mean_vals, norm_255);
fprintf(stderr, "input shape: %d %d %d %d\\n", in.d, in.h, in.w, in.c);
auto start = chrono::high_resolution_clock::now(); //开始时间
ncnn::Extractor ex = harnet.create_extractor();
ex.input("image", in);//input 是 .param文件中输入节点名称
ncnn::Mat result;
ex.extract("conv2d_441.tmp_1", result);
//pretty_print(result);
auto end = chrono::high_resolution_clock::now(); //结束时间
auto duration = (end - start).count();
cout << "程序运行时间：" << setprecision(10) << duration / 1000000000.0 << "s"
<< "； " << duration / 1000000.0 << "ms"
<< "； " << duration / 1000.0 << "us"
<< endl;
fprintf(stderr, "output shape: %d %d %d %d\\n", result.d, result.c, result.h, result.w);
int shape_d = result.d;
int shape_c = result.c;
int shape_w = result.w;
int shape_h = result.h;
std::vector<float> vec_heap;
pretty_print(result, vec_heap);
scale[0]=center[0]/100;
scale[1]=center[1]/100;
std::vector<Keypoints> all_preds;
std::vector<int> idx;
for (int i = 0; i < shape_c; i++)
auto begin = vec_heap.begin() + i * shape_w * shape_h;
auto end = vec_heap.begin() + (i + 1) * shape_w * shape_h;
float maxValue = *max_element(begin, end);
int maxPosition = max_element(begin, end) - begin;
all_preds.emplace_back(Keypoints(0, 0, maxValue));
idx.emplace_back(maxPosition);
std::vector<cv::Point2f> vec_point;
for (int i = 0; i < idx.size(); i++)
int x = idx[i] % shape_w;
int y = idx[i] / shape_w;
vec_point.emplace_back(cv::Point2f(x, y));
for (int i = 0; i < shape_c; i++)
int px = floor(vec_point[i].x+0.5);
int py = floor(vec_point[i].y+0.5);
if (px > 1 && px < shape_w - 1 && py > 1 && py < shape_h - 1)
float diff_0 = vec_heap[py * shape_w + px + 1] - vec_heap[py * shape_w + px - 1];
float diff_1 = vec_heap[(py + 1) * shape_w + px] - vec_heap[(py - 1) * shape_w + px];
vec_point[i].x += diff_0 == 0 ? 0 : (diff_0 > 0) ? 0.25 : -0.25;
vec_point[i].y += diff_1 == 0 ? 0 : (diff_1 > 0) ? 0.25 : -0.25;
center[0]=ceil(center[0]);
center[1]=ceil(center[1]);
transform_preds(vec_point, all_preds, center, scale, shape_w, shape_h);
int skeleton[][2] = 0, 1, 0, 2, 1, 3, 2, 4, 3, 5, 4, 6, 5, 7, 6, 8,
7, 9, 8, 10, 5, 11, 6, 12, 11, 13, 12, 14,
13, 15, 14, 16, 11, 12;
for (int i = 0; i < all_preds.size(); i++)
if(all_preds[i].score>keypoint_score)
cv::circle(bgr, cv::Point(all_preds[i].x, all_preds[i].y), 3, cv::Scalar(0, 255, 120), -1);//画点，其实就是实心圆
for (int i = 0; i < sizeof(skeleton) / sizeof(sizeof(skeleton[1])); i++)
int x0 = all_preds[skeleton[i][0]].x;
int y0 = all_preds[skeleton[i][0]].y;
int x1 = all_preds[skeleton[i][1]].x;
int y1 = all_preds[skeleton[i][1]].y;
cv::line(bgr, cv::Point(x0, y0), cv::Point(x1, y1),
cv::Scalar(0, 255, 0), 1);
cv::imwrite("demo.jpg",bgr);
cv::imshow("image", bgr);
cv::waitKey(0);
return 0;
测试结果
/home/ubuntu/CLionProjects/untitled3/cmake-build-debug/prj
input shape: 1 128 96 3
output shape: 1 17 32 24
程序运行时间：0.005697311s； 5.697311ms； 5697.311us
图片
六、mnn代码
ubuntu@ubuntu:~/MNN/build$ ./MNNConvert -f ONNX --modelFile /home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/paddlepose_sim.onnx --MNNModel /home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/paddlepose_sim.mnn --bizCode MNN
The device support dot:0, support fp16:0, support i8mm: 0
Start to Convert Other Model Format To MNN Model...
[17:16:06] /home/ubuntu/MNN/tools/converter/source/onnx/onnxConverter.cpp:40: ONNX Model ir version: 8
Start to Optimize the MNN Net...
inputTensors : [ image, ]
outputTensors: [ argmax_0.tmp_0, conv2d_441.tmp_1, ]
Converted Success!
cmkelists.txt
cmake_minimum_required(VERSION 3.16)
project(untitled22)
set(CMAKE_CXX_FLAGS "-std=c++11")
set(CMAKE_C_FLAGS "$CMAKE_C_FLAGS -fopenmp ")
set(CMAKE_CXX_FLAGS "$CMAKE_CXX_FLAGS -fopenmp")
set(CMAKE_CXX_STANDARD 11)
include_directories($CMAKE_SOURCE_DIR)
include_directories($CMAKE_SOURCE_DIR/include)
include_directories($CMAKE_SOURCE_DIR/include/MNN)
find_package(OpenCV REQUIRED)
#message(STATUS $OpenCV_INCLUDE_DIRS)
#添加头文件
include_directories($OpenCV_INCLUDE_DIRS)
#链接Opencv库
add_library(libmnn SHARED IMPORTED)
set_target_properties(libmnn PROPERTIES IMPORTED_LOCATION $CMAKE_SOURCE_DIR/lib/libMNN.so)
add_executable(untitled22 main.cpp)
target_link_libraries(untitled22 $OpenCV_LIBS libmnn )
测试代码
#include <iostream>
#include <algorithm>
#include <vector>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/opencv.hpp>
#include<MNN/Interpreter.hpp>
#include<MNN/ImageProcess.hpp>
using namespace std;
using namespace chrono;
struct Keypoints
float x;
float y;
float score;
Keypoints() : x(0), y(0), score(0)
Keypoints(float x, float y, float score) : x(x), y(y), score(score)
;
struct Box
float center_x;
float center_y;
float scale_x;
float scale_y;
float scale_prob;
float score;
Box() : center_x(0), center_y(0), scale_x(0), scale_y(0), scale_prob(0), score(0)
Box(float center_x, float center_y, float scale_x, float scale_y, float scale_prob, float score) :
center_x(center_x), center_y(center_y), scale_x(scale_x), scale_y(scale_y), scale_prob(scale_prob),
score(score)
;
void rotate_point(float *pt, float angle_rad, float *rotated_pt)
float sn = sin(angle_rad);
float cs = cos(angle_rad);
float new_x = pt[0] * cs - pt[1] * sn;
float new_y = pt[0] * sn + pt[1] * cs;
rotated_pt[0] = new_x;
rotated_pt[1] = new_y;
void _get_3rd_point(cv::Point2f a, cv::Point2f b, float *direction)
float direction_0 = a.x - b.x;
float direction_1 = a.y - b.y;
direction[0] = b.x - direction_1;
direction[1] = b.y + direction_0;
void get_affine_transform(float *center, float *scale, float rot, float *output_size, float *shift, bool inv,
cv::Mat &trans)
float scale_tmp[] = 0, 0;
//scale_tmp[0] = scale[0] * 200.0;
//scale_tmp[1] = scale[1] * 200.0;
float src_w = scale[0];
float dst_w = output_size[0];
float dst_h = output_size[1];
float rot_rad = M_PI * rot / 180;
float pt[] = 0, 0;
pt[0] = 0;
pt[1] = src_w * (-0.5);
float src_dir[] = 0, 0;
rotate_point(pt, rot_rad, src_dir);
float dst_dir[] = 0, 0;
dst_dir[0] = 0;
dst_dir[1] = dst_w * (-0.5);
cv::Point2f src[3] = cv::Point2f(0, 0), cv::Point2f(0, 0), cv::Point2f(0, 0);
src[0] = cv::Point2f(center[0] + scale_tmp[0] * shift[0], center[1] + scale_tmp[1] * shift[1]);
src[1] = cv::Point2f(center[0] + src_dir[0] + scale_tmp[0] * shift[0],
center[1] + src_dir[1] + scale_tmp[1] * shift[1]);
float direction_src[] = 0, 0;
_get_3rd_point(src[0], src[1], direction_src);
src[2] = cv::Point2f(direction_src[0], direction_src[1]);
cv::Point2f dst[3] = cv::Point2f(0, 0), cv::Point2f(0, 0), cv::Point2f(0, 0);
dst[0] = cv::Point2f(dst_w * 0.5, dst_h * 0.5);
dst[1] = cv::Point2f(dst_w * 0.5 + dst_dir[0], dst_h * 0.5 + dst_dir[1]);
float direction_dst[] = 0, 0;
_get_3rd_point(dst[0], dst[1], direction_dst);
dst[2] = cv::Point2f(direction_dst[0], direction_dst[1]);
if (inv)
trans = cv::getAffineTransform(dst, src);
else
trans = cv::getAffineTransform(src, dst);
void
transform_preds(std::vector<cv::Point2f> coords, std::vector<Keypoints> &target_coords, float *center, float *scale,
int w, int h)
float temp_scale[] = scale[0] * 200, scale[1] * 200;
float rot=0;
bool inv=true;
float shift[2]=0;
cv::Mat trans;
float heap_wh[2]=(float)w,(float)h;
get_affine_transform(center, temp_scale, rot, heap_wh, shift, inv, trans);
//std::cout<<trans<<std::endl;
double mat_00=trans.at<double>(0,0);
double mat_01=trans.at<double>(0,1);
double mat_02=trans.at<double>(0,2);
double mat_10=trans.at<double>(1,0);
double mat_11=trans.at<double>(1,1);
double mat_12=trans.at<double>(1,2);
for (int i = 0; i < coords.size(); i++)
target_coords[i].x = coords[i].x * mat_00 + coords[i].y *mat_01+1*mat_02;
target_coords[i].y = coords[i].x * mat_10 + coords[i].y *mat_11+1*mat_12;
int main(int argc, char **argv)
float keypoint_score=0.3f;
cv::Mat bgr = cv::imread("/home/ubuntu/PaddleDetection/demo/hrnet_demo.jpg");
//cv::Mat rgb;
//cv::cvtColor(bgr, rgb, cv::COLOR_BGR2RGB);
float center_x=bgr.cols/2.0;
float center_y=bgr.rows/2.0;
float scale_x=bgr.cols;
float scale_y=bgr.rows;
float image_target_w = 128;
float image_target_h = 96;
float center[2] = 0, 0;
float scale[2] = 0, 0;
center[0]=center_x;
center[1]=center_y;
scale[0]=scale_x;
scale[1]=scale_y;
float rot = 0;
float shift[] = 0, 0;
bool inv = false;
float output_size[] = image_target_h, image_target_w;
cv::Mat trans;
get_affine_transform(center, scale, rot, output_size, shift, inv, trans);
std::cout << "trans= "<<trans << std::endl;
cv::Mat detect_image;//= cv::Mat::zeros(image_target_w ,image_target_h, CV_8UC3);
cv::warpAffine(bgr, detect_image, trans, cv::Size(image_target_h, image_target_w), cv::INTER_LINEAR);
//std::cout << detect_image.cols << " " << detect_image.rows << std::endl;
//std::cout << detect_image<<std::endl;
auto start = chrono::high_resolution_clock::now(); //开始时间
// MNN inference
auto mnnNet = std::shared_ptr<MNN::Interpreter>(MNN::Interpreter::createFromFile("/home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/paddlepose_sim.mnn"));
MNN::ScheduleConfig netConfig;
netConfig.type = MNN_FORWARD_CPU;
netConfig.numThread = 4;
auto session = mnnNet->createSession(netConfig);
auto input = mnnNet->getSessionInput(session, nullptr);
mnnNet->resizeTensor(input, 1, 3, (int) image_target_w, (int) image_target_h);
mnnNet->resizeSession(session);
MNN::CV::ImageProcess::Config config;
const float mean_vals[3] = 0.485f * 255.f, 0.456f * 255.f, 0.406f * 255.f;
const float norm_255[3] = (1 / 0.229f / 255.f), (1 / 0.224f / 255.f), (1 / 0.225f / 255.f);
std::shared_ptr<MNN::CV::ImageProcess> pretreat(
MNN::CV::ImageProcess::create(MNN::CV::BGR, MNN::CV::RGB, mean_vals, 3,
norm_255, 3));
pretreat->convert(detect_image.data, (int) image_target_h, (int) image_target_w, detect_image.step[0], input);
mnnNet->runSession(session);
auto heatmap = mnnNet->getSessionOutput(session, "conv2d_441.tmp_1");
MNN::Tensor heatmapHost(heatmap, heatmap->getDimensionType());
heatmap->copyToHostTensor(&heatmapHost);
std::vector<float> vec_heap;
for (int i = 0; i < heatmapHost.elementSize(); i++)
//std::cout << heatmapHost.host<float>()[i] << " ";
vec_heap.emplace_back(heatmapHost.host<float>()[i]);
auto end = chrono::high_resolution_clock::now(); //结束时间
auto duration = (end - start).count();
cout << "程序运行时间：" << setprecision(10) << duration / 1000000000.0 << "s"
<< "； " << duration / 1000000.0 << "ms"
<< "； " << duration / 1000.0 << "us"
<< endl;
int shape_c = heatmapHost.channel();
int shape_w = heatmapHost.width();
int shape_h = heatmapHost.height();
fprintf(stderr, "output shape: %d %d %d %d\\n", heatmapHost.dimensions(), shape_c, shape_h, shape_w);
scale[0]=center[0]/100;
scale[1]=center[1]/100;
std::vector<Keypoints> all_preds;
std::vector<int> idx;
for (int i = 0; i < shape_c; i++)
auto begin = vec_heap.begin() + i * shape_w * shape_h;
auto end = vec_heap.begin() + (i + 1) * shape_w * shape_h;
float maxValue = *max_element(begin, end);
int maxPosition = max_element(begin, end) - begin;
all_preds.emplace_back(Keypoints(0, 0, maxValue));
idx.emplace_back(maxPosition);
std::vector<cv::Point2f> vec_point;
for (int i = 0; i < idx.size(); i++)
int x = idx[i] % shape_w;
int y = idx[i] / shape_w;
vec_point.emplace_back(cv::Point2f(x, y));
for (int i = 0; i < shape_c; i++)
int px = floor(vec_point[i].x+0.5);
int py = floor(vec_point[i].y+0.5);
if (px > 1 && px < shape_w - 1 && py > 1 && py < shape_h - 1)
float diff_0 = vec_heap[py * shape_w + px + 1] - vec_heap[py * shape_w + px - 1];
float diff_1 = vec_heap[(py + 1) * shape_w + px] - vec_heap[(py - 1) * shape_w + px];
vec_point[i].x += diff_0 == 0 ? 0 : (diff_0 > 0) ? 0.25 : -0.25;
vec_point[i].y += diff_1 == 0 ? 0 : (diff_1 > 0) ? 0.25 : -0.25;
center[0]=ceil(center[0]);
center[1]=ceil(center[1]);
transform_preds(vec_point, all_preds, center, scale, shape_w, shape_h);
int skeleton[][2] = 0, 1, 0, 2, 1, 3, 2, 4, 3, 5, 4, 6, 5, 7, 6, 8,
7, 9, 8, 10, 5, 11, 6, 12, 11, 13, 12, 14,
13, 15, 14, 16, 11, 12;
for (int i = 0; i < all_preds.size(); i++)
if(all_preds[i].score>keypoint_score)
cv::circle(bgr, cv::Point(all_preds[i].x, all_preds[i].y), 3, cv::Scalar(0, 255, 120), -1);//画点，其实就是实心圆
for (int i = 0; i < sizeof(skeleton) / sizeof(sizeof(skeleton[1])); i++)
int x0 = all_preds[skeleton[i][0]].x;
int y0 = all_preds[skeleton[i][0]].y;
int x1 = all_preds[skeleton[i][1]].x;
int y1 = all_preds[skeleton[i][1]].y;
cv::line(bgr, cv::Point(x0, y0), cv::Point(x1, y1),
cv::Scalar(0, 255, 0), 1);
cv::imwrite("demo.jpg",bgr);
cv::imshow("image", bgr);
cv::waitKey(0);
mnnNet->releaseModel();
mnnNet->releaseSession(session);
return 0;
结果
七、openvino模型测试结果
首先转openvnino模型
ubuntu@ubuntu:~$ python3 /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo.py --input_model /home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/paddlepose_sim.onnx --output_dir /home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/FP16 --input_shape [1,3,96,128] --data_type FP16
或者
ubuntu@ubuntu:~$ python3 /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo.py --input_model /home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/paddlepose_sim.onnx --output_dir /home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/FP16 --input_shape [1,3,96,128] --mean_values=[123.675,116.28,103.53] --scale_values=[58.395,57.12,57.375] --data_type FP16 --reverse_input_channels
好像结果有点问题，蛀牙处在interpolate上，修正中，但是偶尔会存在检测正确的，结果很玄幻
cmake_minimum_required(VERSION 3.16.1)
set(CMAKE_CXX_STANDARD 14)
project(untitled6)
find_package(OpenCV REQUIRED)
find_package(ngraph REQUIRED)
find_package(InferenceEngine REQUIRED)
include_directories(
$OpenCV_INCLUDE_DIRS
$CMAKE_CURRENT_SOURCE_DIR
$CMAKE_CURRENT_BINARY_DIR
)
add_executable(untitled6 main.cpp)
target_link_libraries(
untitled6
$InferenceEngine_LIBRARIES
$NGRAPH_LIBRARIES
$OpenCV_LIBS
)
测试代码
#include <opencv2/opencv.hpp>
#include <inference_engine.hpp>
#include <iostream>
#include <chrono>
#include <opencv2/dnn/dnn.hpp>
#include <cmath>
using namespace std;
using namespace cv;
using namespace InferenceEngine;
using namespace std;
using namespace chrono;
struct Keypoints
float x;
float y;
float score;
Keypoints() : x(0), y(0), score(0)
Keypoints(float x, float y, float score) : x(x), y(y), score(score)
;
struct Box
float center_x;
float center_y;
float scale_x;
float scale_y;
float scale_prob;
float score;
Box() : center_x(0), center_y(0), scale_x(0), scale_y(0), scale_prob(0), score(0)
Box(float center_x, float center_y, float scale_x, float scale_y, float scale_prob, float score) :
center_x(center_x), center_y(center_y), scale_x(scale_x), scale_y(scale_y), scale_prob(scale_prob),
score(score)
;
void rotate_point(float *pt, float angle_rad, float *rotated_pt)
float sn = sin(angle_rad);
float cs = cos(angle_rad);
float new_x = pt[0] * cs - pt[1] * sn;
float new_y = pt[0] * sn + pt[1] * cs;
rotated_pt[0] = new_x;
rotated_pt[1] = new_y;
void _get_3rd_point(cv::Point2f a, cv::Point2f b, float *direction)
float direction_0 = a.x - b.x;
float direction_1 = a.y - b.y;
direction[0] = b.x - direction_1;
direction[1] = b.y + direction_0;
void get_affine_transform(float *center, float *scale, float rot, float *output_size, float *shift, bool inv,
cv::Mat &trans)
float scale_tmp[] = 0, 0;
//scale_tmp[0] = scale[0] * 200.0;
//scale_tmp[1] = scale[1] * 200.0;
float src_w = scale[0];
float dst_w = output_size[0];
float dst_h = output_size[1];
float rot_rad = M_PI * rot / 180;
float pt[] = 0, 0;
pt[0] = 0;
pt[1] = src_w * (-0.5);
float src_dir[] = 0, 0;
rotate_point(pt, rot_rad, src_dir);
float dst_dir[] = 0, 0;
dst_dir[0] = 0;
dst_dir[1] = dst_w * (-0.5);
cv::Point2f src[3] = cv::Point2f(0, 0), cv::Point2f(0, 0), cv::Point2f(0, 0);
src[0] = cv::Point2f(center[0] + scale_tmp[0] * shift[0], center[1] + scale_tmp[1] * shift[1]);
src[1] = cv::Point2f(center[0] + src_dir[0] + scale_tmp[0] * shift[0],
center[1] + src_dir[1] + scale_tmp[1] * shift[1]);
float direction_src[] = 0, 0;
_get_3rd_point(src[0], src[1], direction_src);
src[2] = cv::Point2f(direction_src[0], direction_src[1]);
cv::Point2f dst[3] = cv::Point2f(0, 0), cv::Point2f(0, 0), cv::Point2f(0, 0);
dst[0] = cv::Point2f(dst_w * 0.5, dst_h * 0.5);
dst[1] = cv::Point2f(dst_w * 0.5 + dst_dir[0], dst_h * 0.5 + dst_dir[1]);
float direction_dst[] = 0, 0;
_get_3rd_point(dst[0], dst[1], direction_dst);
dst[2] = cv::Point2f(direction_dst[0], direction_dst[1]);
if (inv)
trans = cv::getAffineTransform(dst, src);
else
trans = cv::getAffineTransform(src, dst);
void
transform_preds(std::vector<cv::Point2f> coords, std::vector<Keypoints> &target_coords, float *center, float *scale,
int w, int h)
float temp_scale[] = scale[0] * 200, scale[1] * 200;
float rot = 0;
bool inv = true;
float shift[2] = 0;
cv::Mat trans;
float heap_wh[2] = (float) w, (float) h;
get_affine_transform(center, temp_scale, rot, heap_wh, shift, inv, trans);
//std::cout<<trans<<std::endl;
double mat_00 = trans.at<double>(0, 0);
double mat_01 = trans.at<double>(0, 1);
double mat_02 = trans.at<double>(0, 2);
double mat_10 = trans.at<double>(1, 0);
double mat_11 = trans.at<double>(1, 1);
double mat_12 = trans.at<double>(1, 2);
for (int i = 0; i < coords.size(); i++)
target_coords[i].x = coords[i].x * mat_00 + coords[i].y * mat_01 + 1 * mat_02;
target_coords[i].y = coords[i].x * mat_10 + coords[i].y * mat_11 + 1 * mat_12;
int main(int argc, char const *argv[])
std::vector<float> meanVals = -0.485f * 255.f * (1.0f / 0.229f) * (1.0 / 255.f),
-0.456f * 255.f * (1.0f / 0.224f) * (1.0 / 255.f),
-0.406f * 255.f * (1.0f / 0.225f) * (1.0 / 255.f);
std::vector<float> normVals = (1.0f / 0.229f) * (1.0 / 255.f),
(1.0f / 0.224f) * (1.0 / 255.f),
(1.0f / 0.225f) * (1.0 / 255.f);
float keypoint_score = 0.3f;
cv::Mat bgr = cv::imread("/home/ubuntu/PaddleDetection/demo/hrnet_demo.jpg");
cv::Mat rgb;
cv::cvtColor(bgr, rgb, cv::COLOR_BGR2RGB);
float image_target_w = 128;
float image_target_h = 96;
float center_x = bgr.cols / 2.0;
float center_y = bgr.rows / 2.0;
float scale_x = bgr.cols;
float scale_y = bgr.rows;
float center[2] = 0, 0;
float scale[2] = 0, 0;
center[0] = center_x;
center[1] = center_y;
scale[0] = scale_x;
scale[1] = scale_y;
float rot = 0;
float shift[] = 0, 0;
bool inv = false;
float output_size[] = image_target_h, image_target_w;
cv::Mat trans;
get_affine_transform(center, scale, rot, output_size, shift, inv, trans);
std::cout << "trans= " << trans << std::endl;
cv::Mat detect_image;//= cv::Mat::zeros(image_target_w ,image_target_h, CV_8UC3);
cv::Mat resize_img;
resize(rgb, resize_img, Size(image_target_h, image_target_w));
cv::warpAffine(resize_img, detect_image, trans, cv::Size(image_target_h, image_target_w), cv::INTER_LINEAR);
// std::vector<cv::Mat> rgb_channel(3);
// cv::split(detect_image, rgb_channel);
// for (int i = 0; i < rgb_channel.size(); i++)
// rgb_channel[i].convertTo(rgb_channel[i], CV_32FC1, normVals[i], meanVals[i]);
//
// for (int j = 0; j < rgb_channel[i].rows; j++)
// for (int k = 0; k < rgb_channel[i].cols; k++)
// std::cout << rgb_channel[i].at<float>(j, k) << " ";;
//
// std::cout << std::endl;
//
//
// cv::Mat image_dst;
//
// // norm_image.convertTo(image_dst, CV_32FC3, 1);
// // std::cout << "original image_dst type " << image_dst.depth() << std::endl; //CV_32FC3
// //没必要处理 ，因为rgb_channel 是CV_32FC1
// cv::merge(rgb_channel, image_dst);
// // std::cout << "process type " << image_dst.depth() << std::endl; //CV_32FC3
// // std::cout << "last convertTo bgr vaule" << std::endl;
for (int i = 0; i < image_dst.channels(); i++)
for (int j = 0; j < image_dst.rows; j++)
for (int k = 0; k < image_dst.cols; k++)
std::cout << (float) image_dst.at<cv::Vec3f>(j, k)[i] << " ";;
std::cout << std::endl;
std::cout << detect_image.cols << " " << " " << detect_image.rows << " " << std::endl;
auto start = chrono::high_resolution_clock::now();
ExecutableNetwork _network;
OutputsDataMap _outputinfo;
//参数区
Core ie;
auto cnnNetwork = ie.ReadNetwork(
"/home/ubuntu/CLionProjects/untitled3/tinypose_128x96/model/FP16/paddlepose_sim.xml");
//输入设置
InputsDataMap inputInfo(cnnNetwork.getInputsInfo());
InputInfo::Ptr &input = inputInfo.begin()->second;
string _input_name = inputInfo.begin()->first;
input->setPrecision(Precision::FP32);
input->getInputData()->setLayout(Layout::NCHW);
ICNNNetwork::InputShapes inputShapes = cnnNetwork.getInputShapes();
SizeVector &inSizeVector = inputShapes.begin()->second;
cnnNetwork.reshape(inputShapes);
//输出设置
_outputinfo = OutputsDataMap(cnnNetwork.getOutputsInfo());
for (auto &output : _outputinfo)
output.second->setPrecision(Precision::FP32);
//获取可执行网络
//_network = ie.LoadNetwork(cnnNetwork, "GPU");
_network = ie.LoadNetwork(cnnNetwork, "CPU");
size_t img_size = image_target_h * image_target_w;
InferRequest::Ptr infer_request = _network.CreateInferRequestPtr();
Blob::Ptr frameBlob = infer_request->GetBlob(_input_name);
InferenceEngine::LockedMemory<void> blobMapped = InferenceEngine::as<InferenceEngine::MemoryBlob>(
frameBlob)->wmap();
float *blob_data = blobMapped.as<float *>();
//nchw
std::vector<cv::Mat> rgb_channel(3);
cv::split(detect_image, rgb_channel);
for (int i = 0; i < rgb_channel.size(); i++)
rgb_channel[i].convertTo(rgb_channel[i], CV_32FC1, normVals[i], meanVals[i]);
for (int j = 0; j < rgb_channel[i].rows; j++)
for (int k = 0; k < rgb_channel[i].cols; k++)
//std::cout << rgb_channel[i].at<float>(j, k) << " ";;
blob_data[img_size * i + j * int(image_target_w) + k] =rgb_channel[i].at<float>(j, k);
//std::cout << std::endl;
// for (size_t row = 0; row < image_target_h; row++)
// for (size_t col = 0; col < image_target_w; col++)
// for (size_t ch = 0; ch < 3; ch++)
// blob_data[img_size * ch + row * int(image_target_w) + col] =
// float(detect_image.at<Vec3f>(row, col)[ch]);
// // std::cout<<blob_data[img_size * ch + row * int(image_target_w) + col]<<" ";
//
// //std::cout<<std::endl;
//
//
//执行预测
infer_request->Infer();
//获取各层结果
std::vector<float> vec_heap;
int shape_d = 0;//result.d;
int shape_c = 0;//result.c;
int shape_w = 0;//result.w;
int shape_h = 0;//result.h;
for (auto &output : _outputinfo)
auto output_name = output.first;
std::cout << output_name << std::endl;
if (strcmp(output_name.c_str(), "conv2d_441.tmp_1") == 0)
shape_d = output.second.get()->getDims()[0];//result.d;
shape_c = output.second.get()->getDims()[1];//result.c;
shape_w = output.second.get()->getDims()[2];//result.w;
shape_h = output.second.get()->getDims()[3];//result.h;
printf("output shape: %d %d %d %d\\n", shape_d, shape_c, shape_w, shape_h);
Blob::Ptr blob = infer_request->GetBlob("conv2d_441.tmp_1");
LockedMemory<const void> blobMappedResult = as<MemoryBlob>(blob)->rmap();
const float *output_blob = blobMappedResult.as<float *>();
for (int i = 0; i < shape_c ; i++)
for(int j=0;j<shape_h*shape_w;j++)
vec_heap.push_back(output_blob[i*shape_h*shape_w+j]);
//std::cout<<output_blob[i*shape_h*shape_w+j]<<" ";
//std::cout<<std::endl;
auto end = chrono::high_resolution_clock::now(); //结束时间
auto duration = (end - start).count();
cout << "程序运行时间：" << setprecision(10) << duration / 1000000000.0 << "s"
<< "； " << duration / 1000000.0 << "ms"
<< "； " << duration / 1000.0 << "us"
<< endl;
scale[0]=center[0]/100;
scale[1]=center[1]/100;
std::vector<Keypoints> all_preds;
std::vector<int> idx;
for (int i = 0; i < shape_c; i++)
auto begin = vec_heap.begin() + i * shape_w * shape_h;
auto end = vec_heap.begin() + (i + 1) * shape_w * shape_h;
float maxValue = *max_element(begin, end);
int maxPosition = max_element(begin, end) - begin;
all_preds.emplace_back(Keypoints(0, 0, maxValue));
idx.emplace_back(maxPosition);
std::vector<cv::Point2f> vec_point;
for (int i = 0; i < idx.size(); i++)
int x = idx[i] % shape_w;
int y = idx[i] / shape_w;
vec_point.emplace_back(cv::Point2f(x, y));
for (int i = 0; i < shape_c; i++)
int px = floor(vec_point[i].x+0.5);
int py = floor(vec_point[i].y+0.5);
if (px > 1 && px < shape_w - 1 && py > 1 && py < shape_h - 1)
float diff_0 = vec_heap[py * shape_w + px + 1] - vec_heap[py * shape_w + px - 1];
float diff_1 = vec_heap[(py + 1) * shape_w + px] - vec_heap[(py - 1) * shape_w + px];
vec_point[i].x += diff_0 == 0 ? 0 : (diff_0 > 0) ? 0.25 : -0.25;
vec_point[i].y += diff_1 == 0 ? 0 : (diff_1 > 0) ? 0.25 : -0.25;
center[0]=ceil(center[0]);
center[1]=ceil(center[1]);
transform_preds(vec_point, all_preds, center, scale, shape_w, shape_h);
int skeleton[][2] = 0, 1, 0, 2, 1, 3, 2, 4, 3, 5, 4, 6, 5, 7, 6, 8,
7, 9, 8, 10, 5, 11, 6, 12, 11, 13, 12, 14,
13, 15, 14, 16, 11, 12;
for (int i = 0; i < all_preds.size(); i++)
if(all_preds[i].score>keypoint_score)
cv::circle(bgr, cv::Point(all_preds[i].x, all_preds[i].y), 3, cv::Scalar(0, 255, 120), -1);//画点，其实就是实心圆
for (int i = 0; i < sizeof(skeleton) / sizeof(sizeof(skeleton[1])); i++)
int x0 = all_preds[skeleton[i][0]].x;
int y0 = all_preds[skeleton[i][0]].y;
int x1 = all_preds[skeleton[i][1]].x;
int y1 = all_preds[skeleton[i][1]].y;
cv::line(bgr, cv::Point(x0, y0), cv::Point(x1, y1),
cv::Scalar(0, 255, 0), 1);
cv::imwrite("demo.jpg",bgr);
cv::imshow("image", bgr);
cv::waitKey(0);
return 0;
测试结果就不放了～
训练和我之前写的mmpose制作数据集和训练方法一样，数据格式也是一样
Windows下运行PaddleDetection例子
一、依赖安装
先进入工程目录，然后进入之前创建的python3.9环境
cd 工程目录
activate paddle_env
克隆代码
git clone https://github.com/PaddlePaddle/PaddleDetection.git然后进入工程目录
cd PaddleDetection进行安装检查
pip install -r requirements.txt出现如下错误
cl: 命令行 error D8021 :无效的数值参数“/Wno-cpp”
error: command 'D:\\\\Program Files\\\\Microsoft Visual Studio\\\\2022\\\\Preview\\\\VC\\\\Tools\\\\MSVC\\\\14.30.30528\\\\bin\\\\HostX86\\\\x64\\\\cl.exe' failed with exit code 2
解决办法
1、首先需要下载Cython_bbox源码，Cython_bbox，点击Download files下载。
2、解压文件。
3、在目录中使用文本打开setup.py，找到第31行，把其中extra_compile_args=[’-Wno-cpp’]的替换为extra_compile_args='gcc': ['/Qstd=c99']。
4、保存改动，返回cython_bbox-0.1.3文件目录，调用cmd并跳转至此目录后，使用命令行
进入python环境activate paddle_env
进入下载目录D:\\temp\\cython_bbox-0.1.3
python setup.py build_ext install 
然后安装pip install -e D:\\temp\\cython_bbox-0.1.3
然后通过pip list查看是否成功安装了cython_bbox-0.1.3
都成功后安装
python setup.py install提示安装成功
测试
python ppdet/modeling/tests/test_architectures.py
如果提示 No module named 'matplotlib'
输入python -m pip install matplotlib进行安装
二、测试
PP-Tracking GUI界面测试版进行测试
https://github.com/yangyudong2020/PP-Tracking_GUi
https://github.com/yangyudong2020/PP-Tracking_GUi下载工程后进行安装
pip install -r requirements.txt提示版本不兼容，然后根据提示进行降级
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. paddlepaddle-gpu 2.2.1. post112 requires numpy<=1.19.3,>=1.13; python_version >= "3.5" and platform_system == "Windows", but you have numpy 1.22.0 which is incompatible. 降级numpy
pip install -U numpy==1.19.3
然后检查通过，运行
python main.py成功运行
Github地址
PP-Tracking之手把手玩转多目标跟踪 - 飞桨AI Studio - 人工智能学习与实训社区PP-Tracking之手把手玩转多目标跟踪 - 飞桨AI Studio - 人工智能学习与实训社区
https://aistudio.baidu.com/aistudio/projectdetail/3022582数据集格式说明PaddleDetection/PrepareMOTDataSet_cn.md at develop · PaddlePaddle/PaddleDetection · GitHubhttps://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/PrepareMOTDataSet_cn.md
以上是关于56训练PaddleDetection的tinypose128x96关键点模型部署ncnn框架mnn框架openvino框架的主要内容，如果未能解决你的问题，请参考以下文章
算法SOTA功能全面性能最佳，PaddleDetection 2.0重磅升级！
AI达人创造营基于PaddleDetection的红细胞形状异常检测