fast_rcnn_r50 预训练转换为托管在 Triton 模型服务器中的 ONNX
Posted
技术标签:
【中文标题】fast_rcnn_r50 预训练转换为托管在 Triton 模型服务器中的 ONNX【英文标题】：faster_rcnn_r50 pretrained converted to ONNX hosted in Triton model server 【发布时间】：2022-01-08 12:59:25 【问题描述】：我在这里查看了 mmdetection 文档以将 pytorch 模型转换为 onnx link
所有安装都正确，我正在使用 onnxruntime==1.8.1，ONNX Runtime MMCV_WITH_OPS 的自定义运算符。
我正在使用 configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py 以获得更快的 rcnn link 并使用 R-5-FPN 预训练模型 link
我使用它来将预训练模型转换为 onnx，并成功保存了一个名为 fasterrcnn.onnx 的 onnx 文件
python tools/deployment/pytorch2onnx.py \
configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py \
checkpoints/faster_rcnn/faster_rcnn_r50_fpn_mstrain_3x_coco_20210524_110822-e10bd31c.pth \
--output-file checkpoints/faster_rcnn/fasterrcnn.onnx \
--input-img demo/demo.jpg \
--test-img tests/data/color.jpg \
--shape 608 608 \
--dynamic-export \
--cfg-options \
model.test_cfg.deploy_nms_pre=-1 \
我正在使用该 onnx 文件在 NVIDIA triton 模型服务器中托管模型。
fasterrcnn_model | 1 | READY
Triton 的 onnx 模型的模型总结如下图
"name": "fasterrcnn_model",
"platform": "onnxruntime_onnx",
"backend": "onnxruntime",
"version_policy":
"latest":
"num_versions": 1
,
"max_batch_size": 1,
"input": [
"name": "input",
"data_type": "TYPE_FP32",
"dims": [
3,
-1,
-1
]
],
"output": [
"name": "labels",
"data_type": "TYPE_INT64",
"dims": [
-1
]
,
"name": "dets",
"data_type": "TYPE_FP32",
"dims": [
-1,
5
]
],
"batch_input": [],
"batch_output": [],
"optimization":
"priority": "PRIORITY_DEFAULT",
"input_pinned_memory":
"enable": true
,
"output_pinned_memory":
"enable": true
,
"gather_kernel_buffer_threshold": 0,
"eager_batching": false
,
"instance_group": [
"name": "fasterrcnn_model",
"kind": "KIND_CPU",
"count": 1,
"gpus": [],
"profile": []
],
"default_model_filename": "model.onnx",
"cc_model_filenames": ,
"metric_tags": ,
"parameters": ,
"model_warmup": []
摘要概述了输出具有“标签”和“数据”类别
在向 triton 发送带有示例图像的推理请求后，我收到以下响应。 标签
[[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.
18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35.
36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53.
54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71.
72. 73. 74. 75. 76. 77. 78. 79. 0. 1. 2. 3. 4. 5. 6. 7. 8. 9.
10. 11. 12. 13. 14. 15. 16. 17. 18. 19.]]
数据
[[[-1.0000e+00 -1.0000e+00 -1.0000e+00 -1.0000e+00 0.0000e+00]
[-3.0000e+02 -3.0000e+02 -3.0000e+02 -3.0000e+02 0.0000e+00]
[-5.9900e+02 -5.9900e+02 -5.9900e+02 -5.9900e+02 0.0000e+00]
[-8.9800e+02 -8.9800e+02 -8.9800e+02 -8.9800e+02 0.0000e+00]
[-1.1970e+03 -1.1970e+03 -1.1970e+03 -1.1970e+03 0.0000e+00]
[-1.4960e+03 -1.4960e+03 -1.4960e+03 -1.4960e+03 0.0000e+00]
[-1.7950e+03 -1.7950e+03 -1.7950e+03 -1.7950e+03 0.0000e+00]
[-2.0940e+03 -2.0940e+03 -2.0940e+03 -2.0940e+03 0.0000e+00]
[-2.3930e+03 -2.3930e+03 -2.3930e+03 -2.3930e+03 0.0000e+00]
[-2.6920e+03 -2.6920e+03 -2.6920e+03 -2.6920e+03 0.0000e+00]
[-2.9910e+03 -2.9910e+03 -2.9910e+03 -2.9910e+03 0.0000e+00]
[-3.2900e+03 -3.2900e+03 -3.2900e+03 -3.2900e+03 0.0000e+00]
[-3.5890e+03 -3.5890e+03 -3.5890e+03 -3.5890e+03 0.0000e+00]
[-3.8880e+03 -3.8880e+03 -3.8880e+03 -3.8880e+03 0.0000e+00]
[-4.1870e+03 -4.1870e+03 -4.1870e+03 -4.1870e+03 0.0000e+00]
[-4.4860e+03 -4.4860e+03 -4.4860e+03 -4.4860e+03 0.0000e+00]
[-4.7850e+03 -4.7850e+03 -4.7850e+03 -4.7850e+03 0.0000e+00]
[-5.0840e+03 -5.0840e+03 -5.0840e+03 -5.0840e+03 0.0000e+00]
[-5.3830e+03 -5.3830e+03 -5.3830e+03 -5.3830e+03 0.0000e+00]
[-5.6820e+03 -5.6820e+03 -5.6820e+03 -5.6820e+03 0.0000e+00]
[-5.9810e+03 -5.9810e+03 -5.9810e+03 -5.9810e+03 0.0000e+00]
[-6.2800e+03 -6.2800e+03 -6.2800e+03 -6.2800e+03 0.0000e+00]
[-6.5790e+03 -6.5790e+03 -6.5790e+03 -6.5790e+03 0.0000e+00]
[-6.8780e+03 -6.8780e+03 -6.8780e+03 -6.8780e+03 0.0000e+00]
[-7.1770e+03 -7.1770e+03 -7.1770e+03 -7.1770e+03 0.0000e+00]
[-7.4760e+03 -7.4760e+03 -7.4760e+03 -7.4760e+03 0.0000e+00]
[-7.7750e+03 -7.7750e+03 -7.7750e+03 -7.7750e+03 0.0000e+00]
[-8.0740e+03 -8.0740e+03 -8.0740e+03 -8.0740e+03 0.0000e+00]
[-8.3730e+03 -8.3730e+03 -8.3730e+03 -8.3730e+03 0.0000e+00]
[-8.6720e+03 -8.6720e+03 -8.6720e+03 -8.6720e+03 0.0000e+00]
[-8.9710e+03 -8.9710e+03 -8.9710e+03 -8.9710e+03 0.0000e+00]
[-9.2700e+03 -9.2700e+03 -9.2700e+03 -9.2700e+03 0.0000e+00]
[-9.5690e+03 -9.5690e+03 -9.5690e+03 -9.5690e+03 0.0000e+00]
[-9.8680e+03 -9.8680e+03 -9.8680e+03 -9.8680e+03 0.0000e+00]
[-1.0167e+04 -1.0167e+04 -1.0167e+04 -1.0167e+04 0.0000e+00]
[-1.0466e+04 -1.0466e+04 -1.0466e+04 -1.0466e+04 0.0000e+00]
[-1.0765e+04 -1.0765e+04 -1.0765e+04 -1.0765e+04 0.0000e+00]
[-1.1064e+04 -1.1064e+04 -1.1064e+04 -1.1064e+04 0.0000e+00]
[-1.1363e+04 -1.1363e+04 -1.1363e+04 -1.1363e+04 0.0000e+00]
[-1.1662e+04 -1.1662e+04 -1.1662e+04 -1.1662e+04 0.0000e+00]
[-1.1961e+04 -1.1961e+04 -1.1961e+04 -1.1961e+04 0.0000e+00]
[-1.2260e+04 -1.2260e+04 -1.2260e+04 -1.2260e+04 0.0000e+00]
[-1.2559e+04 -1.2559e+04 -1.2559e+04 -1.2559e+04 0.0000e+00]
[-1.2858e+04 -1.2858e+04 -1.2858e+04 -1.2858e+04 0.0000e+00]
[-1.3157e+04 -1.3157e+04 -1.3157e+04 -1.3157e+04 0.0000e+00]
[-1.3456e+04 -1.3456e+04 -1.3456e+04 -1.3456e+04 0.0000e+00]
[-1.3755e+04 -1.3755e+04 -1.3755e+04 -1.3755e+04 0.0000e+00]
[-1.4054e+04 -1.4054e+04 -1.4054e+04 -1.4054e+04 0.0000e+00]
[-1.4353e+04 -1.4353e+04 -1.4353e+04 -1.4353e+04 0.0000e+00]
[-1.4652e+04 -1.4652e+04 -1.4652e+04 -1.4652e+04 0.0000e+00]
[-1.4951e+04 -1.4951e+04 -1.4951e+04 -1.4951e+04 0.0000e+00]
[-1.5250e+04 -1.5250e+04 -1.5250e+04 -1.5250e+04 0.0000e+00]
[-1.5549e+04 -1.5549e+04 -1.5549e+04 -1.5549e+04 0.0000e+00]
[-1.5848e+04 -1.5848e+04 -1.5848e+04 -1.5848e+04 0.0000e+00]
[-1.6147e+04 -1.6147e+04 -1.6147e+04 -1.6147e+04 0.0000e+00]
[-1.6446e+04 -1.6446e+04 -1.6446e+04 -1.6446e+04 0.0000e+00]
[-1.6745e+04 -1.6745e+04 -1.6745e+04 -1.6745e+04 0.0000e+00]
[-1.7044e+04 -1.7044e+04 -1.7044e+04 -1.7044e+04 0.0000e+00]
[-1.7343e+04 -1.7343e+04 -1.7343e+04 -1.7343e+04 0.0000e+00]
[-1.7642e+04 -1.7642e+04 -1.7642e+04 -1.7642e+04 0.0000e+00]
[-1.7941e+04 -1.7941e+04 -1.7941e+04 -1.7941e+04 0.0000e+00]
[-1.8240e+04 -1.8240e+04 -1.8240e+04 -1.8240e+04 0.0000e+00]
[-1.8539e+04 -1.8539e+04 -1.8539e+04 -1.8539e+04 0.0000e+00]
[-1.8838e+04 -1.8838e+04 -1.8838e+04 -1.8838e+04 0.0000e+00]
[-1.9137e+04 -1.9137e+04 -1.9137e+04 -1.9137e+04 0.0000e+00]
[-1.9436e+04 -1.9436e+04 -1.9436e+04 -1.9436e+04 0.0000e+00]
[-1.9735e+04 -1.9735e+04 -1.9735e+04 -1.9735e+04 0.0000e+00]
[-2.0034e+04 -2.0034e+04 -2.0034e+04 -2.0034e+04 0.0000e+00]
[-2.0333e+04 -2.0333e+04 -2.0333e+04 -2.0333e+04 0.0000e+00]
[-2.0632e+04 -2.0632e+04 -2.0632e+04 -2.0632e+04 0.0000e+00]
[-2.0931e+04 -2.0931e+04 -2.0931e+04 -2.0931e+04 0.0000e+00]
[-2.1230e+04 -2.1230e+04 -2.1230e+04 -2.1230e+04 0.0000e+00]
[-2.1529e+04 -2.1529e+04 -2.1529e+04 -2.1529e+04 0.0000e+00]
[-2.1828e+04 -2.1828e+04 -2.1828e+04 -2.1828e+04 0.0000e+00]
[-2.2127e+04 -2.2127e+04 -2.2127e+04 -2.2127e+04 0.0000e+00]
[-2.2426e+04 -2.2426e+04 -2.2426e+04 -2.2426e+04 0.0000e+00]
[-2.2725e+04 -2.2725e+04 -2.2725e+04 -2.2725e+04 0.0000e+00]
[-2.3024e+04 -2.3024e+04 -2.3024e+04 -2.3024e+04 0.0000e+00]
[-2.3323e+04 -2.3323e+04 -2.3323e+04 -2.3323e+04 0.0000e+00]
[-2.3622e+04 -2.3622e+04 -2.3622e+04 -2.3622e+04 0.0000e+00]
[-1.0000e+00 -1.0000e+00 -1.0000e+00 -1.0000e+00 0.0000e+00]
[-3.0000e+02 -3.0000e+02 -3.0000e+02 -3.0000e+02 0.0000e+00]
[-5.9900e+02 -5.9900e+02 -5.9900e+02 -5.9900e+02 0.0000e+00]
[-8.9800e+02 -8.9800e+02 -8.9800e+02 -8.9800e+02 0.0000e+00]
[-1.1970e+03 -1.1970e+03 -1.1970e+03 -1.1970e+03 0.0000e+00]
[-1.4960e+03 -1.4960e+03 -1.4960e+03 -1.4960e+03 0.0000e+00]
[-1.7950e+03 -1.7950e+03 -1.7950e+03 -1.7950e+03 0.0000e+00]
[-2.0940e+03 -2.0940e+03 -2.0940e+03 -2.0940e+03 0.0000e+00]
[-2.3930e+03 -2.3930e+03 -2.3930e+03 -2.3930e+03 0.0000e+00]
[-2.6920e+03 -2.6920e+03 -2.6920e+03 -2.6920e+03 0.0000e+00]
[-2.9910e+03 -2.9910e+03 -2.9910e+03 -2.9910e+03 0.0000e+00]
[-3.2900e+03 -3.2900e+03 -3.2900e+03 -3.2900e+03 0.0000e+00]
[-3.5890e+03 -3.5890e+03 -3.5890e+03 -3.5890e+03 0.0000e+00]
[-3.8880e+03 -3.8880e+03 -3.8880e+03 -3.8880e+03 0.0000e+00]
[-4.1870e+03 -4.1870e+03 -4.1870e+03 -4.1870e+03 0.0000e+00]
[-4.4860e+03 -4.4860e+03 -4.4860e+03 -4.4860e+03 0.0000e+00]
[-4.7850e+03 -4.7850e+03 -4.7850e+03 -4.7850e+03 0.0000e+00]
[-5.0840e+03 -5.0840e+03 -5.0840e+03 -5.0840e+03 0.0000e+00]
[-5.3830e+03 -5.3830e+03 -5.3830e+03 -5.3830e+03 0.0000e+00]
[-5.6820e+03 -5.6820e+03 -5.6820e+03 -5.6820e+03 0.0000e+00]]]
labels 响应看起来像常规的 COCO 类 (80)，但我很难解码 dets 响应。这看起来像边界框坐标 4 和置信阈值 1。制作形状 (1,100,5)。关于 dets 类别应该代表什么的任何想法？输出通常取决于模型本身，但我认为 onnx 转换正在将输出更改为 labels 和 dets
【问题讨论】：
【参考方案1】：看转换脚本好像dets是盒子加分数的组合
boxes (Tensor): The bounding boxes of shape [N, num_boxes, 4]
scores (Tensor): The detection scores of shape [N, num_boxes, num_classes]
tuple[Tensor, Tensor]: dets of shape [N, num_det, 5]
and class labels of shape [N, num_det].
dets = torch.cat([boxes, scores], dim=2)
https://github.com/open-mmlab/mmdetection/blob/master/mmdet/core/export/onnx_helper.py#L197
【讨论】：
以上是关于fast_rcnn_r50 预训练转换为托管在 Triton 模型服务器中的 ONNX的主要内容，如果未能解决你的问题，请参考以下文章
手把手写深度学习(14)：如何利用官方预训练模型做微调/迁移学习？（以Resnet50提取图像特征为例）
使用 tf slim 重新训练预训练的 ResNet-50 模型以进行分类
如何使用 Pytorch 中的预训练权重修改具有 4 个通道作为输入的 resnet 50？