markdown TVM基准实验

Posted 2021-05-05

|TVM batch|vgg-19|resnet-50|resnext-50|inception_v3|drn-c-26|dcn-resnet-101|
|--|--|--|--|--|--|--|
|1|2.0128 ms|1.2898 ms|1.1088 ms|2.4062 ms|1.6831 ms|3.8074 ms|
|16|21.5044 ms|7.926 ms|8.6468 ms|16.4986 ms|19.9313 ms|26.3714 ms|
|16/16=1|1.344 |0.4954 |0.5404 |1.0312|1.2457|1.6482|
|1080:1bs|2.65|1.43|1.28|2.62|2.13|3.97|
|1080:16bs|1.76|0.65|0.7|1.22|1.6|2.19|

------
1. vgg-19 (batch=1): 2.012773891 ms
2. resnet-50 (batch=1): 1.2898257181666668 ms
3. resnext-50 (batch=1): 1.1087786321666666 ms
4. inception_v3 (batch=1): 2.406170417666667 ms
5. drn-c-26 (batch=1): 1.6831447698333333 ms
6. dcn-resnet-101 (batch=1): 3.8073995348333334 ms
------
1. vgg-19 (batch=16): 21.504362825666664 ms
2. resnet-50 (batch=16): 7.925978392499999 ms
3. resnext-50 (batch=16): 8.646780021 ms
4. inception_v3 (batch=16): 16.498573984666667 ms
5. drn-c-26 (batch=16): 19.931346115166665 ms
6. dcn-resnet-101 (batch=16): 26.371414398833338 ms

|TRT batch|vgg-19|resnet-50|resnext-50|inception_v3|drn-c-26|dcn-resnet-101|
|--|--|--|--|--|--|--|
|1|2.4264ms|1.14405ms|5.85555ms|2.00563ms|1.58284ms||
|16|16.7398ms|6.44741ms|12.2726ms|7.27698ms|15.2867ms||
|16/16=1|1.0462375|0.402963125|0.7670375|0.606415|0.95541875||
|32|32.2673ms||

------
1. TensorRT onnx_model/resnet-50.onnx (batch=1) int8 Avg. Time: 1.12706ms Each Image Time: 1.12706
2. TensorRT onnx_model/resnet-50.onnx (batch=2) int8 Avg. Time: 1.7049ms Each Image Time: 0.85245
3. TensorRT onnx_model/resnet-50.onnx (batch=3) int8 Avg. Time: 2.35729ms Each Image Time: 0.785764
4. TensorRT onnx_model/resnet-50.onnx (batch=4) int8 Avg. Time: 2.61804ms Each Image Time: 0.65451
5. TensorRT onnx_model/resnet-50.onnx (batch=5) int8 Avg. Time: 3.07354ms Each Image Time: 0.614708
6. TensorRT onnx_model/resnet-50.onnx (batch=6) int8 Avg. Time: 3.34202ms Each Image Time: 0.557004
7. TensorRT onnx_model/resnet-50.onnx (batch=7) int8 Avg. Time: 3.57236ms Each Image Time: 0.510336
8. TensorRT onnx_model/resnet-50.onnx (batch=8) int8 Avg. Time: 3.97659ms Each Image Time: 0.497074
9. TensorRT onnx_model/resnet-50.onnx (batch=9) int8 Avg. Time: 4.15201ms Each Image Time: 0.461335
10. TensorRT onnx_model/resnet-50.onnx (batch=10) int8 Avg. Time: 4.82598ms Each Image Time: 0.482598
11. TensorRT onnx_model/resnet-50.onnx (batch=12) int8 Avg. Time: 5.28415ms Each Image Time: 0.440346
12. TensorRT onnx_model/resnet-50.onnx (batch=13) int8 Avg. Time: 5.58143ms Each Image Time: 0.429341
13. TensorRT onnx_model/resnet-50.onnx (batch=14) int8 Avg. Time: 6.079ms Each Image Time: 0.434214
14. TensorRT onnx_model/resnet-50.onnx (batch=15) int8 Avg. Time: 6.27433ms Each Image Time: 0.418288
15. TensorRT onnx_model/resnet-50.onnx (batch=16) int8 Avg. Time: 6.49506ms Each Image Time: 0.405941


-----
1. TensorRT onnx_model/vgg-19.onnx (batch=1) int8 Avg. Time: 2.4264ms
2. TensorRT onnx_model/resnet-50.onnx (batch=1) int8 Avg. Time: 1.14405ms
3. TensorRT onnx_model/resnext-50.onnx (batch=1) int8 Avg. Time: 5.85555ms
4. TensorRT onnx_model/inception_v3.onnx (batch=1) int8 Avg. Time: 2.00563ms
5. TensorRT onnx_model/drn-c-26.onnx (batch=1) int8 Avg. Time: 1.58284ms
-----
1. TensorRT onnx_model/vgg-19.onnx (batch=16) int8 Avg. Time: 16.7398ms
2. TensorRT onnx_model/resnet-50.onnx (batch=16) int8 Avg. Time: 6.44741ms
3. TensorRT onnx_model/resnext-50.onnx (batch=16) int8 Avg. Time: 12.2726ms
4. TensorRT onnx_model/inception_v3.onnx (batch=16) int8 Avg. Time: 7.27698ms
5. TensorRT onnx_model/drn-c-26.onnx (batch=16) int8 Avg. Time: 15.2867ms

------
1. resnet-50 (batch=1): 4.841296195983887 ms
2. vgg-19 (batch=1): 6.2340850830078125 ms
3. resnext-50 (batch=1): 8.653332233428955 ms
4. inception_v3 (batch=1): 11.25043511390686 ms
5. drn-c-26 (batch=1): 7.524022817611694 ms
6. dcn-resnet-101 (batch=1): 10.298750877380371 ms
------
1. resnet-50 (batch=16): 25.634759187698364 ms
2. vgg-19 (batch=16): 51.15072774887085 ms
3. resnext-50 (batch=16): 38.369793176651 ms
4. inception_v3 (batch=16): 40.72020173072815 ms
5. drn-c-26 (batch=16): 81.2522509098053 ms
6. dcn-resnet-101 (batch=16): 70.67626214027405 ms