将冻结图转换为 TRT 图时 Jetson Nano 上的 TensorRT 错误

Posted 2023-04-15

【中文标题】将冻结图转换为 TRT 图时 Jetson Nano 上的 TensorRT 错误

【英文标题】：TensorRT Error on Jetson Nano when converting a Frozen Graph to TRT Graph

【发布时间】：2019-11-18 09:21:18

【问题描述】：

大家好，对于 Tensorflow 和 TensorRT 来说都是新手，我无法将现有的冻结图转换为 tensorRT 图。我认为我拥有的代码没有成功转换我的图表。在 Nvidia Jetson Nano 上运行它。

我已尝试遵循此处所示的指南：https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#using-frozengraph

def load_object_detection_model(self):
        # Load TensorFlow object detection model

        gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=.5)

        EXPORTED_OBJECT_DETECTION_MODEL = 'frozen_model_x.pb'
        self.graph_obj = tf.Graph()
        with self.graph_obj.as_default():
            od_graph_def = tf.GraphDef()
            with tf.gfile.GFile(EXPORTED_OBJECT_DETECTION_MODEL, 'rb') as fid:
                serialized_graph = fid.read()
                od_graph_def.ParseFromString(serialized_graph)
                tf.import_graph_def(od_graph_def, name='')

         # Optimize Graph with TensorRT

        trt_graph = trt.create_inference_graph(
            input_graph_def=od_graph_def,
            outputs=['num_detections', 'detection_boxes', 'detection_scores', 'detection_classes'],
            max_batch_size=1,
            max_workspace_size_bytes=4000000000,
            precision_mode='FP16')

        print('reading graph')

        output_node = tf.import_graph_def(
            trt_graph,
            return_elements=['num_detections', 'detection_boxes', 'detection_scores', 'detection_classes'])

        self.graph_obj = output_node # Replace frozen graph with optimized graph 

        print('converted graph')

我得到的错误输出是：“在 load_object_detection_model ops = self.graph_obj.get_operations() AttributeError: 'list' object has no attribute 'get_operations'" 对应于以下代码：

# get handles to objects in object detection graph
        ops = self.graph_obj.get_operations()
        all_tensor_names = output.name for op in ops for output in op.outputs
        self.tensor_dict = 
        for key in [
            'num_detections', 'detection_boxes', 'detection_scores',
            'detection_classes', 'detection_masks'
        ]:
            tensor_name = key + ':0'
            if tensor_name in all_tensor_names:
                self.tensor_dict[key] = self.graph_obj.get_tensor_by_name(tensor_name)

        self.obj_image_tensor = self.graph_obj.get_tensor_by_name('image_tensor:0')
        self.logger.debug('created object detection model graph from '.format(EXPORTED_OBJECT_DETECTION_MODEL))

        # create session for object detection
        self.sess_obj = tf.Session(graph=self.graph_obj)
        self.logger.debug('created object detection model session')

（上面这段代码紧跟在前面的代码 sn-p 之后）。

运行 Ubuntu 18.04、Python 3.6.8、TensorFlow 1.13.1。 TensorRT 详情如下：

ii  graphsurgeon-tf                            5.0.6-1+cuda10.0                                arm64        GraphSurgeon for TensorRT package
ii  libnvinfer-dev                             5.0.6-1+cuda10.0                                arm64        TensorRT development libraries and headers
ii  libnvinfer-samples                         5.0.6-1+cuda10.0                                all          TensorRT samples and documentation
ii  libnvinfer5                                5.0.6-1+cuda10.0                                arm64        TensorRT runtime libraries
ii  python-libnvinfer                          5.0.6-1+cuda10.0                                arm64        Python bindings for TensorRT
ii  python-libnvinfer-dev                      5.0.6-1+cuda10.0                                arm64        Python development package for TensorRT
ii  python3-libnvinfer                         5.0.6-1+cuda10.0                                arm64        Python 3 bindings for TensorRT
ii  python3-libnvinfer-dev                     5.0.6-1+cuda10.0                                arm64        Python 3 development package for TensorRT
ii  tensorrt                                   5.0.6.3-1+cuda10.0                              arm64        Meta package of TensorRT
ii  uff-converter-tf                           5.0.6-1+cuda10.0                                arm64        UFF converter for TensorRT package

【参考方案1】：

由于 pyCUDA，Jetson 平台不支持 TensorRT Python API。但是，python 解析器运行良好。以下是一些供您参考的替代方案：

Python -> [Wrapper] -> C++ inference TensorFlow-TensorRT

您可以使用 Cython 包装 TensorRT C++ 代码，以便您可以从 python 调用它们。有关详细信息，请参阅 Cython 的文档。

还有一个关于 Jetson Nano 的示例可能会有所帮助：Running TensorRT Optimized GoogLeNet on Jetson Nano