使用docker+tensorflow-serving进行模型热部署

Posted 2020-12-23 techi

部署多个模型

（1）直接部署两个模型faster-rcnn与retina，构建代码的文件夹。
文件夹结构为：

mutimodel

├── faster_rcnn
│ └── 1
│ ├── assets
│ ├── saved_model.pb
│ └── variables
│ ├── variables.data-00000-of-00001
│ └── variables.index
├── model.config
├── retina
│ └── 1
│ ├── assets
│ ├── saved_model.pb
│ └── variables
│ ├── variables.data-00000-of-00001
│ └── variables.index

model.config的内容为：

model_config_list {
  config {
    name: ‘rcnn‘,
    model_platform: "tensorflow",
    base_path: ‘/models/mutimodel/faster_rcnn‘  # 这里的base_path是docker中的映射路径
  },
  config {
    name: ‘retina‘,
    model_platform: "tensorflow",
    base_path: ‘/models/mutimodel/retina‘  # 这里的base_path是docker中的映射路径
  }
}

（2）启动docker
sudo docker run -p 8501:8501 -p 8500:8500 --mount type=bind,source=/home/techi/techi/code/model_saved_files/mutimodel,target=/models/mutimodel -t tensorflow/serving --model_config_file=/models/mutimodel/model.config
其中，target为docker挂载地址，model_config_file对应的地址也是挂载地址

编写客户端代码

import tensorflow as tf
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc

options = [(‘grpc.max_send_message_length‘, 1000 * 1024 * 1024),
           (‘grpc.max_receive_message_length‘, 1000 * 1024 * 1024)]

channel = grpc.insecure_channel(‘xxx‘, options=options)

stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)

(_, _), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_test = tf.reshape(x_test, (10000, -1))
# print(x_test)
x_test = tf.cast(x_test, dtype=tf.float32)
x_test = (x_test - 127.5) / 127.5

request = predict_pb2.PredictRequest()
request.model_spec.name = "yoloV3"
request.model_spec.signature_name = ‘serving_default‘
request.inputs[‘input_1‘].CopyFrom(tf.make_tensor_proto(x_test[:20], shape=(20, 784)))  # 在远程传输中，使用tensorproto格式进行传输
response = stub.Predict(request, 10.0)
output = tf.make_ndarray(response.outputs[‘dense_2‘])  # 这里的名称使用 saved_model_cli show --dir=‘相对路径/绝对路径‘ --all 进行查看
output = tf.argmax(output, axis=-1)
print(output)
print(y_test[:20])