图像语义分割出的json文件和原图，用plt绘制图像mask

Posted 2020-11-06 47谢远鹏

1、弱监督

　　由于公司最近准备开个新项目，用深度学习训练个能够自动标注的模型，但模型要求的训练集比较麻烦，，要先用ffmpeg从视频中截取一段视频，在用opencv抽帧得到图片，所以本人只能先用语义分割出的json文件和原图，合成图像的mask。

2、环境安装

操作系统：windows 7

python环境：3.6.4

所需要的库：numpy，matplotlib，PIL，opencv-python

软件：ffmpeg

3、截取视频

　　截取一段视频中一直有同一个人出现的视频段。

# -*- coding: utf-8 -*-
import os

def cut(filename, start, end):
    assert os.path.exists(filename) is True, "The soruse file is not exists."

    start1 = start.replace(":","")
    end1 = end.replace(":","")
    #print(start1 + " " + end1)
    videoname = "{}{}-{}.mp4".format(filename.rsplit(".",1)[0],start1,end1)
    cmd = "ffmpeg -i {} -vcodec copy -acodec copy -ss {} -to {} {} -y".format(filename,start,end,videoname)
    result = os.popen(cmd)
    return result

if __name__ == "__main__":
    file = input("需要截取的视频:")
    start = input("起始时间(HH:MM:SS):")
    end = input("结束时间(HH:MM:SS):")
    print(cut(file, start, end))

输出结果：

左边这个是原视频，右边这个是截取的视频

4、视频抽帧

　　从视频中，每隔40帧抽取一张图片。

import cv2

def get_video_pic(name,zhen):
    path = name.rsplit(".",1)[0]
    cap = cv2.VideoCapture(name)
    for i in range(1,int(cap.get(7)),zhen):
        cap.set(1, i)
        rval, frame = cap.read()
        if rval:
            picname = "{}{}.jpg".format(path,str(i))
            cv2.imwrite(picname, frame)
        cap.release()

if __name__ == "__main__":
    video = r"C:/Users/yuanpeng.xie/Desktop/test/yongcun-3.30-3.36.mp4"
    frame = 40
    get_video_pic(video,int(frame))
    print("over")

输出结果：

视频总共好像是131帧，每隔40帧抽取一次就是下面这四张图片

5、图像予以分割标注

　　自己找工具去标，保存成json文件

输出结果：

　　json文件部分内容

6、用json文件和原图，用plt绘制图像mask

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as img
from PIL import Image
import os
import json

def PictureToMask(d_object, sourcePicture):
    \'\'\'得到原图的宽度和高度\'\'\'
    im = Image.open(sourcePicture)
    size = list(im.size)
    width = size[0]
    height = size[1]
    
    \'\'\'将图片的像素的宽度和高度换算成英寸的宽度和高度\'\'\'
    dpi = 80                 #分辨率
    ycwidth = width/dpi      #宽度(英寸) = 像素宽度 / 分辨率
    ycheight = height/dpi    #高度(英寸) = 像素高度 / 分辨率

    color = ["g","r","b","y","skyblue","k","m","c"]
    fig, ax = plt.subplots(figsize=(ycwidth,ycheight))
    for region in d_object:
        \'\'\'将传进来的x轴坐标点和y轴坐标点转换成numpy数组，相加后转置成多行两列\'\'\'
        x = np.array(d_object[region][0])
        y = np.array(d_object[region][1]) * -1
        xy = np.vstack([x,y]).T
        \'\'\'
        #设置画框的背景图片为原图
        fig = plt.figure(figsize=(ycwidth,ycheight),dpi=dpi)
        bgimg = img.imread(sourcePicture)
        fig.figimage(bgimg)
        \'\'\'
        \'\'\'将numpy中的坐标连城线，绘制在plt上\'\'\'
        plt.plot(xy[:,0],xy[:,1],color=color[int(region)])
        plt.fill_between(xy[:,0],xy[:,1],facecolor=color[int(region)])  #对该分割区域填充颜色
    plt.xticks([0,width])
    plt.yticks([0,-height])
    plt.axis("off")
    #保存图片
    path = sourcePicture.rsplit(".",1)[0]
    print(sourcePicture)
    print(path)
    plt.savefig(path + "-mask.png", format=\'png\', bbox_inches=\'tight\', transparent=True, dpi=100) # bbox_inches=\'tight\' 图片边界空白紧致, 背景透明
    #plt.show()

def getJson(filepath):
    \'\'\'从文件夹获取json文件内容，返回字典\'\'\'
    files = os.listdir(filepath)
    for file in files:
        if file.split(".")[1] == "json":
            jsonfile = filepath + file
            break
    jsonstr = open(jsonfile,"r",encoding="utf8").read()
    d_json = json.loads(jsonstr)
    #print(d_json)
    return d_json

def getPath():
    \'\'\'输入图片文件夹路径\'\'\'
    filepath = input("图片文件夹路径：")
    if filepath.endswith != "/" or filepath.endswith != "\\\\":
        filepath = filepath + "/"
    return filepath

def main():
    filepath = getPath()
    d_json = getJson(filepath)
    for key in d_json:
        data = d_json.get(key)
        pictureName = data["filename"]
        d_object = {}
        for region in data["regions"]:
            l_object = []
            x = data["regions"][region]["shape_attributes"]["all_points_x"]
            y = data["regions"][region]["shape_attributes"]["all_points_y"]
            l_object.append(x)
            l_object.append(y)
            d_object[region] = l_object
        sourcePicture = filepath + pictureName
        PictureToMask(d_object, sourcePicture)

if __name__ == "__main__":
    main()

输出结果：

　　图像mask

7、小问题

　　有一个小问题，就是mask的尺寸会比原图尺寸大，因为保存是会把整个figure保存，等于mask多了个边框，后来翻资料，将mask的大小调成和figure成一样。

plt.axes([0,0,1,1])

 

然后再把图片保存

去掉代码中的bbox_inches=\'tight\'这句话，就可以将mask保存成和原图一样的尺寸了