CARLA--车辆添加环视RGB相机-显示并存储数据[超详细]--[入门-3]
Posted 问题多多快快改
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了CARLA--车辆添加环视RGB相机-显示并存储数据[超详细]--[入门-3]相关的知识，希望对你有一定的参考价值。
系列文章目录
CARLA pygame window界面大小调节两种方法-Ubuntu18.04
如何在carla中加入车辆群[基于traffic manager]
CARLA--车辆添加segmentation语义相机[超详细]--[入门-2]
前言
博客上一些常见的RGB相机demo在ubuntu18.4上运行时往往会出现摄像头画面只有一帧的情况，而在windows上似乎是可以正常运行的，对于这些代码，我没能找出问题的具体原因，只是怀疑是由于是系统差异造成的，知道成因和解决方法的朋友欢迎在评论区留言讨论。
下面展示一个能在ubuntu18.4上跑通的rgb环视demo,具体可以实现的效果有：
1.实时连续显示RGB图像画面
2.可搭载在车上指定位置
3.视角可随意在三轴坐标系转换，实现水平/鸟瞰等任意视角的显示
4.定义对应的场视角，组合多个模块的摄像头，可达到环视效果
5.存储采集的单独的数据画面
一、demo模块说明
1.定义对应的场视角，组合多个模块的摄像头，达到环视效果需要“360/场视角”个摄像头，这里展示三个摄像头的效果
#-------------------------- 添加rgb相机--------------------------#
sensor_queue = Queue()
cam_bp = blueprint_library.find('sensor.camera.rgb')
# 可以设置一些参数 set the attribute of camera
cam_bp.set_attribute("image_size_x", "".format(IM_WIDTH))
cam_bp.set_attribute("image_size_y", "".format(IM_HEIGHT))
cam_bp.set_attribute("fov", "60")
#场视角，需要组成环视需要“360/场视角”个摄像头
# cam_bp.set_attribute('sensor_tick', '0.1')f"IM_HEIGHT")
2.视角可随意在三轴坐标系转换，实现水平/鸟瞰等任意视角的显示：
cam01 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=-1.3,z=args.sensor_h),carla.Rotation(yaw=180 )), attach_to=ego_vehicle)
def get_image(data):
data.save_to_disk('/home/lyr/CARLA/CAM_BACK/%06d.jpg' % data.frame)
sensor_callback(data, sensor_queue, "rgb_behind")
cam01.listen(get_image)
sensor_list.append(1)
carla.Transform(carla.Location(0,0,1.8),carla.Rotation(yaw=60 ))
location是相对车辆底盘中心的平移位置，调整可实现车上任意位置摄像头的安装
Rotation是相对车辆底盘中心的旋转位置
Yaw（偏航）：欧拉角向量的y轴
Pitch（俯仰）：欧拉角向量的x轴
Roll（翻滚）： 欧拉角向量的z轴
这里默认Pitch=0 Roll=0,对其进行调整就可以实现任意视角的显示
关于Yaw-Pitch-Roll的说明可以看这篇博客：python旋转矩阵与欧拉角互转
3.继续添加相机模块
cam02 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=120 )), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_right_behind")
data.save_to_disk('/home/lyr/CARLA/CAM_BACK_RIGHT/%06d.jpg' % data.frame)
cam02.listen(get_image)
sensor_list.append(2)
cam03 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=60 )), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_right_front")
data.save_to_disk('/home/lyr/CARLA/CAM_FRONT_RIGHT/%06d.jpg' % data.frame)
cam03.listen(get_image)
sensor_list.append(3)
cam04 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=0)), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_front")
data.save_to_disk('/home/lyr/CARLA/CAM_FRONT/%06d.jpg' % data.frame)
cam04.listen(get_image)
sensor_list.append(4)
cam05 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=300)), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_left_front")
data.save_to_disk('/home/lyr/CARLA/CAM_FRONT_LEFT/%06d.jpg' % data.frame)
cam05.listen(get_image)
sensor_list.append(5)
cam06 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=240)), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_left_behind")
data.save_to_disk('/home/lyr/CARLA/CAM_BACK_LEFT/%06d.jpg' % data.frame)
cam06.listen(get_image)
sensor_list.append(6)
#-------------------------- 相机设置完毕 --------------------------#
4.相机画面组合及可视化展示
w_frame = world.get_snapshot().frame
print("\\nWorld's frame: %d" % w_frame)
try:
rgbs = []
for i in range (0, len(sensor_list)):
s_frame, s_name, s_data = sensor_queue.get(True, 1.0)
print(" Frame: %d Sensor: %s" % (s_frame, s_name))
sensor_type = s_name.split('_')[0]
if sensor_type == 'rgb':
rgbs.append(_parse_image_cb(s_data))
# 仅用来可视化 可注释
rgb=np.concatenate(rgbs, axis=1)[...,:3] # 合并图像
cv2.imshow('vizs', visualize_data(rgb))
cv2.waitKey(100)
if rgb is None or args.save_path is not None:
# 检查是否有各自传感器的文件夹
mkdir_folder(args.save_path)
filename = args.save_path +'rgb/'+str(w_frame)+'.png'
cv2.imwrite(filename, np.array(rgb[...,::-1]))
except Empty:
print(" Some of the sensor information is missed")
5.存储数据
if rgb is None or args.save_path is not None:
# 检查是否有各自传感器的文件夹
mkdir_folder(args.save_path)
filename = args.save_path +'rgb/'+str(w_frame)+'.png'
cv2.imwrite(filename, np.array(rgb[...,::-1]))二、整体代码
import glob
import os
import sys
import time
try:
sys.path.append(glob.glob('../carla/dist/carla-*%d.%d-%s.egg' % (
sys.version_info.major,
sys.version_info.minor,
'win-amd64' if os.name == 'nt' else 'linux-x86_64'))[0])
except IndexError:
pass
import carla
import random
import numpy as np
import cv2
from queue import Queue, Empty
import random
random.seed(10)#决定车辆生成新位置
# args
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--host', metavar='H', default='127.0.0.1', help='IP of the host server (default: 127.0.0.1)')
parser.add_argument('--port', '-p', default=2000, type=int, help='TCP port to listen to (default: 2000)')
parser.add_argument('--tm_port', default=8000, type=int, help='Traffic Manager Port (default: 8000)')
parser.add_argument('--ego-spawn', type=list, default=None, help='[x,y] in world coordinate')
parser.add_argument('--top-view', default=True, help='Setting spectator to top view on ego car')
parser.add_argument('--map', default='Town04', help='Town Map')
parser.add_argument('--sync', default=True, help='Synchronous mode execution')
parser.add_argument('--sensor-h', default=2.4, help='Sensor Height')
parser.add_argument('--save-path', default='存储路径', help='Synchronous mode execution')
args = parser.parse_args()
# 图片大小可自行修改
IM_WIDTH = 400
IM_HEIGHT= 800
actor_list, sensor_list = [], []
sensor_type = ['rgb']
def main(args):
# We start creating the client
client = carla.Client(args.host, args.port)
client.set_timeout(5.0)
world = client.get_world()
# world = client.load_world('Town04_Opt')
# world.unload_map_layer(carla.MapLayer.Buildings)
# world.unload_map_layer(carla.MapLayer.Foliage )
# world.unload_map_layer(carla.MapLayer.Ground )
# world.unload_map_layer(carla.MapLayer.Walls )
blueprint_library = world.get_blueprint_library()
try:
original_settings = world.get_settings()
settings = world.get_settings()
# We set CARLA syncronous mode
settings.fixed_delta_seconds = 0.05
settings.synchronous_mode = True
world.apply_settings(settings)
spectator = world.get_spectator()
# 手动规定
# transform_vehicle = carla.Transform(carla.Location(0, 10, 0), carla.Rotation(0, 0, 0))
# 自动选择
transform_vehicle = random.choice(world.get_map().get_spawn_points())
ego_vehicle = world.spawn_actor(random.choice(blueprint_library.filter("model3")), transform_vehicle)
actor_list.append(ego_vehicle)
# 设置traffic manager
tm = client.get_trafficmanager(args.tm_port)
tm.set_synchronous_mode(True)
# 是否忽略红绿灯
# tm.ignore_lights_percentage(ego_vehicle, 100)
# 如果限速30km/h -> 30*(1-10%)=27km/h
tm.global_percentage_speed_difference(10.0)
ego_vehicle.set_autopilot(True, tm.get_port())
#-------------------------- 添加rgb相机--------------------------#
sensor_queue = Queue()
cam_bp = blueprint_library.find('sensor.camera.rgb')
# 可以设置一些参数 set the attribute of camera
cam_bp.set_attribute("image_size_x", "".format(IM_WIDTH))
cam_bp.set_attribute("image_size_y", "".format(IM_HEIGHT))
cam_bp.set_attribute("fov", "60")
# cam_bp.set_attribute('sensor_tick', '0.1')
cam01 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=-1.3,z=args.sensor_h),carla.Rotation(yaw=180 )), attach_to=ego_vehicle)
def get_image(data):
data.save_to_disk('/home/lyr/CARLA/CAM_BACK/%06d.jpg' % data.frame)
sensor_callback(data, sensor_queue, "rgb_behind")
cam01.listen(get_image)
sensor_list.append(1)
cam02 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=120 )), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_right_behind")
data.save_to_disk('/home/lyr/CARLA/CAM_BACK_RIGHT/%06d.jpg' % data.frame)
cam02.listen(get_image)
sensor_list.append(2)
cam03 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=60 )), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_right_front")
data.save_to_disk('/home/lyr/CARLA/CAM_FRONT_RIGHT/%06d.jpg' % data.frame)
cam03.listen(get_image)
sensor_list.append(3)
cam04 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=0)), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_front")
data.save_to_disk('/home/lyr/CARLA/CAM_FRONT/%06d.jpg' % data.frame)
cam04.listen(get_image)
sensor_list.append(4)
cam05 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=300)), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_left_front")
data.save_to_disk('/home/lyr/CARLA/CAM_FRONT_LEFT/%06d.jpg' % data.frame)
cam05.listen(get_image)
sensor_list.append(5)
cam06 = world.spawn_actor(cam_bp, carla.Transform(carla.Location(x=1,z=args.sensor_h),carla.Rotation(yaw=240)), attach_to=ego_vehicle)
def get_image(data):
sensor_callback(data, sensor_queue, "rgb_left_behind")
data.save_to_disk('/home/lyr/CARLA/CAM_BACK_LEFT/%06d.jpg' % data.frame)
cam06.listen(get_image)
sensor_list.append(6)
#-------------------------- 设置完毕 --------------------------#
while True:
# Tick the server
world.tick()
# 将CARLA界面摄像头跟随车动
loc = ego_vehicle.get_transform().location
spectator.set_transform(carla.Transform(carla.Location(x=loc.x,y=loc.y,z=35),carla.Rotation(yaw=0,pitch=-90,roll=0)))
w_frame = world.get_snapshot().frame
print("\\nWorld's frame: %d" % w_frame)
try:
rgbs = []
for i in range (0, len(sensor_list)):
s_frame, s_name, s_data = sensor_queue.get(True, 1.0)
print(" Frame: %d Sensor: %s" % (s_frame, s_name))
sensor_type = s_name.split('_')[0]
if sensor_type == 'rgb':
rgbs.append(_parse_image_cb(s_data))
# 仅用来可视化 可注释
rgb=np.concatenate(rgbs, axis=1)[...,:3] # 合并图像
cv2.imshow('vizs', visualize_data(rgb))
cv2.waitKey(100)
if rgb is None or args.save_path is not None:
# 检查是否有各自传感器的文件夹
mkdir_folder(args.save_path)
filename = args.save_path +'rgb/'+str(w_frame)+'.png'
cv2.imwrite(filename, np.array(rgb[...,::-1]))
except Empty:
print(" Some of the sensor information is missed")
finally:
world.apply_settings(original_settings)
tm.set_synchronous_mode(False)
for sensor in sensor_list:
sensor.destroy()
for actor in actor_list:
actor.destroy()
print("All cleaned up!")
def mkdir_folder(path):
for s_type in sensor_type:
if not os.path.isdir(os.path.join(path, s_type)):
os.makedirs(os.path.join(path, s_type))
return True
def sensor_callback(sensor_data, sensor_queue, sensor_name):
# Do stuff with the sensor_data data like save it to disk
# Then you just need to add to the queue
sensor_queue.put((sensor_data.frame, sensor_name, sensor_data))
# modify from world on rail code
def visualize_data(rgb, text_args=(cv2.FONT_HERSHEY_SIMPLEX, 0.3, (255,255,255), 1)):
canvas = np.array(rgb[...,::-1])
return canvas
# modify from manual control
def _parse_image_cb(image):
array = np.frombuffer(image.raw_data, dtype=np.dtype("uint8"))
array = np.reshape(array, (image.height, image.width, 4))
array = array[:, :, :3]
array = array[:, :, ::-1]
return array
if __name__ == "__main__":
try:
main(args)
except KeyboardInterrupt:
print(' - Exited by user.')三、效果展示
carla 环视rgb摄像头数据采集
carla中在斑马线附近的指定位置生成车辆
在carla中，车辆位置的生成一般都是在可以生成车辆的点的地方随机分配一个，这就导致生成的车辆的位置不固定。目前有个需求，就是要在固定位置生成车辆，而且还是斑马线附近,解决办法如下(使用的场景是Town03)。
1. 首先找到carla场景中可以生成的点的做坐标。
在carla中使用
spaw_points=world.get_map().get_spawn_points()
get_spawn_points()函数为carla.Map模块中的函数，用来获得地图中所有车辆生成点。返回值是一个包含carla.Transform的list。
把生成的点进行升序排序，然后放到一个txt文件中，命名为spawn_points.txt
2. 找到carla场景中所有斑马线的坐标
在calar中使用
crosswalks_list =world.get_map().get_crosswalks()
get_crosswalks()函数为carla.Map模块中的函数，用来获得地图中所有斑马线的长方形区域四个点的坐标。其中第一个点的坐标会重复，标志长方形的开始和结束点。
x=92.76921081542969,y=-257.09197998046875,z=0.0
x=94.82572174072266,y=-258.790283203125,z=0.0
x=91.5273666381836,y=-268.68878173828125,z=0.0
x=89.21411895751953,y=-267.7293395996094,z=0.0
x=92.76921081542969,y=-257.09197998046875,z=0.0
返回值是一个包含carla.Location的list。
把生成的点放到一个txt文件中，命名为cross_points.txt
3.查找斑马线附近的点，并生成车辆
查找spaw_points.txt文件中和cross_points.txt文件中相邻的点
例如，在cross_points.txt中找到了一个斑马线的位置
x=9.167159080505371,y=124.02804565429688,z=0.0
x=9.19526195526123,y=121.52638244628906,z=0.0
x=-17.256269454956055,y=120.25283813476562,z=0.0
x=-17.7617130279541,y=122.73088836669922,z=0.0
x=9.167159080505371,y=124.02804565429688,z=0.0
然后在spawn_points.txt文件中找9.16附近的点
x=9.284539222717285,y=-105.3431625366211,z=0.27530714869499207 pitch=0.0,yaw=-88.58641815185547,roll=0.0
使用这个点来生产车辆
location = carla.Location(x=9.284539222717285,y=-105.3431625366211,z=0.27530714869499207)
rotation = carla.Rotation(pitch=0.0, yaw=-88.58641815185547, roll=0.0)
transform = carla.Transform(location,rotation)
self.player = self.world.try_spawn_actor(blueprint,transform)
1650440807888692
4 在地图中获得车辆实时的位置
上面的方法是获得所有的点，但是并不知道车实时在地图上的位置。如果能获得车在地图上实时位置，再去上面获得的文本文件中去找，就能对应起来。
实时获得车位置的代码如下。
def get_car_location(self):
if self.player is not None:
return self.player.get_location()
return None
get_location()函数就是实时获得actor的位置。返回值是location.有了这个值然后再去上面的表中去查靠近这个点的斑马线或者生成点，就能更精确的知道生成点在地图上位置。
参考资料
https://carla.readthedocs.io/en/latest/python_api
以上是关于CARLA--车辆添加环视RGB相机-显示并存储数据[超详细]--[入门-3]的主要内容，如果未能解决你的问题，请参考以下文章