飞行机器人专栏-- 异构多视角视觉系统

Posted Techblog of HaoWANG

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了飞行机器人专栏-- 异构多视角视觉系统相关的知识,希望对你有一定的参考价值。


  1. 感知系统架构

为满足天空端主控制器的诸如RGB-D图像处理等大容量数据吞吐、高速并行计算、实时运动控制以及通信和可视化任务的计算算力需求,同时优化功耗表现,采用了结构紧凑、功耗表现优异的边缘计算硬件NVIDA IJetson AGXOrin 。该开发者套件包含高性能、高能效的 Jetson AGX Orin 模组,算力高达 275 TOPS是上一代多个并发 AI 推理管道性能的 8 倍,运行于 NVIDIA AI 软件堆栈,广泛应用于图像处理、嵌入式控制、并行计算等场景,可以为机器人、制造、控制和电力等行业打造先进的机器人和边缘 AI 应用。系统架构如图所示。

2. Realsense2 D435i & USB Monocular

  1. librealsense & realsense2_camera ROS package

  • 安装SDK库:

librealsense: V2.50.0 + https://github.com/IntelRealSense/librealsense/releases/tag/v2.50.0

在/home下进入librealsense,并安装依赖项
cd librealsense
sudo apt-get install libudev-dev pkg-config libgtk-3-dev
sudo apt-get install libusb-1.0-0-dev pkg-config
sudo apt-get install libglfw3-dev
sudo apt-get install libssl-dev
  • 安装依赖:

安装依赖项和编译

sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && udevadm trigger 
mkdir build
cd build
cmake ../ -DBUILD_EXAMPLES=true
make
sudo make install

测试安装结果

这时候可以连接摄像头了,输入以下命令查看结果

realsense-viewer 

如果成功,可以看到点云图像


  • 编译ROS PKG:

在工作空间src下克隆项目:

cd ~/catkin_ws/src
git clone https://github.com/IntelRealSense/realsense-ros.git
git clone https://github.com/pal-robotics/ddynamic_reconfigure.git
cd ~/catkin_ws && catkin_make

ROS中测试

roslaunch realsense2_camera rs_camera.launch 
  • ERROR:undefined symbol: _ZN2cv3MatC1Ev【librealsense2_camera.so: undefined symbol: _ZN2cv3MatC1Ev】

https://github.com/IntelRealSense/realsense-ros/issues/2467

  • 解决方法:

Now im testing this solution but it looks promising. (even work with -DFORCE_RSUSB_BACKEND:=false):

find_package( OpenCV REQUIRED )
include_directories(
include
$realsense2_INCLUDE_DIR
$catkin_INCLUDE_DIRS
$OpenCV_INCLUDE_DIRS
)
target_link_libraries($PROJECT_NAME
$realsense2_LIBRARY
$catkin_LIBRARIES
$CMAKE_THREAD_LIBS_INIT
$OpenCV_LIBRARIES
)

修改后重新编译,加参数 -DFORCE_RSUSB_BACKEND:=false

catkin_make  install -DFORCE_RSUSB_BACKEND:=false
  1. usb_cam ROS package

  • 安装usb 相机驱动程序ROS功能包:

sudo apt-get install ros-melodic-usb-cam
或sudo apt-get install ros-noetic-usb-cam
  • 参数表:

  • launch文件

<launch>
<!--Launch Usb Camera via usb_cam package -->
  <node name="usb_cam" pkg="usb_cam" type="usb_cam_node" output="screen" >
    <param name="video_device" value="/dev/Monocular" />
    <param name="image_width" value="640" />
    <param name="image_height" value="480" />
    <param name="pixel_format" value="yuyv" />
    <param name="camera_frame_id" value="usb_cam" />
    <param name="io_method" value="mmap"/>
  </node>
  <node name="image_view" pkg="image_view" type="image_view" respawn="false" output="screen">
    <remap from="image" to="/usb_cam/image_raw"/>
    <param name="autosize" value="true" />
  </node>
</launch>
  1. RVIZ

  • 数据及图像可视化节点,实时显示目标检测结果和视觉传感器图像

<launch>
<!-- Launch Multi-sensor drivers and filters, with Lidar range sensor and Usb Camera  -->
    <!-- Loading param files  -->
    <rosparam file="$(find multisensor_fusion)/cfg/sensor_cfg.yaml" command="load" />
    <!--<rosparam file="$(find multisensor_fusion)/cfg/ros_pkg_info.yaml" command="load" />-->

    <!--Launch USB 2.0 Camera launch file -->

    <include file="$(find contact_force)/launch/contact_force_launch.launch">
    </include>

    <include file="$(find multisensor_fusion)/launch/usb_camera_launch.launch">
    </include>

   <!-- Launch network state monitor and publisher node -->
    <node
    name="pub_range_sensor"
    pkg="multisensor_fusion"
    type="pub_range_sensor_pub.py" />
    
    <node
    name="pub_actuator_power"
    pkg="multisensor_fusion"
    type="pub_actuator_effort.py" />


    <node
    name="sensor_status"
    pkg="multisensor_fusion"
    type="sensor_status" />

    <!-- Launch rivz display -->
    <!--<node name="rviz" pkg="rviz" type="rviz" args="-d $(find dual_arm_robot_description)/rviz/dual_arm.rviz" output="screen" />-->

</launch>

3.GUI &集成控制终端

  1. 通信架构

  1. 系统功能

以上是关于飞行机器人专栏-- 异构多视角视觉系统的主要内容,如果未能解决你的问题,请参考以下文章

飞行机器人专栏-- AGX Xavier 通信控制及视觉应用开发

飞行机器人--- ZED2& YOLO &ROS第一视角目标检测

飞行机器人--- ZED2& YOLO &ROS第一视角目标检测

在 Django 中建模异构多对多关系的最佳方法是啥?

飞行机器人专栏-- 多传感器固定端口分配

飞行机器人专栏-- 多传感器固定端口分配