原本的Fast-Drone-250项目是基于x86平台实现的。将其迁移到Jetson orin nx上面临着两个问题，一是arm架构与x86架构的代码兼容性问题，二是nx的cpu算力不足，需要部署vins-gpu。

本文Jetson orin nx使用的是jetpack 5.1.4，Ubuntu20.04，ROS noetic。项目完全从一个刚刷上的全新系统从0开始部署。

详细的修改方法：【实机飞行！】在Jetson Orin NX上部署Fast-Drone-250进行实机飞行:https://blog.csdn.net/weixin_66375409/article/details/156237705?spm=1001.2014.3001.5501

1. 配置CUDA环境

2.1安装jtop

sudo apt-get update
sudo apt-get install python3-pip
sudo pip3 install -U jetson-stats

查看机载电脑的当前环境

jetson_release

1.2 安装与配置CUDA与CUDNN

1.2.1 安装CUDA与CUDNN

安装cuda11.4与cudnn

sudo apt-get update
sudo apt-get install cuda-toolkit-11-4
sudo apt-get install libcudnn8

软连接

export PATH=/usr/local/cuda-11.4/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH
source ~/.bashrc

验证

nvcc -V

1.2.2 配置CUDA

下载功能包并设置cuda

sudo apt-get install -y cmake libavcodec-dev libavformat-dev libavutil-dev \
    libglew-dev libgtk2.0-dev libgtk-3-dev libjpeg-dev libpng-dev libpostproc-dev \
    libswscale-dev libtbb-dev libtiff5-dev libv4l-dev libxvidcore-dev \
    libx264-dev qt5-default zlib1g-dev libgl1 libglvnd-dev pkg-config \
    libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev mesa-utils     
 
sudo apt-get install python2.7-dev python3-dev python-numpy python3-numpy
# To fix OpenGL related compilation problems 
 
cd /usr/lib/aarch64-linux-gnu/
sudo gedit /usr/local/cuda/include/cuda_gl_interop.h

注释cuda_gl_interop.h 62~68行的下列部分，保留#include <GL/gl.h>

//#if defined(__arm__) || defined(__aarch64__)
//#ifndef GL_VERSION
//#error Please include the appropriate gl headers before including cuda_gl_interop.h
//#endif
//#else
 #include <GL/gl.h>
//#endif

2. 安装并配置OpenCV

2.1 卸载原有的opencv 4.2.0

cd ~/Downloads
sudo apt-get purge libopencv*
sudo apt autoremove
sudo apt-get update

【此方法可能会误删一些ROS rqt之类的文件，建议后面鱼香ROS重新补充安装一下】

2.2 安装opencv3.4.18和opencv_contrib3.4.18

OpenCV3.4.18的文件已经存放到项目目录下：

OpenCV3.4.18项目地址:https://github.com/opencv/opencv/tree/3.4.18

OpenCV_contrib3.4.18项目地址：https://github.com/opencv/opencv_contrib/tree/3.4.18

在文件夹中编译安装

cd ~/Fast-Drone-250-NX/opencv-3.4.18
mkdir build && cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
        -D CMAKE_INSTALL_PREFIX=/usr/local \
        -D WITH_CUDA=ON \
        -D CUDA_ARCH_BIN=7.2 \
        -D CUDA_ARCH_PTX="" \
        -D ENABLE_FAST_MATH=ON \
        -D CUDA_FAST_MATH=ON \
        -D WITH_CUBLAS=ON \
        -D WITH_LIBV4L=ON \
        -D WITH_GSTREAMER=ON \
        -D WITH_GSTREAMER_0_10=OFF \
        -D WITH_QT=ON \
        -D WITH_OPENGL=ON \
        -D CUDA_NVCC_FLAGS="--expt-relaxed-constexpr" \
        -D WITH_TBB=ON \
        ../
make -j8    
sudo make install

验证opencv与cuda的安装

jetson_release

2.3 安装eigen3

sudo apt-get remove libeigen3-dev 
cd ~/Downloads/
wget -O eigen.zip https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.zip #check version
unzip eigen.zip
mkdir eigen-build && cd eigen-build
cmake ../eigen-3.3.7/ && sudo make install
pkg-config --modversion eigen3 # 检查 Eigen 版本，判断是否安装成功

3. ROS与实验软件的安装

3.1 ROS的安装

wget http://fishros.com/install -O fishros && . fishros

详见：ros安装（一键最简安装，吹爆鱼香ROS，请叫我鱼吹）

3.2 补充nvidia源

添加源

sudo bash -c 'echo "deb https://repo.download.nvidia.cn/jetson/common r35.6 main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list'
sudo bash -c 'echo "deb https://repo.download.nvidia.cn/jetson/t234 r35.6 main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list'
sudo bash -c 'echo "deb https://repo.download.nvidia.cn/jetson/ffmpeg r35.6 main" >> /etc/apt/sources.list.d/nvidia-l4t-apt-source.list'
sudo apt update

3.3 实验软件的安装

sudo apt install terminator
sudo apt install openssh-server
sudo apt install ros-noetic-plotjuggler
sudo apt install ros-noetic-plotjuggler-ros

3.4 VScode的安装

步骤参考：ROS学习笔记：VScode下ROS环境的配置

4. 安装ceres与glog

进入项目文件夹中，解压3rd_party.zip压缩包

4.1 安装glog

进入glog文件夹打开终端，给予可执行权限

chmod +x autogen.sh
chmod +x configure

执行脚本

./autogen.sh && ./configure && make && sudo make install
sudo apt-get install liblapack-dev libsuitesparse-dev libcxsparse3 libgflags-dev libgoogle-glog-dev libgtest-dev

4.2 ceres1.14编译安装

进入ceres文件夹打开终端，创建一个build文件夹并进入

mkdir build
cd build

编译安装

cmake ..
sudo make -j8
sudo make install
sudo apt-get install ros-noetic-ddynamic-reconfigure

5. 配置VINS-GPU环境

5.1 修改cv_bridge

进入opt/ros/melodic/share/cv_brige_cmake，更改cv_brigeConfige.cmake

cd /opt/ros/noetic/share/cv_bridge/cmake
sudo gedit cv_bridgeConfig.cmake

修改line 94-118，直接复制覆盖！

#if(NOT "include;/usr/include/opencv4 " STREQUAL " ")
#  set(cv_bridge_INCLUDE_DIRS "")
#  set(_include_dirs "include;/usr/include/opencv4")
set(libraries "cv_bridge;/usr/local/lib/libopencv_core.so.3.4.18;/usr/local/lib/libopencv_imgproc.so.3.4.18;/usr/local/lib/libopencv_imgcodecs.so.3.4.18") 
  if(NOT "https://github.com/ros-perception/vision_opencv/issues " STREQUAL " ")
    set(_report "Check the issue tracker 'https://github.com/ros-perception/vision_opencv/issues' and consider creating a ticket if the problem has not been reported yet.")
  elseif(NOT "http://www.ros.org/wiki/cv_bridge " STREQUAL " ")
    set(_report "Check the website 'http://www.ros.org/wiki/cv_bridge' for information and consider reporting the problem.")
  else()
    set(_report "Report the problem to the maintainer 'Vincent Rabaud <vincent.rabaud@gmail.com>' and request to fix the problem.")
  endif()
  foreach(idir ${_include_dirs})
    if(IS_ABSOLUTE ${idir} AND IS_DIRECTORY ${idir})
      set(include ${idir})
    elseif("${idir} " STREQUAL "include ")
      get_filename_component(include "${cv_bridge_DIR}/../../../include" ABSOLUTE)
      if(NOT IS_DIRECTORY ${include})
        message(FATAL_ERROR "Project 'cv_bridge' specifies '${idir}' as an include dir, which is not found.  It does not exist in '${include}'.  ${_report}")
      endif()
    else()
      message(FATAL_ERROR "Project 'cv_bridge' specifies '${idir}' as an include dir, which is not found.  It does neither exist as an absolute directory nor in '\${prefix}/${idir}'.  ${_report}")
    endif()
    _list_append_unique(cv_bridge_INCLUDE_DIRS ${include})
  endforeach()
#endif()

5.2 安装Mavros

这个地方需要等待较长时间

sudo apt-get install ros-noetic-mavros
cd /opt/ros/noetic/lib/mavros
sudo ./install_geographiclib_datasets.sh

5.3 编辑px4.launch

roscd mavros
cd launch
sudo gedit px4.launch

找到波特率，将波特率从"57600"改成"921600"

6. 配置Realsense环境

6.1 安装realsenseSDK

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key  F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE 
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key  F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE
sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key FB0B24895113F120
sudo apt-get install librealsense2-dkms
sudo apt-get install librealsense2-utils
sudo apt-get install librealsense2-dev
sudo apt-get install librealsense2-dbg

先下载源码

git clone https://github.com/jetsonhacks/installRealSenseSDK.git

进入安装包开始安装

cd ./installRealSenseSDK
./installLibrealsense.sh

将Realsense摄像头（D435i）插入USB3.0的接口，验证

realsense-viewer

6.2 安装realsenseROS

在终端输入

sudo apt-get install ros-noetic-realsense2-camera

7. 编译与运行ego_planner

7.1 编译Fast-Drone-250项目

进入工作空间中，编译项目

cd Fast-Drone-250-NX
catkin_make
source devel/setup.bash

7.2 运行ego_planner仿真

roslaunch ego_planner single_run_in_sim.launch

在Rviz内按下键盘G键，再单击鼠标左键以点选无人机目标点

7.3 实机飞行

7.3.1 检查飞控链接并赋予权限

查看端口

ls /dev/tty*

查看是否有ttyACM0这个端口，并赋予权限

sudo chmod 777 /dev/ttyACM0

7.3.2 检查IMU频率

打开一个终端，进入工作空间中

cd Fast-Drone-250-NX
roslaunch mavros px4.launch

再打开一个终端，查看频率是否再200Hz左右

rostopic hz /mavros/imu/data_raw

7.3.3 建图模块验证

打开一个终端，进入工作空间，运行脚本

cd Fast-Drone-250-NX
sh shfiles/rspx4.sh

再打开一个终端，运行ego

roslaunch ego_planner single_run_in_exp.launch

再打开一个终端，打开rviz可视化界面

roslaunch ego_planner rviz.launch

7.3.4 检查realsense驱动正常

打开一个终端，运行realsense相机

roslaunch realsense2_camera rs_camera.launch

再打开一个新的终端，查看图像

rqt_image_view

查看/camera/infra1/image_rect_raw,/camera/infra2/image_rect_raw,/camera/depth/image_rect_raw话题正常

7.3.5 VINS内参标定

打开一个终端，运行realsense相机

roslaunch realsense2_camera rs_camera.launch

打开一个新的终端，打印信息

rostopic echo /camera/infra1/camera_info

进入Fast-Drone-250-NX\src\realflight_modules\vins_gpu\src\VINS-Fusion-gpu\config，把终端中的K矩阵中的fx,fy,cx,cy填入left.yaml和right.yaml 例如我这里是D435i相机，则为

projection_parameters:
   fx: 387.8890075683594
   fy: 387.8890075683594
   cx: 318.77215576171875
   cy: 238.25390625

7.3.6 VINS外参标定

在主目录中新建一个文件夹vins_output。

修改配置文件，打开"Fast-Drone-250-NX\src\realflight_modules\vins_gpu\src\VINS-Fusion-gpu\config\fast_drone_250.yaml"。

cd Fast-Drone-250-NX\src\realflight_modules\vins_gpu\src\VINS-Fusion-gpu\config
gedit fast_drone_250.yaml

在line 16将其中的output_path改成你自己的路径：

output_path: "~/vins_output"

打开一个终端，运行脚本

cd Fast-Drone-250-NX
sh shfiles/rspx4.sh

再打开一个新的终端

rostopic echo /vins_fusion/imu_propagate

拿起飞机沿着场地尽量缓慢地行走，场地内光照变化不要太大，灯光不要太暗，不要使用会频闪的光源，尽量多放些杂物来增加VINS用于匹配的特征点。把vins_output/extrinsic_parameter.txt里的内容替换到fast-drone-250.yaml的body_T_cam0和body_T_cam1

重复上述操作直到走几圈后VINS的里程计数据偏差收敛到满意值（一般在0.3米内）

7.3.7 实机参数调整

调整Fast-Drone-250-NX/src/realflight_modules/px4ctrl/config/ctrl_param_fpv.yaml下的：

mass：修改为无人机的实际重量
hover_percent：修改为无人机的悬停油门，可以通过px4log查看，具体可以参考文档如果你的无人机是和课程的一模一样的话，这项保持为0.3即可。如果更改了动力配置，或重量发生变化，或轴距发生变化，都请调整此项，否则自动起飞时会发生无法起飞或者超调严重的情况。
gain/Kp,Kv：即PID中的PI项，一般不用太大改动。如果发生超调，请适当调小。如果无人机响应较慢，请适当调大。
rc_reverse：这项使用乐迪AT9S的不用管。如果在自动起飞中，发现飞机的飞行方向与摇杆方向相反，说明需要修改此项，把相反的通道对应的值改为true。其中throttle如果反了，实际实验中会比较危险，建议在起飞前就确认好，步骤为：
- roslaunch mavros px4.launch
- rostopic echo /mavros/rc/in
- 打开遥控器，把遥控器油门从最低满满打到最高
- 看echo出来的消息里哪项在缓慢变化（这项就是油门通道值），并观察它是不是由小变大
- 如果是由小变大，则不需要修改throttle的rc_reverse，反之改为true
- 其他通道同理
  7.3.8 Vins悬停遥控飞行
  进入到工作空间中

cd Fast-Drone-250-NX

启动Vins

sh shfiles/rspx4.sh

打印vins数据，摇晃无人机查看Vins的位置情况

rostopic echo /vins_fusion/imu_propagate

遥控器5通道拨到内侧，六通道拨到下侧，油门打到中位打开一个新的终端

roslaunch px4ctrl run_ctrl.launch

起飞！

sh shfiles/takeoff.sh

7.3.9 ego_planner飞行实验

进入到工作空间中

cd Fast-Drone-250-NX