Programmed a Unitree Go1 robot for autonomous exploration and human detection in an unknown environment using C++, Python, ROS 2, and 3D SLAM.

Overview

Rescue missions in disaster-stricken areas pose significant challenges due to their dangerous and unpredictable nature. The intricacies of these environments make it difficult to send humans for exploration, necessitating the utilization of ground robots or aerial drones. While conventional robots encounter limitations such as maneuvering through tight spaces and overcoming obstacles, quadrupeds offer promising solutions with their superior mobility.

To address this challenge, I programmed a Unitree Go1 robot to autonomously explore unknown environments and search for survivors in disaster zones. Equipped with facial recognition capabilities, the robot can efficiently navigate through confined spaces and hazardous terrain, enhancing the effectiveness of rescue operations.

The project is divided into Three main sections:

Autonomous Exploration and Mapping

Human Detection

Facial Recognition

Setup and Hardware

The robot used in this project is the Unitree Go1. It is equipped with a Robosense 3D Lidar, an Intel® RealSense™ Depth Camera D435i and a NVIDIA® Jetson Orin Nano to handle all the computation.

Autonomous Exploration and Mapping

Mapping

A crucial aspect of the project involves the robot’s autonomous mapping of unfamiliar surroundings, facilitated by the RTAB-Map package. This RGB-D, Stereo, and Lidar Graph-Based SLAM approach utilize an incremental appearance-based loop closure detector.

Key features implemented include:

Sensor Fusion: Fusion of Lidar and camera data allows the robot to effectively map the environment and identify loop closures.

Lidar Point Clouds: Used to detect obstacles and provide a wider field of view.

Infrared Camera Data: From the RealSense camera enables loop closures, enhancing mapping accuracy.

RTAB-Map Integration: The robot employs the RTAB-Map package to fuse Lidar and camera data, creating a comprehensive map of the environment as it explores.

Autonomous Exploration and Navigation

The project centers on the autonomous exploration capability of a robot in unknown environments. This is achieved through a custom exploration algorithm implemented as a ROS 2 node in Python. The algorithm, a variation of the frontier exploration algorithm, seamlessly integrates with the Nav2 stack for path planning and obstacle avoidance. It operates based on Occupancy Grid data of the environment.

Key features include:

Adaptation: The algorithm is designed to be universally applicable to any robot equipped with Occupancy Grid information.

State Machine: Utilizes a state machine for transitioning between IDLE, EXPLORING, and MOVING states, based on the robot’s actions.

Frontier Identification: During exploration, the robot detects frontiers, delineating explored and unexplored areas, by scanning the map.

Computational Optimization: Downsampling of the map and binary erosion operations enhance computational efficiency, obstacle detection accuracy, and navigational precision.

Goal Determination: Evaluates frontier points based on predefined criteria and selects a suitable goal destination for the robot to navigate towards.

Path Planning: Utilizes the Nav2 stack to plan a path towards the selected goal frontier, facilitating autonomous navigation.

Human Detection

Human Detection and classification is done using the YOLOv8 model which is a realtime object detection and segmentation model. The model is trained on the COCO dataset and is able to detect and classify 80 different classes of objects. This was filtered to only detect human beings. Utilizing the depth information from the realsense camera, the Unitree Go1 is able to accurately and reliably determine the position (x,y,z coordinates relative to the robot) of the human beings in the environment. This is visualized as red cubic markers in Rviz2.

Facial Recognition

I utilized the DeepFace Python package, which is a lightweight face recognition and facial attribute analysis framework integrating various cutting-edge models like VGG-Face, Google FaceNet, OpenFace, Facebook DeepFace, DeepID, and ArcFace. After testing, DeepID exhibited the highest accuracy at 94%.

Implemented functionality includes:

Storage: Images of individuals are stored using a store service.

Detection: The detect service compares the stored image with the current image from a RealSense camera. If a match is found, a picture of the person is displayed in Rviz2.

Future Work

Although this project has achieved significant milestones, there are several areas that require further development. These include:

1. Exploration Algorithm Optimization: The current exploration algorithm can be further optimized to enhance its efficiency and robustness. This can be done by adding an information gain metric and an object classifier to provde the robot with more information about the environment and determine exactly which areas it should give priority to explore.
2. Human Detection and Recognition: Currently, the human detection system operates by capturing still frames during exploration, interrupting the process to capture, process, and return the position of detected humans. This method was adopted due to the inability to process video streams in real-time accurately while running RTAB Map and the Nav2 stack simultaneously. The position of the human detected was displayed inaccurately using this method and hence was not used in the final implementation. For real-world scenarios, an efficient solution would entail real-time human detection and classification, along with determining their position and distance while the robot moves around. One viable approach would involve downsizing the model and deploying it on a more powerful system.
3. Facial Recognition: Although I successfully developed a package for accurate face storage and recognition, I aimed to integrate facial recognition into the entire pipeline. This enhancement would allow the robot to store the face of a specific person it intends to find and search for that individual within an unknown environment. Such a feature would be invaluable in scenarios like searching for specific individuals in disaster zones. Unfortunately, due to time constraints, I couldn’t implement this feature.