Sensor Products for Enterprises Creating Technological Opportunities in Remote Sensing (SPECTORS) is a four-year-long innovation program with 20 Dutch and German small-and-medium-sized enterprises. The goal of the project is to exploit the market potential of civil drone technology through sensor innovations for remote sensing and remote monitoring. The project's website is located at https://www.spectors.eu.
Role of RaM in the project
The main role of RAM is the technological development of an aerial robot capable of establishing a robust mechanical surface contact for remote inspection.
Drones are already currently used for visual inspection of many infrastructures in the civil, petro-chemical, off-shore, and other industries. This allows the observation of remotely located structures under test without the need to reach them in person. Many of these remote sensing applications are operational and economically successful.
However, some inspections require physical contact with the surfaces to apply non-destructive testing (NDT). Such sensing modalities are for example used to measure the thickness of coatings, quality of the surface, and other physical states which cannot be measured remotely or not accurately enough.
Sensors that are used can be based on ultrasound, eddy currents, induction, or other methodologies, but they all require physical contact with the surface. Investigators need to physically reach the remote locations of interest by building scaffold or with cranes and these approaches are costly and dangerous. The possibility to get in physical contact with the remote surface via a drone has an incredible market value and would drastically reduce costs and increase safety.
Framework
The inspection tasks that the aerial robot will perform are subject to interaction with the environment. Interaction is characterized by the exchange of energy between the robot and the environment. Thus to control the interaction means to control the dynamic relation between the force and velocity of the end-effector not just one of the two. The interaction problem is addressed in the context of the port-Hamiltonian framework.
Research activities
Study and review of the literature on fully-actuated multi-rotor UAVs
The first meaningful step to carry out our research has been a deep study and review of the literature on aerial physical interaction, in particular in relation to the novel state-of-the-art fully-actuated multi-rotor UAV designs, which have been demonstrated more effective means in this field compared to traditional solutions. As a matter of fact, conventional multi-rotor UAV designs, epitomized by the well-known quadrotor, are optimized for maximum flight-time. The propellers of all these vehicles have parallel directions to collectively counteract gravity. Consequently, these robots have an under-actuated dynamics, which causes a tight coupling between their translational and rotational motion, namely the need to re-orient themselves in order to produce a lateral displacement. On the other hand, novel fully-actuated multi-rotor UAV designs are instead optimized to generate forces and moments in all the directions of their configuration space without the need to change their orientation, which is desirable for aerial interaction applications. This is accomplished by arranging the propellers in particular (typically non-parallel) configurations, either in a fixed or in an actively-tilting fashion. We described the most relevant designs in a survey paper.
Comparison of an under-actuated (left) and a fully-actuated (right) multi-rotor UAV designs. Credits of the picture: Ph.D. defense of D. Bicego
Scientific outcomes
Design and implementation of a fully-actuated multi-rotor UAV
Once prepared an adequate mathematical description of the robot dynamics based on the physics, and after a preliminary phase of extensive simulations, we proceeded with the design and the physical implementation of the real aerial platform. The chosen configuration is a hexarotor with the six propellers tilted, in an alternated fashion, in a fixed way around the frame arms of an angle of 30 deg. Our fully-actuated aerial robot was developed in-house based on an off-the-shelf carbon fiber frame with a diameter (rotor hub to rotor hub) of 0.68 m and a nominal total mass less than 2 kg, without batteries. For the generation of thrust, the system uses 11-inch propellers driven by six CM-2217 brushless motors with DYS SN40A electronic speed controllers. Each rotor has the capability to produce a maximum thrust of 12.5 N.
The vehicle is equipped with a Pixhawk flight controller with integrated sensors. The Pixhawk runs the PX4 software, which handles interfacing to the sensors and actuators in addition to providing a modular framework which allows adding new control schemes. We augmented the original PX4 software with custom-made modules that enables the control of fully-actuated multi-rotors. This involves modifying the PX4 control allocation module and the low-level orientation controller.
To see some videos of the fully-actuated hexarotor designed and built at the University of Twente, refer to video1 and video2.
Photos of the fully-actuated hexarotor designed and built at the University of Twente.
Developed control algorithms for aerial physical interaction
Once a precise model of the robot dynamics is available, a basic requirement to achieve stable interaction is the design of a proper control algorithm. We approached the control problem of fully actuated UAVs in a geometric port-Hamiltonian framework. The UAV was modeled as a floating rigid body on the special Euclidean group SE(3), while a unified near-hovering motion and impedance controller was derived exploiting the energy-balancing passivity-based control technique. We carried out a deep analysis of the closed-loop system’s behavior for both the free-flight stability and the contact stability of the UAV. Furthermore, we validated the robustness of the control system to uncertainties with several realistic simulations and real experiments, in which the UAV was driven near its actuator limits. The experiments showed the ability of the UAV, controlled with the proposed algorithm, to hover at its maximum allowed orientation angles and apply its maximum allowed lateral force normal to a planar surface whose geometric description was available, without the input saturation destabilizing the system.
Snapshot of the Gazebo simulation (left) and photo of the real experiment (right) with our fully-actuated hexarotor while interacting with a known planar surface.
Moreover, we employed a momentum-based wrench observer to estimate the interaction wrench without the use of a force/torque sensor, in order to avoid additional payloads. Furthermore, the concept of energy tanks was also used on top of the energy-based impedance controller to guarantee the system’s overall contact stability to arbitrary passive environments.
Schematic representation of the impedance controller designed to achieve stable physical interaction.
Scientific outcomes
Challenging application of interaction: aerial writing
In order to further test the effectiveness and the robustness of our interaction controller in another challenging scenario, we made our fully-actuated hexarotor draw our research group logo on a white-board, thanks to a marker pen attached to the robot end-effector. To accomplish this task, the robot needs to precisely regulate the wrench generated on the surface. The nice results achieved in this experiment can be appreciated in video3.
Snapshot of the aerial writing experiment.
Combining AI and interaction
To achieve consistent performance during impedance control tasks, an a-priori knowledge of the environment parameters is needed to adjust the controller’s impedance parameters accordingly. In order to relieve the user from the tedious tuning of the controller parameters, and motivated by the wish to automatize our system, we tackled this problem. Concentrating on tasks requiring constant impedance parameters throughout the operation, we designed a model-free learning framework to autonomously find the suitable parameter values. The framework relies on Bayesian optimization and episodic reward calculation requiring the UAV to repeatedly perform a predetermined task in the environment actively searching in the impedance parameter space.
Schematic representation of the impedance controller designed to achieve stable physical interaction.
Scientific outcomes
-
Conference paper at 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) → Best paper awards
Combining computer vision and interaction
A fundamental step for autonomous interactive aerial robots is to be able to perceive the environment they will interact with. Achieving consistent and repeatable interaction behavior requires prior knowledge of the environment’s geometric properties, which is not suitable for autonomous robots in unstructured scenarios. A solution for that is combining computer vision and machine learning techniques with interaction control techniques.
In this regard, we introduced a novel UAV navigation method for 3D reconstruction of the target structures when an a-priori knowledge of the object shape, size, and complexity are not available. The proposed method relies on the use of a stereo camera mounted on the UAV for real-time point cloud reconstruction and localization. The real-time point cloud is analyzed in a software loop where the entropy of the point cloud and the surface normals are calculated. The low entropy positions (which indicate the 3D areas with less point density and less information) and the surface normals are used for calculating the next inspection point which can be targeted by the drone in order to enhance the point cloud at best. Path planning through these automatically selected target points is done during the flight almost in real-time. Preliminary validation of such an algorithm was performed with Gazebo simulations within ROS, using realistic robot and camera parameters.
Snapshots of the Gazebo simulation showing the automatic reconstruction of the object of interest, whose dimensions and cylindric shape is not known a-priori by the planner.
Again in the perspective of tackling the problem of contact-based inspection of a target of unknown geometry and pose, by exploiting state of the art techniques in computer graphics, tuned and improved for the task at hand, we designed a framework for the projection of a desired trajectory for the robot end-effector, initially specified by the user in 2D for convenience, on a generic 3D surface to be inspected. Combining these results with the aforementioned achieved works on energy-based interaction control and object reconstruction, we proposed a vision-based impedance control paradigm for the contact inspection of unknown surfaces. To demonstrate the feasibility and the effectiveness of our methodology, we presented the results of both realistic ROS/Gazebo simulations and preliminary experiments with a fully-actuated hexarotor interacting with heterogeneous curved surfaces whose geometric description is not available a priori. A video describing our methodology and presenting the results achieved so far is available on video4.
Graphical description of our vision-based interaction planner. Left: schematic visualization of the projection idea. Right: photo of our fully-actuated aerial robot scanning a curved unknown surface prior to interaction.
Outcomes of projection algorithm, tested with different trajectories and heterogeneous surfaces.
The aerial robot and the target object in the simulated environment in Gazebo (right) and a snapshot of the successful execution of the vision-based interaction planner obtained using RVIZ (right).
Scientific outcomes
-
Conference paper at 2020 IEEE International Conference on Intelligent Robots and Systems (link yet to come)
Join us for a student internship!
We are always looking for enthusiastic motivated roboticists (BSc or MSc) willing to join the aerial robotics community. Interested students can contact Davide Bicego (d.bicego@utwente.nl) or Ramy Hashem (r.a.m.rashadhashem@utwente.nl) for available assignments.
