This is an article related to the approaches for visual servo control of a quadrotor helicopter ( UAV - unmanned aerial vehicle ) and especially for retrieving 3D information from 2D image. This is a part from my second thesis for my Master Degree in Control Systems. It is developed to collaborate to a PhD thesis of another student.
Position, altitude and orientation measurements are usually sufficient for the control of UAVs operating at high altitudes. Therefore, conventional avionics that include GPS and IMU provide the required information for flight control and waypoint navigation. To achieve realistic missions in such complex environments, the flight controller requires precise estimation of both the vehicle pose and its surrounding environment. Among many sensors for environment mapping and obstacles detection, ultrasonic sensors, Laser Range Finder (LRF), radar and vision have distinct advantages when applied to UAVs. We are however, interested in enabling UAVs to achieve advanced flight behaviors in cluttered environments. The limited payload of these small platforms precludes the use of conventional and standard sensors. Therefore, small cameras, which are available today at low-cost, are attractive sensors for mini UAVs flying near the ground in hazardous environments. Indeed, vision provides a viable and useful solution to sensing on UAVs because cameras are too light to fit the limited payload capabilities of MAVs. Furthermore, visual sensors are passive and contain rich information about the vehicle motion and the environment structure. Unlike GPS, which does not work in the shadow of satellite visibility, vision works in a cluttered urban environment or even in indoor spaces.
The basic scene model will be presented briefly:
Quadrotor control system |
The control system is quite complex. The original manufactured drone is equipped with a remote radio control. In this project it is revised so that it can be connected to a PC equipped with a real-time ADC and DAC system and real-time software. Moreover, the control law is defined to the PC and all the calculations are happening to the PC. As the computer vision algorithm is hard to work in real-time, a separate PC is used for this reason. The computer vision result is used as a reference to the control system. Well, it is mix of two systems as solely one is working in real-time. In the shown scene it is used just a small amount of the advantages of the presented algorithm, because it is used to implement so called PVTOL ( Planar Vertical Take-Off and Landing ) - or in other words - a flight in a vertical plane. The aim is in flight time the color object to be tracked and the quadrotor to be posed above it.
My purpose is to present you all the capabilities of the computer vision algorithm.
Retrieving 3D information from 2D image
Well, we have a wireless camera looking down on the board of the quadrotor. We use the receiver and a USB TV tuner to acquire the signal and to import it to Matlab. The main idea is to retrieve 3D info from the image, so we chose a simple color-based method for the segmentation and tracking purpose because the stress is not in the tracking. It is essential the color of the object to be specific for the scene and there are no other object with the same color. Some words for the color-based segmentation:
- We use color filtering via thresholding in the HSV color space.
HSV color space |
The image contain a raw RGB information, however it is hard to be done a color segmentation in the RGB color space because if we threshold the R, G and B values we can extract another color with RGB values in the same range. So we need to transform the RGB color space to HSV color space. So, we need to compute the H (Hue), S (Saturation) and V (Value) values using the following equations:
This algorithm is applied to every pixel in the image. It is worthy to be mentioned, that in Matlab there is a ready to use function for RGB to HSV transformation. So, after we do the transformation and HSV thresholding we receive a binary image with a white blob for the object we were searching for. The H, S and V values should be defined and assigned to the code apriori. As the RGB image is represented like a 3 dimensional array, the same kind of array is used for representing the HSV array.
RGB 3D array |
- 3D info retrieving from 2D image
The next step it is needed to be done is to analyze the blob in the binary image. So we find the area of the blob in the image so that later we can use it to estimate the distance to the object which is related to the area.
Then we need to estimate the geometry center of the blob which can be represented by the following figure:
The geometry center of the blob |
We estimate its coordinates as an average value of the "x" and "y" coordinates of all of the pixels with value 1.
As a result of an identification we use some calculations to estimate the distance to the object. For this reason we need to do an identification of the camera and to define the view angle which will be presented later.
This approach has some issues. For example if the quadrotor is rotated and the plane of the camera is not in parallel of the scene plane, the estimation will be wrong. So there are some assumptions (for example, the cut image size and the same visible part of the scene in both of the axis as a result) which are mentioned in the next video presenting my project for the Mathworks Simulink Student Challenge 2011.
Here is the link of all of the competitives:
http://www.youtube.com/playlist?list=PLFBE7DBBD466439DB
Няма коментари:
Публикуване на коментар