Image processing based robot using MATLAB and Simulation using Proteus ISIS

Build a robot using MATLAB

For the beginners, who are attempting to make an image processing based  robots, here comes few steps to guide you through the process. Remember that the robots sees what actually  the programmer wanted it to see by using a camera as its sensor along with proper image processing algorithms. Usually it involves high computation and hence a normal microcontroller would not be enough. So let us use MATLAB for processing the images and an ordinary microcontroller to execute the commands from the MATLAB.

As I always follow the KISS concept(Keep It Simple Stupid), I will first explain how to control the motors from MATLAB using serial communication(RS232). In the explanation, I have used the basic 8051 microcontroller to execute the command from the MATLAB(you can use any controller of your choice). To reduce the time in setting up the hardware assembly of motors and microcontroller, I have interfaced proteus ISIS and MATLAB(since our aim is to check whether proper control commands are sent to the controller based on the image processing algorithm developed). The following video shows how the commands from MATLAB is received by the controller via serial com port and which in turn controls the motors actions,

And now, we know how to control a motor from MATLAB. All we have to do now is to develop a suitable image processing algorithm based on our needs and to introduce the motor command function ‘fwrite()’ at appropriate places.

Image Processing:

Let us consider an image is an MxN matrix. So,processing an image is nothing but manipulating the values in the MxN matrix as per their needs. The entire picture that can be seen from a camera is the Field Of View(FOV) of the camera and the desired region in which the processing has to be done(or a feature to be extracted) is known as the Region Of Interest(ROI). There are lot of resources available in internet to learn image processing, but, after gaining some basic knowledge about image processing, Please do think and try to develop your own algorithm or just try to combine several algorithms and check for your output(Am sure it will be more fun than just implementing an already existing algorithm).

In the following video, I have used the lane detection sample video of MATLAB and applied global thresholding on it. I have marked and extracted the ROI from the entire FOV of the video. Let us assume, this ROI is a few meters ahead of the robot in which obstacles has to be detected. The threshold plot from the label matrix gives the obstacles on the road(search google for label matrix and thresholding). The pink line indicates the preset threshold value for obstacles and when the real time threshold exceeds the preset value, it can be taken as an indication of an obstacle and appropriate motors can be activated.( this is where you have to use the motor commands). Here, the white color is considered as an obstacle and black color indicates obstacle free area.

To know whether the direction of the obstacle, multiple ROIs can be used(left and right ROI).  So, based on the direction of the obstacle, activate the respective motors. The following MATLAB commands will capture the images from the selected camera,

vid=videoinput('winvideo',1, 'YUY2_160x120');

preview(vid);

pause(3);

while(1)

img=getsnapshot(vid);

% Do all image processing and analysis here

end

‘img’ is the image matrix and you can apply all your algorithms to it inside the while loop. And, as usual, when you are aiming for higher accuracy, you may focus more on the computation part and a better control loop for the motors(PID controlling is often used most cases).

Snap shots of the videos are given below,