Wednesday, 4 June 2014

Image Processing in Android : A kick start to Augmented Reality

I never thought that I would be developing Android Applications a few months back. I sincerely thank Dr. Pinnamaneni Bhanu Prasad for introducing me into this interesting domain and guiding me through the process of step by step development of Augmented Reality applications.

For those who have experience in image processing at a computer level interfaces, it would be very exciting if you see your algorithm running in your own android mobile ( I have experienced this on my very first app, and surely you`ll!).  Moreover, you can take it along with you anywhere and you can demonstrate your algorithm to all your friends/family in any android mobile (which you cannot do with the computer level algorithm you used to develop previously, no matter how complex the algorithm is!).

So, I have decided to write some basic steps that can guide a newbie (new to android but with good programming skills) to develop their own android application for image processing. So, if you like to develop an app, all you need is INTEREST and PATIENCE and all it gives in return is FUN!(if you are a curious techie like me). I have provided some links which would be very useful for anyone to start over ( these  links really helped me a lot than other tutorials on the internet...just my opinion!, you can find your own link that suits you well)

The very first thing you need is an IDE with all libraries and tools of android. It is good to start with the  Android Developer portal, where you can find tutorials about android and setting up any IDE (Eclipse, in my case) for android programming. But from my experience, manual setup of android tools/OpenCV to an IDE is very complicated and you may experience lot of compile errors even with minor directory changes. So, I strongly recommend you to use the NVIDIA Tegra Android Development Pack which will install all the necessary libraries along with OpenCV, thus making your work simpler. Install the Tegra android pack and go through the basic tutorial " Build your first App" in Android developer portal. This will give you a clear idea on various files/folders and its functionality in android development.

Once you are clear with the basics of the android development, different files used in it, accessing various sensors/hardware components, you can start looking after the example codes first and then you`ll be able to develop your own code upon analyzing the example codes.

This is a good link with clear explanation of all required files to build a basic android app. After going through that tutorial, you can start with the image processing in android. To do this, click on the file menu, choose import and then choose existing android application. Now navigate to OpenCV-Android sample programs that you have in your computer (click here to see how to do).

One must always keep in mind  that the image processing methods are same in both android and windows platform. All you need to take care is, the input image and the output image format. The Android camera`s format is RGBA, which means the image what you will be acquiring will be in RGBA format and you need to return the image of same format after doing your processing.

1. Acquire RGBA image
2. Convert it into required format (RGB/Ycrcb/Grayscale etc)
3. Do all your image processing analysis as you would do in OpenCV
4. Convert the output image into RGBA
5. Return the output image to display

These are all the basic steps that one should always keep in mind while programming. Here is a video of my first android application in which I have implemented the following,

1. Thresholding (60/255)
2. Skin Detection and contour drawing over detected skin
3. Canny edge detection
4. Circle detection

I have used a simple switch case to toggle between these four processing when a menu in the android is selected. Which means, all the processing and logics are same for both windows and Android. All you need to do is to think that 'it is easy to work on', spent some time in understanding the basics and eventually you will succeed.

 Once you know how it is done, then you can proceed with some advanced techniques and can do high level  image processing with your own android mobile. And with further interest and patience, you can go through some tutorials about Augmented Reality and can start developing your own Augmented reality apps (as I am doing right now). Have FUN! and feel free to contact me for further assistance.

Object Recognition in MATLAB

Recently, I was working in a conveyor belt setup, where, a camera was used to recognize the objects in the belt and an actuator to eject the unwanted objects. As I always like to simulate the algorithm before an actual implementation, I have used MATLAB for image recognition and based on the object classification made by MATLAB, the actuator will be actuated (to eject unwanted objects).

For simulation purpose, I have established serial communication between MATLAB and Proteus ISIS ( where I can have microcontroller, LEDs, Actuators etc). I have used a microcontroller of my choice and enabled its serial receive pin, so that it can receive the data that MATLAB transmits. MATLAB recognizes a known object by matching its SURF features and serially transmit a unique character/integer for each object. The microcontroller is coded in such a way that, if a specific character/integer is found in the receive buffer, then the actuator will be actuated.

Below shown is a simulated video of a conveyor belt setup. The video was taken using a 2MP cellphone camera. LEDs were interfaced with the microcontroller in Proteus ISIS to check the image recognition algorithm developed in MATLAB ( Every object has its own LED interfaced). Microcontroller coding was done in Keil and the .hex file is dumped into the simulated microcontroller in Proteus ISIS.

The detection algorithm was a bit slow in the video used. However, with the use of Bluecougar-X-X120aG GigE camera, improvement in detection speed was observed. In addition to this, I have also tested the similar object recognition algorithm in OpenCV. By my experience in working with both MATLAB and OpenCV, the detection speed is remarkably high in OpenCV when compared to MATLAB