Wednesday, 4 June 2014

Image Processing in Android : A kick start to Augmented Reality

I never thought that I would be developing Android Applications a few months back. I sincerely thank Dr. Pinnamaneni Bhanu Prasad for introducing me into this interesting domain and guiding me through the process of step by step development of Augmented Reality applications.

For those who have experience in image processing at a computer level interfaces, it would be very exciting if you see your algorithm running in your own android mobile ( I have experienced this on my very first app, and surely you`ll!).  Moreover, you can take it along with you anywhere and you can demonstrate your algorithm to all your friends/family in any android mobile (which you cannot do with the computer level algorithm you used to develop previously, no matter how complex the algorithm is!).

So, I have decided to write some basic steps that can guide a newbie (new to android but with good programming skills) to develop their own android application for image processing. So, if you like to develop an app, all you need is INTEREST and PATIENCE and all it gives in return is FUN!(if you are a curious techie like me). I have provided some links which would be very useful for anyone to start over ( these  links really helped me a lot than other tutorials on the internet...just my opinion!, you can find your own link that suits you well)

The very first thing you need is an IDE with all libraries and tools of android. It is good to start with the  Android Developer portal, where you can find tutorials about android and setting up any IDE (Eclipse, in my case) for android programming. But from my experience, manual setup of android tools/OpenCV to an IDE is very complicated and you may experience lot of compile errors even with minor directory changes. So, I strongly recommend you to use the NVIDIA Tegra Android Development Pack which will install all the necessary libraries along with OpenCV, thus making your work simpler. Install the Tegra android pack and go through the basic tutorial " Build your first App" in Android developer portal. This will give you a clear idea on various files/folders and its functionality in android development.

Once you are clear with the basics of the android development, different files used in it, accessing various sensors/hardware components, you can start looking after the example codes first and then you`ll be able to develop your own code upon analyzing the example codes.

This is a good link with clear explanation of all required files to build a basic android app. After going through that tutorial, you can start with the image processing in android. To do this, click on the file menu, choose import and then choose existing android application. Now navigate to OpenCV-Android sample programs that you have in your computer (click here to see how to do).

One must always keep in mind  that the image processing methods are same in both android and windows platform. All you need to take care is, the input image and the output image format. The Android camera`s format is RGBA, which means the image what you will be acquiring will be in RGBA format and you need to return the image of same format after doing your processing.

1. Acquire RGBA image
2. Convert it into required format (RGB/Ycrcb/Grayscale etc)
3. Do all your image processing analysis as you would do in OpenCV
4. Convert the output image into RGBA
5. Return the output image to display

These are all the basic steps that one should always keep in mind while programming. Here is a video of my first android application in which I have implemented the following,

1. Thresholding (60/255)
2. Skin Detection and contour drawing over detected skin
3. Canny edge detection
4. Circle detection

I have used a simple switch case to toggle between these four processing when a menu in the android is selected. Which means, all the processing and logics are same for both windows and Android. All you need to do is to think that 'it is easy to work on', spent some time in understanding the basics and eventually you will succeed.

 Once you know how it is done, then you can proceed with some advanced techniques and can do high level  image processing with your own android mobile. And with further interest and patience, you can go through some tutorials about Augmented Reality and can start developing your own Augmented reality apps (as I am doing right now). Have FUN! and feel free to contact me for further assistance.


  1. hello bro , i'm a biginner in android from Tunisia , i request you to help me out in the basic thresholding part . can you give me a link to a goog tuto ? thx

    1. Hi,

      Have a look at the following link to know about basic thresholding and its C++ coding,

  2. hi,
    please send me the source code for this app.. if possible the apk file too..
    its for my project work...

  3. Hi,
    Sir I am a student of final year engineering from Pune,India. I saw u r video and read this full blog will you please send me this app of image processing by mobile phone i need it for my Final Year Project.
    I am doing project for virtual keyboard and for it i need the image processing via mobile phone.
    My Mail id
    Waiting for your positive reply

  4. Does anybody know how can we display image from 2d-array in android studio ?