Image Processing in Android : A kick start to Augmented Reality

I never thought that I would be developing Android Applications a few months back. I sincerely thank Dr. Pinnamaneni Bhanu Prasad for introducing me into this interesting domain and guiding me through the process of step by step development of Augmented Reality applications.

For those who have experience in image processing at a computer level interfaces, it would be very exciting if you see your algorithm running in your own android mobile ( I have experienced this on my very first app, and surely you`ll!).  Moreover, you can take it along with you anywhere and you can demonstrate your algorithm to all your friends/family in any android mobile (which you cannot do with the computer level algorithm you used to develop previously, no matter how complex the algorithm is!).

So, I have decided to write some basic steps that can guide a newbie (new to android but with good programming skills) to develop their own android application for image processing. So, if you like to develop an app, all you need is INTEREST and PATIENCE and all it gives in return is FUN!(if you are a curious techie like me). I have provided some links which would be very useful for anyone to start over ( these  links really helped me a lot than other tutorials on the internet...just my opinion!, you can find your own link that suits you well)

The very first thing you need is an IDE with all libraries and tools of android. It is good to start with the  Android Developer portal, where you can find tutorials about android and setting up any IDE (Eclipse, in my case) for android programming. But from my experience, manual setup of android tools/OpenCV to an IDE is very complicated and you may experience lot of compile errors even with minor directory changes. So, I strongly recommend you to use the NVIDIA Tegra Android Development Pack which will install all the necessary libraries along with OpenCV, thus making your work simpler. Install the Tegra android pack and go through the basic tutorial " Build your first App" in Android developer portal. This will give you a clear idea on various files/folders and its functionality in android development.

Once you are clear with the basics of the android development, different files used in it, accessing various sensors/hardware components, you can start looking after the example codes first and then you`ll be able to develop your own code upon analyzing the example codes.

This is a good link with clear explanation of all required files to build a basic android app. After going through that tutorial, you can start with the image processing in android. To do this, click on the file menu, choose import and then choose existing android application. Now navigate to OpenCV-Android sample programs that you have in your computer (click here to see how to do).

One must always keep in mind  that the image processing methods are same in both android and windows platform. All you need to take care is, the input image and the output image format. The Android camera`s format is RGBA, which means the image what you will be acquiring will be in RGBA format and you need to return the image of same format after doing your processing.

So,

1. Acquire RGBA image

2. Convert it into required format (RGB/Ycrcb/Grayscale etc)

3. Do all your image processing analysis as you would do in OpenCV

4. Convert the output image into RGBA

5. Return the output image to display

These are all the basic steps that one should always keep in mind while programming. Here is a video of my first android application in which I have implemented the following,

1. Thresholding (60/255)

2. Skin Detection and contour drawing over detected skin

3. Canny edge detection

4. Circle detection

I have used a simple switch case to toggle between these four processing when a menu in the android is selected. Which means, all the processing and logics are same for both windows and Android. All you need to do is to think that 'it is easy to work on', spent some time in understanding the basics and eventually you will succeed.

 Once you know how it is done, then you can proceed with some advanced techniques and can do high level  image processing with your own android mobile. And with further interest and patience, you can go through some tutorials about Augmented Reality and can start developing your own Augmented reality apps (as I am doing right now). Have FUN! and feel free to contact me for further assistance.

Object Recognition in MATLAB

Recently, I was working in a conveyor belt setup, where, a camera was used to recognize the objects in the belt and an actuator to eject the unwanted objects. As I always like to simulate the algorithm before an actual implementation, I have used MATLAB for image recognition and based on the object classification made by MATLAB, the actuator will be actuated (to eject unwanted objects).

For simulation purpose, I have established serial communication between MATLAB and Proteus ISIS ( where I can have microcontroller, LEDs, Actuators etc). I have used a microcontroller of my choice and enabled its serial receive pin, so that it can receive the data that MATLAB transmits. MATLAB recognizes a known object by matching its SURF features and serially transmit a unique character/integer for each object. The microcontroller is coded in such a way that, if a specific character/integer is found in the receive buffer, then the actuator will be actuated.

Below shown is a simulated video of a conveyor belt setup. The video was taken using a 2MP cellphone camera. LEDs were interfaced with the microcontroller in Proteus ISIS to check the image recognition algorithm developed in MATLAB ( Every object has its own LED interfaced). Microcontroller coding was done in Keil and the .hex file is dumped into the simulated microcontroller in Proteus ISIS.

The detection algorithm was a bit slow in the video used. However, with the use of Bluecougar-X-X120aG GigE camera, improvement in detection speed was observed. In addition to this, I have also tested the similar object recognition algorithm in OpenCV. By my experience in working with both MATLAB and OpenCV, the detection speed is remarkably high in OpenCV when compared to MATLAB

  

Image matching in MATLAB

Image matching- which means comparing two images for identical features. Often its practical usages are biometric applications such as finger print identification, Iris matching etc. Two images matches when their features coincides and these feature could be edges, corners, blobs, color, shape etc. By identifying these specific features in an image and comparing it with the features of other image, we can determine whether the images matches or not. Below shown is a demonstration of the image matching in the application of finger print recognition in MATLAB,

In this program, the number of edges in the image1 is compared with that of the number of edges in the image2. If the image matches, the GUI returns a 'Match' dialogue box.

Algorithm:

1. Load the image into MATLAB
2. Convert to grayscale image
3. Apply the edge detector of your choice after removing noises, if any.
4. Create a copy of this image for comparison (say img1 and img2)
5. Now traverse through the every pixel of both the images using a looping statement,
6. And, check the pixel value at the corresponding pixel
7. If the comparison percentage if greater than certain percentage( say 90, as per your wish), then the finger prints matches, else, the finger prints are not a match



Note: This not the exact way how the conventional biometric scanners work, but, this method can be used for rough matching purpose

H Bridge simulation in NI multisim

H Bridge configuration has many applications in controlling a motor. Generally, a motor can be either switched ON or OFF(uni directional rotation) based on the needs and the direction of rotation depends on the polarity. But, in the field of Robotics and Medical applications, a single motor must have the ability to rotate in two directions(clockwise and anti clockwise). For this purpose, an H bridge configuration is preferred. As the name indicates, the circuit will be in the shape of alphabet “H”. The motor is made to rotate in two directions by changing the polarity of the motor.

 

Look at the following connections, 

Here, the direction of rotation of the motor varies with the change in its polarity. A simulation of the above schematic is shown below,

A combination of these two schematic gives a H Bridge circuit and the switching between polarities is done using the transistors(electronic switches) as shown below,

From the above circuit, it can be seen that when the switches(SW1 and SW3) closed, the motor will be in one polarity and when the switches(SW2 and SW4) are closed while the other switches are open, the motor will be in another polarity. And this is how an H Bridge circuit works and the transistors are used as the switches.  Below shown is the simulation of the H Bridge circuit in NI multisim,

Here, when the gate is ON, the MOSFET is turned ON, hence forms a closed circuit and the motor receives the polarity based on the above explanations. 

Making of my Autonomous Vehicle

I have built an Autonomous Vehicle as a part of my final year project, which was sponsored by “Analog Devices, Inc”.The vehicle have the ability to move from one point to another point based on the GPS coordinates of both points and also plots the path it travels in google maps. In order to avoid obstacles in its path, the vehicle uses a grid of sensors that includes laptop based RADAR, camera, proximity sensors. Following are few snapshots of the prototype that was built (camera is not shown in the picture),

Front view

Side view

Here are some videos of making of my Autonomous Vehicle,

Initial obstacle avoidance algorithm testing 1 - outdoor

Initial obstacle avoidance algorithm testing 2 - outdoor

Optimized algorithm testing 1 - outdoor

Optimized algorithm testing 2 - indoor

Testing after changing vehicle dynamics 1 (vehicle turning radius test)

Testing after changing vehicle dynamics 2 (vehicle turning radius test)

Testing after changing vehicle dynamics 3 + Acquiring GPS data of start and end points

Image processing algorithm testing ( webcam+ MATLAB Simulink)

Plotted GPS data in google earth (while vehicle is moving)

Plotted GPS data in google earth (while vehicle is stationary)

Preliminary model built to test the algorithms 

Heart rate measurement in LabVIEW

This is one of my project in which I have designed a transducer that can measure the variations in oxygenated hemoglobin and deoxygenated hemoglobin ( simply known as an oximeter).The sensor was developed based on the key principle that these two parameters have two different optical spectra in the range of 500nm to 1000nm. Hence, two light sources of two different wavelength(red-660nm and infra red-940nm) are used. A photo diode was used to sense absorption rate and the output of the photo diode was obtained in the LabVIEW using a data aquisition card(NI 6211).  The photo plethysmograph(PPG) waveform was obtained in LabVIEW by driving the leds using PWM signals of 25% duty cycle. Hence, based on the PPG waveform, the heart rate(Beats Per Mintue) was found.

This is the video of the PPG waveform obtained in the LabVIEW. Since ordinary sensors was used, there were lot of noise interference in the reading and the BPM was very fluctuating.

(the PPG waveform in the video is a recorded measurement file of the actual experiment)

This is the PWM pulse which was generated to drive the LEDs from the LabIEW. LEDs are switched alternatively at same duty cycle.

Shown below is the transducing part which I have developed in my college laboratory,

“Simulation” is always a good idea before an actual execution of a project. I have simulated the transducing circuit in NI Multisim and it helped me a lot in actual implementation. Following is the video of the simulation. 

I have used the current source since the output of a photo diode is current and used an I-V converter to convert it into voltage( since input of NI 6211 must be in volts). Here I have also added a sample and hold circuit to hold one led`s value while other is being sampled and vice versa. The manual switch in the simulation was replaced with the control signal from the LabVIEW.

Though the acquired results got some flaws, obtaining a biological parameter was really awesome!

The Free Radicals

Human body consists of n number of feedback loops right from a single cell to the  major parts of the body. Alteration to any of these cells, vessels or parts, leads the regular bio feedback loop to collapse and in some cases, may even results in death. These process often acts like a butterfly effect.

One of the major issue to consider is our immune system and free radicals. Our body`s defense system releases free radicals(a by-product of metabolic process of oxidation)to fight against viruses and bacteria. But the excess free radicals produced due to pollution, smoking, stress, along with the indigenous unterminated  free radical chain tends to steal(in order to get paired) electrons available in the body in every part of the tissue. And this chain goes on until all the free radicals are perfectly bonded.

This is the root cause for majority of heart disease, artery blockage and cancer since these free radicals are more fond of electrons in the region of heart and brain. Unfortunately there are no effective drugs have been developed so far to address this issue. And since free radicals are essential, it cannot be terminated too. But the excess free radicals bonded with the oxygen to form oxygen free radicals can be reversed or neutralized with proper intake of glutathione peroxidase and anti oxidants.

One of the impact of these excess free radicals is that, its tendency to steal electrons from the DNA. If this process is succeeded, the DNA is subjected to mutation and incase of pregnant women, it could even damage the fetus. Thus the effective control of the free radicals must be considered as a preventive step for proper health condition. To do so, keep away from stress and pollution. Also consume anti oxidant rich foods. Vitamin C, E and glutathione peroxidase can reverse the oxygen free radicals into pure oxygen and also prevent excess free radicals and heart blockage. Hope this information helps!!

(I have written this article for my college magazine during my third year of Engineering. These are my perception of free radicals and body chemistry. and I am not an expert in medicine or biochemistry)