Activity 1: Adaptive Color Tracking

When we are asked to describe a person physically, one of the characteristics that we often notice or remember is the color of the person's skin. Indeed, skin color is one of the main traits that is being sought after in pattern recognition techniques. Usual techniques that employ color-based tracking uses a static skin model. But the problem is, based from our daily experience, we easily notice that the skin color changes as illumination conditions vary. In this activity, we reproduce an adaptive color-tracking technique that was developed by Soriano et. al. that utilizes a chromaticity based constraint to select training pixels in a scene for updating a dynamic skin color model. The technique is successfully applied to color-based face tracking in indoor and outdoor videos.

Figure 1. Varying skin color across different races.

Experiment

I used a captured video of a labmate, Kirby Cheng, a 3^rd year BS Applied Physics student, under various illuminations inside the Instrumentation Physics Laboratory and at the NIP corridor as shown in the figure below.

Figure 2. Images of Kirby under different illuminations: (a) - (b) with yellow lighting inside the Instrumentation Physics laboratory, (c) - (d) daylight at the National Institute of Physics (NIP) corridor and outside the NIP.

The Kirby video was first parsed into image frames. From one image frame, we cropped a sub-image of the region of interest (ROI), i.e. his skin, and computed its rg-histogram. The procedure was then repeated for frames where there were drastic changes in skin color. We estimated the color locus of that object using the computed rg-histograms and we performed histogram backprojection per frame.

Figure 3. Image overlayed with the corresponding bounding box

As seen in Figure 3, the size of the bounding box varies under varying illumination. Note that a larger bounding box is observed for the images with yellow lighting given that the color of Kirby's skin blends with the surrounding, thanks to his Chinese genes, while a relatively smaller bounding box appears for the daylight illumination. The image frames exhibit an unbroken tracking of Kirby’s skin under different illuminations.

Tips:

1. Make sure to first convert the rgb images to black and white using the im2bw command in Matlab.

2. Apply morphological operations to binarize the image and remove non-ROI blobs.

3. Use bwlabel to label the index blobs and regionprops to find blob features.

4. To speed up processing for real-time applications, confine object segmentation in the next frame within a slightly larger bounding box than the blob bounding box.

I give myself a grade of 10 for this activity since I was able to satisfy the main objective, which is to track the skin color of the sample under varying illuminations. As a side note, I spent a greater time on this activity mainly because I needed more time in reviewing the syntax and commands available in Matlab.



Reference:

[1]M. Soriano, B. Martinkauppib, S. Huovinenb and M. Laaksonenc, "Adaptive skin color modeling using the skin locus for selecting training pixels" Pattern Recognition 36 (2003) 681 – 690

For teaching concerns please visit: https://sites.google.com/site/alongjas/