Activity 5: Shape from texture using local spectral moments

Upon examining a picture of a curved 3D object made up of uniform patterns, we will notice that the size of the patterns and the spaces in between them seem to diminish as they approach the edges while the shape is somewhat distorted. It gives us an impression that the patterns differ in distance with respect to your line of vision. This disparity in the observed patterns is mainly due to the object being curved.

In this activity, I will outline how to recover the shape of the curved surface from texture information using a non-feature based solution. This technique assumes that the image texture variation is due to the surface curvature rather than to surface texture variation. Texture in this activity refers to the patterns present in the object.

Experiment

We were tasked to take a picture of a 3D object with repeating texture or pattern in its surface. I used a picture of a cylinder (3D object) with repeating black dot patterns of uniform size, shape and spacing on its surface that was used in Super and Bovik’s paper.

Figure 1. Sample image of a curved 3D surface (image size: 164 x 310)

The goal is to reconstruct the surface shape from the recovered tilt and slant using Super and Bovik's technique. A successful implementation of this activity will result in a 3D rendition of the object's surface.

The Gabor filter set we used was composed of radial frequencies ranging from 12 to 96 cycles/image and angular frequencies ranging from -70^o to +90^o in 20^o increments.

Gabor filter set:

Frequency = [12 17 24 34 48 69 96];

Orientation = [-70 -50 -30 -10 10 30 50 70 90]*pi/180;

Steps from Super and Bovik's paper:

1) Convolve the image with Gabor functions and their partial derivatives, and smooth the filter output amplitudes by convolving them with a Gaussian.

The sigma value in the Gaussian window can be varied but as a rule it should be at least 10% the size of the test image.

2) Compute the normalized (a, b, c) image moments from the filter outputs.

3) Optional (for textures with local inhomogeneity): smooth the moments.

4) Compute the canonical moments (M, m, 0) at each point using Equation 8.

5) Find the point x0 of minimum slant (point of minimum sqrt(Mm)). Assume this is a frontal point. Set M_s, ms to be M(xo), m(xo), respectively.

6) Compute sigma, tau at each point from M(x), m(x), q(x), and M_s, m_s using Equations 21- 23.

To reconstruct an estimate of the surface, we used shapeletsurf.m. It reconstructs an estimate of a surface from its surface normals by correlating the surface normals with those of available shapelet basis functions. The correlation results are summed to produce the reconstruction. The summation of shapelet basis functions results in an implicit integration of the surface while enforcing surface continuity.

Figure 2. Reconstructed shape from texture details.


Figure 2 shows the reconstructed shape of the sample cylinder image.

Tips:

1. Dr. Soriano suggested to use the axis equal command to make the size of the reconstructed image equal to the sample image. Unfortunately, when i used the command, the rendered image was very thin. I had to stick with Figure 2 as it looks a lot better than the former.

2. Remember when viewing the surface you should use axis ij so that the surface corresponds to the image slant and tilt axes.

I give myself a grade 0f 10 for this activity.

Reference:

[1]. Super B, Bovik A, Shape from texture using local spectral moments, IEEE Trans Pattern Analysis and Machine Intelligence, 17(4): 333-343,1995

For teaching concerns please visit: https://sites.google.com/site/alongjas/