finding similar images from a database using image only not via tag

Well the question is simple i want to find similar images given a query image, similar to what TinEye does. Suppose I have a shirt with the following description

Sleeve length : full

collar : present

pattern : striped

(The above data is just to give you a feel of image i actually dont have this data)

First image is the query image and the next should be the output of the similarity finding algorithm. So based on the example we have a flexibility like we can show the user an image with a changed color, we can see all the images have the same pattern, the same collar type or sleeve length. So i have to show the output which are visually similar.

There are similar thread on stack also link from stack and not only this but there are many other. But i am confused about the approach to follow.

In my case i dont have to search in another category I have to search in the same category like if the input is shirt i will search in the shirt category only. That part has been done.

So the question is what are the approaches to handle this problem. for the color it is no big issue. Color information can be easily extracted through color histogram. Lets say the input is TShirt round neck i.e. without collar, half sleeve and printed at center with text. Now the output should be images similar to those like half sleeve, round collar, and printed text at center. thought the text may vary. I tried K-Means clustering and P-hash but that didnt work. Please enlighten me

PS : I have to find similar images not duplicates.

asked May 16, 2015 in hibernate by rajesh
0 votes

1 Answer

0 votes

I would try to split this problem into 3 smaller problems:

  • checking whether image shows shirt with long or short sleevs
  • checking pattern (stipped, plain, something else?)
  • determining color of shirt

Checking whether image shows shirt with long or short sleevs
This one is in my opinion the easiest. You mentioned that you have category name, but basing on google graphics it seems that it may not be obvious whether Shirt or TShirt has long or short sleevs.
My solution is quite simple:

  • Find face on image
  • Use grabcut algorithm to extract face mask from image
  • Mask face (so after this step only face is left - everythin else is black). Note that this step is not necessary - i've mentioned it only, because it's shown on final image.
  • Convert image to HSV color space
  • Using face mask calculate histogram for H and S color channels of FACE ONLY (without rest of the image)
  • Calculate back projection of hsv image using histogram from previous step. Thanks for that you will get only regions which color (in HSV) is similar to color of face - so you will get only regions which contains skin.
  • Threshold the result (there is always some noise :) )

The final result of this algorithm is black and white image which shows skin regions. Using this image you can calculate number of pixels with skin and check whether skin is only on face or maybe somewhere else. You can try to find contours as well - generally both solututions will give chance to check if hands are visible. Yes - shirt has short sleevs, no - long sleevs.

Here are the results (from top-left corner - original image, face mask (result of grabcut algorithm), masked face, hsv image, result of calculating back projection, result of using threshold on previous one): 

answered May 16, 2015 by rajesh