ABS-pLSA

Absolute position pLSA (ABS-pLSA) attaches location information to each visual word by localizing it to one of X 揵ins?in the image. Here, ${\displaystyle \displaystyle x}$ represents which of the bins the visual word falls into. The new equation is:

${\displaystyle \displaystyle P(w|d)=\sum _{z=1}^{Z}P(w,x|z)P(z|d)}$

${\displaystyle \displaystyle P(w,x|z)}$ and ${\displaystyle \displaystyle P(d)}$ can be solved for in a manner similar to the original pLSA problem, using the EM algorithm

A problem with this model is that it is not translation or scale invariant. Since the positions of the visual words are absolute, changing the size of the object in the image or moving it would have a significant impact on the spatial distribution of the visual words into different bins.

TSI-pLSA

Translation and scale invariant pLSA (TSI-pLSA). This model extends pLSA by adding another latent variable, which describes the spatial location of the target object in an image. Now, the position ${\displaystyle \displaystyle x}$ of a visual word is given relative to this object location, rather than as an absolute position in the image. The new equation is:

${\displaystyle \displaystyle P(w,x|d)=\sum _{z=1}^{Z}\sum _{c=1}^{C}P(w,x|c,z)P(c)P(z|d)}$

Again, the parameters ${\displaystyle \displaystyle P(w,x|c,z)}$ and ${\displaystyle \displaystyle P(d)}$ can be solved using the EM algorithm. ${\displaystyle \displaystyle P(c)}$ can be assumed to be a uniform distribution.

Selecting words

Words in an image were selected using 4 different feature detectors:^[1]

• Kadir–Brady saliency detector
• Multi-scale Harris detector
• Difference of Gaussians
• Edge based operator, described in the study

Using these 4 detectors, approximately 700 features were detected per image. These features were then encoded as Scale-invariant feature transform descriptors, and vector quantized to match one of 350 words contained in a codebook. The codebook was precomputed from features extracted from a large number of images spanning numerous object categories.

Possible object locations

One important question in the TSI-pLSA model is how to determine the values that the random variable ${\displaystyle \displaystyle C}$ can take on. It is a 4-vector, whose components describe the object抯 centroid as well as x and y scales that define a bounding box around the object, so the space of possible values it can take on is enormous. To limit the number of possible object locations to a reasonable number, normal pLSA is first carried out on the set of images, and for each topic a Gaussian mixture model is fit over the visual words, weighted by ${\displaystyle \displaystyle P(w|z)}$. Up to ${\displaystyle \displaystyle K}$ Gaussians are tried (allowing for multiple instances of an object in a single image), where ${\displaystyle \displaystyle K}$ is a constant.

Initialization

The dataset must be initialized, or seeded with an original batch of images which serve as good exemplars of the object category to be learned. These can be gathered automatically, using the first page or so of images returned by the search engine (which tend to be better than the subsequent images). Alternatively, the initial images can be gathered by hand.

Model learning

To learn the various parameters of the HDP in an incremental manner, Gibbs sampling is used over the latent variables. It is carried out after each new set of images is incorporated into the dataset. Gibbs sampling involves repeatedly sampling from a set of random variables in order to approximate their distributions. Sampling involves generating a value for the random variable in question, based on the state of the other random variables on which it is dependent. Given sufficient samples, a reasonable approximation of the value can be achieved.

Classification

At each iteration, ${\displaystyle \displaystyle P(z|c)}$ and ${\displaystyle \displaystyle P(x|z,c)}$ can be obtained from model learned after the previous round of Gibbs sampling, where ${\displaystyle \displaystyle z}$ is a topic, ${\displaystyle \displaystyle c}$ is a category, and ${\displaystyle \displaystyle x}$ is a single visual word. The likelihood of an image being in a certain class, then, is:

${\displaystyle \displaystyle P(I|c)=\prod _{i}\sum _{j}P(x_{i}|z_{j},c)P(z_{j}|c)}$

This is computed for each new candidate image per iteration. The image is classified as belonging to the category with the highest likelihood.

Addition to the dataset and "cache set"

In order to qualify for incorporation into the dataset, however, an image must satisfy a stronger condition:

${\displaystyle \displaystyle {\frac {P(I|c_{f})}{P(I|c_{b})}}>{\frac {\lambda _{Ac_{b}}-\lambda _{Rc_{b}}}{\lambda _{Rc_{f}}-\lambda _{Ac_{f}}}}{\frac {P(c_{b})}{P(c_{f})}}}$

Where ${\displaystyle \displaystyle c_{f}}$ and ${\displaystyle \displaystyle c_{b}}$ are foreground (object) and background categories, respectively, and the ratio of constants describes the risk of accepting false positives and false negatives. They are adjusted automatically at every iteration, with the cost of a false positive set higher than that of a false negative. This ensures that a better dataset is collected.

Once an image is accepted by meeting the above criterion and incorporated into the dataset, however, it needs to meet another criterion before it is incorporated into the 揷ache set敆the set of images to be used for training. This set is intended to be a diverse subset of the set of accepted images. If the model were trained on all accepted images, it might become more and more highly specialized, only accepting images very similar to previous ones.