Paper ReadingDeep Supervised Hashing for fast Image Retrieval

Posted 2020-10-17 VincentCheng

what has been done:

This paper proposed a novel Deep Supervised Hashing method to learn a compact similarity-presevering binary code for the huge body of image data. 

Data sets: 

CIFAR-10: 60,000 32*32 belonging to 10 mutually exclusively categories(6000 images per category)

NUS-WIDE: 269,648 from Flickr, warpped to 64*64 

content based image retrieval: visually similar or semantically similar. 

Traditional method: calculate the distance between the query image and the database images. 

Problem: time and memory

Solution: hashing methods(map image to compact binary codes that approximately preserve the data structure in the original space)

Problem: performace depends on the features used, more suitable for dealing with the visiual similarity search rather than the sematically similarity search.  

Solution: CNNs, the CNNs successful applications of CNNs in various tasks imply that the feature learned by CNNs can well capture the underlying sematic structure of images in spite of significant appearance variations. 

Related works:

Locality Sensitive Hashing(LSH)：use random projections to produce hashing bits

cons: requires long codes to achieve satisfactory performance.(large memory)

data-dependent hashing methods: unsupervised vs supervised

unsupervised methods: only make use of unlabelled training data to lean hash functions

• spectral hashing(SH): minimizes the weighted hamming distance of image pairs
• Iterative Quantization(ITQ): minimize the quantization error on projected image descriptors so as to allievate the information loss

supervised methods: take advantage of label inforamtion thus can preserve semantic similarity

• CCA-ITQ： an extension of iterative quantization
• predictable discriminative binary code: looks for hypeplanes that seperate categories with large margin as hash function.
• Minimal Loss Hashing(MLH): optimize upper bound of a hinge-like loss to learn the hash functions

problem: the above methods use linear projection as hash functions and can only deal with linearly seperable data. 

solution: supervised hashing with kernels(KSH) and Binary Reconstructive Embedding(BRE).

Deep hashing: exploits a non-linear deep networks to produce binary code. 

Problem : most hash methods relax the binary codes to real-values in optimizations and quantize the model outputs to produce binary codes. However there is no guarantee that the optimal real-valued codes are still optimal after quantization .

Solution: DIscrete Graph Hashing(DGH) and Supervided Discrete Hashing(DSH) are proposed to directly optimize the binary codes. 

Problem : Use hand crafted feature and cannot capture the semantic information.

Solution:   CNNs base hashing method

 

Our goal: similar images should be encoded to similar binary codes and the binary codes should be computed  efficiently.

Loss function:

Relaxation:

Implementation details:

Network structure:

3*卷积层：

3*池化层：

2*全连接层：

Training methodology:

• generate images pairs online by exploiting all the image pairs in each mini-batch. Allivate the need to store the whole pair-wise similarity matrix, thus being scalable to large-scale data-sets.
• Fine-tune vs Train from scratch

Experiment:

CIFAR-10

GIST descriptors for conventional hashing methods

NUS-WIDE

225-D normalized block-wise color moment features

Evalutaion Metrics

mAP: mean Average Precision

precision-recall curves(48-bit)

mean precision within Hamming radius 2 for different code lengths

Network ensembles? 

Comparison with state-of-the-art method

CNNH: trainin the model to fit pre-computed discriminative binary code. binary code generation and the network learning are isolated

CLBHC: train the model with a binary-line hidden layer as features for classification, encoding dissimilar images to similar binary code would not be punished. 

DNNH: used triplet-based constraints to describe more complex semantic relations, training its networks become more diffucult due to the sigmoid non-linearlity and the parameterized piece-wise threshold function used in the output layer. 

Combine binary code generation with network learning 

Comparision of Encoding Time