FREQUENCY DECODED LOCAL BINARY PATTERN

1

Face Retrieval using Frequency Decoded Local
Descriptor

arXiv:1709.06508v1 [] 16 Sep 2017

Shiv Ram Dubey, Member, IEEE
Abstract—The local descriptors have been the backbone of most of the computer vision problems. Most of the existing local
descriptors are generated over the raw input images. In order to increase the discriminative power of the local descriptors, some
researchers converted the raw image into multiple images with the help some high and low pass frequency filters, then the local
descriptors are computed over each filtered image and finally concatenated into a single descriptor. By doing so, these approaches do
not utilize the inter frequency relationship which causes the less improvement in the discriminative power of the descriptor that could be
achieved. In this paper, this problem is solved by utilizing the decoder concept of multi-channel decoded local binary pattern over the
multi-frequency patterns. A frequency decoded local binary pattern (FDLBP) is proposed with two decoders. Each decoder works with
one low frequency pattern and two high frequency pattern. Finally, the descriptors from both decoders are concatenated to form the
single descriptor. The face retrieval experiments are conducted over four benchmarks and challenging databases such as PaSC, LFW,
PubFig, and ESSEX. The experimental results confirm the superiority of the FDLBP descriptor as compared to the state-of-the-art
descriptors such as LBP, SOBEL LBP, BoF LBP, SVD S LBP, mdLBP, etc.
Index Terms—Local Descriptor, High Frequency, Low Frequency, Unconstrained, Retrieval, Face, Decoder.

F

1

I NTRODUCTION

1.1

Motivation

F

ACIAL analysis is a very active research area in the computer vision field. In the early days, the face recognition
was being done in a very controlled environment, where the
images were being captured in frontal pose with uniform
background, etc. From last decade, the research focus has
been shifted towards the unconstrained and robust face
recognition. Wright et al. followed a sparse representation
for face recognition [1]. Ding et al. did the pose normalization for face recognition against pose variations [2]. The
pose-invariant face recognition is converted into a partial
frontal face recognition and then a patch-based face description is employed in [3]. Face recognition against pose,
illumination and blur is conducted by Punnappurath et al.
by modeling the blurred face as a convex set [4]. The survey
in face recognition is also conducted by several researchers
[5], [6], [7]. Due to the increasing demand of real life facial
analysis, unconstrained and robust algorithms are needed
to be investigated.
The facial analysis approaches can be categorized into
following three areas: deep learning based, learning descriptor based, and handcrafted based. The deep learning
based approaches are proposed recently such as DeepFace
[8] and FaceNet [9]. These approaches have gained very
appealing result, but at the cost of huge computation. More

This research is funded by IIIT Sricity, India through the Faculty Seed
Research Grant.
S.R. Dubey is associated with the Indian Institute of Information Technology
Sricity, Chittoor District, AP-517646, India
Preprint Submitted

complex computing resources as well as huge amount of
data are required to run the deep learning based approaches.
Moreover, the database biased-ness has been seen in the
deep learning based approaches. Several learning based
descriptors have been investigated for face recognition task,
such as learning based descriptor (LE) [10], descriptor with
learned background [11], learned discriminant descriptor
[12], learned binary descriptor [13], and simultaneous binary learning and encoding [14]. These descriptors rely over
some codebook or clustering methods to learn the training
data statistics and cannot learn the features that are not
present in the training data. Whereas, on the other hand,
the handcrafted descriptors are easy to design and follow,
independent of the data, does not require complex computing resources and huge amount of data for the training. In
the rest of this paper, the hand-designed descriptors are the
focus of interest.
The local binary pattern (LBP) is widely adapted local
descriptor [15]. It encodes the local relationship of the pixels. Basically, it finds a binary pattern for each pixel and
represent it into the histogram form. A binary pattern for
a pixel consists of the 8 binary values corresponding to
8 local neighbors of the concerned pixel distributed in a
circular fashion. For a neighbor, the binary value is 1 if
the intensity value of neighboring pixel is greater than or
equal to the intensity value of the center pixel, otherwise
the binary value is 0. Originally, LBP was proposed for
the texture classification [15]. After its invention, the LBP
has been proved as the foundation for many computer
vision problems [16], [17]. Several variants of LBP have been
investigated by several researchers for various problems
such as local image matching [18], content based image
retrieval [19], [20], [21], texture classification [22], [23], [24],
biomedical image analysis [25], [26], [27], [28], etc.
In 2006, Ahonen et al. utilized the LBP concept for face

FREQUENCY DECODED LOCAL BINARY PATTERN

representation and recognition which is later regarded as
the state-of-the-art in face recognition [29]. Inspired from the
success of LBP in face recognition as well as other problems,
many improvements in LBP have been proposed for the face
recognition [17]. Huang et al. presented a literature survey
over LBP based descriptors in 2011 for the facial image
analysis [30]. In 2013, one more survey over LBP based face
recognition is conducted by Yang and Chen [31]. A local
ternary pattern (LTP) is proposed in [32] for illumination
robust face recognition by extending the binary concept of
LBP into ternary with the help of a threshold. Finding the
suitable threshold in LTP is one of the problems. The local
derivative pattern (LDP) utilizes the LBP concept over high
order derivatives [33]. Basically, LDP first computes the four
derivative images in 0o , 45o , 90o , and 135o directions respectively. Then, it finds the LBP histogram over each derivative
image and finally concatenate into a single descriptor. Vu
et al. proposed the patterns of oriented edge magnitudes
(POEM) descriptor by incorporating the gradient magnitude
and orientation information [34]. In order to further improve
the POEM descriptor, the whitened principal-componentanalysis dimensionality reduction technique is adapted by
Vu and Caplier [35]. Later on, in 2013, Vu proposed patterns
of orientation difference (POD) descriptor and combined
with the POEM descriptor to boost the performance for face
recognition [36].
A local vector pattern (LVP) is investigated by Fan and
Hung from the vector representations of each pixel [37].
The vector representation of a center pixel in LVP utilizes
the relationship between the center pixel and its adjacent
neighbors at various distances in different directions. In
dimensionality reduced local directional pattern (DR-LDP)
[38], first a local directional pattern is computed for each
pixel of the face image, then the image is divided into multiple blocks, and finally a single code per block is generated
by XORing the local directional pattern codes of each pixel
of that block. The local directional pattern uses the lowpass Kirsch filters in eight directions to get the directional
information. Ding et al. introduced multi-directional multilevel dual-cross pattern (MDML-DCP) for face images [39].
In order to reduce the impact of differences in illumination, they employed the first derivative of a Gaussian with
varying scale. The dual-cross pattern is then computed from
the multi-radius neighborhood in two different neighboring
sets, one consisting of horizontal and vertical neighbors and
another one consisting of diagonal neighbors. The gradient orientations in two directional gradient responses in
horizontal and vertical directions are used to design the
local gradient order pattern (LGOP) for face representation
[40]. Recently, a local directional gradient pattern (LDGP)
exploited the relationship between the referenced pixel in
four different directions in high order derivative space for
face recognition [41]. The local directional ternary pattern
(LDTP) uses eight Robinson compass masks as the lowpass filters to generate the eight directional images and then
computes the final ternary pattern based on the primary and
secondary directions [42].
Most of the LBP variants have focused over the derivative, direction and coding pattern and overlooked the frequency information. Some descriptors used high-pass information, but only for the direction purpose. The high-

2

pass information in the image corresponds to the details
and low-pass information corresponds to the coarse of the
image. Zhao et al. proposed the Sobel local binary pattern
(SOBEL LBP) from the Sobel horizontal and vertical filters
for face representation [43]. In SOBEL LBP, two filtered
images are obtained using Sobel operators, then the LBP
histograms are computed over both and concatenated to
produce the final descriptor. In order to increase the robustness of the descriptor, a pre-processing in the form of
high-pass frequency filter is used in semi-structure local
binary pattern (SLBP) for facial analysis [44]. The 3 × 3 mask
of filter used in SLBP is [1,1,1;1,1,1;1,1,1]. Recently, a bagof-filtered local binary pattern (BoF LBP) is proposed for
content-based image retrieval [20]. BoF LBP concatenates
the LBP computed over the several filtered images. Five
filters, including one low-pass and four high-pass filters are
used in BoF LBP. The local SVD decomposition has recently
been used as the pre-processing step with the existing
descriptors such LBP to develop the SVD S LBP descriptor
[45]. It is explored in [45] that the S sub-band of SVD is
more discriminative and worth for the face retrieval over
near-infrared images. A concept of multi-channel decoded
local binary pattern (mdLBP) is proposed recently for color
image retrieval, where the LBP is first computed over each
color channel separately and then passed through a decoder
to utilize the inter-channel relationship in the final feature
vector [21]. Some recent color descriptors are local color occurrence descriptor [46], rotation invariant structure based
descriptor [47], illumination invariant color descriptor [48],
etc.
In this paper, the decoder concept is utilized with the
multi-frequency filtered images. Instead of simply concatenating the LBP features computed over different filtered
image in BoF LBP or SOBEL LBP, here it is passed to
the decoder to encode the inter-frequency information and
then the output channels are combined. The BoF LBP has
used five filters with one low-pass and four high-pass. The
dimension of the final descriptor will be quite high if all five
frequencies is used with one decoder. In order to resolve
this problem, two decoders with one low-pass frequency
information and two high-pass frequency information are
used in this work. The combination of different frequency
information is evaluated empirically. The main contributions of this paper are summarized as follows:
•

•

•

•
•

Multiple frequency information, including one lowpass and four high-pass is used to increase the discriminative power and robustness of the descriptor.
In the proposed frequency decoded local binary
pattern (FDLBP), the inter-frequency information is
encoded with the help of decoder concept using LBP
binary bits.
In order to reduce the dimensionality of the descriptor, two decoders are used with one low-pass and
two high-pass filters. Different frequency decoder
combinations are also explored through the experiment.
The concept of proposed frequency decoder is also
extended for the color face images.
The image retrieval experiments are conducted over
four very challenging face databases including Point

FREQUENCY DECODED LOCAL BINARY PATTERN

•

3

and Shoot Challenge, Labeled Faces in Wild, Public
Figures, and ESSEX.
The proposed descriptor is tested with different distance measures such as Euclidean, Cosine, Emd, L1,
D1, and Chi-square.

The rest of the paper is structured in the following manner: Section II describes the proposed descriptor; Section
III illustrates the experimental setup; Section IV reports the
experimental results, comparison and analysis; and finally
Section V sets the concluding remarks.

2

P ROPOSED FDLBP D ESCRIPTOR

In this section, the proposed frequency decoded local binary
pattern (FDLBP) computation process is described in detail.
The whole procedure is divided into several steps such as
image filtering with low-pass and high-pass filters, LBP
binary code computation, inter-frequency coding, feature
vector computation, and color channel consideration.

2.1

2.2

LBP Binary Code Generation

Once the input face image is transferred into filtered domain
using different filters, the local binary pattern (LBP) [29]
codes are generated for each pixel of each filtered face
image. As per the convention, Ik for k ∈ [hv, d, sv, sh] is
the input to the LBP code generator. Suppose Ik (i, j) is
the value in ith row and j th column of filtered image Ik .
Consider, LBP is generated from the N number of local
neighbors evenly distributed in a circular fashion at a radius
R. The number of binary bits in the LBP code is same as
the number of local neighbors, i.e., N . The tth neighbor
for a reference pixel Ik (i, j) is denoted by Ikt (i, j), where
t ∈ [1, N ] as shown in Fig. 3. The 1st neighbor is considered
in the right side of the reference pixel and rest neighbors
are considered w.r.t. the 1st neighbor in a circular fashion in
the counter-clockwise direction (see Fig. 3). The coordinate
of tth neighbor of center pixel Ik (i, j) can be given by
(i − R × sin(θt ), j − R × cos(θt )), where θt is the angular
displacement of tth neighbor w.r.t. the 1st neighbor and
computed as follows,

θt = (t − 1) × θ

(2)

where θ is defined as the angular displacement between any
two adjacent neighbors and can be written as follows,

Image Filtering

The low-pass filters generate the coarse information,
whereas the high-pass filters generate the detailed information of the image. The image filtering with one low-pass and
four high-pass filters are done in this paper. These filters
are the same as used in BoF LBP [20] and shown in Fig.
1. The average filter denoted by Fa is used for the lowpass frequency filtering. The horizontal-vertical difference
filter (i.e., Fhv ), diagonal difference filter (i.e., Fd ), Sobel
vertical edge filter (i.e., Fsv ), and Sobel horizontal edge filter
(i.e., Fhh ) are used for the high-pass frequency filtering.
Suppose, the input grayscale image is represented by I
with dimension x × y (i.e. image I is having x rows and
y columns) and the filtered images are denoted by Ia , Ihv ,
Id , Isv , and Ish by using Fa , Fhv , Fd , Fsv , and Fsh filters
respectively. Mathematically, Ik |∀k=a,hv,d,sv,sh can be given
as follows,

I k = I ∗ Fk

(1)

where ’∗’ represents the 2-D convolution operation between
two matrices and Fk is a 3×3 filter with k ∈ [a, hv, d, sv, sh].
Note that the input image I is padded with one pixel (i.e.,
nearly half of the size of the filter) in all directions to obtain
the same size filtered image Ik .
The Fig. 2 presents the five filtered face images for an input face image using the five filters, Fk |k ∈ [a, hv, d, sv, sh].
The input face image is taken from the LFW database [49].
It can be seen that the filtered image using a low-pass filter
(i.e., Fa ) in Fig. 2a is having the coarse information of the
face image, whereas other hand the filtered images using
high-pass filters (i.e., Fhv , Fd , Fsv , and Fsh ) are having
the detailed information with corresponding edges. More
specifically, Fig. 2c and 2d is having the horizontal-vertical
difference and diagonal difference information, whereas,
Fig. 2e and 2f are having the horizontal edge and vertical
edge information.

360
.
(3)
N
The N bit binary LBP code for a reference pixel Ik (i, j)
is computed as follows,
θ=

βk (i, j) = [βk1 (i, j), βk2 (i, j), ..., βkt (i, j), ..., βkN (i, j)]

(4)

where βkt (i, j) is the binary bit value computed for the tth
neighbor of reference pixel Ik (i, j). The βkt (i, j) is generated
from the center value Ik (i, j) and tth neighboring value
Ikt (i, j) as follows,

βkt (i, j) = sign(∆(Ikt (i, j), Ik (i, j)))

(5)

where ∆(α1 , α2 ) encodes the amount of change in between
α1 and α2 and derived as follows,

∆(α1 , α2 ) = α1 − α2

(6)

and sign(α) converts the α in binary form with the help of
following rule,
(
1, if α ≥ 0
sign(α) =
(7)
0, otherwise
In next sub-section, the LBP binary bits (i.e., βk ) computed
over each filtered image (i.e., ∀k ∈ [a, hv, d, sv, sh]) are used
as the input to the frequency decoder.
2.3

Frequency Decoder

The concept of decoder is used in this work to utilize the
relationship among different frequency filtered images. The
decoder concept is also used in mdLBP with the Red, Green,
and Blue color channels of the image [21]. Here, in this work,
two decoders are used to control the number of output
images. The number of output channels for a single decoder
is 2γ , where γ is the number of input channel. So, in case of
one decoder, the number of output image will be 25 = 32

FREQUENCY DECODED LOCAL BINARY PATTERN

(a) Fa , i.e., Average Filter

(b)
Fhv ,
i.e.,
Horizontal-vertical
Filter

4

(c) Fd , i.e., Diagonal
Filter

(d) FSV , i.e., Sobel Vertical Edge Filter

(e) FSH , i.e., Sobel Horizontal Edge Filter

Fig. 1: The five filters, including one low-pass and four high-pass filters used in this paper.

(a) I

(b) Fa

(c) Fhv

(d) Fd

(e) FSV

(f) FSH

Fig. 2: An example demonstrating the filters used to generate the difference frequency images. (a) Original Image, I , taken
from the LFW database [49]. Images obtained using (b) Fa , i.e., Average Filter, (c) Fhv , i.e., Horizontal-vertical Filter, (d)
Fd , i.e., Diagonal Filter, (e) FSV , i.e., Sobel Vertical Edge Filter, and (f) FSH , i.e., Sobel Horizontal Edge Filter.
where ωt is a weighting factor for tth bit and computed as
follows,
ωt |t∈[1,N ] = (2)(t−1)
(9)

Fig. 3: The N neighbors Ikt (i, j)|∀t∈[1,N ] in the local
neighborhood of pixel Ik (i, j) at radius R, where k ∈
[a, hv, d, sv, sh]. Note that the first neighbor is considered
in the right side and rest neighbors are considered w.r.t. first
neighbor in counter-clockwise direction.

for γ = 5 input images. Whereas, two decoders with γ = 3
input image each, have the 2 × 23 = 16 output images. So,
in multi-decoder framework with a Γ number of decoders,
each having a γ number of input images, the total number
of output images is Γ×2γ . Suppose, Ψ represents the output
image of the decoders. The ζ th output of the η th decoder is
given by Ψζη , where ζ ∈ [1, 2γ ] and η ∈ [1, Γ]. The Ψζη (i, j)
is the value of face representation in the ith row and j th
column of ζ th output image of the η th decoder and given as
follows,

Ψζη (i, j) =

N
X
t=1

Ψζ,t
η (i, j) × ωt

(8)

th
and Ψζ,t
binary value for pixel (i, j) in the ζ th
η (i, j) is the t
th
output image of η decoder and calculated with the help of
input binary bits.
In this paper, Γ = 2 number of decoders are used and
γ = 3 number of inputs are given to each decoder. So, each
decoder generates ζ = 2γ = 8 number of output images.
One of the inputs to both decoders is the LBP binary codes
of low-pass filtered image with average filtering, i.e., βa . The
rest of two inputs to 1st decoder are the LBP binary codes
of high-pass filtered images with horizontal-vertical and
diagonal filtering, i.e., βhv , and βd respectively. Similarly, the
other two inputs to 2nd decoder are the LBP binary codes
of high-pass filtered images with Sobel-vertical and Sobelhorizontal filtering, i.e., βsv , and βsh respectively. So, as per
the above convention, the output of 1st and 2nd decoder is
ζ
ζ,t
ζ
denoted by Ψ1 and Ψ2 respectively. The Ψ1 (i, j) is the tth
th
binary value for pixel (i, j) in the ζ output image of 1st
decoder and calculated as follows,
(
t
1, if ζ = Ω(βat (i, j), βhv
(i, j), βdt (i, j))
ζ,t
(10)
Ψ1 (i, j) =
0, otherwise

where Ω is used to decode the three binary input values into
a decimal value as follows for three binary inputs b1 , b2 , and
b3 ,
Ω(b1 , b2 , b3 ) = (b1 × 4) + (b2 × 2) + b3 + 1.
(11)
ζ,t

Similarly, the Ψ2 (i, j) for 2nd decoder is calculated as
follows,
(
t
t
1, if ζ = Ω(βat (i, j), βsv
(i, j), βsh
(i, j))
Ψζ,t
2 (i, j) =
0, otherwise
(12)

FREQUENCY DECODED LOCAL BINARY PATTERN

5

Fig. 4: The illustration of frequency decoded local binary pattern. (a) An example face image taken from LFW database
[49]. (b) Five images obtained by applying the four high-pass filter and one low-pass filter (i.e., middle one). These filters
are also used in BoF LBP [20]. (c) The LBP binary N − bit code computed for every pixel of filtered face images using LBP
operator [29]. (d) The concept of decoder is used to utilize the inter-frequency relationship of LBP binary representation.
Note that the decoder concept in mdLBP is applied over the Red, Green and Blue color channels of the image [21], whereas
it is applied over the different filtered images, including one low-pass filter and two high-pass filters in this work. (e) The
sixteen output face representations, including eight from each decoder are shown for the considered example face image.
where Ω is defined in Eq. (11). After finding the binary
ζ,t
values of output channels for both decoders, i.e., Ψ1 (i, j)
ζ,t
γ
and Ψ2 (i, j) for ζ ∈ [1, 2 ], γ = 3 and t ∈ [1, N ], the output
ζ
ζ
values, i.e., Ψ1 (i, j) and Ψ2 (i, j) are computed using Eq. (8).

The concept of frequency decoder is illustrated in Fig.
4 with the help of an example face image taken from the
LFW database [49]. The Fig. 4a shows the example face
image. The output filtered images are displayed in Fig. 4b by
applying the Fhv , Fd , Fa , Fsv , and Fsh filters over the input
example face. Next, the LBP binary codes are generated over
each filtered image in the Fig. 4c by using the algorithm
presented in sub-section ”LBP Binary Code Generation”.
Two decoders with three inputs each are used in Fig. 4d.
The three inputs for the first decoder are LBP binary codes
obtained over the filtered images generated using Fhv , Fd ,
and Fa filters, whereas the inputs for the second decoder
are LBP binary codes obtained over the filtered images
generated using Fa , Fsv , and Fsh filters. Finally, Fig. 4e
presents the output images of the frequency decoders.

2.4

Feature Vector Computation

After finding the face representations by applying the frequency decoder over the multiple filtered images, the next
task is to convert the representations into a feature vector
form. The standard histogram strategy is used to do so.
Suppose the final feature vector is denoted by ξ and given
as follows,

ξ =< ξ1 , ξ2 >

(13)

=<2η=1 ξη >

where the < υ1 , υ2 > symbol is used for the concatenation
of the two vectors υ1 and υ2 , and ξ1 and ξ2 are the feature
vectors computed by using the output images of 1st and 2nd
decoder respectively. The feature vector for the η th decoder,
i.e., ξη is computed as follows,
γ

ξη =< ξη1 , ξη2 , ...., ξηζ , ...., ξη2 >
γ

=<2ζ=1 ξηζ >

(14)

where ξηζ is the feature vector generated as the histogram
over the ζ th output image of η th decoder. Mathematically,

FREQUENCY DECODED LOCAL BINARY PATTERN

6

Fig. 5: The face retrieval framework using proposed FDLBP descriptor.
the ξηζ is calculated as follows,

ξηζ

=<

2.5

ξηζ (1), ξηζ (2), ..., ξηζ (λ), ...., ξηζ (2N

N
=<2λ=1−1

ξηζ (λ)

>

− 1) >

(15)

where N is the number of neighbors initially considered to
get the LBP binary codes, λ is the bin number, and ξηζ (λ) is
the number of occurrences of λ in the output image ξηζ as
follows,
x−R
X y−R
X
ξηζ (λ) =
ρ(λ, Ψζη (i, j))
(16)
i=R+1 j=R+1

where R is the radius of local neighborhood considered initially for LBP binary code generation, x × y is the dimension
of the image, and ρ is a comparator function to check that
two values are same or not. The ρ for inputs α1 and α2 is
defined by the following equation,
(
1, if α1 = α2 ,
ρ(α1 , α2 ) =
(17)
0, otherwise.
Suppose, the dimension of feature vector of the ζ th
output image of η th decoder is represented by Dηζ and given
as follows,
(N )
Dηζ = (2)
−1
(18)
where ζ ∈ [1, 2γ ], η ∈ [1, Γ], γ is the number of inputs
to the decoder, Γ is the number of decoders, and N is
the number of local neighbors of LBP binary code. The
dimension of feature vector computed using η th decoder,
i.e., Dη is computed as follows,

Dη = (2)(γ) × ((2)

(N )

− 1)

(19)

Similarly, the dimension of final feature vector (D) is given
as,
(N )
D = Γ × (2)(γ) × ((2)
− 1)
(20)
In this work, the default values of Γ, γ and N are 2, 3 and 8
respectively. Thus, the dimension of the final feature vector
is 2 × (2)(3) × ((2)(8) − 1) = 2 × 8 × 256 = 16 × 256.
In order to make the feature vector invariant against the
image resolution, it is normalized as follows,

ξ(µ)
ξnorm (µ) = PD
τ =1 ξ(τ )

(21)

for ∀ µ ∈ [1, D]. The normalized version of feature vectors
is used in the experiments.

Color FDLBP Descriptor

In previous sub-sections the frequency decoded local binary pattern (FDLBP) feature vector is computed over the
grayscale image. The mdLBP is proposed for the color
images [21]. In order to show the significance of frequency
decoder over color channel decoder, the color FDLBP (i.e.,
cFDLBP) and frequency mdLBP (i.e., FmdLBP) are also
proposed in this paper. The cFDLBP computes the FDLBP
feature vector over each color channel independently. Thus,
six frequency decoder is used with two per color channel.
Finally, FDLBP feature vectors of all frequency decoders
are concatenated to find the single cFDLBP feature vector.
Whereas, on the other hand, the FmdLBP uses the color
decoder of original mdLBP [21]. Total five color decoders
corresponding to five filters are used with three inputs each.
The three inputs to a color decoder are from the three color
channels respectively filtered with a particular filter.

3

E XPERIMENTAL S ETUP

The experiments using the framework of image retrieval are
conducted in this paper. The face retrieval using proposed
frequency decoded local binary pattern (FDLBP) descriptor
is depicted in Fig. 5. For a query face image, the top
matching face images are retrieved from a database using
FDLBP descriptor. The FDLBP descriptor is computed over
query face image and all database face images. The similarity scores are calculated between the LDRP descriptors of
query and database images with the help of some distance
measure techniques. Note that high similarity score (i.e.,
low distance) between two descriptors indicates that the
corresponding faces are more similar and vice-versa.
3.1

Distances Measures

For image matching, the performance is highly dependent
upon the distance measures used for finding the similarity
scores. Based on the best similarity scores, top n number
of faces are retrieved. In most of the experiments, the Chisquare distance measure is adapted, whereas other distances
like Euclidean, Cosine, Earth Mover Distance (Emd), L1, and
D1 are also experimented with the proposed descriptor to
find its suitability [19], [21].

FREQUENCY DECODED LOCAL BINARY PATTERN

3.2

7

Evaluation Criteria

The precision, recall, f-score, and retrieval rank metrics are
generally used to evaluate the image retrieval algorithms.
In order to judge the performance over a database, the
metrics are computed by converting all the images of that
database as the query image one by one. The average
retrieval precision (ARP) and average retrieval rate (ARR)
over whole database are calculated by taking the average of
mean precision (MP) and mean recall (MR) of each category
of that database respectively. The MP and MR for a category
are computed by taking the mean of precision and recall by
turning all of the images in that category as the query image
one by one respectively. The precision (P r) and recall (Re)
for a query image is calculated as follows,

#Correct Retrieved Images
#Retrieved Images
#Correct Retrieved Images
Re =
#Similar Images In Database

Pr =

(22)

The F-score is computed from the ARP and ARR values as
follows,
ARP × ARR
F − score = 2 ×
(23)
ARP + ARR
The average normalized modified retrieval rank (ANMRR)
metric is also used to test the rank of correctly retrieved faces
[50]. The higher value of ARP, ARR and F-Score indicates
the better retrieval performance and vice-versa, whereas
the lower value of ANMRR indicates the better retrieval
performance and vice-versa.
3.3

Databases Used

In this paper, four challenging face databases are considered to reveal the performance of the proposed FDLBP
descriptor. The PaSC [51], LFW [49], PubFig [52], and ESSEX
[53] face databases are used for the retrieval experiments.
All these databases are having the color images that are
converted into the grayscale except for the color descriptors.
The face regions in the images are detected and cropped by
using Viola Jones object detection method [54]. All the face
images are down-sampled to 64 × 64 resolution. The PaSC
still images face database is one of the appealing databases
and have the difficulties such as blur, pose, and illumination
[51]. The 293 subjects with total 9376 images (i.e., 32 images
per subject) are present in this database. After face detection,
8718 faces left in the PaSC database that are successfully
detected using Viola Jones detector. The unconstrained face
retrieval is very challenging and desirable in the current
scenario. The LFW and PubFig databases are having the
images from the Internet. These images are captured in a totally unconstrained manner without subjects cooperations.
The variations like pose, lighting, expression, scene, camera,
etc. are part of these databases. The LFW database [49] is
used in this work for the experiments. In the image retrieval
framework, it is required to retrieve more than one (typically
5, 10, etc.) best matching images. So, it is important that
the sufficient number of images should be available in each
category of the database. Thus, the subjects having at least
20 images in LFW database are considered. Total 2984 face
images from 62 individuals are present in the LFW database
used in this paper. The Public Figure database (i.e., PubFig)

has the images from 60 subjects with 6472 number of total
images [52]. The images are downloaded from the Internet
directly following the urls given in this database (dead urls
are removed). The ESSEX face database is a very challenging
database with a lot of variations like background, scale,
illumination, blur, and extreme variation of expressions [53].
Total 7740 faces are present from 392 subjects with nearly 20
images per subject in ESSEX database.

4

FACE R ETRIEVAL E XPERIMENTAL R ESULTS

In this section, the experimental results are reported and
discussed. First of all, the retrieval results using proposed
descriptor are compared with the state-of-the-art descriptors. Second, the performance of the proposed frequency
decoder is tested with different combinations of frequency
decoder. Third, the suitability of distance measures are
experimented with the proposed descriptor. Forth, the color
channels are used with the frequency decoder to test the
performance over the color face images.
4.1

Results Comparison

The retrieval results are computed using ARP(%), ARR(%),
F-score(%) and ANMRR(%) metrics over four challenging
constrained and unconstrained face databases including
PaSC, LFW, PubFig and ESSEX. The results of the proposed
FDLBP descriptor are compared with the existing state-ofthe-art descriptors such as LBP [29], LDP [33], SOBEL LBP
[43], SLBP [44], BoF LBP [20], LDGP [41], LVP [37], and
SVD S LBP [45]. Note that except BoF LBP descriptor,
rest of descriptors are proposed for the face images. It is
also worth to note that SOBEL LBP, SLBP, BoF LBP and
SVD S LBP descriptors are using some filters for the preprocessing.
Fig. 6 demonstrate the ARP, ARR, F-score and ANMRR
plots against the number of retrieved images (n). The 1st ,
2nd , 3rd and 4th rows correspond to the results over PaSC,
LFW, PubFig and ESSEX databases respectively. The 1st ,
2nd , 3rd and 4th columns have the results using ARP,
ARR, F-score and ANMRR metric respectively. The x-axis
shows the number of retrieved images (n), whereas the yaxis shows the ARP, ARR, F-score and ANMRR values in
percentage. It is clear from the results that FDLBP is having
the best result among all descriptors as the values of ARP,
ARR and F-score are highest and the value of ANMRR is
lowest for the FDLBP descriptor. The PaSC and ESSEX are
the constrained databases, but having many variations like
blur, scale, illumination, expressions, and pose. The LFW
and PubFig databases are having the unconstrained images.
The results of Fig. 6 confirm the improved performance of
the proposed frequency decoder based descriptor FDLBP.
4.2

Experiment with Different Decoder Combinations

In the FDLBP descriptor, by default two decoders are
used with three inputs each. The first decoder has the
inputs as the outputs of Fa , Fhv and Fd filters and represented in this subsection as (Fa , Fhv , Fd ). Similarly the
second decoder has the inputs coming from Fa , Fsv and
Fsh filters and represented as (Fa , Fsv , Fsh ). So, in this

FREQUENCY DECODED LOCAL BINARY PATTERN
PaSC

PaSC

40

9
LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

5
4

20
3
2

3

4

5

6

7

8

9

10

2

3

6

7

8

9

10

40
20
4

5

6

7

5

4

8

9

10

4

2

3

4

5

6

7

8

9

(e) ARP over LFW Database

(f) ARR over LFW Database

PubFig

40

7

7

3

4

5

6

7

8

9

10

2

3

4

2

3

Number of Top Matches

4

5

6

7

8

9

5

6

7

60

30
20

2

3

2

9

3

4

5

6

7

8

9

6

7

Number of Top Matches

1

8

9

10

1

2

3

4

5

6

7

8

9

PaSC

2

3

4

5

6

7

8

9

10

PubFig

60

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

40
20
0

10

1

2

over

30
20

10

4

5

PubFig(l) ANMRR
Database

6

7

8

9

10

over

PubFig

ESSEX
30

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

40

3

Number of Top Matches

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

20

10

0

10
5

10

Number of Top Matches

10
4

9

over

20

10

ESSEX

50

50
1

8

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

80

5

(k)
F-Score
Database

F-Score (%)

70

ARR (%)

80

8

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

60

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

40
LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

7

40

Number of Top Matches

ESSEX

90

6

60

Number of Top Matches

ESSEX

5

LFW

6

10

(i) ARP over PubFig Database (j) ARR over PubFig Database
100

4

(d)
ANMRR
Database

PubFig

7

1
1

3

(g) F-Score over LFW Database (h) ANMRR over LFW Database

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

4

2

Number of Top Matches

ANMRR (%)

3

0
10

Number of Top Matches

2

20
2

9

80

4
1

8

0
1

F-Score (%)

ARR (%)

60

6

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

8

10

PubFig

5
LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

5

6
1

Number of Top Matches

80

ARP (%)

3

LFW

Number of Top Matches

100

2

9

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

3
3

20

Number of Top Matches

F-Score (%)

60

40

1
1

LFW

ARR (%)

ARP (%)

5

(b) ARR over PaSC Database (c) F-Score over PaSC Database
6

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

80

ARP (%)

4

LFW

2

7

Number of Top Matches

(a) ARP over PaSC Database

1

8

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

60

6
1

Number of Top Matches

100

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

ANMRR (%)

1

80

ANMRR (%)

60

6

ARR (%)

80

PaSC

PaSC

ANMRR (%)

LBP
LDP
SOBEL_LBP
SLBP
BoF_LBP
LDGP
LVP
SVD_S_LBP
FDLBP

F-Score (%)

100

ARP (%)

8

1
1

Number of Top Matches

2

3

4

5

6

7

8

9

10

Number of Top Matches

(m) ARP over ESSEX Database (n) ARR over ESSEX Database (o) F-Score over ESSEX Database

2

3

4

5

6

7

8

9

10

Number of Top Matches

(p) ANMRR
Database

over

ESSEX

Fig. 6: The results over PaSC (1st row), LFW (2nd row), PubFig (3rd row), and ESSEX (4th row) face databases in terms of
the ARP (1st column), ARR (2nd column), F-Score (3rd column), and ANMRR (4th column) vs number of retrieved images.
The x-axis shows the number of retrieved images (n). The y-axis represents the ARP, ARR, F-score, and ANMRR metric
values.
subsection, < (Fa , Fhv , Fd ), (Fa , Fsv , Fsh ) > is the representation of the default decoder setting. For example,
< (Fa , Fhv ), (Fa , Fd ), (Fa , Fsv ), (Fa , Fsh ) > represents four
decoders with each having two inputs. In this experiment, some other frequency decoder combinations are also
tested over each database as illustrated in Table 1. The
top n = 5 face images are retrieved and ARP is reported.
It is found that the performance of FDLBP is generally
better when low-pass and high-pass information are decoded except over ESSEX database. It is also pointed out
through this experiment that among the high-pass filters
the Fhv and Fd filters are more important than the Fsv
and Fs h filters. Though, the results in previous subsection are computed for one combination of decoders (i.e.,
< (Fa , Fhv , Fd ), (Fa , Fsv , Fsh ) >), its performance can be

further boosted by opting the right decoder combination for
a particular database.

4.3

Experiment with Different Similarity Measures

The Chi-square distance is used for the similarity measures
in the rest of the experiments. In this experiment, the performance of FDLBP descriptor is investigated with different
distances, such as Euclidean, Cosine, Emd, L1, D1, and Chisquare. The ARP for n = 5 number of retrieved images using the FDLBP descriptor with different similarity measures
are summarized in Table 2. It is noticed that the Chi-square
distance is better suited with the FDLBP descriptor. The D1
distance is the second best performing similarity measure.

FREQUENCY DECODED LOCAL BINARY PATTERN

9

TABLE 1: The ARP(%) using proposed FDLBP descriptor with different decoder combinations over the PaSC, LFW, PubFig,
and ESSEX face databases. The number of retrieved images (n) is 5 in this experiment. The highest ARP values are expressed
in bold for each database. The < (Fa , Fhv , Fd ) > represents that only one decoder is used in this case with three inputs
as the filtered images obtained using Fa , Fhv , and Fd filters respectively. The < (Fhv , Fd ), (Fsv , Fsh ) > represents that
two decoders with two inputs each are used in this case with filtered images using Fhv and Fd filters in first decoder and
filtered images using Fhv and Fd filters in second decoder.
Face Databases

Decoder Combination

< (Fa , Fhv , Fd ) >
< (Fa , Fsv , Fsh ) >
< (Fhv , Fd ), (Fsv , Fsh ) >
< (Fa , Fhv ), (Fa , Fd ), (Fa , Fsv ), (Fa , Fsh ) >
< (Fa , Fhv , Fsv ), (Fa , Fd , Fsh ) >
< (Fa , Fhv , Fd ), (Fa , Fsv , Fsh ) >

TABLE 2: The ARP(%) using proposed FDLBP descriptor
with Euclidean, Cosine, Emd, L1, D1, and Chi-square distance measures over the PaSC, LFW, PubFig, and ESSEX
face databases. The number of retrieved images (n) is 5 in
this experiment. The highest ARP values are expressed in
bold for each database.
Face Databases

Distance

Euclidean
Cosine
Emd
L1
D1
Chi-square

PaSC

LFW

PubFig

ESSEX

26.63
27.02
23.78
29.79
29.95
32.00

28.73
28.93
26.09
31.47
31.58
32.46

35.50
35.94
32.15
39.11
39.19
41.73

92.33
92.29
84.27
93.86
93.86
93.77

TABLE 3: The ARP(%) using mdLBP, FDLBP, cFDLBP, and
FmdLBP descriptors over the PaSC, LFW, PubFig, and ESSEX face databases. The number of retrieved images (n) is 5
in this experiment. The highest ARP values are expressed in
bold for each database.
Face Databases

Descriptors

mdLBP
FDLBP
cFDLBP
FmdLBP

4.4

PaSC

LFW

PubFig

ESSEX

26.55
32.00
32.72
27.11

29.65
32.46
34.46
33.04

34.47
41.73
42.42
38.71

91.52
93.77
94.46
92.75

PaSC

LFW

PubFig

ESSEX

32.53
28.92
24.56
32.59
31.64
32.00

32.36
30.02
28.75
32.31
32.76
32.46

41.17
38.22
36.49
41.52
41.68
41.73

95.05
90.32
79.74
94.15
92.86
93.77

using color decoder using each filter independently. The
retrieval results in terms of the ARP for n = 5 number of
retrieved images over the PaSC, LFW, PubFig, and ESSEX
face databases are reported in Table 3. The ARP of FDLBP
is greater than the ARP of mdLBP and the ARP of cFDLBP
is greater than the ARP of FmdLBP. Thus, it can be easily
observed that the frequency decoder is far better than the
color decoder over face databases.

5

C ONCLUSION

In this paper, a frequency decoder based local descriptor
is proposed for the face representation. The descriptor first
computes the low-pass and high-pass filtered images of the
input face images. Then, it uses two decoders with three input channels each. The inputs to the decoder are computed
as the binary codes with the help LBP operator over the
filtered images. The feature vectors are generated over each
output image of both decoders and finally combined into
a single frequency decoded local binary pattern (FDLBP)
feature vector. The proposed descriptor is tested in the
image retrieval framework over four very challenging constrained and unconstrained face databases. The results are
compared with the state-of-the-art descriptors mostly face
descriptors. The experimental results reveal the superiority
of the proposed descriptor. The FDLBP descriptor is also
tested with different frequency decoder combinations and it
is suggested that its performance can be further improved
by choosing the most suitable decoder combination over a
face database. It is also noticed that Chi-square distance is
a better choice for the similarity measure. In case of color
face images, it is suggested to use the frequency decoder
over color channels instead of the color decoder over filtered
images.

Experiment with Color Channels

In order to demonstrate the superiority of frequency decoder over a color decoder, this experiment is conducted.
Following four descriptors computed in different scenario
are considered: 1) original mdLBP using color decoder
without filtering [21], 2) FDLBP using frequency decoder
without color information, 3) cFDLBP using frequency decoder over each channel independently, and 4) FmdLBP

R EFERENCES
[1]

[2]

J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust
face recognition via sparse representation,” IEEE transactions on
pattern analysis and machine intelligence, vol. 31, no. 2, pp. 210–227,
2009.
L. Ding, X. Ding, and C. Fang, “Continuous pose normalization
for pose-robust face recognition,” IEEE Signal Processing Letters,
vol. 19, no. 11, pp. 721–724, 2012.

FREQUENCY DECODED LOCAL BINARY PATTERN

[3]
[4]

[5]
[6]
[7]
[8]

[9]

[10]
[11]

[12]
[13]
[14]

[15]

[16]

[17]
[18]

[19]

[20]

[21]
[22]
[23]
[24]

10

C. Ding, C. Xu, and D. Tao, “Multi-task pose-invariant face recog[25] S. R. Dubey, S. K. Singh, and R. K. Singh, “Local diagonal exnition,” IEEE Transactions on Image Processing, vol. 24, no. 3, pp.
trema pattern: a new and efficient feature descriptor for ct image
980–993, 2015.
retrieval,” IEEE Signal Processing Letters, vol. 22, no. 9, pp. 1215–
1219, 2015.
A. Punnappurath, A. N. Rajagopalan, S. Taheri, R. Chellappa, and
G. Seetharaman, “Face recognition across non-uniform motion
[26] ——, “Local bit-plane decoded pattern: A novel feature descriptor
blur, illumination, and pose,” IEEE Transactions on Image Processfor biomedical image retrieval,” IEEE Journal of Biomedical and
ing, vol. 24, no. 7, pp. 2067–2082, 2015.
Health Informatics, vol. 20, no. 4, pp. 1139–1147, 2016.
W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face
[27] ——, “Local wavelet pattern: A new feature descriptor for image
recognition: A literature survey,” ACM computing surveys (CSUR),
retrieval in medical ct databases,” IEEE Transactions on Image
vol. 35, no. 4, pp. 399–458, 2003.
Processing, vol. 24, no. 12, pp. 5892–5903, 2015.
X. Zhang and Y. Gao, “Face recognition across pose: A review,”
[28] ——, “Novel local bit-plane dissimilarity pattern for computed
Pattern Recognition, vol. 42, no. 11, pp. 2876–2896, 2009.
tomography image retrieval,” Electronics Letters, vol. 52, no. 15,
C. Ding and D. Tao, “A comprehensive survey on pose-invariant
pp. 1290–1292, 2016.
face recognition,” ACM Transactions on intelligent systems and tech[29] T. Ahonen, A. Hadid, and M. Pietikainen, “Face description with
nology (TIST), vol. 7, no. 3, p. 37, 2016.
local binary patterns: Application to face recognition,” IEEE transY. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closactions on pattern analysis and machine intelligence, vol. 28, no. 12,
ing the gap to human-level performance in face verification,” in
pp. 2037–2041, 2006.
Proceedings of the IEEE conference on computer vision and pattern
[30] D. Huang, C. Shan, M. Ardabilian, Y. Wang, and L. Chen, “Local
recognition, 2014, pp. 1701–1708.
binary patterns and its application to facial image analysis: a
F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified
survey,” IEEE Transactions on Systems, Man, and Cybernetics, Part
embedding for face recognition and clustering,” in Proceedings
C (Applications and Reviews), vol. 41, no. 6, pp. 765–781, 2011.
of the IEEE Conference on Computer Vision and Pattern Recognition, [31] B. Yang and S. Chen, “A comparative study on local binary pattern
2015, pp. 815–823.
(lbp) based face recognition: Lbp histogram versus lbp image,”
Z. Cao, Q. Yin, X. Tang, and J. Sun, “Face recognition with
Neurocomputing, vol. 120, pp. 365–379, 2013.
learning-based descriptor,” in Computer Vision and Pattern Recogni- [32] X. Tan and B. Triggs, “Enhanced local texture feature sets for face
tion (CVPR), 2010 IEEE Conference on. IEEE, 2010, pp. 2707–2714.
recognition under difficult lighting conditions,” IEEE transactions
L. Wolf, T. Hassner, and Y. Taigman, “Effective unconstrained
on image processing, vol. 19, no. 6, pp. 1635–1650, 2010.
face recognition by combining multiple descriptors and learned
[33] B. Zhang, Y. Gao, S. Zhao, and J. Liu, “Local derivative pattern
background statistics,” IEEE transactions on pattern analysis and
versus local binary pattern: face recognition with high-order local
machine intelligence, vol. 33, no. 10, pp. 1978–1990, 2011.
pattern descriptor,” IEEE transactions on image processing, vol. 19,
Z. Lei, M. Pietikäinen, and S. Z. Li, “Learning discriminant face
no. 2, pp. 533–544, 2010.
descriptor,” IEEE Transactions on Pattern Analysis and Machine [34] N.-S. Vu, H. M. Dee, and A. Caplier, “Face recognition using the
Intelligence, vol. 36, no. 2, pp. 289–302, 2014.
poem descriptor,” Pattern Recognition, vol. 45, no. 7, pp. 2478–2488,
J. Lu, V. E. Liong, X. Zhou, and J. Zhou, “Learning compact binary
2012.
face descriptor for face recognition,” IEEE transactions on pattern [35] N.-S. Vu and A. Caplier, “Enhanced patterns of oriented edge
analysis and machine intelligence, vol. 37, no. 10, pp. 2041–2056, 2015.
magnitudes for face recognition and image matching,” IEEE TransJ. Lu, V. Erin Liong, and J. Zhou, “Simultaneous local binary
actions on Image Processing, vol. 21, no. 3, pp. 1352–1365, 2012.
feature learning and encoding for face recognition,” in Proceedings
[36] N.-S. Vu, “Exploring patterns of gradient orientations and magof the IEEE International Conference on Computer Vision, 2015, pp.
nitudes for face recognition,” IEEE Transactions on Information
3721–3729.
Forensics and Security, vol. 8, no. 2, pp. 295–304, 2013.
T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray[37] K.-C. Fan and T.-Y. Hung, “A novel local pattern descriptorlocal
scale and rotation invariant texture classification with local binary
vector pattern in high-order derivative space for face recognition,”
patterns,” IEEE Transactions on pattern analysis and machine intelliIEEE transactions on image processing, vol. 23, no. 7, pp. 2877–2891,
gence, vol. 24, no. 7, pp. 971–987, 2002.
2014.
S. R. Dubey, “Robust image feature description, matching
[38] C. M. PVSSR et al., “Dimensionality reduced local directional patand applications,” Ph.D. dissertation, IIIT Allahabad,
tern (dr-ldp) for face recognition,” Expert Systems with Applications,
https://sites.google.com/site/shivram1987/publications/SRD PhD Thesis.pdf,
vol. 63, pp. 66–73, 2016.
6 2016.
[39] C. Ding, J. Choi, D. Tao, and L. S. Davis, “Multi-directional
M. Pietikäinen, A. Hadid, G. Zhao, and T. Ahonen, “Local binary
multi-level dual-cross patterns for robust face recognition,” IEEE
patterns for still images,” in Computer vision using local binary
transactions on pattern analysis and machine intelligence, vol. 38, no. 3,
patterns. Springer, 2011, pp. 13–47.
pp. 518–531, 2016.
S. R. Dubey, S. K. Singh, and R. K. Singh, “Rotation and illumina[40]
C.-X. Ren, Z. Lei, D.-Q. Dai, and S. Z. Li, “Enhanced local gradient
tion invariant interleaved intensity order-based local descriptor,”
order features and discriminant analysis for face recognition,”
IEEE Transactions on Image Processing, vol. 23, no. 12, pp. 5323–5333,
IEEE transactions on cybernetics, vol. 46, no. 11, pp. 2656–2669, 2016.
2014.
[41]
S. Chakraborty, S. K. Singh, and P. Chakraborty, “Local directional
S. Murala, R. Maheshwari, and R. Balasubramanian, “Local tetra
gradient pattern: a local descriptor for face recognition,” Multimepatterns: a new feature descriptor for content-based image redia Tools and Applications, vol. 76, no. 1, pp. 1201–1216, 2017.
trieval,” IEEE Transactions on Image Processing, vol. 21, no. 5, pp.
[42] B. Ryu, A. R. Rivera, J. Kim, and O. Chae, “Local directional
2874–2886, 2012.
ternary pattern for facial expression recognition,” IEEE TransacS. R. Dubey, S. K. Singh, and R. K. Singh, “Boosting local binary
tions on Image Processing, 2017.
pattern with bag-of-filters for content based image retrieval,” in
[43] S. Zhao, Y. Gao, and B. Zhang, “Sobel-lbp,” in Image Processing,
Electrical Computer and Electronics (UPCON), 2015 IEEE UP Section
2008. ICIP 2008. 15th IEEE International Conference on. IEEE, 2008,
Conference on. IEEE, 2015, pp. 1–6.
pp. 2144–2147.
——, “Multichannel decoded local binary patterns for contentbased image retrieval,” IEEE Transactions on Image Processing, [44] K. Jeong, J. Choi, and G.-J. Jang, “Semi-local structure patterns for
robust face detection,” IEEE Signal Processing Letters, vol. 22, no. 9,
vol. 25, no. 9, pp. 4018–4032, 2016.
pp. 1400–1403, 2015.
S. Liao, M. W. Law, and A. C. Chung, “Dominant local binary
[45] S. R. Dubey, S. K. Singh, and R. K. Singh, “Local svd based nir face
patterns for texture classification,” IEEE transactions on image proretrieval,” Journal of Visual Communication and Image Representation,
cessing, vol. 18, no. 5, pp. 1107–1118, 2009.
2017.
L. Liu, Y. Long, P. W. Fieguth, S. Lao, and G. Zhao, “Brint: binary
[46] ——, “Local neighbourhood-based robust colour occurrence derotation invariant and noise tolerant texture classification,” IEEE
scriptor for colour image retrieval,” IET Image Processing, vol. 9,
Transactions on Image Processing, vol. 23, no. 7, pp. 3071–3084, 2014.
no. 7, pp. 578–586, 2015.
X. Qi, R. Xiao, C.-G. Li, Y. Qiao, J. Guo, and X. Tang, “Pairwise
rotation invariant co-occurrence local binary pattern,” IEEE Trans[47] ——, “Rotation and scale invariant hybrid image descriptor and
actions on Pattern Analysis and Machine Intelligence, vol. 36, no. 11,
retrieval,” Computers & Electrical Engineering, vol. 46, pp. 288–302,
pp. 2199–2213, 2014.
2015.

FREQUENCY DECODED LOCAL BINARY PATTERN

[48] ——, “A multi-channel based illumination compensation mechanism for brightness invariant image retrieval,” Multimedia Tools
and Applications, vol. 74, no. 24, pp. 11 223–11 253, 2015.
[49] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled
faces in the wild: A database for studying face recognition in
unconstrained environments,” Technical Report 07-49, University
of Massachusetts, Amherst, Tech. Rep., 2007.
[50] K. Lu, N. He, J. Xue, J. Dong, and L. Shao, “Learning view-model
joint relevance for 3d object retrieval,” IEEE Transactions on Image
Processing, vol. 24, no. 5, pp. 1449–1459, 2015.
[51] J. R. Beveridge, P. J. Phillips, D. S. Bolme, B. A. Draper, G. H.
Givens, Y. M. Lui, M. N. Teli, H. Zhang, W. T. Scruggs, K. W.
Bowyer et al., “The challenge of face recognition from digital pointand-shoot cameras,” in Biometrics: Theory, Applications and Systems
(BTAS), 2013 IEEE Sixth International Conference on. IEEE, 2013,
pp. 1–8.
[52] N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar, “Attribute
and simile classifiers for face verification,” in Computer Vision, 2009
IEEE 12th International Conference on. IEEE, 2009, pp. 365–372.
[53] L. Spacek, “University of essex face database.” [Online]. Available:
http://cswww.essex.ac.uk/mv/allfaces/
[54] P. Viola and M. Jones, “Rapid object detection using a boosted
cascade of simple features,” in Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society
Conference on, vol. 1. IEEE, 2001, pp. I–I.

Shiv Ram Dubey has been with the Indian Institute of Information Technology, Sricity (IIIT Sricity) since June 2016, where he is currently the
Assistant Professor of Computer Science and
Engineering.
He received the Ph.D. degree in Computer
Vision and Image Processing from Indian Institute of Information Technology, Allahabad (IIIT
Allahabad) in 2016. Before that, from August
2012-Feb 2013, he was a Project Officer in the
Computer Science and Engineering Department
at Indian Institute of Technology, Madras (IIT Madras). He was a recipient of several awards including the Early Career Research Award
from SERB, Govt. of India and NVIDIA GPU Grant from NVIDIA. He
received Certificate of Reviewing Awards from Biosystems Engineering
and Computers in Biology and Medicine, Elsevier in 2015 and 2016
respectively. He received the Best Paper Award in IEEE UPCON 2015,
a prestigious conference of IEEE UP Section. His research interest
includes Computer Vision, Deep Learning, Image Processing, Image
Feature Description, Image Matching, Content Based Image Retrieval,
Medical Image Analysis and Biometrics.

11

