A Double Filtered GIST Descriptor for Face Recognition

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 79 (2016) 533 – 542

7th International Conference on Communication, Computing and Virtualization 2016

A Double Filtered GIST Descriptor for Face Recognition Vinay.A, Gagana B, Vinay.S.Shekhar, Anil B, K N Balasubramanya Murthy, Natarajan S PES University, 100 Feet Ring Road, BSK III Stage, Bengaluru – 560085, Karnataka

Abstract We are in the era of wearable technologies, biometrics and multi-factor authentication, where one’s face is increasingly becoming a digital identifier for access control and authentication. Compared to the other biometrics such as Fingerprint, Iris and Palm print, Face Recognition (FR) has the distinctness of being non-intrusive, and has hence garnered substantial mainstream attention. As the devices that incorporate FR are evolving into miniaturization, there is a need to develop more robust algorithms that are computationally less expensive. Hence, in an effort to provide a computationally effective FR methodology, we extend the costeffective GIST descriptor that was designed primarily for object recognition, to be commensurate with FR. This paper proposes a double filtered GIST based descriptor for FR that embodies certain inventive preprocessing steps such as edge detection via Prewitt descriptor, DCT and IDCT transformation to reduce noise prior to feature description with GIST. We will demonstrate by performing extensive experimentations on the ORL and IIT-K face databases that the proposed methodology is capable of effectively performing FR, even in the presence of sharp variations in a number of crucial FR parameters among the faces being compared. © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license © 2016 The Authors. Published by Elsevier B.V. (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review underresponsibility responsibility Organizing Committee of ICCCV Peer-review under of of thethe Organizing Committee of ICCCV 20162016. Keywords: Type your keywords here, separated by semicolons ;

1. Main text Pattern recognition has played a vital role in making humans socially complex and technically advanced; Scientists and engineers have long been striving to emulate the pattern recognition ability of humans to rapidly and accurately recognize patterns. Among the wide gamut of research in this field, Face Recognition (FR)has received consistent mainstream attention. FR [13] is not a new phenomenon. The first generation of face recognition systems can be traced back to the 1960s. Although, the research during the earlier period was passive, today the technology behind such systems is more automatic, much more mathematical and underpinned by sophisticated computing abilities. The earlier models mandated a greater level of human intervention. The contemporary FR systems are ubiquitous as they form a critical part of the security and consumer infrastructure. The technology behind face recognition systems is entering

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the Organizing Committee of ICCCV 2016 doi:10.1016/j.procs.2016.03.068

534

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

uncharted territories. FR is being deployed all forms of everyday applications; for instance, social networks like Facebook is using FR to tag “friends”. Similarly Google utilizes to perform image-based searches; Microsoft employs FR in Windows 10 OS for authentication purpose; Apple is treading new grounds in automating the sharing of photos with ‘tagged’ friends to its iOS users. Furthermore, it is reported that nearly thirty churches have deployed the FR software Churchix to identify persons attending meetings. In terms of security, a team from Tsinghua University and Tzekwan Technology in China has developed the world’s first face recognition ATM where in cameras deployed in the ATM counter captures images of the end user and compares them with the ID photos for verification. Such systems are expected to minimize the risk of illegal withdrawals. Since, FR systems operate by matching the given probe face image against the faces in the database, the speed of recognition depends on the size of the database, among other factors. In order to effective perform FR on smaller machines such as smart-watches, smart phones and so on, it is necessary to employ FR algorithms that are computationally cost-effective, as the GPU and processing power of these devices is limited. Hence in an effort to develop such a system, we utilize the GIST [8] technique, which was developed by Oliva et al., for feature description, and make several modifications to make it commensurate with faces. GIST has been demonstrated to be considerably cost-effective as it matches without taking the similarity/dissimilarity of individual pixels in to account. Furthermore, we add a pre-filtering phase in order reduce noise prior to carrying out feature description with GIST. The pre-filtering phase consists of edge detection with Prewitt descriptor and transformation with Discrete Cosine Transformation (DCT) and IDCT (Inverse Discrete Cosine Transform). Subsequently, the matching of the probe and training image is performed using Sum of Squared Differences (SSD). We have conducted our experimentations on two prominent databases, ORL [11] and IIT-Kanpur [14] under 4 different database configurations in order to ensure that a wide variety of variations in parameters such as scale, zoom, illumination, expression, pose etc. exists, in order to establish the robustness of the proposed methodology in real-time scenarios. The rest of the paper is organized as follows: Section 2 details the background of the related methodologies; Section 3 furnishes the proposed methodology, Section 4 deals with matching of the descriptors, Section 5 proffers the experimental results and finally, Section 6 provides a discussion of the proposed approach and outlines future work. 2. Background 2.1. Facial Feature Descriptors Typically face recognition systems are large-scale systems that require efficient image indexing techniques for description, storage and retrieval of billons of images. Facebook estimates that its users upload approximately 350 million photos every day, bringing their database count to 280 billion photos. Extraction and storage of all features contained in facial image is unfeasible. Similarly, extracting and storing fewer features will not suffice the matching process. Thus, it is very crucial to describe face images as densely as possible and develop matching system that is more efficient. Recently, the concept of image descriptors (interest point descriptors) is gaining wider scope in image processing and related domains. Image descriptors can be classified into two categories, Meta information based and image based. Some of the Meta information based image descriptors include EXIF-data and Geo Tags. On the other hand image based descriptors can be further classified into pixel based image descriptors that include tiny image, color histogram, text on histogram, etc. and structure based image descriptors like Fourier, gradient, geometric and Gist. Some of the popular FR feature extractors are SIFT [22], SURF [21], ORB [18], FAST [20], FREAK [23], BRIEF[19] and so on. They have been demonstrated to be effective even when there is a wide variation in certain crucial parameters such has scale, zoom, rotation, illumination, pose, expression etc. among the faces being compared. A thorough evaluation of prominent face feature descriptors is provided in [15][16].

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

535

2.2. GIST Descriptor GIST [6][7][8] is an image descriptor developed by olive et al. [8] that is based on a very low dimensional representation a given image which is dubbed the Spatial Envelope. They proposed a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) in order to represent the dominant spatial structure of a given scene. Subsequently, they demonstrated that these dimensions could be effectively estimated by utilizing the spectral and coarsely localized information. Their model generated a multidimensional space in which scenes that share membership in semantic categories such as streets, highways, coasts were projected closed together. The performance of the spatial envelope model proved that specific information about object shape or identity was not necessary for scene categorization and that the modeling holistic representation of the scene provided information about its probable semantic category. Oliva et al. [8] applied the methodology on object recognition .In our approach, we extend their philosophy to face recognition [12]. 3. Proposed Framework The proposed methodology, as described in Fig.1 involves inputting the probe and training images and subjecting them to the pre-filtering phase where they undergo preprocessing in the form of edge detection via Prewitt descriptor [3] and are further subjected to DCT transform and IDCT transform to carry out noise reduction (to ensure there are no false matches due to noise in the input face). We refrain from elaborating on the Edge Detection via Prewitt [3], DCT [4] and IDCT [5] methodologies, as they have been exhaustively detailed in several seminal works [3][24][25]. The conventional GIST [6][7][8] methodology as described in Oliva et al. [8], employs GIST to perform object recognition by using 8 orientations per scale and 5 scales (or blocks). In our experimentations, we use GIST to perform feature description and aim to make GIST commensurate with face recognition by using 6 orientations per scale and use 4 scales (blocks). Hence we have a 2 level envelope i.e. DCT/IDCT along with GIST as described in [8]. Furthermore, we carry out matching using SSD (Sum of Squared Differences) [9], its variants [26, 27] and we declare based on the distance the match /mismatch status based on the heuristic thresholds (determined experimentally).

Fig.1 Framework of the proposed Methodology The gist descriptor of a given facial image is obtained by analyzing the orientation and spatial frequency. The global descriptor operates by combining the amplitudes of the output of K Gabor filters at different scales S and orientations O [6][7][8]. In order reduce the size of the feature vector, each facial image in the filter output is resized to a size N*N blocks, where N is typically between 2 and 16, which yields a vector of dimension N*N*K i.e. N*N*S*O [8][12]. This dimension is further reduced by using Weighted Principal Component Analysis (WPCA). Typically, PCA seeks a projection that maximizes the sum whereas the WPCA seeks a projection that maximizes the weighted sum [10]. In our proposed system, for an input face image, a GIST descriptor is computed by:

536

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

1. 2. 3.

Convolving the image with 24 Gabor filters at 4 scales, 6 orientations, producing 24 feature maps of the same size of the input face image. Dividing each feature map into 16 regions using a 4x4 grid and then averaging the feature values within each region. Concatenating the 16 averaged values of all 24 feature maps, resulting in a 16x24=384 GIST descriptor.

Essentially, GIST summarizes the gradient information (scales and orientations) for different parts of an image, which yields a rough description (the gist) of the scene. The task of sifting through a database of millions of images requires substantial amount of computational effort and time. These are typically a couple of gigabytes (GBs)in size and the search speed also depends on the medium on which they are saved, in additional to requiring mutual comparisons. One important characteristic of GIST, which can yield faster computation, is that while comparing images, it does not matter whether the single pixels are similar or different. This was the rationale behind our employment of GIST [17].

Fig.1 Edge Orientation [17] The GIST descriptor of each image is pre-calculated [8][12];and the image is divided into tiles and for each of the tile edges, the different frequencies and angular orientations are pertinently detected and represented by a number (as shown in Fig.2). For the comparison of two images, the aforementioned numbers corresponding to each are compared component-by-component and subsequently summed up in order to yield a single value that describes the similarity between the two images [6][7][12]. The GIST descriptors for the images in the database do not change over time. Hence it is reasonable to pre-calculate them and read them instantly at our convenience. In our approach, the GIST algorithm only utilizes the grayscale information about the images [17]. Constructing a voluminous database may consume a couple of hours on a modern CPU with multiple cores. But with GIST, browsing the gallery for similarities for a given input image takes only about a few minutes. 4. Matching of the Descriptors 4.1. Sum of Squared Differences SSD [1][2] is a form of Correlation based matching, which generally tends to provide dense depth maps by computing the disparity at each pixel within a given neighborhood. This is performed by taking a square window of certain size around the pixel of interest in the reference image and subsequently locating the homologous pixel within the window in the target image, simultaneously moving along the corresponding scan line. This is done with a purpose to locate the corresponding pixel within a certain disparity range d (d E [0,….,dmax]) that tends to minimize the associated error and maximizes the similarity [1]. Overall, the matching process involves the calculation of the similarity measure for each disparity value, which has a subsequent aggregation and optimization step. All of these steps require substantial processing power and there are considerable speed-performance advantages in optimizing the matching algorithm.

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

537

The matching process involves choosing either the left image as the reference i.e. left-to-right matching (direct matching) or right image as the reference i.e. right-to-left matching (reverse matching) [1][2]. Here, the differences are squared and aggregated within a given square window and consequently optimized using the winner-take-all (WTA) strategy. This measure demonstrates a higher computational complexity when compared with the other similarity measures as it involves numerous multiplication operations [26][27]. 4.2. Other Similarity Measures Sum of Absolute Differences(SAD) [26][27] is another popular similarity measure which is computed via the subtraction of the pixels within a given square neighborhood between the reference image I1 and the target image I2which is subsequently followed by the aggregation of the absolute differences within the square window along with optimization using the WTA strategy. If the left and the right images are an exact match, then the resultant will be zero.

Normalized Cross Correlation (NCC) [26] involves more complexity than SAD and SSD algorithms, as it encompasses numerous multiplication, division and square root operations.

Sum of Squared Differences Right-to-Left (SSD R2L)[26]when employed with a descriptor, the end result would be dense disparity map, which can viewed as a sized array, where each of the elements of the array is a data structure that is composed of 3 values: Disparity value along the xx axis, Disparity value along the yy axis and the similarity measure value of the correspondent pair of pixels. Therefore, this data structure can be viewed as a set of 3 arrays. As there exists is a direct relation between the calculated correlation values and the achieved matching performance, the similarity measure array can be viewed as a condense map of the final result. Hence, it can be employed to select the pixel coordinates corresponding to the best match of the whole process. Other distance measures such as Zero-mean Absolute Differences (ZSAD), Locally Scaled Sum of Absolute Differences (LSAD), Zero-mean Sum of Squared Differences (ZSSD), Locally Scaled Sum of Squared Differences (LSSD), Normalized Cross Correlation (NCC) and Zero-mean Normalized Cross Correlation (ZNCC) were also experimented with to investigate as which one is more effective in terms of retrieval performance. An elaborate discussion of distinct between the aforementioned similarity measures along with an in-depth elaboration on their working can be found in Roma et al. [26] and Patil et al. [27]. 5. Results and Discussion This section provide the experimental results of our FR deliberations on the faces of the popular ORL[11] and IIT Kanpur [14] datasets. In our database design, we have four different configurations: GIST-1, IGIST-1 and GIST-2, IGIST-2.On the ORL database, GIST-1 and IGIST-1 had 230 / 400 images for testing and GIST-2 and IGST-2 had 180/400 images for testing. GIST indicates the basic GIST and IGIST is the proposed double filtered GIST. The rest images of the images were used for training. On the IITK database, GIST-1 and IGIST-1 had 336 (61 subjects *6 images per subject) / 671 (61X11) images for testing and GIST-2 and IGST-2 had 427 (61X7) /671 images for testing. The rest of the images were for training.

538

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

Table 1: ORL Dataset

SL

IMAGE

SSD Distance

Distance =0. 8771 (Bad Match, FP)

1.

2 Distance=0.54466 (Bad Match, TN)

3 Distance =0.1782 (Good Match, TP)

4 Distance = 0.0652 (Good Match, TP)

Table 2: Results over the ORL

Method

Sensitivity TPR

Specificity TNR

Precision PPV

Accuracy ACC

Recall

F1 Score

FAR

FRR

HTER

TER

ORL Dataset GIST-1

93.877551

82.35294

96.842105

92.173913

93.8776

95.3368

0.17647

0.06122

0.11885

0.18007

IGIST-1

94.9239

96.144

0.15152

0.05076

0.10114

0.1519

94.923858

84.84848

97.395833

93.478261

GIST-2

96

83.33333

96.644295

93.888889

96

96.3211

0.16667

0.04

0.10333

0.14333

IGIST-2

96.02649

86.2069

97.315436

94.444444

96.0265

96.6667

0.13793

0.03974

0.08883

0.12857

539

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

Table 3:IIT Kanpur Dataset

SL

IMAGE

SSD Distance

Distance = 0.1240 (Good Match, TP)

1.

2 Distance = 0.1168 (Good Match, FP)

3 Distance = 0.1779 (Bad Match, TN)

4 Distance = 0.2382 (Bad Match, FN)

Table 4: Results over the IIT Kanpur Datasets Method

Sensitivity TPR

Specificity TNR

Precision PPV

Accuracy ACC

GIST-1

94.42623

72.13115

94.42623

90.710383

IGIST-1

95.737705

73.77049

94.805195

GIST-2

94.219653

72.83951

IGIST-2

94.555874

75.64103

Recall

F1 Score

FAR

FRR

HTER

TER

94.4262

94.4262

0.27869

0.05574

0.16721

0.22295

92.076503

95.7377

95.2692

0.2623

0.04262

0.15246

0.19508

93.678161

90.163934

94.2197

93.9481

0.2716

0.0578

0.1647

0.22251

94.555874

91.100703

94.5559

94.5559

0.24359

0.05444

0.14902

0.20346

IIT Kanpur Dataset

540

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

Table 5: Results of using Different Similarity Measures

Matching Technique / time NCC 4.96 s

Disparity Map

Matching Technique / time LNCC 7.81 s

Disparity Map

Matching Technique / time SAD 5.77 s

LSAD 9.48 s

ZSAD 5.81 s

SSD 9.46 s

LSSD 9.32 s

ZSSD 7.89 s

SSDR2L 1.61 s

Disparity Map

The results in Table 5 demonstrate that the Sum of Squared Differences (SSD) Right to Left (SSD R2L) matching technique is the optimum choice for the proposed technique. Additionally the sample ROC and CMC curves [12] have been provided for the adopted methodologies in Table 6. We can observe from Table 3, in terms of accuracy, we obtain optimal results in IGIST-2 configuration, as it was more effective than the other three configurations (0.556% over GIST-2, 0.9662% over IGIST-1, 2.2705% over GIST-1). Hence, for a given real-time database, the probe to training image ratio can be kept similar to the IGIST-2 database configuration, in order to obtain similar results. Table 6: ROC and CMC Curves

ROC Curve for GIST-1 on ORL

ROC Curve for IGIST-1 on ORL

ROC Curve GIST-2 on IITK

541

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542

ROC Curve for IGIST-2 on IITK

CMC Plot GIST-2 on IITK

CMC Plot for IGIST-2 on IITK

Our experimentations establish that the double filtered GIST is capable of efficiently matching the probe and test faces images and is hence is a feasible approach for FR. The most optimum approach with less computation time was found to be the IGIST descriptor with the SSDR2L similarity measure. 6. Conclusions and Future Work We have proffered an improved double filtered face recognition methodology based on the GIST called IGIST technique and demonstrated that it is capable of effectively performing Face Recognition task. We experimented with four different configurations to ensure that considerable variation in FR parameters existed in order to establish the robustness of the proposed approach in real-time scenarios. Our experimentations revealed that the proposed GIST methodology was proficiently capable of performing FR, even under a wide variety of variations in a number of crucial FR parameters. Future work aims towards incorporating the Sparse PCA technique [10] as a post-processing step to carry out additional noise reduction and redundancy elimination, in order to investigate if it can provide further performance and accuracy boost. References 1.

2. 3. 4.

5. 6. 7. 8. 9. 10. 11. 12. 13. 14.

Kumar, Ankit, Ashish Joshi, Anil Kumar, Ankush Mittal, and D. R. Gangodkar. "Template Matching Application In Geo-Referencing Of Remote Sensing Temporal Image." International Journal of Signal Processing, Image Processing and Pattern Recognition 7, no. 2 (2014): 201-210. Kilthau, Steven L., Mark S. Drew, and TorstenMöller. "Full search content independent block matching based on the fast fourier transform." In Image Processing. 2002. Proceedings. 2002 International Conference on, vol. 1, pp. I-669. IEEE, 2002. Prewitt Operator: http://www.tutorialspoint.com/dip/Prewitt_operator.htm,accessed Nov 05, 2015. Chen, Weilong, MengJooEr, and Shiqian Wu. "Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain." Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on 36, no. 2 (2006): 458466. Hafed, Ziad M., and Martin D. Levine. "Face recognition using the discrete cosine transform." International Journal of Computer Vision 43, no. 3 (2001): 167-188. Sikirić, Ivan, Karla Brkić, and SinišaŠegvić. "Classifying traffic scenes using the GIST image descriptor." arXiv preprint arXiv:1310.0316 (2013). Douze, Matthijs, HervéJégou, HarsimratSandhawalia, Laurent Amsaleg, and CordeliaSchmid. "Evaluation of gist descriptors for web-scale image search." In Proceedings of the ACM International Conference on Image and Video Retrieval, p. 19. ACM, 2009. Oliva, Aude, and Antonio Torralba. "Modeling the shape of the scene: A holistic representation of the spatial envelope." International journal of computer vision 42, no. 3 (2001): 145-175. Moghaddam, Baback, and Alexander P. Pentland. "Face recognition using view-based and modular eigenspaces." In SPIE's 1994 International Symposium on Optics, Imaging, and Instrumentation, pp. 12-21. International Society for Optics and Photonics, 1994. Jolliffe, Ian. Principal component analysis. John Wiley & Sons, Ltd, 2002. AT&T Database of Faces “ORL Face Database”: http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html (accessed Nov 02, 2015) R.M. Bolle, J.H. Connell, S. Pankanti, N.K. Ratha and A.W. Senior, “The Relation Between the ROC Curve and the CMC”, In the proceedings of Fourth IEEE Workshop onAutomatic Identification Advanced Technologies, pp. 15 – 20, 2005. B. Heisele, P. Ho, J. Wu, and T. Poggio, "Face recognition: component-based versus Global Approaches ", Computer Vision and Image Understanding, vol 91, nos 1-2, pp 6- 21, 2003. IIT Kanpur Face database: http://www.iitk.ac.in/infocell/iitk/newhtml/storyoftheweek24.htm, accessed Nov 02, 2015.

542

A. Vinay et al. / Procedia Computer Science 79 (2016) 533 – 542 15. Bhumika G. Bhatt, Zankhana H. Shah “Face Feature Extraction Techniques: A Survey”, National Conference on Recent Trends in Engineering & Technology, 13-14 May 2011 16. Yan, Shuicheng, Huan Wang, Xiaoou Tang, and Tingwen Huang. "Exploring feature descriptors for face recognition." In Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, vol. 1, pp. I-629. IEEE, 2007. 17. GIST Scene Completion: http://cybertron.cg.tu-berlin.de/pdci11ws/scene_completion/implementation.html, accessed Nov 04, 2015. 18. Rublee, Ethan, Vincent Rabaud, Kurt Konolige, and Gary Bradski. "ORB: an efficient alternative to SIFT or SURF." In Computer Vision (ICCV), 2011 IEEE International Conference on, pp. 2564-2571. IEEE, 2011. 19. Calonder, Michael, Vincent Lepetit, Mustafa Ozuysal, Tomasz Trzcinski, Christoph Strecha, and Pascal Fua. "BRIEF: Computing a local binary descriptor very fast." Pattern Analysis and Machine Intelligence, IEEE Transactions on 34, no. 7 (2012): 1281-1298. 20. Tuzel, Oncel, FatihPorikli, and Peter Meer. "Region covariance: A fast descriptor for detection and classification." In Computer Vision– ECCV 2006, pp. 589-600. Springer Berlin Heidelberg, 2006. 21. Bay, Herbert, Andreas Ess, TinneTuytelaars, and Luc Van Gool. "Speeded-up robust features (SURF)." Computer vision and image understanding 110, no. 3 (2008): 346-359. 22. Lowe, David G. "Distinctive image features from scale-invariant keypoints." International journal of computer vision 60, no. 2 (2004): 91110. 23. Alahi, Alexandre, Raphael Ortiz, and Pierre Vandergheynst. "Freak: Fast retina keypoint." In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 510-517. Ieee, 2012. 24. Nagamalai, Dhinaharan, Eric Renault, and Murugan Dhanuskodi. Advances in digital image processing and information technology: first international conference on digital image processing and pattern recognition, DPPR 2011, Tirunelveli, Tamil Nadu, India, September 2325, 2011, proceedings. Vol. 205. Springer Science & Business Media, 2011. pp: 278-280. 25. NPTEL Courses, “Discrete Cosine Transform (DCT)” http://nptel.ac.in/courses/117105083/pdf/ssg_m3l9.pdf, accessed Nov 05, 2015. 26. Roma, Nuno, José Santos-Victor, and José Tomé. "A Comparative Analysis Of Cross-Correlation Matching Algorithms Using a Pyramidal Resolution Approach." (2002). 27. Patil, Suyog, Joseph Simon Nadar, JimitGada, Siddhartha Motghare, and Sujath S. Nair. "Comparison of Various Stereo Vision Cost Aggregation Methods." International Journal of Engineering and Innovative Technology 2, no. 8 (2013): 222-226.