Gradient Direction Transform

Gradient Direction Transform Magdi Mohamed QCT, Multimedia and Standards R&D, Computer Vision Systems, 2017:06:02

Magdi Mohamed, 2017:06:02

1 / 81

Agenda  Motivations  Existing Solutions  Gradient Direction Transform - Definition & Characteristics  Iris Detection Application  Handwriting Recognition Application

 Future Thoughts  Summary & Conclusions


2 / 81

Optical Eye Tracking System Diagram – Motivation for Idea

Frame Normalization

Image Preprocessing

Image Segmentation

Feature Extraction

Analysis & Classification

Postprocessing

Size Light Pose

Binarization Noise Filtering Smoothing

Morphological Operations, Clustering, & Relaxation Techniques

Edge Operations: Detection Linking Thinning

Search & Image Interpretation Tasks

Spatial Relationships, Sanity Checks & Accept / Reject

Hough Transform Processing And / Or Other Methods (Timm & Barth)

Iris detection is essential for several applications including gaze tracking Magdi Mohamed, 2017:06:02

3 / 81

Overview of Techniques

Practical solutions are usually engineering combination of multiple techniques Magdi Mohamed, 2017:06:02

4 / 81

Hough Transform for Iris Detection – Existing Solution

y

Image Space ( x - α1 ) 2 + ( y – β1 ) 2 = r 2

(x1,y1)

x

Hough Space for Circles β

(α− x1 ) 2 + ( β − y1 ) 2 = r2

Iris Detection Requirements Deals with variations in light and reflections Deals with partially occluded, missing, and noisy features No special markers or makeup required Features realtime processing Quantifies action codes

(α1, β 1)

α

Hough method can detect multiple curves and is resilient to noisy inputs Magdi Mohamed, 2017:06:02

5 / 81

Other Methods: Timm & Barth – Existing Solution {timm,barth}@inb.uni-luebeck.de

Proposing a faster and novel extension of this method Magdi Mohamed, 2017:06:02

6 / 81

Gradient Direction Transform: New Idea

after initializing the transform matrix to zeros, for each gradient vector in the input image region of interest, increment / decrement the value of the cells in the transform matrix inline with the gradient vector accordingly Magdi Mohamed, 2017:06:02

7 / 81

Gradient Direction Transform: Solution for Iris Detection i-image Pre-Processing n-image

Blur-Convolution

Edge-Convolution

e-image

b-image

Vertical-Edge

Horizontal-Edge

Otsu Dynamic Threshold

h-image

v-image

o-image

Gradient Direction Transform

Thinning

g-image

t-image

Weighted Gradient Direction

Hough Transform for Elliptical Shapes

w-image

h-space

Find Eye

iris-location Magdi Mohamed, 2017:06:02

8 / 81

Bresenham’s Algorithm – Implementation Details


9 / 81



10 / 81



11 / 81

Gradient Direction Transform: Implementation Details


12 / 81

References • Hough: Method and means for recognizing complex patterns https://docs.google.com/viewer?url=patentimages.storage.googleapis.com/pdfs/US3069654.pdf • Ballard: Generalizing Hough transform to recognize arbitrarily shapes http://www.cs.utexas.edu/~dana/HoughT.pdf • Mohamed & Nasir: Method and system for parallel processing of Hough transform computations https://docs.google.com/viewer?url=patentimages.storage.googleapis.com/pdfs/US7406212.pdf • Timm & Barth: Accurate eye center localization by means of gradients http://cjee.lakeheadu.ca/public/journals/22/TiBa11b.pdf • Bresenham: A rasterizing algorithm for drawing curves http://members.chello.at/easyfilter/bresenham.pdf


13 / 81

Major Characteristics – Complexities & Capabilities  Face detection takes

~ 30 ms

(24 x 24)

 Eye corner detection takes

~ 12 ms

(256 x 256)

 Iris detection time complexity for region of interest (C x R) 2

 T&B = K1 (C * R)  GDT = K2 (C * R) * C

(K1 ~ cost for normalized floating point vector dot product) (K2 ~ cost for integer addition and bit-wise operations, C>R)

 T&B ignores sign of vector in squaring dot products to avoid square root computations

 GDT is capable of efficient  Consideration of sign of vectors at no extra cost (inward/outward directions)  Extension of gradient normal-vector (by choice) to other directions such as gradient tangent-vector that may suit describing other (binary/gray/color) image analysis tasks


14 / 81

Timm & Barth Versus Gradient Direction Transform Measured Speedup Ratio = T1 / T2 Method


Size (C x R)

Conventional T&B T1 (Seconds)

Novel GDT T2 (Seconds)

Speedup Ratio T1 / T2

040 x 030

0001.291

0000.053

024.538

080 x 060

0022.630

0000.345

065.672

160 x 120

0394.155

0002.689

146.580

240 x 180

2063.608

0013.426

153.705

320 x 240

6603.897

0036.665

180.116

15 / 81

Sample Image 320x240


16 / 81

Input Image 320x240


17 / 81

Gradient Image (Gx)


18 / 81

Gradient Image (Gy)


19 / 81

Gradient Direction Transform 320x240 (In Grey)


20 / 81

Gradient Direction Transform 320x240


21 / 81



22 / 81



23 / 81

Timm & Barth Transform 320x240


24 / 81



25 / 81



26 / 81

Iris Detection – Major Challenges         

Eye boarder localization Occlusion Eyelashes Glasses Makeup Shadows Resolution Non-circularity Wrinkles


        

Redness Water Abrupt motion Blinks Differences between left and right eyes Pose, color variations, and light reflections Age sensitive Manual annotation and evaluation metrics Strong/Weak eye concerns for gaze

27 / 81

Iris Detection - Notes On Used Data  The data is not constructed particularly for evaluating iris detection algorithms  It contains illegitimate images, and less accurate annotation for iris centers  It contains some faces with sun glasses  It contains many cases with corneal reflections from strong light sources  It significantly represents almost frontal faces

 It contains mostly colored images  It represents different categories (age groups, makeup, styles, …)  Since it is the only annotated data available to us, we used it to construct our facial landmark detector, and to evaluate the iris detector

 Ideally, a balanced data set with “casual” iris positions, not mostly looking at the camera, with more accurate manual annotation of iris centers is needed


28 / 81

Iris Detection – Experiment # 1 Setup  Using 15800 (mostly frontal view) face images with 40 landmarks per face manually annotated as Ground Truth,

 Using Omron Face Detector,  Using Omron Eye Corner Locator,  Conducting a Blind Test to Compare GDT Versus T&B Solution,  Compute normalized Cumulative Error Distribution (CED) for each estimator


29 / 81

Iris Detection – Experiment # 1 Performance Summary


30 / 81


 Using Omron Face Detector,  Using Omron Eye Corner Locator,  Conducting a Blind Test to Compare GDT Versus Omron Solution,  Compute normalized Cumulative Error Distribution (CED) for each estimator


31 / 81



32 / 81


 Using Omron Face Detector,  Using Supervised Newton Facial Landmark Detection method with Qualcomm feature descriptor (HSG) and training algorithm,

 Conducting 5 fold cross validation for evaluation,  Compute normalized Cumulative Error Distribution (CDF) for each estimator


33 / 81



34 / 81


35 / 81


36 / 81


37 / 81


38 / 81


39 / 81


40 / 81


41 / 81


42 / 81


43 / 81


44 / 81


45 / 81


46 / 81


47 / 81


48 / 81


49 / 81


50 / 81


51 / 81


52 / 81


53 / 81


54 / 81


55 / 81

Handwriting Recognition Using BAR Features

A Character Image and the Feature Image for the Foreground Horizontal Direction

Pseudo-Code for Computing the Bar Transform on the Foreground

BAR transform is an existing technique proven to provide superior performance Magdi Mohamed, 2017:06:02

56 / 81

Handwriting Recognition Using BAR Features

An Upper Case “B” and the Foreground and Background Bar Transform Feature Images Corresponding to the East-West Directions

BAR Feature Vector Uses Both Foreground & Background BAR Transforms Magdi Mohamed, 2017:06:02

57 / 81

GDT for Handwriting (Normal Direction)


58 / 81

GDT for Handwriting (Tangent Direction)


59 / 81

Handwriting Recognition Using GDT Features


60 / 81

Handwriting Recognition Using GDT Features


61 / 81


62 / 81


63 / 81

Testing


64 / 81

Testing


65 / 81

Testing


66 / 81

Testing


67 / 81

Testing


68 / 81

Testing


69 / 81

Testing


70 / 81

Testing


71 / 81

Testing


72 / 81

Testing


73 / 81

Testing


74 / 81

Testing


75 / 81

Testing


76 / 81

Testing


77 / 81

Testing


78 / 81

Testing


79 / 81

Testing


80 / 81

Testing


81 / 81