university of calcutta automatic attendance

UNIVERSITY OF CALCUTTA Asutosh College Department of Computer Science

Major Project Report On

AUTOMATIC ATTENDANCE MANAGEMENT SYSTEM By Suman Dey Reg. No.: 053-1121-0004-12 Roll: CIS/15/04 No.:001

Ankur Guha Reg. No.: 012-1121-1297-12 Roll: CIS/15/04 No.:012

Dibyarup Basu Reg. No.: 223-1121-0195-12 Roll: CIS/15/04 No.:013

Sudipta Banerjee Reg. No.: 434-1121-0205-11 Roll: CIS/15/04/No.:016

under the supervision of Prof. Gautam Mahapatra Associate Professor and Head

Prof. Atrayee Chatterjee Contractual Whole Time Teacher

Prof. Joyshree Nath Contractual Whole Time Teacher

Sri Suman Mitra Faculty Member

Preface Computer Science is an application oriented subject. Major Project is a mandatory part of Masters in Computer and Information Science course. Every project comes with a time bound but most of the project requires more time than the scheduled time. So there is a trade-off between time and completeness of the project. Also there are some resource limitations as all the required resources for such project are generally not readily available in an academic environment. A project requires guidance from faculties and since their availability is not always ensured, so there is a limitation of guidance. This project deals with attendance management, since most of us are aware of the term attendance as it is vastly used in educational institutions, it helped us to understand the problem of manual attendance thus helping renovate new ideas to overcome the problems. We are constantly motivated and supported by our mentors and project guides to select and continue to work on this project topic. We have tried our level best to add completeness to this project work. Project work helps to gain real world experiences which are useful for future in professional fields. While working on this project topic, Automatic Attendance Management System, in spite of the resources, guidance and time limitations we have been able to learn a lot of new things which has not only enriched our knowledge but also enabled us to gather real life experiences.

Contents 1 Introduction 1.1 Broad Topic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 1 1

2 Software System Structure 2.1 Life Cycle Model . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 The Need For A Software Life Cycle Model . . 2.1.2 Different Software Life Cycle Models . . . . . . 2.2 Prototyping Model . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 What is a Prototype? . . . . . . . . . . . . . . . . 2.2.2 Need for a Prototype in Software Development 2.3 Software Requirements Specification(SRS) . . . . . . . 2.4 Context Diagram . . . . . . . . . . . . . . . . . . . . . . . 2.5 Data Flow Diagram(DFD) . . . . . . . . . . . . . . . . . 2.5.1 Types Of DFD . . . . . . . . . . . . . . . . . . . . . 2.5.2 DFD Components . . . . . . . . . . . . . . . . . . . 2.5.3 Importance Of DFD . . . . . . . . . . . . . . . . . 2.6 Level 1 DFD . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Level 2 DFD . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Level 3 DFD . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 Level 4 DFD . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . .

2 2 2 2 2 2 3 4 4 5 5 5 5 6 7 8 9

3 Technical Overview 3.1 What is an digital image? . . . . . 3.2 Face Detection . . . . . . . . . . . . 3.2.1 Viola Jones Face Detection 3.3 Face Recognition . . . . . . . . . . . 3.3.1 Feature Extraction . . . . . 3.3.2 Classification . . . . . . . . . 3.4 Attendance Marking . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

10 10 10 10 12 12 13 14

4 Database Design 4.1 Entity Relationship Diagram (ERD) . . . . . . . . . . . . . . . . . . . . . . . . .

15 15

5 Database Schema 5.1 student master 5.2 student image 5.3 faculty master 5.4 course master 5.5 registered user 5.6 student course 5.7 student model . 5.8 attendance . . . 5.9 logrecord . . . .

17 17 17 17 18 18 18 19 19 19

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

6 Implementation 6.1 System Configuration . . . . . . . . . . . . . . . . . . . . . 6.2 MDI Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 User Registration . . . . . . . . . . . . . . . . . . . . . . . . 6.3.1 User Registration for User Of Admin type . . . . 6.3.2 User Registration for User Of Non-Admin type . 6.4 User Log In . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Image Registration . . . . . . . . . . . . . . . . . . . . . . . 6.5.1 For Registered User of Administrator Type . . . . 6.5.2 For Registered User of Non-Administrator Type 6.6 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.1 For Registered User of Administrator Type . . . . 6.6.2 For Registered User of Non-Administrator Type 6.7 Recognization . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.7.1 For Registered User of Administrator Type . . . . 6.7.2 For Registered User of Non-Administrator Type 6.8 Attendance Update . . . . . . . . . . . . . . . . . . . . . . . 6.9 Attendance Report . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

21 21 22 23 24 25 28 29 29 31 32 33 34 35 36 37 39 40

7 Testing and Verification 7.1 Prerequisite Unavailability 7.2 User Registration . . . . . . 7.3 User LogIn . . . . . . . . . . 7.4 Image Registration . . . . . 7.5 Training . . . . . . . . . . . . 7.6 Recognition . . . . . . . . . . 7.7 Attendance Update . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

41 42 43 51 52 57 62 68

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

8 Future Scopes

74

9 References

75

10 Appendix A

76

11 Appendix B

140

1.

Introduction

Attendance defines the presence of an or a number of individual at a place or event. The technique for taking attendance prevails in all working places such as school, college, university, office to name some. The attendance marking method follows the manual or traditional method till now. Another way for marking attendance is the automatic mode where the manual labor is excluded. In traditional method an Attendance Register Book is maintained where the daily attendance are recorded manually. Advantage of this method is that it is not dependent on electricity usage so works uniformly as location independent form. It can be updated at any time. This method is much more acceptable to the end users as there is human involvement. If the work done by them is more accurate then the manual method is readily acceptable and reliable to the authority and users. In automatic attendance marking method there remains a constraint for every user as they have to be present within a particular time frame otherwise their attendance will not be recorded. For instruments there are no relaxation based of humanity ground and other issues situation that a student is late to college for some emergency matter but that will not be taken into account if the mode of taking attendance is automatic whereas in traditional or manual marking system this time relaxation is given to the individuals. The disadvantages of this method are time becomes a major issue here because manual labor is involved in this technique and increased paper work consumes time because maintaining all the register books properly are required and automatic usable report generations for decision making and management process is very much time consuming and some cases it is in-feasible. Proxies can be detected more easily with this automatic system as it requires the student to be present in the class which contributes to the actual evaluation of the student attendance. The classroom activities of the students can be monitored and analyzed using machine learning which helps the teachers to evaluate the attentiveness and performance of the students as it is tough for a teacher to do board works and also monitoring each student’s activities simultaneously for a large class with considerable number of students. This monitoring of students with the help of automated machine learning process actually helps the students themselves to acquire good and quality education from an institution. Identification of students who are disturbing others by the help of this system helps increasing the effectiveness and fruitfulness of the classes.

1.1

Broad Topic

System Automation, Machine Learning and Image Processing.

1.2

Objective

The main objective of our project is to build an automatic attendance management system thereby to replace the traditional manual attendance system which is highly time consuming. It will also helps in evaluation process of individual students, that is whether the students are at all attentive or not and also whether they are attending all classes regularly according to the class routine.

1.3

Literature Review

Real-time detection of objects from image frames extracted from video are discussed in [1]. For face detection, the object is human face in an image. Viola Jones Face Detector working procedure has further elaborated in [2]. Concepts of integral image for the speedup the detection procedure has been discussed by [5]. Working principle of histogram of oriented gradient along with machine learning methods for implementation using MATLAB for face detection with Viola Jones approach are discussed, also it guides how to use machine learning methods like classifier to classify different human faces for face recognition based on the feature extracted using histogram of oriented gradient method from the face detected images in [7]. Insights about how the design of a software system should be made and about the models to follow for better results are provided in [9].

1

2.

Software System Structure

This part describes the overall system structure that is going to be implemented and also the life cycle model which is followed for designing this software.

2.1

Life Cycle Model

The life cycle of a software represents the series of identifiable stages through which it evolves during its life time. With this knowledge of a software life cycle it can further define Software development life cycle(SDLC) model(also called software life cycle model and software development process model. A life cycle model graphically represents the different phases through which a software evolves. It is usually accompanied by a textual description of the different activities that need to be carried out during each phase.

2.1.1

The Need For A Software Life Cycle Model

The development team must identify a suitable life cycle model for the particular project and then adhere to it. Without using of a particular life cycle model the development of a software product would not be in a systematic and disciplined manner. When a software product is being developed by a team there must be a clear understanding among team members about when and what to do. Otherwise it would lead to chaos and project failure. A software life cycle model defines entry and exit criteria for every phase. A phase can start only if its phase-entry criteria have been satisfied. So without software life cycle model the entry and exit criteria for a phase cannot be recognized. Without software life cycle models it becomes difficult for software project managers to monitor the progress of the project.

2.1.2

Different Software Life Cycle Models

Many life cycle models have been proposed so far. Each of them has some advantages as well as some disadvantages. A few important and commonly used life cycle models are as follows: 1. Classical Waterfall Model. 2. Iterative Waterfall Model. 3. Prototyping Model. 4. Evolutionary Model. 5. Spiral Model. In this project for Software structure design Prototyping Model will be used.

2.2

Prototyping Model

2.2.1

What is a Prototype?

The prototype model suggests building a working prototype of the system, before the development of the actual software . A prototype is a toy and crude implementation of a system. It has limited functional capabilities, low reliability, or inefficient performance as compared to the actual software. A prototype can be built very quickly by using several short cut. The short cut usually involve developing inefficient, inaccurate, or dummy functions.

2

2.2.2

Need for a Prototype in Software Development

There are several uses of a prototype. Some are listed below: • To illustrate the input data formats, messages, reports, and the interactive dialogues to the customer. This is a valuable mechanism for gaining better understanding of the customers needs. • It is impossible to get the perfect product in the first attempt.The experience gained in developing the prototype can be used to develop the final product • A prototyping model can be used when technical solutions are unclear to the development team. A prototype can help them to critically examine the technical issues associated with product development.

Figure 2.1: Prototyping Model of software development The above figure is that of a prototyping model of software development. As shown in the figure, software is developed through two major activities: 1. Prototype development: Prototype development starts with an initial requirements gathering phase. A quick design is carried out and a prototype is built. The developed prototype is submitted to the customer for evaluation. Based on the customer feedback, the requirements are refined and the prototype is modified. This cycle of obtaining customer feedback and modifying the prototype continues till the customer approves the prototype. 2. Iterative development: Once the customer approve the prototype, the actual software is developed using the iterative waterfall approach. In spite of the availability of a working prototype,

3

the SRS document is usually needed to be developed since the SRS document is invaluable for carrying out traceability analysis, verification, and test case design during later phases. However, for GUI parts, the requirements analysis and specification phase becomes redundant since the working prototype that has been approved by the customer serves as an animated requirements specification.

2.3

Software Requirements Specification(SRS)

The SRS document usually contains all the user requirements in a structured though an informal form. The SRS document should describe the system to be developed as a black box, and should specify only the externally visible behavior of the system. For this reason, the SRS document is also called the black-box specification of the software being developed. An SRS document should clearly document the following aspects of a software: 1. Functional Requirements: • The whole set up should be connected to a network. • An image capturing system is required to capture images of the students. • To store images and other information resource that is database is required. 2. Non-Functional Requirements: • Software maintenance that is update to new version of the software must be made. • For operating the system daily power source should be uninterrupted. In case of power breakdown there should be a provision for power back up mechanism from where the system will get it’s required power for operation.

2.4

Context Diagram

• The context diagram is the most abstract data flow representation of a system. It represents the entire system as a single bubble. This bubble is labeled according to the main function of the system. The various external entities with which the system interacts and the data flow occurring between the system and the external entities are also represented. The data input to the system and the data output from the system are represented as incoming and outgoing arrows. These data flow arrows should be annotated with the corresponding data names. • The name context diagram is well justified because it represents the context in which the system is to exist, i.e. the external entities who would interact with the system and the specific data items they would be supplying the system and the data items they would be receiving from the system. The context diagram is also called as the level 0 DFD. • The bubble in the context diagram is annotated with the name of the software system being developed.This is expected since the purpose of the context diagram is to capture the context of the system rather than its functionality. The figure on the next page shows the context diagram designed for our software system. It is discussed below: • The CLASS IMAGES and the ATTENDANCE REPORT are the external entities. • The entire system AUTOMATIC ATTENDANCE MANAGEMENT SYSTEM is represented in the single bubble. • The data flow is shown using the pointed arrows.

4

Figure 2.2: Context Diagram

2.5

Data Flow Diagram(DFD)

Data flow diagram is a hierarchical graphical representation of data flow in an information system. It is capable of depicting incoming data flow, outgoing data flow and stored data. The DFD does not mention anything about how data flows through the system.

2.5.1

Types Of DFD

Data Flow Diagrams are of two types: 1. Logical DFD:This type of DFD concentrates on the system process and flow of data in the system. 2. Physical DFD:This type of DFD concentrates on the mapping of data flow onto respective physical system.

2.5.2

DFD Components

DFD can represent Source, destination, storage and flow of data using the following set of components 1. Entities:Entities are source and destination of information data. Entities are represented by rectangles with their respective names. 2. Process:Activities and action taken on the data are represented by Circle or Round-edged rectangles. 3. Data Storage:There are two variants of data storage - it can either be represented as a rectangle with absence of both smaller sides or as an open-sided rectangle with only one side missing. 4. Data Flow:Movement of data is shown by pointed arrows. Data movement is shown from the base of arrow as its source towards head of the arrow as destination.

2.5.3

Importance Of DFD

• The DFD technique is so popular is probably because of the fact that DFD is simple to understand and use. • Starting with a set of high-level functions that a system performs, a DFD model hierarchically represents various sub-functions. In fact, any hierarchical model is simple to understand. Human mind is such that it can easily understand any hierarchical model of a system because in a hierarchical model, starting with a very simple and abstract model of a system, different details of the system are slowly introduced through different hierarchies. 5

• The data flow diagramming technique also follows a very simple set of intuitive concepts and rules. • DFD is an elegant modeling technique that turns out to be useful not only to represent the results of structured analysis of a software problem, but also for several other applications such as showing the flow of documents or items in an organization.

2.6

Level 1 DFD

Figure 2.3: Level 1 DFD The above figure is demonstrated below: • Images of the students are captured and faces are detected from them. • These detected images undergo pre-processing and are stored in the database along with the student’s educational information. • These stored images in the database are subjected to feature extraction and finally to design a model for that particular class. • After classification and recognizing the students their attendance are updated in the attendance register. • A report is sent of individual student attendance statistics to the user according to user’s query.

6

2.7

Level 2 DFD

Figure 2.4: Level 2 DFD The above figure is discussed below: • Images captured using the capturing system is stored in the student image database. • Initially when the classifier model is being built, the images comprise of both training and testing images. Later as time passes by a feedback system is maintained which updates the image data set to adapt facial changes. • The images for training as well as for testing are randomly selected and the ratio of training to testing images should be maintained with the number of training images being in more numbers than that of testing images. • The training images set now undergoes a feature extraction process. The feature of each of the images extracted stores discriminative information about each face in a compact vector. • A machine learning model is then used to fit a model of the appearance of the faces in the image database so that we can discriminate between the faces of different people in the database. • The output of the previous stage is a classifier or model that will be used to recognize the input images.

7

2.8

Level 3 DFD

Figure 2.5: Level 3 DFD The above figure is described below: • The testing set images undergoes a feature extraction process using the same method used in case of training images. • Now these image features are set to the classifier or model already created in the previous DFD level. • The output is a label which signifies which person from the database the input/test image belongs to. • Depending on the prediction result of the previous stage the attendance is updated for the recognized student by the attendance registration system.

8

2.9

Level 4 DFD

Figure 2.6: Level 4 DFD The above figure is discussed below: • The user queries for an individual student statistics report to the attendance analyzer. • The attendance analyzer takes data from the student attendance database to compute the user’s query. • The Attendance Report Generator System takes the computed data from the Attendance Analyzer and the other information about the student from the Student attendance database and it generates a report based on these retrieved information. • Then the Attendance Report Generator System sends the final generated report to the user.

9

3.

Technical Overview

This part of the report elaborates the process of how a classroom image is being used to automatically mark attendance of the students present at the lecture.

Figure 3.1: Basic overview of this technical part

3.1

What is an digital image?

A digital image is defined as a 2-D function, f(x,y), where x and y denote spatial coordinates and f is called intensity or gray level at the point of (x,y). Digital image is a matrix of fundamental units called pixels. There are different types of images like binary image, grayscale image, color image etc.

(b) A grayscale image

(a) A binary image

(c) A color image

Figure 3.2: Different types of image.

3.2

Face Detection

Face detection is a computer technology being used in a variety of applications that identifies human faces in digital images[4][10] . Human faces have some features which makes it different from other objects. These common facial features of human are used to determine if a digital image consists of human face or not. Given an arbitrary image, the goal of face detection is to determine whether or not there are any faces in the image and, if present, return the image location and extent of each face. While this appears to be a trivial task for human beings, it is a very challenging task for computers. The difficulty associated with face detection can be attributed to many variations in scale, location, view point, illumination, occlusions, etc. One of the most popular known methods of face detection is Viola Jones face detection[1] algorithm.

3.2.1

Viola Jones Face Detection Technique

The Viola-Jones face detector[1] contains four main ideas that make it possible to build a successful face detector that can run in real time: Haar like rectangular features, the image integral, classifier learning with AdaBoost, and the cascade structure. 10

Haar like rectangular features Human face has some common features like, the eye portion is darker than the upper cheeks, the nose bridge tends to be brighter than the eyes, the forehead region is brighter than the eye region below etc. These common features are detected using Haar like rectangular features shown below:

Figure 3.3: Haar like rectangular features. These rectangular features are similar like kernels used to detect edges using convolution. These features are looked for on the face by subtracting the sum of all pixels in the white region from the sum of all pixels in the black region yielding a single value corresponding to that feature on the image. To calculate these summation more efficiently integral image is used. Integral Image Integral image, also known as a summed area table, is an algorithm for quickly and efficiently computing the sum of values in a rectangle subset of a pixel grid. The integral image at location x, y contains the sum of the pixels above and to the left of x, y, inclusive. Below is a simple example showing the integral image of the corresponding input image.

Figure 3.4: Integral image. Now to find the summation of a given rectangular region in an image we just need the four values from the four corners of the rectangle from the integral image. In the below example we want to find out the summation of the pixel values in the shaded portion from the original image.

Figure 3.5: Integral image example. If we calculate the summation of all 9 values in the original image we get: 2 + 5 + 1 + 4 + 6 + 3 + 3 + 2 + 4 = 30 Now using the corresponding integral image we can find out the summation by: 53 + 2 − 15 − 10 = 30 Using the integral image we need to access only four of the integral image pixel value compared to 9 values (in this case) of the original image. This improves performance by lowering the number of computations required every time during evaluating the summation of the pixels in a specified region of the image. 11

AdaBoost Learning Given a feature set and a training set of positive and negative images, any number of machine learning approaches could be used to learn a classification function. The Viola-Jones face detection[1] uses a variant of AdaBoost to both select a small set of features and train the classifier. A single AdaBoost classifier consists of a weighted sum of many weak classifiers, where each weak classifier is a threshold on a single Haar-like rectangular feature. The weight associated with a given sample is adjusted based on whether or not the weak classifier correctly classifies the sample.

F(x)=α1 f1 (x) + α2 f2 (x) + α3 f3 (x) + ... x: image sub block, F(): Strong Classifier, fi (): Weak Classifier, αi : Weight

Cascade Classifier The cascaded classifier is composed of stages each containing a strong classifier from AdaBoost. The job of each stage is to determine whether a given sub-window is definitely not a face or maybe a face. When a sub-window is classified to be a non-face by a given stage it is immediately discarded. Conversely a sub-window classified as a maybe face is passed on to the next stage in the cascade. It follows that the more stages a given sub-window passes, the higher the chance the sub-window contains a face.

Figure 3.6: Cascading of classifiers. The complexity of the classifiers used are arranged in an increasing order, which ensures complex classifications tasks are done only when the input window have some features close to human facial features.

3.3

Face Recognition

Face recognition is the process to uniquely identify or verify a person’s identity by comparing and analyzing patterns based on the person’s facial contours. In this project we are using face recognition to automate the traditional manual attendance system. Face recognition requires learning of a classifier model with the features, extracted from the detected face images. Later this learned classifier model is used it to predict the identity of the detected faces in the classroom images which in turn leads to the automatic attendance marking to the students present in the lecture.

3.3.1

Feature Extraction

An image consists of a huge amount of data in the form of pixels. This huge data consists of some relevant and useful data which can be processed to extract that relevant information from it. There are also some unwanted data which are rejected. Feature extraction is a method of dimensionality reduction that represents the interesting parts of an image in a compact feature vector[7] . The densest features are the pixels of the image itself[7] . Histogram of Oriented Gradients (HOG) is one of such feature extraction process which represents the structure of objects in the image and is invariant to the illumination and image pixels of that image.

12

Histogram of Oriented Gradients (HOG) The input image is converted into grayscale image and is divided into a number of sub blocks of same size. Each center pixel of these sub blocks are compared against it’s neighboring pixel values to find the direction at which the pixels are getting darker (i.e. pixels with lower gray level values). An arrow or line is drawn along this direction across the block. This process is repeated for all the sub blocks present in the image to produce the HOG of the whole input image.

Figure 3.7: Gradient calculation on a 3 x 3 block. The above image illustrates the HOG process on a sub block size of 3 x 3 with the arrow indicating the direction of the darker portion of the block[6] . Below is the HOG visualization of an image containing a face.

Figure 3.8: HOG visualization of a face image.

3.3.2

Classification

In machine learning, classification is the problem of identifying to which of a set of categories a new observation belongs, on the basis of a training set of data containing observations whose category membership is known. An example would be assigning a given email into ”spam” or ”non-spam” classes. In the terminology of machine learning, classification is considered an instance of supervised learning, i.e. learning where a training set of correctly identified observations is available. An algorithm that implements classification is known as a classifier. The term ”classifier” sometimes also refers to the mathematical function, implemented by a classification algorithm that maps input data to a category. There are many classifiers some of them are: 1. Naive Bayes Classifiers, 2. K Nearest Neighbors (KNN) Classifier, 3. Support Vector Machine (SVM) Classifier, 4. Error Correcting Output Codes (ECOC) Classifier. Any of the multiclass classifiers can be used to classify the extracted features of the detected faces in an image. The classifier is trained with the feature set extracted from the detected faces of registered students in the training dataset. After training, the generated classification model is stores in database for face recognition purpose. The classification model is regenerated after a predefined period with new training set enabling the system to adapt with the gradual facial feature changes of the registered students with time.

13

Figure 3.9: Training and prediction phases of classification. The image above illustrates the two phase of the classification i.e. training phase where the classifier is feed with labeled feature data for learning different features of each of the different classes and the prediction phase where an unknown non-labeled feature is given to the model and it predicts the most probable class of that feature by generating a label against that feature.

3.4

Attendance Marking

All the recognized students in the classroom image automatically gets their attendance updated into the attendance registration system. A display may also be incorporated in the classroom where the recognized student IDs would be displayed so that students can report for wrong detection to the teacher.

14

4.

Database Design

Here the database design for the whole system is depicted using Entity Relationship (E-R) diagram. Before going further, first we need to know what is an E-R diagram and why it is so important in designing a database.

4.1

Entity Relationship Diagram (ERD)

An ”Entity Relationship Diagram (ERD)” shows the relationships of entity sets stored in a database. An entity in this context is a component of data. In other words, ER diagrams illustrate the logical structure of databases. • For database designing Top Down approach is known as E-R diagram. • Here we are focusing on top down approach of designing database. It is a graphical technique, which is used to convert the requirement of the system to graphical representation. So that it can become well understandable.

Figure 4.1: E-R diagram of the registered user schema with logrecord schema.

15

Figure 4.2: E-R diagram of the whole system.

16

5.

Database Schema

This part describes the underlying database schema of Automatic Attendance Management System (AAMS). There are 9 database/schemas which are as follows:1. student master, 2. student image, 3. faculty master, 4. course master, 5. registered user, 6. student course, 7. student model, 8. attendance, 9. logrecord. Each of the above schema is described below.

5.1

student master

This schema consists of the following attributes: studentid: This attribute is given as primary key, which is unique. Student id is used to identify each and every student uniquely. name: This attribute stores every student’s name.

5.2

student image

This schema consists of the following attributes: studentid: This attribute is given as foreign key, which is also unique and this attribute is connected with the primary key attribute of ”Student master” schema. Student id is used to identify each and every student uniquely. picture: This attribute stores every student’s face detected image. date: This attribute store the date of the image stored.

5.3

faculty master

This schema consists of the following attributes: 17

facultyid: Uniquely identifies each of the faculties. name: Name of the faculty is stored. designation: Designation of the faculty is stored.

5.4

course master

This schema consists of the following attributes: courseid: Uniquely identifies each of the courses available. coursename: Name of each of the courses available are stored.

5.5

registered user

userid: Uniquely identifies each of the users of this system. usertype: User type i.e. admin or non admin is stored. password: Password of the user is stored in hashed form using SHA 512. name: Name of the user is stored. gender: Gender of the user is stored. contactno: Stores the contact number. email: Stores the email address detail.

5.6

student course

This schema consists of the following attributes: studentid: Unique student identifier. It is a foreign key and references the table student master. 18

courseid: Unique course identifier. It is a foreign key and references the table course master. session: Session detail.

5.7

student model

This schema consists of the following attributes: coursid: Unique course identifier. It is a foreign key and references the table course master. session: Session detail. model: Trained classifier model.

5.8

attendance

This schema consists of the following attributes: studentid: Unique student identifier. It is a foreign key and references the table student master. facultyid: Unique faculty identifier. It is a foreign key and references the table faculty master. courseid: Unique course identifier. It is a foreign key and references the table course master. session: Session detail. datetime: Date and time of the day for the attendance record. type: The type of the attendance i.e. automatic or manual is stored.

5.9

logrecord

This schema consists of the following attributes: userid: Unique user identifier. It is a foreign key and references the table registered users.

19

action: User actions like image registration, training model, recognition etc are recorded in this field. datetime: Date and time of the day for the log record.

State Of The Art The following are the state of the art for this software system: 1. Image Processing 2. Machine Learning 3. MATLAB 4. Eclipse IDE for Java 5. MySQL

20

6. 6.1

Implementation System Configuration

At first the system configuration interface window is opened which consists of MySQL Database user name, password, database name, working path for the whole system, working path for the MATLAB functions as shown below:

Figure 6.1: System Configuration

21

6.2

MDI Interface

Now when the system is started the interface opens into a MDI(Multiple Document Interface) containing a menu bar which comprises of the following menu items: • User Registration • Log In • Image Registration • Training • Recognization • Attendance The above discussed window looks like the following figure:

Figure 6.2: MDI Window

22

6.3

User Registration

For a new user, he/she has to register first. No user can Log In without registration. For registration the button Register on the User Registration item is selected from the menu items of the MDI Interface which opens into a new window as shown below. The basic buttons functionality are also shown:

Figure 6.3: User Registration Window

23

6.3.1

User Registration for User Of Admin type

For an admin type user registration, a password is asked from the user for verifying whether the user is allowed to be an administrator or not. If the password matches with the already stored password for verification then the user is allowed to select the type field as Admin otherwise not. The other fields has to be filled up next and the User ID becomes the User Name for the user as shown below:

Figure 6.4: User Registration for User Of Admin type

24

6.3.2

User Registration for User Of Non-Admin type

For a non-admin type user registration, the text fields are filled by the user and the User ID becomes the User Name for the user:

Figure 6.5: User Registration for User Of Non-Admin type

25

When the button Check Availability is clicked it checks whether the User ID is present in the database or not as shown below:

Figure 6.6: Operation of Check Availability Button

26

When the button Register is clicked it registers the user details in the database as shown below:

Figure 6.7: Operation of Register Button

27

6.4

User Log In

The following window is displayed when Log In item is selected from the menu bar. The buttons functionality are also shown:

Figure 6.8: User Log In Window

28

6.5

Image Registration

On clicking on the Image Registration button from the menu bar of the MDI window, the following window is displayed where the basic buttons functionality has been explained for registered users of administrator type:

Figure 6.9: Image Registration Window

6.5.1

For Registered User of Administrator Type

If the registered user is of administrator type the following window is displayed:

Figure 6.10: Image Registration for Admin users

29

After filling up the required fields in the above window when the Process button is clicked the MATLAB engine starts and uses the video from the specified path to detect faces from every frames of the video and stores those face images in the file system. The section of Java code that starts the MATLAB engine and processes the video is shown below:

Figure 6.11: Java code to start the MATLAB engine with required parameters for processing the video The section of MATLAB code that executes when the Process button is clicked is shown below:

Figure 6.12: MATLAB code to detect the faces from each frames of the input video

30

6.5.2

For Registered User of Non-Administrator Type

If the registered user is of non-administrator type the following window is displayed:

Figure 6.13: Image Registration for Non-Admin users The buttons functionality in the above window is same as that of the window displayed for the users of administrator type.

31

6.6

Training

Training process will display different windows for the registered users of administrator type and for registered users of non-administrator type. The basic buttons functionality of the training window for registered users of administrator type is shown below:

Figure 6.14: Training Window

32

6.6.1



Figure 6.15: Training Window For Registered User of Administrator Type

The section of Java code where the the required parameters are passed to MATLAB for the generation of the training model is shown below:

Figure 6.16: Java code to start the MATLAB engine to generate training model with the required parameters

33

The section of MATLAB code required for the Process button operation is given below:

Figure 6.17: MATLAB code to train and generate a trained classifier model of the face images

6.6.2



Figure 6.18: Training Window For Registered User of Non-Administrator Type The buttons functionality in the above window is same as that of the window displayed for the users of administrator type.

34

6.7

Recognization

Recognition process will display different window for the registered users of administrator type and for registered users of non-administrator type. The basic functionality of the buttons for registered users of administrator type are shown below:

Figure 6.19: Recognization Window

35

6.7.1



Figure 6.20: Recognition for Admin users

The section of Java code for starting the MATLAB engine and sending the model and the image to be recognized.

Figure 6.21: Java code to start the MATLAB engine with required parameters for recognition

36

The section of MATLAB code for operation of the Recognize button is shown below:

Figure 6.22: MATLAB code to fit the test image to the trained classifier to recognize the students in the image belongs to which class

6.7.2



Figure 6.23: Recognition for Non-Admin users The buttons functionality in the above window is same as that of the window displayed for the users of administrator type.

37

The Capture button enables us to take a picture of the students in the classroom through the capturing device required at the time of recognition. When the button is selected the following window opens:

Figure 6.24: Capturing images of the students

38

6.8

Attendance Update

On selecting the Attendance Update button the following window is displayed for both Administrator type and Non-Administrator type users. The basic buttons functionality of this window is shown below:

Figure 6.25: Attendance Update Window The above window has the following functionality: • In the above window the recognized faces can be seen and if the recognition is not correctly recognized then it can be rectified with the correct name, ID of the student. • If the recognized image is not a face it can be discarded. • If any of the student was not recognized, their attendance can also be updated. • All the above updates are possible when the respective Faculty ID, Session, Course ID are specified by the user. • Two types of attendance marking are possible here one is manual and the other is automatic. The update which requires user’s intervention is stated as manual type of attendance marking done by whereas the update that does not require user’s intervention is termed as automatic type of attendance marking. • Manual Update button facilitates manual attendance marking and Automatic Update button facilitates automatic attendance marking.

39

6.9

Attendance Report

On selecting the Attendance Report button the following window is displayed for both Administrator type and Non-Administrator type users:

Figure 6.26: Attendance Report Window This window above shows the Student ID, Faculty ID, Course ID, Session, Date, Time and Mode of Attendance in a tabular format.

40

7.

Testing and Verification

Software products are normally tested first at the individual component(or unit) level. Unit Testing is performed as each of the different units or modules are taken at a time and are subjected to a set of test inputs and it is observed if the program behaves as expected. If the program fails to behave, then the conditions under which error occurs are noted as discussed below. The units or modules selected for testing purpose are: • Prerequisite Unavailability • User Registration • User Log In • Image Registration • Training • Recognition • Attendance Update

41

7.1

Prerequisite Unavailability

If the software is not configured, then while executing the software the following error message will be displayed:

Figure 7.1: Error instance 1 If the software is wrongly configured, then while executing the software the following error message will be displayed:

Figure 7.2: Error instance 2 If the system does not have the MySQL database installed and configured, then while executing the software the following error message will be displayed:

Figure 7.3: Error instance 3 If the system does not have MATLAB installed and configured, then while executing the software the following error message will be displayed:

42

Figure 7.4: Error instance 4

7.2

User Registration

In the User Registration window if the Name text field is empty the following error will be displayed:


43

In the User Registration window if the Contact No. text field is empty the following error will be displayed:


44

In the User Registration window if the Email text field is empty the following error will be displayed:


45

In the User Registration window if the User ID text field is empty the following error will be displayed:


46

In the User Registration window if the Password text field is empty the following error will be displayed:


47

In the User Registration window if the user wants to register as an Administrator type user, then he/she is required to enter an password which has to match with that of a previously set password otherwise the following error will be displayed:


48

If the user enters an User ID that is already present in the database, on clicking the Check Availability button it will display the following error:


49

If the user tries to register himself/herself with an User ID already existing in the database, then on selecting the Register button the following exception is prompted:


50

7.3

User LogIn

In the User Log In window if the user enters incorrect User ID and password that is different from the User ID and password with which the user registered, the following error message is displayed:


51

7.4

Image Registration

In the Image Registration window if the Student ID text field is empty the following error is displayed:


52

In the Image Registration window if the Upload button is clicked before selecting the video file to process and clicking the process button, the following error message will be displayed:


53

In the Image Registration window if the Process button is clicked without selecting the video file path to process, the following error message will be displayed:


54

In the Image Registration window if the Upload button is clicked before clicking the process button, the following error message will be displayed:


55

If the user enters an incorrect User ID that does not matches with any of the existing User ID present in the database, then the following error message will be displayed:


56

7.5

Training

In the Training window if the Course ID text field is empty then the following error message will be displayed:


57

In the Training window if the Session text field is empty then the following error message will be displayed:


58

In the Training window if the Process button is clicked first after entering only the Course ID and the Session, the following error message will be displayed:


59

If the user clicks the Download button after entering the Course ID and the Session and if no such images of the student exist corresponding to the Course ID and the Session, the following error message is displayed:


60

If the user at first clicks Upload Training Model button after entering only the Course ID and the Session, the following error message will be displayed:


61

7.6

Recognition

In the Recognition window if the Course ID text field is empty, the following error message will be displayed:


62

In the Recognition window if the Session text field is empty, the following error message will be displayed:


63

If the Download button is clicked and if no such training model exists corresponding to the entered Course ID and Session in the database, the following error message will be displayed:


64

If the user clicks the Recognize button after entering the Course ID, Session and downloading the corresponding training model without selecting or capturing an image to be recognized, the following error message will be displayed:


65

If the user clicks the Recognize button after entering the Course ID, Session and selecting an image file path or capturing an image using the capturing device for recognition before downloading the corresponding training model, the following error message will be displayed:


66

If the user clicks the Recognize button after entering the Course ID, Session and specifying an incorrect file path that is if the image file does not exist, the following error message will be displayed:


67

7.7

Attendance Update

In Attendance Update window after recognition if any student who were present in the image but were not recognized then only clicking on the Add IDs button without adding the IDs (comma separated) of those students in the appropriate field will display the following error message:


68

If any of the Student ID given by the user does not match with that of the Student ID in the database on clicking on the Add IDs button the following error message will be displayed:


69

After the user enters the undetected Student IDs, if he/she clicks on the Manual Update button keeping the Faculty ID text field empty, the following error message will be displayed:


70

After the user enters the undetected Student IDs, if he/she clicks on the Manual Update button keeping the Course ID text field empty, the following error message will be displayed:


71

After the user enters the undetected Student IDs, if he/she clicks on the Manual Update button keeping the Session text field empty, the following error message will be displayed:


72

If the Faculty ID entered does not matches with the database, the following error message will be displayed:

Figure 7.35: Error instance 6 The error says: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (’aams’,’attendance’,CONSTRAINT ’attendance ibfk 2’ FOREIGN KEY (’facultyid’) REFERENCES ’faculty master’ (’facultyid’))

If the Course ID entered does not matches with the database, the following error message will be displayed:

Figure 7.36: Error instance 7 The error says: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot add or update a child row: a foreign key constraint fails (’aams’,’attendance’,CONSTRAINT ’attendance ibfk 3’ FOREIGN KEY (’courseid’) REFERENCES ’course master’ (’courseid’))

73

8.

Future Scopes

• Further work can be done on this project to alert the student by sending SMS regarding their attendance. SMS alert can be given to the parent of the student. • This software system can be further designed to work properly during examination sessions where attendance is highly essential. This will not only eliminate student identification such as checking identification card or admit card of the student thus not interfering the ongoing examination process but also help in detecting whether the students are using any sort of unfair means during the course of examination. • The automatic attendance management system will not only be limited to school, colleges, it can be implemented in offices also thus monitoring the employee activities. • The system may be implemented along side with any existing surveillance systems for recognition and tracing purposes.

74

9.

References

1. Viola, Jones: Robust Real-time Object Detection, IJCV 2001 2. Cen, Kaiqi. ”Study of Viola-Jones Real Time Face Detector [PDF file]”, web.stanford.edu. N.p., 2017. Web. 1 August 2017. 3. ”Statistical classification”. Wikipedia. N.p., 2017. Web. 1 August 2017. 4. ”Face detection”. Wikipedia. N.p., 2017. Web. 1 August 2017. 5. ”Summed-area table”. Wikipedia. N.p., 2017. Web. 1 August 2017. 6. ”Machine Learning is Fun! Part 4: Modern Face Recognition with Deep Learning”. Medium. N.p., 2017. Web. 1 August 2017. 7. Nehemiah, Avi, ”Face Recognition with MATLAB [Video]”. MathWorks. 2017. Web 1 August 2017. 8. Firyn, Bartosz. ”Webcam Capture API”, Webcam Capture Sarxos. N.p., 2017. Web. 1 August 2017. 9. Lecture Notes on Software Engineering By H.S.Behra, Asst. Prof. K.K.Sahu and Asst. Professor Gargi Bhattacharjee. 10. ”Face detection”. facedetection.com. N.p., 2017. Web. 1 August 2017.

75

10.

Appendix A

Java Code Class: Main package aams; import import import import import import import import import import import

java.io.BufferedReader; java.io.File; java.io.FileReader; java.sql.Connection; java.sql.DriverManager; java.sql.PreparedStatement; java.sql.ResultSet; java.sql.Statement; java.text.DateFormat; java.text.SimpleDateFormat; java.util.Date;

import javax.swing.JOptionPane; import javax.swing.SwingUtilities; import javax.swing.UIManager; public class Main { static String static String static String static String static String static String public static { try {

matlabFunctionPath=null; dbname=null; dbusername=null; dbpassword=null; workingPath=null; adminAuthenticationPassword=”asutosh”; boolean isInteger(String str)

Integer.parseInt(str); } catch(Exception e) { return(false); } return(true); } public static void deleteAll(File path) { if(path.exists()) { File[] l = path.listFiles(); for (File f : l) { if (f.isDirectory()) deleteAll(f); else

76

f.delete(); } path.delete(); } } public static void deleteFiles(String path) { File file=new File(path); if(file.exists()) { File[] listOfFiles=file.listFiles(); for(int i=0;i=1) { if(!token[0].equalsIgnoreCase(””)) { for(int i=0;i”); prevb=new JButton(”0) { try { currentindex−−; if(listOfFiles[currentindex].getName().endsWith(”.jpg”)) { file=new File(listOfFiles[currentindex]. getAbsolutePath()); mypic=ImageIO.read(file); //System.out.println(”Read Ok”); img=mypic.getScaledInstance(200, 200, Image. SCALE SMOOTH); //System.out.println(”Scaled Ok”); imagel.setIcon(new ImageIcon(img)); //System.out.println(”setIcon done”); ridtf.setText(listOfFiles[currentindex].getName(). substring(0,listOfFiles[currentindex].getName ().lastIndexOf(” ”))); rnametf.setText(Main.getStudentNameByID(ridtf. getText())); prevb.setEnabled(false);

128

nextb.setEnabled(false); } else { JOptionPane.showMessageDialog(null, ”Not a .jpg file.”, ”Invalid image!”, JOptionPane. WARNING MESSAGE); } } catch(Exception e) { JOptionPane.showMessageDialog(null, e, ”Exception!”, JOptionPane.ERROR MESSAGE); } if(currentindex−1