Security using cognitive service

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 29 ISSUE 1 – MAY 2018 - ISSN: 2349 – 9303

Security using cognitive service B Venkatesan

P Praveen Kumar

Paavai Engineering College, Information Technology, [email protected]

Paavai Engineering College, Information Technology, [email protected]

A Kishore Chowdary

M Prabakaran

Paavai Engineering College, Information Technology [email protected]

Paavai Engineering College, Information Technology [email protected]

Abstract — Security using cognitive service is a project that can identify people by using their faces. Each face will have an unique measurements in terms of the parts like nose, ears, eyes, eyebrows, cheeks, chin and etc. The pictures of all the authenticated users will be stored in the face list as JSON scripts. These scripts are generated by the face API. Now, the photos of users will be taken for the authentication purpose and then processed using face API to generate the JSON script. Finally, this script is matched with the available face values in the face list. Here each face value that is stored in the face list will have its own unique ID. Whenever the measurements are matched, the availability of that specific face ID is checked in the face list. If it is available, then the person is authorized one. If there is no match for that specific face ID, then the person is concluded as unauthorized and concerned action can be taken over him. This can also be taken to a next level by using an IoT device namely Raspberry Pi for reaching people in a crowd without disturbing them or making them to know what we are doing. Thus, we will make use of a drone to carry the Raspberry Pi device in the air. Index Terms — Azure, Cognitive Service, Face API, IoT. ——————————  ——————————

1 INTRODUCTION

s

cannot assure that a security guard is capable of guarding 24/7 since the guard is a human and they too require some resting time for sure. Same way, considering a dog, it cannot guard all the time since it also requires the kind of maintenance in form of feeding, and maintaining the same. This means, even the dogs cannot be used for security purposes in a reliable manner. Coming to the next one namely the cameras and other security gadgets, all these are not completely secure and reliable at some point. Today there are lot of technologies that are available in market which can be used to jam all these devices from doing their works. Out of all, managing and maintain all these kind of security features will cost us a lot. Even the initial deployment of all these services is too high.

ecurity is most important concern in this modern world. As the technology grows on, the need of security is also increasing on. This security can be in the form of physical security, financial security, digital security or even personal security. For all these kind of security purposes, we will be depending on the security guards, security cameras, scanners, dogs and even some other electronic gadgets that can help us in achieving these goals. This paper is all about the security feature that takes the term security to a next level. Using this, we can implement security in an advanced level with minimal cost. For achieving this goal, we can make use of azure cognitive service that works with face API. To make this feature little more advanced, we will use IoT device so that we will be able to reach the people in a crowd with ease. Thus, we will be able to achieve security at the maximum level. With this we will be able to avoid spending money for maintaining security guards or security devices. Along with this, we can completely reduce the cost and time which we invest for the security. Thus, the security using cognitive service can be implemented for this.

3 Security Using Cognitive Service The Security Using Cognitive Service makes use of the modern technologies like face API, IoT and Windows apps. The user will be able to achieve the security goal with a minimal cost. The measurements of faces will be generated using face API and is then processed using applications.

2 EXISTING GENERAL SECURITY SYSTEMS

3.1 Microsoft Azure Cognitive Services

There are many systems that are in existence for achieving the goal of security. Security guards, security cameras like CCTV, security dogs and scanners have their own kind of approach in achieving this goal. But still all these have their own drawbacks in terms of their processing. Appointing a security guard requires for making a payment for the guard every month in the name of salary. Also, we

Microsoft Cognitive Services (formerly Project Oxford) are a set of APIs, SDKs and services available to developers to make their applications more intelligent, engaging and discoverable. Microsoft Cognitive Services expands on Microsoft’s evolving portfolio of machine learning APIs and enables developers to easily add intelligent features – such as emotion and video detection; facial, speech and vision recognition; and speech and language 47

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 29 ISSUE 1 – MAY 2018 - ISSN: 2349 – 9303 understanding – into their applications. Our vision is for more personal computing experiences and enhanced productivity aided by systems that increasingly can see, hear, speak, understand and even begin to reason.

This means that Custom Vision Service identifies whether an image is of a particular object, but not where that object is within the image. Very few images are required to create a classifier -- 30 images per class is enough to start your prototype. The methods Custom Vision Service uses are robust to differences, which allows you to start prototyping with so little data. However, this means Custom Vision Service is not well suited to scenarios where you want to detect very subtle differences (for example, minor cracks or dents in quality assurance scenarios).

3.1.1 Academic Knowledge API With this service, you will be able to interpret user queries for academic intent and retrieve rich information from the Microsoft Academic Graph (MAG). The MAG knowledge base is a web-scale heterogeneous entity graph comprised of entities that model scholarly activities: field of study, author, institution, paper, venue, and event. The MAG data is mined from the Bing web index as well as an in-house knowledge base from Bing. As a result of on-going Bing indexing, this API will contain fresh information from the Web following discovery and indexing by Bing. Based on this dataset, the Academic Knowledge APIs enables a knowledge-driven, interactive dialog that seamlessly combines reactive search with proactive suggestion experiences, rich research paper graph search results, and histogram distributions of the attribute values for a set of papers and related entities.

3.1.6 Linguistic Analysis These APIs provide access to natural language processing (NLP) tools that identify the structure of text. The current release provides three types of analysis namely Sentence separation and tokenization, Part-of-speech tagging, Constituency parsing.

3.1.7 Speaker Recognition Voice has unique characteristics that can be used to identify a person, just like a fingerprint. Using voice as a signal for access control and authentication scenarios has emerged as a new innovative tool – essentially offering a level up in security that simplifies the authentication experience for customers. Speaker Verification APIs can automatically verify and authenticate users using their voice or speech.

3.1.2 Bing Autosuggest Documentation Bing Autosuggest API lets you send a partial search query term to Bing and get back a list of suggested queries that other users have searched on. For example, as the user enters each character of their search term, you'd call this API and populate the search box's dropdown list with the suggested query strings.

3.1.8 Face API This is a cloud-based service that provides the most advanced face algorithms. Face API has two main functions: face detection with attributes and face recognition. The following API is going to be used for implementing this project. Let us have a deeper overview on all the features that are available in the API.

3.1.3 Bing Entity Search The Entity Search API sends a search query to Bing and gets results that include entities and places. Place results include restaurants, hotel, or other local businesses. For places, the query can specify the name of the local business or it can ask for a list (for example, restaurants near me). Entity results include persons, places, or things. Place in this context is tourist attractions, states, countries, etc.

3.1.8.1 Face Detection Face API detects up to 64 human faces with high precision face location in an image. And the image can be specified by file in bytes or valid URL.

3.1.4 Computer Vision API Version 1.0 The cloud-based Computer Vision API provides developers with access to advanced algorithms for processing images and returning information. By uploading an image or specifying an image URL, Microsoft Computer Vision algorithms can analyze visual content in different ways based on inputs and user choices.

3.1.5 Custom Vision Service Custom Vision Service is a tool for building custom image classifiers, and for making them better over time. For example, if you want a tool that could identify images of "Daisies", "Daffodils", and "Dahlias", you could train a classifier to do that. You do so by providing Custom Vision Service with images for each tag you want to recognize. Custom Vision Service works best when the item you are trying to classify is prominent in your image. Custom Vision Service does "image classification" but not yet "object detection."

Fig 3.1.8.1 Face Detection Rectangles Face rectangle (left, top, width and height) indicating the face location in the image is returned along with each detected face. 48

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 29 ISSUE 1 – MAY 2018 - ISSN: 2349 – 9303 Optionally, face detection extracts a series of face related attributes such as pose, gender, age, head pose, facial hair and glasses.

After a person group has been created and trained, identification can be performed against the group and a new detected face. If the face is identified as a person object in the group, the person object will be returned.

3.1.8.2 Face Recognition

3.1.8.6 Understanding and using face landmarks

Face recognition is widely used in many scenarios including security, natural user interface, image content analysis and management, mobile apps, and robotics. Four face recognition functions are provided: face verification, finding similar faces, face grouping, and person identification.

Face landmarks are a series of specifically detailed points on a face; typically points of face components like the pupils, canthus or nose. Face landmarks are optional attributes that can be analysed during face detection. You can either pass 'true' as a Boolean value to the returnFaceLandmarks query parameter when calling the Face Detect or use the returnFaceLandmarks optional parameter for the FaceServiceClient class DetectAsync method to include the face landmarks in the detection results.

3.1.8.3 Face Verification Face API verification performs an authentication against two detected faces or authentication from one detected face to one-person object.

By default, there are 27 predefined landmark points. The following figure shows how all 27 points are defined:

3.1.8.4 Face Grouping Given one set of unknown faces, face grouping API automatically divides them into several groups based on similarity. Each group is a disjointed proper subset of the original unknown face set and contains similar faces. And all the faces in the same group can be considered to belong to the same person object.

3.1.8.5 Face Identification Face API can be used to identify people based on a detected face and people database (defined as a person group) which needs to be created in advance and can be edited over time. The following figure is an example of a person group named "myfriends". Each group may contain up to 10,000-person objects. Meanwhile, each person object can have up to 248 faces registered.

Fig 3.1.8.2 Person Group Fig 3.1.8.3 Face Landmarks

The points returned are in units of pixels, just like the face rectangular frame. Therefore, making it easier to mark specific points of interest in the image. The following code demonstrates retrieving the locations of the nose and pupils: var faces = await faceServiceClient.DetectAsync(imageUrl, returnFaceLandmarks:true); foreach (var face in faces) { var rect = face.FaceRectangle; var landmarks = face.FaceLandmarks; double noseX = landmarks.NoseTip.X; double noseY = landmarks.NoseTip.Y; double leftPupilX = landmarks.PupilLeft.X; double leftPupilY = landmarks.PupilLeft.Y; double rightPupilX = landmarks.PupilRight.X; double rightPupilY = landmarks.PupilRight.Y; } 49

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 29 ISSUE 1 – MAY 2018 - ISSN: 2349 – 9303 framework and the same will be deployed into IoT device. The application will be written in a format where it will be working with the support of face API for generating face landmarks.

4 Internet of Things IoT or Internet of Things is a cut edge advanced technology that is conquering the technical market today. It has leverages a lot by providing different kind of services for different set of users. The concept works by making use of an IoT device that can be connected and controlled via internet by making use of applications. There are different set of IoT devices that are available in market. Each of these devices have their own kind of usage and specifications. For implementation of this project, the IoT device namely Raspberry Pi is going to be used. To make the device move places, a drone is going to be used.

5.1 Drone

4.1 Raspberry Pi

6.0 Working of Security Using Cognitive Services

A drone is a device that can fly and capturing the images of a place, person, object or an entity from air. This setup can be used for porting the raspberry pi device to places so that the images of the people in an area can be captured and those images can be used for authenticating the people who are available in there.

SOLUTION USING THESE COMPONENTS

All the above-mentioned technologies will be combined into one and then they will be used in different ways to implement this security feature. Two applications will be deployed where one will be deployed to the IoT device where as one more application is going to act as a client to receive the images and all other metrices that are given by IoT device. The application in IoT device will be taking images of people with the help of camera that is available in it. After taking the images, it will be sending the same to client application where in which those images will be processed. The client application will have two options where in which one option is to store images into face group where as one more image will be for matching the faces. If the new user has to be added into the group, the taken picture can be added into it. If a taken picture has to be matched with available face IDs in the face group, then it can be compared with the list of available face IDs. If the ID gets matched, then the user will be an authenticated person. If the ID does not match, then the user will be considered as unauthenticated person. Here the face IDs will be generated automatically by the face API when the image is processed in the application.

4.1 Raspberry Pi This is an IoT device that can be connected to internet and triggered by writing a few set of programmers. The Raspberry Pi is like a mini computer without large memory space. It has something called GPIO pins that can connect with devices and triggers actions of the same. Raspberry is platform independent and hence it can be used in different scenarios. For implementation of this project, we are going to make use of Windows 10 IoT core operating system. The application that is going to be implemented is going to be an universal windows application.

The Raspberry Pi will be using cloud service to send data between the nodes. To make this setup a portable one, the raspberry pi can be fixed in a drone and it can be flied to the people and images of them can be captured in that way. Even the same application ca n be installed in PC and can be connected to camera to achieve same goal. Here we can use the CCTV cameras for same purpose. This is how the whole setup works. Furthermore, the same can also be enhanced to a more advanced level by adding the other APIs like speech recognition and synthesis to make this whole setup an automated one. But for present consideration, this setup will be enough to achieve the titled goal of security.

4.1.1 Windows 10 IoT core Windows 10 IoT Core is the smallest version of the Windows 10 editions that leverages the Windows 10 common core architecture. These editions enable building low-cost devices with fewer resources. Development for Windows 10 IoT Core leverages the Universal Windows Platform.

7.0 Algorithm of Model 1. 2. 3.

5 Universal Windows Application The universal windows application is the supported application format of raspberry pi since windows 10 IoT core operating system is used. An application is going to be developed by using .Net

4. 5. 50

Start. Capture image from camera available in raspberry Pi. Send the application from raspberry pi to the client application in PC. Process image using client application and generate the face landmarks using face API. If the image has to be stored into the face group, add the

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 29 ISSUE 1 – MAY 2018 - ISSN: 2349 – 9303 6. 7. 8. 9.

[4] Microsoft ―web for entities and places― Link https://docs.microsoft.com/en-us/azure/cognitive-services/bingentities-search/search-the-web [5] Microsoft ―Bing Image Search‖ Link https://docs.microsoft.com/en-us/azure/cognitive-services/bingimage-search/ [6] Microsoft ―Microsoft Speech API‖ Link https://docs.microsoft.com/en-us/azure/cognitiveservices/speech/home [7] Microsoft ―Bing Spell Check‖ Link https://docs.microsoft.com/en-us/azure/cognitive-services/bingspell-check/ [8] Microsoft ―Computer Vision API Version 1.0‖ Link https://docs.microsoft.com/en-us/azure/cognitiveservices/computer-vision/home [9] Microsoft ―Custom Decision Service‖ Link https://docs.microsoft.com/en-us/azure/cognitiveservices/custom-decision-service/custom-decision-serviceoverview [10] Microsoft ―Face API‖ Link - https://docs.microsoft.com/enus/azure/cognitive-services/face/overview [11] Microsoft ―Emotion API‖ Link https://docs.microsoft.com/en-us/azure/cognitiveservices/emotion/home [12] Nithya Mathan ―Implementation Of Face API Using Microsoft Azure And Visual Studio‖ Link http://www.csharpcorner.com/article/implementing-a-face-api-using-microsazure/ [13] Viral Jain ―Introduction To Face API In Azure‖ Link http://www.c-sharpcorner.com/article/introduction-to-face-api-inazure/ [14] Gourav Jain ―Understanding Face API In Azure‖ Link http://www.c-sharpcorner.com/article/understanding-face-api-inazure/ [15] Sibeesh Venu ―Create A Microsoft Azure Cognitive Service Face API Application In 30 Minutes‖ Link - http://www.csharpcorner.com/article/create-a-microsoft-azure-cognitiveservice-face-api-application-in-30-minutes/ [16] Manikandan Murugesan ―Deploying The Face API Using Visual Studio 2017‖ Link http://www.csharpcorner.com/article/deploying-the-face-api-using-visualstudio-2017/ [17] Vijayaragavan S ―Retrieving Face Attributes Using Cognitive Service Face API In UWP With Azure, XAML And C#‖ Link http://www.c-sharpcorner.com/article/retrieving-face-attributesusing-cognitive-service-face-api-in-uwp-with-azure-x/ [18] Prasanna Murali ―Face API In C# Using Cognitive Services‖ Link - http://www.c-sharpcorner.com/article/face-api-in-c-sharpusing-cognitive-services/ [19] Saad Mahmood ―Build A Windows Phone And Store App For Project Oxford Face API‖ Link http://www.csharpcorner.com/UploadFile/5c2d70/build-a-windows-phoneand-store-app-for-project-oxford-face/ [20] Ravi Shankar ―Install Windows 10 IoT Core And Remote Login On Raspberry Pi 3‖ Link http://www.csharpcorner.com/article/install-windows-10-iot-core-and-remote/ [21] Nithya Mathan ―Creating An IoT Hub On The Azure Portal‖ Link - http://www.c-sharpcorner.com/article/creating-an-iot-hub-

face ID and landmark measurements into the face group. If the face has to be matched with the generated face IDs that are available in face group, compare the face IDs. If the face ID matches with any other available IDs in the face group, the person is authorized. If the face ID does not match with available IDs in the face group, the person is unauthorized. Stop.

SCOPE OF THE MODEL 8.0 Advantages     

Initial investment for this setup is very less when compared to manual models. Designing is very simple. More reliable when compared to physical model. Can be used all time seamlessly without having any kind of downtime. Security can be achieved to the maximum level using this feature.

9.0 Technologies and Components Required   

Visual Studio 2017. Raspberry Pi 3. Drone.

10 CONCLUSION The implementation of this concept will help us in achieving this goal of security in a reliable way with more advanced features. Along with this, the initial deployment cost will be avoided. The process will make the concept of security an advanced one. The features like these face APIs that are mostly used in software products can also be used in this way proving that any technology can be used in any format by making use of IoT and a creative mind. The security service will be taken to places using the drone which is far more advanced feature. Even a large area with mass crowd can be easily monitored by making use of this feature. This even leads to a development of lot more features in the domain of security. These kinds of feature will also bring changes in the more complex and secure locations like army bases or secured zones where security must be monitored all the time.

REFERENCES [1] Microsoft ―What is Cognitive Services?‖ Link https://docs.microsoft.com/en-us/azure/cognitiveservices/welcome [2] Microsoft ―Academic Knowledge API‖ Link https://docs.microsoft.com/en-us/azure/cognitiveservices/academic-knowledge/home [3] Microsoft ―Bing Autosuggest‖ Link https://docs.microsoft.com/en-us/azure/cognitive-services/bingautosuggest/ 51

INTERNATIONAL JOURNAL FOR TRENDS IN ENGINEERING & TECHNOLOGY VOLUME 29 ISSUE 1 – MAY 2018 - ISSN: 2349 – 9303 on-the-azure-portal/ [22] Nithya Mathan ―Creating A Simulated Device For IoT Hub Using .NET‖ Link http://www.csharpcorner.com/article/simulated-devices-for-iot-on-our-pcusing-net/ [23] Mahesh Chand ―What Is Azure IoT Edge‖ Link http://www.c-sharpcorner.com/article/what-is-azure-iot-edge/ [24] Jaymin Dangi ―Overview Of IoT Security Risks Associated With The Healthcare Industry‖ Link - http://www.csharpcorner.com/article/overview-of-iot-security-risksassociated-with-healthcare-industry/ [25] Vijayaragavan S ―Creating IoT Led Blinking App With Raspberry Pi 2‖ Link - http://www.c-sharpcorner.com/article/iotled-blinking-uwp-app-with-raspberry-pi-2/ [26] Abdul Rasheed Feroz Khan ―Kickstart IoT (Internet of Things) With Raspberry Pi‖ Link http://www.csharpcorner.com/article/kick-start-for-iot-internet-of-things-withraspberry-pi/ [27] Jayesh Vyas ― Introduction To Internet of Things (IOT)‖ Link http://www.c-sharpcorner.com/article/introduction-to-iot/

52