prospects and peers scan business card and easily add his/her to their contacts. 2. Advertising: - One .... In this theytake account of QR codes basics, its real time application in day to day life and ..... decoding of such codes on Android devices.
A Dissertation Report on
Region Identification and Decoding Of Security Markers Using Image Processing Tools Submitted for the partial fulfillment of the award of Master of Technology in Computer Science
By Pragya Mishra (14536)
To Department of Computer Science, AIM&ACT Under the supervision of Dr. Amol P Bhondekar Pricipal Scientist CSIR-CSIO, Chandigarh
Mr. K.F. Rahman Department of Computer Science, AIM & ACT Banasthali Vidyapith- Rajasthan
BANASTHALI VIDYAPITH
(2014-16)
Certificate Certified that
Pragya Mishra has carried out the research work titled
“Region Identification and Decoding of Security Markers” from 6 July, 2015 to
10 may, 2016
for the award of the Master of Technology in
Computer Science from CSIR-CSIO, Chandigarh under my supervision. The thesis embodies result of original work and studies carried out by student herself and the contents of the thesis do not form the basis for the award of any other degree to the candidate or to anybody else.
(Name of Supervisor/Guide)
Designation:
Place: CSIR-CSIO, Chandigarh Date:
i|Page
Certificate This is to certify that Ms Pragya Mishra, Roll No. 14536 a student of Master of Technology in Computer Science
4th Semester of Banasthali
Vidyapith has completed the research work titled “Region Identification and Decoding of Security Markers” in CSIR-CSIO, Chandigarh of One year duration in partial fulfillment of Master of Technology in Computer Science.
Internal Guide Mr. K.F. Rahman Banasthali Vidyapith
ii | P a g e
Abstract
QR (Quick Response) Code is 2-Dimensional barcode, developed by Denso WaveJapan in 1994 which is now-a-days very popular. The aim of this work is to develop covert, colored QR Codes in which, capacity to store data is increased and security is also enhanced as compared to traditional QR codes, which is composed of only black and white colors. To accomplish this, first of all, we have generated three QR codes having red, blue and green color and merged them to generate RGB Colored QR code by using python, PyQRCode, QR Tool and OpenCV. Decoding of RGB Colored QR Code is one of the challenging tasks since, printers are CMYK based, so after printing QR on paper its color changes. To deal with this problem, encoding of QR is being done in CMYK color space. In this, data is encoded in cyan, magenta and yellow channels and while decoding, information is decoded from complementary red, green and blue channels resulting from capturing devices. Further, to test parallel encoding capability of QR Code, we have carried out encoding and decoding operations of QR Codes in HSV color space. Results shows that we can encode more than three messages parallely in colored QR Code. Developed patterns will have a significant importance in covert applications.
Keywords— CMYK, Decoding, Encoding, HSV, OpenCV, PyQRCode, QR Code, RGB.
iii | P a g e
Acknowledgement
I have taken efforts in this project. However it would not have been possible without the kind support and help of individuals and organization. I wish to express my gratitude to all those individuals who have contributed their ideas, time and energy in this project. First of all, I would like to express my deep sense of gratitude towards Director CSIO and Dr. G.N. PUROHIT, Dean AIM & ACT, Banasthali University for providing me the opportunity to work in one of the well reputed branches of CSIR. I deem it as a proud privilege to express my sincerest regards and gratitude towards Dr. Amol P Bhondekar, my project guide and Principal Scientist -CSIO, without him this project would not have been possible. His professional approach, mentoring and style will remain with me forever as I continue my career. I express my sincerest gratitude towards Mr. Ritesh Kumar Scientist- CSIO, who always provided their support and motivation. The wealth of knowledge, guidance, and technical ideas that he has brought into this project is invaluable. It is my privilege to thank Mr. Shankho Turjo Sarkar (PhD. Scholar), Mr. Shambo Roy Chowdhury (PhD. Scholar) and all the helping hands of Agrionics (V1) lab, for their constant encouragement, the wealth of knowledge, guidance and technical ideas they shared, in this project is invaluable and without them it would have not been possible to achieve the target.
Lastly, but most importantly, I thank my parents, family and friends, for their immense love and support. Pragya Mishra M.tech.(Computer Science) Smart ID:ABMTC14199
iv | P a g e
Table of Contents Certificate ..................................................................................................................i Certificate ..................................................................... Error! Bookmark not defined. Abstract ................................................................................................................... iii Acknowledgement .................................................................................................... iv Table of Figures ................................................................................................... viii List of Tables ............................................................................................................ x List of Abbreviations ................................................................................................ xi CHAPTER 1:
INTRODUCTION ....................................................................... 1
1.1
Introduction: ................................................................................................ 1
1.2
Barcodes: .................................................................................................... 1
1.3
QR Code: .................................................................................................... 2
1.4
Versions of QR Code:.................................................................................. 3
1.5
Structure of QR Code: ................................................................................. 5
1.6
Characteristics of QR Code: ........................................................................ 7
1.7
Advantages of QR Code: ............................................................................ 9
1.8
Applications of QR Codes: ........................................................................ 10
1.9
Difference between Barcode and QR Code: ............................................... 12
1.10
Invisible Ink: ......................................................................................... 13
1.11
Challenges with Covert Colored QR Codes: ........................................... 15
CHAPTER 2: 2.1
LITERATURE SURVEY ......................................................... 16
Understanding QR Codes Compared With Barcodes: ................................ 16
v|Page
2.2
Encoding And Decoding Techniques: ........................................................ 17
2.3
Various Approaches For Increasing Capacity ............................................ 28
CHAPTER 3:
MOTIVATION AND OBJECTIVES ....................................... 32
3.1
Motivation ................................................................................................. 32
3.2
Objectives ................................................................................................. 32
CHAPTER 4:
METHODOLOGY .................................................................... 34
4.1
User Interface "PIMAD"........................................................................... 34
4.2
Algorithm For QR Region Identification ................................................... 34
4.3
Proposed Encoding Steps........................................................................... 36
4.4
CMYK ColorSpace ................................................................................... 37
4.5
Proposed Decoding Steps .......................................................................... 41
4.6
HSL and HSV Colorspace ......................................................................... 42
4.7
Encoding and Decoding in HSV ................................................................ 46
4.8
Image Segmentation .................................................................................. 47
4.9
Techniques For Image Segmentation ......................................................... 48
4.10
Thresholding .......................................................................................... 49
CHAPTER 5:
DESIGN OF EXPERIMENTS ................................................. 51
5.1
Hardware Interfaces:.................................................................................. 51
5.2
Software Interfaces: ................................................................................... 51
5.3
Python( Open Source) .............................................................................. 51
5.4
OpenCV( Open Source) ............................................................................ 52
5.5
PyQRCode( Open Source) ........................................................................ 53
5.6
QR Tools( Open Source) .......................................................................... 53
5.7
Design For Generation of QR Code ........................................................... 53
5.8
Design of Experiment For Decoding Of QR Code ..................................... 55
vi | P a g e
CHAPTER 6:
RESULTS AND DISCUSSION ................................................ 56
6.1
Developed Interface................................................................................... 56
6.2
Results For Region Identification In QR Code ........................................ 57
6.3
Results For Generation of QR Code ......................................................... 58
6.4
Results for Decoding of QR Code .............................................................. 61
6.5
Result for Encoding QR Code in HSV ...................................................... 62
6.6
Result for Decoding QR Code in HSV ...................................................... 62
CHAPTER 7:
CONCLUSION AND FUTURE SCOPE .................................. 64
APPENDICES ....................................................................................................... 65 7.1
APPENDIX-A: Diagrams And Documentation Tool ................................. 65
7.2
APPENDIX-B: Theoretical And Conceptual Framework:.......................... 67
References .............................................................................................................. 68 List of Publications ................................................................................................. 71
vii | P a g e
Table of Figures Figure 1-1. One Dimensional Barcode. ...................................................................... 2 Figure 1-2. QR Code ................................................................................................. 3 Figure 1-3. Colored QR Code .................................................................................... 3 Figure 1-4. Structure of QR Code .............................................................................. 6 Figure 5-1. Flow Diagram for Region Identification ................................................ 35 Figure 5-2. Encoding Procedure............................................................................... 37 -Figure 5-4. Subtractive Colors ................................................................................ 38 Figure 5-5. RGB-CMYK Color Space Model .......................................................... 38 Figure 5-6. Decoding Procedure .............................................................................. 40 Figure 5-7. HSV Model ........................................................................................... 43 Figure 5-8. Amplitude v/s Wavelength ................................................................... 43 Figure 5-9. Conversion from RGB to HSV .............................................................. 44 Figure 5-10. Conversion From RGB to HSL ............................................................ 45 Figure 5-11. Flow Diagram for Encoding in HSV .................................................... 46 Figure 5-12. Flow diagram for Decoding in HSV..................................................... 47 Figure 6-1. Design for Generation of QR Code ........................................................ 54 Figure 6-2. Design for Decoding of QR Code .......................................................... 55 Figure 7-1."PIMAD" interface ................................................................................. 56 Figure7-2.Original image ......................................................................................... 57 Figure 7-3. HSV image ............................................................................................ 57 Figure7-4RGB histogram......................................................................................... 57 Figure 7-5. Region Identification ............................................................................. 57 Figure 7-6. Encoded image I .................................................................................... 59 Figure 7-7. Encoded image II................................................................................... 59 Figure 7-8. Encoded image III ................................................................................. 59 Figure 7-9. Generated RGB colored QR Code ......................................................... 59 Figure 7-10. CMYK Encoding from RGB ............................................................... 60 viii | P a g e
Figure 7-11. CMYK Colored QR Code.................................................................... 60 Figure 7-12. Result for Encoding in HSV ................................................................ 62 Figure 7-13. Result for Decoding in HSV ................................................................ 63 Figure 9-1. Level 0 DFD .......................................................................................... 65 Figure 9-2. Level 1 DFD .......................................................................................... 66 Figure 9-3. Level 2 DFD .......................................................................................... 66
ix | P a g e
List of Tables Table 1-1. Versions of QR Code ................................................................................ 4 Table 1-2. Encodable Character Set ........................................................................... 7 Table 1-3. Data characters per symbol ....................................................................... 9 Table 1-4. Difference between QR Code and Barcode ............................................. 12 Table 1-5. Invisible Ink Examples ........................................................................... 14 Table 4-1. Conversion from RGB to CMYK ............................................................ 39 Table 4-2. Conversion from CMYK to RGB ............................................................ 40 Table 6-1. Result for Generation of QR Code .......................................................... 58 Table 6-2. Results for Decoding.............................................................................. 61
x|Page
List of Abbreviations
1. AIM: Automatic Identification Manufacturer. 2. CMYK: Cyan, Magenta, Yellow, Key 3. DIPT: Double Inverse Perspective Transformation. 4. EDI: Electronic Data Interchange. 5. HSL: Hue, Saturation, Lightness. 6. HSV: Hue, Saturation, Value. 7. ISO: International Standard Organization. 8. JIS: Japan Industrial Standards. 9. NLM: Non Location Map. 10. QR: Quick Response. 11. RFID: Radio Frequency Identification. 12. RS: Reed Solomon.
xi | P a g e
CHAPTER 1: INTRODUCTION
1.1
INTRODUCTION:
Covert (meaning invisible to human eye) Quick Response codes are two dimensional symbology, focused with the goal of high speed reading and encoding capacity compared to traditional barcodes. They have taken control over the market due to factors like, small size, readability from any direction, dirt and damage resistant, high capacity, kanji and kana encoding capability. Quick Response code (QR) is a type of matrix barcode, first designed for an automotive industry by Denso Wave in Japan (1994) and today is admired in industries due to its fast readability and greater storage capacity as compared to barcode[1] . It contributed greatly for the management of automotive industry to work efficiently in a wide range of tasks from production to shipping to the issuing of transaction slips. The QR Code became a medium which was indispensable and could store a great amount of information on these processes. QR code has several advantages over traditional barcodes due to which it is preferred in industry. First, QR code has great error amendment capability, which can restore 30% data for the maximum error correction level i.e. level H. Second, QR Code is capable of 360 degree (Omni-directional), high speed reading. QR Code accomplishes this task through position detection patterns located at the three corners of the symbol. These position detection patterns guarantee stable high-speed reading, circumventing the negative effects of background interference. Third, QR code supports various different encoding types and versions. One can choose an appropriate encoding type and version to reduce the size of QR code. 1.2
BARCODES:
A barcode is an optical machine-readable representation of data and used to encode information on different products and services. Initially it is represented by various systematically arranged parallel lines with varying width and spacing between them and are referred as linear or one-dimensional (1D) [2]. 1|Page
Figure 1-1. One Dimensional Barcode [3]
Later they emerged into rectangles, dots, hexagons and also other geometric representation in two dimensions (2D). Despite 2D patterns use a variety of different symbols, they are also generally referred to as barcodes. Initially barcode readers are being used to read barcode but now-a-days various scanners and softwares are available on various devices such as on our smart phones to read them. One dimensional (1-D) barcodes are less secure as compared to two dimensional (2-D) barcodes as former one is very easy to decode by just scanning the lines and spaces[2]. Scanning of one dimensional barcodes must be in one direction for better readability purpose while in two dimensional barcodes wide range of scanning angles are available to scan them making readability of 2-D barcodes much better as compared to 1-D barcode. Storage size of 1-D barcodes is also small as compared to 2-D barcodes as in 1-D we only store data in one dimension while in 2-D we can store in various dimensions. [1]. Due to all such limitation of 1D Barcode storage, only a few data which demands less security or are not much vulnerable like product identification is stored in 1D Barcode. 1.3
QR CODE:
QR code (Quick Response Code) is the trademark for a type of barcode. In traditional one dimensional bar codes we can only store a maximum of approximately 20 digits, whereas in QR Code we can handle several dozen to several hundred times more information. In QR Code we are capable of encoding the same amount of data in approximately one-tenth of the space as compared to the traditional barcode since in QR code we can store data both horizontally and vertically. For a smaller printout size, Micro QR Code is also available.
2|Page
Figure 1-2. QR Code
Figure 1-3. Colored QR Code
In order to increase the storage capacity of information being encoded in QR code, different colors can be used [2]. The information encoded is directly proportional to Log2N, where N is number of colors used . Therefore, using more number of colors means encoding more information. Figure 1.3 show colored QR code, containing three colors. This means the information encoded in Figure 1.3 is Log2 (3). Since the perceived surface color also depends on illuminant spectrum with surface reflectance spectrum so using more numbers of colors will raise problems [2]. Furthermore while decoding, determining the color index of every surface plot is a challenging task [2]. Other parameters include: color drifts, color fading, square patterns not being identified correctly, unknown or poorly calibrated camera color response, camera blur, color mixing from two nearby patches due to blur, quantization and noise [2]. One can choose an appropriate encoding type and version to reduce the size of QR code. 1.4
VERSIONS OF QR CODE:
In QR Code we can encrypt byte, binary, numeric, alphanumeric and kanji data along with symbols, binary data, control codes, and multimedia data unlike one dimensional barcodes [4]. There are 40 different versions of QR Code range from version-1 (which contains 21*21 modules) to version-40 (which contains 177*177 modules) in which each version expands from 4 modules in the right and 4 to the bottom. We can encode data in both horizontal and vertical direction of these modules. Examples of QR code with different versions can be seen in following Table 1.1:
3|Page
Table 1-1. Versions of QR Code
Version 1 QR Code
Version 10 QR Code
Version 5 QR Code
Version 20 QR Code
Version 40 QR Code Version 30 QR Code
4|Page
One of the important characteristics of QR code is it can be read even when the image is blur, damaged or dirty. This is possible due to Error Correction Capability which is carried out in QR Code using Reed Solomon error correction due to which decoding can be done effectively and efficiently and can restore all the encoded data even if 35 percent of encoded image is damaged [4]. 1.5
STRUCTURE OF QR CODE:
In QR code each region has specified functionalities which makes it easy to understand and in use for its users. Each QR Code can be defined mainly in two regions: First is Encoded Region and second one is Function Pattern. Function Pattern can be sub-divided into finder, separator, timing patterns and alignment patterns [1]. Individual functions of each region of QR pattern are as following: 1. Finder pattern: These patterns are situated at the three corners of QR code i.e. at top-right, top-left and lower-left corner and determines the position, size, and the angle of inclination of QR code [2].Each of these patterns can be considered as a three superimposed concentric squares which is consisted of 7*7 dark , 5*5 light and 3*3 dark modules [5] and ratio of module width of each finder pattern is 1:1:3:1:1 [1].
2. Alignment Pattern: This pattern is used to correct the distortion of QR code. Central coordinate of this pattern as shown in figure 1.6 is known to correct the distortion. For easier detection of the central coordinate a black isolated cell is placed in the alignment pattern [6].
3. Timing pattern: This pattern is arranged in both horizontal and vertical directions and used to identify the central coordinate of each cell in QR symbol as shown in above figure [5].
4. Quiet zone: This pattern provides the required margin for reading the QR code so that it becomes much easier for decoder to identify the code among the rest of the images. In figure 1.4 it is shown by green color [6]. 5|Page
Figure 1-4. Structure of QR Code
5. Data Area: Data Area contains the data which has to be encoded in binary numbers '0' and '1' and will be arranged based on the encoding rule. This area also contain reed Solomon codes and have error correction functionality [5].
6. Version Information: This gives the information about the version of QR code. Versions of QR code range fromversion-1 to version-40 each exceeding by four modules in both horizontal and vertical direction. In figure 1.6 it is shown by blue color.
7. Format Information: The red color in the figure 1.6 represents the area which includes the format information of the particular QR code generated. The format string is 15 bits long which contains the error correction level and mask pattern. There are eight mask patterns used in QR codes used to change the output matrix. Each pattern changes the bits according to the coordinates in the QR matrix. The purpose of the mask pattern is to make the QR code easier to be read by the scanner. Error correction codes are used to restore the data when a part of QR code is missing. The restoration rate varies according to the error correction level used. For example:
6|Page
if the damaged area is 15% of the entire code size. The data can be recovered by using error correction level "M". 1.6
CHARACTERISTICS OF QR CODE: 1. Encodable character set: - QR code has high capacity to encode characters ranging from numeric to alphanumeric, kanji and kana characters to Unicode characters as shown in table 1.2. Since all of the Shift JIS characters have representations in UTF-8, it is possible to use byte mode for Kanji with UTF-8 encoding. However, Kanji in UTF-8 are encoded with three bytes (or four, in rare cases), whereas Shift JIS characters are encoded with two or one bytes. In other words, it will not be possible to fit as many characters into the QR code if using UTF-8 in byte mode for Kanji. Using Kanji mode for Shift JIS Kanji gives the highest capacity. Table 1-2. Encodable Character Set
1.
Numeric Data (Digits 0-9).
2. Alphanumeric Data (Digits 0-9, space, $, %, +, *, -, /, ., :, upper case letters (A to Z).
3. 8-bit byte data.
4.
7|Page
Kanji characters
2. Representation of data: - A dark module is binary one and white module is binary zero [4]. 3. Symbol size (Not including Quiet Zone):- Version 1 (21×21 modules) to Version 2 (177×177 modules), versions 1 to 40 increasing by 4 modules in both horizontal and vertical directions. We can encode data in both horizontal and vertical direction of these modules. One of the important charectistics of QR code is it can be read even when the image is blur, damaged or dirty.[4].
4. Selectable error correction levels:- Four levels named L (recovers 7% data), M (recovers 15% data), Q (recovers 25% data), H (recovers 30% data) are present [4]. Before the error correction code words are generated, the data code words are broken into smaller blocks. Error correction code word contains two groups and within each group, the data code words may be further broken into blocks sequentially. The smaller code words need not be broken into smaller blocks. The bit sequences for each error correction level L, M, Q and H are 01, 00, 11, and 10 respectively. Reed Solomon error correction method is adopted generally for generating code words of error correction including polynomial long division method along with Galois field. Galois field is essentially a restricted set of numbers as well as some mathematical operations that creates numbers which are there in the existing set.
5. Data characters per symbol:- As the version goes on increasing, more data can be stored in the code with the point to be kept in mind that larger the version and smaller the encoded data in it leads to easy and fast access of finding the finder patterns in order to decode the information. Maximum data capacity is for version 40 (177×177 modules) and minimum error correction level L. The Capacity to encode data of such QR code is shown in Table 1.3.
8|Page
Table 1-3. Data characters per symbol
1.7
1.
Numeric Data- 7,089 characters.
2.
Alphanumeric Data- 4,296 characters
3.
8 bit-byte Data- 2,953 characters.
4.
Kanji Data- 1,817 characters.
ADVANTAGES
OF QR CODE:
1. High Capacity: -While conventional barcodes have capability of encoding 20 digits app. QR codes can encode several hundreds of digits [7].
2. Small print out size: -It encodes data both horizontally and vertically, therefore it requires much less space as compared to barcodes [7].
3. Kanji and Kana encoding capability: -As it is first developed in Japan, it is capable of encoding data sets of Kanji and Kana characters[7].
5. Dirt and damage Resistant: -Data can be restored even if the symbol is partially dirty or damaged. Maximum of 30% code words can be restored [7].
9|Page
6. Readable from any direction: -In QR Code wide range of scanning angles are available to scan them making readability of 2-D barcodes much better as compared to 1-D barcode. QR code is capable of 360 degree reading, this task is accomplished by finder patterns located at three corners of code [7].
6. Structured appending feature:- QR code can be divided into multiple data areas and information stored in multiple QR code symbols can be reconstructed as a single data symbol and vice-versa. One data can be stored up to 16 symbols, allowing printing in a narrow area [7] .
7. Secure and Robust:- QR Codes are more secure and robust as compared to one dimensional barcodes as in one dimensional barcodes it is easy for someone to decode the encoded messages by carefully observing the patterns and spacing between them but with QR Code it is very difficult for someone to predict the encoded data just by observing the patterns [7].
8. Various versions available:- QR code supports various different encoding types and versions. One can choose an appropriate encoding type and version to reduce the size of QR code [6]. 1.8
APPLICATIONS OF QR CODES:
1. Business cards: - One can include his/her business name and address, website, blog, or even links to your social media properties. The QR code will let his/her prospects and peers scan business card and easily add his/her to their contacts. 2. Advertising: - One can link prospects to product videos, spec sheets, or “Buy Now” pages. Assigning a URL specific to the QR code in the ad with a re-direct is a great way to track how much traffic came to that page through print advertisement.
10 | P a g e
3. Create a campaign to increase your social following: - Including a QR code to instantly link to your Face book, Twitter, or LinkedIn page on print advertisements, direct mail pieces, or even your email signature.
4. QR codes to product packaging: - link customers to a page with useful resources like customer service phone numbers, user manuals, and related items.
5. Use QR codes for company or customer-facing events: - Add a QR code to an event ticket and link to a Google map, and RSVP page, or materials that attendees need to bring along.
6. Education: - QR code can be used by students to create resumes that link to other content such as their professional website or portfolio. One can create QR code for linking students to examples of quality work, whether it's PowerPoint or slide share for a class presentation.
7. Go green: - One can go green by using QR codes, saving some trees. Instead of taking out more printouts, instructions, announcements or assignments can be encoded in QR codes.
8. Code payments: - In November 2012, huge amount of payments were made from QR codes in Czech Republic. These codes were used to store bank account information and other credit/debit card information which were designed especially for payment provider applications.
9. Website login: - QR codes are shown on the login pages of the computer screen and when a registered user scans it, he/she will automatically logged in to the computer.
11 | P a g e
10. Virtual stores: - According to a study in June 2011, 14 million mobile users scanned the QR code. Wal-Mart, Procter and some other stores have adopted the concept of virtual stores
11. Covert applications: -Colored QR codes can be used for covert applications using invisible markers in future. 1.9
DIFFERENCE BETWEEN BARCODE AND QR CODE: Table 1-4. Difference between QR Code and Barcode
Features
QR Code
Barcode
High Capacity
Up to 7089 numeric digits
10-20 digits
Durability against soil and damage
Reading is possible up to 30% damage
Reading is impossible
Reduced space
40 digits numeric (app. 5mm*5mm)
10 digits numeric (app. 50mm*20mm)
360 degree reading
Supports 360 degree reading
Horizontal reading
Language supported
Numeric, alphanumeric, kanji and kana
Numeric, alphanumeric
12 | P a g e
1.10 INVISIBLE INK: Inks that are invisible to the human eye have been available for years. Only in recent years has there been an interest in using these fluorescing or “invisible” inks for printing machine readable marks, such as bar codes. Just using a clear, “invisible” ink that has specific fluorescing properties, with most types of inkjet, thermal transfer or other common printing methods could not be simpler. As more people attempt to use conventional printing concepts to this specialized printing, there are challenges to the success in their efforts, especially if one cannot clearly see and easily inspect the result of their printing. With the possibility of fluorescent dyes, there is a new weapon in the fight against counterfeiting – invisible QR codes. Security of QR code can be enhanced by coating it with invisible ink which in past were made from various things such as lemon juice, vinegar, diluted blood or even urines. Now a day, these inks are made from fluorescent nano-particles derived from nature, invisible to the naked eye. The printed patch is invisible until it is revealed with the near infra red laser or when exposed to an ultra violet light. Security of QR code can be enhanced by coating it with invisible ink which in past were made from various things such as lemon juice, vinegar, diluted blood or even urines. Modern ink is made from florescent, derived from things in nature which glow when exposed to UV light. These inks contain a photo initiator (an element used in making ink as it absorbs the UV emission of a UV light.) to participate in the reaction and respond to UV radiation. A QR code which is coated with invisible ink in order to enhance the security of embedded data is called Covert QR code. Advantages and uses are as follows: Unobtrusive marking for label or part identification. Tracking of restricted items that are highly subject to counterfeiting such as pharmaceuticals. Covert track and trace of restricted distribution products – wines, cigarettes, promo items. Authentication of Branded Items. Warranty tracking of materials used in construction. Sporting and other Venue ticket authentication. 13 | P a g e
Some of the examples of using invisible inks in images are shown below in table: Table 1-5. Invisible Ink Examples
Images printed by using invisible inks
Images under ultra violet light
Image of cloth coated with invisible blue ink
Resulted Image under UV light
Image of strip coated with invisible red ink.
Resulted Image under UV light
Image of plastic cover coated with Invisible red ink.
14 | P a g e
Resultant image under UV light
1.11 CHALLENGES WITH COVERT COLORED QR CODES: Covert Colored QR Codes have many benefits over monochrome to dimensional QR Codes. Some of them are given as follows: Increases the storage capacity [1]. We can parallely encode information three times greater than monochrome QR codes. The information encoded is directly proportional to Log2N, where N is number of colours used [2]. More secure and robust as compared to monochrome QR Codes as patterns becomes more complicated by using no of colors so, it becomes very difficult for someone to decode the messages by just observing the QR.
In addition to all these benefits it is very challenging task to develop these patterns in real environment due to following reasons The practical implementation of the colored QR code decoding algorithm is a big challenge itself. The accuracy in extracting RGB planes and implementing image processing techniques makes it difficult to implement. With the introduction of covert QR codes the task is more challenging as to read image after allowing UV radiation is bit complicated as various other factors into picture.
15 | P a g e
CHAPTER 2: LITERATURE SURVEY
2.1
UNDERSTANDING QR CODES COMPARED WITH BARCODES:
Rinkalkumar M [6] presented a review paper where recent research works are discussed in detail, along with the explanation of why these two dimensional QR codes are getting wide acceptance in market and industrial areas and gradually replace many applications of one dimensional barcodes. This review paper shows that the QR codes developed in Japan recognized as standards has been approved by AIM standard, JIS standard, ISO standard. This paper concluded that due to storage limitation, one dimensional barcodes is used for storing item identification data. On the other hand two dimensional barcodes due to their better readability, security, high speed decoding capability, is used in various applications like sales, marketing, production and logistics. There are different types of two dimensional barcodes available but due to its basic structure and chacteristics, QR Code is best among another available two dimensional barcodes. Andrew Walsh [8] conducted a research trying various QR codes containing the contact information. The author compared the QR code with the usage of RFID cards which are useful in colleges, library, and recognition of student, staff and non-staff identification. Kato and Tan [9], evaluated six 2D barcodes using some criteria for standardization potential, which are as follows: symbol reading (Omni directional), low-resolution cameras support, robustness in reading under different lighting conditions, barcode reading distance, capability to handle error correction, security issues, support for multiple character sets, and data capacity along with the consideration of the fidelity of the camera phone's captured image as a metric for gauging reading reliability. They used an additional metric – a first –read rate to verify their results and for better gauge reading reliability. Lotlikar et al [10], presented a paper on comparative study of barcode, QR Code and RFID System.In this paper, they compared all the three technologies on various grounds like durability, cost, information capacity, read range etc. to determine best 16 | P a g e
out of it.They have shown a comparable analysis of RFID QR code and barcoding technologies in practice and finally concluded that RFID can deliver more rapid scanning times than barcode QR scanning. I T Asare and D Asare [11] presented a paper on effective use of Quick Response (QR) Code as a marketing tool in which they have uncovered six vital factors referred to as the “Five W‟s and H” that needs to be well considered prior to undertaking a marketing campaign using QR codes. They emphasize on understanding how QR codes can be effectively used in engaging with consumers and concluded that for companies to effectively utilize QR codes as a marketing tool, firstly they need to set clear objectives, decide if this is the suitable approach to reach their target market, whether the content available after a scan is valuable to the consumer, is the placement and position of the QR code easy to scan, is it the right time to implement such a campaign and finally how will the execution be done. Pandya and Galiyawala [5] conducted a survey on QR Codes in context of research and application and highlighted some of possible research areas while considering QR codes. In this theytake account of QR codes basics, its real time application in day to day life and research areas associated and discussed applications, advantages, scope and structure of QR code. 2.2
ENCODING AND DECODING TECHNIQUES:
Graham [12] developed an algorithm which find areas of QR code with potential markers and there after look for its specific features by applying image processing during decoding process. The algorithm has been designed such that markers of approximately any size and orientation can be detected. First the original encoded image was threshold, and then a rough search for finder patterns was applied. Edge detection was applied, if the edges were detected the image was further processed for determining size and orientation and if edges were not detected, they were rejected. Thereafter size and orientation is determined in two steps:Marker Size Determination and Precise Orientation determination. In marker size determination, size is determined using template was then correlated with the appropriate sub-region of the dilated and filled binary image.The precise size of the marker is determined by 17 | P a g e
knowing the size of the structuring element used to perform the dilation and the size of the template just before the output stopped changing.After this precise orientation is determined and if a single pair of guide bars is found, the precise orientation of the code marker is known. If guide bars are detected for more than one orientation or if no guide bars are detected, then the image is rejected as not being a code marker. Lastly all of the encoded code markers were read by determining the coordinates of the upper left fixed corner element relative to the global coordinates of the original image.This algorithm was proven correct for some of the images and incorrect for the images which left number of artifacts after the black and white thresholding steps tend to create problems when looking for the dominant orientation. An attempt to overcome the problem was done by lowering the threshold value but this lead to more false detection. Another difficult problem which evolved was image captured from the extreme end, which generates two problems. One is that the marker can appear rectangular, rather than square. Second is that the marker is more of the diamond shape, rather than square while attempts to reduce these were not effective but they increased the computational time.
Huang et al.[13] Proposed a new algorithm in reversible data hiding, associated with QR codes where QR codes can be hidden into original image and considerable enhancement in embedding capacity was expected. Next they proposed a scheme in which the image of QR code was browsed, hyperlink corresponding to QR code was accessed firstand then the QR code vanished and the original image was recovered to retain the information stored in it. Reversible data hiding is technique of watermarking, unlike conventional watermarking techniques as in it both the hidden data and original multimedia, an image for instance should be perfectly recovered. For data embedding, the goal of the authors was to hide both the location map and the data to be hidden into the original image. First they generated non location map of the original image, followed bychoosing a threshold value and splitting NLM into four sub images and generated its histograms for calculating peak points. If the sum of peak occurrences was larger than the threshold, beginning of NLM was hided into histogram, if not then value of threshold was lowered. The remaining information was then embedded after which image with hidden information and 40 bit side 18 | P a g e
information was achieved as output. The authors concluded with the description of the popularity of QR codes, the usage of QR code in accessing the web pages, reversible data hiding which has the characteristics of abundant capacity for hidden information. They illustrated their proposed algorithm with the example of QR code information which can be deciphered to URL relating to the original image, and more information corresponding to the original image can be discovered by the users, such as online shopping. Pandya et al. [5] discussed applications, advantages, scope and structure of QR code on the basis of research and survey. The architecture of QR code giving information about timing patterns, finder patterns, quiet zone etc. has been explained with keen interest in this paper. Basic characteristics like encoding capacity, versions, error correction levels, applications etc. are discussed in details. Their study reveal that for improving data capacity concept of monochrome barcodes can be extended to color by which data capacity can be increased by three times. Scratch removal techniques, which consist of morphological operations dilation allows scratch to become more obvious and finally applying median filter to increase decoding efficiency. Comparison between QR code embedded technique for invisible marking by using Discrete Cosine Transform and Discreet Fourier Transform was studied in this paper. According to developed reversible data hiding methods, four main technologies used are: compression based technology, difference expansion based technology, and histogram based technology and visible watermarking. They concluded that more and more experiments have to done with QR
codes in different aspects like improving the security, better recognition, reducing redundancy in order to save space, possibility of encoding different kind of data like audio, etc. and since, QR code provides the structural flexibility, it opens up the huge platform for researchers to explore the possibilities to enhance the performance of QR code or to merge QR code with different technologies such as : some kind of encryption technique can be used to encode data first, and then encode it to QR code for better security solutions, finding out the possibility of the use of coding techniques other than RS coding.
Lin et al. [1] explained the two steps of image recognition i.e. image pre-processing and QR code extraction. In image pre-processing focus was made on image de noising, camera shaking which were sorted out by using different binarization 19 | P a g e
techniques which uses modified local threshold method to improve the nonuniformity and uneven light interferences or by generating high resolution image. QR code extraction algorithm uses the position detection patterns of QR code efficiently and accurately to locate the QR code position. It consists of locating the finder patterns of code by using edge detection followed by morphological dilation and closing for more compact regions. Sometimes QR code images may be distorted due to camera perspective projection and due to some other factors so, after locating the QR code position, restoration of a distorted image is done through perspective transformation and resample. So the whole decoding procedure consists of various steps as image binarization, QR code extraction, perspective transformation and resampling, and error correction. By these steps, different types of QR code images are recognized. Reed Solomon Codes helps to correct the error information of distorted images or inappropriate binarization. Finally, the corrected data by Reed-Solomon error correction algorithm is achieved. In general the decoding algorithm consists of computing the syndromes, determining an error locator polynomial, and then solving the error values. Binarization of an image is done in order to recognize the white and black modules accurately by local threshold algorithm i.e. decomposing each image into a series of block and then calculating the threshold value for each block by using: T1 (x, y) = mean (x, y) × 1 +
Standard Deviation (x, y) R
Where R is 1250 by default. Threshold value was calculated for each and then by collecting all points in the image in the ratio of 1:1:3:1:1 both horizontally and vertically finder patterns are recognized. Depending on module size and distance between two finder patterns, version was estimated. Module size can be calculated by:-
Module size =
Width of Finder Pattern 7
And version can be calculated as:-
20 | P a g e
Version =
( Euclidean _distance Module _Size ∗𝑐𝑜𝑠𝜃 −10) 4
Where 𝑓1 is upper left finder pattern and 𝑓2 is upper right finder pattern and 𝜃 is the angle between width of finder pattern and estimated width. Finally alignment patterns and centroids of alignment patterns were calculated and QR code image was reassembled together by using perspective transformation to restore the distorted QR code images. The general representation of Perspective Transformation can be given by:
′
′
′
𝑥𝑦𝑤 =
𝑢𝑣1
𝑎11 𝑎12 𝑎13 𝑎21 𝑎22 𝑎23 31 𝑎32 𝑎33
Where 𝑥 = 𝑥′ ∕ 𝑤′ and 𝑦 = 𝑦′ ∕ 𝑤′.
Wakahara and Yamamoto [14] presented the dotted picture or image multiplexing method over the QR coding and the evaluation of the redundancy characteristics of the QR coding. Then the image of QR code is presented by the animation method. Authors developed a prototype of the QR editor for making the QR code with the doted images and then the characteristics are confirmed by the cellular phone barcode reader. With the increase in version of QR code, the amount of data that can be encoded into it increases with increase in the redundancy factor. In creating QR code redundant code word is filled in vacant area. The prototype application of QR code and image multiplexing QR editor is configured by using java and for recognition purposes, a regular cellular phone is used by the author and his co-authors carried out in normal room lightning conditions. For making the animated QR image, the moving area of the multiplexed image is limited to the center and redundancy is obtained avoiding the function pattern area. The reading characteristics of QR code with multiplexed dotted picture or image is measured and the validity of the animated QR code made by QR Editor is also confirmed. Results shows that QR-JAM is limited to about 40 %, the proposed system can multiplex larger ratio of the dotted images by utilizing the large version. 21 | P a g e
Falas, Kashani [15] designed a two dimensional barcode reading using the camera phones to overcome the Data entry for mobile phones by the phone‟s numeric keypad. Barcode reading applications use the phone camera to capture the image of the QR code and then using the phone processor for decoding. The software is implemented in Java, for platform independence on phones supporting J2ME. It is debated that Java is a feasible alternative if software is carefully developed. A review of attempts in the use of mobile phones as bar-code readers, and application of this technology, with very promising results, is shown in this paper by authors. They demonstrated practically that the approach was promising in wide range of applications from product tracking to linking the physical world with the virtual digital world. The problem encountered while experimenting was not only the camera‟s resolution (even low-end resolution of 640x480 pixels is adequate) but also the poor focusing of the camera on the close-up capture. Until auto-focus devices become common-place, the interim solution is the use of a macro lens accessory attached on the phone, to be able to decode bar-codes from close-up. Another problem they faced was whether the computational power of a mobile phone is enough to replace a hardware based laser scanner and allow the user to experience comparable speed and ease of use. Several factors such as transmission costs and delays encountered when assigning the image decoding part to a back-end server, makes it necessary to perform the decoding part on the phone. The larger versions of codes with less number of data can be easily decoded has been concluded by the authors during their research. Pre- processing steps include: Gray scaling of captured colored image by using standard brightness equations, Local grey scale histogram stretching, Local optimum thresholding by evaluating the optimum threshold for each region, noise filtering by standard closing and opening techniques, cropping to reduce size and processing time, rotation correction before decoding to make it faster and tilt correction to bring the image to its expected square shape before decoding. The results indicate that the techniques used in pre processing of an image has a twofold effect i.e. decoding ability is improved at the expense of increased decoding time. After pre processing the software locates the corners and edge patterns first and then decodes the black and white modules containing data. Decoding strategy included the 22 | P a g e
idea of searching the straight lines in space in order to transform the part of the image into magnitude and angle coordinates (Hough transform). Chen and Lei [7] developed a new technique to encode and study its realization in windows systems. Optimal masking model and version can be choose in their proposed work frame to save coding space to the maximum extent. In this entire coding system consists of seven C + + class and encoding algorithm encodes not only characters, numbers, alphanumeric data, Kanji and Kana characters but also images can be encoded into the QR code. Based on encoding rule data is encoded into the binary numbers of 0 and 1. After that reed Solomon codes are incorporated in data area for data correction capability, data area is rearranged and mask processing is done to accurately binarize the data. In mask processing they have implemented EXOR between the data area cell and the mask pattern (template) cell when encoding the stored data and arranging it into the data area. Authors, test results showed that the encoding system can encode data fast and accurately with low CPU utility. The result analysis showed that the CPU utility is about 6% within 15% and with small fluctuations. The error correction level results showed that the time consumed in L level is less as consumed by H level and for the same version it can accommodate more data as compared to H level. They designed a user interface comprising of menu and tool bar from where a user can import data, example- a text file and the data can be encoded simultaneously. Saving the QR code image, creating a new project, printing the image, displaying the help information etc. are some of the tools present in the authors developed software. Aidong Sun et al. [16] gives an algorithm for the analysis and correction of the distorted QR code image based on the code area finding by four corners detection for 2D barcode. For the detection of the finder patterns in image of QR barcode, author used the edge detection with contours finding algorithm to eradicate the noise and reduce the computational time and utilize the two tangents to approximate the right bottom points. They give a detail description on how to use inverse perspective transformation in rebuilding a QR-code image from a distorted one. The algorithm composed of five tasks, gray-scale image transformation, canny edge detection, external contours 23 | P a g e
finding, inverse perspective transformation and cell grids generating, which are arranged in sequence. For detecting four corner points in QR-code images they performed external contours finding and from the Omni-direction code area in a binary image they have detected three small squares. Second algorithm proposed by the author using external contours distinguishing, canny edge detection and double inverse perspective transformation (DIPT) to locate Omni directional QR codes. Since the input image has a deformed shape because of being captured from the embedded camera device so, to normalize the code shape, inverse perspective transformation is used. The equation used to calculate the coefficients of perceptive transformation which maps the vertices (Xi, Yi) to vertices (Ui, Vi) is as follows where i changes from 1 to 4: 𝑈𝑖 =
C00 × Xi + C01 × Yi + C02 C20 × Xi + C21 × Yi + C22 And
𝑉𝑖 =
C10 × Xi + C11 × Yi + C12 C20 × Xi + C21 × Yi + C22
The new image is mapped from the origin image and then the RGB value of a pixel is extracted by using the following formula: f (x, y) = g (u, v) = g
C00 × X + C01 × Y + C02 , C20 × X + C21 × Y + C22
C10 × X + C11 × Y + C12 C20 × X + C21 × Y + C22
This algorithm assumed that the input image of QR code is not curved for all edges. Although the distorted image is corrected but the rebuild image is not correctly identified, therefore it is further transferred into a cell matrix with 1 and 0.The average reorganization success rate of their algorithm is calculated as 88.19%. Gaikwad and Singh [17] presented a review paper on
the concept of Image
embedded QR codes, an automatic method to hide information using QR codes and to embed QR codes into color images with bounded probability of detection error. Discussed embedding methods are compatible with standard decoding applications. It can be applied to any color or gray scale image with full area coverage and consists of 24 | P a g e
two components. First is to reduce the coarse square structure of the QR code by using half toning techniques for the selection of modified pixels to breakand second is by using the luminance level to which the pixels are to be transformed in such a way that it should not visible to naked eye on the color image. They discussed a method for hiding information in QR Code in which first of all, they divided the original message into smaller parts and encoded each message into QR Code pattern. After that, they encoded each part of QR code into special symbols and generated them. Thereafter those generated patterns are read by optical device such as a scanner or a camera phone, the picture image is analyzed and then decoded to retrieve the hidden data. The QR decoding process contains three basic stages: binarization, grid sampling and decoding of QR code bit stream. Wiputtikul and Vongpradhip [18] introduced method for error detection of the information in QR Code by using Micro QR Code, creating a grayscale QR Code.They used image processing technique before decoding which offers ability of fixing error and restore module that have been damage such as tear, stain or bend before decoding process. They divided the whole experimental procedure into two separate processes. QR Code barcode encoding process with additional error detection method and decoding process by splitting error detection of QR Code to verify the correction of the transcription. Encoding process is again divided into two parts: Encoding of information process to achieve Matrix of QR Code and Micro QR Code for the overlap. They modified source code of “Zxing” based on ISO / IEC 18004 standard to encode and decode Micro QR Code. For implementing error detection on QR Code they overlay bar code with the area of the Finder Pattern on all three corners of the Finder Pattern of QR Code. Decoding process is also break down to two parts which is QR Code image and error detection, encrypted with Micro QR Code. While decoding, they used image processing to extract the error by determining ROI (Region of Interest) on the overlapped area of Micro QR Code located at the three corners of the Finder Pattern. Binarization is used to extract QR Code from processed image by using multilevelthresholding and Opening technique is applied to eliminate edges and the noise from 25 | P a g e
the barcode image since after binarization process, the obtained micro QR Code is not complete due to the edges of the QR Code's modules residual. Aryachandran S. and Jyothi R L [19] introduced a QR Code System which is password protected enhancing the security of colored QR Code and also discussed why one should go for password protection rather than classical encryptions. They divided whole encoding into two parts: encryption and encoding. While encryption they encrypt data by using XOR encryption before encoding them in a QR Code and divided the data into three parts for unity in encoding. After that they placed first character on first QR cod, 2nd on 2nd QR code and so on, thus diffusing the data even before encrypting. This data is XOred with the encryption stream which is generated by random generator within the range of value of password. While encoded they generated three monochromes QR code and replaced black color of each QR Code each with cyan, magenta and yellow respectively. Decoding procedure is divided into three parts as:
CMY estimation, binarization and decoding and
decryption. In CMY estimation image is processed and passed through the method of color estimation. This part includes pre-processing, localization and geometry correction, Image filtering, color image processing and thresholding. After correcting the color of QR Code they performed binarization which results in black and white QR code only in which decoding can be done. In third phase QR codes are then decoded as per the ISO/IEC standardsresulting in encrypted data which is further decrypted by using XOR decryptionfor the information retrieval. They have donecoding inmat lab environment with the help of Google Zxing package for QR code operations and their results shows that algorithm successfully decoded with 95% results. Homkajorn et al
[20] introduced an advance technique on removing scratch or
damage that exists on QR-code. In this technique first of all, they converted captured RGB image into HSV model and then optimize the Threshold level of Saturation Layer in order to get clear boundaries as possible.They performed dilation process in order to achieve clear scratch and compared result with the damaged QR code to remove scratch. After that, they converted the resultant image into binary and to maximize the quality of QR code and for the removal of random noiseMedian filter is 26 | P a g e
used so that it is able to be read and decode by decoder. Their experimental result gives the average BER (Bit Errors Rate) value of 2.54.This technique is beneficial especially when QR-code decoding algorithm is unable to decode if the scratch that applies on the QR-code is more than Error Correcting Level threshold of current QRcode or the damage applies on some curtain area, which consider as information area of the QRcode. Yue Liu and Mingjun Liu [21] proposed an automatic recognition algorithm of Quick Response Code and also described an image processing system based on embedded system which is ablefor the binarization, location, segment, and decoding the QR Code.They developed a high-accuracy, a high-speed binarization methodwhich can locate the finder pattern accurately and integrate the local thresholding method with global thresholding and adapt to various sizes, various gray-level values, and under various lighting conditions of real bar code image.During their encoding procedure,firstly they encoded input data in according to most efficient mode and formed bit stream and divided into code these stream into words which is again divided into blocks. After this, error correction code wordsare added to each block and all these code wordsare putted into a matrix and masked with mask pattern. Finally function patterns are added into the QR symbol. For automatic recognition of QR Code they converted captured code into grayscale image.They introduced an adaptive multi-level thresholding algorithm, to binarize the gray scaled image, which integrate the local threshold with global threshold. After binarizing they determined location and orientation of QR Code with the help of finder pattern and located the central coordinates of alignment pattern for recognition of code. After recognizing the QR Code they decoded the QR Code by using procedure which is just reverse of their encoding procedure and then implemented this application in their architecture. They coded their barcode image processing algorithm by the full assembler code to execute in real-time. Their experimental result shows that there are three kind of execution time corresponding to processing step in which 81.4% image can be binarized with once global thresholding, 15.7% image need secondary thresholding method, and 0.028% image need locally thresholding method.
27 | P a g e
AnuThambi [22] introduced a method for embedding color image on QR Code using genetic algorithm which reduces the processing time and enhances the visual tolerance of the QR code. They used Genetic algorithm to generate the halftone masks which is then applied to the color image and QR Image to make it to bi-level image and then intermediate images are fused to get the result.They applied halftone masking by using the following rule:𝐼𝑝𝑞 < 𝑀𝑝𝑞 ⟶ 𝐵𝑝𝑞 = 0 Ipq > 𝑀𝑝𝑞 ⟶ Bpq = 1 Where, M is a half-toning mast and is an array of threshold values used to convert a continuous tone image, I, to bi level B. The values of I and M are numbers in the same range, typically 0- 255. The values in B are 0 or . After this, Binarization using threshold is applied to QR code to select the pixels for fusion. After binarization they embedded QR Code in to the halftone masked color image as QR image as the background image and half toned color image as foreground image which increases the decoding robustness. Their results shows that genetic algorithms are effective in finding improved halftone masks and also in reducing the search space to particular subsets of possible halftone masks which enhances the overall search performance. 2.3
VARIOUS APPROACHES FOR INCREASING CAPACITY
Henryk Blasinski, et al. [4] proposed a cooler barcode frame work for mobile phone applications. Their proposed providing a low bit error rate and high decoding rate for each color channel when used with the error correction technique. M Ramya [23] designed a QR code for mobile applications by exploiting the spectral diversity Cyan, Magenta and Yellow colorant channels commonly used for color printing and complimentary red, blue and green color channels. His experimental results showed that the proposed work overcomes both single and burst errors providing a low bit error rate and high decoding rate for each colorant channel. The author proposes techniques for converting two dimensional barcodes to color barcodes. FPGA based high speed, high accuracy technology for generation of color 28 | P a g e
QR codes for various texts has been proposed by the author. For generation RS coder is used in burst error detection, input data is arranged in block format and parity blocks are added to each. Three codes with different text messages encoded are then masked in MATLAB to ensure more security. Colors are assigned to every bit and pixel value and then added together to get colored QR code. Prathibha N. Pillai, K. Naresh [24] proposed a new color QR code algorithm which uses multiplexing and color coding techniques. This algorithm created a three times more capacity than the other black and white QR codes. Up to some extent, damaged QR code is decodable using RS codes which are generated using finite fields. The algorithm includes the following steps: Data to be encoded by user and convert it into ASCII equivalent, using the equivalent finite field numbers are generated by using primitive polynomial. The codeword is generated and converted into its binary equivalent and the bits are placed according to the QR pattern. Each QR code is colored and then combined to form a colored QR code. The decoding steps consists of splitting the color QR code image in to RGB planes, estimating the QR patterns, converting binary into decimal for each byte, applying to RS decoder as input, converting them in to characters to give back the original information. Jeevan M Meruga et al. [25] researched that huge amount spend to fight against counterfeiting by government and private companies can be reduced by using lanthanide doped β-NaYF (4) Nano-particles for security printing applications. Inks which were used comprised of Nano-particles with oleic acid as the capping agent in toluene and methyl benzoate with poly(methyl methacrylate) (PMMA) as the binding agent were used for printing purposes of quick response (QR) codes. The QR codes were made using an AutoCAD file and printed. The printed QR codes were invisible under ambient lighting conditions, but were readable using a near-IR laser, and those QR code images successfully scanned using a smart phone. It was also researched that optimization for line width, thickness and stability on different substrates can be done for the security ink and demonstrated that QR codes, which have been used primarily for information sharing applications, can also be used for security purposes. Homayoun Bagherinia, Roberto Manduchi [2] proposed a new approach for color barcode decoding. The paper shows that using more number of colors encounters 29 | P a g e
problem that the surface color depends not only the surface reflectance spectrum but also unknown illuminant spectrum gets added which act as an undesirable parameter. Other factors which needs to be avoided and create interference while decoding are multiple light conditions (indoor\outdoor), color drift during printing, color fading, quantization and noise, camera non linearity or poorly calibrated camera response . They selected the reference color patches uniformly sampled in RGB color space printed normally on paper. Images were taken under different lightening conditions. The number of times the decoder fails to decode the code was calculated and observed that it is possible to reach high information rate with very low error rate. Victor [26] proposed a technique for data compression which in turn helped to increase the data capacity of QR Codes in which he first compressed the data to be encoded by using any of the data compression technique. The compression technique used by author is in two steps: converting the text data into binary form and then generating the hash map data from this binary data. For converting text data into binary form he converted every letter into an 8 bit sequence of 0‟s and 1‟s. After compressing data by using this method he encoded the data in QR Code. For decoding he used kaywa reader and J2ME and performed pre-processing steps and after that finder patterns are detected and original data is decoded. His experimental results shows that by using efficient compression technique 4 Mb of data can be stored inside of a QR Code whereas in normal QR Codes can compress only up to 4 kb of data. Ramya and Joice [27] presented an idea to generate color QR code, in which information are embedded in both horizontal and in vertical directions, while embedding the information in QR image , it causes a blurring effect which reduces the aestheticism and also makes it difficult to decode data at the decoding end. To deal with this, QR image is embedded in High Frequency cover image in various novelty technique, The HF of cover image can be obtained using Discrete Wavelet Transform by applying the combinations of low pass, high pass filters. They also implemented a in which they embedded a series of images instead single image which in result improves the storage area and resolution of QR image and also assures security policy. While generating QR Code they analyzed input data for compression 30 | P a g e
and decided a suitable version for QR Code and then encoded data in the selected mode adding terminator bits at the end of the codeword. After they performed error correction to remove noise during transmission and masking to separate out function patterns from encoded pattern of information Thomas Dean, Charles Dunn [28] proposed a new 3D barcode standard, Quick Layered Response (QLR) codes codes in which codes are superimposed in the RGB color space, to increase the capacity by a factor of 3., and implemented encoding and decoding of such codes on Android devices. They encoded and decoded QLR codes in linearly independent RGB basis colors and also investigated the inclusion of modern error coding schemes in QLR codes. They used zxing image processing library implemented in Java and produced a useful zxtestbench java project that decodes QLR codes from input images.
31 | P a g e
CHAPTER 3: MOTIVATION AND OBJECTIVES
3.1
MOTIVATION
A covert colored quick response (QR) code has been created by the researchers extending the concept of monochrome QR Codes, in an attempt to increase the storage capacity and to enhance the security and robustness as compared to traditional two dimensional monochrome barcodes. In colored QR Codes capacity to store data is three times more than that of traditional 2d monochrome QR. They have taken control over the market due to factors like, small size, readability from any direction, dirt and damage resistant, high capacity, kanji and kana encoding capability. Because these invisible codes are complex, they are hard to replicate. In order to increase the storage capacity of information being encoded in QR code, different colors can be used [2]. The information encoded is directly proportional to Log2N, where N is number of colors used . Therefore, using more number of colors means encoding more information. Since the perceived surface color also depends on illuminant spectrum with surface reflectance spectrum so using more numbers of colors will raise problems [2]. Furthermore while decoding, determining the color index of every surface plot is a challenging task . Other parameters include: color drifts, color fading, square patterns not being identified correctly, unknown or poorly calibrated camera color response, camera blur, color mixing from two nearby patches due to blur, quantization and noise [2]. Due to enhanced security and robustness, these codes will therefore be great when printed on banknotes, bills and other goods that can be counterfeited easily. 3.2
OBJECTIVES
This project aims at the region identification and decoding of covert QR Codes using image processing tools. Whether your purpose is to track possible diverted distribution of products, subtly identify a label or an item in transit to an end
32 | P a g e
destination (either within your facility or outside), or provide authentication of an item or document, this proposed system will fulfill all the requirements. Following points define the work accomplished in this research: 1. Design and development of software to read and display images and plot their HSV and RGB diagrams. 2. Study, analysis and recognition of covert colored QR codes by using image processing techniques. 3. Perform encoding and decoding of covert QR Codes by using suitable algorithms.
33 | P a g e
CHAPTER 4: METHODOLOGY
This chapter deals with the methodology adopted in designing of required interface using opencv python in spyder and region identification, encoding and decoding of colored quick response codes using python, opencv, pyqrcode and qrtool. 4.1
USER INTERFACE "PIMAD"
By using python and open cv a user interface "PIMAD” has been developed in which we can read and display various images plot its hsv and rgb histogram, extract its ROI. 4.2
ALGORITHM FOR QR REGION IDENTIFICATION
By using the following proposed algorithm we can identify the position of finder patterns in given QR Code. These patterns are situated at the three corners of QR code i.e. at top-right, top-left and lower-left corner and determines the position, size, and the angle of inclination of QR code. Steps in this algorithm are: 1) Splitting of the given QR code image to R, G and B planes. 2) For each of these planes, edge detection using canny edge detector and thresholding for different values such as 50,100....etc. has been performed. 3) Then in all of the binary images contour values are determined using OpenCV function cv2.contours (). 4) To remove the unwanted noise in the image, filtering operation has been performed according to the area. 5) After filtering approximate the contour values. 6) Since finder patterns are squares so we have to consider only those contour values for which we are getting four corners. Filter other contour values from the approximated values and consider only those which gives four corner values.
34 | P a g e
Figure 4-1. Flow Diagram for Region Identification
7) Now in this result set there may be some shapes with four corners but not squares, so we have to check other properties- all inner angles must be 90 and all sides must be equal. By using this we again refine our result set by filtering other contour values which are not satisfying these properties. To check the inner angles we have used condition if cos (theta) < 0.1, ie theta > 84 degree, then all angles are at 90 and distance between any two points is calculated by using following formula: 𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 =
𝑥2 − 𝑥1 ² + 𝑦2 − 𝑦1 ²
8) From above formula we can check whether all sides are equal or not. 9) Draw the bounding box around the detected squares. 35 | P a g e
4.3
PROPOSED ENCODING STEPS 1) By using pyqrcode and python first of all we generate three QR codes which contain three different texts (alphanumeric, unicode, numeric) individually encoded in them and save these images in lossless .png format.
2) For region identification, finder patterns of each are has to be determined. To accomplish this above stated algorithm for region identification is applied in spyder python. After this we are having our finder patterns detected and shown by three bounding boxes.
3) As for creating a colored Quick Response Code, each generated code has to be colored with one of the RGB color therefore, a single QR Code in addition to its background is colored with one of the RGB colors. Such kind of three independent encoded colored Covert QR codes are developed, each possessing different color. Position of the inner square is determined by using the dimensions of outer square as we know ratio of module width of each finder pattern is 1:1:3:1:1.
4) Then these three codes are merged in resulting in formation of RGB layered colored QR code, which has triple times the capacity as compared to single QR code which comprise of white and black modules. Further, the generated RGB colored covert QR code is printed and captured using camera but decoding of such code is challenging task since, printers are cmyk based, so after printing QR on paper its colour changes. To deal with this problem, encoding of QR is being done in CMYK colour space.
36 | P a g e
Figure 4-2. Encoding Procedure
4.4
CMYK COLORSPACE
A CMYK color space is all possible colors that can be made from four colourants for cyan, magenta, yellow, and key black [29]. While dealing with web, thumb rule states that RGB is preferred and CMYK (cyan, Magenta, Yellow, Black) is preferred while dealing with printing materials because if printed in RGB, it would result in some other color. On the other hand, computer monitors give off colored light-RGB (CMYK is colored ink) [30]. Printing deals with absorption and reflection of wavelengths of which we perceive as CMYK color. Moreover, computer monitors have a wide range of colors as compared to printers. Therefore, these colored codes needs to be converted in order to get printed and processed further. RGB are also called additive colors as there are no colors and the colors are being added together to get other colors. Eyes do not receive refelected light and perceive the color to be black [31]. When we add portions of RGB, we achieve CMYK colors as shown below.
R+G+B=White, R+B=Magenta R+G=Yellow, G+B=Cyan 37 | P a g e
Figure 4-3. Additive Colors
Furthermore, when we subtract CMYK, we get RGB colors. Therefore CMYK are called subtractive colors. Subtractive colors, resulting in RGB colors are shown below:
Figure 4-4. Subtractive Colors
C + Y = Green, C + M = Blue M + Y =Red , C+M+Y = Black
38 | P a g e
This results to the conclusion that when it comes to decision of using RGB colors, output should be a computer screen. If the piece of work, needs to be printed then CMYK should be used. RGB to CMYK conversion formula is shown in table below: The R,G,B values are divided by 255 to change the range from 0.255 to 0.1, resulting in new values in CMYK as follows:
R N = 1 − R/255 GN = 1 − G / 255 BN = 1 − B / 255 Form the above obtained values, CMYK is calculated accordingly as shown in table below. Table 6. Conversion from RGB to CMYK
COLOR
FORMULA FOR CALCULATION
1.
Cyan
𝐶 = R N − 𝐾 / (1 − 𝐾)
2.
Magenta
𝑀 = GN − 𝐾 / (1 − 𝐾)
3.
Yellow
𝑌 = BN − 𝐾 / (1 − 𝐾)
4.
Black(Key)
S. NO.
39 | P a g e
K = 1-min (𝑅𝑁 , 𝐺𝑁 , 𝐵𝑁 )
Another model showing CMYK Color Space is given as below in figure:
Figure 4-5 RGB-CMYK Color Space Model [29].
For CMYK to RGB conversion, the range of R, G, B values is in range 0.255. The conversion formula for the same is shown below: Table 7. Conversion from CMYK to RGB
S. COLOR NO.
FORMULA FOR CALCULATION
1.
Red
𝑅 = 255 × 1 − 𝐶 × 1 − 𝐾
2.
Green
𝐺 = 255 × 1 − 𝑀 × 1 − 𝐾
3.
Blue
𝐵 = 255 × 1 − 𝑌 × 1 − 𝐾
40 | P a g e
In CMYK encoding steps are given below1) By using Pyqrcode and python we generate three encoded QR images and save these images in lossless .png format. 2) Since generated QR is a three channel image, we take one channel from each of the generated image and assume them as R, G , B channel respectively. 3) By using above given series of formulas we convert RGB channels into corresponding CMYK channel. 4) These channels merged to generate colored CMYK QR which is printed and captured using camera. 4.5
PROPOSED DECODING STEPS
The data retrieval process of encoded data consists of binarizing the grayscale image to reduce the impact of print and capture noise, followed by data decoding. While capturing the image, light effects gets added up in image which needs to be corrected by using image processing techniques like thresholding in order to decode the captured image again. Thresholding is the simplest method of image segmentation. From a grayscale image, thresholding is used to create binary images. A certain threshold value is chosen and pixels are replaced if in the image containing the black pixels, if the image intensity is less than some fixed constant (say T) or if the white pixels has image intensity greater than threshold value. Here we are utilizing Local Thresholding algorithm commonly used for monochrome barcodes. Local Thresholding: The local thresholding algorithm is extensively used in monochrome barcodes to account for illumination variations across the captured image. Steps for decoding procedure can be given as follows: Step 1: The decoding process is just the opposite of encoding. The finder patterns of the layered QR code are again recognized and un layering of each code is done by removing the layers of red, green and
41 | P a g e
Step 2: Decoding of every de-layered Covert QR code is accomplished. Further, the generated RGB colored covert QR code is printed with the size of 3×3cm (widely used standard size of printing QR code) and captured using camera. While capturing the image, light effects gets added up in image which needs to be corrected by using image processing techniques like thresholding in order to decode the captured image again.
Figure 4-6. Decoding Procedure
4.6
HSL AND HSV COLORSPACE
A color space helps to understand the color capabilities of a particular device [32] or file and represents what a camera can see, a monitor can display or a printer can print etc. It is a mathematical representation of a set of colors. There are variety of color spaces known such as RGB, CMY, HSV, HIS etc. All color spaces can be obtained from RGB information provided by the devices such as camera, scanners etc. HSL (hue, saturation, lightness) and HSV (hue, saturation, value) are the two most important representation of the RGB color model shown in figure below. 42 | P a g e
Figure 4-7 HSV Model [29].
Another way of creating colors on monitor is based on hue, saturation and brightness values.
Figure 4-8 Amplitude v/s Wavelength [32].
Saturation is an expression of relative relative bandwidth of the visible output from a light source. In the figure, the saturation is represented by the steepness of the slopes 43 | P a g e
of the curves. Red curve represents a color having low saturation, the green curve represents a color having greater saturation, and the blue curve represents a color with fairly high saturation. As saturation increases, colors appear more clear and as saturation decreases, colors appears washed-out. Brightness is a relative expression of the intensity of the energy output of a visible light source. It can be expressed as a total energy value (different for each of the curves in the diagram), or as the amplitude at the wavelength where the intensity is greatest (identical for all three curves). In the RGB color model, the amplitudes of red, green, and blue for a particular color can each range from 0 to 100 percent of full brilliance. These levels are represented by the range of decimal numbers from 0 to 255, or hexadecimal numbers from 00 to FF. These cylindrical geometries are used today in image analysis, computer vision and image editing. Starting at red primary color at 0 degrees, green at 120 degrees and blue at 240 degrees and then back to red at 360 degrees as shown in figure 5.8. The central vertical axis comprises the range from black (value 0) to white (value 1). The R,G,B values are divided by 255 to change the range from 0.255 to 0.1. The conversion of RGB to HSV is done as:
Figure 4-9 Conversion from RGB to HSV [29].
44 | P a g e
The HSL color space, also called HLS or HIS or HSB. HSI stands for Hue, Saturation, Lightness (also Luminance or Luminosity) / Intensity/ Brightness. Conversion from RGB to HSL is done as:
Figure 4-10. Conversion From RGB to HSL [29]
45 | P a g e
4.7
ENCODING AND DECODING IN HSV
We encode five parallel QR Code messages in HSV Color Space and decode them back to generate that encoded messages. 1) While encoding in HSV first of all we generate five encoded monochrome QR Codes by using Pyqrcode. 2) Since generated images are three channel image we took one channel from each image and normalize their values between 0-1. 3) Integrate all normalized images in 2i points and put into one image of same dimension as generated one. 4) Divide Hue values into 2i points as H keeping Saturation and Value constant. Where, i is total number of images generated, and H contains all those points. 5) Assign these HSV values to the integrated image so that resulting image is in HSV Color Space.
Figure 4-11. Flow Diagram for Encoding in HSV
46 | P a g e
At the time of decoding, 1) if image is captured from camera then it is again converted back into HSV and split image into hue, saturation and value channels. 2) After that find out indexes where hue value equals to the values stored in H. 3) Convert them into binary and assign them to i no of images to retrieve encoded images back.
Figure 4-12. Flow diagram for Decoding in HSV
4.8
IMAGE SEGMENTATION
Image Segmentation is the process of digital image portioning into sets of pixels to simplify the image for easier analysis [33]. It is typically used to locate objects and boundaries. In image segmentation, it means assigning a label to every pixel such that pixels having same label share certain characteristics. Each pixel in a region is similar with respect to some features like color, intensity or texture. Some of the practical implementations of image segmentation are:-
47 | P a g e
1. Medical imaging: This application includes measuring of tissue volumes, planning of surgery, virtual surgery simulation, diagnosis and study of structures, locating tumors etc.. 2. Object detection: This application includes face detection, pattern detection, locating objects in images etc.. 3. Object recognition: Face recognition, finger print recognition etc.. are application included in recognition part. 4. Other applications: Other applications include robot guidance, process controls in machine vision, deals with the problem of searching the digital images in large data base, closed circuit television etc.. 4.9
TECHNIQUES FOR IMAGE SEGMENTATION
Several techniques are present for image segmentation, which are as follows: 1. Thresholding: This method is based on threshold value which on being calculated turns a gray scale image into binary image [34]. 2. Clustering Methods: The K-means algorithm is used as an iterative technique used to partition an image into k clusters. The factor can be selected manually or randomly. 3. Compression based Method: This method states that the image segmentation results best when the coding length of the data is minimized. The method describes each segment by its texture and boundary shape. 4. Histogram based method: This is one of the most efficient method amongst all others as they need only one pass through the pixels. A histogram is computed from all the pixels in the image and the peaks are used to locate the clusters in the image. The histograms can be done in multiple fashions when multiple frames are to be considered [35]. 5. Edge detection: As there is always a sharp adjustment in intensity at the region boundaries, region boundaries and edges are closely related. The edges identified by 48 | P a g e
this process are often disconnected and the segment method can be applied to the edges detected by the edge detectors [12]. 6. Dual clustering methods: This method is the combination of the three characteristics: Partition based on histogram analysis checked by the high compactness of the object and high gradients of the borders. 7. Region growing methods: This method rely on the assumption that the neighboring pixels within one region have similar values. The common procedure is to compare one pixel with its neighbors. If the criteria is fulfilled, then the pixel can be set to belong to the clusters as one or more of its neighbors. 8. Partial differential equation base methods: We can segment the image by using numerical scheme. Curve propagation is the popular technique with the numerous applications of object extraction, object tracking, stereo reconstruction, etc. The central idea is to evolve an initial curve towards the lowest potential of a cost function, where its definition reflects the task to be addressed. As for most inverse problems, the minimization of the cost functional is non-trivial and imposes certain smoothness constraints on the solution, which in the present case can be expressed as geometrical constraints on the evolving curve. 9. Parametric methods: This is based on parameterizing the contour according to some sampling strategy and then evolves each element according to image and internal terms. Such technique is fast and efficient. 10. Level set methods: It can be used to efficiently address the problem of curve/surface/etc. propagation in an implicit manner. The central idea is to represent the evolving contour using a signed function whose zero corresponds to the actual contour. 4.10 THRESHOLDING Thresholding is the simplest method of image segmentation. From a grayscale image, thresholding is used to create binary images [34]. A certain threshold value is chosen and pixels are replaced if in the image containing the black pixels, if the image 49 | P a g e
intensity is less than some fixed constant (say T) or if the white pixels has image intensity greater than threshold value. To make thresholding completely automated, it is necessary for the computer to automatically select the threshold value. For this some methods are:
1. Clustering-based methods, where the gray-level samples are clustered in two parts as background and foreground (object), or alternately are modeled as a mixture of two Gaussians. 2. Histogram shape-based methods, where, for example, the peaks, valleys and curvatures of the smoothed histogram are analyzed. 3. Entropy-based methods result in algorithms that use the entropy of the foreground and background regions, the cross-entropy between the original and binarized image, etc. 4. Object Attribute-based methods search a measure of similarity between the graylevel and the binarized images, such as fuzzy shape similarity, edge coincidence, etc. 5. Spatial methods use higher-order probability distribution and/or correlation between pixels 6. Local methods adapt the threshold value on each pixel to the local image characteristics. In these methods, a different T is selected for each pixel in the image. Color images can also be thresholded. One approach is to designate a separate threshold for each of the RGB components of the image and then combine them with an AND operation. This reflects the way the camera works and how the data is stored in the computer, but it does not correspond to the way that people recognize color. Therefore, the HSL and HSV color models are more often used; note that since hue is a circular quantity it requires circular thresholding. Histogram shape-based methods in particular, but also many other thresholding algorithms, make certain assumptions about the image intensity probability distribution. The most common thresholding methods work on bimodal distributions, but algorithms have also been developed for uni-modal distributions, multimodal distributions and circular distributions 50 | P a g e
CHAPTER 5: DESIGN OF EXPERIMENTS
5.1
5.2
HARDWARE INTERFACES:
Processor: Intel(R) Core(TM) i7
Speed: 2.53 GHz
RAM: 3.00 GB
Hard Disk: 320 GB
SOFTWARE INTERFACES:
Software: Spyder Python, Open CV, pyqrcode, qrtool Coding language: Python, PyQt 5.3
Operating system: Windows 7
PYTHON( OPEN SOURCE)
Python was developed by Guido van Rossum in the late eighties and early nineties at the National Research Institute for Mathematics and Computer Science in the Netherlands and now maintained by a core development team at the institute [36]. Python is a high-level, interpreted, interactive and object-oriented scripting language. It is designed to be highly readable and uses English keywords which make it more user friendly language since it has fewer syntactical constructions than other languages. Key Features:1) It is easy-to-learn since it has few keywords, simple structure, and a clearly defined syntax which makes it more student friendly. 2) It is easy-to-read since it is more clearly defined and visible to the eyes. 3) Python's source code is fairly easy-to-maintain. 4) Python has broad standard library which is very portable and cross-platform compatible on UNIX, Windows, and Macintosh.
51 | P a g e
5) It support for an interactive mode which allows interactive testing and debugging of snippets of code. 6) Python codes are Portable. It can run on a wide variety of hardware platforms and has the same interface on all platforms. 7) Python codes are extendable as one can add low-level modules to the Python interpreter which enable programmers to add to or customize their tools to be more efficient. 8) Python provides interfaces to all major commercial databases. 9) It has supports for GUI applications that can be created and ported to many system calls, libraries, and windows systems, such as Windows MFC, Macintosh, and the X Window system of Unix. 10) It can be easily integrated with C, C++, COM, ActiveX, CORBA, and Java. 11) Its support for automatic garbage collection. 12) Python provides a better structure and support for large programs than shell scripting. Various functions, variables, Matrix and Array operators, algebraic commands are supported by this technical language and a user helping guide and documentations are present within the software with examples for support and easy understanding of users. Python is available on a wide variety of platforms including Linux and Mac OS X. 5.4
OPENCV( OPEN SOURCE)
OpenCV was started at Intel in 1999 by Gary Bradsky and its first release came in 2000. It supports a wide variety of programming languages like C++, Python, Java etc and is available on different platforms including Windows, Linux, OS X, Android, iOS etc. OpenCV-Python is the open source Python API of OpenCV. It combines the best qualities of OpenCV C++ API and Python language. It is a Python wrapper around original C++ implementation and has support for Numpy which makes the task more easier. Numpy is a highly optimized library for numerical operations. Several other libraries like SciPy, Matplotlib which supports Numpy can be used with this [36]. 52 | P a g e
So OpenCV-Python being an open source is an appropriate tool for fast prototyping of computer vision problems. A user helping guide and documentations are present within the software with examples for support and easy understanding of users 5.5
PYQRCODE( OPEN SOURCE)
The pyqrcode module is a QR code generator that is simple to use and written in pure python. It is compatible with Python 2.6, 2.7, and 3.x. Its all of the automation can be controlled manually and codes can be generated in SVG, EPS, PNG (by using the pypng module), and plain text. By using this we can set all of the properties of QR Code manually including version number, its error correction level and even we can create QR Code with different background colors. 5.6
QR TOOLS( OPEN SOURCE)
Python-qrtools is a backend ("library") for creating and decoding QR Codes in python and it depends on qrenconde and zbar. QtQR is a Qt GUI (front-end) for python-qrtools that makes easy creating and decoding the codes. 5.7
DESIGN FOR GENERATION OF QR CODE
The quick response codes were developed using Python and PyQRCode, which were experimented by encoding different texts, alphanumeric characters, email identities. Moreover, to generate three a single colored qr code, it was important that initialy generated black and white code should be encoding different texts and the generated code ought to have the same version number. Versions of the generated codes should be same, because version determines the size of the code as a result while adding the codes, they all are properly aligned on top of each other. If the size of the image is same then only one can multiplex the images properly. According to the objective, we had to develop a colored quick response code which could only be designed, if after the generation of black and white codes, those codes were colored individually with different colors. Different colors were needed for different codes because, if we
53 | P a g e
multiplexed the same colored codes one above the other it, would have been impossible to separate them in order to get decoded. For encoding we generate three QR Codes with different text and color each of them with one of the color depending on which color we want to create our QR Code. This was done by changing the value of white pixels to pixel value of the color which needs to be filled. After the generation of these individually colored codes, encoding different texts in same version with same color (one color/code), these codes were added up together to generate a single colored quick response code. This code has the capacity of three separate codes. In other words, we can say that its encoding capacity is changed to thrice as compared to a single black and white encoded quick response code and occupy space of only one code while printing. The encoding process is described in Fig below.
Figure 5-1. Design for Generation of QR Code
54 | P a g e
5.8
DESIGN OF EXPERIMENT FOR DECODING OF QR CODE
The generated colored quick response code was decoded by separating the individual codes which were added to form colored code. For this procedure, every layer of the background color in addition to its color in finder patterns were extracted back. After extracting each channel local thresholding is applied on each channel and then decoding is performed using QR Tools. The process of recognition is just the opposite of generation. The decoding procedure is shown in figure below:
Figure 5-2. Design for Decoding of QR Code
55 | P a g e
CHAPTER 6: RESULTS AND DISCUSSION
This chapter deals with the results achieved till now while designing of interface, generation and recognition of Covert Quick response codes. 6.1
DEVELOPED INTERFACE
Below is the figure shown of the developed interface named "PIMAD". In lower left splitter we can display image and in lower right splitter we can display region of interest of the loaded image. By using Browse option we can browse and load any image.
Figure 6-1."PIMAD" interface
56 | P a g e
In addition to this RGB and HSV of the loaded image can be displayed in a separate window by using HSV and RGB buttons. In following figures HSV and RGB plot of a loaded image is shown.
Figure 6-2. Original image
6.2
Figure 6-3. HSV image
Figure 6-4 RGB histogram
RESULTS FOR REGION IDENTIFICATION IN QR CODE
By using the proposed algorithm for region identification in QR Code, finder patterns in the given QR is detected and shown by green color bounding box as shown in figure below:
Figure 6-5. Region Identification
By using these finder patterns we can determine the position and inclination of the given QR Code. Position of inner square can be determined from the position of outer square as each of these patterns can be considered as a three superimposed concentric squares which is consisted of 7*7 dark , 5*5 light and 3*3 dark modules and ratio of module width of each finder pattern is 1:1:3:1:1 [1]. 57 | P a g e
6.3
RESULTS FOR GENERATION
OF QR CODE
While generation, the three white and black module QR codes were encoded by using PyQRCode as shown in table below: Table 8. Result for Generation of QR Code
S.NO.
Data Encoded
1. “www.csio.res.in”
2.
"www.facebook.com"
3.
"www.banasthali.org”
58 | P a g e
Generated QR Code
These generated QR codes are then saved as .png format and read in spyder python and then algorithm is applied for region identification. The results for this are shown below:
Figure 6-6. Encoded image I
Figure 6-7. Encoded image II
Figure 6-8. Encoded image III
These three colored Covert QR codes are added together and care is taken of the images for its pixel values which can be changed while adding or superimposing the previously generated QR code. Result for merging three color codes to form a single RGB coloured Covert QR code (with increased capacity) is shown in figure below:
Figure 6-9. Generated RGB colored QR Code
Now after printing QR code on paper its color changes so, encoding of QR Code is done in CMYK Color Space. For these RGB channels are converted into CMYK channel and then merged together to generate CMYK Colored QR Code. Whole procedure for this is shown in fig below.
59 | P a g e
In given figure three monochrome QR codes are generated encoding messages M1, M2 and M3. After that from each QR Code we took one channel and converted them into cyan, magenta and yellow channels according to above mentioned formulae. At the end merged all channels together to generate encoded CMYK Colored QR Code as a result.
Figure 6-10. CMYK Encoding from RGB
Result for merging c, m, y, k channels to generate colored QR Code is shown in figure below:
Figure 6-11. CMYK Colored QR Code
60 | P a g e
6.4
RESULTS FOR DECODING OF QR CODE
After generating the CMYK QR code, it is again converted back to RGB for decoding process. Then de-layering process is performed in OpenCV by removing the layers of RGB colours one by one and then converting it to gray scale so as to obtain the three originally encoded QR codes. Then local thresholding is done and output is given to open source QR decoder -QR Tools to decode them and to verify results The results verified that the messages decoded were the same as the encoded messages in each of the Quick Response codes initially.
Table 9. Results for Decoding
S.NO.
Retrieved QR Images
Data Decoded
1. “www.csio.res.in”
2.
"www.facebook.com"
3.
"www.banasthali.org”
61 | P a g e
6.5
RESULT FOR ENCODING QR CODE IN HSV
Since we are facing some difficulties while decoding in RGB and CMYK color spaces so we performed both encoding and decoding of QR codes in HSV color space In this we have tried to encode 5 messages parallely whereas in RGB and CMYK we have encoded only three messages and combined all of them to generate HSV Colored QR code. For encoding QR Code in HSV encoded five QR Codes are generated. We took one channel from each code and integrated them in 32 points and put this in an image of same dimension as of generated QR Codes. After that keeping S and V constant we have divided H into 32 points and assign these HSV values to the integrated image so that resulting image is in HSV Color Space as shown in fig below
Figure 6-12. Result for Encoding in HSV
6.6
RESULT FOR DECODING QR CODE IN HSV
At the time of decoding if image is captured from camera then it is again converted back into HSV and split image into hue, saturation and value channels. After this we 62 | P a g e
find out indexes where hue value equals to the values stored in H and convert them into binary and assign them to i no of images to retrieve encoded images back. Result for decoding QR in HSV is shown below in figures. Since results are in HSV so black color of QR pattern turns into blue color and white color changes in dark red color.
Figure 6-13. Result for Decoding in HSV
63 | P a g e
CHAPTER 7: CONCLUSION AND FUTURE SCOPE
From the above results for generation and recognition of Covert QR codes, it is concluded that the Covert (meaning invisible to human eye) Quick Response codes is a two dimensional technology, focused with the goal of high speed reading and encoding capacity compared to traditional barcodes. They have taken control over the market due to factors like, small size, readability from any direction, dirt and damage resistant, high capacity, kanji and kana encoding capability, structured appending feature. Some of the applications include product tracking, marketing, ticketing [1] etc In order to increase the storage capacity of information being encoded in QR code, different colours can be used and the information encoded is directly proportional to Log2N, where N is number of colours used [2]. Therefore, using more number of colours means encoding more information. The problems encountered while using more number of colours is that the resulting observed surface colour depends on surface reflectance spectrum. Furthermore while decoding, determining the colour index of each is a challenging task. Other parameters include: colour drifts, colour fading, square patterns not being identified correctly, unknown or poorly calibrated camera colour response, camera blur, false detection, quantization and noise. The methodologies proposed in this paper provides an effective method for extending monochrome barcodes to colour barcodes . Decoding of colored QR Code is one of the challenging task since, there is significant cross-channel coupling observed among the printer and camera colour channels [4]. In future, different colored inks of invisible markers visible under ultraviolet or infra red light, having different wavelengths can be used for generating covert quick response codes which can be decoded using scanner.
64 | P a g e
Appendices
7.1
APPENDIX-A: DIAGRAMS AND DOCUMENTATION TOOL
DATA FLOW DIAGRAM:
It is graphical representation which depicts the "flow" of data through a system and abbreviated as DFD. It shows what type of data will be input to and output from the system, where the data will come from and go to, and where the data will be stored [37]. Now proposed DFD for this system is given below:
LEVEL 0 DFD:
It is top level diagram also known as context diagram. It only shows the main processes of the system.
Input Image
User
Decoder Decoded Message
Figure 0-1. Level 0 DFD
LEVEL 1 DFD: This level of DFD depicts how the process is divided into sub- processes, and which together provide all of the functionality of the system as awhole.
65 | P a g e
Input
Encoded Image
Read Image
Process
Find QR Code
Get Byte Data
Decoder
Decode
D
Figure 0-2. Level 1 DFD
LEVEL 2 DFD:
Level 1 DFD is further sub-divided to form level 2 DFD which shows the whole system at more refine level. Below figure shows level 2 DFD.
Plot
D
Find HSV
Image HSV
Read values
Input Image
Read RGB Component
Plot
D
Find Histogram Read Input
Identify Region
Extract
Decoder Find ROI
Decode
D
Decoded message
Figure 0-3. Level 2 DFD
66 | P a g e
RGB Hisatogram
D Extract ROI
Decoded Message
7.2
APPENDIX-B: THEORETICAL AND CONCEPTUAL FRAMEWORK:
For increasing the storage capacity of QR code so that more information can be encoded in it, different colors can be used. The information encoded is directly proportional to Log2N, where N is number of colors used [2]. Therefore, using more number of colors means encoding more information. The problems encountered while using more number of colors is that the resulting observed surface color depends on surface reflectance spectrum. Furthermore while decoding, determining the color index of each is a challenging task. Other parameters include: color drifts, color fading, square patterns not being identified correctly, unknown or poorly calibrated camera color response, camera blur, false detection, quantization and noise.
67 | P a g e
References
[1]
J. A. Lin and C. S. Fuh, “2D barcode image decoding,” Math. Probl. Eng., vol. 2013, no. 3, 2013.
[2]
H. Bagherinia and R. Manduchi, “A theory of color barcodes,” Proc. IEEE Int. Conf. Comput. Vis., pp. 806–813, 2011.
[3]
A. Denso, “Qr code essentials,” Retrieved fr om http//www. nacs. org/LinkClick. aspx, pp. 1–12, 2011.
[4]
H. Blasinski, O. Bulan, and G. Sharma, “Per-colorant-channel color barcodes for mobile applications: An interference cancellation framework,” IEEE Trans. Image Process., vol. 22, no. 4, pp. 1498–1511, 2013.
[5]
K. H. Pandya and H. J. Galiyawala, “A Survey on QR Codes: in context of Research and Application,” Int. J. Emerg. Technol. Adv. Eng. Website www.ijetae.com ISO Certif. J., vol. 9001, no. 3, pp. 258–262, 2250.
[6]
R. R. M, P. G. Student, and P. Institue, “A Review on 1D & 2D Barcode with QR Code Basic Structure and Characteristics,” vol. 4, no. 01, pp. 4–7, 2014.
[7]
E. Chen and L. Lei, “Research and realization of encoding on QR code,” 2012 5th Int. Congr. Image Signal Process. CISP 2012, no. Cisp, pp. 1299–1302, 2012.
[8]
A. Walsh, “QR Codes – using mobile phones to deliver library instruction and help at the point of need,” J. Inf. Lit., vol. 4, no. 1, pp. 55–64, 2010.
[9]
H. Kato and K. T. Tan, “Pervasive 2D barcodes for camera phone applications,” IEEE Pervasive Comput., vol. 6, no. 4, pp. 76–85, 2007.
[10] T. Lotlikar, R. Kankapurkar, A. Parekar, and A. Mohite, “Comparative study of Barcode, QR-code and RFID System,” Int. J. Comput. Technol. Appl., vol. 4, no. 5, p. 817, 2013. [11] I. T. Asare, W. Yan, A. Road, and D. Asare, “The Effective Use of Quick Response ( QR ) Code as a Marketing Tool,” vol. 2, no. 12, 2015. [12] A. B. Graham, “Image Processing Algorithm for Detection of TwoDimensional Visual Code Markers,” Computer (Long. Beach. Calif)., pp. 1–5, 68 | P a g e
2006. [13] H. C. Huang, F. C. Chang, and W. C. Fang, “Reversible data hiding with histogram-based difference expansion for QR code applications,” IEEE Trans. Consum. Electron., vol. 57, no. 2, pp. 779–787, 2011. [14] T. Wakahara and N. Yamamoto, “Image processing of 2-dimensional barcode,” Proc. - 2011 Int. Conf. Network-Based Inf. Syst. NBiS 2011, no. 1, pp. 484–490, 2011. [15] T. Falas and H. Kashani, “Two-dimensional bar-code decoding with cameraequipped mobile phones,” Proc. - Fifth Annu. IEEE Int. Conf. Pervasive Comput. Commun. Work. PerCom Work. 2007, pp. 597–600, 2007. [16] A. Sun, Y. Sun, and C. Liu, “The QR-code reorganization in illegible snapshots taken by mobile phones,” Proc. - 2007 Int. Conf. Comput. Sci. its Appl. ICCSA 2007, pp. 532–536, 2007. [17] A. Gaikwad and K. R. Singh, “Information Hiding using Image Embedding in QR Codes for Color Images : A Review,” vol. 6, no. 1, pp. 278–283, 2015. [18] T. Wiputtikul and S. Vongpradhip, “A Technique to Add Error Detection of QR Code Decoding by Using Micro QR Code,” Int. J. Electr. Energy, vol. 1, no. 3, pp. 168–172, 2013. [19] S. Aryachandran and R. L. Jyothi, “Secure Color QR Codes,” vol. 2, no. c, pp. 77–85. [20] K. Homkajorn, M. Ketcham, and S. Vongpradhip, “A Technique to Remove Scratches from QR Code Images,” pp. 127–131, 2012. [21] Y. Liu and M. Liu, “Automatic Recognition Algorithm of Quick Response Code Based on Embedded System,” Sixth Int. Conf. Intell. Syst. Des. Appl., vol. 2, pp. 783–788, 2006. [22] N. Communications and E. Technologies, “Embedding of color image on QR code with reduced processing time using Genetic Algorithm,” vol. 1, no. 2, pp. 14–18, 2015. [23] M. Ramya and M. Jayasheela, “Improved Color QR Codes for Real Time Applications with High Embedding Capacity,” Int. J. Comput. Appl., vol. 91, no. 8, pp. 8–12, 2014. 69 | P a g e
[24] K. Naresh and P. N. Pillai, “QR Verification System Using RSA Algorithm,” vol. 10, no. 2, pp. 433–437, 2014. [25] J. M. Meruga, W. M. Cross, P. Stanley May, Q. Luu, G. a Crawford, and J. J. Kellar, “Security printing of covert quick response codes using upconverting nanoparticle inks,” Nanotechnology, vol. 23, no. 39, p. 395201, 2012. [26] N. Victor, “Enhancing the Data Capacity of QR Codes by Compressing the Data before Generation,” vol. 60, no. 2, pp. 17–21, 2012. [27] S. Ramya and C. S. Joice, “An Optimized Image and Data Embedding in Color QR Code,” vol. 23, pp. 66–72, 2015. [28] T. Dean and C. Dunn, “Quick layered response (qlr) codes,” … Electr. Eng. Stanford Univ. Tech. …, 2012. [29] P. M. Nishad and R. Manicka Chezian, “Various Colour Spaces and Colour Space Conversion,” J. Glob. Res. Comput. Sci., vol. 4, no. 1, pp. 44–48, 2013. [30] “M0122445.pdf.” . [31] “S. Agrawal, N. K. Verma, P. Tamrakar and P. Sircar, „Content Based Color Image Classification using SVM,‟” p. 5945183, 2011. [32] Microsoft, “Color spaces,” pp. 15–34, 2010. [33] A. Mukherjee and S. Kanrar, “Enhancement of Image Resolution by Binarization,” Int. J. Comput. Appl., vol. 10, no. 10, pp. 15–19, 2010. [34] S. H. Shaikh, A. Maiti, and N. Chaki, “Image binarization using iterative partitioning: A global thresholding approach,” 2011 Int. Conf. Recent Trends Inf. Syst. ReTIS 2011 - Proc., pp. 281–286, 2011. [35] S. Rajput and S.R.Suralkar, “Comparative Study of Image Enhancement Techniques,” Int. J. Comput. Sci. Mob. Comput., vol. 2, no. 1, pp. 11–21, 2013. [36] A. Mordvintsev, “OpenCV-Python Tutorials Documentation,” 2014. [37] D. S. Le Vie, “Understanding Data Flow Diagrams.”
70 | P a g e
List of Publications
Conference Paper in MICTDI 2016 Pragya Mishra1, Shambo Roy Chowdhury2, Ritesh Kumar1 and Amol P. Bhondekar1, “Region Identification and Decoding of Security Markers ". (Communicated, IJCA 2016) 1 2
Banasthali Vidyapith Rajasthan CSIR-CSIO, Chandigarh
71 | P a g e
