Detection of Counterfeit by the Usage of Product ... - ScienceDirect

Available online at www.sciencedirect.com

ScienceDirect Procedia CIRP 26 (2015) 430 – 435

12th Global Conference on Sustainable Manufacturing

Detection of counterfeit by the usage of product inherent features Matthias Blankenburga,* , Christian Hornb , J¨org Kr¨ugera,b a Fraunhofer-Institute b Technische ∗

for Production Systems and Design Technology , Pascalstr. 8-9, D-10587 Berlin, Germany Universit¨at Berlin, Institute for Machine Tools and Factory Managment, Pascalstr. 8-9, D-10587 Berlin, Germany

Corresponding author. Tel.: +49-30-314-28689; fax: +49-30-3917517. E-mail address: [email protected]

Abstract One aspect of the economical dimension of sustainable business development is the protection of high value products from counterfeiting. This holds especially true for consumer goods since the sustainable manufacturing process gains a more and more important role, e.g. in the creation of a brand image. In this paper we propose a method for detecting counterfeit by capture of inherent features indissolubly linked with the product induced by the production process itself. Since a counterfeiter gains margin by the use of inferior production processes and material the differences between genuine product and counterfeit can be captured in an automated fashion. The proposed method not only renders the application of artificial security tags obsolete which helps reducing the material usage but also gives enhanced protection against counterfeiting as the inherent characteristics cannot be removed from the article. © B.V. This is an open access B.V. article under the CC BY-NC-ND license c 2015  2014 Elsevier The Authors. Published by Elsevier (http://creativecommons.org/licenses/by-nc-nd/3.0/). Peer-review under responsibility of Assembly Technology and Factory Management/Technische Universit¨at Berlin. Peer-review under responsibility of Assembly Technology and Factory Management/Technische Universität Berlin. Keywords: Counterfeit Detection; Pattern Recognition; Machine Learning; 2D and 3D Image Processing; Electronic Nose

1. Introduction Figure 1, taken from the annual ”Report on EU customs enforcement of intellectual property rights” of the European Union in 2012 [1], shows a continuous upward trend in the number of shipments suspected of violating intellectual property rights for the last years. In 2011 more than 90 thousand cases of detained articles were reported. The value to their equivalent genuine products is estimated to be over 1.2 billion Euro and this covers only Europe. To get an idea of the worldwide amount of economic damage for the last years the report ”The Economic Impact of counterfeiting and piracy” [2] of 2008 estimates a total loss of 250 billion dollars in the year 2007. This report covers the analysis of international trade in counterfeit and pirated products, but these estimates do not include domestically produced and consumed counterfeit and pirated digital products being distributed via the Internet. If these were also considered, the magnitude of counterfeiting and piracy worldwide could be several hundred billion dollars more in 2007. Furthermore, if we compare these numbers to the amount of cases reported in Figure 1, they probably doubled in 2011. The effect of counterfeiting and piracy is an intermission of innovation and thus impairment of economic growth. The economic damage affects in particular countries that use advanced production and manufacturing processes based on intensive research and development to produce high quality goods.

Fig. 1. Cases of customs enforcements of intellectual property rights at the European border, from [1]

Another very important argument to enable the differentiation between brand products and their counterfeits is safety. It is stated in the OECD report that the products counterfeiters and pirates produce and distribute are often of minor quality and can even be dangerous and health hazards. Common standards that ensure the safety of products can be ignored by product pirates and the used materials can be dangerous. With the magnitude of counterfeiting and piracy in mind, these reports emphasize the need for more effective enforcement to combat the counterfeiting and piracy on the part of governments and businesses alike.

2212-8271 © 2015 Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/). Peer-review under responsibility of Assembly Technology and Factory Management/Technische Universität Berlin. doi:10.1016/j.procir.2014.07.062

Matthias Blankenburg et al. / Procedia CIRP 26 (2015) 430 – 435

Fig. 2. Current scenario for counterfeit detection through customs officials

431

Fig. 3. Desired scenario for counterfeit detection through customs officials

2. State-of-the-Art Technology A key component for this enforcement is the development of new methods for automated counterfeit detection. The review of copyright infringement of registered trademarks and products is not easy to implement. Due to the high number of pending trademarks and constantly added new applications it is very difficult for the executive bodies, such as customs, to register violations of trademark rights immediately and in a comprehensive manner. The awareness to all registered brands and products is for the executive organs not possible and therefore necessarily, trademark infringement remains unnoticed. The current scenario for products entering a market in a foreign country is displayed in Figure 2. Here it is shown how customs officials usually handle the inspection of products at the border. First the goods arrive at a specific check point, usually via sea- or airfreight. If the customs officer notices some anomaly in the paperwork, he will check the cargo containers. As discussed earlier the officer is often not an expert for the shipped product, so he could not detect a counterfeit. Instead the company producing the genuine product is contacted to send their own expert, which can verify the product. This is a time-consuming and expensive process, therefore most containers in question often remain unnoticed. To overcome these limitations in the checkup routine an automated expert-system is necessary that can support the customs officials, as shown in Figure 3. Given that the officer could verify the shipped cargo by himself while the company issues the authentication system for their products. This idea was adopted more recently through an application of artificial security features to products. The issues of such security labels are in part the high cost, and additionally the integration into the product. On the other hand high-quality branded products, as the target of counterfeiting, have usually, due to the production processes and materials used, and in view of its processing machinery and equipment, a grade of high quality. The specific conditions of production, manufacturing technologies and materials generate specific features, which identify the product uniquely. These features may be detected multimodal by man, including tactile (plasticity, elasticity, thermal conductivity, surface structure), visual (shape, color, surface texture, transparency), olfactory (smell) or acoustic (sound) perceptions. In general, only the person familiar with the manufacture of the product can combine these inherent characteristics in their entirety so that it can differentiate the genuine product from a clear counterfeit. The innovation of this text is the detection of these features in an automated fashion through the combination of digital sensing and machine learning, rendering the application of artificial security labels obsolete.

Common automated counterfeit detection methods require nowadays additional security features at the product itself. Several methods have been developed, but main advantages and disadvantages remain similar. Additional security features require further steps in production to add these features to the product. This raises expenses, manufacturing time and development efforts, which is clearly a disadvantage. On the other hand the security is enhanced and an original brand is easy to detect in an automated fashion, since there is a specific feature to look for. But this could also be a main disadvantage, if the security feature itself is easy to reproduce and could be added to any forged product. Another challenge is to link the security label to the brand product in a way it cannot be removed or stolen. This way product pirates could label their counterfeits easily as an original with an original security label. Counterfeit detection without artificial security tags is a solution to these problems, if the counterfeit is distinguishable from the original brand. 3. Product-Inherent Features The Inherent ID Project adopts a novel approach to protecting high-value products from counterfeiting. The approach is based on the stationary and mobile capture of key product features indissolubly linked with the product which enable its production process to be traced. This not only renders obsolete the application of security tags but also gives enhanced protection against counterfeiting as the inherent characteristics that the high-quality production process impregnate in the genuine product are combined with one another to serve as proof of product identity. They form the basis on which electronic certificates of authenticity can be issued without the need for complicated explicit security markings. Methods for the capture and control of identity characteristics are being elaborated in the Inherent ID project for system integration using intelligent cameras and an electronic nose. The identity characteristics captured by this range of sensors serve both for the product identification and product authentication. At the same time this also offers opportunities for improving documentation of product flows in the supply chain. Full documentation serves as a complement to the inherent characteristics of the authentic product and offers valuable information of verification of the genuine article, thus serving to safeguard against counterfeits. Optical 2D and 3D characteristics as well as olfactory characteristics are combined with one another to serve as proof of product identity. They form the basis on which electronic certificates of authenticity can be issued without the need for complicated explicit security markings. The identity characteris-

432

Matthias Blankenburg et al. / Procedia CIRP 26 (2015) 430 – 435 1.8

User Input

Gaussian Gradient Laplacian

1.6

Global fractal dimension f(α)

1.4

Gaussian Smoothing

Laplacian Filtering

Gradient Filtering

1.2

1

0.8

0.6

0.4

0.2

0

Calculate Local Fractal Dimension

1

1.5

2 2.5 Local fractal dimension α

3

3.5

1.8 Gaussian Gradient Laplacian

1.6

1.4

Global fractal dimension f(α)

De
ne Sets from LFD Map

Calculate Global Fractal Dimension for Sets

1.2

1

0.8

0.6

0.4

0.2

Texture Signature

Fig. 4. Workflow for generating a textural signature

tics captured by this range of sensors serve both for product identification and product authentication. At the same time this also offers opportunities for improving documentation of product flows in the supply chain. 3.1. Texture Features The ability to characterize visual textures and extract the features inherent to them is considered to be a powerful tool and has many relevant applications. A textural signature capable of capturing these features, and in particular capable of coping with various changes in the environment would be highly suited to describing and recognizing image textures [3]. As humans, we are able to recognize texture intuitively. However, in the application of Computer Vision it is incredibly difficult to define how one texture differs from another. In order to understand, and manipulate textural image data, it is important to define what texture is. Image texture is defined as a function of the spatial variation of pixel intensities [4]. Furthermore, the mathematical description of image texture should incorporate, identify and define the textural features that intuitively allow humans to differentiate between different textures. Numerous methods have been designed, which in the past have commonly utilized statistical models, however most of them are sensitive to changes in viewpoint and illumination conditions [3]. For the purposes of mobile counterfeit detection, it is clear that this would be an important characteristic for the signature to have, as these conditions can not be entirely controlled. Recently a description method based on fractal geometry known as the multifractal spectrum has grown in popularity and is now considered to be a useful tool in characterizing image texture. One of the most significant advantages is that the multifractal spectrum is invariant to the bi-Lipschitz transform, which is a very general transform that includes perspective and texture surface deformations [3]. Another advantage of Multrifractal Spectra is that it has low dimension and is very efficient to compute [3] in comparison to other methods which achieve invariance to viewpoint and illu-

0

1

1.5

2 2.5 Local fractal dimension α

3

3.5

Fig. 5. Multi-Fractal-Spectra of texture of a textile product (top) and its counterfeit (bottom)

mination changes such as those detailed in [5], [6]. One of the key advantages of multifractal spectra, which is utilized here is that they can be defined by many different categorizations or measures, which means that multiple spectra can be produced for the same image. This is achieved through the use of filtering, whereby certain filters are applied to enhance certain aspects of the texture, to create a new measure. Certain measures are more or less invariant to certain transforms, and the combination of a number of spectra achieves a greater robustness to these. The worklfow is depicted in Figure 4 and an example is given in Figure 5. 3.2. Shape Features Since manual detection is often done visual by customs officials, visual features are also important for any automatic detection mechanism. Besides detecting features through two dimensional image processing, three dimensional data capture is necessary for counterfeit detection, because it provides important additional information. To capture a real-world object in three dimensions a 3D scanner, or range camera, can be used. The basic principles of 3D scanners available on the market are triangulation, time-offlight or interferometric approaches, whereas each principle has its advantages or disadvantages. For a profound insight into that topic refer to [7]. We use a mobile structured-light 3D scanner for our application, but in general any three dimensional data acquisition method can be used to capture a real-world object. But while using different kinds of scanning techniques the results may vary. One distinguishable feature of brand products is the shape itself. Shape matching is a well studied topic and several publications can be found over the last 15 years. Despite many different approaches available, most practical applications still use the 1992 introduced Iterative Closest Point Algorithm (ICP) [8] or its optimized variants to match objects. This is due to the fact that most newer approaches are neither easy to implement nor able to run at a reasonable speed for the use in commercial

Matthias Blankenburg et al. / Procedia CIRP 26 (2015) 430 – 435

Fig. 6. Key Points for two Scans of the same Shoe

433

Fig. 7. Transformation of shape features

software. Approaches using global features are not suitable for counterfeit detection, where minor details of an object can be highly important. Therefore only approaches detecting local features were taken into consideration. Our automatic local-featurebased matching algorithm consists of two major parts: a feature detector and a feature descriptor. The classification is done later after the texture and odor features are combined with the shape features trough feature fusion. Feature Detector The feature detector finds points of interest on a given mesh which are usually extrema in a specific mathematical notation. In two-dimensional approaches well known techniques like corner detection are used. In three dimensions new approaches based on two-dimensional image processing algorithms that use feature-based approaches have been developed. Examples are the Harris-3D-feature detector [9], several portations of the SIFT-algorithm to three dimensions [10,11] or the 3D equivalent of SURF [12]. Other approaches use for example HeatKernel-Signatures [13] or maximally stable extremal regions (MSER) [14] to detect features. For counterfeit detection we use a Scale Space approach to detect keypoints [7]. The Scale Space is usually constructed by repeatedly applying a filter to a given mesh. L(x, y, z, σ) = F(x, y, z, α) ∗ M(x, y, z) whereas M is the mesh and F is the filter-kernel. The difference of the resulting meshes is then examined for extrema. As filterkernel a finite difference approximation of the Laplace operator 1 αi Pi n i=1 n

G(x, y, z, α) =

was used, where α is a weighting factor and Pi are the neighbors of the regarded point. The advantage of this smoothing approach is that each point keeps is relative position, keeping the shape itself of the whole object. The differences of the mean curvature at each point is the criterion for constructing the Scale Space. Figure 6 shows detected keypoints of two different scans of the same shoe. Feature Descriptor The feature descriptor transforms the area at the detected keypoint into an easy comparable and meaningful description. Usually the approaches combine feature detectors and feature descriptors into one method. Well known methods like MeshSIFT [11] or 3D-SURF [12] use their three-dimensional counterpart of feature descriptors developed for two-dimensional applications. Approaches using Heat Kernel Signatures [13,15] use these for both – detection and description. In contrast to

Fig. 8. Olfactory pattern of a genuine jersey (top) and a counterfeit (bottom)

that another approach called Spin-Images [16] is a feature descriptor only. It is able to describe an object locally or globally. In [17] this concept was adopted to a scale-invariant version encoding local information. Figure 7 shows a transformation of the area surrounding keypoints into a 2D dense map using Spin Images [16]. Here a 3D mesh is transformed into several 2D maps, each related to a keypoint S O : R3 → R2 The 2D dense map is constructed using the equation  (α, β) = ( x − p2 − (n · (x − p))2 , n · (x − p)) where (α, β) describe the new 2D coordinates. It is a cylindric coordinate system with its point of origin in the regarded point of the mesh. A set of ranked Spin Images describes the object itself, so it can be matched to the abstract brand model. 3.3. Odor Features There are many ambient influences to odor sensing. For example humidity and temperature are different in Germany and Malaysia. Additionally a mathematical expression for the composition of odor is not linear, so odorous influences cannot be filtered out easily. Given these facts and that the used Artinos 16-channel metal-oxide Sensor returns most unspecific data it is a challenge to filter environmental influences. To meet the challenge of extracting desired signals in a robust fashion and filter the environmental noise we use a similar approach to blind source separation, where two different measurements are conducted. The first one is a pattern from the environment without test object. The second is a pattern from the desired sample in the before mentioned environment. The first signal can then be used to extract the plain odor of the object itself from the second signal. The components can be identified and thus the ambient influence can be filtered. Since

434

Matthias Blankenburg et al. / Procedia CIRP 26 (2015) 430 – 435

Fig. 10. Core information of a textile sample, arb. unit

User Input

2D Image Capturing

Textur Analysis

Logo Matching

Fig. 9. The Independent Components with reasonable high similarity measure are indicated by arrows. Noise contribution was omitted. arb. unit N

Acquire 3D Data?

Y

the electronic nose measurement data delivers a nonlinear mixture of the environmental and sample odor there is no obvious connection between these two patterns. One approach to divide the signals into their components is the Independent Component Analysis (ICA). Here the separation is done by statistical means. At most the ICA can return as many independent components as the number of sensors used for capturing the input data, whilst reducing the complexity. In general the ICA has two major problems. The first problem is that the independent components are permuted. The sequence of two algorithmic cycles might not be the same even with the same data. The second problem is the loss of variance information in the independent components, since it cannot be restored. The independent components were extracted by an extended Bell & Sejnowski Algorithm [18] with adjusted break condition. Here the covariance criterion [19] was used.

Classi
cator/ Decission Capture 3D Model

3D Matching

Acquire Smell Data?

N

Y

Capture Odour

Odour Matching

Fig. 11. Sophisticated workflow for counterfeit detection

4. Workflow E{g(u)u } = I T

If this equation is true the gi (yi ) and y j are uncorrelated for i  j. Therefore this can be seen as a nonlinear variant of principal component analysis. The next step after the ICA is to check the integrity of the independent components. There are a some independent components which seem to be noise. An autocorrelation analysis identifies a possible noise contribution. These independent components can be omitted. Afterwards the similarity between the sample and the environmental independent components are evaluated by applying the cosine distance. The results are shown in Figure 9. The independent components with the strongest connection are the independent components which represent the environment in the sample data and could be omitted as well. The arrows in Figure 9 indicate the corresponding independent components. The signals which are exclusive to the sample measurement represent the the core information on the odor of the test object. Figure 10 shows the extracted signal patterns which characterize the textile sample.

With the features described above there is a strong basis for automated classification of patterns. The key point for a robust and reliable counterfeit detection is the combination of these features and additional user information with the aim to derive a decision whether the probe is likely to be a counterfeit. An advantage of the proposed algorithms for feature extraction is the possibility to utilize statistical frameworks since the features are represented by probability density distributions. In general there are various approaches possible. Starting with a direct fusion of the features as proposed in [20], or a more sophisticated approach which is taking the process of probing into account. Such a workflow is depicted in Figure 11. Here the decision process is not necessarily based on the utilization of all features, since some of them are dispensable or could be misleading. Think of the probing of shirt, obviously the 3D geometry cannot give a relevant contribution to the decision process and the 3D scanning can therefore be omitted. The classification itself is done with an adjusted Bayesian approach where special account was given to the detection of novel and

Matthias Blankenburg et al. / Procedia CIRP 26 (2015) 430 – 435

435

7. Acknowledgements

Fig. 12. Future Scenario for Counterfeit Detection

therefore unknown patterns. This was done with estimation of the Level of Significance distribution, which gives a decision information and an additional value of the plausibility of this decision, cf. [21].

5. Future Implementation The approach of the project Inherent ID can be adopted to a possible future scenario for counterfeit detection. As shown in Figure 12 the approach could be ported to to work with consumer electronics like smartphones, since 3D cameras are already available there. The textural features and the shape features of an object could be detected with the built-in cameras. The classification itself can then be done with an approach using Service Oriented Architectures (SOA), where the features are transfered from the smartphone over the Internet to a server. This is necessary because even recent smartphones with multicore cpu’s are too slow to compute the proposed algorithms in a timely fashion. This enables not only customs officials to detect counterfeits, any customer would be able to do that using the detection app. This could lead to a whole new market driven combat against product piracy.

6. Conclusion It was shown that the Inherent-ID Project adopts a novel approach to protecting high-value products from counterfeiting. The approach is based on the stationary and mobile capture of key product features indissolubly linked with the product which enable its production process to be traced. This not only renders the application of security tags obsolete but also gives enhanced protection against counterfeiting as the inherent characteristics that the high-quality production process impregnate in the genuine product are combined with one another to serve as proof of product identity. They form the basis on which electronic certificates of authenticity can be issued without the need for complicated explicit security markings. Methods for the capture and control of identity characteristics are being elaborated in the Inherent-ID project for system integration using intelligent cameras and an electronic nose. The identity characteristics captured by this range of sensors serve both for the product identification and product authentication. At the same time this also offers opportunities for improving documentation of product flows in the supply chain. Full documentation serves as a complement to the inherent characteristics of the authentic product and offers valuable information of verification of the genuine article, thus serving to safeguard against counterfeits.

The authors acknowledge the funding of the research project Inherent-ID by the senate of the state Berlin and the European Regional Development Fund. The project is embedded in the Fraunhofer Cluster of Innovation Next Generation Identity Berlin Brandenburg. Furthermore we would like to acknowledge the work done by our students, namely Evelyn Jungnickel, Maximilian Fechteler and Norman Franke. 8. References [1] European Commission. Report on EU customs enforcement of intellectual property rights, Results at the EU border - 2011. ISBN 978-92-79-25362-1; 2012. pp. 10-19. [2] OECD. The Economic Impact of Counterfeiting and Piracy. Paris: OECD; 2008. www.oecd.org/sti/counterfeiting [3] Xu Y, Ji H, Ferm¨uller C. Viewpoint Invariant Texture Description Using Fractal Analysis. Int J Comput Vision 2009;83. pp. 85-100. [4] Tuceryan M, Jain AK. Texture Analysis, Handbook of Pattern Recognition & Computer Vision. 2nd ed.: World Scientifc Publishing Co. Ptc. Ltd.; 2001. [5] Varma M, Zisserman A. Classifying images of materials: Achieving viewpoint and illumination independence. ECCV 2002; Volume 3. pp. 255-271. [6] Varma M, Zisserman A. Texture Classification: are filter banks necessary?. CPVR 2003; Volume 2. pp. 691-698. [7] J¨ahne B. Digital Image Processing. Heidelberg: Springer; 2005. [8] Besl P, McKay N. A Method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 1992; 14(2). pp. 239256. [9] Sipiran I and Bustos B. A Robust 3D Interest Points Detector Based on Harris Operator. Proc. EUROGRAPHICS Workshop on 3D Object Retrieval (3DOR) 2010. [10] Flitton G, Breckon T, and Bouallagu N. Object Recognition using 3D SIFT in Complex CT Volumes. In Proc. British Machine Vision Conference 2010, BMVA Press. pp 11.1-11.12. [11] Maes C, Fabry T, Keustermans J, Smeets D, Suetens P, Vandermeulen D. Feature detection on 3D face surfaces for pose normalisation and recognition. In Proc. BTAS 2010. [12] Knoop J et al. Hough Transform and 3D SURF for robustthree dimensional classification. In Proc ECCV 2010. [13] Sun J, Ovsjanikov M, Guibas L. A concise and provably informative multiscale signature based on heat diffusion. Eurographics Symposium on Geometry Processing (SGP) 2009. [14] Litman R, Bronstein A, Bronstein M. Diffusion-geometric maximally stable component detection in deformable shapes. Arxiv preprint 2010. arXiv:1012.3951. [15] Bronstein A, Bronstein M, Guibas L and Ovsjanikov M. Shape google: Geometric words and expressions for invariant shape retrieval. ACM Transactions on Graphics (TOG) 2011; Volume 30 Issue 1. [16] Johnson A, Hebert M, Using spin images for efficient object recognition in cluttered 3d scenes. IEEE PAMI 21 1999. pp. 433-449. [17] Darom T, Keller Y. Scale invariant features for 3dmesh models. IEEE Transactions on Image Processing 2010. [18] Bell AJ, Sejnowski TJ. An information maximisation approach to blind separation and blind deconvolution. Neural Computation 1994; 7, 6. pp. 1129-1159 [19] Hyvaerinen A, Karhunen J, Oja E. Independent Component Analysis. New York: John Wiley and Sons; 2004. pp. 229-232. [20] Mitchell HB. Multi-Sensor Data Fusion: An Introduction. Heidelberg: Springer publishing; 2007. [21] K¨uhn S. Stochastic Engineering – Berechnung, Entwicklung und Modellierung bei unsicherer Information. Doctoral thesis: TU Berlin; 2010 ISBN 978-3-8322-9188-4. pp. 103-115.