Introduction to Matlab

83 downloads 330 Views 1MB Size Report
Color Measurement. • Part 2 : MatLab – Intro. • Computer Vision techniques. – Color measurement in CIE Lab units „polygonal‟. – Color segmentation ...
Computer Vision Based Image Analysis for Color Measurement and Segmentation

Vural Gökmen

Outline • Part 1 : Color in Foods – Intro • Color Measurement • Part 2 : MatLab – Intro • Computer Vision techniques – Color measurement in CIE Lab units „polygonal‟ – Color segmentation

Need to measure color • The color is the first sensation that the consumer perceives and uses as a tool to accept or reject, because the color observation allows the detection of certain anomalies or defects of a product. • Color changes during processing/storage may be indicative!

Color is important variable of foods • We can basically explain the importance of color in the food industry by following steps; – Color as a quality indicator of foods – Determining the defects and anomalities on food surface according the color differences, – Color as a process control variable in food manufacturing.

Color is important variable in “thermally processed foods” • Color can be used as a process control variable in thermal processes. • Browning in foods is a usual consequence of thermal processes such as frying, baking and roasting. • Degree of browning may be indicative !

to

t1

t2

t3

t4

Lo ao bo

L1 a1 b1

L2 a2 b2

L3 a3 b3

L4 a4 b4

Maillard Reaction and Color + amino compound

Aldose Sugar

N-substituted glycosylamine

+ H2O

Amadori rearrangement Amadori rearrangement product (ARP) 1-amino-1-deoxy-2-ketose -2H2O > pH 7

-3H2O ≤ pH 7

> pH 7 Fission products (acetol, diacetyl, pyruvaldehyde, etc)

Reductones -2H +2H

Schiff’s base of hydroxymethylfurfural (HMF) or furfural + H2O

Dehydroreductones

- amino compound

+ amino compound -CO2 Strecker degradation +α amino acid

Aldehydes

HMF or Furfural

Aldols and N-free polymers

+ amino compound

Degree of browning may be indicative for early and/or intermediate MRPs. This makes color measurement critical in thermally processed foods.

+ amino compound

Aldimines and Ketimes

Melanoidins (brown nitrogenous polymers)

Browning

Color is an indicator ! •



Color formation, in terms of browning, is mainly a surface phenomenon. This is somehow true for acrylmide formation. This fact simply proposes that these two process variables can be correlated.

However; • •

Color is not homogenously distributed on the surface, Rates of color and acrylamide formation may be similar or different,

~300 µg/kg

~2500 µg/kg

Color Measuring Devices • Color spectrometers

• Portable devices & color mouses

• Computer-vision systems

Color Measurement Traditional color measurement

• Color of foods is usually measured in units L*a*b* which is an international standard for color measurements, adopted by the CIE (Commission Internationale d'Eclairage). • The lightness ranges between 0 and 100 while chromatic parameters (a, b) range between -120 and 120.

CIE Color Space

Color models - conversion Among the color space models, CIE Lab model is the most complete one. It is a device independent model.

Once color is measured in any color space model, it can be converted to other color spaces. It is a built-in function in MatLab.

Conditions affect color measurement

The conditions should be constant during the color measurement to make different results comparable. In color measuring devices, this is instrumentally taken under control.

In computer-vision systems, the conditions should be fixed when taking digital images.

Layout of an image acqusition system Camera (megapixel) Light Source (D65)

Image Transfer (USB) Light Source (D65)

90o

45o

45o

Object

Image Analysis Station

Simple setup

D65 Lamps How different is the color at the end of baking?

ΔE There is no indutry standard for COLOR of processed foods. We, as food scientist, mostly deal with color changes during processing. This means DIFFERENCE or ΔE !

Difference (ΔE) •

Uniform changes of components in the L*a*b* color model aim to correspond to uniform changes in perceived color. So the relative perceptual differences between any two colors in L*a*b* can be approximated by treating each color as a point in a three dimensional space (with three components: L*, a*, b*) and taking the Euclidean distance between them. This Euclidean distance in L*a*b* space is ΔE (often called “Delta E”).

E  ( L*2  L*1 ) 2  (a2*  a1* ) 2  (b2*  b1* ) 2

Thermally processed foods non-homogeneously colored surfaces •



Color measuring devices cannot measure the whole surface. To take representative mean values, the measurements should be repeated.

Sample to be measured

specimen hole and sample geometries do not fit !!! Specimen hole

Image processing

>> RGB=imread(„crisp.jpg‟);

>> Imshow(RGB); MatLab >> renklab Lab __Value = 73.7869 7.4724 59.4035

Define the region of interest as you wish !

Result - Perception Similarity

Which „mean‟ value represents the sample better?

Outline • Part 1 : Color in Foods – Intro • Color Measurement • Part 2 : MatLab – Intro • Computer Vision techniques – Color measurement in CIE Lab units „polygonal‟ – Color segmentation

Introduction to Matlab

What is MATLAB? •

MATLAB is a program for doing numerical computation. It was originally designed for solving linear algebra type problems using matrices. It‟s name is derived from MATrix LABoratory.



MATLAB has built-in functions for solving problems requiring data analysis, image processing, curve fitting, optimization, and several other types of scientific computations. It also contains functions for 2-D and 3-D graphics and animation.

Matlab Desktop

Desktop Tools (MATLAB v7) • Command Window – type commands

• Workspace – view program variables – clear to clear – double click on a variable to see it in the Array Editor

• Command History – view past commands – save a whole session using diary

• Launch Pad – access tools, demos and documentation

Matlab Files (.m) •

Use predefined functions or write your own functions



Reside on the current directory or the search path – add with File/Set Path



Use the Editor/Debugger to edit, run

Vector / Matrix • a vector

x = [1 2 5 1]

x = 1

2

5

• a matrix

1

x = [1 2 3; 5 1 4; 3 2 -1]

x =

1 5 3

2 1 2

3 4 -1

Matrices y=x(2,3)

• x(i,j) subscription

y = 4

x = 1 5 3

2 1 2

3 4 -1

• whole row

y=x(3,:)

y = 3

• whole column

y=x(:,2)

y = 2 1 2

2

-1

Calculations at the Command Line MATLAB as a calculator

»

-5/(4.8+5.32)^2 ans = -0.0488 » (3+4i)*(3-4i) ans = 25 » cos(pi/2) ans = 6.1230e-017 » exp(acos(0.3)) ans = 3.5470

Assigning Variables » a = 2; » b = 5; » a^b ans = 32 » x = 5/2*pi; » y = sin(x)

Semicolon suppresses screen output Results assigned to “ans” if name not specified

y = 1

» z = asin(y) z =

1.5708

() parentheses for function inputs

Basic Matrix Operations Matrix Multiplication >> a=[ 2 3 4; 5 6 7]; >> b=[1 2 3]; >> a*b ans = 35 44 53 65 83 101

Array Multiplication >> a= [1 2 3 4; 5 6 7 8]; >> b= [1 2 3 4; 1 2 3 4]; >> c= a.*b c= 1 5

4 9 16 12 21 32

Toolboxes • Collections of functions to solve problems from several application fields. – – – – – – – –

Signal Processing Toolbox Image Processing Toolbox Curve Fitting Toolbox Neural Network Toolbox Fuzzy Logic Toolbox Statistics Toolbox Partial Differential Equations Toolbox And many others…

Image Processing with MATLAB

What is image processing? • Any form of information processing for which the input is an image, such as photographs or frames of video; the output is not necessarily an image, but can be for instance a set of features of the image. • Most image-processing techniques involve treating the image as a two-dimensional signal and applying standard signal-processing techniques to it.

Image Formats • The following image formats are suppoorted by MATLAB: – – – – – –

BMP HDF JPEG / JPG PCX TIFF XWB

Reading the Image with MATLAB • If an image is stored as a JPEG-image on the disc we first read it into Matlab. RGB=imread(‘Filename.jpg’)

• The picture is read as RGB format pixel-by-pixel between 0 - 255.

Case Studies • Case 1 : Color measurement in CIE Lab units – Matlab code „renklab.m‟

• Case 2 : Color segmentation – Matlab code „VectorQuantize.m‟

Renklab.m • Execute “renklab” at Command Prompt 1. Renklab converts the picture into a different format : Normalization between 0 and 1. (Divided each pixel by 255) A= im2double(A)

Renklab.m (cont.)

2. Selection of area in a polygonal shape Z=roipoly(A) 3. Renklab converts

it from RGB form to Lab form cform = makecform('srgb2lab'); lab = applycform(Z,cform);

Result

Calculated means of Lab values >> Lab__value = L

a

b

55.8246 20.8705 53.6507

Mean Lab of all pixels defined in this area !

Possible Implication of Renklab • Renklab measures the surface color non-invasively, or non-destructively. • It can be integrated into food processing lines for online monitoring the changes in color.

VectorQuantize.m • VectorQuantize is a MatLab code that segments any digital images based on predefined color references. • It is developed for the segmentation of fried potato images, to find a correlation with acrylamide level. • In its current form, it assumes that there are three typical regions in a fried potato image.

• These regions are tentatively determined!

Region 3 Region 1

Region 2

Image Segmentation • Read the image : Z=imread(‘Filename.jpg’) • Convert it into double form: Z=im2double(Z) • Before segmentation 4 different reference areas should be selected: – a=yellowish – b=brownish – c=dark brown – d= background

Reference Values a= yellowish

b= brownish

a=imcrop(Z);

b=imcrop(Z);

Select reference yellowish area

Select reference brownish area

d= background

c= dark brown

c=imcrop(Z);

d=imcrop(Z);

Select reference dark brown area

Select reference background area

Mean Values for Reference Areas •

Take double mean of the reference values: >>a=mean(a); (takes the mean value of columns) >>a=mean(a); (takes the mean value of raws)

Mean 1



Mean 2

Repeat it for all reference values.

Reference Matrix

Obtaining matrix of reference values >> u=[a(:,:,1) a(:,:,2) a(:,:,3);b(:,:,1) b(:,:,2) b(:,:,3); c(:,:,1) c(:,:,2) c(:,:,3); d(:,:,1) d(:,:,2) d(:,:,3)] >>u = 0.7523 0.5597 0.4966 0.8166

>> u=u' 0.5260 0.2568 0.1922 0.8063

0.0145 0.0069 0.0090 0.7713

u= 0.7523 0.5597 0.4966 0.8166 0.5260 0.2568 0.1922 0.8063 0.0145 0.0069 0.0090 0.7713

Segmented Images >> VectorQuantize(Z,u);

>> 0.7959 79.59 % brownish area

>> 0.1658 16.58 % brownish area

Practical Implications of Segmentation • VectorQuantize can practically be applied to the image any thermally processed food to monitor „browning‟. • The reference mean values are easily modified to find out any potential correlation with process contaminants. • Online application is possible for inspection purpose.

Perceptible Numerical Info !

50% BROWNING

Correlation with acrylamide Case Study : Potato Chips •

There are three different colors (equivalently three different kinds of pixels) in a fried potato image; – bright yellow (region 1) – brownish yellow (region 2) – darker brown (region 3)

Hint ! It was experimentally observed that Region-2 has a high probability of containing the highest concentration of acrylamide. Number of Pixels in Set  2 NA2  Number of Pixels in Entire Image

Region 3 Region 1 Region 2

Correlation with acrylamide Looking at the correlation between NA2 and acrylamide level 6000

Acylamide level, ng/g

5000

y = 5264.4x r = 0.989

4000 3000 2000 1000

FIGURE The correation between NA2 and acrylamide in potato chips

0 0

0.2

0.4

0.6 NA2

0.8

1

Further readings

Further readings