Image Processing and Related Fields - Stanford University

Image Processing and Related Fields!

http://en.wikipedia.org/wiki/File:CVoverview2.svg!

Bernd Girod: EE368 Digital Image Processing!

Introduction no. 22!

What is an image?!  

     

Ideally, we think of an image as a 2-dimensional light intensity function, f(x,y), where x and y are spatial coordinates, and f at (x,y) is related to the brightness or color of the image at that point.! In practice, most images are defined over a rectangle.! Continous in amplitude („continous-tone“)! Continous in space: no pixels!!


Point Operations no. 1!

Digital Images and Pixels!  

 

 

A digital image is the representation of a continuous image f (x,y) by a 2-d array of discrete samples f [x,y]. The amplitude of each sample is quantized to be represented by a finite number of bits. ! Each element of the 2-d array of samples is called a pixel or pel (from „picture element“) ! Think of pixels as point samples, without extent. !



Image Resolution!

200x200

100x100

50x50

25x25!

•  These images were produced by simply picking every n-th sample horizontally and vertically and replicating that value nxn times. ! •  We can do better! –  prefiltering before subsampling to avoid aliasing –  Smooth interpolation



Color Components! Monochrome image!

R[x,y] = G[x,y] = B[x,y]

Red R[x,y]

Green G[x,y]


Blue B[x,y] Point Operations no. 4!

Different numbers of gray levels! 256

32

16

4

2

„Contouring“

8



How many gray levels are required?!  

Contouring is most visible for a ramp!

32 levels! 64 levels! 128 levels! 256 levels!

 

Digital images typically are quantized to 256 gray levels. ! Bernd Girod: EE368 Digital Image Processing!


Brightness discrimination experiment!  

Can you see the circle?!

I + ΔI

125! 126! 128! 129! 130! 127!

 

I

Note: I is luminance, measured in cd m 2

Visibility threshold!

ΔI I ≈ KWeber ≈ 1…2% Bernd Girod: EE368 Digital Image Processing!

„Weber fraction“! „Weberʻs Law“! Point Operations no. 7!

Contrast with 8 bits according to Weberʻs Law!  

Assume that the luminance difference between two  successive representative levels is just at visibility threshold!

I max 255 = (1 + KWeber ) I min    

For! KWeber = 0.010.02 Typical display contrast!    

 

I max = 13156 I min

Cathode ray tube 100:1! Print on paper 10:1!

Suggests uniform perception in the log(I) domain („Fechnerʻs Law“)! Bernd Girod: EE368 Digital Image Processing!


Gamma characteristic!  

Cathode ray tubes (CRTs) are nonlinear! Luminance!

I! I ~ Uγ γ = 2.0 . . . 2.3!

Voltage U!  

Cameras contain γ-predistortion circuit!

U ~ I1 γ



log vs. γ-predistortion! U

U ~ log(I )

U ~ I1 γ Imax = 100 Imin

I

 

Similar enough for most practical applications! Bernd Girod: EE368 Digital Image Processing!


Photographic film! Hurter & Driffield curve (H&D curve)! for photographic negative!

Luminance!

slope -γ!

density d

2.0

shoulder!

I = I 0 ⋅10 − d = I 0 ⋅10 −( − γ log E +d0 )

„linear“ region!

1.0

toe!

= I 0 ⋅10 − d0 ⋅ E γ

log E

0

d0

E is exposure

 

γ measures film contrast!    

 

General purpose films: γ = -0.7 . . . -1.0! High-contrast films: γ = -1.5 . . . -10!

Lower speed films tend to have higher absolute γ!



Intensity scaling! Original image!

Scaled image!

f [ x, y ]

a ⋅ f [ x, y ]

Scaling in the γ-domain is equivalent to scaling in the linear luminance domain! γ

γ

I ~ ( a ⋅ f [ x, y ]) = a ⋅ ( f [ x, y ]) γ

. . . same effect as adjusting camera exposure time.!



Adjusting γ! Original image!

f [ x, y ]

γ increased by 50%!

γ

a ⋅ ( f [ x, y ])

with

γ = 1.5

. . . same effect as using a different photographic film . . .! Bernd Girod: EE368 Digital Image Processing!


Image Processing Examples! Face morphing!

Source: Yi-Wen Liu and Yu-Li Hsueh, EE368 class project, spring 2000.!



Gray level histograms!  

 

 

Distribution of gray-levels can be judged by measuring a  histogram:!  

For B-bit image, initialize 2B counters with 0 !

 

Loop over all pixels x,y!

 

When encountering gray level f [x,y]=i, increment counter #ι

Normalized histogram may be thought of as an  empirical probability distribution.! You can also use fewer, larger bins to trade off amplitude  resolution against sample size.!



#pixels!

Example histogram!

Cameraman! image! gray level!



#pixels!

Example histogram!

gray level!


Pout! image!


Histogram equalization!  

Idea: find a non-linear transformation!

g=T(f) !to be applied to each pixel of the input image f [x,y], such  that a uniform distribution of gray levels in the entire range  results for the output image g[x,y].

  Analyse ideal, continuous case first, assuming! 0 ≤ g ≤1   0

≤ f ≤ 1   T(f) is strictly monotonically increasing, hence, there exists! 0 ≤ g ≤1 f = T −1 ( g )  

Goal: pdf pg(g) = 1 over the range !0 ≤ g ≤ 1



Histogram equalization example! . . . after histogram equalization!

#pixels!

#pixels!

Original image Pout!

gray level!


gray level!


Histogram equalization example!

Original image Pout!


Pout! after histogram equalization! Point Operations no. 21!


#pixels!

#pixels!

Original image Cameraman!

gray level!


gray level!



Original image Cameraman!


Cameraman! after histogram equalization!



#pixels!

#pixels!

Original image Moon!

gray level!


gray level!



Original image Moon!


Moon! after histogram equalization!


Adaptive histogram equalization!  

 

Apply histogram equalization based on a histogram obtained from a portion of the image!

Sliding window approach: 

Tiling approach: 

different histogram (and mapping)  for every pixel!

subdivide into overlapping regions,  mitigate blocking effect by smooth blending  between neighboring tiles !

Must limit contrast expansion in flat regions of the image,  e.g. by clipping individual histogram values to a maximum! Bernd Girod: EE368 Digital Image Processing!


Adaptive histogram equalization!

Original!

Global histogram!


Tiling  8x8 histograms!

Tiling  32x32 histograms!


Adaptive histogram equalization!

Original image Tire!

Tire after   equalization of  global histogram!


Tire after   adaptive histogram equalization  8x8 tiles!


Point operations for combining images!    

   

Image averaging for noise reduction! Combination of different exposures for  high-dynamic range imaging! Image subtraction for change detection! Accurate alignment is always a requirement!



Image averaging for noise reduction! 1 image

2 images

4 images!

http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm!



High-dynamic range imaging! 16 exposures, one f-stop (2X) apart!

[Debevec, Malik, 1997] ! Bernd Girod: EE368 Digital Image Processing!

Combined image! Point Operations no. 33!

Where is the defect?!

Image A (no defect)!


Image B (w/ defect)!


Absolute difference between two images!

w/o alignment!


w/ alignment!


Digital subtraction angiography!

-!

+!

Contrast! enhancement!

http://www.isi.uu.nl/Research/Gallery/DSA/!



Gray-level thresholding! How can holes be filled?!

Original image! Peter f [x,y]!

Thresholded ! Peter m [x,y]


f [ x, y ] ⋅ m [ x, y ]

Thresholding no. 1!

Error probability for thresholding!

pdf!

Background!

Foreground!

error  probability!


Gray level!

Thresholding no. 2!

Optimal supervised thresholding! “MAP (Maximum A Posteriori) detector”! pdf!

Background!

Foreground!

error  probability!

Gray level!

If different outcomes are associated with different costs:   more general “Bayes minimimum risk detector”! Bernd Girod: EE368 Digital Image Processing!

Thresholding no. 3!

Unsupervised thresholding (cont.)!


Thresholding no. 6!

Unsupervised thresholding (cont.)!


Thresholding no. 7!

Chroma keying!    

Color is more powerful for pixel-wise segmentation: 3-d vs. 1-d space! Take picture in front of a blue screen (or green, or orange)!


Thresholding no. 8!

MAP detector in RGB-space!

Original image!

Skin color detector!

[S. Leahy, EE368 class project, 2003]!


Thresholding no. 11!

Region labeling and counting!  

How many bacteria in this picture?!

Original Bacteria image!    

after thresholding!

Which pixels belong to the same object (region labeling)?! How large is each object (region counting)?! Bernd Girod: EE368 Digital Image Processing!


Example: region labeling!



Hole filling as dual to small region removal! Mask with holes!


After NOT operation,! (background) region labeling,! small region removal,! and second NOT operation!


Sometimes, a global threshold does not work!

Original image! Bernd Girod: EE368 Digital Image Processing!

Thresholded with  Otsuʼs Method! Thresholding no. 20!

Locally adaptive thresholding!    

Slide a window over the image! For each window position, decide whether to do thresholding!    

 

 

Can use variance or other suitable criterion! Thresholding should not be performed in uniform areas!

Non-uniform areas: apply Otsuʼs method (based on local histogram) ! Uniform areas: classify the entire area as foreground or background based on mean value!



Locally adaptive thresholding (example)!

Original image!

Global threshold!


Local threshold!


Dilation!  

Binary dilation operator!

 

g ⎡⎣ x, y ⎤⎦ = OR ⎡⎣W f ⎡⎣ x, y ⎤⎦ ⎤⎦ := dilate f ,W Effects!

{

     

}

(

)

Expands the size of 1-valued objects! Smoothes object boundaries! Closes holes and gaps!

Original (178x178)!

dilation with ! 3x3 structuring element!


dilation with ! 7x7 structuring element!

Morphological image processing no. 3!

Example: blob separation/detection by erosion! Original! binary! image! circles!

Erosion! by 11x11 ! structuring! element!

Erosion ! by 21x21 ! structuring! element!

Erosion! by 27x27 ! structuring! element!



Example: counting coins!

Original

dilation

1 connected component

thresholded

thresholded after dilation

22 connected components

Courtesy: P. Salembier!



Morphological edge detector!

g-f


dilation g

(g-f) thresholded


Courtesy: P. Salembier!

original f

Example! n  n 

Image subsampling 2:1 horizontally and vertically! Small input image of size 8x8, output image size 4x4! ! # # # # Hx = Hy = # # # # # "

n 

1 0 0 0 0 0 0 0

0 0 1 0 0 0 0 0

0 0 0 0 1 0 0 0

0 0 0 0 0 0 1 0

$ & & & & & & & & & %

Nice for unified treatment, but NOT recommended for implementation! Bernd Girod: EE368 Digital Image Processing!

Image Filtering and Deconvolution no. 9!

Example (cont.)! n 

A somewhat better technique for 2:1 image size reduction! ! # # # # Hx = Hy = # # # # # "

n  n 

0.5 0 0 0 $ & 0.5 0 0 0 & 0 0.5 0 0 & 0 0.5 0 0 & & 0 0 0.5 0 & 0 0 0.5 0 & 0 0 0 0.5 & & 0 0 0 0.5 %

Can you figure out what this does to the image?! Why is this a better technique than the previous one?! Bernd Girod: EE368 Digital Image Processing!


Side by side comparison!

! # # # # Hx = Hy = # # # # # "

1 0 0 0 0 0 0 0

0 0 1 0 0 0 0 0

0 0 0 0 1 0 0 0

0 0 0 0 0 0 1 0

$ & & & & & & & & & %


! # # # # Hx = Hy = # # # # # "

0.5 0 0 0 $ & 0.5 0 0 0 & 0 0.5 0 0 & 0 0.5 0 0 & & 0 0 0.5 0 & 0 0 0.5 0 & 0 0 0 0.5 & & 0 0 0 0.5 %


Filtering Examples!

Original! Cameraman!

Cameraman blurred by convolution! Filter impulse response! ! 1 1 !1 1 1 $ # 1 1 !1 1 1 & & 1 # 1 1 1 1 1 [ ] # & 25 # # 1 1 " 1 1


1 1

1 1 && 1 1 %


Filtering Examples!


Cameraman blurred horizontally! Filter impulse response! ! 1 1 1 ![1] 1 1 5

(


)


Filtering Examples!


Cameraman blurred vertically! Filter impulse response! ! !1 $ # 1 & ! & 1# # [1] & 5# & 1 # & 1 " %



Filtering Examples!


Cameraman sharpened! Filter impulse response! ! " 0 !1 0 % 1$ ! !1 8 !1 ' 4 $$ # 0


[ ] !1

' 0 '&


Filtering Examples!


Cameraman sharpened! Filter impulse response! ! " 0 !1 0 % $ !1 !5 !1 ' [ ] $ $# 0


!1

' 0 '&


Why do we process images?!  

Acquire an image! –  Correct aperture and color balance –  Reconstruct image from projections

 

Prepare for display or printing! –  Adjust image size –  Halftoning

 

Facilitate picture storage and transmission! –  Efficiently store an image in a digital camera –  Send an image from Mars to Earth

 

Enhance and restore images! –  Remove scratches from an old movie –  Improve visibility of tumor in a radiograph

 

Extract information from images! –  Read the ZIP code on a letter –  Measure water pollution from aerial images


Introduction no. 2!

Image Processing Examples! Restoration of image from Hubble Space Telescope!

!

Source: IVPL Northwestern University, Chicago


Introduction no. 3!

Color balancing example!

Original Lena! Bernd Girod: EE368 Digital Image Processing!

Scale-by-max  color balancing! Color no. 27!

Image Processing Examples! Noise reduction!

Degraded image

Noise-reduced image! Source: Jungwon Lee, EE 368 class project, Spring 2000!


Introduction no. 5!

Image Processing Examples! Halftoning!


Introduction no. 7!

Wiener filtering example!

Image with motion blur!

restored by Wiener filtering! Source: http://www.cs.kun.nl/~ths/rt2/col/h5/5restoratieENG.html!



Image Processing Examples! Special Effects!

Photo!

Simulated   color pencils!

Simulated   oil painting!

source: Feng Xiao, EE368 class project, spring 2000.


Introduction no. 6!

Image Processing Examples! Pseudocolor enhancement for security screening!

Source: Gonzalez+Woods, Fig. 6.24!


Introduction no. 8!

Image Processing Examples! Extraction of settlement area from an aerial image!

source: INRIA, Sophia-Antipolis, France!


Introduction no. 9!

Image Segmentation!

http://cs.stanford.edu/group/roadrunner/stanley.html!



Image Processing Examples! Face Detection!

source: Henry Chang, Ulises Robles, EE368 class project, spring 2000.



Image Processing Examples! Face Detection!

source: Michael Bax, Chunlei Liu, and Ping Li, EE368 class project, spring 2003.



Image Processing Examples! FaceDetection! Blurring for Privacy Protection! Face



Image Processing Examples! Handwriting recognition!



Image Processing Examples! License Plate Recognition!

http://www.platerecognition.info/1103.htm! http://ezine.motorola.com/government?a=325!



Image Processing Examples! Biometrics: Fingerprint recognition!

FBIʼs   Integrated  Automated  Fingerprint  Identification  System! IAFIS!



Image Processing Examples! Biometrics: Iris recognition!

Source: J. Daugman, U. Cambridge!



Image Processing Examples! Mugshot retrieval!

Source: MIT Media Lab!



Spring 2006 Project:  Visual Code Marker Recognition!



Spring 2007 Project:  Painting Recognition!

1!

2!

3!

4!

5!

6!

7!

8!

9! Bernd Girod: EE368 Digital Image Processing!

10! Introduction no. 27!

Spring 2008 Project:  CD Cover Recognition!

