ccv.wordpress.fos.auckland.ac.nz ... to start with image sharpening for removing the blur ..... windows of the local operator contain original data as well as.
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Point, Local, and Global Operators1 Lecture 05
See Section 2.1 in Reinhard Klette: Concise Computer Vision Springer-Verlag, London, 2014 ccv.wordpress.fos.auckland.ac.nz
1
See last slide for copyright information. 1 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Agenda
1
Image Processing
2
Gradation Functions
3
Local Operators
4
Point, Local, and Global Operators
2 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Image Processing Mapping of an image into another image Often as image preprocessing, i.e. prior to image analysis
Here: Input image mapped into an image showing high-frequency information only 3 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Example: Noise Removal Noise removal is one of the reasons for mapping a given image into a “better” image Noise is unwanted data For example, challenges of outdoor image recording: Difficulties with lighting, motion blur, or sudden changes in scenes Figure on next page shows video frames recorded in a car (for vision-based driver-assistance), illustrating three examples of noise Definition: Ego-vehicle = the vehicle where “the system” is operating in (see stereo or night vision systems in modern cars, or lane departure warning and distance measurement in modern trucks)
4 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Illustration of Noise
5 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
2nd and 3rd Noise Example (1) Top row: Pair of stereo images taken time-synchronized but of different brightness; see the shown gray-level histograms Goal: Transform images such that resulting images are “as taken at uniform illumination” (2) Bottom left: Blurring caused by rain and obstruction of view by a wiper Goal: Reduce impacts of rain and wiper when analysing this image (e.g. when calculating distance to car in front of the ego-vehicle); we could try to start with image sharpening for removing the blur (3) Bottom right: Gaussian noise in a scene recorded at night Goal: Remove the Gaussian noise, enhance the image contrast
6 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Agenda
1
Image Processing
2
Gradation Functions
3
Local Operators
4
Point, Local, and Global Operators
7 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Point Operators Time-efficient methods for image processing Gradation Functions Transform image I into image Inew of the same size: Map value u at pixel location p in I by gradation function g onto value v = g (u) at the same pixel location p in Inew Change only depends on value u at location p, thus a point operator defined by gradation function g
8 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Which Gradation Function? Which function g was used to map the left into the right image?
A historic photo of events in April 1960 at Korea University, Seoul, leading to an uprising against the autocratic ruler Syngman Rhee of South Korea. 9 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Histogram Transforms Histogram transforms are examples of point operators Histogram Transform If the goal is that Inew satisfies some properties defined in terms of its histogram then we speak about a histogram transform Example: Histogram Equalization Goal: Transform I such that all gray levels appear equally often in Inew Ideal goal: For all u ∈ {0, 1, . . . , Gmax } HInew (u) =
|Ω| Ncols Nrows = = const Gmax + 1 Gmax + 1
Unfortunately: Not possible in general due to having digital images (and not continuous functions) 10 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
The Raging Bull Example
Left: Input image RagingBull with histogram Right: The same image after histogram equalization Example: A binary image would remain binary after histogram equalisation 11 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Histogram Equalization: How To Do Given: Ncols × Nrows scalar image I with absolute frequencies HI (u), 0 ≤ u ≤ Gmax Transform I into Inew of the same size by applying a gradation function Use the following gradation function g g (u) = cI (u) · Gmax where cI is the relative cumulative frequency function Recall: What exactly is the definition of cI ? Figure on previous page illustrates this histogram-equalization transform 12 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
General Histogram Transform Goal: A particular distribution of scalar values described by a histogram 1
2
3
The given histogram specifies a relative cumulative histogram function, and this is all what we need to define a gradation function g1 for mapping (assumed) data, having this intended histogram, into an image having an (approximately) equalised histogram For given image I , calculate graduation function g2 for histogram equalisation Gradation function g (u) = g1−1 (g2 (u)) maps I into an image which has (approximately) the intended histogram
Note: The use of the inverse g1−1 assumes that we have a one-one function g1 . How to do if this is not the case? 13 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Linear Scaling Not a histogram transform, but another example for a point operator Image I has positive histogram values in interval [umin , umax ] Goal: Map values used in I linearly onto the whole scale [0, Gmax ] umin = min {I (x, y ) : (x, y ) ∈ Ω} umax = max{I (x, y ) : (x, y ) ∈ Ω} Definition of gradation function: a = −umin
and b =
Gmax umax − umin
g (u) = b(u + a) Results: Pixels having value umin in I now have value 0 in Inew Pixels having value umax in I now have value Gmax in Inew 14 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Linear Scaling: Graph Graph of gradation function for linear scaling Gmax
g(u)
0
u 0
umin
umax Gmax
Defined by being incident with points (umin , 0) and (umax , Gmax ): Derive the formula as given on the previous page 15 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Linear Scaling: Example Left: umin = 12 and umax = 195
Right: umin = 0 and umax = 255
16 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Conditional Scaling Not a histogram transform, but another example for a point operator Goal: Map image J into image Jnew such that Jnew has the same mean and variance as image I Definition of gradation function: σI − µI σJ g (u) = b(u + a) a = µJ ·
and b =
σJ σI
Calculation of Jnew : Replace value u at p in J by v = g (u) Result: µJnew = µI and σJnew = σI Might solve noise issue (1) on Page 6 if brightness difference between both images is uniformly defined 17 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Conditional Scaling: Example Image I
Image J
Conditional scaling of J
Equalized J 18 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Conditional Scaling: Synthetic Example Image I
Image J
Conditional scaling of J
Equalized J
19 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Agenda
1
Image Processing
2
Gradation Functions
3
Local Operators
4
Point, Local, and Global Operators
20 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
General Concept of a Local Operator
Map input image I into a new image J: 1
Given: Ncols × Nrows image I
2
Sliding window Wp of size (2k + 1) × (2k + 1)
3
Reference point p at the center of the window
4
Reference point p moves into all possible pixel locations of I
5
At these locations we perform a local operation
6
Result of the operation defines new value at p
21 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Moves of Reference Point p of Window Wp through I Defined by a scan; upper left shows default scan:
22 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Two Examples: Local Mean and Maximum Example: Local operation is mean J(p) = µWp (I ) ; for p = (x, y ) we have µWp (I ) =
+k X +k X 1 · I (x + i, y + j) (2k + 1)2 i=−k j=−k
Example: Local operation is the maximum; for p = (x, y ) we have J(p) = max{I (x + i, y + j) : −k ≤ i ≤ k ∧ −k ≤ j ≤ k}
Maximum in 41 × 41 input window (left) copied into p on the right What happens if input window (i.e. p) moves one pixel to the right? 23 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Border-pixel Strategy Needed for windows centered at p and not completely contained in I
There is no general rule: Define meaningful operation depending on the given local operation
24 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Example: Dilation & Erosion of Mathematical Morphology
Top, left: Original image. Right: Local maximum for k = 3 (dilation of bright pixels). Bottom, left: Local minimum for k = 5 (erosion of bright pixels). Right: Local operator using 3 × 3 filter kernel as on Page 27 25 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Linear Operators and Convolution Linear local operators are a class of local operators A linear local operator is defined by a convolution of I with a filter kernel Reference point p = (x, y ), weights wi,j ∈ R, scaling factor S > 0: +k +k 1 X X J(p) = I ∗ W (p) = wi,j · I (x + i, y + j) S i=−k j=−k
Array of (2k + 1) × (2k + 1) weights and S define the filter kernel
Example: All wi,j > 0 and S =
+k P
+k P
wi,j
i=−k j=−k
26 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Common Way for Filter Kernel Representation
Left: General representation for a 3 × 3 filter kernel Middle: Filter kernel for approximating derivation in x-direction, S = 1 Right: Filter kernel of a 3 × 3 box filter, S = 9 Box Filter Local mean on Page 23 is a linear local operator known as box filter All weights equal to 1, and S = (2k + 1)2 is the sum of all weights
27 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Local Operators: Summary 1
Operations are limited to windows, typically of size (2k + 1) × (2k + 1)
2
Window moves through the image following a selected scan order
3
Operation in the window is the same at all locations
4
Pixels close to the border of the image: case by case decisions
5
Results are either replacing values at reference points in input image I (sequential local operator); new values propagate like a “wave” over original values; windows of the local operator contain original data as well as already-processed pixel values 2 written into a second array, leaving the original image unaltered this way (parallel local operator); this kind of a local operator can be implemented on parallel hardware 1
28 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Agenda
1
Image Processing
2
Gradation Functions
3
Local Operators
4
Point, Local, and Global Operators
29 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Point, Local, or Global Operator Cases for (2k + 1) × (2k + 1) window size: (1) k = 0, i.e. window is just a single pixel: A point operator Example: Operators defined by a gradation function (2) k such that whole image is covered by the window: A global operator Examples: 2D Fourier transform of an image, e.g. I(0, 0) is the mean of the whole image, or any result of 2D Fourier filtering (3) If not global operator then a local operator (including point operators) Examples: Local mean (box filter), local maximum 30 / 31
Image Processing
Gradation Functions
Local Operators
Point, Local, and Global Operators
Copyright Information
This slide show was prepared by Reinhard Klette with kind permission from Springer Science+Business Media B.V. The slide show can be used freely for presentations. However, all the material is copyrighted. R. Klette. Concise Computer Vision. c
Springer-Verlag, London, 2014. In case of citation: just cite the book, that’s fine.
31 / 31
