Edge-preserving smoothing algorithms attempt to reduce noise and ... 6. Fall 2005. Correction and Calibration. 11. Selective Median Filter (cont.) horizontal.
Correction and Calibration Reading: Chapter 7, 8.1–8.3
ECE/OPTI 531 – Image Processing Lab for Remote Sensing
Fall 2005
Preprocessing • Required for certain sensor characteristics and systematic defects • Includes: – noise reduction – radiometric calibration – distortion correction
• Usually performed by the data provider, as requested by the user (see Data Systems)
Correction and Calibration
2
Fall 2005
1
Correction and Calibration • Noise Reduction • Radiometric Calibration • Distortion Correction
Correction and Calibration
3
Fall 2005
Noise Reduction • Requires knowledge of noise characteristics • Since noise is usually random and unpredictable, we have to estimate its characteristics from noisy images • Global Noise – Random DN variation at every pixel – LPFs will reduce this noise, but also smooth image – Edge-preserving smoothing algorithms attempt to reduce noise and preserve signal
Correction and Calibration
4
Fall 2005
2
Sigma Filter • Average moving window pixels that are within a threshold difference from the DN of the center pixel, sigma filter near edges and lines edge feature
line feature
5 x 5 window: only the green
row m, column n
c
pixels are
c
averaged for the output pixel c
row m, column n+1
c
c
c
row m, column n+2
Correction and Calibration
5
Fall 2005
Nagao-Matsuyama Filter • Calculate the variance of 9 subwindows within a 5×5 moving window • Output pixel is the mean of the subwindow with the lowest variance c
c
c
c
c
c
c
c
c
Correction and Calibration
6
Fall 2005
3
Example: SAR Noise Reduction
original
Correction and Calibration
5 x 5 LPF
5 x 5 sigma (k=2)
Nagao-Matsuyama
7
Fall 2005
Noise Reduction (cont.) • Local Noise – Individual bad pixels or lines – Often DN = 0 (“pepper”), 255 (“salt”) or 0 and 255 (“salt and pepper”) – Median filter reduces local noise because it is insensitive to outliers (Chapter 6) • For bad lines, set filter window vertical
Correction and Calibration
8
Fall 2005
4
Example: Bad-line Removal horizontal image details also removed
after 3 x 1 median filter
single noisy partial scanline (Landsat MSS)
Correction and Calibration
9
Fall 2005
Selective Median Filter • Detection by spatial correlation – Use global spatial correlation statistics to limit median filter
CDF of left image 1.05 1 2% 1% 0.5%
0.95
CDF
0.9 0.85 0.8 0.75
0.7
squared-difference image !ij between each pixel and its immediate neighbor in the preceding line
Correction and Calibration
10
0.65 0
0.02
0.04
0.06 !
0.08
0.1
0.12
Fall 2005
5
Selective Median Filter (cont.)
horizontal image details preserved
98% threshold mask for median filter
Correction and Calibration
after 3 x 1 selective median filter
11
Fall 2005
PCT Component Filter • Detection by spectral correlation – Use PCT to isolate spectrally-uncorrelated noise into higher order PCs – Remove noise and do inverse PCT TM2
TM3
TM4
PC 1
PC2
PC 3 noise from TM3
Correction and Calibration
12
Fall 2005
6
Periodic (Coherent) Noise • Repetitive pattern, either consistent throughout the image (global) or variable (local) • Destriping – Striping is caused by unequal detector gains and offsets in whiskbroom and pushbroom scanners – Destripe before geometric processing – Global, linear detector matching • Adjust pixel DNs from each detector i to yield the same mean and standard deviation over the whole image
– Nonlinear detector matching • Adjust each detector to yield the same histogram over the whole image (CDF reference stretch)
Correction and Calibration
13
Fall 2005
Periodic Noise (cont.) • Spatial filtering approaches
data flow for amplitude Fourier filtering noisy image
– Detector striping causes characteristic “spikes” in the Fourier transform of the image – For line striping, the spike is vertical along the v-axis of the Fourier spectrum – A “notch” amplitude filter (removes selected frequency components) can reduce striping without degrading image Correction and Calibration
FT
phase spectrum
amplitude spectrum design filter FT -1
cleaned image
14
Fall 2005
7
Destriping Examples
noisy image
original spectrum
striping notch-filter
removed noise
striping and within-line-filter
removed noise
Correction and Calibration
15
Fall 2005
Automatic Destriping • “Automatic” periodic noise filter design – Attempt to avoid manual noise identification in spectrum – Requires two thresholds
Automatic Periodic Noise Filter 1. 2. 3. 4.
HPF spectrum
noise filter
cleaned image
removed noise
Apply “soft” (Gaussian) high-pass filter to noisy image to remove image components Threshold HPF-filtered spectrum to isolate noise frequency components Convert thresholded spectrum to 0 (noise) and 1 (nonnoise) to create noise amplitude notch filter Apply filter to noisy image
Correction and Calibration
16
Fall 2005
8
Debanding • Use combination of LPFs and HPFs (Crippen, 1989) noisy image
* * *
1 row x 101 column LPF 33 row x 1 column HPF 1 row x 31 column LPF
!
cleaned image
Correction and Calibration
17
Fall 2005
Example: TM Debanding
original
101 column LP
33 row HP
water-masked
31 column LP
water-masked
Correction and Calibration
18
filtered
water-masked
Fall 2005
9
Correction and Calibration • Noise Reduction • Radiometric Calibration • Distortion Correction
Correction and Calibration
19
Fall 2005
Radiometric Calibration • Many remote sensing applications require calibrated data – comparison of geophysical variables over time – comparison of geophysical variables derived from different sensors – generation of geophysical variables for input to climate models
• Extent of calibration depends on the desired geophysical parameter and available resources for calibration Correction and Calibration
DN
sensor calibration
• cal_gain and cal_offset coefficients
at-sensor radiance • image measurements • ground measurements
atmospheric correction
surface radiance
• atmospheric models • view path atmospheric transmittance • view path atmospheric radiance • solar exo-atmospheric spectral irradiance
solar and topographic correction
• solar path atmospheric transmittance • down-scattered radiance • DEM
surface reflectance
20
Fall 2005
10
Sensor Calibration • Calibrate sensor gain and offset in each band (and possibly for each detector) to get at-sensor radiance • Use pre-launch or post-launch measured gains and offsets pre-flight TM calibration coefficients
Correction and Calibration
21
Fall 2005
Atmospheric Correction • Estimate atmospheric path radiance and viewpath transmittance to obtain at-surface radiance (sometimes called surface-leaving radiance) At-sensor:
– Solve for at-surface radiance Earth-surface:
– In terms of calibrated at-sensor satellite data Earth-surface: Correction and Calibration
22
Fall 2005
11
In-Scene Methods • Path radiance can be estimated with the Dark Object Subtraction (DOS) technique – In-scene method assumes dark objects have zero reflectance, and any measured radiance is attributed to atmospheric path radiance only – Subject to error if object has even very low reflectance – View-path atmospheric transmittance is not corrected by DOS Dark-Object Subtraction 1. 2. 3. 4. 5.
Identify “dark object” in the scene Estimate lowest DN of object, Assume DN values (calibrated to at-sensor radiance) within the dark object assumed to be due only to atmospheric path radiance Subtract from all pixels in band b
Correction and Calibration
23
Fall 2005
Atmospheric Modeling • DOS can be improved by incorporating atmospheric models – Reduces sensitivity to dark object characteristics – For example, coarse atmospheric characterization (Chavez, 1989)
– Can also fit DN0b with λ-K function to determine K, and use fitted model instead of DN0b
• View-path atmospheric transmittance can be estimated using atmospheric models, such as MODTRAN Correction and Calibration
24
Fall 2005
12
Atmospheric Modeling (cont.) • Atmospheric modeling requires knowledge of many parameters – Imaging geometry – Atmospheric profile and aerosol model view path, nadir view path, view angle = 40 degrees 1
transmittance
0.8 0.6 0.4 0.2 0 0.4 Correction and Calibration
0.8
1.2 1.6 wavelength (µm)
2
2.4
25
Fall 2005
Solar Geometry and Topography • Further calibration to reflectance requires 3 more parameters
– solar path atmospheric transmittance (from model or measurements) – exo-atmospheric solar spectral irradiance (known) – incident angle (from DEM)
Correction and Calibration
26
Fall 2005
13
Example: MSI Calibration • Partial calibration with correction for sensor gains and offsets and DOS
note the
band 1
strong
(blue)
correction for Rayleigh scattering
2 (green)
3 (red)
at-sensor radiance
DN
Correction and Calibration
scene radiance
27
Fall 2005
Example: Partial Calibration band
Rayleigh
1
scattering
(blue)
correction 2 (green)
3 (red)
4 (NIR)
5
low solar
(SWIR)
irradiance 7 (SWIR) DN
Correction and Calibration
at-sensor radiance scene radiance
28
Fall 2005
14
Hyperspectral Normalization • Calibration of hyperspectral imagery is particularly important because: – High sensitivity to narrow atmospheric absorption features or the edges of broader spectral features – Spectral band shift in operating sensor – Need for precise absorption band-depth measurements – Computational issues
• A number of “normalization” techniques that use scene models have been developed to partially calibrate hyperspectral data
Correction and Calibration
29
Fall 2005
Normalization (cont.)
effectiveness of various normalization techniques for calibration
Correction and Calibration
30
Fall 2005
15
Example: HSI Normalization • Radiance normalized by flat field and IARR techniques
relative reflectance
AVIRIS at-sensor radiance
250
alunite (radiance) buddingtonite (radiance) kaolinite (radiance)
200
0.8 alunite (IARR) alunite (flat field)
0.7 0.6 0.5 0.4 2000
2050
2100
2150 2200 2250 wavelength (nm)
2300
2350
2400
1.4
150
1.3 relative reflectance
at-sensor radiance (W-m- 2-sr- 1-µ m- 1)
1 0.9
100
50
1.2 1.1 1
buddingtonite (IARR) buddingtonite (flat field)
0.9 0.8 0.7
0 2000
2050
2100
2150 2200 2250 wavelength (nm)
2300
2350
0.6 2000
2400
2050
2100
1.3
mineral
2300
2350
2400
1.2 relative reflectance
discrimination confused by solar background and topographic
1.1 1 0.9
kaolinite (IARR) kaolinite (flat field)
0.8 0.7 0.6 0.5 2000
shading
Correction and Calibration
2150 2200 2250 wavelength (nm)
2050
2100
2150 2200 2250 wavelength (nm)
2300
31
2350
2400
Fall 2005
Example (cont.) • Radiance normalized by flat fielding, compared to reflectance data 1 Alunite GDS84
relative reflectance
0.8
similarity implies
alunite (flat field)
we can use
0.6
reflectance library 0.4
data for image class identification
0.2
0 2000
2050
2100
2150 2200 2250 wavelength (nm)
1
2300
2350
2400
0.8 0.7 relative reflectance
relative reflectance
0.8
0.6 0.4 Buddingtonite GDS85
0.2
buddingtonite (flat field)
0 2000
2050
2100
2150 2200 2250 wavelength (nm)
Correction and Calibration
2300
2350
0.6 0.5 0.4 0.3 0.2
Kaolinite CM9 kaolinite (flat field)
0.1 0 2000
2400
32
2050
2100
2150 2200 2250 wavelength (nm)
2300
2350
2400
Fall 2005
16
Example (cont.) • Radiance normalized by continuum removal 1
140 0.8 120 0.6
100 80
0.4 alunite (radiance)
60
alunite continuum 40
2050
2100
2150
0.8 160 140 120
0.4 buddingtonite (radiance) buddingtonite continuum
0.2
buddingtonite/continuum
2050
2100
2150
2200 2250 wavelength (nm)
Correction and Calibration
2300
2350
2350
0 2400
0 2400
1
0.8
200
0.6 150 0.4 kaolinite (radiance) 100
kaolinite continuum
relative value
0.6
2300
250 at-sensor radiance (W-m- 2-sr- 1-µ m- 1)
1
180
60 2000
background
0.2
2200 2250 wavelength (nm)
relative value
at-sensor radiance (W-m- 2-sr- 1-µ m- 1)
200
80
removes solar
alunite/continuum
20 2000
100
relative value
at-sensor radiance (W-m- 2-sr- 1-µ m- 1)
160
0.2
kaolinite/continuum 50 2000
2050
2100
33
2150
2200 2250 wavelength (nm)
2300
0 2400
2350
Fall 2005
Correction and Calibration • Noise Reduction • Radiometric Calibration • Distortion Correction
Correction and Calibration
34
Fall 2005
17
Distortion Correction • Applications – correct system distortions – register multiple images – register image to map
• Three components to warping process – selection of suitable mathematical distortion model(s) – coordinate transformation – resampling (interpolation) Distortion Correction Implementation 1. 2. 3.
Create an empty output image (which is in the reference coordinate system) Step through the integer reference coordinates, one at a time, and calculate the coordinates in the distorted image (x,y) by Eq. 7-27 Estimate the pixel value to insert in the output image at (xref,yref) from the original image at (x,y) (resampling)
Correction and Calibration
35
Fall 2005
Definitions • Registration: The alignment of one image to another image of the same area. • Rectification: The alignment of an image to a map so that the image is planimetric, just like the map. Also known as georeferencing. • Geocoding: A special case of rectification that includes scaling to a uniform, standard pixel GSI. • Orthorectification: Correction of the image, pixel-by-pixel, for topographic distortion.
Correction and Calibration
36
Fall 2005
18
Example: TM Rectification way Speed
Original (Tucson, AZ)
fill Speedway
Rectified
Correction and Calibration
37
Fall 2005
Coordinate Transformation • Coordinates are mapped from the reference frame to the distorted image frame reference frame
image frame column
column
(x,y) (xref,yref)
row
row
– “Backwards” mapping (xref,yref) —> (x,y) avoids “holes” or overlaying of multiple pixels in the processed image Correction and Calibration
38
Fall 2005
19
Polynomial Distortion Model • Generic model useful for registration, rectification and geocoding • Known as a “rubber sheet stretch” • Relates distorted coordinate system (x,y) to the reference coordinate system (xref,yref)
• For example, a quadratic polynomial is written as:
Correction and Calibration
39
Fall 2005
Distortion Types • The coefficients in the polynomial can be associated with particular types of distortion
original
shift
scale in x
shear
y-dependent scale in x
quadratic scale in x
rotation Correction and Calibration
40
quadratic Fall 2005
20
Affine Transformation • A linear polynomial (number of terms K = 3) – Special case that can include: • • • •
shift scale shear rotation
– In vector-matrix notation
Correction and Calibration
41
Fall 2005
Piecewise Polynomial Distortion • For severely-distorted images that can’t be modeled with a single, global polynomial of reasonable order
piecewise quadrilateral warping
b
B
a
c
D
d
Example airborne scanner image
C
A
piecewise triangle warping
b
affine correction
a
B c
d
reference frame
C
A D
distorted frame
quadratic correction Correction and Calibration
42
Fall 2005
21
Application to Wide FOV Sensors • Comparison of actual distortion to global polynomial model – accurate for small FOV sensors such as Landsat, but not for large FOV scanners such as AVHRR ± 5° FOV
± 50° FOV
true distortion quadratic fit
true distortion quadratic fit
% error
1.01
0.001
15
3.5
10
3
5
2.5
0
1
2
-5
% error
% error
scale relative to nadir
0
scale relative to nadir
1.005
% error
4
-0.001
0.995
-0.002 -6
-4
-2
0 2 scale angle (degrees)
Correction and Calibration
4
1.5
-10
1
-15
0.5
6
-20 -60
-40
-20
0 20 scan angle (degrees)
43
40
60
Fall 2005
Ground Control Points (GCPs) • Use to control the polynomial, i.e. to determine its coefficients – Characteristics: • • • •
high contrast in all images of interest small feature size unchanging over time all are at the same elevation (unless topographic relief is being specifically addressed)
– Often located by visual examination, but can be automated to varying degree, depending on the problem
Correction and Calibration
44
Fall 2005
22
Automatic GCP Location • Use image “chips” – Small segments that contain one easily identified and well-located GCP
• Normalized cross-correlation between template chip T (reference) and search area S in image to be registered
– normalization adjusts for changes in mean DN within area Correction and Calibration
45
Fall 2005
Area Correlation Cross-correlation of one area
search chip target chip
target area T two relative shift positions
search area S
i,j = 0,0
i,j = 0,1
Chip layout over full scene search chip
target chip
T1 T2
+
+
46
S2
+
+ T3
+
reference image Correction and Calibration
S1
S3
+
distorted image Fall 2005
23
Finding Polynomial Coefficients • Set up system of simultaneous equations using GCPs and solve for polynomial coefficients • Example with quadratic polynomial (number of terms K = 6) – Given M pairs of GCPs – For each GCP pair, m, create two equations
– Then, for all M GCP pairs, in vector-matrix notation
Correction and Calibration
47
Fall 2005
Solving for Coefficients • So, for each set of GCP x- and y-coordinate pairs, we can write a linear system – Determined case (M = K, just enough GCPs) solution: M = K:
solution:
• Exact solution which passes through GCPs, i.e. they are mapped exactly
– Overdetermined case (M > K, more than enough GCPs): M
≥ K:
solution:
• is called the pseudoinverse of W • solution results in least-squares minimum error at GCPs:
Correction and Calibration
48
Fall 2005
24
Example: 1-D Curve Fitting linear 90
y = 13.228 + 0.82085x
8
R= 0.93425
70
60
60
M0
24.141
M1
0.011693
M2
0.0095018
R
0.95128
y
80
70
y
80
50
50
40
40
30
30
20
20 0
10
20
30
40
50
60
70
80
0
10
20
30
40
x
third order 8
M0
90
70
8
-0.14393
M3
0.0011585
R
0.98745
M0
90
5.6886
M2
8
70
-2.5272 0.21642
M3
-0.0049359
M4
3.4942e-05
R
0.99999
80 70
50
40
40
40
30
30
30
20
20 50
60
70
33.718
M1
-3.0378
M2
0.2515
M3
-0.0059674
M4
4.8406e-05
M5
-6.4246e-08
R
1
60
50
40
9
M0
y
M2
y
y
90
50
30
80
fifth order
31.173
M1
80
60
20
70
Y = M0 + M1*x + ... M8*x + M9*x
9
Y = M0 + M1*x + ... M8*x + M9*x
60
10
60
fourth order
9
-24.455
M1
80
50
x
Y = M0 + M1*x + ... M8*x + M9*x
0
second order
9
Y = M0 + M1*x + ... M8*x + M9*x 90
80
20 0
10
20
30
x
40
50
60
70
80
0
10
20
30
x
Correction and Calibration
40
50
60
70
80
x
49
Fall 2005
Example: Registration • Register aerial photo to map in an urban area • Locate 6 GCPs in each (black crosses) • Locate 4 Ground Points (GPs) in each (red circles) – not used for control, only for testing
GCP1 GCP6
aerial photo (x,y)
GCP2 GCP5
GCP3
GCP4
GCP1 GCP6 GCP2 GCP5
GCP3
Correction and Calibration
50
map reference (xref,yref)
GCP4
Fall 2005
25
GCP Error
Error at GCPs and Ground Points (GPs) as function of polynomial order 5
RMS deviation (pixels)
4
GCPs GPs
3
2
1
0 3
4
5
6
number of terms
Mapping of GCPs
Correction and Calibration
51
Fall 2005
GCP Refinement • Analyze GCP error and remove outliers
five GCPs, with outlier deleted 1.5
1
1 deviation in y (pixels)
deviation in y (pixels)
original six GCPs 1.5
0.5 0 -0.5 -1 -1.5 -1.5
0.5 0 -0.5 -1 -1.5
-1
-0.5 0 0.5 deviation in x (pixels)
Correction and Calibration
1
-1.5
1.5
52
-1
-0.5 0 0.5 deviation in x (pixels)
1
1.5
Fall 2005
26
Final Registration Result
Correction and Calibration
53
Fall 2005
Resampling • The (x,y) coordinates calculated by are generally between the integer pixel coordinates of the array • Therefore, must estimate (interpolate or resample) a new pixel at the (x,y) location reference frame
image frame
column
column
DNr(x,y) DN(xref,yref)
row
row Correction and Calibration
54
Fall 2005
27
Resampling (cont.) resampling distances
• Pixels are resampled using a weighted-average of the neighboring pixels • Common weighting functions:
INT(x,y) !x 1 - !x
– nearest-neighbor: fast, but discontinuous
!y
A
E
DNr (x,y)
1 - !y C
– bilinear: slower, but continuous
B
F
D
1.2 1
• Implement 2-D bilinear resampling as two successive 1-D resamplings
n-n linear
value
0.8 0.6
– resample E between A and B – resample F between C and D – resample DN(x,y) between E and F
0.4 0.2 0 -0.2 -3
-2
-1
0 ! (pixels)
1
2
3
Correction and Calibration
55
Fall 2005
Resampling (cont). – bicubic: slowest, but results in sharpest image • piecewise polynomial; special case of Parametric Cubic Convolution (PCC)
where Δ is the distance from (x,y) to the grid points in 1-D • “standard” bicubic is
α = -1; superior bicubic is α = -0.5
• High-boost filter characteristics; amount of boost depends on amplitude of side-lobe, which is proportional to α 1.2 1
PCC, " = -1 PCC, " = -0.5 sinc
0.8
value
0.6 0.4 0.2 0 -0.2 -0.4 -4
-3
-2
-1
0
1
2
3
4
! (pixels)
Correction and Calibration
56
Fall 2005
28
PCC Resampling • PCC resampling procedure – resample along each row, A-D, E-H, I-L, M-P etc. – resample along new column Q-T
A
B
E
F
Q
C
D
G
H
!x 1 - !x !y
R
DNr (x,y)
1 - !y I
J
S
K
L
M
N
T
O
P
Correction and Calibration
57
Fall 2005
Example: Magnification (“Zoom”) 1× nearest-neighbor
bilinear
2×
3×
Correction and Calibration
58
Fall 2005
29
Example: Rectification
nearest-neighbor
bilinear
PCC (α = -0.5) Correction and Calibration
PCC (α = -1.0) 59
Fall 2005
Interpolation Quality bilinear
nearest-neighbor
resampled image surface plots (4x zoom)
parametric cubic convolution (α = -0.5)
Correction and Calibration
60
Fall 2005
30
Resampling Differences • Difference images between different resampling functions
nearest-neighbor – bilinear
bilinear – PCC
– polynomial distortion model affects global geometric accuracy – resampling function affects local radiometric accuracy
Correction and Calibration
61
Fall 2005
31
