Image Processing

Image Processing Prof. Christian Bauckhage

outline lecture 02

image acquisition and digital photography

representations of digital images (part 1)

summary

image acquisition and digital photography

general principle

visualize, record, and store light

light

electromagnetic radiation where energy propagates in form of electromagnetic waves a particle or quantum phenomenon where photons carry the electromagnetic force

speed of light

we have c = νλ where ν ⇔ frequency λ ⇔ wavelength

energy of light

we have E = hν = h

c λ

where h ⇔ Planck’s constant 6.6260689633 × 10−34 Js ⇒ high frequent / small wavelength light carries more energy

λ

electromagnetic spectrum

1m 100

1mm 10−1

10−2

10−3

1nm

1µm 10−4

10−5

10−6

10−7

10−8

10−9 wavelength [m]

108

109

1010

1011

1012

1013

1014

1015

1016

1017

1018 frequency [Hz]

VHF micro waves

infrared

ultraviolet

X−rays name

visible spectrum

visualizing light reflected light absorbed light emitted light

photographic images, scanned images, . . . X-ray images, . . . infrared images, . . .

active imaging

time-of-flight cameras record echos of emitted infrared light (similar to Radar)

consumer product

depth image

hyperspectral imaging

record light from across several bands of the em-spectrum

hyperscpectral image

spectral responses

recording technology

digital (color-)photography relies on CCD cameras


digital (color-)photography relies on CCD cameras CCD = charged-coupled device = semiconductor device consisting of thousands of photosensitive elements


digital (color-)photography relies on CCD cameras CCD = charged-coupled device = semiconductor device consisting of thousands of photosensitive elements

photo sensors of modern consumer cameras contain at least 2592 × 1944 elements (≈ 3M-pixels)

note

in general, CCD elements are not perfectly square

CCD chip

schematic CCD matrix

intensity images

rays of light incident on the camera lens hit the CCD matrix light intensity at each element of the matrix is recorded as a number

color images (1)

split light into basic colors (typically red, green, blue) use a trichroic prism assembly and record each basic color on a separate CCD array – this is expensive

trichroic prism

three separate CCD arrays

color images (2)

superimpose a mosaic of color filters (a Bayer filter) over a single CCD array – this is (much!) cheaper

Bayer filter

single array

recording technology in detail

photons incident on a CCD element generate a charge and the more light there is, the more electrons are freed


photons incident on a CCD element generate a charge and the more light there is, the more electrons are freed an electric field gathers electrons into packets


photons incident on a CCD element generate a charge and the more light there is, the more electrons are freed an electric field gathers electrons into packets packets are read and quantized to gray scale values


photons incident on a CCD element generate a charge and the more light there is, the more electrons are freed an electric field gathers electrons into packets packets are read and quantized to gray scale values depending on camera quality, a single CCD element stores 100,000 to 350,000 electrons before saturating


photons incident on a CCD element generate a charge and the more light there is, the more electrons are freed an electric field gathers electrons into packets packets are read and quantized to gray scale values depending on camera quality, a single CCD element stores 100,000 to 350,000 electrons before saturating representing >100,000 gray values would require a storage capacity of at least 17 bit per CCD element


photons incident on a CCD element generate a charge and the more light there is, the more electrons are freed an electric field gathers electrons into packets packets are read and quantized to gray scale values depending on camera quality, a single CCD element stores 100,000 to 350,000 electrons before saturating representing >100,000 gray values would require a storage capacity of at least 17 bit per CCD element this is a) too memory intensive b) unnecessary because of the physiology of the human visual system (details later in the course)

intensity quantization

intensity resolution is reduced using a certain mapping 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15

1

2

3

4

5



2

3

4

5

6

7

8

9 10 11 12 13 14 15

1

2

3

4

5

mapping can be stored as a lookup table, i.e. the camera does not have to do the math every time a picture is taken



2

3

4

5

6

7

8

9 10 11 12 13 14 15

1

2

3

4

5

mapping can be stored as a lookup table, i.e. the camera does not have to do the math every time a picture is taken typically, images are stored using 256 = 28 intensity levels per color channel



2

3

4

5

6

7

8

9 10 11 12 13 14 15

1

2

3

4

5

mapping can be stored as a lookup table, i.e. the camera does not have to do the math every time a picture is taken typically, images are stored using 256 = 28 intensity levels per color channel this requires 8 bit (= 1 byte) per CCD element (high quality cameras or scanners store up to 14 bit per color channel)

problems of CCD technology

geometric distortions due to rectangular CCD elements; may necessitate computational correction



blooming overexposure may free too many electrons; these may overflow to neighboring elements and cause highlight effects



blooming overexposure may free too many electrons; these may overflow to neighboring elements and cause highlight effects

dark noise CCD sensors are sensitive to parts of the invisible em-spectrum (UV or IR radiation) and may thus record “inexplicable” phantoms or apparitions

storing light

numerous different file formats (more details later) RAW JPG GIF PNG TIFF .. .

representations of digital images (part 1)

implications of digital recording technology

domain and range of digital images are discrete ⇔ digital images are digitized and quantized

the smallest, basic elements of a digital image are called pixels (picture elements) ⇒ a digital image consists of a matrix of pixels

pixels and pixel arrays

in memory, digital intensity images are stored as (2D) pixel arrays which are also called pixmaps

i

pij

j


color images are stored in 3 to 4 pixmaps, one per color channel and a possible α-layer

i

prij

j


the resolution of a digital image is characterized through the width and the height of the pixmap, typically in a ratio of 4:3 (say, 1024 × 768) or 16:9 (say, 1920 × 1080)


the resolution of a digital image is characterized through the width and the height of the pixmap, typically in a ratio of 4:3 (say, 1024 × 768) or 16:9 (say, 1920 × 1080) given width, height, and color depth of an image, its memory requirement can be readily calculated


the resolution of a digital image is characterized through the width and the height of the pixmap, typically in a ratio of 4:3 (say, 1024 × 768) or 16:9 (say, 1920 × 1080) given width, height, and color depth of an image, its memory requirement can be readily calculated stored in an uncompressed image format, a 1024 × 768 color image with 256 intensity levels per color channel has a size of 1024 × 768 × 3 byte = 2, 414, 592 byte = 2.3 Mbyte


the resolution of a digital image is characterized through the width and the height of the pixmap, typically in a ratio of 4:3 (say, 1024 × 768) or 16:9 (say, 1920 × 1080) given width, height, and color depth of an image, its memory requirement can be readily calculated stored in an uncompressed image format, a 1024 × 768 color image with 256 intensity levels per color channel has a size of 1024 × 768 × 3 byte = 2, 414, 592 byte = 2.3 Mbyte

image formats

data formats for storing digital images on Internet servers, private hard drives, DVDs, . . . there are pixmap (a.k.a. bitmap or raster graphics) formats e.g. JPG, GIF, PNG, TGA, TIFF, PPM, . . . vector graphics formats e.g. SVG, FIG, . . .

example: PPM (portable pixmap) format

P6 # CREATOR: XV Version 3.10a Rev: 12/29/94 (PNG patch 1.2) 74 82 255 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff fe fe fe fe fe fe ff ff ff fe fe fe fe fe fc ff ff fd c7 c6 c4 19 15 ...

1st line: magic number P1, P2, P3: ASCII format P4, P5, P6: binary format then: comments then: image width and image height then: maximum color value (same for all color channels) finally: image data (hexadecimal values) in r, g, b triplets

image compression

data formats for compressed storage of images have their own scientific discipline in this course, we shall not study image compression

image compression

data formats for compressed storage of images have their own scientific discipline in this course, we shall not study image compression at this point, we may note compression reduces memory requirements but it annihilates information differences are hardly visible to the eye but they are there

example

enlarged portrait of a Gallic warrior

example (cntd.)

JPG hex dump 00000000 00000010

ff d8 ff e0 00 10 4a 46 00 01 00 00 ff fe 01 23

49 46 00 01 01 00 00 01 43 52 45 41 54 4f 52 3a

|......JFIF......| |.......#CREATOR:|

0a 72 3a 70 69 68 01 01 01 01 00 01 01 01 01 02 01 05 03

52 31 39 32 30 0a 01 01 01 01 01 01 01 01 00 00 00 c4 00

|..CREATOR: XV Ve| |rsion 3.10a Rev| |: 12/29/94 (PNG | |patch 1.2) Qual| |ity = 100, Smoot| |hing = 0....C...| |................| |................| |................| |................| |.C..............| |................| |................| |................| |.........R.J..".| |................| |................| |................| |...........}....|

... 000000e0 000000f0 00000100 00000110 00000120 00000130 00000140 00000150 00000160 00000170 00000180 00000190 000001a0 000001b0 000001c0 000001d0 000001e0 000001f0 00000200

0a 73 20 61 74 69 01 01 01 01 43 01 01 01 01 11 01 06 02

43 69 31 74 79 6e 01 01 01 01 01 01 01 01 01 01 01 07 04

52 6f 32 63 20 67 01 01 01 01 01 01 01 01 ff 03 01 08 03

45 6e 2f 68 3d 20 01 01 01 01 01 01 01 01 c0 11 00 09 05

41 20 32 20 20 3d 01 01 01 01 01 01 01 01 00 01 00 0a 05

54 33 39 31 31 20 01 01 01 01 01 01 01 01 11 ff 00 0b 04

4f 2e 2f 2e 30 30 01 01 01 01 01 01 01 01 08 c4 00 ff 04

3a 30 34 29 2c ff 01 01 01 01 01 01 01 01 52 1f 00 00 00

20 61 20 20 20 db 01 01 01 01 01 01 01 01 00 00 00 b5 01

58 20 28 20 53 00 01 01 01 01 01 01 01 01 4a 00 00 10 7d

56 20 50 51 6d 43 01 01 01 01 01 01 01 01 03 01 01 00 01

20 52 4e 75 6f 00 01 01 01 01 01 01 01 01 01 05 02 02 02

56 65 47 61 6f 01 01 01 01 ff 01 01 01 01 22 01 03 01 03

65 76 20 6c 74 01 01 01 01 db 01 01 01 01 00 01 04 03 00

example (cntd.)

PPM hex dump 00000000 00000010

50 36 0a 23 20 43 52 45 20 56 65 72 73 69 6f 6e

41 54 4f 52 3a 20 58 56 20 33 2e 31 30 61 20 20

|P6.# CREATOR: XV| | Version 3.10a |

50 51 6d 38 ff ff ff ff ff 12 7e 00 01 de e4 e5 fe c8 ff

68 3d 20 ff ff ff ff ff ff 97 aa 05 01 02 dd 40 c7 ff ff

|PNG patch 1.2) | |Quality = 100, S| |moothing = 0.74 | |82.255..........| |................| |................| |................| |................| |................| |....-...........| |˜gU......9(.....| |................| |................| |................| |................| |.......?@B......| |............:89.| |................| |................|

... 000000e0 000000f0 00000100 00000110 00000120 00000130 00000140 00000150 00000160 00000170 00000180 00000190 000001a0 000001b0 000001c0 000001d0 000001e0 000001f0 00000200

4e 75 6f 32 ff ff ff ff ff 17 67 00 01 c8 e7 e6 fd c6 ff

47 61 6f 0a ff ff ff ff fe 10 55 04 00 b1 d7 e7 fb ff ff

20 6c 74 32 ff ff ff ff fe 0a fc 00 00 1f dc eb ff ff ff

70 69 68 35 ff ff ff ff fe 2d d8 00 04 12 e0 e5 ff fd ff

61 74 69 35 ff ff ff ff fe 1c b8 06 02 09 e0 e6 fd ff ff

74 79 6e 0a ff ff ff ff fe 12 eb 00 03 0b e4 e8 c9 ff ff

63 20 67 ff ff ff ff ff fc b1 c8 00 03 06 e7 3f c9 ff ff

20 20 3d ff ff ff ff fe ff 80 39 00 02 8f e1 42 af ff ff

31 31 20 ff ff ff ff fe fd ca 28 00 0f 8b e4 1d ad ff ff

2e 30 30 ff ff ff ff fe c7 ae 18 03 06 88 de 1d ae ff ff

32 30 0a ff ff ff ff fe c6 96 08 00 00 ed e3 1d 3a ff ff

29 2c 37 ff ff ff ff fe c4 1c 00 00 b7 ed e7 e7 38 ff ff

20 20 34 ff ff ff ff fe 19 07 00 04 a1 eb dd e7 39 ff ff

20 53 20 ff ff ff ff ff 15 00 07 03 8a e0 e2 e7 c9 ff ff

example (cntd.)

result of ls -lh -rw------- 1 user users 10K Apr -rw------- 1 user users 19K Apr

7 7

2003 asterix-head.jpg 2003 asterix-head.ppm

summary

we now know about

imaging methods CCD technology image formats (sort of)

exercises

describe in your own words how a CCD camera works what is dark noise? what is blooming?