possible, of course, the conversion between the images saved in different ways.
... In practice, more frequently used is saving the pixel brightness using 8 bit ...
dr inż. Jacek Jarnicki, dr inż. Marek Woda Institute of Computer Engineering, Control and Robotics Wroclaw University of Technology {jacek.jarnicki, marek.woda}@pwr.wroc.pl
Multimedia & Computer Visualization
Exercise #2 Digital image in computer The purpose of this exercise is to learn the basic ways to store digital images in computer memory. Each of them is used for different image types and it has a specific application. It is possible, of course, the conversion between the images saved in different ways. It should be noted that presented methods do not use any compression algorithms, and each image is saved point by point, the only difference is meaning of information related to an encoded pixel and manner of its encoding using the numbers. It will be considered four types of images: §
monochromatic (intensity images),
§
binary (binary images),
§
RGB color (RGB images),
§
indexed (indexed images).
1. Monochromatic images (intensity images) Monochrome images are images for which a point (pixel) is encoded by a single number, and the whole image using a rectangular array of numbers describing the intensity (brightness, gray value) of individual points. A value of 0 usually represents black, and the maximum value from a permissible range represents white. In MATLAB intensity of a monochrome image point can be coded as: §
8-bit unsigned integer (uint8) – 256 degrees of brightness from 0 to 255 ,
§
16-bit unsigned integer (uint16) – 65636 degrees of brightness from 0 to 65635,
§
Floating point number (double) – practically continuous range from 0 to 1.
Multimedia & Computer Visualization Wrocław 2014
Commonly used method of storing pixel brightness is coding of image points using integer (8 or 16 bit) and in this way the pixel data is usually saved as graphic files (regardless of their format). In practice, more frequently used is saving the pixel brightness using 8 bit integers, since 256 degrees of brightness is due to the characteristics of human visual perception - fully adequate to achieve the impression of a continuous scale. Floating point format is used rather as an intermediate format when we want to perform some mathematical operations on the image elements and it is more convenient to do so by using floating point arithmetic. MATLAB with Image Processing Toolbox provides tools to identify how the data is saved in graphic files and many conversion methods between formats. · Coding of a pixel using 8-bit integer For example, to analyze the contents stored in the image Lena_gray_8.tif it is needed to run three commands: >> I = imread(’Lena_gray_8.tif’); >> imshow(I) >> whos Their execution will: read image from file and display it in the graphics window and then print the variable characteristics in the console, which shows that table I created by the usage imread(…) command contains a number of 8-bit unsigned integers (uint8), it has a size of 256 x 256 and occupies 65,536 bytes. As you can see, in this way the image size and method of data brightness encoding of individual pixels have been identified. This was possible thanks to the running imread(…)command, which provides mechanisms for correct interpretation of information contained in the file that we load up. Such a file (in this case 'Lena_gray_8.tif') usually consists of a header, which in some way provides information about the format of the data describing the image, and the way how it is saved along with the data itself. Data can be saved as compressed or uncompressed. In order to determine how the data is actually sin the image - run the following commands: >> imfinfo(’Lena_gray_8.tif’); On the console it will be printed full information on how the data is encoded and characteristics of the image. More important parameters are: On the console there will be printed full information on how to encode the data and characteristics of the image file. Important parameters are FileSize: 70622 – size of the graphics file Width: 256 – image width Height: 256 – image height ColorType: 'grayscale' – data interpretation from color reproduction point of view
Multimedia & Computer Visualization Wrocław 2014
BitsPerSample: 8 – number of bits per sample image PhotometricInterpretation: 'BlackIsZero'– interpretation of numbers describing the image points SamplesPerPixel: 1 – number of samples per pixel Please note that table I holding the image points is 65,536 bytes, while Lena_gray_8.tif file in TIFF format is a little bigger and it occupies 70,622 bytes. The difference stems from, inter alia, that in addition to the data file contains a header. · Coding of a pixel using 16-bit integer For a comparison how a file header containing an image with the points coded on 16 bits looks like, you can execute the commands: >> J = imread(’Lena_gray_16.tif’); >> imfinfo(’Lena_gray_16.tif’) >> whos You can also display image stored in the array J in order to see that the monochrome images with 8 and 16 bit brightness point encoding are not much different. · Coding of a pixel using floating point number The last method of monochrome image encoding is to record image points using floating point numbers from the range [0,1].In order to convert integer (uint8) or (uint16) to floating point number (double) it is convenient to use the im2double(…)function available in "Image Processing Toolbox". For example to convert an image with 16-bit brightness encoding stored in the array J, you may use the command: >> K = im2double(J); This way it will be created K array, in which the image from J array is stored using floating point numbers from the range [0,1]. To make sure, you can type in following commands: >> J(45, 167) >> K(45, 167) to show a corresponding elements of array J and K and find out that the conversion was done properly. Of course, a set of functions contained in "Image Processing Toolbox" provides the ability to reverse operations, namely to go back from a floating point to an integer. This can be done using the functions im2uint8(…) or im2uint16(…).
Multimedia & Computer Visualization Wrocław 2014
2. Binary images Binary images are images for which individual points are encoded using Boolean variable receiving the value 0 or 1. If we assume that the value 0 corresponds to black, and white is the value 1 then we can display a binary image. Binary images are often being acquired from monochrome or color images in order to simplify further processing, in procedures such as as object recognition, quantitative analysis of images content or other. Way of getting binary image from the monochrome one is quite simple. At the beginning you need to establish so-called ‘binarization’ threshold (level), namely to find a number in the range in which brightness points of monochrome image are located. For example, if the source image is a 8-bit monochrome brightness coded image, a threshold can be any number in the range [0, 255]. Next you only need to calculate binary values for the individual points of image using following dependencies:
ì0 if y(i, j ) = í î 1 if
x(i, j ) £ level x(i, j ) > level
where: x(i, j) – brightness point of (source) monochrome image, y(i, j) – Boolean value for point of (output) binary image, level – ‘binarization’ threshold. · Monochrome to binary image conversion The task is to write a simple converter program transforming a monochrome image to a binary one with adjustable binarization threshold. The source image is a microscopic view on biological material, showing the cells decomposition, taken from brain of the fish called "zebra fish" and it comes from the Max Planck Institute in Dresden. Source image named Cells.tif and a threshold is any given number from [0, 1] range, for example, 0.5. The next steps of image conversion algorithm are as follows (names given in brackets are MATLAB functions recommended to use to implement particular steps of algorithm): §
Set the binarization threshold by a variable declaration and initialization named the level e.g. level = 0.5;
§
Read the source image from Cells.tif file and save in (imread(…) function).
§
Perform conversion of the image stored in table I (saved as integers - uint8) into floating point (double) image format and store it table J (im2double(…)function).
§
Determine the image size stored in the J array (size(…)function)
Multimedia & Computer Visualization Wrocław 2014
I
table
§
Declare table K in line with the size image that contains logical zeros. This can be done with the command K = logical(zeros(n, m)); where n and m are image sizes determined in the previous step
§
Examine the consecutive image points stored in J array, and if the brightness exceeds a specified threshold (level) change the corresponding element in K array from 0 to 1 Note: In order to view consecutive image points you should use double for loop, while for threshold testing please use if conditional statement (please refer to the MATLAB Help system for more details)
§
In one of graphics window to display the source image from an array I and the binary image from array K (subplot(…)function).
§
Save the binary image from K array to a file – use the command imwrite(K, 'Cells_bin.tif', 'tif')
§
Using the command imfinfo(…)check the file parameters of recorder binary image.
Figure 1 shows how the window graphic with the source and calculated images for the binary threshold binarization (level = 0.5) should look like.
Fig. 1. Monochrome and binary images (level = 0.5)
Multimedia & Computer Visualization Wrocław 2014
You can now experiment with the threshold binarization value to see how its value influences the appearance of the output image.
3. Color images (RGB images) Color images produced according RGB color model differs from monochrome ones that a single image point (pixel) is described by not one but and three numbers. These numbers are called color components R, G and B and specify degree of each primary color (red, green and blue) in a pixel. It is generally assumed that when color component equals to 0 means the absence of a primary color, and value 1 means its maximum share. In MATLAB, as it was in the case of monochrome images, color components R, G, B are being encoded using integers and floating point (double) values. There are three basic types of color component notation: §
8-bit unsigned integer (uint8) – 256 degrees of color component 0 to 255,
§
16-bit unsigned integer (uint16) – 65636 degrees from 0 to 65635,
§
Floating point number (double) – practically continuous range from 0 to 1.
First notation using 8 bits is able to save: 2563 = 16.777.216 colors Second notation using 16 bits: 656363 = 282.765.433.979.456 colors whereas in floating point notation is even higher (possible number of colors depends on how the floating point numbers are coded) To find out the difference between color and monochrome file description, use the commands: >> imfinfo(’Lena_gray_8.tif’) >> imfinfo(’Lena_color_256.tif’) and compare the contents of messages shown in MATLAB console. · RGB color components distribution of an image The aim of the task is to read the color image from a file, take the image color components (R, G, and B) apart and then display each component as a separate monochrome image The algorithm goes as follows: §
Read a source image from Spectrum.tif file and stored in I table (imread(…)function).
Multimedia & Computer Visualization Wrocław 2014
§
Determine the size of the image stored in I array (size(…)function). Note: A table that holds the RGB image is three-dimensional.
§
Declare three tables: R, G and B, in which monochrome images will be stored, it can be done using the command zeros(n, m, 'uint8'); where n and m are images sizes determined in the previous step.
§
Save the data from I table into R, G, B arrays. Note: You can do this with or without loops.
§
In one of the graphics window display the source image from I array and the monochrome images from R, G, B arrays (subplot(…)function).
Obtained effect should look like as it is shown in Figure 2. Subtitles shown on the individual images can be displayed using title(…) function (please refer to the MATLAB Help system for more details).
Fig. 2. Color image and R, G, B, components displayed as monochrome images 4. Indexed images In previously described methods of image recording the image contents was always stored in a single array. In the case of monochrome and binary images it was two-dimensional array, whereas for color images it was three-dimensional one. Another image type is called indexed one and differs from previous ones in that are stored in two separate arrays. Using indexing you can encode both monochrome and color images. The way how to store a color image using the indexes is presented on Figure 3 Multimedia & Computer Visualization Wrocław 2014
Data table I (162, 157) = 253
Color table Index
R
G
B
…
…
…
…
(162, 157) 253 …
0.7804 0.4745 0.4745 …
…
…
Fig. 3. Explanation of storing a color image using indexes
Now to encode an image two tables are required: ·
data matrix that is equal to image size, which elements are unsigned integers (uint8 or uint16).
·
colormap matrix built from a number of rows containing three floating point numbers (double) describing the R, G and B color components.
Array elements are simply rows numbers of color array (shown in Figure 3). This method of image coding is undoubtedly, in most cases, more cost-effective than direct entry of each R, G, B component. This facilities quick image alteration too. In order to illustrate how to use indexing to store an image you can perform a simple experiment, consisting in replacing a single row of color array. For this purpose, you should first load up an image saved earlier as indexed one. Such an image is named Lena_color_index.tif, and you can load it up using the command: >> [I, map] = imread(’Lena_color_index.tif’); It should be noted that, unlike to all previous cases, that loaded picture was stored in two tables I and map. The first of these is the previously described data array (with indexes) and the second on is color array. If you execute the command: >> I(162, 157)
Multimedia & Computer Visualization Wrocław 2014
value 253 will be displayed, other words index of color table row corresponding to a pixel represented by coordinates (162, 157). To read color components values of the analyzed pixel type in: >> map(253,:) That will display R, G, B color components of the pixel. These are the numbers 0.7804, 0.4745, 0.4745 (see Figure 3). To replace row elements of the color table please use following statement: >> map(253,:) = 1.0; All three elements of the row, namely color components values will be changed to 1.0 which corresponds to white color. Finally, to see the result, write command: >> imshow(I, map); the effect should be the same as shown in Figure 4.
Fig. 4. Image with one color replaced Interpretation of the result is simple. All pixels in the image, which in the data array had a value of 253, after replacing the row in the color array were displayed in white. · Getting the image in sepia The task involves processing grayscale image saved as indexed one in similar sepia like image.
Multimedia & Computer Visualization Wrocław 2014
The algorithm is as follows: §
Read a source image from Lena_gray_index.tif file and store it in tables I and map (imread(…)function).
§
Declare and initialize alfa variable e.g. alfa = 0.65;
§
Determine
§
Create new color table map_1 – copy to it, two first columns of map color table (that determine R and G color components) and multiply all the values from third column (corresponding to B color component) by alfa.
§
Display in separate windows both images (I, map) i (I, map_1), these should look like as on Figure 5.
§
Try to change value alpha until you obtain the desired appearance of the image in sepia.
§
NOTE: Please use subimage() function in order to display both images in single figure
image
size
stored
in
map
color
table
(size(…)function).
Fig. 5. Source and sepia like images (alfa = 0.65) · RGB to indexed image conversion From the standpoint of saving memory to store an image, indexed images seem to be quite attractive. However, recording devices usually do not save images as indexed ones. Acquisition of data requires involvement of special algorithms and a lot of processing in order to save acquired material as indexed images. A source image that is saved according to RGB model rules usually comprises lots of colors. And in extreme case, each pixel can be described by a slightly different color. For such an image, when you try to save it, indexed color table Multimedia & Computer Visualization Wrocław 2014
will have as many rows as the image has points and additionally you will need to add a data array containing the indexes. A saved image in this case will occupy more memory than a direct description of R, G, B color components for specific pixels. The general conclusion is, that the storing an image as indexed one only pays off when number of image colors is far less number of pixels. But it turns out that due to certain characteristics of color perception by a human eye, we can reduce the number of colors without much loss to general image quality. The idea is simple, and lies in the fact that points with similar colors are assigned to a calculated in some way color. This process is called color quantization and is implemented using various algorithms. The most popular is a uniform quantization and quantization with minimum variance criterion. Uniform quantization is the simplest algorithm of reducing the number of colors in images. The idea of it is easy to explain using the RGB color model in the form of a cube, such as is shown in Figure 6
G (0, 1, 0)
B
R (1, 0, 0)
(0, 0, 1)
Fig. 6. RGB color model in the form of a cube If each of the cube sides we divide into n equal segments then one can imagine that the cube will be divided into n3 small cubes. If we assign for all of these cubes a specific color such as color of inner cube, then a discrete structure describing the space formed by n3 colors will be created. Color quantization algorithm is very simple: §
For each pixel described by R, G, B components, identify in which cube a pixel described by these components is located.
§
As new color coordinates of a pixel, assign the color coordinates corresponding a cube determined in the previous step.
Multimedia & Computer Visualization Wrocław 2014
§
Build a color table with only the colors from cube’s color space, which have their counterparts in the processed image.
As a result of uniform quantization an image will be created, which will contain the points of maximum n3 colors. The algorithm is simple; however, a problem can make the choice of n number. For small n, the number of colors of the processed image may be too small, for large n color table is likely to grow considerably. It should also be noted that allocation of color space does not depend on image content and depends only on n number chosen. Quantization with the minimum variance criterion is much more complicated. RGB color space represented by the cube of Figure 6 is divided in this case in unequal (different size) small cubes, the other way round as it was the case of evenly quantization. The division is dependent on the points’ distribution representing the individual pixels of the image inside the color space cube. If there is large points concentration in a certain area, a small rectangular cuboid (rectangular prism) is being constructed, and if the local density of points is small, cuboid is bigger. The individual points lying inside so designated cuboids are represented in the destination image by the cuboids centers. Such a division of color space allows for a better source image approximation with a given number of colors, making the outcome dependent on the content of the analyzed image and not, as it was in the case of uniform quantization only on an arbitrarily taken parameter. Of course, better quality of approximation requires more computation. „Image Processing Toolbox” offers a function called rgb2ind(…), which converts the RGB color image to indexed one. Both abovementioned methods are implemented. The aim of last task is to examine what results give the two color quantization algorithms and comparing the results of their actions. The steps are as follows: §
Load up Lena_color_256.tif file.
§
Use rgb2ind(…)function in order to perform evenly and minimum variance criterion quantization, to get the smallest possible color table, that gives visual result of indexed image equally good to the source image.
§
Check, what is the file size of saved indexed image obtained using abovementioned (both) methods.
Multimedia & Computer Visualization Wrocław 2014
