ONE-DIMENSIONAL LEAST-SQUARES MODEL-BASED ...

ONE-DIMENSIONAL LEAST-SQUARES MODEL-BASED HALFTONING1 David L. Neuho2 Department of Electrical Engineering and Computer Science University of Michigan, Ann Arbor, MI 48109 Thrasyvoulos N. Pappas Bell Laboratories Lucent Technologies, Murray Hill, NJ 07974 Telephone: (908) 582-2933 FAX: (908) 582-7308 email:[email protected] Nambi Seshadri Communications Research Department AT&T Labs-Research, Murray Hill, NJ 07974 February 27, 1997

Abstract A least-squares model-based approach to digital halftoning is proposed. It exploits both a printer model and a model of visual perception. It produces a halftoned image that minimizes the squared error between the response of the visual model to the original image and the response of the printer and visual models to the halftoned image. For onedimensional halftoning, where each column is processed independently, it is shown that least-squares halftoning can be optimally performed with the Viterbi algorithm. Applying this approach to simple visual and printer models yields better halftones than conventional one-dimensional methods. Though one-dimensional halftoning is seldom used in practice, the results of this method are of interest because their appearance is due only to the models and the fundamental nature of halftoning. Thus they can serve as a guide to the design of two-dimensional approaches for which only approximate least-squares solutions are possible.

1 2

Portions of this work were published in Proc. ICASSP-92, San Francisco, CA, May 1992. Consultant at AT&T Bell Laboratories in 1989-90.

February 27, 1997

2

1 Introduction Digital halftoning is the process of creating a binary approximation to a gray-scale image. It is necessary for display of gray-scale images when direct rendition of gray tones is not possible, for example when printing on paper by conventional means. We propose a least-squares model-based (LSMB) approach to digital halftoning that is capable of exploiting both a printer model and a model of visual perception to produce high quality halftones. This is accomplished by minimizing the squared error between the response of the visual model to the original image and the response of the printer and visual models to the halftoned image. In this paper we focus on the case of one-dimensional halftoning, and show that the LSMB approach can be optimally performed with the Viterbi algorithm for a wide range of printer and visual models. Applying this approach to simple visual printer and visual models yields better halftones than conventional one-dimensional methods. Though onedimensional halftoning is seldom used in practice, the results of this method are of interest because their appearance is due only to the models and the fundamental nature of halftoning. Thus they can serve as a guide to the design of two-dimensional approaches for which only approximate least-squares solutions are possible. Model-based halftoning can be especially useful in transmission of high quality documents using high delity gray-scale image encoders [1, 2]. In such cases halftoning is performed at the receiver, just before printing. Apart from coding eciency, this approach permits the halftoner to be tuned to the individual printer, whose characteristics may vary considerably from those of other printers, for example, 300 vs. 600 dpi or write-black vs. write-white laser printers. Halftoning relies on the assumption that the visual system acts as a spatial low-pass lter. More detailed models of visual perception have been developed, and we show that it is possible to implement halftoning methods that actively exploit them. The visual models we use are based on estimates of the spatial frequency sensitivity of the eye. Halftoning also relies on the model that the \blackness" of a printed binary pattern is proportional to the fraction of ones in the pattern. This means that a \1" in the pattern causes a black dot to be printed, a \0" leaves white space and the areas of the black dot and white space should, ideally, be the same. It follows that the \desired" shape for the

February 27, 1997

3

black dots produced by a printer would be T  T squares, where T is the dot spacing. p However, most printers produce circular dots. Their radius must be at least T= 2, so they are capable of blackening a page entirely. Thus, black dots cover portions of adjacent white spaces, causing the perceived gray level to be darker than the fraction of ones. We refer to this phenomenon as dot overlap. It causes halftoning methods that generate dots in proportion to the desired gray level to produce a nonlinearly distorted gray scale. Moreover, most printers produce black dots that are larger than the minimal size and introduce other distortions that further aect the gray scale. The most commonly used digital halftoning techniques for printers, such as clustered ordered dither, protect against dot overlap by clustering black dots [3]. This causes much of the dot overlap to fall on other black dots, rather than on space that should remain white, thus attaining a nearly linear gray scale. Unfortunately, such clustering constrains the gray-scale and spatial resolutions of the perceived images and increases the visibility of low frequency artifacts. In order to obtain a gray scale that is closer to linear, a monotonic, nonlinear transformation can be applied to the image prior to halftoning to compensate for the nonlinear eect of the printer on the gray scale. This transformation is based on measurements of the re ectance of the characteristic patterns produced by a given halftoning technique. In the case of ordered dither, the transformation can be applied to the threshold matrix instead of the image. This can be done for dispersed-dot order dither techniques as well, but clustered-dot techniques produce a more uniform distribution of gray levels. In the case of error diusion, the transformation is applied to the image [4]. However, this transformation works only when the printer introduces a monotonic nonlinearity. Rather than trying to resist or compensate for printer distortions, the proposed LSMB method actively exploits a model of such distortions in order to attain a linear gray scale, high spatial resolution, and pleasing textures (rather than low frequency artifacts). In addition, we show that it can produce even more gray levels than can be produced by the other approaches. Accordingly, an accurate printer model, such as the dot-overlap and window models given in [5, 6, 7, 8], plays an important role. Several authors have previously proposed model-based halftoning approaches. Allebach [9] proposed halftoning so as to minimize a visual model based distortion measure, but did not give speci c methods for doing so. He also suggested the use of a display model but did not give a speci c one. Anastassiou [10] proposed halftoning so as to minimize a frequency

February 27, 1997

4

weighted squared error between the binary and original images. The latter is equivalent to our formulation except that we incorporate a printer model and permit nonlinearity in the visual model. Anastassiou and Kollias [10, 11] also described a neural network technique that approximately minimized the frequency weighted squared error. Roetling and Holladay [12] proposed a dot-overlap printer model, like the one used in this paper and [5, 6], but used it only to modify ordered dither so that it results in a linear gray scale. Allebach [13] also proposed a modi cation of ordered dither that takes into account dot overlap to improve the gray scale of the printed images. However, ordered dither is a highly constrained method that cannot achieve the high spatial resolution, increased gray-scale resolution, and visually pleasant textures of the proposed model-based approach, nor of the modi ed error diusion mentioned in the next paragraph. Another technique that takes into account dot overlap was proposed by Pryor et al. [14]. Its performance is also inferior to the proposed method. In [5, 6], Pappas and Neuho showed that printer models, especially the dot-overlap model, can be used to modify error diusion to account for printer distortions. The resulting modi ed error diusion (MED) algorithm provides high quality reproductions with reasonable complexity. However, error diusion does not make use of an explicit visual model, and this results in some well known artifacts and asymmetries. The rst to propose the use of a dot-overlap model to account for printer distortions in error diusion was Stucki [15, 16]. In [6] it was found that Stucki's algorithm is more ecient computationally, while the MED algorithm has better performance. Recently, Rosenberg [4] used the dot-overlap printer model proposed in [5, 17] to derive a gray-scale transformation that can be applied to the image before halftoning to compensate for printer distortions. However, as mentioned earlier, this approach works only when the printer introduces a monotonic nonlinearity, and cannot produce as ne a gray scale as the model-based approaches examined here and in [6]. In [7, 8] it was shown how to identify the parameters of sliding-window printer models from measurements of the re ectance of various test patterns. These were then used in MED and LSMB approaches. The least-squares model-based halftoning described in this paper exploits both a printer model and a model of visual perception. It produces an \optimal" halftoned reproduction by nding the binary image that minimizes the squared error between the output of the cascade of the printer and visual models in response to the binary image and the output of

February 27, 1997

5

the visual model in response to the original gray-scale image. The resulting halftones should have high spatial and gray-scale resolution, and pleasing textures. This paper pursues the one-dimensional case, in which each row or column of the image is halftoned independently. One-dimensional halftoning is simpler than two-dimensional halftoning and easier to analyze, but is seldom used in practice because it does not exploit the two-dimensional properties of vision. We show how the one-dimensional least-squares approach can be exactly implemented using the Viterbi algorithm. Our experiments show that it successfully produces an essentially linear gray scale with more gray levels and better spatial resolution and texture than conventional one-dimensional screening techniques. We also nd that the least-squares approach eliminates problems associated with the one-dimensional version of error diusion. As two-dimensional halftoning generally produces more pleasing textures than onedimensional, it is of interest to pursue LSMB halftoning in two dimensions. The latter is a considerably more dicult task. Unfortunately, the inherently one-dimensional structure of the Viterbi algorithm makes it unsuitable for solving the two-dimensional problem. Instead, approximate (and generally iterative) methods have been developed. We discuss them in the next paragraph. Although suboptimal two-dimensional LSMB halftones tend to look better than optimal one-dimensional LSMB halftones, the latter are interesting precisely because they are optimal. Thus, one can be sure that their appearance re ects only the visual model, the printer model, and the fundamental nature of halftoning. In contrast, when evaluating the appearance of two-dimensional LSMB halftoning, it is dicult to say how much is due to the models and how much is due to the approximate nature of the minimization. Aside from their theoretical interest, the one-dimensional results are a valuable guide to understanding the two-dimensional problem. In addition to [10], the two-dimensional LSMB approach has been recently pursued by several authors. An iterative approach, including both visual and printer models was explored by Pappas and Neuho in [18, 19]. In similar eorts, Mulligan and Ahumada [20] assume perfect printing, and Analoui and Allebach [21] consider a dierent display model. Carrara et al. [22] consider variations of the scheme presented in [21] and also adopt our dot-overlap printer model, while Flohr et al. [23] investigate computational issues. Zakhor et al. [24, 25] also assume perfect printing, but the performance of the approach they propose is inferior to the above techniques because it breaks the image into small blocks

February 27, 1997

6

and halftones them independently. Finally, a related technique using Markov random elds is presented by Geist et al. [26]. In the present paper, Section 2 discusses visual models. The printer models are presented Section 3. Section 4 presents the one-dimensional least-squares model-based halftoning approach, including the Viterbi algorithm and a method for reducing its complexity. Section 5 describes our experimental results with this approach. The conclusions are summarized in Section 6.

2 Models of Visual Perception This section describes the kinds of visual models that can be exploited by the LSMB approach and the speci c models on which the one-dimensional LSMB algorithm will be tested. Halftoning works because the human visual system (HVS) acts as a spatial low-pass lter. Numerous researchers have estimated the spatial frequency sensitivity of the visual system, often called the modulation transfer function (MTF). Typical of such is the estimate that Mannos and Sakrison [27] found to be good for predicting the subjective quality of coded images n o H (f ) = 2:6 (0:0192 + 0:114 f ) exp ?(0:114 f )1:1 (1) where f is in cycles/degree. This MTF is plotted in Figure 1. As indicated by (1), the visual system is most sensitive to frequencies around 8 cycles/degree. (Others have variously estimated the peak sensitivity to lie between 3 and 10 cycles/degree [28, p. 55].). The decrease in sensitivity at higher frequencies is generally ascribed to the optical characteristics of the eye (e.g. pupil size) [29]. Notice that the sensitivity has dropped 3 db at about 3 and 16 cycles/degree, 20 db at 35 cycles/degree and about 46 db at 60 cycles/degree. The decrease in sensitivity at low frequencies accounts for the \illusion of simultaneous contrast" (a region with a certain gray level appears darker when surrounded by a lighter gray level than when surrounded by a darker) and for the Mach band eect (when two regions with dierent gray levels meet at an edge, a light band is perceived on the light side of the edge and a dark band is perceived on the dark side of the edge). Besides the MTF, many other characteristics of the human visual system have been studied. A recent, comprehensive reference is [30]. Based on such characteristics, a variety

7

February 27, 1997

of visual models have been proposed for use in various image processing tasks. The simplest, arguably, include just a lter, for example the lter of (1). The next simplest, and perhaps most commonly cited in image processing, [31, p. 292], [28, p. 57], [32, 27], include also a memoryless nonlinearity, as shown in Fig. 2. Such nonlinearities account for Weber's law, which says that the smallest noticeable change in intensity is proportional to intensity. Most commonly it is represented as a logarithm or power law (e.g. intensity1=3). More complex models include, for example, a lter before the nonlinearity [29] or a bank of lters [31, p. 295], [33]{[34],[35]{[36]. For the least-squares halftoning of Section 4, we require a one-dimensional, discretespace model of the form zk = M (xk?m ; : : :; xk+m ) (2) where the xk 's are samples of the image (from one line or column), the zk 's are the model outputs (upon which cognition is based), and M (
) is some sliding-window function with 2m + 1 arguments (m is a non-negative integer). The gray level xk of the image is assumed to vary between 0 (white) and 1 (black). We make xk proportional to \blackness" rather than the usual \whiteness" because it is blackening rather than whitening that requires eort, as in placing ink on paper. Thus, the usual image intensity is (1 ? gray level). The above model can easily incorporate a memoryless nonlinearity and an FIR lter. We experimented with models of the form

zk = n(xk )  hk

(3)

where n(
) is a memoryless nonlinearity, h?m ; :::; hm is the impulse response of an FIR lter and  denotes convolution. Following [27, 37, 38], we tried

n(x) = 1 ? (1 ? x)r

(4)

for various values of r. (In [27], r = 1=3 was found to be best.) It is obviously advantageous to choose m as large as possible, but to limit complexity we chose m = 9 and found a 19-th order FIR lter that roughly matched (1) for samples taken at 300 dpi and viewed at 30 inches. The resulting impulse response is given in Table 1 and plotted along with the frequency response in Fig. 3. We have found that this impulse

February 27, 1997

8

response is well approximated by a Gaussian shape with standard deviation 1.5. This is convenient because one can simply change the standard deviation to account for dierent viewing distances. Unfortunately, m = 9 is not large enough to make the frequency response decrease at low frequencies, which would capture the Mach band eect. If the methods of Section 4 can be simpli ed, then the order of the lter can be increased to account for Mach bands.

3 Printer Models This section describes both the general form of printer model that the LSMB halftoning method requires and the speci c model on which the method will be tested. Our work is oriented to laser printers. For example, we used write-black laser printers with 300 dpi resolution, such as the HP LaserJet II and the Data Products LZR 1260, as our test vehicles. However, the discussion and methods are intended to apply directly, or be adaptable, to a wide variety of printers. To a rst approximation, such printers are capable of producing black dots on a piece of paper, at any and all sites of a Cartesian grid with horizontal spacing of T1 inches and vertical spacing of T2 inches. For simplicity we assume that T1 = T2 = T . The reciprocal of T is the printer resolution in dots per inch (dpi). The printer is controlled by an NW  NH binary array b = [bi;j ], where bi;j = 1 indicates that a black dot is to be placed at site (i; j ), located iT inches from the left and jT inches from the top of the image, and bi;j = 0 indicates that the site is to remain white. Since T is ordinarily quite small, the overall eect of printing b can be summarized by an array p = [pi;j ], where pi;j is the average gray level induced by the ink in a T  T square \site" surrounding the (i; j )-th grid point. In our convention, pi;j can take any value between 0, representing white, and 1, representing black. Experience shows that pi;j depends not only on the bit bi;j corresponding to the (i; j )-th site, but also on the bits corresponding to neighboring sites. For one thing, a black dot is ordinarily round and large enough that it overlaps neighboring sites. It must be large enough to cover its own square site. But in fact, it is usually larger. For example, the dots produced by the HP printer have, approximately, 2.45 times more area than a site, which is 56% larger than needed to cover the site. Thus, even when bi;j = 0, pi;j will be positive if

9

February 27, 1997

one or more neighboring sites contain black dots. There are other intriguing phenomena, as well. For example, the HP printer puts signi cantly more ink on the paper when printing a pattern of alternating black and white (blank) lines than a pattern in which ve black lines appear between each pair of white lines. (This suggests that the charge induced by the laser on the ink attracting drum depends in a complex way on the binary array.) A printer model is simply a function indicating how pi;j depends on bi;j and neighboring bits. Since this paper pursues one-dimensional halftoning, we adopt one-dimensional printer models. The input to such a model is a one-dimensional sequence of bits b = [b1; : : :; bN ], where the site k is now located kT inches from the top (or the left) of the image, and the output is a sequence of gray levels p = [p1; : : :; pN ]. The one-dimensional LSMB method considered in the next section requires a sliding-window model of the form

pk = P (bk?n ; : : :; bk ; : : :; bk+n ) :

(5)

Since images are actually two-dimensional, a more de nitive interpretation of this model is that pk is a prediction of the darkness of the kth row (or column) of the two-dimensional printed image that results when the input to the printer is a two-dimensional binary array with all columns (or rows) identically equal to [b1; : : :; bN ]. Such a model has 22n+1 parameters, namely, the values of P for all possible arguments. A method for identifying these parameters from measurements of the gray level of periodic test patterns was described in [8, 7], along with speci c models with n = 1; 2; 3 for the HP printer. Examples of such patterns are shown in Fig. 4. In particular, the gray levels, as measured by a re ection densitometer, were related to the 22n+1 printer model parameters (the values of P ) through a set of linear equations, and a constrained optimization problem was solved to obtain the parameters. In this paper, we choose to test the LSMB method on a one-dimensional version of the simple dot-overlap model described in [5, 6], which though not quite as good as the models in [8, 7], is conceptually simpler. This makes it easier to interpret the results. Speci cally, as illustrated in Fig. 5, it is assumed that a (one-dimensional) black dot has width T (1+2 ), so

10

February 27, 1997

that it covers  fraction of each of the two neighboring sites. Therefore, assuming 0 <  < 1,

8 0; if bk?1; bk; bk+1 = 000 > > > < ; if bk?1; bk; bk+1 = 001 or 100 pk = P (bk?1; bk ; bk+1) = > 2; if bk?1 ; bk ; bk+1 = 101 > > : 1;

(6)

bk = 1

To identify the dot-overlap parameter  , one may view this model as a kind of projection of the two-dimensional dot-overlap model described in [6]. Accordingly, we deduce from [6] that  = :33 is a good value for the HP printer. With this choice, the gray levels predicted by the model for a one-dimensional sequence are the same as those predicted by the twodimensional dot-overlap model of [6] for each column of a two-dimensional image whose columns equal the one-dimensional sequence. The gray levels produced by this model for various vertically periodic patterns are shown in Table 2 along with the measured re ectance (the table shows 1 ? re ectance) of each printed pattern, obtained using a re ection densitometer. Actually, the densitometer measures re ectance density; the re ectance values are given by re ectance = 10?density. Similar measurements were shown in [8, 6]. One can see that the dot-overlap model does much to explain how patterns with the same numbers of ones can have dierent gray levels. For example, it predicts the relative gray levels of patterns with 3 ones. On the other hand, it does not explain why the pattern 101010 is darker than 110110, 101110, and 111110 which have more ones. Thus, there is room for more accurate printer models. It is occasionally useful to simulate dot overlap at a lower resolution (e.g. 100 dpi) using dots created by printing at a higher resolution (e.g. 300 dpi). For example, Fig. 6 shows a simulation of  = :33 dot overlap at 100 dpi, in which each 100 dpi dot is constructed from ve 300 dpi dots. There are two reasons for doing so. One is that this enables one to test a halftoning algorithm on a \printer" with \true" dot overlap, so that the appearance of the resulting halftoned image can be presumed to be caused by the ability of the algorithm to exploit dot overlap, rather than by the accuracy (or inaccuracy) of the dot-overlap model. A second is that the 300 dpi laser printer output cannot be printed in this journal without substantial additional distortion, while the simulated dot overlap at 100 dpi can be printed more accurately. Accordingly, all halftoning results shown in Section 5 are shown at 100 dpi using simulated dot overlap. Notice that when printing the simulation of .33 dot overlap

February 27, 1997

11

illustrated in Fig. 6, the laser printer \adds" its actual dot overlap p , resulting in an eective dot overlap of e = :33(1+ p) = .44. And journal printing of such can be expected to add a little more dot overlap.

4 Least-Squares Model-Based Halftoning Suppose that an image, a printer model, and a visual perception model are given. We will refer to the cascade of the printer and visual perception models as the perceptual printing model. The least-squares model-based (LSMB) halftoning approach chooses the binary image that minimizes the squared error between the output of the perceptual printing model in response to the binary image and the output of the visual model in response to the original gray-scale image. As mentioned earlier, rather than simply assuming that the visual system acts as a low-pass lter that averages adjacent bits (as in conventional ordered dither and error diusion), this method actively exploits the visual model. And rather than simply trying to be robust to printer distortions, as in conventional clustered ordered dither, this method actively exploits printer distortions to create the best possible halftoned reproduction. The result should be high spatial and gray-scale resolution and pleasing textures. The question arises as to how to implement the least-squares approach. Here, we describe a method based on the Viterbi algorithm [39, 40]. Unfortunately, as indicated earlier, it is applicable only to one-dimensional halftoning, so that other methods are required for two-dimensional LSMB halftoning. The remainder of this section is devoted to describing the (optimal) Viterbi approach to LSMB halftoning in one dimension, with the exception of a brief overview of work on iterative (suboptimal) approaches to LSMB halftoning in two dimensions. Suppose we are given a one-dimensional image x = [x1 ; : : :; xN ], a printer model with the sliding-window form pk = P (bk?n ; : : :; bk+n ) (7) and a visual model of the form speci ed in Section 2 with a memoryless nonlinearity n(x) followed by an FIR lter with impulse response h?m ; : : :; hm . In the least-squares approach

12

February 27, 1997

we seek the halftoned approximation [b1; : : :; bN ] that minimizes the squared error

E=

N X k=1

(zk ? wk )2

(8)

where, as illustrated in Fig. 7,

zk = yk  hk = n(xk )  hk wk = vk  hk = n(pk )  hk pk = P (bk?n; : : :; bk+n ) As boundary conditions, we choose for k < m + n + 1; k > N ? m ? n

xk = 0 ; bk = 0 ;

for k < m + n + 1; k > N ? m ? n

These guarantee that the perceived images (the response of the perceptual printer model to the binary image and the response of the perceptual model to the original gray-scale image) are perfectly white for k < 1 and k > N , and gradually darken to the \true" intensity in a border (m + n + 1 pixels for the former and m + n for the latter). Note that since the visual model and printer models have \memory", the least-squares approach is noncausal. That is, the decisions at any point in the image depend on future as well as past image pixels. In contrast, with error diusion the decisions at any point in the image depend on the past only. This noncausality gives the least-squares approach the freedom to make sharp transitions and track edges better than any other approach. In formulating the minimization so the Viterbi algorithm may be applied, we follow the approach of Ungerboeck [41] and Acampora [42] (also found in [43, pp. 272{277]), because, as we shall see, it permits a reduced complexity approximate solution. As a rst step, one may straightforwardly show N X E =
(zk ? wk )2 k=1

= kz k2 +

NX ?m k=m+1

0 1 kX ?1 @?2vk Zk + vk2H0 + 2vk vj Hk?j A j =k?2m

13

February 27, 1997

= kz k2 +

NX ?m k=m+1

k ;

(9)

where

kzk2 =

N X zk2;

k=1

k = ?2vk Zk + vk2H0 + 2vk Zk = Hk =

kX +m j =k?m kX +m j =k?m

kX ?1 j =k?2m

vj Hk?j ;

(10)

zj hj?k ; hj?k hj :

(11)

From (9) we see that the squared error E is the sum of kz k2 , which does not depend on the bits [b1; : : :; bN ] to be selected, plus the k 's, each depending on a dierent subset of the bits. That is, k depends on (bk?2m?n ; : : :; bk ; : : :; bk+n ). In the Viterbi algorithm, the minimization of E is simpli ed by introducing the notion of state. Speci cally, we de ne the state at time k to be

Sk = (bk?2m?n+1 ; : : :; bk ; : : :; bk+n);

(12)

i.e., it consists of 2m+2n consecutive bits neighboring bk . We consider bk to be the \present" bit and bk+n to be the \most recent" bit. Notice that the state has been de ned so that

k can be determined from Sk?1 and Sk , so that Sk can be determined from Sk?1 and the most recent bit bk+n , and so that Sk contains as few bits as possible. In essence, the state Sk?1 summarizes all that one needs to know to determine k except the most recent bit. It follows from (9), (10) and the de nition of state that

E = kzk2 +

NX ?m k=m+1

(Sk?1 ; Sk);

(13)

where (:; :) is a function determined by (10) and where from the boundary, condition Sm = (0; : : :; 0). Since there is a one-to-one correspondence between sequences of bits [b1; : : :; bN ] and sequences of states [Sm ; : : :; SN ?m], one may minimize E by nding the state sequence

14

February 27, 1997

[Sm ; : : :; SN ?m] that minimizes (13), rather than nding the binary sequence that minimizes (9). It is then a trivial matter to derive the binary sequence [b1; : : :; bN ] from the state sequence. The Viterbi algorithm [39, 40] is an ecient way to nd the minimizing state sequence. (It is, essentially, a form of dynamic programming.) Let S denote the set of all possible states (the set of all binary sequences of length 2m +2n). For each k in the range m +1; m + 2; :::; N ? m and for each state s 2 S , the algorithm nds a state sequence [Sm+1 ; : : :; Sk?1 ; s] (ending at time k in state Sk = s) for which

kzk2 +

k X j =m+1

(Sj?1 ; Sj )

(14)

is minimum among all state sequences ending in s at time k. Let k (s) denote the minimizing state sequence, and let Ek (s) denote the resulting minimum value. Then the state sequence that minimizes (13) (i.e., the desired solution) is (N ?m (s); s), where s is the state for which EN ?m (s ) is smallest. For each k starting with k = m + 1 and each s 2 S , the algorithm nds Ek (s) and k (s) using the recursion:

E (s0) + (s0; s) Ek (s) = min k?1 s ?
k (s) = k (Sk?1 ); s 0

(15) (16)

where Sk?1 is the state s0 achieving the minimum in Ek (s) and where Sm = (0; :::; 0). In regard to the complexity of the algorithm, for any state s there are precisely two states that can precede it. Thus the minimization in (14) involves two computations of (
;
), an addition and a binary comparison. If sucient memory is available, the function  may be precomputed and saved as a matrix. In this case, the complexity of the algorithm, in operations per dot, is proportional to the number of states: Ns = 22m+2n . Thus, complexity increases exponentially with m and n, but is independent of the size of the image. It is intuitively clear that better models can be found for larger values of m and n, and that these should result in better halftoning. However, since complexity increases exponentially with m + n, one is limited by the hardware available for implementation. On the other hand, there are ways to reduce the number of states (complexity), at the cost of

15

February 27, 1997

some suboptimality, i.e., an increase in E . We have used the state reduction method that is possible in the Ungerboeck-Acampora approach [41, 42]. It is based on the following observations: The state at time k ? 1 was de ned in such a way that it contained all bits needed to determine vk?2m ; : : :; vk?1 , which in turn enter into the third term of (10), namely, kX ?1 vj Hk?j (17) 2vk j =k?2m

The summation may be viewed as a ltering of the vj 's by an FIR lter with impulse response H?2m ; : : :; H2m given by (11). Since, as illustrated in Fig. 3, the impulse response hk of the visual model lter tapers to zero, it follows that the impulse response Hk = hk  h?k also tapers to zero with an even longer tail of nearly zero values, as illustrated in Fig. 8. Indeed, some of the last terms of Hk , say H2m?t+1 ; : : :; H2m, are so small that the corresponding terms of the sum, vk?2m+t?1 H2m?t+1 ; : : :; vk?2m H2m , can be dropped from (17) without much eect. In this case, the state at time k may be rede ned as

Sk = (bk?2m?n+t+1 ; : : :; bk ; : : :; bk+n);

(18)

so that now there are only 22m+2n?t possible states. For the lter of Fig. 3, Fig. 8 suggests t can be chosen to be 9 or 10 with virtually no loss. Our experiments, described in the next section, indicate that even larger values of t are feasible. Further reductions in the size of the state-space could be obtained using methods such as that of Duel and Heegard [44]. To emphasize the bene ts of the Ungerboeck-Acampora approach, we mention that a direct implementation of the Viterbi algorithm would choose, as its state at time k, a sequence of 2m + 2n bits from which wk in (8) could be computed. While a reduced set of bits could be used to make an approximate solution, the fact that the impulse response hk does not have as many nearly zero trailing terms as Hk means that the complexity cannot be reduced as much as in the above approach. In other words, t cannot be chosen as large.

Two-Dimensional LSMB Halftoning Two-dimensional least-squares halftoning has been performed using iterative techniques. Such techniques nd a solution that is only a local optimum; the visual quality of the resulting halftone images depends on the initial estimate and the optimization strategy.

February 27, 1997

16

The initial estimate of the binary image could be provided by any halftoning algorithm, e.g. the modi ed error diusion [6], the one-dimensional version of which is brie y described in the next section, or it could be a constant or random image. In some simple iterative schemes [18, 20] the binary image is updated one pixel at a time, in a raster scan. In a similar scheme [22], each pixel of the binary image is either toggled or swapped with one of its 8 neighbors in a given scanning order. Other variations are possible, for example [21] uses swaps only. More sophisticated (and computationally intensive) schemes use simulated annealing [20, 45]. More details about the various schemes can be found in [46, 19].

5 Experimental Results To test the one-dimensional, least-squares model-based (LSMB) halftoning method, we applied it to the ve test images shown in Fig. 9. The size of the `lena', `bank' and `peppers' images is 256  256 (they are subimages of the 512  512 images) and the size of the `ramp' and `step' images is 390  60. For LSMB halftoning we added a white border of m + n pixels at the top and the bottom of each image, as required by the boundary conditions of the previous section. The gray levels in these images are integers ranging from 0 to 255, rather than real numbers from 0 to 1 (as in Sections 2-4), with 255 being black. This convention will be retained throughout this section. The gray levels of the `step' image are 128 and 64, and the gray levels of the `ramp' image range from 255 to 0. This section presents and assesses the results. As indicated earlier, the halftoning is applied to vertical columns. This usually creates a pattern of horizontal lines, which tends to be more visually pleasing than a pattern of vertical lines, created by halftoning horizontal lines. The rst two test images `ramp' and `step', are, eectively, one-dimensional. For this reason only one column must be halftoned and replicated to form a two-dimensional display image. The performance of the method depends strongly on the choices of printer and visual models. We rst consider the method with the following \baseline" choices: the dot-overlap printer model with  = :33, and the visual model of Fig. 2 with the lter shown in Table 1 and Fig. 3 and no nonlinearity. Fig. 10 shows the results of halftoning the ve test images with this baseline system. The printer model is intended for 300 dpi printing with the HP Laser Jet II. The visual model is intended for 300 dpi printing viewed at 30 inches. However, due to the fact that substantial additional distortion would be introduced by the journal

February 27, 1997

17

printing process, the halftoned images are printed at 100 dpi with simulation of .33 dot overlap, as described in Section 3. (The actual dot overlap is a little higher, as discussed there.) Viewing the image from seven and one half feet shows roughly how the visual model intends the image to be viewed. Alternatively, one may tip the page to reduce the apparent line spacing. The state reduction factor for our baseline system was conservatively chosen to be t = 7. Table 4 shows the per pixel squared error produced by the LSMB algorithm applied to the `ramp' and `lena' images for various values of t. One may see that choosing t as high as 10 has little eect on the mean-squared error. As we saw in the previous section, this is because the last 10 values of Hk are close to zero (see Fig. 8). Moreover, we found that the resulting halftoned `ramp' images are identical for t = 4; 5; 6; 7; there are minor dierences for t = 8; 9; 10. For the corresponding values of t, the halftoned `lena' images are nearly identical. As the value of t increases further, the dierences become more and more noticeable. At t = 15, the dierences are signi cant. We now describe two other types of halftoning and compare their performance with that of the above baseline system. First we consider ordered dither. It is a special case of screening, whereby the image array is compared, pixel by pixel, to an array of imageindependent thresholds [3]. A black dot is placed wherever the image gray level is greater than the corresponding threshold. In ordered dither the thresholds are periodic. More speci cally, the threshold array is generated by periodically replicating a threshold matrix. Here we consider a one-dimensional clustered ordered dither scheme. Fig. 11 shows the results of halftoning with a one-dimensional version of a classical screen, whose thresholds are given in Table 5. Notice that small thresholds are clustered, causing black dots (lines) to be clustered. The second type of halftoning we consider is error diusion, whereby each image pixel is compared to a threshold which depends upon \past" image pixels, usually above and to the left. Alternatively, each image pixel is compared to a xed threshold, after a correction factor is added to its original gray level to account for past \errors." Here we consider a one-dimensional version of the modi ed error diusion algorithm (MED) [6], whereby the past errors account for dot overlap as well as quantization eects. A block diagram is shown in Fig. 13. Let x = [x1 ; : : :; xN ] be the original image and b = [b1; : : :; bN ] the halftone image produced by MED. Without loss of generality, we assume that the image is scanned top to

18

February 27, 1997

bottom or left to right. The error ek is de ned as the dierence between the output of the printer model pk and the \corrected" image vk . Previous errors are ltered and subtracted from the current image value xk to obtain the \corrected" value of the gray scale image. The MED equations are

X

vk = xk ? hn ekk?n ( 0; ifn v > t k bk = 1; otherwise

where

(19) (20) (21)

ekn = pkn ? vn for n < k

(22)

pkn = P (bk?n; : : :; bk+n ) for n < k

(23)

is a sliding-window printer model of the form (5), but here only the points bk?n ; : : :; bk?1 have been determined; bk ; : : :; bk+n are assumed to be zero (white). Since only the dotoverlap contributions of the \past" pixels can be used in (23), the \past" errors keep getting updated as more pixel values are computed. Hence the dependence of the error and the printer model output on k. The assumption that the undetermined pixels are white leads to a small bias in the gray scale of the printed image. A multi-pass version of the MED algorithm (MP-MED) eliminates this bias [7, 8]. Fig. 12 shows the results of halftoning with the one-dimensional MP-MED algorithm with the .33 dot-overlap model and the lter shown in Fig. 14. Table 3 lists the per pixel squared error of the ve test images halftoned by the three dierent techniques. We evaluate the performance of the LSMB technique in terms of (i) spatial resolution (i.e. sharpness), (ii) texture (i.e. visibility of halftone patterns), (iii) gray-scale resolution (i.e. number of perceived gray levels), and (iv) gray-scale distortion of the halftoned images. As we will see below, some of these attributes can be interdependent. (i) The spatial resolution of a halftoning algorithm is its ability to follow rapid changes in image gray level, for example where an edge causes a step change. In this regard, LSMB does an exceptional job, particularly at edges. This is especially noticeable in the `step' and the `bank' images, which look sharper even than conventional halftoning with the twodimensional version of classical ordered clustered dither (not shown). This sharpness is due

February 27, 1997

19

to the noncausality of the approach, i.e. the ability of the method to look ahead to see where dots should best be placed. Such precise edge tracking causes the halftoned patterns of adjacent columns to be nearly synchronized, which in turn causes the wavy horizontal line character of the reproductions, and also contributes to false contouring (discussed below). MED also has the freedom to make sharp transitions and, therefore, produces results that are very sharp. It is causal, however, and thus the transitions do not always occur exactly at the boundaries. This accounts for the fact that the halftoned columns do not synchronize as much as with LSMB, so the wavy horizontal lines are not as prevalent, and less false contouring occurs. (ii) The texture of a halftoning algorithm refers to the visibility of the dot patterns that the algorithm creates that are not part of the original image. These patterns should have few low frequency components since they are the most visible. A typical halftoning algorithm produces a set of characteristic dot patterns. For example, in regions of constant gray level, ordered dither algorithms produce a nite set of periodic patterns, as can be seen in the `step' image of Fig. 11. The characteristic patterns of the various halftoning methods are evident in the halftoned versions of the `ramp' image. By comparing these, we see that the one-dimensional LSMB algorithm produces periodic patterns, whereas the MED does not. This is not true in the two-dimensional case [18, 19]. The texture of these patterns is generally better for LSMB and MED than for classical screening, principally because the latter's clustering of dots (lines) introduces more low frequency energy. In addition, MED produces some undesirable artifacts that LSMB does not. For example, MED produces highly visible, widely spaced lines at the bottom of `ramp', whereas LSMB does not, precisely because the visual model predicts that such lines would be visible and, consequently, would increase the squared error. As another example, MED produces a number of light and dark stripes in the `step' image. This is a result of the diusion of \error" over the image. It does not happen with LSMB or classical screening. (iii) The gray-scale resolution of a halftoning algorithm is the number of dierent perceived gray levels that it can generate. More speci cally, it is the ability of the algorithm to render small changes in gray level. It is easy to see from the halftoning of `ramp', that the classical screen of Table 5 produces seven characteristic patterns, with seven distinct gray levels. On the other hand LSMB successfully exploits dot-overlap to produce about ten dierent gray levels, a signi cant increase. Moreover, this is close to the eleven gray

February 27, 1997

20

levels, shown in the last column of Table 2, that are produceable by the dot-overlap model with patterns of period six. The number six represents, approximately, the number of dots over which the eye averages when viewing 300 dpi printing from 30 inches [6]. Thus, eleven is approximately the maximum number of gray levels that are both produceable by the dot-overlap model and distinguishable by the eye. Notice that classical screening produces uniformly spaced gray levels, while the LSMB algorithm produces more shades of gray in the dark areas of the `ramp' than in the light areas. This is because LSMB produces whatever gray levels the printer and visual models permit. An interesting interaction between gray-scale resolution and texture becomes evident when we look at regions of slowly changing gray level. The primary concern here is that of false contouring, which occurs when the halftoned output changes abruptly from one characteristic pattern to another along a line of constant gray level, though the image changes only slightly. This can be seen in the `ramp,' `lena,' and `peppers' images of Fig. 11. False contouring is less noticeable with LSMB than with classical screening. This is principally due to the former having more gray levels. Observe that with MED, false contouring is even less noticeable than in the LSMB `ramp' image. However, this is because the MED algorithm produces patterns that are inherently more random, not because it produces more gray levels. In two-dimensional LSMB halftoning the false contours disappear [18, 19], as they do in the two-dimensional version of error diusion [5, 6]. This is partly because of the increased numbers of gray levels, but also because one-dimensional LSMB halftoning lacks the exibility to break up false contours. A typical way to reduce false contouring, is to increase the randomness of the patterns, i.e. to add texture. This can be done by adding a small amount of white (uniformly distributed) noise to each pixel in the image before halftoning [2]. In the case of ordered dither, an alternative way to eliminate false contouring is by using error diusion between adjacent blocks of pixels [47, 48, 49]. This should not be confused with the pixel by pixel error diusion algorithm. In summary, false contouring is caused by lack of gray-scale resolution, and can be hidden by a sucient amount of texture, albeit at the expense of image graininess. (iv) The gray-scale distortion introduced by a halftoning technique is the degree to which the gray level of the displayed image diers from the desired gray level. An approximation of the gray-scale distortion of a halftoning technique on a particular device can be obtained by measuring and plotting the output of the display device in response to halftoned regions of

February 27, 1997

21

dierent constant gray levels, as shown in Fig. 15. The dashed line shows the ideal behavior of a halftoning technique. The solid line indicates the kind of nonlinearity caused by dot overlap, in that it lies above the ideal line. As mentioned in the introduction, such plots can be used to compensate for printer distortions [4]. Gray-scale distortions are mainly due to inadequate modeling of the characteristics of the display device and they are relatively independent of the characteristics of vision. Printer distortions, in particular, have a large eect on the appearance of a halftoned image. They distort the linearity and even the monotonicity of the gray scale. This is especially true for dispersed ordered dither techniques and standard error diusion in the presence of distortions such as dot overlap. The classical scheme reduces the eects of printer distortions by printing dots in clusters, but this sacri ces spatial resolution and generates more low frequency artifacts. The LSMB approach minimizes printer distortions and, moreover, it exploits them to increase the number of gray levels, while maintaining high spatial resolution. Indeed, the gray scale of the `ramp' of Fig. 10 is fairly linear. The same is true for the the `ramp' produced by MED. Although it cannot be accurately printed in a journal article, we have observed that the 300 dpi halftoned output of `ramp' is not entirely monotonic, which is indicative of the fact that the dot-overlap model leaves room for improvement. Better models can be found in [8]. However, the monotonicity at 100 dpi shown in Fig. 10 indicates that the LSMB method is doing its job. We now discuss the results of several modi cations to the baseline model parameters. First, we experimented with nonlinearities in the visual model having the power law form given in (4). We did not nd, however, that it oered signi cant improvements, and sometimes, it even introduced undesirable distortions. Second, we experimented with a number of dierent visual model lters. Most interesting are the results of modifying the lter to account for dierent viewing distances (with 300 dpi printing). To facilitate this, we used the Gaussian lter, suggested in Section 2, with standard deviation 1:5d=2:5, where d is the viewing distance in feet. Note that since p the lter is Gaussian, Hk is also Gaussian with standard deviation equal to 2 times the standard deviation of the visual model lter. We chose the state reduction factor t so that all points of Hk beyond three standard deviations are dropped. Note that the lter width 2m + 1 should be chosen to be large; the exact value is not important. Figures 16 and 17 show the result of LSMB halftoning with visual model lters tuned to various viewing

February 27, 1997

22

distances. The ramp in Fig. 17 is 512  60. Table 6 lists the corresponding per pixel squared errors. As expected, the per pixel squared error decreases with increasing viewing distance. In several ways these gures demonstrate the ability of the LSMB method to eectively tailor the halftoning to the visual model. As the viewing distance becomes larger, the eye has the ability to average over more dots, and therefore to perceive more gray levels. Correspondingly, the central lobe of the impulse response of the Gaussian lter gets wider and we see that the halftoned `ramp' shows increasing numbers of gray levels, with smaller steps between them. Also, we see thicker black and white spaces, precisely because from large distances the eye can smooth them, while from small distances it cannot. Related to this is the distance between the last two black lines at the bottom of each `ramp'. For large viewing distances, the eye can smooth widely spaced lines, so the LSMB produces such at the bottom of `ramp' to reproduce a very light gray tone. On the other hand, at small viewing distances, the eye can distinguish relatively closely spaced lines; thus the last pair of black lines produced by the LSMB algorithm are fairly close together, representing a not so light gray tone. All lighter gray tones are simply approximated by pure white. Hence, the halftones on the left are truncated. A similar observation can be made at the top of each ramp. The problem is not symmetric, however, because of the dot-overlap. Isolated white lines at the top of the ramp are thinner (approx. 33% of a pixel wide) than isolated black lines at the bottom (approx. 166% of a pixel wide). Similar eects occur, but are not as easily noticeable in Fig. 16. Third, a considerable number of experiments with dierent printer models are described in [8]. Here, we simply mention that Fig. 18 shows the result of halftoning a 512  60 grayscale ramp without a printer model; i.e. with an ideal model that predicts pk = 255bk . The gure is printed at 100 dpi without simulated dot overlap. The per pixel squared error is listed in Table 6. Again, this gure demonstrates the ability of the LSMB method to eectively tailor the halftoning to the visual model. Notice that now the halftoned ramps are approximately symmetrical. This example also con rms that for a printer with dot overlap the LSMB halftoning algorithm produces more dark shades of gray and fewer light shades of gray than for a printer with no dot overlap. This is consistent with Table 2, and is shown more explicitly in Fig. 19, which plots the pro les of the perceived original and halftone images (according to our models) for a viewing distance of three feet, with and without dot overlap. In both

February 27, 1997

23

plots, the pro le of the perceived halftone is shown with a solid line and the perceived original with a dashed line. As shown in Table 6, the overall squared error for the `ramp' image is larger with dot overlap than without (except for the viewing distance of one foot). This is because the squared error penalizes the larger dierences between the perceived original and halftone images in the light regions of the model with dot overlap more than the somewhat smaller dierences in both the light and dark regions of the model without dot overlap. Note also that it is because of the optimality of the one-dimensional LSMB algorithm that we can attribute such observations to the printer and visual models rather than the optimization scheme.

Two-dimensional LSMB We close this section with a brief overview of the performance of two-dimensional LSMB algorithms. As we saw in the previous section, two-dimensional LSMB algorithms are iterative and nd solutions that are only local optima. Thus, the performance of the LSMB method depends on the starting point and the optimization strategy. It also depends on the choice of printer and eye models. For most of the iterative schemes we tried in [46, 19], we found that the MP-MED starting points result in the lowest error and the best visual quality. We also found that dierent starting points are tuned to dierent viewing distance ranges. For example, for a 300 dpi printer with  = 1:25 dot-overlap, we found that at short viewing distances (1 to 1.5 feet), the MP-MED start with a Floyd-Steinberg lter results in the lowest error, while at medium distances (2 to 3 feet), the Jarvis-Judice-Ninke start results in the lowest error [46, 19]. Depending on the optimization scheme, other starting points, and especially random starts result in patterns that look considerably grainier. In [46, 19] we also found that we can control the amount of sharpening by changing the characteristics of the eye lter that corresponds to the continuous-tone image (the other eye lter is xed). As expected, the narrower the impulse response of the lter the sharper the halftone image. Since the two-dimensional algorithms have many more degrees of freedom in selecting binary patterns, they result in much better halftones than one-dimensional techniques. Thus, the two-dimensional MED and LSMB techniques generate more gray levels (even at short viewing distances) and do not suer from false contouring. As the viewing distance increases, one expects an increase in the number of perceived gray levels at the expense

February 27, 1997

24

of coarser textures (which become less visible) and reduced image detail. However, this is very dicult to verify because the results are only approximate and one cannot be sure that resulting textures are optimal. In contrast, in the one-dimensional case, since the LSMB solution can be obtained using the Viterbi algorithm, we were able to show that the optimal textures get coarser as the viewing distance increases. In general, we found that the LSMB metric predictions agree with visual evaluations of image quality. However, it also has serious limitations because it is very dicult to capture image quality using a single number. For example, this metric is aected by image sharpness, gray-scale resolution, texture artifacts, geometric and gray-scale distortions, etc. In addition, it is very sensitive to displacement of the original and halftone images and uniform brightness shifts, even though neither aects image quality signi cantly. On the other hand, when this metric is used carefully, it agrees remarkably well with human perception and provides a powerful tool for algorithm development and evaluation.

6 Conclusions We presented a one-dimensional model-based halftoning technique that minimizes the squared error between the response of a visual model to the original image and the response of visual and printer models to the halftoned image. This technique, which uses the Viterbi algorithm to perform the minimization, exploits the models to produce halftones with high spatial and gray-scale resolution. While two-dimensional halftoning is generally desired, the results of this one-dimensional study indicate the true limits of the least-squares modelbased approach, in that it is only in the one-dimensional case that the minimization can be precisely achieved, and consequently, that the perceptual and printing models can be fully exploited.

February 27, 1997

25

References [1] D. L. Neuho and T. N. Pappas, \Perceptual coding of images for halftone display," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, vol. 4, (Toronto, Canada), pp. 2797{2800, May 1991. (Publisher: Institute of Electrical and Electronics Engineers, Piscataway, NJ.) [2] D. L. Neuho and T. N. Pappas, \Perceptual coding of images for halftone display," IEEE Trans. Image Proc., vol. 3, pp. 341{354, July 1994. [3] R. Ulichney, Digital Halftoning. Cambridge, MA: The MIT Press, 1987. [4] C. J. Rosenberg, \Measurement-based veri cation of an electrophotographic printer dot model for halftone algorithm dot correction," in IS&T's 8th Int. Cong. Adv. NonImpact Printing Techn., (Williamsburg, VA), pp. 286{291, Oct. 25-30, 1992. (Publisher: The Society for Imaging Science and Technology, Spring eld, VA). [5] T. N. Pappas and D. L. Neuho, \Model-based halftoning," in Proc. SPIE, vol. 1453, Human Vision, Visual Proc., and Digital Display II, (San Jose, CA), pp. 244{255, Feb. 1991. [6] T. N. Pappas and D. L. Neuho, \Printer models and error diusion," IEEE Trans. Image Proc., vol. 4, pp. 66{80, Jan. 1995. [7] C.-K. Dong, \Measurement of printer parameters for model-based halftoning," B.S./M.S. Thesis, MIT, May 1992. [8] T. N. Pappas, C.-K. Dong, and D. L. Neuho, \Measurement of printer parameters for model-based halftoning," Journal of Electronic Imaging, vol. 2, pp. 193{204, July 1993. [9] J. P. Allebach, \Visual model-based algorithms for halftoning images," in Proc. SPIE, vol. 310, Image Quality, pp. 151{158, 1981. [10] D. Anastassiou, \Error diusion coding for A/D conversion," IEEE Trans. Circ. Sys., vol. CAS-36, pp. 1175{1186, Sept. 1989. [11] D. Anastassiou and S. Kollias, \Digital image halftoning using neural networks," in Proc. SPIE, vol. 1001, Visual Communications and Image Processing, 1988.

February 27, 1997

26

[12] P. G. Roetling and T. M. Holladay, \Tone reproduction and screen design for pictorial electrographic printing," Journal of Applied Phot. Eng., vol. 15, no. 4, pp. 179{182, 1979. [13] J. P. Allebach, \Binary display of images when spot size exceeds step size," Applied Optics, vol. 19, pp. 2513{2519, Aug. 1980. [14] R. W. Pryor, G. M. Cinque, and A. Rubinstein, \Bilevel displays { a new approach," in Proc. SID, vol. 19, pp. 127{131, 1978 (publisher: Society for Information Display, Playa del Rey, CA). [15] P. Stucki, \MECCA { a multiple-error correcting computation algorithm for bilevel image hardcopy reproduction," Research Report RZ1060, IBM Research Laboratory, Zurich, Switzerland, 1981. [16] P. Stucki, \Advances in digital image processing for document reproduction," in VLSI Engineering (T. L. Kunii, ed.), pp. 256{302, Tokyo: Springer-Verlag, 1984. [17] T. N. Pappas, \Model-based halftoning of color images," IEEE Trans. Image Proc. Scheduled to appear in the July 1997 issue (vol. 6, no. 7). [18] T. N. Pappas and D. L. Neuho, \Least-squares model-based halftoning," in Proc. SPIE, vol. 1666, Human Vision, Visual Proc., and Digital Display III, (San Jose, CA), pp. 165{176, Feb. 1992. [19] T. N. Pappas and D. L. Neuho, \Least-squares model-based halftoning," submitted for publication to IEEE Trans. Image Proc. [20] J. B. Mulligan and A. J. Ahumada, Jr., \Principled halftoning based on models of human vision," in Proc. SPIE, vol. 1666, Human Vision, Visual Proc., and Digital Display III, (San Jose, CA), pp. 109{121, Feb. 1992. [21] M. Analoui and J. P. Allebach, \Model based halftoning using direct binary search," in Proc. SPIE, vol. 1666, Human Vision, Visual Proc., and Digital Display III, (San Jose, CA), pp. 96{108, Feb. 1992. [22] D. A. Carrara, M. Analoui, and J. P. Allebach, \Recent progress in digital halftoning," in IS&T's 8th Int. Cong. Adv. Non-Impact Printing Techn., (Williamsburg, VA),

February 27, 1997

27

pp. 265{270, Oct. 25-30, 1992. (Publisher: The Society for Imaging Science and Technology, Spring eld, VA). [23] T. J. Flohr, C. B. Atkins, and J. P. Allebach, \Can DBS ever be a practical halftoning technique," in SID Digest of Technical Papers, (Seattle, WA), pp. 143{146, May 18{20, 1993. (Publisher: Society for Information Display, Playa del Rey, CA). [24] A. Zakhor, S. Lin, and F. Eska , \A new class of B/W and color halftoning algorithms," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, vol. 4, (Toronto, Canada), pp. 2801{2804, May 1991. (Publisher: Institute of Electrical and Electronics Engineers, Piscataway, NJ.) [25] A. Zakhor, S. Lin, and F. Eska , \A new class of B/W halftoning algorithms," IEEE Trans. Image Proc., vol. 2, pp. 499{509, Oct. 1993. [26] R. Geist, R. Reynolds, and D. Suggs, \A markovian framework for digital halftoning," ACM Transactions on Graphics, vol. 12, pp. 136{159, Apr. 1993. [27] J. L. Mannos and D. J. Sakrison, \The eects of a visual delity criterion on the encoding of images," IEEE Trans. Inform. Theory, vol. IT-20, pp. 525{536, July 1974. [28] A. K. Jain, Fundamentals of Digital Image Processing. Englewood Clis, NJ: Prentice Hall, 1989. [29] C. F. Hall and E. L. Hall, \A nonlinear model for the spatial characteristics of the human visual system," IEEE Trans. Sys., Man, and Cyber., vol. SMC-7, pp. 162{170, Mar. 1977. [30] B. A. Wandell, Foundations of Vision. Sunderland, MA: Sinauer, 1995. [31] A. N. Netravali and B. G. Haskell, Digital Pictures: Representation and Compression. New York: Plenum, 1988. [32] T. J. Stockham, \Image processing in the context of a visual model," Proc. IEEE, vol. 60, pp. 828{842, July 1972. [33] F. W. Campbell and J. G. Robson, \Application of Fourier analysis to the visibility of gratings," J. Physiology, vol. 197, pp. 551{566, 1968.

February 27, 1997

28

[34] N. Graham and J. Nachmias, \Detection of grating patterns containing two spatial frequencies: A comparison of single-channel and multiple-channels models," Vision Res., vol. 11, pp. 252{259, 1971. [35] D. J. Sakrison, \On the role of the observer and a distortion measure in image transmission," IEEE Trans. Commun., vol. COM-25, pp. 1251{1267, Nov. 1977. [36] N. Graham, \Spatial-frequency channels in human vision: Detecting edges without edge detectors," in Visual Coding and Adaptivity (C. Harris, ed.), pp. 215{262, Hillsdsale, NJ: Erlbaum, 1980. [37] S. S. Stevens, \Neural events and the psychophysical law," Science, vol. 170, pp. 1043{ 1050, 1970. [38] R. J. Mans eld, \Visual adaptation: Retinal transduction, brightness and sensitivity," Vision Research, vol. 16, pp. 679{690, 1976. [39] A. J. Viterbi, \Error bounds for convolutional codes and an asymptotically optimum decoding algorithm," IEEE Trans. Inform. Theory, vol. IT-13, pp. 260{269, Apr. 1967. [40] G. D. Forney, Jr., \The Viterbi algorithm," Proc. IEEE, vol. 61, pp. 268{278, Mar. 1973. [41] G. Ungerboeck, \Adaptive maximum-likelihood receiver for carrier-modulated datatransmission systems," IEEE Trans. Commun., vol. COM-22, pp. 624{636, May 1974. [42] A. S. Acampora, \Maximum-likelihood decoding of binary convolutional codes on band-limited satellite channels," in National Telecommunications Conference, (Dallas, TX), pp. 24.5-1{24.5-5, Nov. 29-Dec. 1, 1976. (Publisher: Institute of Electrical and Electronics Engineers, Piscataway, NJ.) [43] A. J. Viterbi and J. K. Omura, Principles of Digital Communications and Coding. New York: McGraw-Hill, 1979. [44] A. Duel-Hallen and C. Heegard, \Delayed decision feedback sequence estimation," IEEE Trans. Commun., vol. COM-37, pp. 428{436, May 1989.

February 27, 1997

29

[45] M. A. Schulze and T. N. Pappas, \Blue noise and model-based halftoning," in Proc. SPIE, vol. 2179, Human Vision, Visual Proc., and Digital Display V, (San Jose, CA), pp. 182{194, Feb. 1994. [46] T. N. Pappas, \Model-based techniques for digital halftoning," in Proc. Int. Conf. Image Processing, vol. II, (Austin, TX), pp. 26{30, Nov. 1994. (Publisher: IEEE Computer Society Press, Los Alamitos, CA.) [47] G. Goertzel and G. R. Thompson, \Digital halftoning on the IBM 4250 printer," IBM J. Res. Develop., vol. 31, pp. 2{15, Jan. 1987. [48] Y.-H. Chen, F. C. Mintzer, and K. S. Pennington, \Panda: Processing algorithm for noncoded document acquisition," IBM J. Res. Develop., vol. 31, pp. 32{43, Jan. 1987. [49] Z. Fan, \Halftoning by combining ordered dithering and error diusion," in Proc. IS&T's 8th Int. Congress on Advances in Non-Impact Printing Technologies, (Williamsburg, VA), pp. 277{279, Oct 25-30, 1992. (Publisher: The Society for Imaging Science and Technology, Spring eld, VA).

February 27, 1997

30

List of Figures 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

18

19

The spatial frequency sensitivity of the visual system. : : : : : : : : : : : : A simple visual model. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : Impulse and frequency response of the selected lter model. : : : : : : : : : Periodic test patterns. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : One-dimensional dot-overlap model. : : : : : : : : : : : : : : : : : : : : : : Simulation of  = :33 dot overlap @ 100 dpi using a 300 dpi printer (without dot overlap. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : The visual and printer perceptual models. : : : : : : : : : : : : : : : : : : : Hk = hk  h?k for the lter of Figure 3. : : : : : : : : : : : : : : : : : : : : Test images: `bank', `lena', `peppers', `step', and `ramp'. : : : : : : : : : : : One-dimensional least-squares model-based halftoning, printed at 100 dpi with  = :33 dot-overlap simulation. : : : : : : : : : : : : : : : : : : : : : : One-dimensional classical clustered ordered dither, printed at 100 dpi with  = :33 dot-overlap simulation. : : : : : : : : : : : : : : : : : : : : : : : : : One-dimensional multi-pass modi ed error diusion, printed at 100 dpi with  = :33 dot-overlap simulation. : : : : : : : : : : : : : : : : : : : : : : : : : One-dimensional modi ed error diusion : : : : : : : : : : : : : : : : : : : : Impulse response of one-dimensional error diusion lter. : : : : : : : : : : The gray scale of a typical halftoning technique. : : : : : : : : : : : : : : : LSMB halftoning of `lena' at viewing distances of 1, 2, 3, 4, 5, and 6 feet, printed at 100 dpi with  = :33 dot-overlap simulation. : : : : : : : : : : : : LSMB halftoning of `ramp' at viewing distances of 1, 2, 3, 4, 5, 6, and 7 feet, and MP-MED halftoning in the last column, printed at 100 dpi with  = :33 dot-overlap simulation. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : LSMB halftoning of `ramp' at viewing distances of 1, 2, 3, 4, 5, 6, and 7 feet, and MP-MED halftoning in the last column, all without a printer model, printed at 100 dpi without dot-overlap simulation. : : : : : : : : : : : : : : Pro les of the perceived original (dashed line) and halftone images (solid line) for a viewing distance of three feet. (a) with dot overlap. (b) without dot overlap. : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :

33 33 34 34 35 35 36 36 37 38 39 40 41 41 41 42

43

44

45

31

February 27, 1997

sample 0 1 2 3 4 5 6 7 8 9

value .272436 .217089 .113336 .039878 .006827 -.003496 -.004864 -.003601 -.002130 -.001069

Table 1: Impulse response of the selected eye lter model. (The impulse response is symmetric around the origin; only the terms on the positive axis are shown.)

Pattern

Frequency of 1's

000000 100000 100100 101000 110000 101010 101100 111000 110110 101110 111100 111110 111111

0.00 0.17 0.33 0.33 0.33 0.50 0.50 0.50 0.67 0.67 0.67 0.83 1.00

Dot-overlap 1?Measured Model Prediction Re ectance  = 0:33 0.00 0.00 0.28 0.45 0.55 0.80 0.55 0.71 0.44 0.56 0.83 0.93 0.72 0.79 0.61 0.63 0.89 0.88 0.89 0.87 0.78 0.71 0.94 0.85 1.00 0.95

Table 2: Gray levels of periodic patterns.

32

February 27, 1997

`bank' `lena' `peppers' `step' `ramp' LSMB 167.60 176.98 170.71 335.91 235.30 classical 1285.75 1506.95 1401.36 1013.49 1102.60 MP-MED 364.89 388.01 349.20 481.04 465.43 Table 3: Per pixel squared error for dierent techniques.

t `ramp' `lena'

4 5 6 7 8 9 10 11 12 13 14 15

235.30 235.30 235.30 235.30 235.36 235.91 239.40 246.99 256.62 260.14 285.49 326.08

176.98 176.98 176.98 176.98 177.03 177.57 179.62 186.81 199.98 206.76 238.07 281.79

Table 4: Per pixel squared error for dierent values of the state reduction factor t. 219 146 73 36 109 182 Table 5: Thresholds of one-dimensional CLASSIC ordered clustered dither. `ramp' `ramp' d `lena' (feet) ( = :33) ( = :33) ( = 0:) 1 2060.21 1637.88 2084.63 2 374.18 426.43 265.90 3 93.20 148.86 78.17 4 27.41 64.15 32.14 5 8.96 32.18 15.31 6 5.34 19.75 9.76 7 { 14.55 5.63 Table 6: Per pixel squared error for dierent viewing distances.

33

February 27, 1997

1.0 visual system sensitivity (Mannos and Sakrison)

0.8 0.6 0.4 0.2 0.0 0

20

40

60

80

spatial frequency in cycles/degree

Figure 1: The spatial frequency sensitivity of the visual system.

x

image

-

n(x)

memoryless nonlinearity

y

-

h

lter

Figure 2: A simple visual model.

z

perceived image

-

34

February 27, 1997

• •

0.2

model impulse response

•

τ = .0064 degrees •

0.1

•

•

0.0

•

•

•

•

•

•

τ

•

-.038

•

•

0

•

•

•

•

.038

degrees

1.0 0.8 0.6 0.4 0.2 0.0

visual system sensitivity (Mannos and Sakrison)

.......................................................... ... . . ... .. .. .. ... .. model frequency response .. .. .. .. ... . . .. .. .. .. .. ... .. .. .. f s = 1/τ = 157.1 cycles/degree .. .. ... ... ... .. ... .. . .. .. ... ... .... fs / 2 .. . . . . .... . .................................. .... . .... . .... . . . . . . ... | ....

0

20

40

60

80

spatial frequency in cycles/degree

Figure 3: Impulse and frequency response of the selected lter model.

100100

101000

101100

Figure 4: Periodic test patterns.

35

February 27, 1997

bk

6 

1



0







1





2

3

4

5

6

7

2T

3T

4T

5T

6T

7T

-

k

-

t

-

t

-

k

u(t)

6

1

- T  0

T

u ~(t)

6

1 2

 0

T

2T

3T

4T

5T





6T

7T

pk

6 

1

 



0 1

2

3

4

5

6

7

Figure 5: One-dimensional dot-overlap model.

6 1

0

1/300 inch



-



1/100 inch

-

-

Figure 6: Simulation of  = :33 dot overlap @ 100 dpi using a 300 dpi printer (without dot overlap.

36

February 27, 1997

eye model - nonlinearity k - lter

xk

y

n(xk )

bk

-

printer model

P ( k?n b

; : : : ; bk

v

+n )

n(pk )

-

wk

-

hk

eye model - nonlinearity k - lter

pk

zk

hk

Figure 7: The visual and printer perceptual models.

Hk sample spacing = .0064 degrees 0.2 •

•

•

• •

0.1 •

•

•

0.0

•

• • • • • • • • • • • • • •

-.038

•

0

• • • • • • • • • • • • •

.038

Figure 8: Hk = hk  h?k for the lter of Figure 3.

degrees

February 27, 1997

Figure 9: Test images: `bank', `lena', `peppers', `step', and `ramp'.

37

February 27, 1997

38

Figure 10: One-dimensional least-squares model-based halftoning, printed at 100 dpi with

 = :33 dot-overlap simulation.

February 27, 1997

39

Figure 11: One-dimensional classical clustered ordered dither, printed at 100 dpi with  = :33 dot-overlap simulation.

February 27, 1997

40

Figure 12: One-dimensional multi-pass modi ed error diusion, printed at 100 dpi with

 = :33 dot-overlap simulation.

41

February 27, 1997

xk

vk

-m ? 6 +

- threshold

bk

P(

bk

t

pk

- printer model ?

n;

: : : ; bk

+

n

-

)

?- ?m+ ek

low-pass lter  hi;j

Figure 13: One-dimensional modi ed error diusion 7 12



5 12

Figure 14: Impulse response of one-dimensional error diusion lter.

perceived gray level

255 measured

160

5

expected 80

0

image gray level 0

80

160

255

Figure 15: The gray scale of a typical halftoning technique.

February 27, 1997

42

Figure 16: LSMB halftoning of `lena' at viewing distances of 1, 2, 3, 4, 5, and 6 feet, printed at 100 dpi with  = :33 dot-overlap simulation.

February 27, 1997

43

Figure 17: LSMB halftoning of `ramp' at viewing distances of 1, 2, 3, 4, 5, 6, and 7 feet, and MP-MED halftoning in the last column, printed at 100 dpi with  = :33 dot-overlap simulation.

February 27, 1997

44

Figure 18: LSMB halftoning of `ramp' at viewing distances of 1, 2, 3, 4, 5, 6, and 7 feet, and MP-MED halftoning in the last column, all without a printer model, printed at 100 dpi without dot-overlap simulation.

45

February 27, 1997

255 200 150 100 50 0 0

100

200

300

400

511

300

400

511

(a)

255 200 150 100 50 0 0

100

200 (b)

Figure 19: Pro les of the perceived original (dashed line) and halftone images (solid line) for a viewing distance of three feet. (a) with dot overlap. (b) without dot overlap.