A Feature-Based Pencil Drawing Method - Semantic Scholar

A Feature-Based Pencil Drawing Method Nan Li and Zhiong Huang Department of Computer Science School of Computing National University of Singapore Singapore 117543 {linan, huangzy}@comp.nus.edu.sg Abstract

Durand et al. [2] decouple strokes and high-level attributes for interactive drawing. In [5], a method using line integral convolution for pencil drawing is proposed. In [11, 12], models were proposed of pencil, paper and how lead pencils interact with drawing paper through a careful investigation of the real drawings using scanning electron microscope. The pen-and-ink model is discussed in [7, 14]. The work on brush painting is also relevant to pencil drawing. Shiraishi and Yamaguchi [9] used a three-step local source image approximation method to generate strokes: stroke distribution, stroke generation and stroke ordering and painting. Haeberli [3] developed an interactive painting system for quickly producing painted representations of still images. Hertzmann [4] used brush strokes represented by spline curves in several layers: smaller brush sizes for upper layers and bigger sizes for lower ones. In previous methods of generating 2D pencil drawings from a 2D digital photo, where our work is addressed, the pencil stroke attributes are determined either only by the stroke location point or by the local region color information, different from ours.

We present a method for generating a pencil drawing from a digital image using the feature geometric attributes obtained by analysis of image moment and texture of each region. Thus, the drawing results represent not only the local region information but also the feature characteristics. It is inspired and devised to correspond to a real pencil drawing process: A painter often divides a scene into regions, observes the distinctive features, and applies the pencil strokes accordingly. We have implemented the method and compared the results with Adobe Photoshop 5.0. CR Categories: I.3.3 (Picture/Image Generation): Display Algorithms; I.4.6 (Segmentation): Pixel Classification Keywords: Non-photo realistic rendering, Enhanced realistic, Image moment function, Texture analysis

1

Introduction

Non-photo realistic rendering (NPR) refers to any techniques which can produce a non-photo realistic image. There are three distinct types: direct rendering of 3D scenes, transformation from a photo, and interactive drawing. This paper focuses on the second one. Our method automatically generates images in the style of pencil drawings. In this method, feature region geometric information is employed. The contribution of our work is a novel way to use the extracted features from a source image to guide the pencil drawing to achieve a feature-driven rendering result. The results of our method represent not only the local region information but also the feature characteristics. This method makes the drawing process correspond to a real process, where a painter often divides a scene into regions, observes the distinctive features, and applies the pencil strokes accordingly.

2

3

Our Method

Our method considers the stroke location point and the local region color information together, a combination of the previous work of brush painting and pencil drawing. In our method, the feature region attributes are used to determine the vector field that controls the pencil stroke orientation and the level of filtering. Using the line integral convolution to produce pencil drawing, vector field affects the overall appearance of the pencil drawing. Stroke direction is very important for conveying the shapes and textures of objects. If a texture presents a directional feature, the strokes should be oriented in a direction matching that of the original texture. Texture directionality analysis is done based on each feature region. If a high directionality exists for one region, the vector field in that region is produced following that texture direction. Otherwise a random direction vector field is generated in that region. The algorithm of generating pencil drawing is as follows:

Related Work

Several existing NPR techniques address the pencil drawings. In an early 2D painting system called PencilSketch [13], a mouse based virtual tablet is provided for interactive drawing. Recently, ___________________________

(1) Use color-based image segmentation to subdivide the source image into different regions. (2) Each region is considered as a feature region, whose boundary is extracted. (3) Apply image moment functions and texture analysis to obtain the feature geometric attributes. (4) White noise image is generated from original image. (5) Apply the feature geometric attributes to generate vector field. (6) Use line integral convolution (LIC) to generate the final rendering.

Copyright © 2003 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail [email protected]. © 2003 ACM 1-58113-578-5/03/0002 $5.00

We now describe each step in detail in the following subsections.

135

Geometric moment of pth degree about the x-axis and qth degree about the y-axis is defined as,

3.1 Image segmentation This step is to segment the image so that each resulting region contains one dominant color or texture. We use color-based image segmentation method to extract features. Then, we merge all regions whose size are smaller than a given threshold into their neighboring large regions.

M pq =

x p y q I ( x, y ) dxdy ,

p,q=0,1,2,…

(3)

ζ

where ζ is the region of the pixel space in which the image intensity function I(x,y) is defined. The immediate digital approximation of Eq. (3) for computing the moment of an NxM image would be

After segmentation, each feature region Fi in the result map has a label i, and all pixels inside the feature region are marked with that label i. Fig 1 shows the label map after image segmentation.

M pq =

N

M

(4)

x p y q I ( x, y ).

x =1 y =1

Another method called δ-method is used for non-rectangular region. The image region defined over a closed convex region ζ can be represented by the set, (, yk), k = 0, 1, 2,…, n, where, x1k and x2k denote the x-coordinates of the boundary points of the kth image row having an ordinate value yk. The image moment can be written as,

M pq =

n

x2 k

k =1 x = x1k

(5)

q

x p yk I ( x, yk ).

A rectangle to approximate the image with centroid (x0, y0), w (shorter edge), h (longer edge) and angle θ between h and the xaxis is given by the following equations:

Fig 1: Feature region after segmentation.

3.2 Feature boundary extraction

x0 = M10/ M00, y0 = M01/ M00,

We scan the label map obtained from segmentation to extract feature region boundaries. Inside each feature region Fi, for y = yk, xmin and xmax are recorded which define Fi boundary in horizontal line y = yk. So the extracted boundary for Fi is a set, (, yk), k = 0, 1, 2,…, n-1, where, x1k, x2k denote the x-coordinates of the boundary points of the kth image row having an ordinate value yk.

θ = 0.5 tan −1

b , a−c

w = 6 ( a + c − b 2 + ( a − c ) 2 ), h = 6 ( a + c + b 2 + ( a − c ) 2 ),

3.3 Local geometric moment approximation This subsection discusses how the local geometric moments [6] can be applied for local source image approximation with a single rectangle.

a=

if d∈[0,d0], (2)

0

M 11 − x0 y0 ), M 00

where a, b and c are defined as,

M 02 − y02 . M 00

In subsection 3.3, the orientation of a rectangle approximation to the image can be derived by image moment functions. However, for a local source image containing textures of objects, texture analysis technique [10] is better. Thus, we apply it to discover the texture directionality. The technique is briefly presented below. A histogram of local edge probabilities is used against their directional angle. It was shown that this histogram represents sufficiently global features of the input picture such as long lines and simple curves. This method utilizes the fact that gradient is a vector, so it has both magnitude and direction. In the discrete case, the magnitude |∆G| and the local edge direction θ are approximated as follows:

(1)

F(d)=

b=2(

3.4 Texture analysis technique

where, d(C1, C2) ∈ R is the distance of two colors, C1 and C2 in the LUV color space. In this perceptually uniform color space, d(C1, C2) is given by the Euclidean distance. F(d) ∈ [0,1] is the function that maps the color distance d to the intensity value. (1-(d/d0)2)2

M 20 − x02 , M 00

c=

3.3.1 Intensity Distribution Image An intensity distribution image is denoted by I(x,y): R2→R [9]. The function value is the intensity at the pixel location (x,y). This intensity distribution image can be obtained from a local source image P and color C. I(x, y) is given by the following equation, I(x, y) = F(d(C, P(x,y)),

(6)

if d∈[d0,∞),

|∆G| = (|∆H |+|∆V |)/2, θ = tan-1 (|∆V |/|∆H |) + π/2,

where d0 is a threshold parameter tuned by experiments.

(8)

where ∆H and ∆V are the horizontal and vertical differences measured by following 3 x 3 operators, respectively,

3.3.2 Geometric Image Moment

136

-1 0 1 -1 0 1 -1 0 1

-1 -1 -1 0 0 0 1 1 1

The resultant θ (0 ≤ θ ≤ π) measures counterclockwise. The desired histogram HD can be obtained by quantizing θ into n bins and counting the points with the magnitude |∆G| over the threshold t; i.e.,

H D (k ) = Nθ (k ) /

n −1 i=0

Nθ (i ), k = 0,1,Λ , n − 1

(9)

where Nθ(k) is the number of points at which (2k-1) π/2n ≤ θ ≤ (2k+1) π/2n and |∆G| ≥ t. Thresholding |∆G| by t is aimed at preventing counting of unreliable directions which cannot be regarded as edge points. By experiments, n is set to 16 and t is set to 12. The shape of each histogram HD is not sensitive to the value of t. A way of measuring the directionality quantitatively from HD is to compute the sharpness of the peaks. A sharp peak indicates the texture has highly directional, whereas a flat peak indicates the texture has no directionality. The approach is to sum the second moments around each peak if multiple peaks exist. This measure can be defined as follows:

Fdir = 1 − r n p

np p =1 φ ∈ϖ p

(φ − φ p ) 2 H D (φ ),

(10) Fig 2: An example of white noise image.

where np φp

number of peaks, pth peak position of HD,

ϖp

range of pth peak between valleys,

r

3.6 Generate vector field Vector field determines the strokes direction that is an important factor contributing to the impression of a pencil drawing. Stroke direction is also important for conveying the shapes and textures of objects. In this algorithm, extracted feature region is used to determine the vector value at each pixel. The vector direction of a pixel is determined by the orientation θ of the feature region Fi that the pixel is inside. Therefore the stroke direction in Fi will follow the orientation of Fi.

normalizing factor related to quantizing levels of φ, quantized direction code (cyclically in modulo 180°).

3.5 Generating white noise The gray-scale tone of a resulting pencil drawing is mainly decided by the white noise image. To match the tone between the input image and the resulting pencil drawing, we use the tone of the input image to guide the distribution of noise. The white noise image is generated in a way that the probability a white value is set for a pixel is proportional to the intensity level of the corresponding pixel in the input image. Let Iinput be the intensity of a pixel in the input image, P is a floating-point number generated with a random function, then the intensity Inoise of the corresponding pixel in the noise image is decided in the following way: Inoise =

Imax, if P >= T P ∈ [0.0, 1.0], 0,

otherwise,

3.7 Generate pencil drawing by applying Line Integral Convolution (LIC) Line Integral Convolution (LIC) algorithm was proposed by Cabral and Leedom [1]. It is a method to filter an image by averaging pixels along a curve segment that follows a given vector field. In a brief description, given a 2D vector field represented as a regular grid and a same size white noise image, the algorithm generates an output image in which the texture has been locally blurred in the direction of the vector field. There is a one-to-one correspondence between the grid cells of the vector field and the pixels of the output image. To decide the value for each pixel in the output image, a local streamline is generated passing through the corresponding grid cell in the vector field. Then, a 1-D low-pass filter kernel is defined on the local streamline. The pixels lying on the streamline in the input image is convoluted with the low pass-filter kernel. LIC was also used in [5, 8] for the similar purpose.

(11)

where T = k (1-I input/I max) and k∈(0.0, 1.0]. Here Imax is the largest index of colormap. It is 255 usually. k is a coefficient for controlling the overall tone of the resulting pencil drawing. Higher value of k will add in more noise. Although a best value of k for producing a matching tone between the input image and the resulting pencil drawing depends also on other factors, such as the composition and texture of a scene, our initial experiment result suggests that a default value of 0.7 produce a visually acceptable result for most scenes. One example is shown in Fig 2.

3.8 Edge and boundary enhancement In pencil drawing, sometimes, outline of objects are drawn to emphasize the shape and contour. To simulate this effect, in our pencil drawing process, outline is extracted from the input image and imposed to the resulting image. For the outline extraction, the differential operator Sobel is applied instead of texture edge detection techniques so as to

137

extract both the region boundaries, as well as the internal structure of a region. Outline internal structure is important for representing the texture within a region. By using Sobel operator, filter Mx and My are applied for each pixel in the input image to get fx and fy.

Mx = -1 0 1 -2 0 2 -1 0 1

4

We have implemented the algorithm described in section 3 using C++ on Unix workstation. For 1024x1024 images, it takes about 1 second average to generate their pencil drawing results on a Pentium 3 PC of 256MB. We are still working on processing video, where a faster algorithm is one important factor for real time processing. Some sample results are shown below in Fig 4 with the original image, the results from Adobe Photoshop 5.0, and the results of our method. It is clear that our results are closer to real drawings in style.

My = 1 2 1 0 0 0 -1 -2 -1

The magnitude of gradient is

| ∇ f |=

2

fx + fy

2

Implementation and Results

(12)

An edge is detected when |∇f | is quite large. The pixel value will be darkened when an edge is detected there. Fig 3 shows the comparison with two pencil drawings.

(a)

(a)

(b) (b) Fig 3: A drawing (a) without and (b) with the edge and boundary enhancement.

(c)

138

(g)

(d)

(h)

(e)

(i)

(f) (j)

139

5

Summary

We have presented and implemented a method for generating a pencil drawing from a digital image using the feature geometric attributes obtained by analysis of image moment and texture of each region. We have compared the results with Adobe Photoshop 5.0 for the drawing quality.

6

Acknowledgment

We would like to thank the reviewers of Graphite 2003. Their comments have been used to improve this paper. The work was partly supported by NUS scholarship for undergraduate study and the research grand R252-000-085-112.

References [1] B. Cabral and L. C. Leedom, Imaging vector fields using line integral convolution, SIGGRAPH '93, 263-270. [2] F. Durand, V. Ostromoukhov, M. Miller, F. Duranleau, and J. Dorsey, Decoupling strokes and high-level attributes for interactive traditional drawing, Proceedings of the 12th Eurographics Workshop on Rendering, June 2001, 71-82.

(j)

[3] P. Haeberli, Paint by numbers: Abstract image representations, Computer Graphics, August 1990. 207-214. [4] A. Hertzmann, Painterly rendering with curved brush strokes of multiple sizes, SIGGRAPH '98, 453-480. [5] X. Mao, Y. Nagasaka and A. Imamiya, Automatic generation of pencil drawing from 2D images using line integral convolution, CAD/Graphics 2001, 240-248. [6] R. Mukundan and K. R. Ramakrishnan, Moment functions in image analysis, theory and applications, published by World Scientific in September 1998. ISBN: 9810235240. [7] M. P. Salisbury, S. E. Anderson and D. H. Salesin, Interactive pen-and-ink illustration, SIGGRAPH '94, 101-108. (k)

[8] H. W. Shen and D. L. Kao, New line integral convolution algorithm for visualizing time-varying flow fields, IEEE TVCG 4(2), 1998, 98-108. [9] M. Shiraishi and Y. Yamaguchi, An algorithm for automatic painterly rendering based on local source image approximation, Non-Photorealistic Animation and Rendering 2000 (NPAR '00), 53-58. [10] J. A. Shufelt, Texture analysis for enhanced color image quantization, Graphical Models and Image Processing, 59(3), May 1997, 149-163. [11] M. C. Sousa and J. W. Buchanan, Observational model of blenders and erasers in computer-generated pencil rendering, Graphics Interface '99, 157-166. [12] M. C. Sousa and J. W. Buchanan, Computer-generated graphite pencil rendering of 3D polygonal models, EG '99, 195207.

(l)

[13] A. H. Vermeulen and P. P. Tanner, PencilSketch-A pencilbased paint system, Graphics Interface '89, 138-143.

Fig 4: (a), (d), (g), and (j) Original images. (b), (e), (h), and (k) Results from Adobe Photoshop 5.0. (c), (f), (i), and (l) Our pencil drawing results.

[14] G. Winkenbach and D. H. Salesin, Computer-generated penand ink illustration, SIGGRAPH '94, 91-100.

140

Figure 4 A Feature-Based Pencil Drawing Method Nan Li, Zhiong Huang

298