Reconstruction of Illumination Functions using Bicubic Hermite Interpolation RUI MANUEL BASTOS1 ´ ANTONIO AUGUSTO DE SOUSA1; 2 FERNANDO NUNES FERREIRA1; 2 1
INESC – Instituto de Engenharia de Sistemas e Computadores Largo Mompiller 22 – Apartado 4433 4007 PORTO CODEX – PORTUGAL 2
FEUP – Faculdade de Engenharia da Universidade do Porto Rua dos Bragas 4099 PORTO CODEX – PORTUGAL 1
[email protected]
ABSTRACT An interpolation technique based on Hermite bicubic surfaces is proposed to improve the accuracy of the reconstruction of illumination functions and avoid Mach band artifacts. The information of radiosity at vertices is used to evaluate approximate tangent planes to the illumination function and determine partials and mixed partials of that function. Typical discontinuities of intensity or derivative in the illumination function are preserved in the reconstructed image. The additional computation involved is modest and the result of real importance.
1
Introduction
The photorealistic image synthesis has been the aim of much research in Computer Graphics. The main goal is to generate images that evoke the same response that the observer would have when looking at the actual environment. There are many limitations related to physiological cues that hamper the achievement of the final purpose [3]. A special limitation is the way our eyes respond to changes and discontinuities in magnitude and/or slope of intensity. This is the so called Mach band effect. It is easy to verify that the most of the light we see does not proceed directly from light sources, but from multiple diffuse, specular or intermediate reflections/transmissions. The most accurate way to render a scene is to evaluate the illumination function for each pixel, but for smoothly-varying illumination functions (diffuse surfaces), this is unnecessarily expensive. In this case, it is much more efficient to sample the illumination function in selected points, and then reconstruct a smooth function to approximate the actual illumination function everywhere in the scene. The main cause of the Mach band effect in synthetic images is the discontinuity in the interpolation of the illumination function between patches where only the values in the vertices are known. In addition to the Mach band effect that can be caused by some kinds of interpolation, the loss in accuracy of the interpolated values also can be noted if compared to the ones evaluated using the same method used to compute the illumination function in the vertices. There are some reconstruction models, and the simplest one is the constant or faceted shading - a single value of the illumination function is used to shade each entire polygon. As the name suggests, this model presents images with faceted appearance, caused by intensity discontinuities at the edges of the mesh of patches. Using this model, it is only possible to have intensity and slope continuity at an edge between two patches if these patches have equal values of the illumination function. Otherwise, there is no continuity at each edge between two patches (C ?1 - discontinuous in intensity).
To reduce discontinuity problems, a bilinear interpolation technique has been introduced by Gouraud [9]. A linear interpolation can ensure continuity of intensity, but cannot eliminate derivative discontinuities (C 0 - continuous in intensity, discontinuous in derivative). The discontinuities in intensity and/or in derivative, in places where the illumination function should be smooth, may appear as Mach band effect in the generated images. To obtain reconstruction functions continuously differentiable, higher-order interpolation schemes must be used. Kirk and Voorhies [10] used a quadratic interpolation that avoided Mach band in almost every case, but did not guarantee derivative continuity between adjacent patches. Another quadratic interpolation, described by Powell and Sabin [12] and Cendes and Wong [5], has been used to reconstruct continuously differentiable functions. Although continuity of derivative is desirable to reconstruction functions, care must be taken in some cases where the illumination functions present magnitude and slope discontinuities. Discontinuities in intensity (C ?1 ) occur, for instance, at regions of contact between surfaces with different orientations or different reflection/emission characteristics. Discontinuities in derivative occur, e.g., in shadows cast by area light sources at penumbra and umbra boundaries. To be faithful to the illumination function, a reconstruction function should preserve these C 0 and C ?1 discontinuities and not smooth them as a continuous differentiable function. Using Powell-Sabin quadratic interpolation looks to be difficult to handle derivative discontinuities, because it imposes a strong connection between neighboring patches. Salesin et al. [14] proposed a bicubic reconstruction scheme that breaks up each triangular patch into three B´ezier subtriangles and introduced discontinuities relaxing constraints on the control points of adjacent triangles. Our approach is similar to the second one, but with a simpler scheme. We propose a bicubic reconstruction scheme based on parametric bicubic Hermite interpolation [7][8]. The scheme approximates tangent planes to the illumination function at each vertex of the geometrical mesh, and, using the evaluated values of the illumination function at those vertices, associates a reconstruction Hermite patch to each geometrical patch. The consideration of selected discontinuities is done in a natural way, duplicating geometrical vertices and edges where a geometrical or illumination discontinuity exists. This reconstruction scheme can be used to approximate several functions in several fields. We used and implemented it to achieve a better interpolant in radiosity. Although the radiosity method very accurately evaluates the illumination function of selected points in a scene, the subsequent and traditional bilinear interpolation of those points introduces undesirable inaccuracies.
2
Mach band effect
It is easy to verify the faceted appearance of any image, when curved objects in the scene are approximated by polygonal meshes where each polygon is shaded individually. This effect occurs if constant or interpolated shading is used and nothing is done to ensure slope continuity across polygon boundaries. The most simple, but expensive, solution of using a finer mesh is not effective, unless all polygons used are smaller than a pixel. This effect of discontinuity is known as Mach band [13] [8], and was first reported in 1865 by the Austrian physicist Ernst Mach. It depends directly on the distribution of the illumination and exaggerates the intensity change at any edge where there is a discontinuity in magnitude or slope of intensity:
At the boundary between two polygons, one dark and the other light, the dark one looks darker and the light one looks lighter. According to Ratliff [13], the basic effect can be verified by holding an opaque card under an ordinary fluorescent desk lamp, preferably in a dark room (figure 1). If the shadow is cast on a piece of paper, part of the paper is fully illuminated, and next to that area is a half-shadow (penumbra) that gets progressively darker until a full shadow (umbra) is reached. Ideally, the distribution of light should be uniformly high in the bright area, uniformly low in the dark area, and smoothly graded between the bright and the dark areas. Looking closely at the edges of the penumbra, a narrow bright band at the bright edge and a narrow dark band at the dark edge can be seen. These are the Mach bands. Observing the distribution of intensity along the bright, penumbra and umbra areas, one can verify that the Mach bands appear in regions where there is discontinuity in magnitude or slope of intensity. In the example presented, discontinuities and its associated Mach bands are desired to represent the actual effect. In cases where a smooth illumination function is desired, discontinuities in magnitude or slope of intensity
LIGHT
Opaque card
Fully illuminated
Penumbra
Umbra Perceived intensity Actual intensity
I
Distance I
Distance
Figure 1: Mach band effect in a shadow cast by a linear light source. It can be seen the actual intensity (analogous to a linear interpolation) and the perceived intensity. Also can be seen a representation of the intensity using constant shading and the correspondent perceived intensity. cannot exist. It can be verified that interpolations of order lower than quadratic cannot guarantee continuity of magnitude and slope of intensity.
3
Definition of the reconstruction problem
The illumination function of a scene returns the spectral sampling at a point, given the spatial coordinates of that point. As said in section 1, for smoothly-varying illumination functions (diffuse surfaces) it is much more efficient to sample the illumination function in selected points, and then reconstruct a smooth function to approximate the actual illumination function everywhere in the scene. That reconstructed or interpolated function is called reconstruction function. As it is too difficult to find an interpolation function for a whole scene, the problem is broken down into small parts. Usually, in radiosity algorithms the objects of the scene are tessellated into geometrical patches [6] [1]. For simplicity, the reconstruction function is tessellated into reconstruction patches associated with the geometrical patches of the scene. Actually, each geometrical patch has more than one associated reconstruction patch - one for each color component (e.g., red, green and blue), but, since the treatment is the same for each color component, the illumination/reconstruction functions will be analyzed as unidimensional quantities. Each geometrical patch is described by its vertices coordinates and characteristics of reflection and emission. At each vertex of each geometrical patch there exists a geometrical normal vector and an intensity (illumination function at that point) is evaluated. Using the given information, we have to find a reconstruction function R that, at any vertex V , returns values equal to the ones calculated, and approximates the illumination function I anywhere else over the geometrical patch:
R(V ) = I (V ); 8 V R(p) ' I (p); 8 p 2 geometrical patch and p 6= V: The same reconstruction function should approximate the first derivative, or gradient, of the illumination function everywhere over the geometrical patches, and must preserve the continuity and discontinuities of the illumination function. The simple association of a reconstruction patch with each geometrical patch does not ensure continuity between reconstruction patches. If two patches share an edge where the illumination function continuity is:
C 1 , the two reconstruction patches must share, at that edge, the same values and its first derivative. C 0 , the two reconstruction patches must share, at that edge, the same values to approximate the
illumination function and the first derivative in the direction of the edge. They do not share the first derivative of the illumination function in any other direction.
C ?1, the two reconstruction patches do not share, at that edge, the same values, nor its first derivative. 4
Bicubic Hermite surfaces to interpolate radiosities
Among the bicubic surfaces the Hermite form was chosen. The motivation to choose this form was its interpolation of derivative data and that this information is directly available from the tangent planes to the illumination functions. A bicubic patch in Hermite form is given by [8][7]:
R(s; t) = S M G M T H
where
= s3 T = t3 S
2
MH
s2
s
t2
3 0 0
1
(1)
(2)
1
?2
2
T
1
t
= ?30 6 6 4
T H
H
(3) 3
1
1
1 0
0 5 0
?2 ?1
7 7
(4)
and, in the case of radiosity reconstruction functions: 2
GH
=
B00
6 6 6 B10 6 6 6 6 @B 00 6 @s 4
j
@B @s
j10
B01
@B
B11
@B
@B @s
@B @s
@t
@t
j00
@B
j10
@B
@t
j01
2 @ B @s@t
j00
j11
2 @ B @s@t
j10
j01
3
7 7 11 7 @t 7 7 7 2 @ B 7 @s@t 01 7 5
j
(5)
j
2 @ B @s@t
j11
The input parameters to the Hermite bicubic form, as can be seen in the coefficient matrix GH , are radiosities 2 @B @ B (Bst ), partials ( @s;t jst) and mixed partials ( @s@t jst), shown in figure 2.
Hermite reconstruction patch
Partials
t
Twists
Geometrical patch
s
Figure 2: The shown points and vectors define a bicubic Hermite patch. To guarantee C 0 continuity at an edge between two Hermite patches, the matching curves of the two patches must be identical, which means that the radiosities and the partials for the two surfaces must be identical along the edge. The necessary conditions for C 1 continuity are that the radiosities and the partials along the edge and the partials and mixed partials (twists) across the edge be equal. With these equalities the storage requirements may be reduced.
5
Tangent planes to the illumination function
It will be assumed that continuity of the geometrical normal vector implies continuity of the reconstruction function. However, an a priori discontinuity meshing algorithm may indicate any kind of discontinuity at any point. In this way, all the reconstruction patches that share a vertex must share a single tangent plane to the illumination function at that point, unless a discontinuity is detected there. A vertex is duplicated if there is any discontinuity that causes a discontinuity in the illumination function at that vertex. In the same way, an edge is duplicated if there is any discontinuity, at least, at one of its vertices. A discontinuity between patches at a vertex/edge occurs if:
the geometrical normal vector is discontinuous; the reflectivity is discontinuous; the emissivity is discontinuous; it is a part of the boundary between umbra and penumbra areas; it is a part of the boundary between penumbra and fully illuminated areas.
To evaluate an approximate tangent plane to the illumination function, the vertex where the function is tangent, the edges that share that vertex and the corresponding patches, and the radiosities at the vertices of the shared edges must be known. Using this information (figure 3.a), a coordinate system ST B is defined for each patch that shares the vertex through the parametric axes s; t and its cross product B (radiosity). In this coordinate system (patch), the radiosity variation vector (radiosity linear interpolation between vertices) is evaluated for the two edges of this geometrical patch that share the vertex. The cross product of the two radiosity variation vectors gives the normal vector, i , to the radiosity variation plane (radiosity bilinear interpolation) that approximates the normal vector (tangent plane) to the illumination function at the shared vertex. The simple use of bicubic interpolation with the above technique to approximate the illumination function tangent plane at every vertex does not give more accurate results than the bilinear interpolation. But more accurate tangent planes can be used taking the average of the tangent planes at each vertex. This average gives good approximations except at vertices of surface boundary.
N
B
B
B1 Ni
Plane of radiosity variation B4 B1
B0 B3
B6
V0 s
V1
Geometrical patch
V3
B0 t
V2
s V0 (a)
Patch A
V1
Patch B
V4
Patch C
V6
(b)
Figure 3: Evaluation of the tangent planes (normal vectors) to the illumination function. (a) 3D sketch at any vertex of any geometrical patch. (b) Two-dimensional analysis of the normal vectors to the illumination function. If there are more than a patch sharing a vertex (figure 3.b), a normal vector to the reconstruction function is evaluated for each patch at each vertex. The approximate normal vector (tangent plane) to the illumination function at the shared vertex is the average vector of the normal vectors of the patches. This
assumption ensures C 1 continuity of the reconstruction function for all the patches sharing the vertex and is a good approximation for the illumination function normal vector (figure 3.b at V1 and V4 ). Using just one plane of radiosity variation (figure 3.a and figure 3.b at V0 and V6 ) to evaluate the illumination function normal vector, the tangent plane is undervalued (dotted line) and a correction must be made to improve the results (solid line). The same problem occurs when two radiosity variation planes share a surface boundary vertex, in the direction of the shared edge.
6
First derivative of the illumination functions - Gradients
To evaluate the partials and mixed partials of the coefficients matrix (5), the approximate normal vectors (tangent planes) to the illumination function are used together with the geometrical normal vectors. Using the fact that the slope of a function may be expressed as the height of a right triangle with base length one, the derivatives are evaluated as the tangent of the angle between the geometrical normal vector, g , and the normal vector to the illumination function, i , at that vertex (figure 4):
N
N
@B @d
j = dB = j d j tan ; j d j= 1 in STB.
(6)
V
s t ss tt
d
N
+ ), and where is a given direction ( , or j + i is the projection of the normal vector to the illumination j function onto the plane defined by the geometrical normal vector and the given direction. Care must be taken with clockwise/counter clockwise angles, depending on g , i , and : @B @d
where
j N N j sign(n d) sin j = cos = N N g
N N
d
i
(7)
V
g
i
n = N (N N ). g
g
i
Ng Nis Ng
Ni
Ng Ni
Ni
Tangent plane
N i st
dB s
Patch A
Patch B
s
Nit st
Patch C
t
s
s
s+t
t
Figure 4: First derivatives evaluation.
7
Implementation
The bicubic interpolation technique, basically, uses three steps in the conventional radiosity pipeline: the meshing, the determination of the tangent planes to the illumination function, partials at each edge and mixed partials at each vertex, and the rendering. The meshing creates information of patches, edges and vertices, duplicating shared edges and vertices when a discontinuity of any kind is detected at a vertex. After knowing the radiosity at each vertex (before the rendering), the tangent planes to the illumination function are approximated and the partials and mixed partials evaluated. During the rendering, at every pixel, a geometrical patch and parametric
coordinates (s; t) over that patch are known. Knowing the correspondent reconstruction patches (one for each color component) to that geometrical patch (same parametric coordinates), the bicubic interpolation of the radiosities is evaluated using expression (1) for each color component (e.g., red, green, blue).
8
Results
The proposed reconstruction technique have been implemented on a SUN Sparc II workstation in C++, as an extension to the radiosity public domain software from India (by Sumant Pattanaik). We have compared a reference solution (a), obtained with a fine mesh, since analytic solution are not known for radiosity problems, a bilinear C 0 interpolation (b), and our C 1 bicubic reconstruction technique (c), to show how the accuracy of the reconstruction functions can be improved accumulating more information about the illumination function per vertex/edge and using a higher order interpolation technique (figure 6). In (b), a value of intensity per vertex, together with a bilinear interpolation, were used to generate the final image. In (a) and (c), a value of intensity per vertex, and partials and mixed partials of the illumination function were used together with a bicubic interpolation to reconstruct improved images. Figure 5 presents a sketch of the simple geometrical model with 136 quadrilaterals. The reference solution was generated using a mesh of 14080 quadrilaterals and the area light source was subdivided in 16 by 16 samples in both cases.
Figure 5: Simple geometrical model. Figure 6 shows a simple illumination function of occluded and unoccluded light where can be analysed the continuity and discontinuities of the reconstruction functions. Note the incorrect Mach band artifacts in (b) that are completly eliminated in (a) and (c). Also note the correct Mach bands on the partial occluded surface, due to penumbra edges, appropriately flagged in the mesh, not smoothed by the bicubic interpolation. Figure 7 plots the red component for the horizontal central line of each image and figure 8 shows the relative errors between the reference and the interpolated reconstruction functions. The lower relative errors of the bicubic interpolation confirm that the presented technique based on the correlation between the geometrical normal vector and illumination function normal vector is much more accurate than a conventional bilinear interpolation. As presented in section 5, the undervalueing of the illumination function normal vector at vertices of surfaces boundary can be seen in the bicubic interpolated image and plots (x = 180), if compared to the reference solution.
9
Conclusion and further work
The radiosity gradient can be evaluated from the radiosity samples in an environment, using a correlation between the geometrical normal vector and the illumination function normal vector at each vertex. Based on this, we have presented a new bicubic reconstruction technique for illumination functions that allows for selected magnitude and derivative discontinuities. The additional computation involved is negligible,
Figure 6:
(a)
(b)
compared to the cost of computing the radiosity samples, and the advantage is an improved interpolation accuracy. Compared to a standard radiosity implementation (bilinear interpolation), the described algorithm requires a list of edges (2 unsigned - vertices - and 6 double - 2 partials per 3 color components) and 12 double per patch - 4 mixed partials per 3 color components. To generate the images in figure 6.(b) and (c) were used 5 minutes and 45 seconds and 6 minutes and 20 seconds, respectively, where 3 minutes and 15 seconds is the time for 500 steps of progressive refinment in both cases. In this way, the rendering took 195 seconds for the bilinear interpolation and 230 seconds for the bicubic interpolation (bicubic=1.18bilinear). Aspects of the presented technique that suggest further research are: A priori and a posteriori adaptive meshing. The a priori meshing is fundamental to indicate the discontinuity edges. To more accurately reconstruct the illumination functions, an a posteriori meshing can resample the environment, avoiding great radiosity variations and allowing more accurate tangent planes to the illumination function. More accurate normals at vertices of surface boundary. The simple scheme presented for estimating normals to the illumination function needs a correction at vertices of the boundary of surfaces. It is possible to approximate the derivative of the illumination function at a vertex of the boundary using quadratic interpolation and values of radiosity and derivative at a neighbour vertex out of the boundary. Triangular patches are better suited to describe complex geometries than are rectangular patches.
RADIOSITY INTERPOLATION (line 255)
250
Reference Bicubic Linear
200 150 Radiosity (red) 100 50 0
0
100
200
300 X position
400
500
Figure 7: Comparision between actual radiosity (reference solution with extremely fine mesh) and interpolated values.
10
Acknowledgments
The first author was supported by the Portuguese Council for Scientific and Technologic Research - JNICT - proc. 435.03.1. We would like to thank Gladimir Baranoski for helpful discussions and Marcelo Walter for his comments on the paper.
References [1] BARANOSKI, Gladimir V. Guimar˜aes. The Parametric Differential Method: An Alternative to the Calculation of Form Factors. Proceedings of EUROGRAPHICS’92. [2] BASTOS, Rui M. R. and FERREIRA, Fernando Nunes. State-of-the-art in Global Illumination using Radiosity. Technical report, INESC/Norte, Porto, Portugal, 1992. (In portuguese). [3] BASTOS, Rui M. R. and LASCHUK, Anat´olio. True 3D Images in Computer Graphics through the use of Computer Generated Holograms. COMPUGRAPHICS’92 Conference, Lisbon, December 14-17, 1992. [4] BASTOS, Rui M. R., et al. Avoiding Mach Band in Radiosity. V Encontro Portuguˆes de Computa¸ca˜ o Gr´afica, Aveiro, Portugal, February 3-5, 1993. (In portuguese). [5] CENDES, Zoltan and WONG, Steven H. C 1 Quadratic Interpolation over Arbitrary Point Sets. IEEE Computer Graphics and Application 7(11):8-16, 1987. [6] COHEN, Michael; GREENBERG, Donald P.; IMMEL, David S. and BROCK, Philip J. An Efficient Radiosity Approach for Realistic Image Synthesis, IEEE Computer Graphics and Applications, p.26-35, Mar. 1986. [7] FARIN, Gerald. Curves and Surfaces for Computer Aided Geometric Design: A Practical Guide. 2nd ed., Academic Press, 1990. 444p. [8] FOLEY, J; van DAM, A.; FEINER, S.; HUGHES, J. COMPUTER GRAPHICS principles and practice. 2nd ed., Reading: Addison Wesley, 1990. 1174p.
100
RADIOSITY INTERPOLATION RELATIVE ERROR (line 255) Bicubic Linear
80 Relative 60 Error (red) 40 20 0
0
100
200
300 X position
400
500
Figure 8: Relative error due to interpolation.
[9] GOURAUD, Henri. Continuous Shading of Curved Surfaces. IEEE Transactions on Computers C-20(6):623-9, 1971. [10] KIRK, David and VOORHIES, Douglas. The Rendering Architecture of the DN10000VT. Computer Graphics (Proceedings of SIGGRAPH’90) 24(4):299-307, 1990. [11] MAX, Nelson. Smooth Appearance for Polygonal Surfaces. The Visual Computer 5:160-73, 1989. [12] POWELL, M. and SABIN, M. Piecewise Quadratic Approximation on Triangles. ACM Transactions on Mathematical Software, 316-25, 1977. [13] RATLIFF, F. Contours and Contrast Scientific American, 226(6):91-101, June 1972. [14] SALESIN, David; LISCHINSKI, Dani and DeROSE, Tony. Reconstructing Illumination Functions with Selected Discontinuities. In: 3rd EUROGRAPHICS Workshop on Rendering (Proceedings), Bristol, 1992.
