Camera Textures Martin Spindler∗ Marco Bubke† Tobias Germer Thomas Strothotte Department of Simulation and Graphics, Otto-von-Guericke University of Magdeburg, Germany
(a) Lens without depth constraint
(b) Lens with depth constraint
(c) Selection of the lens by the user
(d) Lens has adapted to size of skull
Figure 1: Fig. (a) shows a lens without depth constraint. Here, the pelvis and both arms are distorted by the lens. In (b) the scope of the lens is restricted to the pelvis, i.e. the projection of both arms is not influenced by the lens. Fig. (c) and (d) display an extract of a lens-object tracking sequence. In (c), where the complete skeleton is visible, the lens has been selected by the user to magnify the skull. In (d), where only the skull and the ribcage are visible, the lens has adapted to the position and the size of the skull automatically.
Abstract In this paper we introduce a novel real-time rendering technique for camera deformations that can be applied to lens distortions and non-realistic projections. Our technique is based on vertex shader textures and presents a hybrid approach working in both, image and object space. As the primal deformation is achieved by a vertex deformation in object space, our technique does not exhibit any artefacts known from pixel-based scaling techniques. Additionally, we present a novel approach for an automatic lensobject tracking that assists the user in manipulating and interacting 3D objects within the scene. Here, we developed a GPU-based technique for fast bounding box determination that is defined by a set of arbitrary vertices. We also discuss different techniques allowing the user to accurately select these vertices in a nonlinearly distorted environment.
found in the human eye, where the distribution of cones is most compact in the center of the foeva, whereas peripheral regions are more likely to exhibit a sparse distribution [Goldstein 1984]. The same concept is used for information management in human brains, too. According to Furnas [Furnas 1986], we all have a pretty detailed understanding of what is happening in our nearby environment, whereas we only show a rough understanding of distant influences. This is because near events are more important to us, than the ones being farer away, which is a universal rule, allowing us to efficiently make use of limited resources and to improve orientation in a complex world.
CR Categories: I.3.7 [Computing Methodologies]: COMPUTER GRAPHICS—Three-Dimensional Graphics and RealismColor, shading, shadowing, and texture Keywords: mesh distortion, non-realistic projections
1
Introduction
The concept of detail in context has been studied for a long time, not only in the field of computer graphics. Its principles can be ∗ {spindler,germer,tstr}@isg.cs.uni-magdeburg.de †
[email protected]
Figure 2: Ancient drawing of the “Battle of Kadesh”. On the right side Ramses II. and his chariot is depicted exceptional large in order to clarify his specific status and power (it seems as he alone had won the battle). At all times, the idea of detail in context has been inspiring artists, in a way that certain objects are depicted larger than others to point out their importance (see Fig. 2). No wonder, that the field of computer graphics has also been influenced. Thus, many nonlinear distortion approaches have been developed and evaluated (such as fisheye lenses or bifocal displays) in order to provide appropriate tools for navigating in complex datasets or for scientific visualization purposes. A central problem, when dealing with nonlinearly distorted projections, is to find fast rendering techniques capable of being used for real-time applications. Although, such techniques have been pre-
sented in the past already, there is still enough room to explore even faster and more accurate technqiues. Therefore, inspired by the rich feature set of modern graphics hardware, we have developed a novel real-time rendering technique for nonlinear projections, which we call camera textures. Besides this novel rendering technique, our contributions comprise interaction techniques for object/vertex selection in nonlinearly distorted environments, automatic lens-object tracking, and depth constraints to restrict the lens’ scope.
mention of “magic lenses” for the sake of completeness. They were not relevant for our work.
2.2
Non-Realistic Projections
This paper is organized as follows: we start by giving a review of related work (Sec. 2). Then we introduce and discuss the concept of camera textures (Sec. 3), which enables us to render lens distortions and non-realistic projections (Sec. 4). Later, we present our results and discuss quality and performance issues (Sec. 5). Finally, we summarize the contributions of this paper and show directions of future research (Sec. 6). (a) 2D Bifocal Display
2
Related Work
In this section we will discuss previous approaches for partial lens distortions and non-realistic projections. Afterwards, we will complete this section with an overview of present real-time rendering techniques for distortions.
2.1
Partial Lenses
Lenses can be divided into two categories: non-partial lenses and partial lenses [Keahey 1998]. Lenses belonging to the former category completely cover the screen, whereas the ones from the latter category have a limited region of impact. Both categories are qualified for communicating the impression of detail in context. In the following we will discuss partial lenses and refer the reader to the next section (Sec. 2.2) for non-partial lenses. A comparison of various “magnifying” lens types is given by Carpendale et al. [Carpendale and Montagnese 2001] (see Fig. 3). Here, the curve in the upper row shows the gradient profile of each lens starting from the point of focus going towards the border area. The curve in the lower row describes the gradient profile from the border area into the direction of the context.
(b) Polar Fisheye Lens
Figure 4: Non-partial lenses [Leung and Apperley 1994]. Although partial lenses can be converted easily to non-partial lenses (the opposite direction is not that easy), we will only focus on “pure” non-partial lens approaches in this section. Apperley first introduced bifocal displays in 1982 [Apperley et al. 1982]. Here, the display was divided horizontally into 3 viewports with the central viewport providing an undistorted view and the left and right viewport providing horizontally distorted views. In 1989, Leung extended the one-dimensional bifocal display to two dimensions [Leung 1989]. As can be seen in Fig. 4(a), the display is divided into nine viewports. Interestingly, the central viewport and the four corner viewports are not distorted (the corner viewports exhibit the aspect ratio from the display). Later in 1991 and 1993, the bifocal display was augmented with perspective projections which led to the “Perspective Wall” [Mackinlay et al. 1991] and the “Document Lens” [Robertson and Mackinlay 1993]. Another popular non-partial lens is the fisheye lens [Furnas 1981; Sarkar and Brown 1992]. Fig. 4(b) illustrates one of its variants: the rectangular polar fisheye.
Another type of partial lenses – so-called “magic lenses” [Stone et al. 1994; Viega et al. 1996] – is typically not being used for magnifying or zooming purposes. Rather, they provide alternative views of parts of the scene and can be controlled by special user interfaces, e.g. toolglass widgets [Bier et al. 1993]. We only make Figure 5: Inverse perspective (left) and traditional perspective (right) projections [Levene 1998]. Finally, we will name another publication, which we found to be inspiring. In 1998, Levene worked on a framework for non-realistic projections [Levene 1998], which aimed at producing projections, that contradict the human’s imagination, but still have a long tradition in art. Fig. 5 shows one example: the inverse perspective projection.
2.3
Figure 3: Comparison of lens types [Carpendale and Montagnese 2001]: Gaussian, Cosine, Hemisphere, Linear, Inverse Cosine and Manhattan.
Rendering Techniques
One popular approach for rendering interactive nonlinear distortions is an image based technique, first introduced by Keahey and Robertson in 1996 [Keahey and Robertson 1996b; Keahey and Robertson 1997]. Here, in a first step the input image is projected
onto a two-dimensional regular grid via traditional texturing methods. In a second step the regular grid is then deformed resulting in a distorted version of the original image (see Fig. 6).
Another limitation is the two-pass character, inherent in all image based techniques and influencing performance costs negatively. Motivated by modern graphics hardware technology we aimed at solving these problems to provide a simple and robust technique capable of real-time applications with complex 3D scenes.
3.2
(a) Original image with regular grid (b) Distorted image + altered grid
Figure 6: Keahey’s approach for creating distorted images is based on standard texturing methods and 2D grid deformations [Keahey and Robertson 1996b]. Later, this technique was adapted by Keahey [Keahey and Robertson 1996a] to work with 3D scenes, too. This was achieved by simply rendering the 3D scene into the input texture. In 2005, Yang et al. [Yonggao Yang 2005] picked up this approach once again, described their implementation based on the OpenGL library, and discussed its application for lenses and nonlinear perspective projections.
The Camera Texture
Before we introduce the concept of camera textures, we would like to define some spaces, necessary for a proper understanding. We assume, that all objects in a 3D scene belong to the world space also including the camera. If the origin of the world space is translated to the position of the camera by simultaneously aligning its look-at direction with the positive Z-axis, we are in viewing space. The camera space is then obtained by applying the projection matrix to the viewing space. Within the camera space the X- and Y -values range from −1.0 to 1.0, whereas the Z-values range from 0.0 to 1.0. By omitting the Z-values (possibly employing the depth-buffer) we get to a twodimensional space, which – after applying the rasterization stage – is known as the image space representing the final picture of what the camera “sees”.
Although there are other nonlinear distortion rendering techniques, including object based approaches like [Bayarri 1995], we don’t know about a single one being appropriate for real-time applications with complex scenes (with the exception of Keaheay’s and Yang’s image based approach).
3
Camera Textures
In this section we will introduce the camera texture as a novel tool for rendering nonlinear projections. We start with a motivation of our work (Sec. 3.1). Then we present the technique (Sec. 3.2) and explain implementation details (sec. 3.3). Finally, we discuss some problems (Sec. 3.4).
3.1
Figure 7: The camera texture describes a lens by local changes in X and Y direction in camera space, i.e. before the rasterization has been applied. Thus, the depth information is still available.
Why yet another rendering technique?
As already discussed in Sec. 2.3, present object space techniques are not very applicable for nonlinearly distorted real-time applications, especially when large and complex 3D scenes are involved. Hence, image based approaches are typically preferred, even though they exhibit two disadvantages, that could not adequately be solved in the past. The first disadvantage arises when a region is magnified in a way that the input image cannot provide enough sample points, leading to pixel artefacts in the distorted image. A common approach to avoid this problem is to use larger input images, which again introduces downsides by requiring more texture memory and an increased rendering time of 3D scenes. The second disadvantage of image based techniques is their inflexibility regarding a manipulation of visibility depending on depth information, object-ids, or other parameters. For instance, it is not trivial to apply depth constraints to a lens, because the required depth information is missing.
As illustrated in Fig. 7, a camera texture can be thought of as a discrete description of a (non-)partial distortion of a projection. It resides in the XY -plane of the camera space and therefore lies within the image space. Usually, the camera texture exhibits a size similar to the resolution of the final picture. It stores a XY -gradient for every pixel, describing a local shift of all vertices potentially falling onto a pixel. In other words, each ray defined by the projection (the modelview-projection-matrix is meant here) and passing through a pixel, is bended according to the distortion defined by the gradient in the camera texture. In the next section we will explain, how this idea can be efficiently applied to modern graphics hardware, in such a way that the traditional pipeline does not have to change.
3.3
Rendering Pipeline
Vertex shader textures, i.e. textures that are accessible within vertex shaders, provide an elegant and straightforward way to implement
Camera Texture Model Data
Vertex Shader
Primitive Assembly
Fragment Shader
Frame Buffer
Figure 8: Simplified rendering pipeline. 1 2 3 4 5 6 7 8 9
u n i f o r m s a m p l e r 2 D cameraTex ; v o i d main ( ) { mat4 mvp = gl ModelViewProjectionMatrix ; gl Position = mvp ∗ g l V e r t e x ; v e c 2 t e x C o o r d = ( g l P o s i t i o n . xy + 1 . 0 ) ∗ 0 . 5 ; v e c 2 d e f o r m e d = t e x t u r e 2 D ( cameraTex , t e x C o o r d ) ; g l P o s i t i o n . xy += d e f o r m e d ; }
Listing 1: Vertex shader for camera textures.
the concept of camera textures. Vertex shader textures were first publicly available with the launch of Nvidia’s GeForce 6 series in 2004. Currently, Nvidia’s NV40 chips (e.g. GeForce 6 and 7 series) are still the only type of graphics hardware, fulfilling the requirements of Microsoft’s Shader Model 3.0 [Dietrich 2004], which specifies the “vertex shader texture” feature. In Fig. 8, the principle rendering pipeline has been diagrammed. Here, the vertex shader consumes all incoming vertices of the model data. Since it also has access to the camera texture it can easily transform each vertex nonlinearly, as it arrives at the shader. List. 1, presents the source code of a simplified camera texture vertex shader. Here, the traditional orthographic or perspective projection is applied to the incoming vertex in line 4 and 5. The local deformation vector is then retrieved by the camera texture in line 7, which is added to the position in line 8. In our experimental test environment we have used more complex vertex shaders, e.g. we applied code for scaling, color and normal computation, lensdepth-constraints, and bilinear filtering of the camera texture. We leave it to the reader to write her own version of a camera texture vertex shader. The rest of the pipeline has no particular importance for the camera texture technique. The fragment shader is pretty “unchallenged”, because its only job is to apply a traditional phong shading. From the performance’s point of view, this lets enough room for more sophisticated shading effects. The camera texture approach does not suffer from artefacts based on pixel distortions, because all deformations are executed in object space. However, due to sampling issues with the camera texture, there is the posibility of vertex aliasing, which we corrected by implementing a bilinear filtering. Since current graphics hardware only support nearest-point-sampling in vertex shaders, we had to implement our own bilinear-filter function, which requires three additional texture reads plus two calls of the mix() function. In Sec. 5.2 and 5.3 the gain in image quality as well as performance penalties are dicussed.
3.4
“method is problematic”. They argue, that “the normal linear projection of a straight-line segment in 3D space is still a straight-line segment on the projection plane. [...] In the nonlinear projection environment, however, the projected line segment is no longer a straight segment”. They conclude, that they “have to break the 3D line segments into several shorter segments and project them separately”, which would “seriously degrade system performance”. Although our approach theoretically suffers from the “straight-line segment” problem, we can not completely agree with Yang’s conclusion. Instead, we believe, that the time is ripe for a paradigm shift from image based to object based distortion techniques in the field of real-time applications. As the results from Sec. 5.3 prove, current graphics hardware is fast enough to render nonlinear projections from very complex data sets, being one – two orders of magnitude larger in size, than those used in [Yonggao Yang 2005]. Hence, in consideration of performance purposes it is absolutely arguable to use high resolution data sets in order to avoid the “straight-line segment” problem. Nevertheless, we must admit that it is often not prefereable to tesselate a simple shape such as a box on a “brute force” basis. In many cases, a rendering application has to deal with lots of objects, where a global tesselation of all objects would unnecessarily slow down the system. In these cases, it is more desirable to adaptively tesselate on object, whenever the distortion technique requires more vertices. Although we are not able to provide a solution yet for such adaptive tesselation techniques, we believe, that in near future, graphics hardware will provide efficient functionality for such purposes (e.g. “geometry shader”).
4
Lenses
In this section we will briefly explain, how we employed camera textures to define actual lenses (Sec. 4.1). Then we will present and discuss a couple of interacting techniques, including lens-object tracking (Sec. 4.2), object/vertex selection (Sec. 4.3), and a fast algorithm for the real-time computation of axis-aligned bounding boxes (Sec. 4.4).
4.1
Creation of Camera Textures
Both approaches, the image based technique presented by Yang et al. [Yonggao Yang 2005] and our camera texture technique, are based on local shifts in the image plane. They only differ in how the shifts are applied on what. The image based approach handles local shifts by a twodimensional polygonal mesh (vertex), whereas we use the camera texture (pixel). We call this the “vertex vs. pixel” contradiction. On the other hand, in the image based approach pixels are shifted, whereas in the camera texture approach vertices are moved. We call this the “pixel vs. vertex” contradiction. However, the mathematical foundations of both approaches are identical. Therefore, we refer to [Yonggao Yang 2005], in particular to section “Nonlinear perspective projections” (pp76 − 79) and to section “3D magic lenses” (pp80 − 81). By using these mathematical concepts one can easily create a lens with a camera texture.
Problem Discussion 4.2
In [Yonggao Yang 2005] Yang et al. state in the second paragraph of the “Implementation details” section, that they favour an image space approach over an object space approach, because the latter
Lens-Object Tracking
When a user interacts with a 3D scene, he does not want to readjust a partial lens every time an object has been moved, rotated or
scaled. Rather, he expects the system to assist her silently in the background by automatically adjusting the lens.
1 2 3
For this, the system has to keep track of what object is selected. The selected object’s bounding box can then be projected onto the camera texture using its center as the center of the lens. Unfortunately, this approach exhibits the disadvantage, that only whole objects can be tracked by a lens. In practice, however, the user often wants to magnify an arbitrary set of vertices, possibly belonging to different objects.
4 5 6 7 8 9 10 11
Therefore, we developed a more flexible technique for lens-object tracking, which can be divided into two passes (plus a preprocessing pass being executed only once per selection). In the preprocessing pass, the user defines a selection of vertices (Sec. 4.3). Afterwards, for each frame the axis-aligned bounding box of the selected vertices is calculated (first pass, Sec. 4.4) and then the camera texture is applied (second pass, Sec. 3).
4.3
12 13
v a r y i n g vec4 pos ; v o i d main ( ) / / f r a g m e n t s h a d e r { g l F r a g D a t a [ 0 ] = v e c 4 (− p o s . xy , p o s . xy ) ; g l F r a g D a t a [ 1 ] = v e c 4 (− p o s . z , p o s . z , 0 . 0 , 0 . 0 ) ; }
Listing 2: Bounding box shaders.
been processed, the two color buffers of size 1 × 1 are read back to the CPU.
Vertex Selection
During the work with lenses the user often selects a set of vertices, wanting to direct the lens’ focus to it. In our research we have found two different use cases (Fig. 10). In the first case, the user selects all vertices located within the selection also including backface vertices, that are hidden by front-face vertices. We call this end-to-end selection. In the second case only vertices belonging to front-faces are selected. We call this surface selection. The user can choose between a rectangular selection and an arbitrary polygonal selection. We then render the selection polygon into the stencil buffer. To determine the selected vertices, we render their vertex ids into the framebuffer by having the camera texture vertex shader and stencil buffer switched on. This guarantees a correct selection of the vertices in regard to the lens distortion and in regard to the user’s selection. In case of the end-to-end selection we need a second rendering pass to determine the selected front-face and the back-face vertices. Finally, we store the selected vertices in vertex buffer objects (VBOs) to reduce unnecessary CPU–GPU communication.
4.4
v a r y i n g vec4 pos ; v o i d main ( ) / / v e r t e x s h a d e r { pos = ftransform ( ) ; g l P o s i t i o n = vec4 ( 0 . 0 , 0 . 0 , 0 . 0 , 1 . 0 ) ; }
Bounding Box Determination
The axis-aligned bounding box of a set of arbitrary vertices in 3D space is defined by 6 scalar values describing the minimum and maximum extension in each dimension. Since an axis-aligned bounding box potentially can change every frame, we aimed at finding a solution that is real-time capable, even for a huge number of vertices.
5
Results
In this section we will present and discuss the results obtained from our experimental nonlinear deformation system. In Sec. 5.1 we will display example renditions. In Sec. 5.2 we discuss image quality issues and in Sec. 5.3 we will interpret the performance costs.
5.1
Examples
The following figures illustrate different aspects of our work: In Fig. 10, the behaviour of surface selection and end-to-end selection is compared. In Fig. 1, the effects of depth constraints and lens-object tracking are illustrated. Fig. 11 shows some of our nonrealistic projection renderings.
5.2
Quality Issues
Fig. 9 shows two different views of the skeleton dataset. In the left figure, nearest point sampling has been used for camera texture accesses, whereas in the right figure bilinear filtering was applied instead. Although the general impression of both pictures is comparable, there are minor differences in some regions. In the lower right corner of the left figure, one can clearly see the rectangular alignment of the displaced vertices, leading to a chiseled appearance of
Consequently, we decided to avoid unnecessary memory transfers between the CPU and the GPU (the selected vertices are stored in GPU memory) and worked out a GPU solution (see List. 2). Here, we made use of the GL MAX hardware blending equation, which is the essential operation in the process of finding an axis-aligned bounding box. The minus signs in line 11 and 12 can be explained by max(−x) = min(x). To ensure, that each vertex is involved, we shortend the framebuffer to be 1 × 1 pixel in size (see line 5). Because 6 scalar values do not fit into one RGBA-framebuffer, we additionally made use of the OpenGL Multiple Render Traget (MRT) extension allowing a GPU program to write to more than one color buffer simultaniously, which is done in line 11 and 12. After all vertices have
(a) Nearest point sampling
(b) Bilinear filtering
Figure 9: Qualitative comparison of camera texture accesses.
(a) Position of skull during selection
(b) Rotation with surface selection applied
(c) Rotation with end-to-end selection applied
Figure 10: Difference of surface selection and end-to-end selection. The left figure shows the position of the skull when the two selections were applied. After the selection, the skull was rotated to the right, whereas in the middle figure the surface selection had been applied and in the right figure the end-to-end selection. One can clearly see, that the lens of the middle figure is restricted to the teeth (since the surface was selected only), whereas the lens in the right figure embraces the complete neck (since all vertices in Z-direction had been selected).
(a) Inverse perspective
(b) Document lens
(c) 2D bifocal display
Figure 11: Non-realistic projections.
the distorted view as shown in the upper left corner. Bilinear filtering, as used in the right picture, can help to rigorously improve the image quality.
5.3
Performance Costs
All performance evaluation tests have been conducted on an AMD Athlon(tm) XP 2500+ system with 512 MB RAM and NVIDIA’s GeForce 6800 GT graphics chip with 256 MB RAM. The computer runs SuSE Linux 10.0 with NVIDIA’s most up-todate display driver IA32-1.0-8756. Please note, that results will likely differ on computers with different configurations. We used three 3D models in the performance tests: the bronchial tree (1, 126, 090 triangles), the skeleton (249, 489 triangles), and the pharaoh’s bust (11, 622 triangles). For the rendering area size we chose following three pixel resolutions: 1111 × 836 (928, 796 pixels), 540 × 402 (217, 080 pixels), and 201 × 171 (34, 371 pixels). Before we analyze the performance results in more detail, we want to recall, that our distortion technique only requires one render-
ing pass (as opposed to image based techniques, which require two passes). However, if we additionally make use of lens-object tracking, we will need two passes (one for the determination of the bounding box and one for the rendering). In order to provide a better understanding of the impact of each pass, we present their update frequencies separately (see Fig. 12 and 13 for the rendering results and Fig. 14 for the bounding box results). The aggregated update frequency fagg can then be computed according to Equ. 1, where f1 represents the update frequency of the rendering pass and f2 the update frequency of the bounding box pass. Both frequencies are in frames per second (fps).
fagg =
1 f1
1 +
1 f2
=
f1 f2 f1 + f2
(1)
As can be seen in Fig. 12, the camera texture algorithm is vertexrestricted for large datasets, i.e. the size of the framebuffer has only little influence on the rendering speed. In particular, this is evident with the bronchial tree model providing more then 1 million triangles. Here, the frame rate is virtually constant, regardless of
frames per second
resolution (please note, the huge difference between 201 × 171 and 1111 × 836). However, when the size of a dataset is getting smaller (as in Fig. 13), the impact of vertex processing decreases dramatically resulting in considerable changes in speed as the framebuffer resolution changes. The bilinear filtering also reduces the rendering speed by a factor of about 40% for large datasets. This is due to extra texture reads that negatively affect cache hits.
40 skeleton:
non-bilinear
bilinear
bronchial tree:
non-bilinear
bilinear
540 × 402
1111 × 836
30 20 10
0
201 × 171
framebuffer resolution
frames per second
Figure 12: Performance results for the “skeleton” and the “bronchial tree” datasets. For large datasets the algorithm is vertexrestricted, i.e. the size of the framebuffer has only little influence on the rendering speed.
non-bilinear
1200
300
201 × 171
540 × 402
1111 × 836
Figure 13: Performance results for the “pharaoh’s bust” dataset. For smaller datasets the performance depends on both, the vertex processing and the fragment processing.
frames per second
References
BAYARRI , S. 1995. Computing non-planar perspectives in realtime. Computers & Graphics 19, 3, 431–440. framebuffer resolution
1000
B IER , E. A., S TONE , M. C., P IER , K., B UXTON , W., AND D E ROSE , T. D. 1993. Toolglass and magic lenses: The seethrough interface. Computer Graphics 27, Annual Conference Series, 73–80. C ARPENDALE , M. S. T., AND M ONTAGNESE , C. 2001. A framework for unifying presentation space. In UIST ’01: Proceedings of the 14th annual ACM symposium on User interface software and technology, ACM Press, New York, NY, USA, 61–70. D IETRICH , D. S., 2004. Shader Model 3.0 - No Limits. http://www.microsoft.com/whdc/winhec/partners/ shadermodel30_NVIDIA.mspx%.
100
F URNAS , G. 1981. The fisheye view: A new look at structured files. Tech. rep., Bell Laboratories, Murray Hill, New Jersey.
10
10k
100k
1000k
number of vertices Figure 14: Performance results for the computation of axis-aligned bounding boxes. The computation speed is roughly linearly dependent on the number of vertices.
6
Currently, we are working on a real-time “cubist style” rendering system, that aims at stitching together different perspectives smoothly. Here, we extend the concept of camera textures to make use of cubemap textures, that allow the definition of distortions not only along the camera plane, but in all directions. As suggested by Yang et al. [Yonggao Yang 2005], we also work on the simulation of natural phenomena, in particular raindrops on glass and hot-air environments (e.g. deserts and bonfires), which can be described by camera textures elegantly. Furthermore, we plan to apply the lens-object tracking technique to the interactive labeling system presented by G¨otzelmann et al. [G¨otzelmann et al. 2005].
A PPERLEY, M. D., T ZAVARAS , I., AND S PENCE , R. 1982. A bifocal display technique for data presentation. In Proceedings of Eurographics’82, Conference of the European Association for Computer Graphics, 27–43.
600
0
Although our camera texture approach does not suffer from pixel artefacts, as known from image based distortion techniques, it requires well tesselated 3D models to produce distortions of high quality. This limitation is caused by the “straight-line segment” problem, which has been discussed detailly in Sec. 3.4. In near future, this problem can hopefully be solved by adaptive tesselations executed as the need arises directly on the graphics hardware.
bilinear
900
0
textures on modern computer graphics hardware. Our technique is applicable for distortion lenses and non-realistic projections and is particularly suitable for real-time applications. Additionally, we have introduced a technique for automatic lens-object tracking, that assists the user in manipulating and interacting 3D objects within the scene.
Conclusions and Future Work
We have presented a simple and fast rendering technique that computes nonlinear deformations in camera space using vertex shader
F URNAS , G. W. 1986. Generalized fisheye views. In CHI ’86: Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press, New York, NY, USA, 16–23. G OLDSTEIN , E. B. 1984. Sensation and Perception, 2nd ed. Wadsworth Publishing Company, January. ¨ G OTZELMANN , T., A LI , K., H ARTMANN , K., AND S TROTHOTTE , T. 2005. Form Follows Function: Aesthetic Interactive Labels. In Computational Aesthetics in Graphics, Visualization and Imaging, Eurographics Association, L. Neumann, M. Sbert, B. Gooch, and W. Purgathofer, Eds. G UTWIN , C., AND F EDAK , C. 2004. A comparison of fisheye lenses for interactive layout tasks. In GI ’04: Proceedings of
the 2004 conference on Graphics interface, Canadian HumanComputer Communications Society, School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada, 213– 220. G UTWIN , C., AND F EDAK , C. 2004. Interacting with big interfaces on small screens: a comparison of fisheye, zoom, and panning techniques. In GI ’04: Proceedings of the 2004 conference on Graphics interface, Canadian Human-Computer Communications Society, School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada, 145–152. K EAHEY, T., AND ROBERTSON , E. 1996. Techniques for NonLinear Magnification Transformations. In Proceedings IEEE Visualisation ’96 Information Visualization Symposium. K EAHEY, T. A., AND ROBERTSON , E. L. 1996. Non-Linear Image Magnification. Tech. Rep. 460, Department of Computer Science, Indiana University, USA, April. K EAHEY, T. A., AND ROBERTSON , E. L. 1997. Nonlinear Magnification Fields. In Proceedings IEEE Symposium on Information Visualization (INFOVIS ’97), 51–58. K EAHEY, T. A. 1998. The Generalized Detail-In-Context Problem. In Proceedings IEEE Symposium on Information Visualization 1998 (INFOVIS ’98), 44–51. L EUNG , Y. K., AND A PPERLEY, M. D. 1994. A review and taxonomy of distortion-oriented presentation techniques. ACM Trans. Comput.-Hum. Interact. 1, 2, 126–160. L EUNG , Y. K. 1989. Human-computer interface techniques for map based diagrams. In Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.), Elsevier Science Inc., New York, NY, USA, 361–368. L EVENE , J. 1998. A Framework for Non-Realistic Projections. Master’s thesis, Massachusetts Institute of Technology. M ACKINLAY, J. D., ROBERTSON , G. G., AND C ARD , S. K. 1991. The perspective wall: detail and context smoothly integrated. In CHI ’91: Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press, New York, NY, USA, 173–176. ROBERTSON , G. G., AND M ACKINLAY, J. D. 1993. The document lens. In UIST ’93: Proceedings of the 6th annual ACM symposium on User interface software and technology, ACM Press, New York, NY, USA, 101–108. S ARKAR , M., AND B ROWN , M. H. 1992. Graphical fisheye views of graphs. In Human Factors in Computing Systems, CHI’92 Conference Proceedings: Striking A Balance, ACM Press, P. Bauersfeld, J. Bennett, and G. Lynch, Eds., 83–91. S TONE , M. C., F ISHKIN , K., AND B IER , E. A. 1994. The movable filter as a user interface tool. In CHI ’94: Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press, New York, NY, USA, 306–312. V IEGA , J., C ONWAY, M. J., W ILLIAMS , G., AND PAUSCH , R. 1996. 3d magic lenses. In UIST ’96: Proceedings of the 9th annual ACM symposium on User interface software and technology, ACM Press, New York, NY, USA, 51–58. YONGGAO YANG , J IM X. C HEN , M. B. 2005. Nonlinear Perspective Projections and Magic Lenses: 3D View Deformation. IEEE Computer Graphics and Applications 25, 1 (January/February), 76–84.
