Paper Number - Realtime Technologies

2004-01-2641

High-Frequency Terrain Content and Surface Interactions for Off-Road Simulations Michael M. Morrison, Richard A. Romano Realtime Technologies, Inc.

Alexander A. Reid, David J. Gorsich U.S. Army RDECOM, National Automotive Center Copyright © 2004 SAE International

ABSTRACT Standard visual database modeling practices in driving simulation reduce geometric complexity of terrain surfaces by using texture maps to simulate high frequency detail. Typically the vehicle dynamics model queries a correlated database that contains the polygons from the high level of detail of the visual database. However the vehicle dynamics database does not contain any of the high frequency information included in the texture maps. To overcome this issue and enhance both the visual and vehicle dynamics databases, a mathematical model of the high frequency content of the ground surface is developed using a set of Non-Uniform Rational B-Splines (NURBS) patches. The patches are combined in the terrain query by superimposing them over the low-frequency polygonal terrain, reintroducing the missing content. The patches are also used to generate Bump Map textures for the image generator so that the visual representation matches the terrain query. An imagery-based approach to generating the mathematical surfaces and Bump Maps from the database’s decal textures is also presented. Future research will include full implementation in a driving simulator. This research was supported by an SBIR Phase I Contract (No. W56HZV-04-C-0101) in conjunction with the U.S. Army TARDEC.

generally constructed to the specifications and constraints of the Image Generator, rather than the dynamics model. Surface roughness characteristics can rarely be captured in the polygonal model due to image generator fill-rate and polygonal constraints. In addition, if the terrain was constructed from Geo-Specific data there were certainly compromises made in the polygonal structure in order to minimize geometric load. Simulation databases targeted at “on-road” driving may have an additional splinar or other mathematical surface used as the terrain model for typical drivable surfaces. In general, these surfaces are designed to further smooth the terrain over the polygonal model and do not address the resolution and high frequency roughness issues presented here. Reid [1] suggests that since the polygonal surfaces in the visual model may exceed 30 meters (or more) in size, any notion of “off road” terrain has been lost by using the visual database in the terrain query. This is because the high frequency roughness of the surface is no longer adequately captured in the visual model, and therefore the terrain model. The resulting terrain and visual databases lack realism in the simulation system because this high frequency content is missing.

ROUGHNESS GENERATION BACKGROUND Historically, simulation systems rely on the visual database geometry to provide the terrain for autonomous entities to traverse. The high Level-OfDetail (LOD) component of the visual database is extracted and used to create a polygonal surface. In offroad simulations, this surface is primarily what the dynamics model and other entities in the simulation interact with. Because the visual database is the same as the surface terrain, correlation and ease of generation are achieved. However, the high LOD model of the visual database is

The goal was to generate a set of roughness data that could be overlaid in the visuals, as well as presented to the dynamics model through the terrain query. To complete this process the roughness data had to be converted into a normal map for the image generator and converted to additional correlated data for the runtime terrain query. Where frequency content and RMS roughness for surfaces are known, a parametric model can be used to generate a NURBS surface to represent the roughness. Where frequency content and RMS roughness values are not known, a height field can be derived from the

surface texture maps to represent the roughness. Nontextured surfaces with unknown frequency content are considered “flat.” Parametric method Where frequency content and RMS roughness for surfaces are known, Reid suggests using Spectral Fractal Geometry (or Fractional Brownian Motion) to create a spectral representation of the surface. This spectral content is then transformed into the spatial domain using an inverse two dimensional FFT where it is then scaled by the RMS roughness values of the surface. The resulting spatial surface is then used to derive control points for a NURBS patch. A series of adjoining patches could then be smoothed and their control points saved for later use at runtime for the terrain query. To this end, a MATLAB program was used to construct and visualize NURBS surfaces. To describe the frequency content of a surface, this software takes min/max wavelength, RMS roughness, and fractal dimension as parameters. Figure 1 shows a NURBS surface generated using this software inside the MATLAB viewer.

so that shadows and specular effects are minimized. This provides the best looking textures for run time, since lighting effects are generally added by the image generator, but the resulting images can leave few cues as to the relief of the original surface. While photometric techniques may exist to automatically extract height information from a single image, we chose to use a relatively simple method to convert the RGB image into a height field. This technique is used by both ATI and Nvidia in their conversion tools. The basic technique is to simply “average” the RGB pixel values using the formula: (R+G+B)/3. The result is a new grayscale image that is declared to be the height field for the surface. In this image “white” (or value 255) represents the highest elevation, and black (or value 0) represents the lowest elevation. Figure 2 shows the height field captured from a portion of a grass texture.

Figure 2: RGB to Height Field

Figure 1:

MATLAB NURBS Generator

This software generates an external file that describes the NURBS surface. A separate library was developed to allow loading and querying of the resulting surface. Imagery method Where frequency content and RMS roughness values are not known, the frequency content may be derived from the surface texture maps. A representative height field can be extracted from the texture using a straightforward technique. This height field is also used for the runtime terrain query.

As is evident from the technique, automatically generating height fields from texture surfaces is problematic. The final result may appear appealing to the eye, and contain frequency content similar to the original image, but it may not accurately represent the original surface relief. In the case of Figure 2, the simple averaging filter seems to have fairly accurately captured the surface relief. In other cases, such as an image of black bricks with white recessed mortar, the height field would be erroneously inverted. This could have negative consequences elsewhere in the system, but has not proven to be an issue for us thus far. Deriving the height field from the texture map has the added advantage that the Bump Map and texture appear tightly correlated on the image generator. Using a NURBS generated Bump Map that varies too far from the image texture can cause the imagery to appear somewhat unrealistic. The height fields are stored in SGI’s RGB format. Cross-platform readers and writers are freely available for this format, and GIMP and Photoshop both support it as well. TERRAIN CORRELATION AND QUERY

Texture maps are usually single, monoscopic images taken in highly ambient (or overcast) lighting conditions

The high-resolution terrain needs to be correlated with the low-resolution terrain and formulated into a surface for the terrain query routines. The runtime terrain query will determine which polygonal surface is under the query point, the overlying high resolution terrain, and how to appropriately combine the two for a surface query.

frequency content for each surface with an associated NURBS. The NURBS patch is virtually tiled as described above.

HOT = Zt+Zs Two correlation methods for the terrain query have been implemented. A NURBS query and a general purpose height field query. In either case the surface is represented by a rectangular patch that contains the roughness characteristics for a particular surface type. Surface type to surface patch mapping is controlled through configuration files. Both queries are currently implemented in an orthographic "top-down" manner. This means that the slope of the terrain is not taken into account when "mapping" the high resolution surface to the terrain for the query. Figure 3 shows how the patches overlay the polygonal terrain. For each NURB or height field, the code constructs a single rectangular patch representing the high frequency content for that surface type. The terrain is queried to determine the coarse Z value and the surface type from the polygonal surface. Then the appropriate patch for the specific surface type is queried. The patches are virtually tiled such that the terrain query position modulo the dimensions of the patch will always land on the patch. These two values are then added to form the final height of the terrain at the query point. This is shown in Figure 4.

Zt Zs Polygonal Terrain

High-Resolution Terrain

Figure 4: Terrain Query Method

Since the NURBS patches are not discretized, the query supports essentially infinite resolution within the patch. Height field terrain query Support for arbitrary height field patch queries was implemented in a manner similar to the method used for NURBS queries. The code associates a height field with each surface specified as having one. Again, the height fields are virtually tiled as described above. The implementation includes support for patch dimensions and scale and bias factors. This allows the user to modify the characteristics of the patch based on a visual inspection of the actual surface. In order to determine a height value, the runtime query finds the four “texels,” or grid posts, which bound the query position within the patch and bilinearly interpolates the Z value from them. VISUAL CORRELATION

Figure 3: NURBS Surface Query

NURBS terrain query As part of a separate development a real time NURBS patch query was added to our existing terrain query library. Figure 3 shows how the NURBS surfaces overlay the polygonal terrain. The code constructs a single rectangular NURBS patch representing the high

Bump Mapping (specifically “DOT3” or “Normal Mapping”) was used to add the high frequency content to the visual model. James Blynn first suggested the technique of Bump Mapping in his seminal work [4]. The majority of the techniques in use today originated from his work in the area.

height field of a NURBS surface as generated by the MATLAB program. It is gridded to show sampling points that would be used when generating a 256x256 Normal Map.

Figure 7: NURBS Surface Sampling

Figure 5: Bump Mapped Surface

The basic idea behind Bump Mapping (Figure 5) is to simulate a rough, or bumpy, surface by using a specially formulated texture map rather than by adding additional geometric detail to the model. This texture map contains perturbations to the surface which are to be used in the lighting equation. During the lighting operation a perpixel texture read is performed to extract that pixel’s normal vector. This vector is then used in the lighting equation to color the pixel. The resulting surface will have the appearance of greater detail due to the variations in shading. DOT3 Bump Mapping requires the perturbed surface representation to be a “Normal Map.” The I,J,K components of the normal vectors are encoded in the RGB color components of the texture, where a value of 255 represents +1.0, and a value of 0 represents -1.0.

The image on the right shows a zoomed view of a surface depicting normals at individual sample points. The normals are computed by determining partial derivatives (or tangent and binormal vectors) of the surface at each sample point. The cross product of these vectors is stored in the texture map as an RGB color value. The result is the Normal Map on the left of Figure 8.

Figure 8: NURBS Normal Map (left) and Bump Mapped Grass Texture (right)

The image on the right is of a single quad with a grass texture that has been modulated by the Normal Map. Additional specularity has been added by the lighting model to give a shiny appearance. When combined with the enhanced terrain query, the end result is a visual model that closely matches the terrain model and has recaptured the high frequency content at the polygonal level. Figure 6: Decal Texture (left) and Normal Map (right) [7]

Figure 6 shows a decal texture on the left and an associated Normal Map on the right. The chalky blue appearance of the image is due to the fact that a normal for a “flat” surface (or IJK of 0.0, 0.0, 1.0) is an RGB color of 128,128,255. Normal map generation Command line tools were developed to generate Normal Maps from NURBS surfaces and height fields. The tools read a surface and write a Normal Map in SGI RGB file format. The image on the left in Figure 7 shows the

Normal Maps for height fields can be generated using the same sampling method described above. The image in Figure 9 is based on the decal texture and height field shown in Figure 2.

NURBS and Height Field Normal Mapping tools were used to generate the Bump Maps and high-resolution surfaces. The extensions to the terrain query were used to query the NURBS or Height Field surfaces at runtime. An extended visuals subsystem on a PC generated the graphics using custom Cg shaders.

Figure 11: Simulation Run

Figure 9: Bump Mapped Grass

CONCLUSION As part of this research a demonstration of the technologies working together in a test database was developed. The database consisted of sixteen patches on a 4x4 grid, each with differing surface types, textures, high resolution surfaces, and Bump Maps. A top-down view of this database is shown in Figure 10. Each square segment of the database is 200 meters by 200 meters.

Figure 11 shows several out-the-window views from a simulation run. The image on the left is of a section of the database that used height fields generated from the grass decal texture for the Normal Maps and high frequency terrain query. The image on the right is of a section of the database using NURBS for the Normal Maps and high frequency terrain query. As mentioned earlier, this work was completed under a SBIR Phase I contract. RTI’s Phase II proposal has subsequently been accepted to continue the research and implement the technologies as products for the U.S. Army TARDEC. RTI’s existing terrain query library and visuals component products will be used as the basis for ongoing development. These components are already commercial products that have been sold into industry as well as TARDEC. RTI develops the SimCreator simulation authoring tool, individual subsystem component libraries, and complete turn-key simulation systems which will contain this technology.

REFERENCES

Figure 10: Top Down View of Test Database

A Realtime Technologies, Inc (RTI) SimCreator simulation system was used to demonstrate a visuals and dynamics model interacting with this database. The

1. Reid, A.A. (1999). “Correlated Parametric Terrain Surface Development using Non-Uniform Rational B-Splines and Spectral Fractal Geometry,” Ph.D. Dissertation, Oakland University, Rochester, MI. 2. Reid, A.A. “Development of a High-Resolution Virtual Terrain to Support the Development And Testing Of Intelligent Systems,” 2002 NDIA 2nd Annual Intelligent Vehicle Systems Symposium, June 2002. 3. Vlachos, Peters, Boyd, Mitchell. “Curved PN Triangles,” ACM Symposium on Interactive 3D Graphics, 2001. 4. Blynn, James F. “Simulation of Wrinkled Surfaces,” Computer Graphics (SIGGRAPH 1978 Proceedings), pp. 286-292, August 1978. 5. Gouraud, H. “Continuous Shading of Curved Surfaces,” IEEE Transactions on Computers, vol. C20, pp. 623-629, June 1971. 6. Phong, Bui Tong. “Illumination for Computer Generated Pictures,” Communications of the ACM, vol. 18, no. 6, pp. 311-317, June 1975.

7. ATI Corporation, various Bump Mapping examples, http://www.ati.com/developer 8. Moller, Tomas, Haines, Eric. “Real Time Rendering,” Chapter 4 - “Visual Appearance”, Section 5.7.5 - “Bump Mapping” 9. Dietrich, Sim. “Per Pixel Shading,” Australian Game Developers Conference, http://developer.nvidia.com/object/agdc_perpixelsha ding.html. 10. Grantham, “Advanced Shading I,” Proceeds from SIGGRAPH99: “Lighting and Shading Techniques for Interactive Applications,” http://www.sgi.com/software/opengl/advanced98/not es/node108.html

CONTACT Mike Morrison Realtime Technologies, Inc. 333 West Drake Suite 30 Fort Collins, CO 80526 (970) 377-1567