cost of developing the system is kept to a minimum over the entire life cycle (rather than just during the initial design phase). Good software design will also.
Desktop Simulation Luke Kelty Paul Beckett RMIT University MELBOURNE VIC 3000 Lucien Zalcman Air Operations Division Aeronautical & Maritime Research Laboratory Defence Science & Technology Organisation (DSTO) 506 Lorimer St, Fishermens Bend MELBOURNE, VIC 3207 Keywords: Simulation, Flight Simulation, 3D Graphics, OpenGL, Direct3D, GLUT, Software Maintenance, Life-Cycle Costs ABSTRACT: Real-time simulation and data visualisation are tasks traditionally restricted to expensive, high performance computers. Recent advances in desktop computing open new possibilities for the development of simulation applications such as a human-in-the-loop military aircraft simulator. The increasing capability and reducing cost of desktop computing hardware focuses the discussion on the software component of the life cycle. This paper discusses the effects of recent hardware and software developments on application development costs from a total life cycle point of view.
1.
Introduction
In the past real-time simulation has been restricted to the arena of high performance computing. With recent improvements in hardware, operating systems and industry standard graphics Application Programming Interfaces (API), it is clear that the personal computer has matured to become a viable alternative for simulation applications [1,2]. This paper investigates current hardware, tools and techniques available for the development of real-time simulation on personal computers - especially with respect to the high demands of “human-in-the-loop” simulation. 1.1
Human-in-the-loop Simulation
A human-in-the-loop simulator has rigorous requirements for the generation of images. The simulator needs to provide high quality images at rapid frame rates as well as responding quickly to the demands of the user. Issues such as latency and image fidelity are critical to the way that a user will perceive the simulation. However, just as important are the “software engineering” issues of code development time, and ease of maintenance. Traditionally, these issues have been at odds in the low-end PC environment. The demands of high performance have meant that it has not been possible to fully exploit the benefits of machine independent APIs. However, if a software system is to be maintainable in any form, a solution must be developed that is portable and re-usable. The objective must be to ensure that the cost of developing the system is kept to a minimum over the entire life cycle (rather than just during the initial design phase). Good software design will also have the effect of significantly extending the life span of the system.
1.2
Future Trends
With recent advances in desktop computing, it has been suggested that the high end ‘Graphics Workstation’ is in danger of becoming obsolete or extinct [3]. However, just as technologies such as the mainframe, Fortran and Unix are still important [4], there is no reason to expect that the workstation has reached the end of its life. The real question remaining is: how will the desktop computer and its high performance counterparts co-exist in the future?
2.
Desktop Technologies
There is fierce competition in the desktop computing industry aimed at the consumer market. This continues to result in increasingly cost-effective computing hardware and software. In addition to a major reduction in cost, recent advances in desktop computing include: • Significant improvement in microprocessor speed; • Motherboards which support multiple processors and gigabytes of RAM; • High performance 3D graphics hardware; • Operating Systems which can access these hardware developments. All of these features are becoming standard as manufacturers vie with one another for a dominant market share. Although meaningful benchmarks are hard to find [5], it is clear that these advances are moving architectures well into the domain of mid-range workstations. It is therefore possible to directly compare a high end PC running WinNT against a mid-range workstation - noting that the former is roughly one third the price of the latter. A possible disadvantage is that some manufacturers may be tempted to sacrifice long-term reliability in order to compete effectively. Cutting a few dollars off
the manufacturing price of a motherboard has real benefits in a volume market. It is also clear that the hardware manufacturers can become caught between competing API standards. The situation is still dynamic with new standards constantly emerging. In the past, the only concern was the "API war" between DirectX [6] and OpenGL [7]. Now, new standards such as Fahrenheit are emerging, with the intention of combining the best features of both - but with the potential to split the market yet again. The key issue here is whether hardware performance will suffer as manufacturers alter chipsets to comply with the dominant standards.
3.
Hardware
3.1.
Intel Processor
Improvements in the performance of the Intel processor over the last few decades are well known. Current offerings from Intel include the 450 MHz Pentium II processor. Intel has announced the availability (March 1999) of a 500 MHz Pentium III processor with enhanced multimedia extensions (MMX2) [8], including additional, enhanced 3D graphics instructions. The 64 bit Merced processor (IA64) [9], co-developed with Hewlett-Packard, is scheduled for release in late 1999. It is clear that the standard x86 instruction set has become the target that other manufacturers must meet or exceed if they are to compete in the consumer marketplace. New offerings from manufacturers (eg. AMD, Cyrix) seek to be faster and cheaper, while still meeting system and software compatibility requirements [10]. 3.2.
3D Graphics Cards
The potential of the desktop architecture for real-time simulation was fully realised with the advent of the 3D graphics accelerator card. There are a number of 3D cards on the market with varying levels of quality. High performance cards (eg. Intergraph’s Wildcat 4100 [11] Hewlett-Packard’s Visualise fx6 [12] etc.) provide 2432 bit z buffers, true colour, hardware texture mapping, minimum texture memory of 32 Mbytes, hardware geometry acceleration etc. [13] They can provide performance levels comparable with some high-end Silicon Graphics Incorporated (SGI) configurations at a cost of several thousand dollars (for the graphics card). Low-end cards, based on graphics chips such as 3DFx [14], Permedia2 [15], Intel I740 [16], nVidia TNT [17] etc., can produce reasonable graphics performance at a cost of less than several hundred dollars. In general most modern graphics cards come with hardware accelerated, industry standard, OpenGL and Direct3D drivers for Windows 95 and Windows NT. Newer, multi-threaded drivers can take full advantage of multiple processors while Pentium III aware drivers target the new MMX2 instructions. Linux drivers are also becoming available. 3.3.
Intel’s Accelerated Graphics Port
Intel’s Accelerated Graphics Port (AGP) [18] provides a means whereby the graphics controller can
obtain texture information directly from the PC’s memory. To provide out-the-window imagery of sufficient fidelity, for example in a low flying humanin-the-loop simulator, a considerable amount of texture information may be required. If this texture information cannot be stored locally, due to insufficient texture memory in the graphics card, the data must be made available to the graphics processor via the graphics controller’s PCI interface bus. The PCI bus is a general purpose, shared, interface bus and the graphics controller must obtain control of the bus from other interfaces before it is able to transfer texture information. This is a potential bottleneck. The AGP is a dedicated, high-speed graphics bus, introduced by Intel with the Pentium II processor, which is a costeffective solution when a graphics controller does not have sufficient texture memory available. Alternatively, for high-end graphics cards, adding more texture memory is a better solution.
4.
Application Programming Interfaces
An Application Programming Interface (API) defines the software interface to a set of software routines or applications. When a set of APIs is accredited by an accepted industry organisation, such as the IEEE or the OpenGL Architecture Review Board, the API set (eg. OpenGL) may be described as "open", ie. an industry standard. A widely used, although proprietary, API set (eg, Microsoft’s Direct3D) may also be described as a (pseudo) industry standard. In the graphics area a number of these industry standard API’s exist. Use of these industry standard API’s reduces maintenance as long as the required API (ie driver) is available for the graphics controller (hardware) being used. 4.1.
OpenGL
Silicon Graphics released the OpenGL API in 1992, having developed its precursor (GL) for use on its own workstations. The objective was to provide a ‘low-level, vendor-neutral’ software interface for developing both 2D and 3D graphics. Referred to as the "Assembler Language" of computer graphics, OpenGL allows for the development of graphics components across an impressively wide range of platforms. 4.2.
Direct3D
The main competitor to the OpenGL API has been Microsoft’s Direct3D [19] which is part of DirectX. The Direct3D API provides software-based rendering including transformations, lighting and rasterization as well as transparent hardware acceleration. Direct 3D features supported by 3D accelerator cards include zbuffering, anti-aliasing, alpha blending, mip mapping, atmospheric effects and perspective correct texture mapping. In addition the Direct3D API integrates with Microsoft’s DirectX and ActiveX providing video mapping and hardware 3D rendering in 2D overlay planes. Integration with the DirectDraw APIs provides 2D drawing and texture services for 3D rendering. The popularity of the Direct3D API has been driven by the computer gaming industry. As a result the
majority of hardware vendors support the Direct3D API. 4.3.
GLUT
GLUT (OpenGL Utility Toolkit) [20] is an operating system independent, toolkit which implements a windowing API for OpenGL. GLUT provides a portable API so you can write a single OpenGL program that works on both Win32 PCs and X11 workstations. Since GLUT is designed to be highly portable it has been extended to other operating systems such as OS/2 and Macintosh operating systems. The GLUT library has bindings for the C, C++ (same as C), FORTRAN, and Ada programming languages. Source code for GLUT is freely available. GLUT is not a ‘full–featured’ window toolkit. It is only intended for small to medium sized OpenGL programs. In order to develop a highly sophisticated GUI it is still necessary to use a native window system toolkit such as Motif. The GLUT library supports the following functionality: • Multiple windows for OpenGL rendering. • Callback driven event processing. • An `idle' routine and timers. • Utility routines to generate various solid and wire frame objects. • Support for bitmap and stroke fonts. • Miscellaneous window management functions. GLUT makes uses of software-only OpenGL routines. Once a supported 3D-accelerator card is used the driver automatically takes over allowing GLUT to use hardware acceleration.
4.6.
Fahrenheit
Recently Microsoft and Silicon Graphics announced that they will join forces to develop a new set of graphics APIs under the code-name ‘Fahrenheit’ which they hope will become the primary graphics API for both consumer and professional applications on Windows [23]. The Fahrenheit project is intended to create a suite of APIs for the Microsoft DirectX multimedia architecture on the Windows operating system and Silicon Graphics UNIX-based platform. It will evolve from Direct3D, DirectDraw, and OpenGL while providing full backward compatibility with applications and hardware device drivers written for Microsoft Direct3D as well as maintaining functional compatibility with SGI's OpenGL technologies. Intel will become involved to ensure maximum support for current hardware. This will ensure that the new suite of APIs are optimised for the Pentium II/III processors as well as including support for MMX, MMX2/Katmai instructions, and the new 64-bit Intel Merced chip. Fahrenheit will consist of three main components: • Fahrenheit Scene Graph API • Fahrenheit Large Model Visualisation Extensions (both the API and the LMVE are due for release in the first half of 1999) • Fahrenheit low-level API (Due for release sometime in 2000) 4.6.1.
The Scene Graph API
GlutMaster [21] is a set of C++ wrappers which uses C++ objects to allow simultaneous, multiple GLUT windows. Only one window may be active at a particular instance in time. GlutMaster is simple and portable, and can therefore provide a common environment for programmers to independently develop and integrate graphics code.
A scene graph is an interface to a low-level API, typically either OpenGL or Direct 3D in the simulation domain. It consists of data structures to create a world, specify objects, textures, and behaviors, and so on. The Scene Graph API will be capable of making use of hardware acceleration. It will incorporate technology from existing SGI products including Optimizer, Performer, OpenInventor, and Cosmo 3D as well as Hewlett-Packards Direct Model API. Similar work has already been undertaken by SGI for the creation of OpenGL++. Combined with the Large Model Visualisation Extensions, this tool has the potential to change the way interactive graphics is developed for the desktop computer.
4.5.
4.6.2.
4.4. GlutMaster Adaptation of GLUT
An
Object
Oriented
MESA
The Mesa API [22] is a graphics library which for legal reason cannot be described as an implementation of OpenGL. For all intents and purposes it is a substitute for OpenGL as most applications written for OpenGL are able to use Mesa without alteration to source code. The Mesa graphics library is supported on WinNT as well as Linux. GLUT may be used in conjunction with Mesa to develop applications that are portable between WinNT and Xfree86. The performance of the Mesa graphics library is directly proportional to CPU speed. Recent releases of Mesa provide support for symmetric multi-processing and 3D hardware acceleration [22].
The Large Model Visualization Extensions
The Large Model Visualization Extensions will serve two purposes. Firstly they will make it easy to render complex models (e.g. CAD models) in real time. Secondly they will integrate with the Scene Graph API to add functionality such as ‘multi-resolution simplification’ allowing programmers to easily write applications that interact with large visual databases. This technology will also be designed to enhance legacy applications with new large model visualization capabilities [23].
4.6.3.
The Low-Level API
Finally the Fahrenheit project will include a low-level API to address the ‘short-comings’ of both OpenGL and Direct 3D, as well as to provide for future requirements of a 3D Graphics API. The first two APIs in the Fahrenheit project will be developed using the existing OpenGL and Direct3D APIs. 4.7.
Java 3D API
In order to extend the functionality of the basic Java core, Sun Microsystems and its partners have developed the Java Media and Communication APIs [24]. These contain support for graphics programming with Java 2D and 3D. The APIs provide the ability to write interactive graphics applications and applets. However, Java is faced with seemingly overwhelming competition from the existing OpenGL standard. OpenGL is a low level, procedural API that is optimised for speed and allows the developer great control over the rendering process. The Java3D API, on the other hand, is high-level and object-oriented. It works through a scene graph-based 3D graphics model. The benefit of this is ease of use [25]. Where possible the Java runtime environment enables the scene graph to generate renders in the fastest time possible. For this reason the Java3D API is more useful for real-time interactive graphics environments such as games, simulations and low-latency situations than it is for high quality graphics applications such as a rendering. One attractive feature of the Java 3D API is the ability to import content into the Java runtime environment. Sun and the Web 3D Consortium have recently developed the Java 3D VRML97 file loader and browser. Java 3D also provides support for peripheral devices including wands, data gloves and headsets. The com.sun.j3d.utils.trackers package includes support for Fakespace, Logitech and Polhemus devices. While this API has some very useful functionality, the technology is not entirely problem-free. Firstly, since Java 3D is classified as an extension to the Java core, vendors are not required to implement the API. Due to the amount of effort required to maintain the Java core alone, Sun is currently the only major vendor supporting Java 3D and then only for Solaris and Win32. As a result the Java API does not support as many platforms as could be expected. In comparison OpenGL is available for every flavour of Unix, Windows and many other operating systems. Secondly, and of critical importance to its adoption as a standard API, there is a distinct lack of documentation provided by Sun. In contrast the OpenGL Consortium's Web site provides comprehensive information. This is not a minor point: the relative complexity of 3D graphics APIs make good documentation absolutely essential. The next issue is that details of the rendering-pipeline are hidden from the developer. Instead the Java 3D runtime environment manages these details. This makes the Java 3D API unsuitable for a number of
programming problems where such low-level control is essential. Finally the Java 3D API mixes heavyweight (ie using native non-java peers) and lightweight components. The result is that GUI development is quite disorganised.
5.
Software Design for Simulation
In the real-time simulation the issues of performance versus standardisation need to be considered very carefully. While performance is usually critical, development costs (including costs incurred in the maintenance phase) must be kept to a minimum over the entire life cycle of a simulation system. The use of industry standard APIs (ie drivers) reduces overall life cycle costs. Whilst removing source code one level of abstraction from hardware does create a slight degradation in performance, the continuous increase in performance of current desktop hardware more than compensates for the performance loss of the higher level abstraction software. The advantage is that hardware can be upgraded without the need to redevelop software. In the desktop computing domain, software re-development costs often greatly exceed the cost of new hardware. Abstraction from the hardware through the use of device drivers and application programming interfaces is necessary to ensure that code is portable between hardware devices. 5.1.
Portability
The most popular operating systems used for HIL simulation applications are WinNT, Linux and Irix. Any APIs and/or SDKs (Software Development Toolkits) used to develop applications should support these operating systems thus any code developed will automatically be operating system independent and portable. Of the APIs mentioned above OpenGL and GLUT are: • Operating system independent; • Available for Windows NT, 95, Linux, IRIX, etc.; • Available for a range of popular programming languages; • Freely obtained over the Internet 5.2.
Code Re-use
Rapid improvements in desktop computing highlight the importance of abstracting code from computer hardware. Abstracting code from hardware by using industry standard APIs or SDKs, such as OpenGL, provides code that is portable from hardware device to hardware device. When a new device is installed (ie a faster graphics card) the new relevant driver (eg. the OpenGL driver) is installed on the computer. In Windows NT, because the OpenGL driver (opengl32.dll) is a Dynamic Link Library (DLL) object the application code does not even have to be recompiled. The DLL driver is loaded at runtime. Once the new hardware and OpenGL DLL driver have been installed, the application will automatically take advantage of the new, faster hardware. It is a simple 5
minute procedure to replace a graphics card and run a driver installation program on a PC. Moving one level of abstraction higher, using APIs or SDKs such as GLUT produces re-usable, highly portable code that can be used, with little or no change, from operating system to operating. It must however be recompiled on the new operating system. The next level of code reuse (as well as portability) is to create data objects that can be used cross platform without the need to re-compile. This ideal is (almost) achieved with the Java 3D API by running software components in a virtual machine on top of the hardware and operating system, abstracting such details as endian format and native data types. However, this solution is currently sub-optimal with respect to performance, portability and the level of control possible. 5.3.
Rapid Application Development Tools
Another important consideration is the amount of time and level of skill required for development of software components. Consequently it is advantageous to use Rapid Application Development (RAD) tools for the automatic generation of graphical components. VPI’s VAPS toolkit is a graphically orientated toolkit which allows the user to quickly and easily create complicated instrumentation from a library of simple primitives. Of course it is important that a RAD tool produces code that adheres to functional requirements including performance levels, portability and maintainability.
6.
A Simple Application
A simple application was developed at DSTO’s Air Operations Division to demonstrate the benefits of using the APIs and SDKs described above. The OpenGL and GLUT APIs were used to produce a generic (graphical) front end of a radar simulator running on a SGI computer. The application was written in C++ running on the IRIX operating system. The application took six man weeks of effort to develop and comprised a few hundred lines of C++ code. Once fully developed the application was copied to an Intel Pentium Pro PC, running a Matrox Millenium graphics card, and recompiled using the Windows NT version of GLUT and the required Matrox Millenium OpenGL driver. The application executed on the Windows NT PC without any change to the source code. A 3DFx Voodoo 1 based graphics card is a dedicated, 3D only, graphics card and must run in conjunction with a normal 2D capable card, such as the Matrox Millenium, to run the windows part of the Windows NT operating system. Windowing software cannot be run on a Voodoo card. The Voodoo 3D add-on card can only run a single, full screen, non-windows, application. The application code was recompiled using the OpenGL compatible Mesa driver (see section 4.5 above) as a validated 3DFx Voodoo 1 OpenGL driver was not available. Again the application executed without any change in source code. However, the OpenGL (through the Mesa driver) radar graphics front end appeared on the second monitor attached to the
Voodoo 1 card but the corresponding GLUT window appeared on the monitor attached to the 2D Matrox Millenium card as the Voodoo card did not support any (2D) windowing capabilities and the GLUT software was using the Matrox Millenium OpenGL driver. Thus this simple, graphical, radar front-end application elegantly demonstrated the benefits of using widely available, standard APIs and SDKs.
7.
Summary and Conclusions
The continuously increasing power of desktop architectures is changing the way that simulation software is designed. In the past the main concern was to optimise the code for performance with many desktop simulators and games written in assembler. Today, PC hardware is powerful enough to compensate for the overheads of abstracting source code from hardware through the use of device drivers plus industry standard APIs and SDKs. The following points should be considered when developing a (HIL simulation) computing application: • The increasing capability and reducing cost of desktop computing hardware is driving the focus of the total application life cycle cost to software development; • A major component of the total software development life cycle is overall maintenance and this aspect should be considered in detail at the initial design phase of the application; • The cost of software maintenance can be significantly reduced by the use of “low level” (industry and pseudo proprietary) standard APIs and SDKs such as the OpenGL API. • In operating systems, such as Windows NT, where the OpenGL library is installed as a software module loaded at run-time, only the new device with its associated driver (API) needs be installed and the software should run without being recompiled. Alternatively the software may have to be recompiled but the code should not require modification; • The use of “high level”, operating system independent, standard APIs and SDKs, such as GLUT, should result in highly portable and reusable code which requires less maintenance, lowers application development costs and extends and/or maximises the life of an application.
8. 1. 2. 3. 4.
References Zalcman, L., Cost Effective "Flying Desk" Simulators, Proceedings SimTecT97, (1997), pp 313 - 315. Zalcman, L., Cost Effective Simulators - Twelve Months On, Proceedings SimTecT98, (1998), pp 275 - 279. Graphics PCs Will Put Workstation Graphics in the Smithsonian, Proc. Sigraph 96, ACM Press, New York, 1996, p. 506. The Times They Are A-Changing: PC Graphics Moves In, William Ribarsky, Georgia Institute of
5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22.
23.
Technology,IEEE Computer Graphics and Applications, May/June 1998 http://www.specbench.org http://www.microsoft.com/directx/default.asp http://www.opengl.org/ http://www.sgi.com/software/opengl/overview.html http://www.intel.com/pentiumiii/ http://developer.intel.com/design/processor/future/ ia64.htm Bursky, D., Advanced Processes and Architectures Boost x86 CPU Performance Levels, Electronic Design, November 16, 1998, pp69-82 http://www.intense3d.com/ http://www.hp.com/kayak/graphics/visualise_fx6.ht ml Scott Spanbauer, High-End 3D Comes Down to Earth New Media, August 12 1996. http://www.3dfx.com/ http://www.3dlabs.com/ http://www.intel.com/design/graphics/740/index.ht m http://www.nvidia.com/home.html http://www.intel.com/technology/agp/ http://www.microsoft.com/directx/pavilion/d3d/ default.asp http://reality.sgi.com/mjk_asd/glut3/glut3.html http://www.opengl.org/Documentation/GLUT.html http://www.duke.edu/~stetten/GlutMaster/GlutMast er.html http://www.mesa3d.org/ http://www.opengl.org/Documentation/ Implementations/Mesa.html http://ftp.llp.fu-berlin.de/lsoft/F/3/PMESA.html http://www.sgi.com/developers/technology/graphic s/fahrgraphics.html
http://www.gamasutra.com/features/programming/ 19980410/fahrenheit_01.htm 24. http://java.sun.com/ 25. http://www.gamasutra.com/features/programming/ 19990115/java3d_01.htm
9.
Author Biography
Luke Kelty graduated from the University of Queensland with a Bachelor of Engineering (Electrical and Electronic). He is currently completing a Master of Engineering (Information Technology) at RMIT University. The work described in this paper was completed during a work assignment at the Air Operations Division of the Aeronautical and Maritime Research Laboratories Paul Beckett is Head of the Department of Computer Systems Engineering at RMIT. He received both his Bachelor of Engineering and Master of Engineering degrees from RMIT University. He was team leader of the group which developed the prototype for the Hawker Pilot Trainer – a multi-crew reconfigurable cockpit simulator now marketed by Tenix. Dr Lucien Zalcman graduated from Melbourne University with a BSc (Hons.) in 1973. He was awarded a PhD in Physics from Melbourne University in 1980 and a Graduate Diploma in Computing Studies from RMIT in 1984. In 1984, he joined DSTO as an Information Technology Officer. Since 1992 he has been employed as a Senior Professional Officer in Air Operations Division of AMRL specialising in the field of Distributed Interactive Simulation.
