Building OpenWorlds™ Paul J. Diefenbach, Prakash Mahesh, and Daniel Hunt DRaW Computing Associates Inc.
Abstract This paper discusses the history and design decisions behind OpenWorlds, the first C++ toolkit for VRML 2.0 integration. OpenWorlds is a highly extensible set of libraries which supports scene graphs, various scripting languages including Java, prototyping, and event routing. OpenWorlds was purposely designed to be easily incorporated in many software systems, to run on different platforms, to be used with various graphics libraries, and to perform many functions. CR Categories and Subject Descriptors: I.3.6 [Methodology and Techniques] Graphics data structures and data types; I.3.7 [Three-Dimensional Graphics and Realism]: Virtual Reality. Additional Keywords: VRML, library, Script
1. INTRODUCTION The Virtual Reality Modeling Language (VRML) is a file format for describing interactive 3D objects and worlds. It is designed to be used on the Internet, intranets, and local client systems. VRML is also intended to be a universal interchange format for integrated 3D graphics and multimedia. It may be used in a variety of application areas such as engineering and scientific visualization, multimedia presentations, entertainment and educational games and titles, web pages, and shared virtual worlds. -VRML Specification ISO/IEC DIS 14772-1 Introduction The Virtual Reality Modeling Language (VRML) is a standard language for describing interactive 3-D objects and worlds delivered across the Internet. -VRML Repository web page 3508 Market Street, Suite 203, Philadelphia, PA 19104 pjdief/prakash/
[email protected]
Since VRML originated as a graphical language for the Web, it has garnered the latter quote’s very narrow application, despite the stated purpose of the VRML specification [1]. VRML 2.0 is viewed primarily as a graphics scene description, with additional scripting capabilities thrown in. VRML scripting is also extremely Java-centric. VRML is typically viewed in the context of a web-browser, and solely within that context, whereas connection to existing systems is highly EAI-centric (External Authoring Interface). The OpenWorlds view of VRML is different: VRML is not only a powerful tool for graphics interchange and exchange, but foremost an extensible scripting language which, almost incidentally, handles graphics scenes. This is not to say that the graphics aspects of VRML aren’t important; rather that the graphics aspects are inherent in the scripting notions of VRML 2.0. VRML is a tool whose roots are in browser technologies, but whose promise lies in being a medium of exchange for all graphics and simulation systems. Adding VRML capabilities to an existing CAD system not only brings in the ability to import new geometry, but also incorporates the ability to run VRML animations. A CAD/CAE system which previously had only modeling capabilities could now have the ability to script custom animations, to download a dynamics or kinematics constraint VRML script, and to output an entire simulation to a VRMLcompatible ray-tracer. As companies have made large investments in existing graphics systems with inherent capabilities far beyond the traditional browser, it makes far more sense to bring VRML to the application rather than the application to VRML. (The latter is the EAI approach.) This is the philosophy behind the design of OpenWorlds: an open library accessible by all; open to platforms, open to scripting languages, open to change. OpenWorlds is essentially a VRML scene-graph execution engine which traverses a scene graph, executes nodes, and passes events. The actions of the nodes are external to OpenWorlds. Nodes can be Java script nodes; they can be C++ built-in nodes such as Shape and Transform using OpenGL; they can be built-in nodes based on a proprietary rendering architecture written in C; they can use a high level rendering library like SGI’s Irix Performer or OpenGL Optimizer, or use primitives inherent in a CAD system’s API. They can even be script nodes written in LISP used to drive an AI simulation. OpenWorlds is designed to be modular, with interface classes for every area of functionality. In addition to the built-in and scripting node interfaces, there are database, GUI, parser, language, and other interfaces as well. OpenWorlds provides
sample implementations and template code; even so, all of these features are customizable. This paper discusses the architecture of OpenWorlds, and how this architecture has permitted creation of a variety of VRML products. Section 2 first details related work, and how OpenWorlds differs from these approaches. Section 3 then gives some background and history on OpenWorlds. Section 4 of the paper introduces the basic architecture of OpenWorlds. The next four sections discuss the various components of this architecture, including in order: the Browser, the Script Interfaces, the Application Interfaces, and the Nodes. Section 9 then introduces three separate applications all developed using the OpenWorlds library. Section 10 reports on the current status and limitations of OpenWorlds, and the paper concludes with a discussion of future work.
2. RELATED WORK OpenWorlds can be used to make a VRML browser. OpenWorlds can also be used as a general-purpose graphics library. OpenWorlds is neither of these, however. OpenWorlds differs from other VRML browsers in that the VRML nodes are not inherent in the system, nor is the rendering library. An OpenWorlds developer can re-define the behavior of any node, whether it is a VRML-specification node or a custom node. OpenWorlds can be used to make an application into a VRML browser, while still retaining the functionality of the original application. In this manner, EXTERNPROTOs can be natively implemented within a browser to add custom node types yet maintain valid VRML 2.0 syntax. OpenWorlds differs from rendering libraries in that rendering is not inherent in the OpenWorlds design, unlike systems which can support VRML geometry such as Java3D or OpenGL Optimizer. Instead, OpenWorlds can use these rendering platforms of various levels of complexity such as OpenGL, Performer, Java3D, or proprietary systems, and extends these to provide full VRML parsing and run-time support. As the Java3D specification[5] suggests, “A developer can use Java 3D functionality to build a VRML loader and browser much as he or she would use C and OpenGL to write a VRML loader and associated browser.” An OpenWorlds application can be written which uses different node libraries (in both C and Java) for different platforms, without modifying the application itself.
Jack lacked the ability to develop complex simulations which coordinated many scripting behaviors. C++ modules and Lisp finite-state machines could not communicate, and there was no high-level monitoring and control. In addition, even the simplest tasks required extensive programming. DRaW’s concept for an extension to Jack was an interface which would connect and coordinate scripting behaviors written in different scripting languages. At this time VRML 2.0 was taking form, and it had this notion of scripting languages and behaviors written into the specification. It was decided that VRML would be used as the simulation language and simulation mechanism to connect various Jack behaviors together, as well as to provide control over these behaviors. Jack objects would be encompassed in VRML nodes, and Script Nodes would communicate with these objects. VRML 2.0 geometry was simply a “bonus” for this system. While designing this Jack behavior system, now named HOW (Human OpenWorlds), we realized that such a behavior system could fit into many architectures as well, such as a CAD/CAE system or other 3-D graphics package. It was decided to design a system which would be open to as many systems, platforms, and uses as possible, hence the name OpenWorlds.
4. ARCHITECTURE OpenWorlds is designed around the notion of the node, whether it is a built-in or a script node. In addition, OpenWorlds is designed to provide similar APIs for all types of nodes. To achieve this, OpenWorlds maintains internal classes for all browser and node data, and presents the developer with external classes for accessing the internal members. The other concept central to OpenWorlds is its openness for its integration. At every level, OpenWorlds was designed not to force a developer to use OpenWorlds in any one particular way. For example, OpenWorlds has a built-in execution loop, but it can also be called on a per-frame basis. New types of VRML nodes are easily added without changing the core OpenWorlds system, as is support for other scripting languages.
3. HISTORY OpenWorlds emerged out of a VR project DRaW is developing for NASA’s Marshall Space Flight Center. DRaW is developing an immersive human VR system by creating a modular extension to the human simulation system, Jack®, developed by the University of Pennsylvania. Jack, now commercially available as Transom Jack™ from Transom Technologies Inc., is an advanced human posturing, sizing, and animation system with inverse kinematics, high-level behaviors such as balance, and extensible scripting languages including Lisp and C++ modules.
Figure 4.1
In Figure 4.1, the user-application communicates with the OW class, which encompasses the Browser, the Script Interfaces, and the Application Interfaces. The Nodes, both built-in VRML specification nodes (i.e. Sphere) and user-defined Script nodes, are external to OW and communicate through the interface classes which provide an API (such as the VRML specified Java API[1]) to the browser. The user-application also has access to the browser - more so than the nodes - as well as access to the Script Interfaces and Application Interfaces. Built-in nodes have a more restrictive view, and Script nodes are still more restrictive.
5. BROWSER The OpenWorlds browser is actually three browsers: the actual internal browser class, an external view of the browser for the application, and an external view of the browser for the nodes. The OpenWorlds browser accepts URLs to load in either directly from the application or from one of its application interfaces. The URL is retrieved, the world file is parsed, and the corresponding prototypes, nodes, and routes are created. The OpenWorlds browser holds all of the VRML prototypes and nodes which are read in. This includes the built-in prototypes, which are parsed in just like other .wrl files. The browser stores the node and prototype’s field and event information, along with a reference to where the code implementing the prototype’s behavior actually resides. There are two basic methods to have the browser traverse the scene graph: executeLoop or executeFrame. The executeLoop function enters a browser loop, which repeatedly calls executeFrame with the current time. In addition, every application interface is also called for each frame. Each time the browser’s executeFrame method is called, the browser performs several functions. The browser first traverses through all of the sensor nodes, spawning off any sensor events. Each event in turn can trigger an event cascade. The browser next traverses through the top-level built-in nodes, which perform the appropriate action (via the appropriate script interface) through the script implementing the node’s behavior. These nodes, like the sensor nodes, can additionally trigger event cascades. Note that it is the node, itself, which continues traversal of the VRML scene hierarchy. The node calls down to its children, which in turn call their children. The browser is only responsible for calling top-level nodes. This allows the implementation of the traversal to proceed as required in either a top-down or bottom-up recursive process, depending on the implementation of the nodes’ scripts.
6. NODES When a node instance is read in from a world file, the browser creates an internal representation of the node. It then creates a new script interface to the code (C++, Java, etc.) which
implements the behavior of the node. This node code uses the browser’s API to get the data of the node’s fields and events. When an SFNode or MFNode field references another node, it is actually referencing this internal representation of the node, not the actual code implementing the behavior. The C++ API for C++ nodes was modeled after the Java Script API [1]. Both our C++ API and Java API follow the Java Script API with some additions for implementing built-in nodes. As mentioned in the previous section, the traversal of the scene graph is actually performed by the nodes themselves; i.e. a node calls down to its children. To accomplish this, an action member function was added to the internal built-in node class, which in turn calls the action function on the code implementing the external node. This permits a C++ node to call the action function of a Java node, and vice-versa, thereby permitting intermingling of Java and C++ built-in nodes. The action functions of a geometry node might build up a rendering database, as with our Iris Performer version, or may perform the actual rendering itself as in the OpenGL version. A simple OpenGL built-in node from the OpenWorlds Programming Guide [2] is seen below: #include #include #include class Sphere : public BuiltinScript{ protected: SFFloat radius; public: Sphere(); void action(SFTime& t, ActionType at); }; Sphere::Sphere(): radius((SFFloat&)getField(“radius”)) {} void Sphere::action(SFTime &t, ActionType at) { if (at != SCENE) return; GLUquadricObj* quadObj = gluNewQuadric(); gluQuadricDrawStyle(quadObj, GLenum(GLU_FILL)); gluQuadricNormals(quadObj, GLenum(GLU_SMOOTH)); gluSphere(quadObj, radius.getValue(), 16, 16); }
A more complex node might have a processEvents member function, or might call down to a child node. This is demonstrated in the OpenGL Group node: class Group : public BuiltinScript{ protected: MFNode &children; … public: Group(); void addChildren(FMNode&, SFTime&); … void *action(SFTime&, ActionType, void*); void processEvent(Event&); private: deque builtins; };
Group::Group(): children(MFNode&)getExposedField(“children”) { getBuiltin(children, builtins); } Group::action(SFTime& t, ActionType at, void *) { for (int i=0; iaction(t,at); } Group::processEvent(Event &e) { //switch based on which event … }
New built-in nodes are added to OpenWorlds by creating a PROTO for that node and including in the PROTO a special field which contains the URL of the code implementing the behavior of the node, analogous to the URL field of the Script node. The appropriate Script Interface is chosen by the extension of the URL name.
7. SCRIPT INTERFACES Script Interfaces are interface classes to the external node implementations. Although called Script Interfaces, they are interfaces for both Script and Built-in Script nodes. An application can register multiple Script Interfaces. The two most common are the Java and C++ Script Interfaces. The C++ interface generally supports built-in nodes; the Java interface generally supports Script nodes (although you can make Java built-in nodes and C++ Script nodes as well). The Script Interface is the bridge between the internal node representation held by the browser (created from the VRML .wrl file) and the external node script. The internal node holds the data of the node; the external node script holds the behavior. The Script Interface permits the browser to call functions on the external node, and for the external node to get its data from the internal node and to call other nodes. The Script Interfaces maintain a reference to the external node implementation. In C++, this is an instance of the node class; in Java, this is an object of the class. When a C++ node “foo” calls the action function of a Java node “bar”, the external node foo calls the action function on the internal node bar, which finds bar’s Java script interface instance and calls its action function. This, in turn, calls the action function on the external Java node bar. When dealing with nodes of the same language, there is a method to avoid this indirection by calling the Script Interface only once and saving the direct pointer to the other node for future reference.
8. APPLICATION INTERFACES Application Interfaces are classes which communicate with and are called by the OpenWorlds browser. They can be used to implement a graphical user interface, a window manager, etc. If the user is calling executeLoop, the application interface is the
only way in which the user can communicate and interact with the browser. In essence it gives the user the ability to register a callback function. There are three main functions, or phases, associated with the Application Interfaces. These are: preProcess, process, and postProcess. The browser interacts with the interfaces by first calling each interface’s preProcess function, followed by the process function. After the process function, the browser calls the sensors and the nodes. After the nodes are traversed, the browser calls the interface’s postProcess function. A typical example of the interfaces is the graphics interface. The nodes create the scene by drawing onto an offscreen buffer once every frame; the interface handles all the rest of the work. In this example, the preProcess function prepares the top-level matrices and handles other setup functions necessary for the nodes. Then, the postProcess switches the old display buffer with the newly created one.
9. OPENWORLDS APPLICATIONS OpenWorlds has been used to create many different types of applications. The first commercially released product was our VRML 2.0 shipped by SGI for their Iris Performer™ libraries. This chapter discusses three different uses of OpenWorlds which use the full functionality of OpenWorlds run-time support. The first is the Jack VRML module, the next are stand-alone applications, and the third is a browser plug-in.
9.1
Jack Module
As mentioned previously, the impetus for OpenWorlds was to create a Jack behavior manager. The OpenWorlds Jack module accomplishes this and presents a GUI scene graph viewer/editor to the user. The VRML built-in nodes for Jack were at first implemented using our Iris GL nodes, as Jack was originally an Iris GL application. Newer nodes implement the geometry directly as Jack’s native “peabody” objects. Again, this extensibility demonstrates the benefit of the open approach. The Jack module is a dynamic shared object (DSO) which gets dynamically linked into Jack at run-time. This OpenWorlds module has a library of Jack-specific nodes in addition to support for the standard VRML 2.0 nodes. Both types of nodes can be routed together. An “h-anim” Human Animation Working Group humanoid[3] prototype wraps around each human figure. This gives a VRML file the ability to control the human figure as it would any “h-anim” figure. In addition, this h-anim figure has all of Jack’s inherent capabilities as well, such as Inverse Kinematics (IK), balance, grouped joint-limits, anthropometric scaling, constraints, and motors.
9.2
Standalone Browser
Three standalone VRML 2.0 browsers were developed in order to test and to demonstrate the OpenWorlds functionality. One was developed using SGI’s IRIS Performer, one using OpenGL Optimizer, and the other using OpenGL. The two main components necessary in all of these browsers were the graphics interface, which handled displaying and interacting with the scene, and the graphics specific nodes, such as geometry nodes, textures, etc. The OpenGL version has a top-down approach to drawing the scene. As the nodes are traversed the scene is drawn into the “current” buffer, which is maintained by the graphics interface . When postProcess() is called on the graphics interface, this buffer is switched with the buffer currently being displayed.
Figure 9.1.1 Complex simulations can be created with the GUI editor. A 2-D node graph displays routes between nodes, and permits graphically adding nodes and routes. For example, to monitor Neutral Body Posture deviation (joint deviation from a set body posture) for a NASA simulation (see Figure 9.1.1), a node routes the deviation total to a graphing node, whose type field was set to draw a bar-chart. In addition, the deviation is monitored by a node, and if it crosses a threshold, the node sends an event to an AudioClip node to play a warning message. Using the editor, the user can quickly and easily change the graph to a line graph or pie chart, or route the data to a node which records the deviation.
The Performer and Optimizer versions, on the other hand, use a bottom-up approach. In Performer, a database containing the scene is assembled by the nodes. The top level node then passes this to the graphics interface which is responsible for traversing and displaying the database. By letting the nodes handle the traversal, rather than doing it internally, we are able to handle these differences easily. In order to show how OpenWorlds fit into existing applications, the Performer version was combined with Performer’s perfly demo. Perfly, in essence, acted as the graphics interface for the browser. This allowed the browser to take advantage of perfly’s features, such as fog, database tree displaying, existing user interface, etc. An added bonus that came with perfly was the ability to load files that weren’t in the VRML format.
Figure 9.2.1
Figure 9.1.2 Figure 9.1.2 shows another simulation in Jack where Jack watches and interacts with “Floops” as he walks around. This simulation uses Jack’s IK to track floop’s position, which is passed to Jack through a VRML route.
In the perfly demo (Figure 9.2.1), any interaction with the user and the scene was passed through the graphics interface. An example of this is VRML viewpoints . The viewpoints were added to the scene at the top level. It was also necessary to give the user a means of selecting them. In order to accomplish this, pointers to the viewpoints were passed to the graphics interface. This was done so that the interface could set the top transformation to the appropriate matrix. The user interface was then able to call the graphics interface to obtain the names. This set-up allowed information to be sent from the user interface to the Viewpoint nodes via the pointers in the graphics interface.
9.3
Netscape Plug-in
11. CONCLUSIONS AND FUTURE WORK
A more obvious use of OpenWorlds is in constructing a custom VRML 2.0 browser plug-in (Figure 9.3.1). A preliminary implementation of this was debuted at the VRML 97 conference. As the windowing and event handling of the application are not inherent to the OpenWorlds libraries, a browser plug-in can use the HTML browser’s features for this. OpenWorlds simply renders to the directed window. With the open nature of the system, OpenWorlds can also use Netscape’s Java Virtual Machine and its web services. This is accomplished by replacing the default Java script interface and OpenWorlds’ URL services, both of which are accessible to the developer.
We have presented an overview of the OpenWorlds VRML 2.0 toolkit and the design philosophy behind this system. Unlike pure rendering libraries which have some VRML geometry support, OpenWorlds provides general node and scripting support. This enables use of not only VRML geometry, but VRML animations and simulations as well. In summary, OpenWorlds is intended to be easily integrated and extended to provide VRML support in any application. In this vein, more and more aspects of OpenWorlds are becoming modifiable by the user. For example, malloc() calls are being replaced by user-definable owMalloc() calls. This will allow the user to take charge of some memory management issues if so desired. The VRML 2.0 specification presented numerous behavioral extensions to the VRML 1.0 framework. OpenWorlds is a system designed around these extensions, and it will continue to evolve to support future VRML developments.
REFERENCES [1] ISO/IEC 14772-1, The Virtual Reality Modeling Language, 1997. [2] DRaW Computing Associates, Inc. OpenWorlds Programming Guide, version 0.3. http://www.openworlds.com/ow_prog_guide_0_3.html, September 30, 1997. [3] Human Animation Working Group, Specification for a Standard VRML Humanoid, Version 1.0, August 12, 1997. Figure 9.3.1 The plug-in can even support various rendering layers if new ones emerge (i.e. Java 3D), simply by swapping in new built-in nodes.
10. CURRENT STATUS At the time of writing, more features are being designed and completed. Javascript support is being added to the script node. Full multiple browser support is working in house, but not yet released. Tied in with the completion of multiple browsers is EXTERNPROTO support, which required the ability to parse two files simultaneously. More varieties of the External Authoring Interface (EAI)[4] support are in the works. A fully functional C++ EAI has been created, and the Java implementation is currently being designed. Along with the C++ EAI, an RPC (Remote Procedure Call) implementation is planned. This could potentially give multiple networked users quick and easy access to an interactive VRML database.
[4] C. Marrin, Proposal for a VRML 2.0 Informative Annex, External Authoring Interface Reference, January 21, 1997.
[5] Sun Microsystems Inc., Java™3D API Specification, Version 1.0, August 1, 1997.
