Articulating a 3D metaphor for displaying and exploring ... - Springer Link

Topical Editor: Prof. Dr. Georgios A. Triantafyllidis, 3D Res. 03, 01(2012)6 10.1007/3DRes.01(2012)6

3DR EXPRESS

w

Articulating a 3D metaphor for displaying and exploring vacation packages

N. Vidakis • D Akoumianakis • C. Katimeri

Received: 29 September 2011 / Revised: 28 October 2011 /Accepted: 19 November 2011 © 3D Research Center, Kwangwoon University and Springer 2012

Abstract * This paper presents a 3DTV application that enables displaying and exploration of thematic information in a 3D environment using the book and carousel metaphor. To achieve its goal the system initially deposits all necessary information to be displayed in an XML data repository. The stored data is then displayed through dynamically created 3D artifacts that build up a 3D carousel with hanging 3D books and flipping 3D pages separated into different sections. The user can interact and explore the displayed information through a toolbar which gives him the ability to activate the carousel, to flip pages and section of pages and to zoom on information. Keywords 3D book metaphor, 3D data visualization, Electronic 3DTV publishing

1. Introduction 3D virtual books are rapidly evolving as a metaphor for digital documents and books. For example, the British Library23 has developed an electronic 3D book called “Turning the pages” for presenting its books. Some people think of 3D virtual books as design elements in their own right12. 3D virtual books could be used to proof real books for print runs. Electronic versions of large text books used as reference, such as computer manuals, machinery operation manuals, navigation manuscripts, or academic publication proceedings, would also be compatible with the 3D book metaphor. The relevant literature reports on a variety of 3D virtual books developed to provide a different approach of their physical correspondent to access, display and explore text (see for example, “Turning the pages” of British Library23, “WebBook” of Stuart K. Card, George G. N. Vidakis ( ) • D Akoumianakis • C. Katimeri Applied Informatics and Multimedia Dept., Technological Educational Institute of Crete, Heraklion, Greece email: [email protected]

Robertson and William York12, Zinio Reader24 and FlipViewer25). Our approach of 3D virtual books for accessing and exploring data discussed in this paper uses suitable metaphors like the carousel. The key difference with other 3D virtual books is the XML data repository which is used to store data and then display it through dynamically created 3D artifacts that build up a 3D carousel with hanging 3D books and flipping 3D pages rather than using static images of book pages. The rest of the paper is structured as follows. The next section reviews the state of the art in 3d graphics libraries, 3D metaphors and graphical representation of data by discussing main techniques methods and toolkits. Then we present the BookCarousel system, its design specifications and metaphor used for displaying and exploring the data. Following this, we briefly discuss the implementation of the 3DTV system and the paper concludes with a comparison of similar systems and future improvements.

2. Experimental 2.1. 3D graphics libraries 3D graphics have become so popular, particularly in video games, that specialized APIs (Application Programming Interfaces) have been created to ease the processes in computer graphics generation. The first 3D graphics framework was probably Core, published by the ACM in 197720. In general 3D APIs are categorized in Low-level which are mainly concerned with rendering and High-level which provide additional functionality on top of the lowerlevel APIs. Indicative exemplars of such APIs are: (a) lowlevel: X3D, OpenGL and the OpenGL Shading Language, OpenGL ES, Direct3D, RenderMan, RenderWare, Glide API, QuickDraw 3D, WebGL, etc. and (b) high-level: Java 3D, ClanLib, CrystalSpace, HOOPS 3D Graphics System,

2

3D Res. 03, 01(2012)6

Irrlicht Engine, JMonkey Engine, Mobile 3D Graphics API, JT Open, Nvidia Scene Graph, OGRE, Open Inventor, OpenGL Performer, OpenSceneGraph, OpenSG, CopperLicht, O3D, Away3D etc. 3D API libraries such as Open Graphics Language (OpenGL) and Microsoft DirectX® (DX) hide the implementation algorithms of rendering, such as lighting, from the developer. Shading Languages such as OpenGL

Shading Language (GLSL) or DirectX High Level Shading Language (HLSL) allow developers to make changes to these built-in rendering algorithms. Graphics applications created using 3D APIs that do not override the built-in functionality are said to be using the “fixed functionality pipeline”, and the applications that do override the built-in functionality are said to be using the “programmable functionality pipeline”.

Fig. 1 Scene-graph

2.1.1. The Java 3D API Java 3D is a scene graph-based 3D API32 for the Java platform. It runs atop either OpenGL or Direct3D. Compared to other solutions, Java 3D is not only a wrapper around these graphics APIs, but an interface that encapsulates the graphics programming using a real, objectoriented concept. Here a scene is constructed using a scene graph (Fig. 1) that is a representation of the objects that have to be shown. This scene graph is structured as a tree containing several elements that are necessary to display the objects. Additionally, Java 3D offers extensive specialized sound support. Making 3D graphics interactive is a long−standing problem, as evidenced by its long history of algorithms, APIs, and vendors. The dominant industry standard for interactive 3D graphics is OpenGL, created by Silicon Graphics (SGI). OpenGL was designed as a cross−platform rendering architecture and is supported by a variety of operating systems, graphics card vendors, and applications. The OpenGL API is written in the C programming language, and hence not directly callable from Java. A number of open source and independent programming efforts have provided simple Java wrappers over the OpenGL API that allow Java programmers to call OpenGL functions, which are then executed in native code that interacts with the rendering hardware. One of the most popular is GL4Java31. However, there are few disadvantages to using a Java wrapper over OpenGL, as opposed to coding in C and calling OpenGL directly. Although programmers can use

the more friendly Java APIs, they must incur the overhead of repeated calls through the Java Native Interface (JNI) to call the native OpenGL libraries. Java 3D relies on OpenGL or DirectX to perform native rendering, while the 3D scene description, application logic, and scene interactions reside in Java code. When Sun set out to design Java 3D, although they did not have the resources or industry backing to replace OpenGL, they wanted to leverage more of Java’s strengths as an object−oriented programming (OOP) language instead of merely delegating to a procedural language such as C. Whereas OpenGL’s level of description for a 3D scene consists of lists of points, lines, and triangles, Java 3D can describe a scene as collections of objects. By raising the level of description and abstraction, Sun not only applied OOP principles to the graphics domain, but also introduced scene optimizations that can compensate for the overhead of calling through JNI5.

2.1.2. Example systems developed with 3D technology SciDelik (Fig. 2) is an advanced scientific visualization and animation system based on Java and the 3D scene-graph graphics API, Java 3D, of Sun Microsystems, Inc. SciDelik has been under development for the past several years and is arguably the first full-featured, turnkey, scientific visualization package of its kind based on Java19.

3D Res. 03, 01(2012)6

3

Another example of 3D technology systems is the Open Wonderland (originally Project Wonderland) (Fig. 3) open source toolkit for creating collaborative 3D virtual worlds. Important goals of Wonderland is for the environment to be

completely extensible, to support content creation within the world and to support importing art from open source 3D content creation tools as well as professional 3D modeling and animation applications18.

Fig. 2 SciDelik

Fig. 3 Project Wonderland

4

3D Res. 03, 01(2012)6

Fig. 4 Looking Glass 3D

Project Looking Glass 3D (LG3D) (Fig. 4) is a now inactive free software project under the GPL to create an innovative 3D desktop environment for Linux, Solaris, and Windows. It was sponsored by Sun Microsystems. Looking Glass is programmed in the Java language using the Java 3D system to remain platform independent17.

2.2. 3D metaphors

Fig. 6 The Room metaphor

Fig. 5 The (Desktop) metaphor

A metaphor is a literary figure of speech that uses an image, story or tangible thing to represent a less tangible thing or some intangible quality or idea. Metaphor may also be used for any rhetorical figures of speech that achieve their effects via association, comparison or resemblance1,2. Aristotle22 used both this sense and the regular, current sense above. Metaphors are used to describe things or situations and convey information to humans in a more understandable manner. Developers use metaphors to impress users and draw their interest4.

Hamilton8 refers to the Indurkhya definition of a metaphor: “A metaphor is a description of an object or event, real or imagined, using concepts that cannot be applied to the object or event in a conventional way. The object or the event being described is called the target, and the concepts that cannot be applied conventionally are called the source. The metaphor is made meaningful by interpreting the source unconventionally in the target. The unconventional interpretation can be arrived at on the basis of some underlying similarity between the source concepts and the target.” Dubinsky and Hazzan11 note that there must be a difference between the target and the source otherwise the metaphor becomes the object itself. An Interface metaphor is a set of user interface visuals, actions and procedures that exploit specific knowledge that users already have of other domains. The purpose of the interface metaphor is to give the user instantaneous knowledge about how to interact with the user interface.

3D Res. 03, 01(2012)6

5

Interface metaphors are used to help conceive unknown or difficult to understand things. The most commonly used interface metaphor is the one of the desk (desktop metaphor) (Fig. 5). The desktop metaphor treats the monitor of a computer as if it is the user's desktop,

upon which objects such as documents and folders of documents can be placed. Another popular metaphor is the “room metaphor”. Rooms used in software are similar to the rooms of a building (Fig. 6). The room metaphor is commonly used in collaborative environments4.

Fig. 7 Indicative exemplars of 3D Books

2.2.1. The “Book metaphor” The book metaphor has been used in applications by a number of people for some time. The main characteristics of this metaphor are the distinct role of same pages e.g. content page, the grouping of pages to chapter etc. Apart from the above characteristics readers tend to interact with traditional book in more ways than turning pages21 such as underline things, write notes, bend pages etc4. In bibliography we can find many different developments of the book metaphor. Indicative exemplars (Fig. 7) are the British Library’s “Turning the pages” project23 which uses Macromedia Director technology for photo capturing, the WebBook12 of Card, Robertson και York (1996), the 3Book3 of Hong, Card, Chen, Zinio Reader and the Flip Viewer. We will examine them in more detail in the next sections

2.3. Graphical Representation of data The term graphical representation of data includes concepts such as data visualization, information graphics, knowledge visualization, scientific visualization visual design9,10. According to Friedman13 the main goal of data visualization is to communicate information clearly and effectively through graphical means. Post et al14 says that data visualization connected the two knowledge areas scientific and information visualization. Jacques Bertin26, a French cartographer published “The Semiology of Graphics”. Edward Tufte in 198327 published a theory for data graphics giving emphasis on maximize the density of useful information. Tukey in 197728 published a work on Exploratory Data Analysis giving emphasis on the usage of graphs. Gleveland and McGill29 wrote a book on Dynamic Graphics for Statistics. The first use of the term “Information representation” was made by Robertson, Card and Mackinlay30. Shneiderman in 19926 developed the “dynamic queries” while Card, Robertson and Mackinlay7 developed the use of animation for interacting with large volumes of data in a system called “Information Visualizer”. Different techniques and methods have been developed over the years for the graphical representation of data. Some

of the most important are: (a) the “Halo” method which is used for display large papers in small displays. “Halo” has been used for map presentation on PDA’sand mobile phones, (b) the “Cladograms” used for relationships representation, (c) “Color alphabet”, (d) “Dendrogram” uses hierarchical grouping, (e) “Hyperbolic Tree”, (f) “Treemapping” tree based representation of data etc.

3. Related work Lately a few projects have started for the digitization of large numbers of books and documents. The Million Book Project of Carnegie Mellon University is one of them and its goal is to scan 1 million books and give them to the public. Recently Google, Yahoo, and Microsoft announced separately plans for digitizing millions of printed books with search facilities over the internet3. In order to capture the appearance and the feeling of a printed book the British Library has developed an interactive system called the “Turning the pages”23. With the use of animations, visitors have the capability to turn pages (Fig. 7). “WebBook” of Stuart K. Card, George G. Robertson and William York (Fig. 7) it has been developed as a component of the Web Forager and it is a 3D book using HTML pages. The user assembles HTML pages by “dropping” them into a book. Then he can navigate through the book via mouse clicks and hyperlinks. The “WebBook” has a limitation on the number of pages as if they exceed the 30 pages it becomes very slow. Learning from the “WebBook” experience Lichan Hong, Stuart K. Card, and Jindong (JD) Chen) have developed the “3Book” which they plan to use as a basic component on ebook systems, digital libraries and information workspaces. “3Book” (Fig. 7) tries to provide a natural feeling on page turning and thus it contains a page bending while turning. Unlike “WebBook”, the “3Book” is able to handle unlimited sized books. Zinio Reader24 and FlipViewer25 are two commercial products for presenting pages in a 3D manner.

6

3D Res. 03, 01(2012)6

None of the above systems uses a generic data repository such as XML which provides fast and easy access to data and adds extroversion characteristics.

4. The BookCarousel system This section describes the system architecture and design specifications as well as the system itself. The 3DTV BookCarousel system was implemented with the Java 3D library. Apart from Java 3D the JDOM library33 was used for manipulating data stored in XML from within Java15,16.

JDOM is an open source Java-based document object model for XML that was designed specifically for the Java platform so that it can take advantage of its language features. JDOM interoperates well with existing standards such as Document Object Model (DOM)34 and Simple API for XML (SAX)35. Our reference data model is presented in Fig. 15, 16 & 17 which detail a relational view upon vacation packages in the area of regional tourism from the eΚοΝΕΣ project (see Acknowledgement) and could be easily adopted (only the data in the XML need to be changed) to present context from different context areas.

Fig. 8 The turning pages mechanism

Design Specifications Turning pages The main concern of the method used for turning the pages of the 3D books that “hang” on the Carousel was to support unlimited number of books hanged and unlimited number of

pages per book. To do so we have developed a method for turning pages where at any given time only four pages of the book are constructed (Fig. 8). At the beginning as the book is closed two pages are constructed. As the user turns pages backwards or forwards pages are constructed or destroyed accordingly and switch positions between top and bottom.

3D Res. 03, 01(2012)6

7

Fig. 9 Initial XML file structure

8

3D Res. 03, 01(2012)6

Fig. 10 General XML file structure

Fig. 11 Mapping of initial XML with general XML

3D Res. 03, 01(2012)6

9

Fig. 12 XML management architecture

Fig. 13 Class Diagram–XML management

XML to display All information presented with the 3D BookCarousel is stored in XML files. At the present version of the system changes to the information content is made manually by changing the XML files. At a future version of the system an information content management system is planned to be developed to support automatic translation of information from original form, such as plain text etc., to the XML format supported by the system. Each page of the 3D book presents information such as name, description, tel. etc. concerning the business partners of the vacation package. Business partners are separated

into business categories. Each category is represented as a different 3D book. The class XmlCorrelator recovers the data from the initial XML (Fig. 9) file with the use of the JDOM library. Then the system associates the recovered data with a general XML structure (Fig. 10) which is used for the data presentation. The association of the initial XML file with the general XML file can be seen in Fig. 11. The general XML file is then read by the class XMLParser which is responsible for inserting data into pages and sections. The model used for the data management is showed in Fig. 12 and the class diagram of the presentation of the data is showed in Fig. 13.

10

3D Res. 03, 01(2012)6

Fig. 14 Class Diagram – Book

Fig. 15 XML data and 3D Book mapping

3D Res. 03, 01(2012)6

11

Fig. 16 Class and 3D Book mapping

12

3D Res. 03, 01(2012)6

Fig. 17 Page turning

Fig. 18 Turning the top textbox

The Application Each 3D book contains pages and a toolbox for navigation. Each page contains 3 sections arranged vertically. The top and lower sections contain text and have flipping page capabilities where the middle section contains images with multiple image page capabilities. Images are dynamically loaded from the web as the XML file contains the necessary image address. The relationships and objects involved in the implementation of the 3D presentation of data described above are showed in Fig. 14. The data mapping between the general XML and the 3D presentation is showed in Fig. 15

where the mapping of the class diagram with the 3D presentation is showed in Fig. 16 Fig. 17 shows the page flipping which uses the turning mechanism described earlier. If we follow image 1 through to image 4 we can see o orthogonal rectangle to flip from right to left. The 3D native component “rectangle” of the Java 3D library has been used. As mentioned earlier each page contains three sections. The top and bottom section present text and have flipping abilities similar to the one described for the page. Fig. 18 from image 1 to 4 illustrates the sections turning ability of the BookCarousel system.

3D Res. 03, 01(2012)6

13

Carousel

Zoom out book Fig. 19 Carousel metaphor

Fig. 20 Six book Example

Fig.21 Eight book Example

The Carousel Metaphor

5. Summary and conclusions

The carousel metaphor has been used for setting up books in a cyclical arrangement (Fig. 19). At the top of the carousel stands the “book title” which can differ according to the data presented below. Books or chapters are attached to the carousel with straight lines. Each line represents a 3D Book or chapter which can be rotated upon user selection. The 3D Book that the user selects comes in the front scene by carousel rotation. The number of 3D books, chapters and pages is unlimited and depends on the data stored in the XML file. The BookCarousel system displays any number of books and pages that will be found in the XML file. Fig. 20 and 21 show a 6-book carousel and a 8-book carousel accordingly.

This paper presents a 3DTV system named BookCarousel that uses the carousel and the book metaphor for displaying and exploring vacation package information. The technology used is the Java3D and JDOM libraries. The use of JDOM was necessary in order to recover data from the XML files. The 3D artifacts (carousel, book, page and section) are dynamically created depending on the data encountered in the XML data repository used. 3D books are positioned in a cyclical order and by selecting them the user activates the carousel which turns and brings the selected book in the front scene.

14

3D Res. 03, 01(2012)6 7.

Comparison with similar systems The BookCarousel system has some similarities with other related work systems and some differences. The table below summarizes the main similarities and differences with other systems. BookCarousel

WebBook

3Book

Yes No

Yes No

Yes Yes

Yes

No

No

Yes

No

Yes

Yes

No

No

Page Turn Page Bend Active page sections Unlimited Pages 3D book animation

8. 9.

10. 11. 12. 13.

Future Improvements

14.

At a future version of the system an information content management system is planned to be developed to support automatic translation of information from original form, such as plain text etc., to the XML format supported by the system. More interaction through annotation facilities would be another area of improvement. Predefined page setup templates from which the user could choose would allow users to construct their personal display of the 3D books. Finally a more natural feeling of page turn with page bending would add a better appearance of the animation. Different reference models from areas such as 3DTV travel documentary film or 3D television program that describes travel in general or tourist attractions will be considered in our future work.

15.

Acknowledgment

22.

This work is motivated by eKoNEΣ which is funded by GSRT, Greek Ministry of Development, in the context of the Operational program of the Region of Crete.

References 1. 2. 3. 4. 5. 6.

J. Keränen (2005) Using metaphors in computer science education – cross cultural aspects. University of Joensuu. T. R. Colburn, G. M. Shute (2008) Metaphor in computer science. Department of Computer Science, University of Minnesota, Duluth, MN, USA L. Hong, S. K. Card, J. D. Chen (2006) Turning Pages of 3D Electronic Books. Palo Alto Research Center, USA D. Akoumianakis (2006) Human Computer Interfaces, a modern approach, Kleidarithmos D. Selman (2002) Java 3D Programming C. Ahlberg, C. Williamson, B. Shneiderman (1992) Dynamic queries for information exploration: An implementation and evaluation, Proc. ACM CHI'92: Human Factors in Computing Systems, p.p. 619-626.

16. 17. 18. 19. 20.

21.

23. 24. 25. 26. 27. 28. 29. 30.

31. 32. 33. 34. 35.

S. K. Card, G. G. Robertson, J. D. Mackinlay (1993) "Information Visualization Using 3D Interactive Animation". In: Communications of the ACM, 36(4): 57-71. A. Hamilton (2000) Metaphor in theory and practice: the influence of metaphors on expectations. ACM J. Comput. Doc. 24: 237-253 S. Chithirapoovizhi, R. Parthasarathi, T. V. Geetha (2003) A Tool for Document Visualization. Resource Centre for Indian Language Technology Solutions – Tamil, School of Computer Science and Engineering, Anna University, Chennai, India. S. K. Card, J. D. Mackinlay, B. Shneiderman (1999) Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann Publishers. D. Yael, Hazzan, Orit (2003) Using Metaphors in eXtreme Programming Projects, in Lecture Notes in Computer Science, Springer Berlin / Heidelberg, volume 2675 S. K. Card, G. G. Robertson, W. York (1996) The WebBook and the Web Forager: An Information Workspace for the World-Wide Web, ACM Press, pages 111-117 V. Friedman (2008) Data Visualization and Infographics, Graphics, January 14th, 2008. F. H. Post, G. M. Nielson, G. P. Bonneau (2002) Data Visualization: The State of the Art, Research paper TU delft E. R. Harold (2002) Processing XML with Java, http://www.cafeconleche.org/books/xmljava/chapters/index.h tml The apache XML project, http://xmlbeans.apache.org/index.html Project Looking Glass, http://www.sun.com/software/looking_glass Project Wonderland, https://lg3d-wonderland.dev.java.net/ SciDelik. Scientific Visualization And Animation System, http://sites.google.com/site/scidelik/ T. H. Cormen, J. Wegmann, D. M. Nicol (1997) Multiprocessor out-of-core FFTs with distributed memory and parallel disks (extended abstract). In Proceedings of the fifth workshop on I/O in parallel and distributed systems (IOPADS '97). ACM, New York, 68-78. Y.-C. Chu, I. H. Witten, R. Lobb, D. Bainbridge (2003) How to turn the page. In Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries (JCDL '03). IEEE Computer Society, Washington, 186-188. Aristotle. Poetics. Trans. I. Bywater (1984) in The Complete Works of Aristotle: The Revised Oxford Translation, 2 Vols. Ed. Jonathan Barnes. Princeton, Princeton University Press. http://www.bl.uk/onlinegallery/ttp/ttpbooks.html https://www.zinio.com/account/download-reader-page.jsp http://www.flipviewer.com/ Jacques Bertin, Sémiologie Graphique, ISBN 2-7132-1277-4, 442 pp., Les Re-impressions des Editions de l'Ecole des Hautes Etudes En Sciences Sociales E. Tufte (2001) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press. J. W. Tukey (1977) Exploratory Data Analysis, publish by Addison Wesley. W. S. Cleveland, M. E. McGill (1988) Dynamic Graphics for Statistics, published by Springer. G. G. Robertson, S. K. Card, J. D. Mackinlay (1989) The cognitive coprocessor architecture for interactive user interfaces, In Proceedings of the ACM SIGGRAPH Symposium on User Interface Software and Technology, pages 10-18. http://www.jausoft.com/gl4java/ http://java.sun.com/docs/books/java3d/ http://www.jdom.org http://www.w3.org/DOM http://www.saxproject.org/