An OWL-based extensible transcoding system for mobile multi-devices

Journal of Information Science http://jis.sagepub.com

An OWL-based extensible transcoding system for mobile multi-devices I. -Ching Hsu and Shang-Juh Kao Journal of Information Science 2005; 31; 178 DOI: 10.1177/0165551505052346 The online version of this article can be found at: http://jis.sagepub.com/cgi/content/abstract/31/3/178

Published by: http://www.sagepublications.com

On behalf of:

Chartered Institute of Library and Information Professionals

Additional services and information for Journal of Information Science can be found at: Email Alerts: http://jis.sagepub.com/cgi/alerts Subscriptions: http://jis.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav

Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 15, 2008 © 2005 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

An OWL-based extensible transcoding system for mobile multi-devices

I.-Ching Hsu and Shang-Juh Kao Department of Computer and Information Science, National Chung-Hsing University, Taichung, Taiwan, ROC Received 21 September 2004 Revised 11 January 2005

Abstract. Due to the rapid Internet development, an answer to the challenging question of how to automatically convert heterogeneous information into the desired XML format that can be accepted by a designated device is urgently requested. In this paper, we propose an OWL-based extensible transcoding system (OETS) which is based on an extensible transcoding policy infrastructure, with a hierarchical structure of ontologies, and can automatically render an HTML document in mobile devices. In order to create common knowledge of OETS, we develop two top level ontologies: procedural sensor/effector ontology (PSEO) and markup file taxonomy ontology (MFTO). PSEO combines with situated logic to support sensor/effector procedures attached to an ontology class, while MFTO supports the markup file and structural components classification. Developers can exploit the extensible transcoding capability of OETS to satisfy various layout-specific structures of HTML pages by providing dedicated domain ontologies.

Keywords: OWL; OETS; situated logic; transcoding; agent; ontology Correspondence to: I-Ching Hsu, Department of Computer and Information Science, National Chung-Hsing University, 250 Kuo-Kuang Rd, Taichung, Taiwan, ROC. E-mail: phd9203 @cs.nchu.edu.tw

178

1. Introduction Along with the current trend that various mobile devices, such as mobile phones, personal digital assistants (PDAs), and automobiles, can access the Internet, the same web content needs to be rendered accordingly. Taking into account the physical restrictions on end devices, such as screen size, memory amount, and connection bandwidth, a significant challenge occurs in automatically converting heterogeneous information (such as that in HTML, XML, WML, VoiceXML, etc. formats) on the Internet into the desired XML format which can be accepted by a specific device (such as browser, WAP phone, PDA, telephone, etc.). Heterogeneous markup language document conversion is a type of transcoding, which is the process of dynamically mutating and customizing a document based on the characteristics of the requesting device and on the user’s preferences. Essentially, there exist three different approaches for the generic transcoding of markup language documents. First, the flat-based transcoding approach allows the transcoder to directly convert a markup language file from one format to another without referring to any external data. This transcoding approach is the simplest one. HTML Tidy [1], which converts HTML files to XHTML files, is an example. The main problem with flat-based transcoding is its lack of flexibility in transcoding policy for transcoders. In other words, the fixative transcoding policy is embedded in transcoders and focuses on specific tasks with relatively small dimensions. The second approach, template-based transcoding, utilizes a template expressed in transformation language to describe rules for transforming XML documents into other XML documents. A lack of flexibility

Journal of Information Science, 31 (3) 2005, pp. 178–195 © CILIP, DOI: 10.1177/0165551505052346 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 15, 2008 © 2005 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

I.-C. HSU AND S.-J. KAO

in transcoding policy can be solved by making use of template-based transcoding. An example is Extensible Stylesheet Language (XSL) [2], which has been defined by the World Wide Web Consortium (W3C) for XML documents transformation. XSL includes both a transformation language (XSLT [3]) and a formatting language. XSLT provides elements that define rules for transcoding one XML document into another XML document, while formatting language includes an XML vocabulary for specifying formatting. The transcoding from a source markup document to the target markup document is described by means of an XSL stylesheet which consists of templates and transformation rules. The main limitations in template-based transcoding are the limited tags of the transformation language and lack of knowledge representation. The third approach, annotation-based transcoding, creating a metadata by annotating markup documents, is one of the Semantic Web [4] techniques for transcoding agents that can automatically convert heterogeneous information to a desired XML format. In this approach it is important to note that the result of applying an annotation to a target markup document depends on a particular transcoding policy and on knowing the particular needs of the target device. The role of the external annotation is to provide hints for a transcoding policy to make better decisions on content adaptation given the device’s characteristics. In this paper, we present an extendable transcoding policy infrastructure that uses OWL (Ontology Web Language) [5] to present knowledge and an ontologybased transcoding agent to perform transformation using the semantic content of annotation. Our OWLbased extensible transcoding system (OETS) has two major advantages over other annotation-based transcoding systems. First, OETS takes an ontology multiple-layered approach to facilitate transcoding policy with extensive flexibility by providing a variety of domain ontologies for various layout-specific structures of HTML pages. Second, OETS is based on the emerging web ontology language OWL rather than RDF [6]. The main advantages of OWL over RDF are efficient reasoning support, sufficient expressive power, and convenient expression. Within OETS, two top level ontologies are developed: procedural sensor/effector ontology (PSEO), which combines with situated logic [7] to support sensor/effector procedures attached to an ontology class, and markup file taxonomy ontology (MFTO), which provides the markup file and structural components classification. In order to demonstrate the feasibility of OETS, we developed HTML forms ontology (HFO), a domain

ontology, to describe HTML forms layout structure and to track routing information about form items. In addition, we set up an ontology-based routing model (OBRM) to provide the non-sequential navigation feature for HTML forms applications, such as ecommerce and Internet questionnaires. By applying the OETS, we show how to automatically convert HTML forms to WML forms for a WAP phone, and VoiceXML forms for a voice phone. The structure of the paper is organized as follows. We give the formal definition of ontology multiplelayered architecture, and describe the framework of the OETS next. In Section 3, the development of two top level ontologies, PSEO and MFTO, is described. HFO, OBRM, and an automatic transcoding algorithm are included in Section 4. An application scenario and system extensibility are presented in the following section. Finally, summary and concluding remarks are included.

2. System architecture OWL, defined by W3C, is more capable of expressing semantics than XML, RDF, and RDF Schema [8], and it is superior to others in marking itself as machine readable. In our transcoding system, we propose an OWL-based multiple-layered approach to allow extensive flexibility for annotation-based transcoding policy. 2.1. Ontology multiple-layered approach An ontology is commonly defined as an explicit, formal specification of a shared conceptualization of a domain of interest. That is, an ontology describes some application-relevant part of the world in a machine understandable way [9]. The core ingredients of an ontology include a set of concepts, a set of properties, and the relationships between the elements of these two sets. In this paper, the ontology supports the following functions: (1) retrieving the appropriate information from documents by providing a structure to annotate the contents of a document with semantic information [10, 11], for instance, an annotation of an HTML page associates with HFO; (2) ensuring consistency and correctness by formulating constraints on the content of information [12], for instance, development of PSEO is written in OWL and combines with situated logic to support sensor/effector procedures attached to an ontology class;


179

OWL-based extensible transcoding system

(3)

supporting inference to derive additional knowledge from a set of facts [13], for instance, OWL adds more vocabulary for describing properties and classes: relations between classes (e.g. disjointness), cardinality (e.g. ‘exactly one’), equality, richer typing of properties, characteristics of properties (e.g. symmetry), and enumerated classes; (4) providing the high level taxonomy from a specific domain to include super- and sub-concepts [14], for instance, MFTO aims at providing the infrastructure for interoperability of terms in the domains of markup language. It has been widely accepted that ontologies and metadata are the core elements of the Semantic Web. In the following, we introduce a formal model of

ontologies and metadata, focusing specifically on the interaction of ontologies and association of metadata with natural language. By integrating these concepts, we construct an ontology multiple-layered architecture which is composed of four layers: meta layer, language layer, ontology layer, and instance layer, as shown in Figure 1. Definition 1 (meta layer). The meta layer contains rdfs:Class only. The rdfs:Class is the root class in the RDF data model. Any other class is regarded as an instance of the rdfs:Class. Definition 2 (language layer). RDF, RDFS, DAML+OIL [15,16], and OWL are the candidates in the language

r df s :s ubC las sOf

Meta Layer

rdfs: C lass

rd f s : s u b P r o p e r t y O f rdf:type

rdfs: R esource

Language Layer rdfs: Property owl:DatatypeProperty

Ontology layer (Top Level Ontology)

pseo:parameterName ps eo:ef f ec tor p s e o : se n s o r

PSEO

mfto:component

pseo: content

mfto:XMLFile p s e o : In p u t pseo: O utput

mfto:For m mfto:Frame

mfto:WMLFile mfto:HTMLFile mfto:XHTMLFile hfo:formURL

hfo:triggerOutput hfo:tidyOutput

hfo:tidyInput

hfo:selectItem hfo: value

hfo: Tidy hfo:extractionOutput hfo:jumpTo hfo:triggerInput hfo: Extraction hfo:HTMLForm hfo:formTit le hfo:itemValue hfo:extractionInput hfo:itemSet h f o : d e s c r i p t i o n hfo:formItem

HFO

Instance Layer Annotation

mf t o : M a r k u p F il e m ft o : Fr a g me n t

pseo: attachment

pseo: C lass

hfo:triggerNext

MFTO

ow l:Cla s s

owl:ObjectProperty

pseo:procedureSet

Ontology Layer (Domain Ontology)

rdfs: C lass

Qu e s t i on-1 forGuest Qu es tio n -2 Qu es tio n -3 Ques tion- 4 Q ue s t ion-5

answer-No answer-Yes

Fig. 1. Ontology multiple-layered architecture.

180



layer. OWL has more facilities for expressing meaning and semantics than XML, RDF, and RDF Schema, and thus OWL goes beyond these languages in its ability to represent machine readable content on the Web. OWL is a revision of DAML+OIL, and aims to be the standardized and broadly accepted ontology language of the Semantic Web. Therefore, OWL should integrate well with other W3C recommendations and not change too much from DAML+OIL. These ontology languages are adopted to create ontologies: top level ontology and domain ontology. Definition 3 (ontology layer). A web-based ontology is a tuple O = , where: • C is a set of concepts representing classes in an ontology; • P is a set of relations representing properties in an ontology; • α is the hierarchical relation function for classes; α: C → C, α(C1) = C2 means that C1 is a subclass of C2; • β is the hierarchical relation function for properties; β: P → P, β(P1) = P2 means that P1 is a subproperty of P2; • γ is the attribute relation function between classes; γ: P → C ⫻ C, γ(P1) = (C1, C2) means that the domain of P1 is C1 and the range of P1 is C2; • Σ is a set of ontology axioms, expressed in an appropriate logical language, e.g. first order logic; • Π is a set of RDF-based ontology language, such as RDF Schema, DAML+OIL, or OWL. Generally, there are two different types of ontologies: a top level ontology and a domain ontology [17]. The former describes a general concept upon which other ontologies are created. The latter defines the terminology and specification relevant to a particular topic of interest. The main purpose of the top level ontology is to provide a unified view through which various domain ontologies can inherit common knowledge and semantic schema [18]. Our approach is to take into consideration the mobile device interface for HTML pages. For adaptation, HTML page structures require rearrangement, i.e. transcoding, to better fit on a mobile device interface, especially when layout-specific HTML structures like forms, tables and frames are used. In our OETS, we employ domain ontology to describe the semantics of layout-specific HTML structures [19]. Developers can exploit domain ontologies for various layout-specific structures of HTML pages to plug into our OETS to enhance transcoding capability.

In OETS, the HFO is a semantic schema of annotation which is created manually and is used to record the metadata of an HTML form. These ontologies and annotations will be described in more detail later. Definition 4 (instance layer). An ontology-based annotation is a tuple A = , where • O is an ontology representing the semantic schema of annotations; • I is a set of instance identifiers; • Λ is the class instance function; Λ: C → 2I, Λ(C1) = I1 means that each member of I1 is an instance of C1; • Υ is the property instance function; Υ: P → 2I⫻I, Υ(P1) = I1 ⫻ I2 means that each member of (I1 ⫻ I2) is an instance of P1. The instance layer is the annotation repository of web pages, which represents domain-specific individuals. The annotation refers to adding metadata to markup documents to provide a higher level interpretation of the documents. In most existing web annotation applications, such as Yawas [20] and Annotea [21, 22], the annotations do not conform to some common semantic structures that are pertinent to the underlying domains. To overcome this limitation, several attempts have been made to create annotations based on some well-defined semantic structures and knowledge representations, known as ontologies [23]. Existing annotation-based transcoding systems [24–26] are all based on RDF which is the basic technology for ontologies. However, RDF alone does not share some basic common structure, which would help to describe classes of resource and types of relationship between resources. Thus, we need more facilities than RDF offers for expressing semantics. Example. Let us consider a short example of an instantiated ontology and metadata structure as depicted in Figure 1. Here, the top level ontology is {PSEO, MFTO}, the domain ontology is {HFO}, and an annotation is given. The partial ontology architecture can be represented as: C = {pseo:Class, hfo:HTMLForm, hfo:formItem}, P = {pseo:effector, hfo:itemSet, hfo: triggerNext}, α(hfo:HTMLForm) = pseo:Class, α(hfo: formItem) = pseo:Class, β(hfo:triggerNext) = pseo: effector, and γ(hfo:itemSet) = (hfo:HTMLForm, hfo: formItem). Additionally, the following metadata statements are defined. Λ(hfo:HTMLForm) = {forGuest}, Λ(hfo:formItem) = {Question-1, Question-2, Question-3, Question-4, Question-5}.


181


2.2. System components

established by the annotator and they are instances of the domain ontology. The instance base plays the same role as the fact base in a traditional expert system. The transcoding agent is an ontology-based intelligent agent [27] that can parse ontologies and extract information from annotations. Moreover, the transcoding agent can start sensor/effector procedures and communicate with these procedures to automatically convert Internet documents in HTML format into the desired XML format, such as WML and VoiceXML. The transcoding agent can play the role of an inference engine. The information flow of the transcoding agent occurs as follows: (1) The client device sends a request URL with a device type to the transcoding agent. (2) The transcoding agent formats the URL’s request to the web server. (3) The web server sends the target document to the transcoding agent. (4) The transcoding agent accesses the instance base to retrieve annotations related to the target document and the device type.

The OETS architecture is depicted in Figure 2. It contains four components: ontology base, instance base, procedure base, and transcoding agent. The ontology base is composed of ontologies that populate the specific domain of interest. These ontologies are divided into two categories: top level ontology and domain ontology. Top level ontologies, PSEO and MFTO, describe the common concepts for code conversion, while domain ontology quotes from the top level ontologies and describes the specific layout structure of the HTML page. HFO is an example of domain ontology to describe HTML forms layout structure. The ontology base can be treated as the knowledge base in a traditional expert system. The procedure base is composed of sensor/effector procedures that are attached to specific classes. These procedures are started automatically in a particular status by the transcoding agent. The instance base represents domain specific individuals, that is, annotations of HTML pages. These annotations are manually

P r o ce d u r a l Sensor/E ffector Ontology

Workflow reference

HTML Forms Ontology ( d o ma i n o n t o l o g y )

HTML doc XML do c WML doc VoiceXML… . Internet Documents Web Server

manual annotation

Ma rkup F ile Taxonomy Ontology

Tidy P ro c e d u r e [Sensor] Extraction P ro c e d u r e [Sensor] trig g e rNe x t P ro c e d u r e [Effector]

Ontology Base

5 .o ntology

an nota tions of HTML pages

Web Browser

Procedure Base PDA

6.call procedure

I ns t a n c e B a s e 4.annotation 3.response

Transcoding Agent

2. request

WAP Phone 7.transcoded document 1. request

OWL-based Extensible Transcoding System

Phone

…….. Client Device

Fig. 2. OWL-based extensible transcoding system architecture.

182



(5)

Depending upon the annotations, device type, and target document, the transcoding agent performs corresponding transcoding. During the transcoding process, the transcoding agent refers to the ontologies (e.g. domain ontology, top level ontology) for assistance. (6) The transcoding agent triggers the sensor/effector procedures. (7) Finally, the transcoding agent passes the transcoded document to the requesting client. The detailed transcoding algorithm of step 5 and step 6 is described in Section 4.4.

3. Top level ontologies Within OETS, two top level ontologies, PSEO and MFTO, are defined. In the following, we show how to explicitly express PSEO and MFTO in OWL. 3.1. Procedural sensor/effector ontology The PSEO can support sensor/effector procedures attached to an ontology class. The idea of procedural sensor/effector comes from the situated logic program. The sensor procedure is started automatically as a transcoding agent performs the class retrieval, while the effector procedure is started automatically as a

transcoding agent releases the class. Figure 3 shows the PSEO structure between classes, class inheritance, and properties. The pseo is the namespace that represents the ontology of PSEO.1 The xsd and owl are namespaces of XML Schema and OWL, respectively. The pseo:Class is an instance of owl:Class, which represents the common case of class which contains at least a sensor or an effector procedure. We refine the class by using an owl:unionOf to impose a pseo:sensor property or pseo:effector property. The pseo:procedureSet is the power set of pseo:sensor or pseo:effector properties, that is, the set of all sub-properties of pseo:sensor or pseo:effector. This class is needed as the range of certain properties. The pseo:Input describes the information that is expected by sensor/effector procedures. We view an input as a container for one or more sensor/effector procedures. Accordingly, the pseo:Input class must only have a pseo:attachment property and a pseo:parameterName property. The pseo:Output describes information that is generated by the sensor/effector procedure. We again view an output as a container for one or more sensor/effector procedures. Hence, the pseo:Output class must only have a pseo:attachment property and a pseo:parameterName property. The range of the pseo:sensor property is a URL which is the address of the sensor procedure. When pseo:Class is accessed by the transcoding agent, the

owl:Class property

pseo:Class

instance

pseo:effector

pseo:sensor xsd:anyURL

pseo:procedureSet pseo:attachment

pseo:attachment

pseo:Input pseo:parameterName xsd:string

pseo:content pseo:content

owl:Thing

pseo:Output

owl:Thing

pseo:parameterName xsd:string

Fig. 3. Semantic structure of the PSEO. Journal of Information Science, 31 (3) 2005, pp. 178–195 © CILIP, DOI: 10.1177/0165551505052346 Downloaded from http://jis.sagepub.com at PENNSYLVANIA STATE UNIV on April 15, 2008 © 2005 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution.

183


sensor procedure is triggered. Likewise, the range of the pseo:effector property is a URL which represents the address of the effector procedure. After the transcoding agent releases pseo:Class, the effector procedure is invoked as a side-effect action. The pseo:attachment is a bridge between sensor/ effector procedures and pseo:Class. The domain of the pseo:attachment property may be either a pseo:Input or a pseo:Output, and the range of the pseo:attachment property is a pseo:procedureSet. The pseo:parameterName property is the name of the actual parameter, which could be just a literal or perhaps the URI of the process parameter. The domain of the pseo:parameterName property may be either a pseo:Input or a pseo:Output. The pseo:content property is the actual content of the parameter, which could be just a literal or an instance of owl:Thing Class. Similarly, the domain of the pseo:content property may be either a pseo:Input or a pseo:Output.

ontology could be far from complete; we rather aimed at capturing the most widely used concepts and ensuring the feature of extensibility. The class inheritance hierarchy of MFTO is depicted in Figure 4. The mfto is the namespace that represents the MFTO.2 In MFTO, we define two instances of the owl:Class, one is mfto:MarkupFile class that represents all the markup files, and the other one is mfto:Fragment class that represents all markup fragments. The mfto:MarkupFile employs mfto:component property to specify what it is composed of. There are two subclasses of the mfto:MarkupFile, one is mfto:XMLFile, which represents all the XML markup files, and the other one is mfto:HTML, which represents all the HTML markup files. In turn, the mfto:XMLFile has three subclasses: mfto:XHTMLFile, mfto:WMLFile, and mfto:VoiceXMLFile. Similarly, mfto:Frames, mfto:Form, and mfto:Table are subclasses of mfto:Fragment, and they represent different component structures accordingly.

3.2. Markup file taxonomy ontology The MFTO specifies the markup file classification by applying OWL. OWL is useful as a bridging tool between different formalisms to enable interoperability. It is the motivation for providing the markup language classification at a high abstraction level. Another advantage of using the ontological approach and OWL is in its extensibility. As a new markup language becomes available, we can easily extend the existing classes and instances in MFTO. Our current

4. HTML forms ontology In order to demonstrate the feasibility of our OETS, we developed an HTML forms ontology. HFO is a domain ontology used to describe HTML forms layout structure, and used as the semantic schema for annotations to track form items routing information in an HTML form application. In addition, the sensor/effector procedures are also set in the HFO.

property

owl:Class

s u b C la s s O f instance

mfto:component mfto:Fragment

mfto:Table

mfto:Image

mfto:MarkupFile

mfto:Form

mfto:Frame

mfto:HTMLFile

mfto:XMLFile

mfto:List mfto:WMLFile mfto:XHTMLFile

mfto:VoiceXMLFile

Fig. 4. Semantic structure of the MFTO.

184



4.1. Challenges in converting HTML forms for mobile devices HTML forms are widely used in Internet applications, such as e-commerce, questionnaires, invoices, membership applications, etc. Along with the currently popular Internet access form mobile phones, HTML forms must be transformed into WML for WAP phones, or VoiceXML for voice phones. In particular, converting HTML forms to WML forms may face display and non-sequential navigation problems. 4.1.1. Display problem. An HTML form may consist of any kind of form item. If a form item in the HTML form is excessive, that is, the HTML form exceeds the length of the display screen, the user must move the vertical scroll bar to browse. This movement is convenient in a regular desktop browser; however, it could cause a serious problem in small display size mobile devices, such as WAP phones. In general, WAP phones are limited by the two following factors: (1) Small sized display – the WAP phone screen is small and cannot show all HTML forms fields at once [28]. Using the vertical scroll button on a WAP phone to browse the data is difficult. (2) Deck/card organizational metaphor – all information in WML is organized into a collection of cards and decks [29]. Cards specify one or more units of user interaction, such as a choice menu, a screen of text or a text entry field. Logically, a user navigates through a series of WML cards, reviews the contents of each, enters requested information, makes choices, and moves on to another card. WML cards are grouped together into a WML deck. A WML deck is similar to an HTML page, and a field (e.g. a text field, a checkbox, a select item, or an option button) in HTML forms can be displayed in a WML card. 4.1.2. Non-sequential navigation problem. It is a feature of most e-commerce applications, such as Internet questionnaires, that users usually do not browse the pages in sequential order. The questionnaire form may contain a skip pattern where progress through the questionnaire depends on the response of previous questions, so-called routing. We call this the non-sequential navigation problem. For example, if the current question is the third one and the user selects ‘Yes’ then the next question may proceed to the fifth one. In contrast, if the user selects ‘No’, the next question may be the fourth without skipping. WAP phones and PC browsers have different

navigation styles. Usually, it does not cause a routing problem in HTML forms applications, because a PC user can view all the form items on one HTML page. While with a WAP phone, a user must navigate through a series of WML cards. Therefore, the WML cards must manage the routing information by setting the onpick option tag in WML, which can determine the next WML card display depending on the current form item’s answer. HTML does not support related tags or attributes to record non-sequential navigation form items. When converting HTML forms to WML forms, the WML option tag cannot set the onpick attribute automatically. Thus the transcoding agent cannot determine the next WML card display depending on the current form item’s answer. The solution is to record the nonsequential navigation form items in an annotation which is an instance of HFO. These annotations inherit knowledge and the semantics [30] from the HFO. Together with the semantics of PSEO and MFTO, these annotations will help the transcoding agent to accomplish the automatic transformation. The above non-sequential navigation problem in WML also occurs in VoiceXML for voice user interface [31]. The small screen size constraint on the amount of data that can be presented ‘at once’ is similar to the serial nature of voice data presentation. Therefore, the WML solution can also apply to VoiceXML. Depending upon the device type, the transcoding agent can make an appropriate code conversion. 4.2. Classes, properties, and procedural sensor/effector in HFO The class inheritance hierarchy of HFO is depicted in Figure 5. The HFO provides hints for the transcoding policy to make a better decision on content adaptation given the mobile device’s characteristics. In Figure 5, hfo is the namespace that represents the HFO.3 4.2.1. Classes in HFO. The hfo:HTMLForm is a subclass of the pseo:Class and mfto:Form. There must be a sensor or effector procedure attached to this class, such as (1) using an owl:hasValue restriction to limit the hfo:Tidy value to Tidy.class, which is a sensor procedure written in Java program; (2) using an owl:hasValue restriction to limit the hfo:triggerNext value to findNext.class, which is an effector procedure written in Java program. The hfo:formItem is a subclass of pseo:Class. There must be a sensor or effector procedure attached to this class. This class represents all of the items in HTML


185


forms. The two attached procedures are: (1) using an owl:hasValue restriction to limit the hfo:Extraction value to extractData.class, which is a sensor procedure in Java; (2) using an owl:hasValue restriction to limit the hfo:triggerNext value to findNext.class, which is an effector procedure in Java. The hfo:itemValue is an instance of the owl:Class. This class represents all values of form item. By setting an ‘owl:Cardinality = 1’ restriction, we can ensure the hfo:itemValue class only has an hfo:value property. 4.2.2. Properties in HFO. Both hfo:Tidy and hfo:Extraction are sub-properties of pseo:sensor. The

hfo:triggerNext is a sub-property of pseo:effector. All the above are properties of pseo:Class. The hfo:jumpTo property is used to record the ID of the next form item which is to be processed by the transcoding agent. The hfo:value property is used to record the legal value of a form item. hfo:itemSet, hfo:formURL, and hfo: formTitle properties are used to group form items, record the URL, and record the title of an HTML Form, respectively. 4.2.3. Procedural sensor/effector in HFO. HFO quotes the top level ontology PSEO to set the properties hfo:Tidy, hfo:Extraction, and hfo:triggerNext. The

Fig. 5. Semantic structure of the HFO.

186



trigger condition and function of these three attached procedures are described as follows. The Tidy.class is a sensor procedure which is in Java language and represents the value of hfo:Tidy. When the transcoding agent retrieves the hfo:HTMLForm, the sensor will trigger the Tidy.class to load the HTML form and convert it into an XHTML document. This transformation adopts the XHTML 1.0 standard. The extractData.class is a sensor procedure coded in Java language and is the value of hfo:Extraction. When the transcoding agent retrieves the hfo:formItem, the sensor will trigger the extractData.class to extract the fragment of this form item into an XHTML forms file. The findNext.class is an effector procedure which is also written in Java and is the value of hfo:triggerNext. When the transcoding agent finishes an hfo:formItem code conversion and releases the hfo:formItem, the effector will trigger the findNext.class to compute the next hfo:formItem to be processed by the transcoding agent. 4.2.4. Instances in HFO. The input and output parameters of attached sensor/effector procedures include hfo:tidyInput, hfo:tidyOutput, hfo:extractionInput, hfo:extractionOutput, hfo:triggerInput, and hfo: triggerOutput. The hfo:tidyInput is an instance of pseo:Input and is used to describe the related information of the input parameter of hfo:Tidy. For instance, the input parameter ‘fileURL’ has a URL passed by the transcoding agent. The hfo:tidyOutput is an instance of pseo:Output and is used to describe the related information of the output parameter of hfo:Tidy. For instance, the output parameter ‘outFile’ has an XHTML document created by Tidy.class and will be returned to the transcoding agent. The hfo:extractionInput is an instance of pseo:Input and is used to describe the related information of the input parameter of hfo:Extraction. For instance, the input parameter ‘formItem’ has an instance of hfo:formItem passed by the transcoding agent. The hfo:extractionOutput is an instance of pseo: Output and is used to describe the related information of the output parameter of Extraction. For instance, the output parameter ‘outCode’ has an instance of hfo: formItem created by extractData.class and will be returned to the transcoding agent. The hfo:triggerInput is an instance of pseo:Input and is used to describe the related information of the input parameter of hfo:triggerNext. For instance, the input parameter ‘itemID’ has an instance of hfo:formItem passed by the transcoding agent.

The hfo:triggerOutput is an instance of pseo:Output and is used to describe the related information of the output parameter of hfo:triggerNext. For instance, the output parameter ‘nextItemID’ has an instance of hfo:formItem computed by findNext.class and will be returned to the transcoding agent. 4.3. Ontology-based routing model In this section we propose a formal framework, the OBRM, for non-sequential navigation. This framework provides the basis of a navigation model for HTML forms applications, such as e-commerce and Internet questionnaires. The default navigation style of the HTML forms is sequential. However, in this paper, we will focus on the non-sequential navigation problem. In the following, we give the formal definition of the OBRM. Definition 5. The OBRM can be represented as a fourtuple: , where ONTO(Id) = {HFO, PSEO, MFTO} ONTO is a set of ontologies that provides information to enable the transcoding agent to properly interpret the annotations. Id is the unique identifier of an instance of domain ontology which describes a specific layout structure of an HTML page. The HFO serves as a domain knowledge for organizing semantically related data on an HTML form, inherits PSEO to set related attached procedures, and refers MFTO to quote markup language taxonomy. FITEM(Id) = {I1, I2, . . . , In} FITEM is a set of form items in an HTML form application. Each form item Ii is an instance of hfo:formItem and has an associated set of legal value IVALUE(Ii). IVALUE(Ii) = {Vi1, Vi2, Vi3, . . . , Vix} IVALUE is a set of item values in an HTML form application. Each value item Vij is an instance of hfo:itemValue. If value Vij should lead to a non-sequential navigation, then the information is kept in a non-sequential navigation record Sj. NSNR(Id) = {S1, S2, . . . , Sq} NSNR is a set of non-sequential navigation records in an HTML form application. Each non-sequential navigation record Sj has an associated triple . Sj =

When answering the item Ii, the display may jump to answer the item Ir according to the value Vij. In contrast, if no Sj appears in a form item, then it makes the answer in sequential order.


187


4.4. Automatic transcoding algorithm In our OETS, the transcoding agent is an intelligent agent that can parse ontology and extract information from annotations. The transcoding agent can start and communicate with the sensor/effector procedure which is attached to the pseo:Class. The automatic transcoding flow is depicted in Figure 6. The notations used in this section are given in the following. TA: transcoding agent; HQ: original HTML forms; HQ A: HTML forms annotation. XHQ: transcoded XHTML forms; WHQ: transcoded WML forms; VHQ: transcoded VoiceXML forms.

The detailed algorithm is described step-by-step as follows. 1. 2.

3.

4.

5.

6.

188

TA loads an HQA.which is identified by ID. When TA retrieves the ID, the hfo:Tidy will be triggered. 2.1. TA passes the input parameter ‘fileURL’ to hfo:Tidy. 2.2. hfo:Tidy loads HQ through the ‘fileURL’ parameter. 2.3. hfo:Tidy transforms HQ to satisfy 1.0 standards of the XHTML, and the set content of output parameter ‘outFile’ is the XHQ. 2.4. hfo:Tidy returns the ‘outFile’ to TA. TA retrieves the hfo:formTitle content and performs an appropriate action depending upon the device type: (A) WAP_Phone: converts into the first WML card of WHQ. (B) Voice_Phone: converts into the first VoiceXML field of VHQ. When TA releases hfo:HTMLForm or hfo:formItem, hfo:triggerNext will be triggered. 4.1. IF the next form item is computed the first time THEN ‘itemID’ is set to NULL. 4.2. TA passes the input parameter ‘itemID’ to hfo: triggerNext. 4.3. hfo:triggerNext analyzes FItem(ID) by using the ‘itemID’, and gets the next form item, Ii. 4.4. hfo:triggerNext sets output parameter ‘nextItemID’ content to Ii, and returns the ‘nextItemID’ to TA. When TA retrieves the Ii from HQA, the hfo:Extraction will be triggered. 5.1. TA sets the input parameter ‘formItem’ content to Ii and passes it to hfo:Extraction. 5.2. hfo:Extraction parses Ii and retrieves the hfo:description content. 5.3. hfo:Extraction retrieves the related fragment in XHQ through the hfo:description, and sets the output parameter ‘outCode’ content as the fragment. 5.4. hfo:Extraction returns the ‘outCode’ parameter content to TA. TA converts Ii to the desired XML format depending on the device type:

Step1: TA loads HQA

TA triggers hfo:Tidy sensor procedure Step2: HQ is transcoded to XHQ

Step3: creates first WML card / VoiceXML field

TA triggers hfo:triggerNext effector procedure Step4: finds the next Ii to be transcoded

TA triggers hfo:Extraction effector procedure Step5: parses Ii and retrieves related fragment from XHQ

Step6: TA converts Ii to the desired XML format

Step7: TA packs transcoded components and outputs a WHO /VHO document

Fig. 6. The workflow of automatic conversion by the transcoding agent.

(A) WAP_Phone: converts the ‘outCode’ parameter content into the next WML card of WHQ. (B) Voice_Phone: converts the ‘outCode’ parameter content into the next field of VHQ. 6.1. While Pop (Ivalue (Ii)) != NULL 6.1.1. IF Sj = is TRUE THEN TA retrieves the related metadata from the Vij, and integrates the next form item Ir into the desired XML format which keeps the information about the nonsequential navigation. 6.2. When TA completes the conversion of Ii, the hfo:formItem will be released. 6.3. Repeat steps 4, 5 and 6 until each form item in FItem(ID) has been complete.



7.

TA completes transcoding and outputs a document depending on the device type. (A) WAP_Phone: packs the WML cards and converts to WHQ. (B) Voice_Phone: packs the VoiceXML fields and converts to VHQ.

NOTO(forGuest ) = {HFO, PSEO, MFTO} FITEM(forGuest) = {Question-1, Question-3, Question-4, Question-5}

Question-2,

IVALUE(Question-3) = {answer-No, answer-Yes} NSNR(forGuest) = {S1} S1 =

5. An application scenario and system extensibility To explicitly demonstrate the proposed OETS, this section will illustrate how the transcoding agent works. We employ HFO for application and demonstrate the WML document transcoding. 5.1. Annotation of an HTML questionnaire To illustrate the transcoding process proposed in OETS, a designate application of Internet questionnaire4 is designed, as in Figure 7. The Internet questionnaire is an application of HTML forms. The questions in the questionnaire may not be answered in order. For instance, there is a non-sequential navigation occurrence at the third question. If a user answers ‘Yes’ to the third question, then he will be jumped to the fifth question by simply skipping the fourth one. The related information in this non-sequential navigation behavior must be recorded in an annotation, as shown in Figure 8. This annotation is an instance of HFO. The explanation of the annotation is described as follows. (1) The identifier forGuest is an instance of hfo:HTMLForm, which can inherit knowledge and semantics from the hfo:HTMLForm. The hfo:Tidy sensor procedure and hfo:triggerNext effector procedure are attached to the class hfo:HTMLForm. These procedures are inherited from HFO and PSEO. (2) Question-1, Question-2, Question-3, Question-4, and Question-5 are all instances of hfo:formItem. Each has an attachment applied to a sensor procedure, hfo:Extracion, and the effector procedure, hfo:triggerNext. Question-3 has an alternative choice (i.e. non-sequential navigation), so it contains hfo:selectItem to record the related information answer-Yes or answer-No. (3) Both answer-Yes and answer-No are instances of hfo:itemValue. The value determines the next question to be processed. (4) In OBRM, the forGuest annotation has the following corresponding values:

The transcoding agent can apply the annotation and follow the algorithm, described in Section 4.4, to automatically convert the above example to the devicespecific format, such as a WML document5 for WAP phones or a VoiceXML document6 for voice phones. 5.2. WML questionnaire demonstration for WAP phones The transcoded WML document in the previous section is designated for M3GATE WAP simulator, as shown in Figure 9. Except for the first WML card, every WML card must display two frames. The first frame displays the question itself and the second frame displays the answer. Usually, when a user has answered a question the next question will be prompted, except for the third question. For the third question, the next question to proceed to depends on the answer given by the user. After the test, we could find that if the user selects the answer ‘Yes’, the next question to proceed to is the fifth, but if he selects the answer ‘No’, the fourth question will follow. The information needed by non-sequential navigation is kept in the WML card through the annotation. 5.3. Extensibility of OETS In this section, we illustrate the extensibility of OETS. Developers can exploit domain ontologies for various layout-specific structures of HTML pages by plugging into the ontology base of OETS. The Internet questionnaire described in Section 5.1 is a scroll-based HTML form, where all form items are displayed in a single page, and the user must move the vertical scroll bar to browse. This is not the only display format. In fact, most e-commerce applications are designed with screen-based HTML forms. With screen-based forms, a display may include multiple sections where each section is associated with a frame. Figure 10 shows the Frames layout structure that has two frames: menu and questions. Navigation through a screen-based HTML Form is possible using the menu operation or answer section to access the subsequent screen. Screen-based HTML forms raise another challenge to


189


Fig. 7. An HTML questionnaire with a non-sequential navigation.

mobile devices when a user navigates across pages. Therefore, we have developed the HTML Frames Ontology (HFrO) which is a domain ontology used to describe HTML Frames layout structure. HFrO can be seen as the semantic schema for annotations to track the hyperlink information of multi-frames in the screen-based HTML form applications. Besides, HFrO can be referred by HFO through the hfo:formURL property. Figure 11 shows the extensibility of OETS by plugging HFrO into the ontology base. The ontologybased knowledge architecture provides a mechanism to incorporate flexibly with various HTML page 190

structures by applying the features of dynamic reference to other domain ontologies. The OBRM of the above example in Figure 10 has the following structure: NOTO(Id) = {HFO, HFrO, PSEO, MFTO}, where Id is the identifier of the screen-based HTML form shown in Figure 10. FITEM(Id) = {Q1, Q2, Q3, . . . Qs}, where Qs is the Question ID. IVALUE(Q1) = {A11, A12}, IVALUE(Qj) = {Aj1, Aj2}, IVALUE(Qk) = {Ak1, Ak2}, where Aji is the answer ID.

