Web Services: Evolving Techniques in Net-Centric ...

Web Services: Evolving Techniques in Net-Centric Operations Roy Ladner, Elizabeth Warner, Fred Petry, Uday Katikaneni, Kevin Shaw U.S. Naval Research Laboratory, Stennis Space Center, MS 39529 David Aha U.S. Naval Research Laboratory, Washington, DC Kalyan Gupta and Philip Moore ITT Industries, Alexandria, VA U.S. Naval Research Laboratory, Washington, DC ABSTRACT Web Services are often seen as the net-centric enabling technology for many US Navy operations. While Web Services are increasingly used for data distribution in a network centric environment, Web Services only constitute a baseline specification that provides the foundation on which application developers, under current approaches, write specialized software in order to retrieve data over the Internet. Software development and maintenance can increase with the increase in number of different available Web Services due to such factors as new XML schema, XML schema versioning differences and variations in interface methods. In this paper, we provide an overview of Web Services and provide examples of Web Services for Navy net-centric operations as applied to meteorological and oceanographic (MetOc) data. We then present issues related to the evolution of MetOc Web Services to particular Navy needs. Finally, we describe a new project we have begun, the Advanced MetOc Broker (AMB), which will assist with minimizing specialized software development for new and ad hoc Web Services for the MetOc domain. The AMB will apply MetOc ontologies to knowledge-based techniques in order to support an advanced approach to the use of Web Services; namely, the automated identification and retrieval of MetOc data. Key words: Ontology

Web Services, Net-Centric Warfare, Automated Reasoning, Intelligent Systems, MetOc Data, MetOc

1. INTRODUCTION Forces that are networked enjoy dramatically improved capabilities for sharing, accessing, and exchanging information compared to those that are not. These capabilities are intended to provide information superiority in the information age. [1] Web Services are being adopted as an enabling technology to provide net-centric capabilities for many Navy operations. The Navy Enterprise Portal (NEP), for example, is Web Services-based, and the Department of the Navy is promulgating guidance for developing Web Services. Web Services technology is rapidly being adopted, which may lead to an ever-increasing number of data providers offering data via new Web Services. The use of Web Services should reflect special needs that may exist for Navy operations. Applications should exercise a “best fit” approach to Web Services to ensure that the employ of Web Services technology adds to an application and does not detract from or hinder an application. [2] In order to determine “best fit”, it may be necessary to evaluate an application and the environment in which it is used. Issues such as bandwith usage and guaranteed data delivery can be mission critical during some Navy operations. In addition, rapidly integrating applications with new Web Services may be significant to maintaining information superiority in a network-centric world.

In this paper, we describe the components and protocols of Web Services and provide examples of Web Services for Navy net-centric operations as applied to meteorological and oceanographic (MetOc) data. We then present issues related to the evolution of MetOc Web Services for the Navy. Finally, we describe our work on the Advanced MetOc Broker (AMB). The AMB supports an advanced approach to the use of Web Services; namely, the automated identification, retrieval and integration of MetOc data.

2. WEB SERVICES COMPONENTS AND PROTOCOLS Web Services provide data and services to users and applications over the Internet through a consistent set of standards and protocols. The most commonly used standards and protocols include, but are not necessarily limited to, the Extensible Markup Language (XML), Simple Object Access Protocol (SOAP), the Web Services Definition Language (WSDL) and Universal Discovery Description and Integration (UDDI). XML is a language used to define data in a platform and programming language independent manner. XML has become one of the widely used standards in interoperable exchange of data on the Internet. XML Schemas define the structure or building blocks of an XML document. Some of these structures include elements, attributes, element hierarchy, number of occurrences of elements, and data types, among others. [3] WSDL allows the creation of XML documents that define the “contract” for a Web Service. The “contract” details the acceptable requests that will be honored by the Web Service and the types of responses that will be generated.[4] The “contract” also defines the XML messaging mechanism of the service. The messaging mechanism, for example, may be specified as SOAP. A UDDI registry provides a way for data providers to advertise their Web Services and for consumers to find data providers and desired services. Data provided about a Web Service can be categorized much like information in a telephone book into “white” pages, “yellow” pages and, unlike a telephone book, the “green” pages. The white pages include basic provider information such as name, address, business description and contact information. The yellow pages provide services listed by category as determined by the American Industry Classification System and the Standard Industrial Classification. The white and yellow pages include enough information for a consumer to determine whether they need the technical specification for the service, which is contained in the green pages. The green pages may either contain or point to the WSDL file. An interface to a UDDI registry, may allow users to search for Web Services by business category, business name or service. [5] It is, of course, not necessary to register a Web Service with a UDDI registry. However, that would be similar to a business not listing its telephone number in a telephone directory. Not having a listing might make it more difficult for consumers to discover and utilize a Web Service. The Web Services protocol stack as described above consists of four layers: discovery, description, XML messaging and transport. [5] A Web Service publishes its existence in a UDDI registry so that the service may be readily discovered. The Web Service interface is described with a WSDL file. Interfaces often identify SOAP as the required XML messaging protocol. SOAP allows for the exchange of information between computers regardless of platform or language. A sample use of the protocol stack is illustrated in Figure 1. In Figure 1, a Web Service publishes its existence with one or more UDDI registries. Next, a user discovers the service from a UDDI registry and retrieves a description of the service. The user then either automatically invokes the service or writes an application that invokes the service by sending an XML message over the specified transport to the service. The Web Service then returns an XML message over the specified transport. There are applications that provide services on the Web without using all components of the Web Services protocol stack described above. These Web-based services employ diverse methods for discovery, description, messaging and transport. Within these Web-based services adherence to standards and protocols vary.

2

2. Discover/retrieve description

UDDI Registry UDDI Registry UDDI Registry

1. Publish Web Service

3. Invoke service – send message 4. Service sends response

Web Service User

Data Provider Web Service

Figure 1: Illustrated Use of Web Services

3. WEB SERVICES WITHIN THE METOC COMMUNITY This section describes some of the architectures using Web Services and Web-based services. Since our focus is on improving delivery of MetOc data to Navy warfighters, we limit our comments to some of the architectures in the Navy MetOc domain. 3.1 Navy Enterprise Portal The Navy Enterprise Portal (NEP) is a Web Service access portal that is provided by the Department of the Navy. The NEP makes a distinction between user-interfaces and data services. It does this by allowing for a User Facing Service that operates in a Web-browser and which interacts with a Data Oriented Service on a remote server. A Data Oriented Service is not tightly coupled to any client application. The NEP allows the user to simultaneously access multiple User Facing Services from the same Web-browser interface. [6] 3.2 Geospatial Information DataBase (GIDB) The Geospatial Information DataBase (GIDB) System is an example of a Web-based service that invokes Web Services. The GIDB is an object-oriented digital mapping portal system designed by the Digital Mapping, Charting and Geodesy Analysis Program of the Naval Research Laboratory. Development of the system began in 1994. [7] The GIDB currently connects users to over 500 servers offering over 2,500 services. The GIDB’s communications gateway allows users to obtain data from a wide variety of data providers distributed over the Internet. The GIDB System uses XML technologies to obtain data from data providers who offer an XML-based interface. [8] Not all data providers distribute data via Web Services. For those data providers who are Web-based data providers rather than Web Services data providers, the GIDB uses the data provider’s interface of choice. This may be a native API or other mechanism. The data providers accessible through the GIDB include such diverse entities as Fleet Numerical Meteorology and Oceanography Center (FNMOC), the U.S. Geological Survey, Digital Earth/NASA, and the Geography Network/ESRI. A significant FNMOC product is the Coupled Ocean/Atmosphere Mesoscale Prediction System (COAMPS) data. The atmospheric components of COAMPS are used operationally by the US Navy for short-term numerical weather

3

prediction for various regions around the world. The GIDB’s communications gateway provides a convenient means for users to obtain COAMPS data and incorporate it with other vector and raster data in map form. The GIDB establishes a well-defined interface that brings together such heterogeneous data for a common geo-referenced presentation to the user. The GIDB user display includes the capability for 3D terrain visualizations with map overlay.[9] Research has also been conducted into extending the GIDB to use spatio-temporal data mining techniques for Navy planning applications.[10, 11] An illustration of the current application interface for a typical data request is shown in Figure 2.[12] This picture shows the fusion of live satellite imagery with such features as shaded relief, active fires, dams and water gauge stations over central Florida.

Figure 2: GIDB Data Fusion Over the Internet

3.3 Joint MetOc Broker Language (JMBL) The Joint MetOc Broker Language (JMBL) defines a syntax that allows standardized request and response structures for MetOc data queries. JMBL was developed with input from joint forces including Army, Navy, Air Force, etc. A goal of JMBL was to define one Web Service based on jointly defined XML Schemas that would serve all types of MetOc data requests. Each agency would implement this jointly defined Web Service and would therefore have interoperable implementations of the same Web Service. The JMBL Web Service is defined by one WSDL file and several XML Schemas. These Schemas define the structure of requests that the JMBL Web Service will accept and the structure of responses that the JMBL Web Service will provide. The request and response Schemas include several other Schemas, which define global data types and

4

structures. Figure 3 shows this conceptual organization. As shown in Figure 3, several of the global Schemas are included in other global Schemas. Schemas in Figure 3 are represented by “XSD”.

JMBL WSDL

JMBL Request XSD

JMBL Response XSD

Global JMBL Elements XSD

Global JMBL Attributes XSD include

include

Global JMBL Types XSD

Figure 3: Conceptual View of JMBL WSDL and Schemas

3.4 VNE-NCS The Navy’s Virtual Natural Environment – Net Centric Services (VNE-NCS), formerly the Tactical Environmental Data Services (TEDServices), is a new, scaleable and modular environmental data repository, designed to support Warfighters, Weapon Systems, and MetOc data users. VNE-NCS is currently an example of a distributed Web-based service that defines its interface in Java and uses the HTTP transport protocol. VNE-NCS is capable of interfacing with JMBL based Web Services to obtain data. Work is underway to integrate other JMBL capabilities into VNE-NCS. This would allow users to interact with VNE-NCS servers via JMBL requests. VNE-NCS includes a middleware infrastructure that enables data transport between nodes in the system (Figure 4). This figure illustrates the distributed nature of VNE-NCS. Each node is a Web server, which autonomously communicates with other nodes to achieve its objectives. One such objective is the active management of bandwidth and data. Since each node is a Web server, data updates can be continuously transmitted to end-user environments and are pre-staged in a local data cache until cancelled expressly by users or cancelled by the server due to non-use by endusers. This greatly reduces end-user retrieval times for large data sets. VNE-NCS combines the concepts of shared data spaces with Web Services and Web-based services as shown in Figure 5. This figure shows the conceptual installation of a VNE-NCS Web server on a Navy platform in the ship’s OA (weather) Division. All users on the platform have access to the Web server and data pre-staged there. Off-board users have similar access.

5

Data Source

Data Source

Battle Group

Battle Group Data Source

Battle Group

Data Source

Data Source

VNE-NCS Web Server

Figure 4: VNE-NCS Distributed Web Server Topology

Large scale data transfer can be difficult when network communications are unstable. VNE-NCS employs Resumable Object Streams (ROS) for all data traffic between VNE-NCS Web servers across the network to achieve fail-safe data transportation under these conditions. ROS allows either the client or server side of a request to lose network connection, regain it, and have the request continue where it left off. Retransmission of the previously transmitted portion is not necessary in either case. Data requests can still be wrapped in compression and/or encryption. The ROS transmission controls add almost no bandwidth overhead to the communication (approximately 13 bytes).

Data Resource1 Data Resource2

IT - 21 Infrastructure

ASW

OA

MWC

SOF

OA

Strike

ESG

VNE-NCS Web Server

Data Resourcen

In-Situ, Throughthe-Sensor

Planning/Operation Cells

Figure 5: Conceptual Installation of a VNE-NCS Server

6

In addition to ROS, VNE-NCS uses an advanced data compression scheme (LPAC). LPAC provides higher lossless compression ratios than data compression methods previously in use by MetOc data users for large gridded data sets. Data is compressed prior to network transmission. It is also stored in the compressed format and uncompressed only on extraction to end-users. A Java-implementation of spatial DataBlade type functionality provides the methods for complex extractions from these datasets. [13, 12] ROS, LPAC and the shared data space concept take into account the special needs of sea-going Navy Web-based clients who require large chunks of MetOc data on frequent intervals over tenuous communications with limited bandwidth. VNE-NCS represents the type of client/server network topology that can be important in many net-centric operations. Instead of many clients connecting to a single server, VNE-NCS enables many clients to connect to many servers. In this topology, Web servers can act as clients to other Web servers.

4. SPECIALIZED NEEDS “Best fit” of XML and Web Services in some Navy contexts may require that consideration be given to circumstances that may not often be found in many other business contexts. These circumstances involve sea-going clients who may require high volume binary data on frequent intervals over what may be tenuous communication channels with limited bandwidth. In this environment and under these circumstances, consideration should be given to how the use of XML and Web Services might add to the size of the data transmission (XML Bloat), how bandwidth usage might be affected and how data delivery might be delayed if a guaranteed delivery mechanism is not employed. The typical e-commerce approach to Web Services utilizes a data transport mechanism that employs an all or nothing transmission approach, which may expand bandwidth usage while attempting to complete a data transmission and result in delayed data receipt. For example, HTTP does not guarantee completion of a transmission. When communication channels break, client applications using Web Services may then have to re-submit requests. This could be especially problematic for a Navy user afloat who needs high-volume environmental data for mission planning purposes. In such a case, the need to compress the data and offer guaranteed delivery is especially significant. Navy operations may require that data from many sources be fused to form a comprehensive picture of the “battle space” for a given moment. Navy clients may need to acquire this data from multiple Web Services and other Web-based applications. In addition, security measures may make difficult the “discovery” aspect of Web Services and available data. The advertisement of Web Services in a UDDI registry may or may not be desirable for net-centric operations in the Department of Defense (DoD) community. Each of the Web Services and Web-based services previously described has to some extent evolved to satisfy specialized Navy needs while utilizing Web Services technology. Some examples and lessons learned are discussed below. The GIDB, for example, began as a Java applet connected to a single geospatial data server. The GIDB has adapted to the growing use of Web Service technology by making possible the retrieval and fusion of data from over 500 different servers. As the use of Web Services grows, a means of automatically discovering and integrating with new Web Services will be a desired enhancement. JMBL has sought to satisfy specialized Navy needs for joint interoperability through a uniform Web Service that can be separately implemented by multiple data providers. JMBL does this through adoption of jointly agreed upon XML Schema and WSDL. It is believed that this approach will reduce the burden on the client application developers. However, as the use of JMBL increases, a means for client applications to elegantly address Schema version compatibilities and Schema enhancements may be needed. The JMBL v. 3.0 Schema, for example, is not backward compatible with earlier versions. A client side request that conforms to v. 3.0 might not be considered a valid request to a Web Service that has implemented JMBL v. 2.19. Similarly, a client side request that conforms to JMBL v. 2.19 may not be considered a valid request to a Web Service that has implemented JMBL v. 3.0. Conforming client side code is necessary for each non-backward compatible version of the Schema that is implemented by each JMBL based Web

7

Service. Although future versions of JMBL promise to be backward compatible with older versions, updated client side code is still necessary to take advantage of new enhancements to the JMBL Schema. As previously mentioned, there is a need to generate client code in order to interact with Web Services. Several tools exist to automatically generate Java classes from WSDL, but these may not necessarily achieve desired results when complex Schemas are involved. We have used Axis v. 1.1, for example, with JMBL v. 2.19 and v. 3.0. In these cases, Axis v. 1.1 was not able to handle several features of the XML Schema. Figure 3 shows the interrelation of these JMBL Schemas. While JMBL includes global Schemas in the request and response Schemas, the global Schemas are not assigned individual name spaces. The JMBL elements are therefore unqualified. Axis v. 1.1 does not recognize unqualified elements and consequently generates client side Java code that produces invalid instances of the JMBL Schemas. Additionally in those cases where Java classes can be auto-generated from WSDL, developer intervention is still required to integrate the resulting classes into a meaningful application. Frameworks that automatically execute publicly available methods on a Web Service based solely on the WSDL and Schema still show a lack of maturity needed to interact with complex Schemas. While VNE-NCS has addressed bandwidth management, it has also integrated a JMBL client. This approach takes advantage of the benefits of JMBL while maintaining the benefits of VNE-NCS bandwidth management techniques. As the number of Web Services data providers grows, a means of automatically discovering and integrating with these new Web Services will be a desired enhancement to VNE-NCS. Lessons learned indicate that compatibility of XML Schema versions is an inherent issue for Web Service clients. It appears that adopting Web Services has provided the means to achieve some structural interoperability between joint services. Although tools exist to aide in the creation of client applications for Web Services, the tools may not be adequate when dealing with complex XML Schemas. Human resources are still required to implement client applications for these Web Services.

5. FUTURE WORK With Web Services technology playing an ever increasing role in net-centric operations and new Web Services becoming available, the need exists for Navy applications to quickly and easily integrate with these Web Services. Web Services technology has freed developers from platform and programming language constraints, but it has not yet freed developers from writing code that connects to server applications. Web Service technology merely defines the specifications (WSDL and XML Schemas) to which the client application developer must conform. These Schemas may be complex and in addition, structural and semantic differences may exist between Web Services. Since Web Services give the promise of discoverable, self-describing services that stick to common standards, their employ should allow the possibility of an efficient and automated capability to obtain and integrate data. [5] Ideally, with this automated capability it should be possible to obtain and integrate data (1) from alternate sources when data becomes unavailable from a previously reliable source, (2) from newly identified data sources that possibly employ previously unseen Schemas or (3) from a known source that changes its interface definition. Our current work seeks to leverage the potential for system automation offered by Web Services technology. Our approach combines Web Services technology with knowledge-based techniques and intelligent machine reasoning to create a Naval Advanced MetOc Broker (AMB). The AMB could be used to support the automated identification and retrieval of MetOc data from new and ad hoc Web Services. We expect our approach to enable machine “understanding” of a Web Service interface in order to overcome a recognized limitation to a fully automated process [5]. We limit our focus to the MetOc domain. Our technical approach to automating the AMB’s recognition of terms used by new Web Services for data requests and responses is to apply MetOc ontologies to meteorological and oceanographic forms of data. [14, 15, 16] Since the MetOc domain is well understood, this process is expected to overcome many semantic limitations inherent to MetOc Web Services. To do this, the AMB will use a mapping function to resolve semantic differences and integrate data. The description of concepts and terms inherent in MetOc ontologies should assist with resolution of different Schemas that may have varying semantics but describe similar data requests and responses.

8

We are currently designing the system architectural components and defining the ontology. Figure 6, for example, shows a high level conceptual description of the mapping process in which ontology usage may enable an automated mapping process. A data provider uses the term “temp” and the AMB uses the term “air_temperature”. These need to be mapped as equivalent. This is shown in the mapping with source term “temp” mapping to “temperature”. The CONCEPT_AIR in the ontology (mapping dictionary) is used to resolve this mapping. It is expected that many new MetOc Web Services using domain-relevant terminology would be discoverable by the AMB and that the AMB would be able to resolve requests to and responses from these new Web Services. Web Service Registry search algorithms will be significant to the AMB’s identification of new Web Services. Where these are lacking for security purposes, other means will have to be employed to identify data sources’ WSDL and Schema to the AMB. Also, because data confidence levels are important to warfighters, our technical approach will include identification of parameters representing measures of confidence of data’s availability, reliability, suitability and assessment of appropriate weightings. Although our focus is on the MetOc domain, the methodology employed by the AMB could be extended beyond MetOc to other domains. Systems based on this approach would not require extensive client application development for each new Web Service from which data can be retrieved. Similarly, extensive client application maintenance would not be required for each Schema alteration that may be made to the Schemas of existing Web Services.

Provider Providerand andRequester Requester Pair Pair

Mapping Mapping Mapping Dictionary

{ CONCEPT_AIR terms: < temp, air_temperature > attribute: attribute: attribute:

…

…}

temp air_temperature

< air_temperature > …



…

Mapper • Term matching based on higher level concepts,

Figure 6: Conceptual View of Ontology Mapping Process

ACKNOWLEDGEMENTS The authors would like to thank the Naval Research Laboratory’s Base Program, Program Element No. 0602435N for sponsoring this research.

9

REFERENCES [1] Director, Force Transformation, Office of the Secretary of Defense, “Network-Centric Warfare Creating a Decisive Warfighting Advantage”, Washington, DC , Winter 2003. Cleared for Public Release by Department of Defense directorate for Freedom of Information and Security Review 04-S-0272. http://www.oft.osd.mil/library/library_files/document_318_NCW_GateFold-Pages.pdf [2] Jacobs M., Hopkins B., 2001, DON XML Development Guidelines, http://diides.ncr.disa.mil/shade/documents/GIGesDMD/DONXMLguide.ppt#3

[3] XML Schema Tutorial, 2004, http://www.w3schools.com/schema/schema_intro.asp. [4] Web Services Definition Language, 2004, http://www.perfectxml.com/WebSvc3.asp. [5] Cerami, E., Web Services Essentials, pp. 3-25, O’Reilly and Associates, Sebastopol, CA, 2002. [6] Navy Enterprise Portal (2004); https://portal.tfw.navy.mil, http://agendas.ca.com/agendas/SessionDetails.asp?SessionId=1595, http://ams.confex.com/ams/Annual2005/techprogram/paper_86973.htm [7] Chung, M., Cobb, M., Shaw, K., Arctur, D., “An object-oriented approach for handling topology in vpf products”, GIS/LIS'95 Proceedings, Volume 1, pp. 14 -16, 1995 [8] Wilson, R., M. Cobb, F. McCreedy, R. Ladner, D. Olivier, T. Lovitt, K. Shaw, F. Petry, M. Abdelguerfi, “Geographical Data Interchange Using XML-Enabled Technology within the GIDB System”, XML Data Management, Chaudri C., Rashid A., Zicari R. eds., pp. 353-374, Addison-Wesley, Boston MA , 2003. [9] Ladner R., Abdelguerfi, M., Shaw, K., “3d mapping of an interactive synthetic environment”, IEEE Computer, Volume 33(3), bpp. 35-39, 2000. [10] Ladner, R., Petry, F., “Assessment of spatial data mining Tools for integration into an object-oriented GIS (GIDB)”, Proceedings 13th Int. Conference Database and Expert Systems Applications, pp. 113-122, Springer-Verlag, Berlin Germany, 2002. [11] Ladner, R., Petry, F., “Spatio-temporal data mining and knowledge discover: issues overview”, Mining SpatioTemporal Information Systems, Ladner R., Shaw K., Abdelguerfi, M., pp. 1-19, Kluwer Academic Publishers, Norwell MA , 2002. [12] Warner, E., Ladner, R., Petry, F. and Shea, J., “Advanced Techniques in Delivering Data to the Warfighter in a Distributed Information System”, Proceedings of SPIE Sensors, and Command, Control, Communications and Intelligence Technologies for Homeland Security and Homeland Defense III, Volume 5403, pp. 753-761, SPIE, Bellingham WA, 2004. [13] Katikaneni, U., Ladner, R. and Petry, F., “Internet delivery of meteorological and oceanographic data in wide area naval usage environments”, The 13th International World Wide Web Conference Proceedings, pp. 84 - 88, ACM, New York NY, 2004. [14] Alameh, N., “Chaining geographic information web services”, IEEE Internet Computing, Volume 12, pp. 22-29, 2003. [15] Fonseca, F., Egenhofer, M., Agouris, P. and Camara, G., “Using ontologies for integrated geographic information systems”, Transactions in GIS, Volume 6(3), pp. 47-61, 2002. [16] Fonseca, F. and Davis, C., Using the internet to access geographic information in Interoperating GIS, pp. 313-324, Kluwer Academic Publishers, Norwell MA, 1999

10