an increasing need for fast and flexible software development. ... description-driven systems to represent various levels of information that endow flexibility both ...
© Copyright EMSISE’03
A Component Framework for Description-Driven Systems F. Estrella1, 2, Z. Kovacs1, R. McClatchey2, N. Toth * 1, 2 & T. Solomonides2 1
European Organization for Nuclear Research (CERN), 1211 Geneva 23, Switzerland {Richard.McClatchey, Tony.Solomonides}@uwe.ac.uk 2 Centre for Complex Co-operative Systems, University of West of England, Frenchay, Bristol BS16 1QY, UK {Florida.Estrella,Zsolt.Kovacs,Norbert.Toth}@cern.ch
Abstract : Software systems are increasingly impinging on new areas in everyday life; competitive market conditions mean that the requirements for such software are subject to frequent change. As a result, there is an increasing need for fast and flexible software development. Meta-objects can be used in multi-layered description-driven systems to represent various levels of information that endow flexibility both to the managed data and to the system design. This position paper identifies those generic system components that together provide a coherent architecture for maintaining system evolution and data complexity in a holistic approach. We argue that these generic components can be used as building blocks of a system at multiple abstraction layers, introducing reuse and extensibility in higher abstractions, and reducing the complexity of the system architecture. To illustrate the judicious use of components in such development, this paper overviews the CRISTAL system as one example of an existing implementation of a system.
1. Introduction
Modeling Abstraction
Software systems are increasingly impinging on new areas in everyday life; competitive market conditions mean that the requirements for such software are subject to frequent changes, even during the lifecycle of software production. Consequently, there is a need for methods of rapid development of new software products from scratch, as well as for the rapid and frequent modification of existing systems. This approach is advocated by exponents of so-called Agile software development [1]. As a complement, a number of works promoting runtime software development have been published, including [2, 3, 4]. Developers using traditional software development techniques are usually forced to terminate running applications, modify, compile and debug application code. Any new application must then either support multiple versions of data or require data to be migrated to the new version. Systems that support runtime software development would allow some of these changes to take place during program execution, replacing static model structures with first-class objects.
Instance
Model
Meta-Model
Meta-metaClasses
Meta-MetaObjects
Meta-Classes
Classes
Meta-Objects
Objects
Data
Meta-Data
Meta-Meta-Data
Data Abstraction
Figure 1. Description-driven approach. * N. Toth is partially supported by the National Scientific Research Fund (OTKA) through grant T029264.
1
© Copyright EMSISE’03
Systems built on what we have termed a Description-Driven (DD) approach [2, 5] support run-time software development and scalable data management through the use of meta-objects. Figure 1 shows that DD system objects at meta-data and higher abstraction levels represent type- and object propertyspecific information extracted from the lower abstraction level. Solid lines refer to the traditional ‘instance-of’ relationships while dashed lines represent ‘described-by’ relationships. The latter describes and often constrains the creation and behaviour of objects on the lower abstraction level. Instance objects can be simulated [6] by meta-objects, inducing a homomorphism between the two abstractions. Meta-objects being first-class objects are the means for run-time system evolution, while data values extracted to the meta-level provide scalable data management. Depending on the semantics of instance objects being described, meta-objects may represent descriptions of process, workflow activity, data, role as well as relationship type. Due to the multiplicity of information abstraction layers that DD systems operate on and describe, such systems are more complex in their architecture than traditional systems, resulting in additional development and maintenance time. However, DD systems are flexible and highly responsive to changes in the environment of any given application instance, expressed as changes at a descriptive level; the hoped for flexibility across multiple applications appears to be a realistic expectation. A generic component framework, proposed here, supporting the DD approach provides a solution with two benefits: the framework supplies the basic system components that a run-time evolving application would require and still, by leaving some application specific decisions open for developers, by only providing framework interface implementations and guidelines, applications can flexibly utilize the DD approach. The remaining task of software developers is therefore to focus on implementing narrow framework interfaces and (re-)configuring the meta-level(s) of the system. Following the identification of the required framework components, we argue that by exploitation of the resulting components, systems can equally manage multiple layers of information. This is in contrast with other related works that provide different models for object instances and for their runtime class representation. The benefits of our approach include reduced complexity of system architecture, reduced system maintenance and the relatively simple extension of a system towards higher abstraction layers.
2. Elements of the Description-Driven Component Framework The DD approach targets both problems of system evolution management and data complexity management through the judicious use of metadata. Information about the instance model, system configuration decisions, instance object structures through simulation, and property values that are common to a set of instance objects are abstracted and represented by meta-objects. In this section we will identify those system entity parts that are required as generic building blocks of a DD system. The generic system building block (entity) requires a set of properties that are often given as name and value pairs. Properties may reflect entity characteristics (such as unique identity) at instance and metalevels, while meta-level properties may additionally contain the type of instance properties and metainformation. Various semantically different relationships including grouping, aggregation, description and generalization represent specific kinds of object connections that are fundamental parts of an OO system. The management of the semantic information is however left to the software developer in most programming languages. The use of a reified graph pattern [5] provides a generic solution to representing relationships between entities in a reflective and managed form, by raising the semantics of the relationship often from a simple object pointer to a first-class object. As a practical example, information held by these objects may include the number of related objects, the distance between related objects or a mixing ratio of related objects. Any change propagation upon request to a multiple number of related entities, and the handling of relationship information by the reified graph pattern itself can be achieved in a well-established, synchronised and reliable manner, taking away this task from system developers. Objectified relationships at the meta-level simulate instance object structures and hold descriptive information related to the instance object structures, such as reconfigurable structural constraints.
2
© Copyright EMSISE’03
The dynamic maintenance of entity behaviour is one of the main requirements for run-time program development. Workflow management systems provide one such solution by allowing the run-time reconfiguration of activities (adding, removing, reordering, splitting, joining, repeating, etc.), where each activity describes a certain task for execution. Each individual entity having its own workflow management process is capable of evolving independently. Meta-entities hold two types of workflow information: they manage their own workflows and they hold a workflow description for those instance entities that meta-entities describe. As the system is continually allowed to evolve through its description layer, the management of coexisting versions of structure, workflow and data requires the presence of an audit-trail system by recording events that trigger important design changes, thus setting the basis of version control. As events that produce system changes (such as the new version of a script, structural or behavioural changes at any abstraction level) are triggered by the execution of tasks, these tasks may generate resultant data, specific to the executed task. The type of such data is specific to the executed script. Recording such data attached to the triggering event at each level of abstraction will allow the data to evolve, making possible access to all intermediate states at a later stage. Considering the frequent data type evolution of a large number of scripts, we believe that an XML-like data format is best suited to the storage of such information, all the more so since object-oriented and XML formats have wellestablished conversion techniques and rapid type evolution does not necessarily affect the data description of the storage device. Another important feature is the XSD type description of XML document instances, where the format of the document type specification conforms to the format of document instance, opening opportunities to extensibility towards the more abstract layer.
Modeling Abstraction
To produce ‘snapshots’ of an entity, to retrieve coherent versions of a number of entities or to search through result data stored in the audit-trail, each entity element needs to obtain a viewpoint that acts as a query executor. Its task involves query optimization techniques such as query rewriting [7] exploiting the simulation, and query propagation to related entities. Viewpoints may become tools to provide crosscutting, used by aspect oriented software development.
Instance
Model
Meta-Model
Component Framework
Component Framework
Component Framework
Meta-MetaObjects
MetaObjects
Objects
Data
Meta-Data
Meta-Meta-Data
Data Abstraction
Figure 2. The Description-Driven approach based on a component framework. In traditional OO programming public methods represent services to other objects, while methods of other objects specify the sequence of methods to be called. According to our approach, tasks that are specified by workflow activities of an entity need to be completed by the same or another entity. Capabilities of an entity present a list of activity types that the entity offers to perform.
3
© Copyright EMSISE’03
The above system components (highlighted in bold) have been identified in order to provide generic building blocks to model traditional objects and their descriptions in a way that reflects DD systems. Distributed computing and object persistence are additional issues that can be addressed at the level of the entity, by supplying it with remote object manageability and persistence capability. Appropriate information exchange between components and their appropriate organization within and across layers of the multi-layered architecture, establish a holistic framework for the development of DD systems. External to the component framework the system needs to support the overall management of generic entities. These tasks include support for database management, unique identification management, roles and policies and graphical user interfaces to provide run-time application development.. The resulting component framework can be reused for each abstraction layer simplifying system architecture and allowing ease of extension towards higher abstraction layers as shown by Figure 2.
3. A Practical Example – The CRISTAL System at CERN It has been argued above that property, workflow, collection, audit-trail, viewpoint and capability components form a set of generic system elements that can be managed collectively to facilitate the building of DD systems relatively independently of the layer of abstraction. In order to build an application, developers need to adopt these concepts and adapt them to their specific domain and designs. We present the requirements and an overview of the CRISTAL software application as an example to show how a particular DD system copes with the special production management requirements of a large-scale experiment at CERN. Although our example is taken from the application domain of engineering data management, other process-oriented areas such as software development management, business process management and software integration tools are also suitable domains.
3.1 Requirements The CRISTAL software (Cooperative Repositories and Information System for Tracking Assembly Lifecycles) [8, 9] has been developed to meet the production management requirements of one of the physics experiments at CERN. The experiment involves the implementation of a one-of-a-kind complex sub-detector by constructing a prototype and the final product at the same time. The subdetector consists of a large number of compound products. These products are categorized into a large number of types, each described by different parameters and subject to a different set of workflow activities. Physicists, being the main users of the system, often need to apply changes to the detector design such as the specification of additional properties or new workflow activities for an existing product type, the introduction of a new type, or the modification of the workflow for specific products of a type. These changes may happen independently in geographically distributed production centres. The physical characteristics of each product are measured and possible problems diagnosed before and after assembly. All workflow activity results are recorded to provide full traceability. The estimated data by the end of production will accumulate to the order of one Terabyte. This data serves as the basis for maintaining quality control and, more importantly in this scientific setting, an iterative refinement of the production process. The type of measurements defined by a specified workflow activity type may evolve over time as physicists study and learn from previous results. Although the production process of each product is identified by the type of the product, there can be no accurate description of how an individual product needs to be handled, as each product may be subject to an ad-hoc process modification due to the research nature of the production. System evolution therefore occurs as new product types have to be introduced and modified by physicists across the geographically distributed system. These system requirements demand software solutions that show scalable and complex data management and flexibility towards system evolution.
4
© Copyright EMSISE’03
CRISTAL is a distributed product data and workflow management system, which makes use of an OO database for its repository, a multi-layered architecture for its component abstraction and dynamic object modeling for the design of the objects and components of the system. CRISTAL is based on a DDS architecture using meta-objects. The DDS approach has been followed to handle the complexity of such a data-intensive system and to provide the flexibility to adapt to the changing scenarios found at CERN that are typical of any research production system. In the two years of operation of CRISTAL it has gathered over 25 Gbytes of data and been able to cope with more than 30 evolutions of the underlying data schema without code or schema recompilations. These changes included evolutions of type definitions, the additions of new managed entities and on-the-fly redefinitions of workflow components. In addition CRISTAL offers domain-independence in that the underlying data model is generic in concept; for example the Agilium group [10] is currently adapting the Kernel of the CRISTAL system for the purposes of commercial Enterprise Application Integration (EAI).
3.2 Component Organization For the purpose of fitting the production management environment optimally onto the above concepts, the entity acting as the overall container for all components has been separated into two parts. Fig. 3 illustrates that the workflow activities defining and requesting task executions on the Passive Traceable Entity (PTE) is separated from the actual task execution of Active Entity (AE). Manageable Entity represents the common utilities such as object identity, distribution capability and persistence capability. Manageable Entity
Passive Traceable Entity
Active Entity
Capability
Workflow
Collection
Audit-trail
Viewpoint
Property
Job
Data
Figure 3. Component framework organized for production management PTEs in production management may include products, order forms, Bills-of-Material (BOM), etc. Activities contained in the workflow of a PTE describe those tasks that need to be performed on the PTE. However, the tasks themselves need to be executed by an AE. AEs may represent various instruments, human workers and user codes that perform specific calculations.
3.3 Workflow Activities CRISTAL differentiates between two types of workflow activities. Predefined activities support fundamental system manipulation to cope with runtime evolution, while application specific extensions can be added by software developers during runtime to meet application requirements. Predefined activities include manipulating workflow activities and manipulating an entity in the collection structure. For example, an application specific activity is the conversion from inches to
5
© Copyright EMSISE’03
metres and transversal light transmission measurement. One task is assigned to each activity. The runtime evolution of workflow activities is supported by CRISTAL in two ways: predefined steps provide creation and reorganization of workflow activities, while Java beans and a generic communication interface for data exchange provide support for runtime task definition. Workflow activities are graphically described in a number of steps with pre- and post-conditions using the standard CRISTAL object specification mechanisms.
3.4 Workflow Activity Execution The execution of PTE workflow activities is carried out by the AE system participants. Providing multiple AEs with the same capability in a distributed working environment allows the assignment of the next executable activities in such a way that load balancing is supported. The communication channel between PTEs and AEs in the case of CRISTAL is established through a notification service, decoupling the two participants. Executable activity tasks are assigned an AE using the publish/subscribe mechanism and an appropriate business rule. Following the execution of the task, the resulting data (if any) is sent from the AE to the PTE through a direct method invocation. The PTE registers this event along with the data in the audit-trail component.
3.5 Workflow and Workflow Description When instance-level entities are added to the system their descriptions i.e. meta-level entities, need to be assigned to them. Each meta-level entity may hold multiple versions of workflow descriptions acting as templates for instance-level workflows that can be later modified. The creation of a workflow description version is achieved by executing the task of a predefined activity belonging to the meta-level entity. The task provides a tool to construct a new workflow description. In line with any workflow activity execution, the resulting new workflow version – being the resulting data of the task – is recorded in the audit-trail data of the meta-level entity. Some of the activities recur across different workflow versions; by reducing the granularity of the recorded workflow to the level of an activity, CRISTAL increases the design reuse and establishes scalability to the system.
3.6 Data and Data Description Data organized by the audit-trail information permits the exchange of data between workflow activity tasks. Different types and versions of tasks manipulate previously recorded results through the viewpoint and append them with their own results. These resulting data can be generated by user code or an instrument, or can be manually input by a human worker. Generating input forms based on the schema describing the type of the result data offers one way to produce valid, type-conformant data. Instance data descriptions – or schemas – are kept as data of the associated description-level entity.
3.7 Collections CRISTAL uses two types of information stored in the meta-level reified collections. Semantic information interpreted by the system, such as the allowed number of described instance-level entities, and application-specific information that is propagated to the instance-level collection when a new instance-level entity with its collections has been created. This application specific information can then be interpreted by application specific workflow activity tasks.
3.8 Properties and Property Descriptions Properties of an instance-level entity are created in an analogous way to that in which workflows are instantiated from workflow descriptions. Property descriptions are stored in the audit-trail of a dedicated property description entity attached to the meta-level entity. The semantic information of
6
© Copyright EMSISE’03
property descriptions such as optional properties and default values are interpreted by a predefined activity. A runtime manageable XSD schema is attached to each type of property description. Based on this schema information graphical building tools may provide property editor forms for the users that produce XML formatted documents. Valid XML documents can be turned into property objects and attached to the created instance-level entity.
3.9 Towards higher abstractions Extending the system towards the meta-meta-level provides bootstrapping configuration options for meta-level entities. Predefined activities also available at the highest implemented abstraction layer provide initial functionalities to configure the layer to application needs, from which lower abstraction layers can be instantiated in a scalable manner. The definition of meta-meta schemas at the meta-meta layer in CRISTAL puts constraints on meta-layer schemas that include for example the prohibition of specific languages and certain property naming rules.
4. Conclusions Parallel works in run-time system evolution management have shown the need to model different abstraction layers with dedicated software architectures in multi-layered systems (e.g [3 & 4]). This position paper argues that it is possible to represent multiple layers of abstractions by a set of collectively managed reusable generic system components. We support our argument by introducing an implementation of a large-scale production management system as an example [8, 9]. Benefits of the resulting component framework include system integration and interoperability, rapid software development of a simplified system architecture, and runtime adaptability to system evolution. Acknowledgments The authors take this opportunity to acknowledge the support of their home institutes and numerous colleagues responsible for the CRISTAL software. In addition Professor Richard McClatchey acknowledges the support of the Royal Society in the preparation of this paper. References [1] The Manifesto for Agile Software Development. See: http://agilemanifesto.org/ [2] Z. Kovacs. “The Integration of Product Data with Workflow Management Systems”. PhD Thesis, University of the West of England, Bristol, England, 1999. [3] J. W. Yoder & R. Johnson. “The Adaptive Object-Model Architectural Style”, the Working IEEE/IFIP Conference on Software Architecture (WICSA3), Montreal, Canada. August 2002. [4] D. Riehle, S. Fraleigh, D. Bucka-Lassen, N. Omorogbe. “The Architecture of a UML Virtual Machine”. Proceddings of the Object-Oriented Programming, Systems, Languages and Architectures conference, (OOPSLA), Tampa Bay USA. October, 2001. [5] F. Estrella. “Objects, Patterns and Descriptions in Data Management”. PhD Thesis, University of the West of England, Bristol, England, 2000. [6] P. Buneman, S. B. Davidson, M. F. Fernandez, D. Suciu. “Adding structure to Unstructured Data”, Proceedings of the International Conference on Database Theory,(ICDT) Delphi, Greece. January 1997. [7] C. Koch. “Optimizing Queries Using a Meta-level Database”. ArXiv:cs:DB/0205060 v1. 2002. [8] F. Estrella, Z. Kovacs, J-M. Le Goff, R. McClatchey, T. Solomonides & N. Toth, “Pattern Reification as the Basis for Description-Driven Systems”, In press Vol 2 Issue 2 of the Journal of Software and System Modeling, Springer-Verlag ISSN: 1619-1366, 2003. [9] F. Estrella, J-M Le Goff, Z. Kovacs, R McClatchey & S. Gaspard , “Promoting Reuse Through the Capture of System Description”. Lecture Notes in Computer Science Vol 2426 p 101-111 ISBN 3540-44088-7 Springer-Verlag, 2002 (Presented at the OOIS 2002 Workshop on Reuse in ObjectOriented Information Systems Design Montpellier, France. September 2002). [10] S. Gaspard, F. Estrella, R. McClatchey & R. Dindeleux, “Managing Evolving Business Workflows through the Capture of Descriptive Information”. Accepted by the eCOMO'2003 4th Int. Workshop on Conceptual Modeling Approaches for e-Business 22nd International Conference on Conceptual Modeling ER 2003. Chicago, USA. October 2003.
7
