Configuration Programming - A Framework for the Development of ...

Configuration Programming A Framework for the Development of Distributable Systems Jeff Kramer Department of Computing, Imperial College of Science, Technology and Medicine, 180 Queen's Gate, London SW7 2BZ, UK.

ABSTRACT The underlying model commonly used to describe an application domain is that of processing components, loosely coupled by communication streams for each of the different types of information produced or consumed. The underlying model used for implementing distributed and parallel systems is usually similar, consisting of distributable processes communicating by message passing. Why then should different models be introduced in the intervening phases of software development? Configuration programming advocates the use of the same underlying interconnected-component model throughout the software process, from requirements through design and construction to evolution. The description of system structure, as a set of components and their interconnections, is separated from the functional description of individual component behaviour. This general approach has been used in the Conic Environment for the development of distributable software. This has shown that the approach is readily usable, leads to clear and flexible designs, and produces systems which are comprehensible, maintainable and amenable to change. The approach is considered "constructive" since it emphasises the satisfaction of system requirements by composition of components. In this paper we describe the principles which underlie the configuration programming approach and illustrate their usefulness by describing examples of some of the features of the existing Conic environment. As an indication of the future research directions, we then describe the work of the REX project: a recent collaborative ESPRIT II project which is based on the principles of configuration programming, and which intends to improve and extend the supported facilities.

Proc. of IEEE Internatins Conference on Computer Systems and Software Engineering (COMPEURO 90), Israel, May 1990.

Configuration

Programming

1

19/8/91

1. INTRODUCTION There has been considerable research work on the description of the software development process and on new and more powerful tools and environments to support it. Many of the suggested processes place emphasis on automated software construction, specifically identifying the important role of formal specifications as the key to validation of user requirements and generation of the system itself [1]. It is suggested that maintenance is not conducted directly on the system itself, but rather on the specification, with new versions of the system "automatically" re-generated from that new specification. This is the “specification-driven” approach. Although this view of the software process is extremely appealing, it is unfortunately proving rather elusive. For instance, both automated verification and generation of software has proved to be far more difficult in practice than it seemed in those early promising days [15, 4]. Formal specifications are still proving to be extremely difficult to develop, with subsequent interpretation being equally challenging. Automated deduction has made impressive strides forward, but are still a long way from practical use. Declarative programming languages (such as logic and functional languages) certainly provide a sound base for research on transformation techniques to provide more efficient implementations, but leave much to be desired as specifications. They are, after all, programming languages. Part of the reason for our lack of impact on the 'real' world seems to be that, for any non-trivial system, no single specification technique seems to be adequate. Although it may appear otherwise, the intention of this paper is not to criticise the motives or details of the research mentioned above. However, it is rather difficult to see how practitioners of software engineering for large and complex systems can take advantage of any of the advances made in individual fields without a complete revolution in the world of specifications. How then should we proceed? Practical experience in software engineering has taught us that complex systems can be built and managed provided we adhere to some sound principles. Software modularity is essential, and permits the use of composition to form a system. In particular, distributed systems are conveniently described, constructed and managed in terms of their software structure. Descriptions of the constituent software components and their interconnection patterns provide a clear and concise level at which to specify and design systems, and can be used directly by construction tools to generate the system itself. In many cases - particularly embedded applications - it is the structure of the application itself which is used to dictate the structure of the resultant system. This approach has been referred to as "programming-in-the-large" [9], and component-based system building using module interconnection languages [12,26], "processormemory-switch level" programming [2] and "configuration programming" [21,22]. Evolution of the system can be achieved by making extensions or changes to the system configuration by the addition or replacement of components [23]. Since it seems that many 'new' systems are created by modifying or evolving previous versions rather than by design and construction ab initio, there is a need to recognise system evolution as a major portion of the development process. The premise of our approach is that a separate, explicit structural (configuration) description is essential for all phases in the software development process, from system specification as a configuration of component specifications (eg.as in Inscape [37], GRID [36] and PAISLEY [44]), to evolution as changes to a system configuration [18,23]. In contrast to the "specification driven" approach, our model is "constructive". The "specification driven" approach attempts to formalise the decomposition process based only on the system specification. We believe that this process of component identification remains informal as it requires design information not usually included in the system specification. Decomposition is best dealt with through design heuristics. Emphasis should rather be placed on the validation process viewed as "construction" of the system from components [24]. This is the means by which we gain confidence that our design is satisfactory. The formal view of this constructive approach is that of system verification by showing satisfaction of the system specification as a composition of component specifications. This approach thus emphasises composition as a basic operation of configuration, with the need to support composition of actual system components and specifications. Hence, system configuration (structure) should be recognised as the unifying framework upon which to hang specification, design, construction and evolution of systems.

Configuration

Programming

2

19/8/91

Basic Principles of Configuration Programming The basic principles of the configuration programming approach can be summarised as follows: 1. The configuration language used for structural description should be separate from the programming language used for basic component programming. This separation of concerns facilitates the description, comprehension and manipulation, both by man and machine, of the system in terms of its structure. This is achieved by abstracting away from the component programming concerns.The structural nature of the configuration specification makes it amenable to both textual and graphical description. System construction can be performed by translation of the structural configuration description by component creation and interconnection. Furthermore, the configuration language should be declarative, describing what the structure is, not how it is to be constructed. Declarative descriptions tend to be more concise and amenable to analysis, interpretation, and manipulation than their imperative equivalents. 2. Components should be defined as context independent types with well-defined interfaces. Context independence [18] means that the component makes no direct reference to any nonlocal entities, but can be integrated into any compatible context without redefining or recompiling it. We therefore require that components access only local data and use indirect naming (such as local ports) to refer to connected components. Definition as a type permits instantiation and reuse in different contexts. The component interface should describe the interaction points with other components and permits validation of interconnections at configuration time. 3. Using the configuration language, complex components should be definable as a composition of instances of component types. Hierarchies are a natural and convenient means for the support of subcomponent encapsulation and information hiding. Interconnected instances of more basic component types can be composed to form more complex components (ie. an instance hierarchy). These composite components should themselves be component types, available for use in further definitions. Such an approach also permits the definition and construction of recursive structures. Note that the use of an instance hierarchy provides a clearer and more easily distributable system structure than is provided by extending types. For instance, although object-oriented programming languages identify a type (class) inheritance hierarchy for objects, they do not provide explicit information on the object instance structure. This makes comprehension and distribution more complex (e.g. see Emerald [3]). 4. Change should be expressed at the configuration level, as changes of the component instances and/or their interconnections. This follows from the first principle. Given that it is beneficial to utilise a structural description to comprehend and manipulate the system, then change can also be beneficially expressed as structural change. Changes can be made to component instances, which are then of a new and different type. Change at the programming level can be reflected as a change to the component type if all instances are affected. For a number of years, the Conic environment [18,29] has supported the use of a configuration language based on these principles. This has been especially successful with respect to ease-ofuse and flexibility in system design, construction and evolution. Standard design methods such as JSD [16] and Structured Analysis (Data Flow Diagrams) [43,8] produce designs as system configurations but then fail to carry these notions through to distributable implementations with explicit configurations. Environments such as STATEMATE [14] have recognised the power of the structural view for system specification and modelling, but tend to be weaker in their support for system implementation and subsequent evolution. A number of other research projects make use of a separate configuration (or module interconnection) language (DICON [27], Durra [2], Garp [17], Lady [35], C/Mesa [31], Muppet [32], NETSLA [25], RNet [7], Polylith [38], STILE [42], and Linda as a coordination language [5]) but few are as widely distributed and used, and as simple yet versatile as the Conic configuration language. The Conic configuration language includes facilities for hierarchic definition of composite components, for parametrisation of components, for replication of both component instances and interface ports, for conditional configurations with evaluation of guards at component instantiation, and even for Configuration

Programming

3

19/8/91

recursive definition of components [10]. Paper Outline In this paper we briefly describe our experience using 'configuration programming' in the Conic Environment, and assess the benefits of such an approach. Based on this experience, we extend the notions of configuration programming to cover many more aspects of the software development process. Emphasis is placed on the structural decisions and descriptions for each of the phases in the process, and on the need for compositionality. The configuration programming framework has been adopted as the basis of a recent ESPRIT II Project, REX, on "Reconfigurable and Extensible Parallel and Distributed Systems". This major project, which involves 10 industrial and university research groups, is using two large demonstrators in the telecommunications and CIM areas to act as a focus for the work and as a means for demonstrating the techniques and tools developed. A brief description of the REX work is used in the paper to indicate the general research direction in this field and to elucidate some of the issues and possible solutions. 2 . CURRENT SUPPORT FOR CONFIGURATION PROGRAMMING: The CONIC Environment. The Conic environment [18,19,29], developed by the Distributed Computing Group at Imperial College, provides support for configuration programming for distributed and concurrent programs. The environment provides support for two languages, one for programming individual task modules (processes) with explicitly defined interfaces, and one for the configuration of programs from groups of task modules. In addition, the environment provides support for the reuse of modules in different contexts and support for dynamic configuration. This latter facility is achieved using on-line management tools which permit dynamic creation, control and modification of application programs. The Conic environment has been in use for over 5 years, with dynamic configuration as a more recent facility. It has amply demonstrated the utility of configuration level programming and the need for the separate configuration perspective. We now illustrate the use of the Conic structural (configuration) language for describing, constructing, monitoring and changing distributable systems. In order to provide a feel for the approach, we use a simple example: a patient monitoring system [34]. The intensive care ward in a hospital consists of a number of beds. Patients in each bed are continuously monitored for a number of factors, such as pulse, temperature and blood pressure. For each patient the current readings can be displayed both at the bedside and at the nurse unit. If any of the factor readings of a patient are outside of preset limits, then an alarm is sent to the central nurse station. Configuration Design Descriptions in Conic Component Types. The patient monitoring system is constructed from the two context independent component types (referred to as modules in Conic) defined both graphically and textually below in Figures 1 and 2. The interface to a component is defined by typed exit- and entryports. Messages are sent out via exitports and received from entryports. Messages can be of any standard Pascal datatype ( except pointer and file types). The type definitions are imported from definition modules by the use clause. alarm

patient

bed

group module patient; use monmsg: bedtype, alarmstype; exitport alarm:alarmstype; Configuration

Programming

4

19/8/91

entryport bed:signaltype reply bedtype; - The module periodically reads sensors attached to a patient. Readings outside preset ranges cause alarm messages to be sent to the exitport alarm. A request message received on the entryport bed returns the current readings and ranges. end.

Figure 1 - The Patient Component

alarm[1] alarm[2]

bed[1]

nurse

bed[2]

alarm[3]

bed[3]

alarm[4]

bed[4]

alarm[5]

bed[5]

group module nurse (maxbed:integer=5); use monmsg: bedtype, alarmstype; entryport alarm[1..maxbed]:alarmstype; exitport bed[1..maxbed]:signaltype reply bedtype; - the module displays alarms received from alarm[] and requests the display of a particular patient via bed[]. end.

Figure 2 - The Nurse Component Component Hierarchies: In the above, we have described the main component types to be used to construct the patient monitoring system. In fact, each of the two component types used are themselves configurations of components. For example, the internal structure of the patient component is depicted in Figure 3. It is defined by instantiating an instance of each of a scanner and monitor component types and interconnecting their exit- and entryports. The links between exitports and entryports allow components to communicate by message passing. The Conic environment permits only ports of the same type to be connected.

reading

scanner

reading bed

monitor alarm

bed alarm

patient

group module patient; use monmsg: bedtype, alarmstype; use scanner, monitor; exitport alarm:alarmstype; entryport bed:signaltype reply bedtype; create scanner; monitor; link scanner.reading to monitor.reading; bed to monitor.request monitor.alarm to alarm; end.

Figure 3 - Internal structure of Patient Module Configuration

Programming

5

19/8/91

A system in Conic is thus an hierarchic structure of component instances. The components at the bottom of the hierarchy are sequential tasks, implemented in a programming language. In Conic, the internal programming language is Pascal extended to support message passing for the distributed environment. Instances of these task modules execute concurrently. Constructing Systems in Conic Given the hardware depicted in Figure 4, we can construct an initial patient monitoring system consisting of one nurse and one patient by instantiating one instance of each of the above component types and interconnecting their exit- and entryports. Again, the Conic environment permits only ports of the same type to be connected. The configuration description for this initial system is again shown both textually and graphically in Figure 5.

Ethernet

Sun1

Targ1

Targ2

Figure 4 - Hardware Environment

bed1

alarm a l a r m [ 1 ] bed bed[1]

bed[2]

nurse

bed[3]

alarm[2] bed[4]

alarm[3] alarm[4]

bed[5]

alarm[5]

system ward; create bed1:patient at targ1; nurse: nurse at sun1; link bed1.alarm to nurse.alarm[1]; nurse.bed[1] to bed1.bed; end.

Figure 5 - Initial Patient Monitoring System The system is created by submitting the configuration description to a configuration manager tool which downloads component code into target processors or instantiates processes under Unix as appropriate (figure 6). The configuration management tool and its supporting environment is described in [29]. The configuration description may be submitted directly as text to the configuration management tool or indirectly using a graphical editor described in more detail in [21]. Note that, in addition to instance creation and linking (interconnection), the configuration description includes component location (the at clause) and parameters. For example the nurse has a default parameter setting to the value 5 (Figure 2) however this could have been changed when the instance was specified, eg. create nurse:nurse(3) at sun1.

Configuration

Programming

6

19/8/91

configuration specification

Configuration Management

components

System

bed1

nurse

Figure 6 - System Configuration using Configuration Management Note that the configuration language is declarative. This is more amenable to analysis and validation, and leaves the actual order in which configuration operations are performed to the underlying support system. It can then exploit the inherent parallelism of the underlying architecture where appropriate. If the configuration statements were embedded in a procedural language, the current state of the system configuration would depend on the state of the configuration program (for instance, see [25]). This would complicate the user view of the system and the provision of support for system evolution and dynamic configuration. Dynamic Configuration for System Evolution. In addition to programming initial configurations, the Conic toolkit permits dynamic configuration: changes to running systems. For example, extending the above system to include an additional patient unit can be performed by submitting the following configuration change of the system 'ward' to a configuration manager: change ward; create bed2:patient at targ2; link bed2.alarm to nurse.alarm[2]; nurse.bed[2] to bed2.bed; end

The resulting system is depicted graphically in Figure 7:

bed1

bed2

alarm a l a r m [ 1 ] bed[1] bed

bed[3]

nurse

alarm a l a r m [ 2 ] bed[2] bed

bed[4] bed[5] alarm[3] alarm[4] alarm[5]

Figure 7 - Extended Patient Monitoring System The change can be thought of as an edit, in configuration terms, of the configuration specification and the system itself. As shown in figure 8 it results in both a new specification and a correspondingly changed system. Thus the system itself evolves rather than the traditional approach of regenerating the system ab initio.

Configuration

Programming

7

19/8/91

Change Configuration specification

bed1

i

Specification Configuration specification

i+1

Configuration Management

bed1

nurse

nurse

bed2 System

System

i

i+1

Figure 8 - Correspondence Between System Specification and Actual System Dynamic Change Management As mentioned, changes are specified declaratively at the configuration level in terms of structure. However, these changes must be applied to the system in such a way that the application consistency is preserved. In addition, we would like to minimise the disruption to the application during the change. Recent work [23] has provided a sound basis for performing change while preserving consistency and without disrupting the unaffected components. The structural concerns are separated from those at the application (component) level. Changes are specified purely structurally, in the declarative configuration language. Using a set of change rules, it is possible to derive a procedural change transaction which identifies those nodes in a system affected by the particular change, and includes the control operations and ordering decisions necessary to apply the change transaction to the system itself. This change transaction puts the affected part of the system into a quiescent state in order to ensure that it does not contain the results of partial application transactions. It can be shown that this state is both reachable and sufficient for the application of the change. Disconnecting nodes are then given the opportunity to execute finalisation actions to leave the environment in a consistent state, and newly connected nodes the opportunity to execute initialisation actions to make the node consistent with its environment. The system may then be permitted to actively participate with the rest of the system. The details of this approach are beyond the scope of this paper, but it has affirmed our belief in the separation of configuration concerns as an essential prerequisite for describing and supporting system evolution. Graphical Configuration Monitoring and Programming In practice, we have found it convenient to create initial systems from the configuration specification. Once created, a graphics tool, ConicDraw, can be used to display and manage the system. As depicted in figure 9, ConicDraw maintains a graphic representation of executing Conic systems in terms of the component instances which exist in the system, their interconnections and their execution state. It gathers this information directly from the executing system by communicating with a configuration manager. Thus the graphics tool can be used in a monitoring mode to provide users with an uptodate view of the system configuration. Graphic Model of System (ConicDraw)

Actual System Changes

Status

Figure 9 - Relationship between ConicDraw and System. Configuration

Programming

8

19/8/91

Changes to the system are reported to ConicDraw by configuration management to enable it to maintain an uptodate view of the system. ConicDraw can itself instigate changes to the system as a result of edits to the graphic representation. A change which has been performed graphically can be saved in its text form for later re-use. The patient monitoring system was extended by editing the diagram directly to create Figure 10. These edits caused the tool to send the extension configuration text to a configuration manager to change the actual system accordingly. Figure 10 shows a further bed, bed3, being linked into the existing system using the link tool in the tool palette. The link instruction generated by this graphic operation (link nurse.bed3 to bed3.bed) is shown at the bottom of the figure.

Figure 10 - System linking via ConicDraw. Since ConicDraw always maintains a representation of the actual system, one or more workstations can be connected to the distributed system to allow a number of users to cooperatively manage and monitor the system. ConicDraw contains a comprehensive set of tools for creating and editing Conic configuration diagrams. As such, it can also be used as a standalone diagram editor. A full description of ConicDraw is given in [21]. Summary and Experience This section has briefly outlined how configuration programming is used and supported in the Conic environment. The configuration language and components both conform to the basic principles described in the introduction to this paper. The interested reader can find a detailed description of the Conic Configuration language in [10] and the Conic environment in [29]. Users of the Conic environment include a number of universities and industrial research centres in the UK, Belgium, Germany, France, Greece, Sweden, Finland, Canada and Japan. Their use of configuration programming has included underground monitoring and communications in mines, plant automation, image processing, an object-oriented system manager, and multi-loop self-tuning adaptive controllers. Our experience at Imperial College has been very positive and confirms our belief that the structural configuration level is a useful level of abstraction for system description, construction and evolution. The provision of software tool support has been essential in the successful use of Conic.

3. FUTURE SUPPORT FOR CONFIGURATION PROGRAMMING: The REX Project The basic principles of configuration programming, with their concrete manifestation in Conic, have thus been shown to be useful in providing a uniform framework for system description, construction and evolution. However, we recognise that there are a number of aspects which should be reconsidered and improved, or which should be incorporated. Our current work is Configuration

Programming

9

19/8/91

aimed at these aspects. The main forum for this work is in the REX project supported by the European Economic Community under the ESPRIT II initiative. REX, Reconfigurable and Extensible Parallel and Distributed Systems, aims to “...develop an integrated methodology and associated support tools for the development and management of parallel and distributed systems”. Ten European partners from industry and university research groups are involved. The project started in May 1989 and is to run for 5 years. Two large demonstrators in the telecommunications and CIM areas act as a focus for the work and as a means for demonstrating the techniques and tools developed. Configuration programming concepts have been adopted as the basis of the REX work; as stated in the project synopsis, “...the notion of the system as a configuration of modular software and hardware components will be used as the framework on which to hang the research work on system specification, analysis, construction and modification” [39]. This section discusses our current work and, using the REX project as a context, describes the approach we intend to follow1 . System Specification and Modelling “Development and establishment of a common specification methodology both to express structure and behaviour of distributed and parallel systems and to guide the activities to construct and (re)configure them.” As discussed in the introduction, more formal approaches to the problem of system specification and construction advocate the use of formal specifications with transformation techniques to move to lower level specifications and finally to the system itself [1,4,28]. Although these approaches have produced some elegant techniques, they have still to be proven in practice on large systems. In addition, there has been little discussion of system evolution except for respecification and re-application of the transformations to a completely new system. If such approaches are to have any success in the long term, they must provide a practical structure or framework on which to compose their specifications and generate and evolve their systems. We believe that the configuration perspective provides such a framework, and that the transformation approach could benefit from adoption of structural specifications. For instance, transformation techniques could aid system decomposition into constituent components, each with a formal specification of its behaviour. Composition rules could be used to derive composite group behaviour from constituent component behaviours. These techniques would also be useful in identifying respectively where changes should be applied and for predicting the affect of change. Our current approach actually puts more emphasis on the constructive composition aspect rather than decomposition [24,6]. Decomposition is currently more practical as a less formal process, with composition of component specifications as the more promising means for deriving or proving satisfaction of a system specification. The approach is then analogous to system design description and construction except that the entities being manipulated are not software components but their specifications. The objective is to ensure that the structure of the system and of its specification is the same2 . In fact, the basic configuration programming principles are applicable to formal specifications of component behaviour, where: 1. The configuration language used for structural specification is separate from the specification language used for specifying component behaviour; 2. Component specifications should specify the visible behaviour at the component interface. 3. Using the configuration language, complex specifications should be definable as a composition of component specifications; 4. Changes should be expressed at the configuration level, as changes of the constituent specifications and/or their interaction. In practical terms, the approach is to associate optional specification attributes (figure 11) with 1

The author is the Technical Director of the REX project. Nevertheless, the statements regarding the future work in the project reflect the opinion and biased interests of the author. Since the project is in its initial phase, many of the concepts are still under discussion. Acknowledgement is made to all participants in the project who have contributed to the ideas expressed here. 2 The work by Castro [6], also reported at this Conference, illustrates use of this constructive approach using the same patient monitoring system example. Configuration

Programming

10

19/8/91

actual components. These attributes are intended to include specifications for aspects such as functional behaviour, timing, and resource requirements. An analysis tool could be provided for each type of attribute, thereby providing the means for deriving and checking behaviour, timing constraints and resource requirements of composite components, provided that those attribute specifications are supplied. In addition, attributes are a means of supporting the use of different diverse formalisms for each type of specification. Users could select that formalism most appropriate to the application and their experience, and then use the associated analysis tools. We do not intend (currently) to work on the problems of relating specifications in different formalisms but expect that a single formalism will be used for each type of property where analysis and reasoning is required. Perhaps a process akin to ‘natural selection’ will indicate which is the most useful of the offered techniques.

COMPONENT

{ attribute type: ... attribute type: ... ... }

Figure 11 - Component with Optional Specification Attributes The kinds of property which we hope to check are component compatibility from specified restrictions on component interactions, satisfaction of global constraints, and satisfaction of a composite component specification by the composition of component specifications. We seek techniques which support compositional reasoning, with rules to infer composite behaviour (such as CCS [30]). System Development Methods “The provision and use of development methods for the systematic construction of software for distributed architectures, especially focussed on stepwise refinement of the problem domain, parallelism and decomposition to form a configuration of constituent components.” The development method we require should be based on the principles of configuration programming i.e. early identification of an explicit structural specification, context independent component types and hierarchical composition of component instances. Standard design techniques, such as JSD [16], SASD [43,8] or DARTS/DA [13] could be used to aid the decomposition process provided that the above principles are emphasised. This has led to our constructive design approach [24] which emphasises the bottom-up constructive aspect rather than decomposition. The approach (which is recursive) is shown diagrammatically in figure 12 and consists of the following steps: 1. Structure and Component Identification: Initial design aims to identify the main processing components and produce a structural description indicating the main data flows. This step provides an initial identification of component types. Furthermore, hierarchical decomposition of any of the component types into a configuration of subcomponents may be performed at this stage, or left for later refinement. 2. Interface Specification: Introduce control (synchronisation) between components and refine the configuration, component interface specifications (intercommunication) and component descriptions accordingly. The formulation of precise interface specifications permits the detailed design, implementation and testing of a component type to proceed independently from the rest of the system design. 3. Component Elaboration consists of elaboration of the component types, either by hierarchical decomposition of composite component types into a configuration of subcomponents (as in steps 1 and 2), or by detailed functional description of behaviour for primitive process components. As before, the identification of common component types is emphasised. 4. Construction is by instantiation and interconnection of components to form distributable components (logical nodes), followed by system configuration by allocation and interconnection of distributed components. Configuration

Programming

11

19/8/91

5. Modification and evolution of the system is performed by the replacement or addition of components and connection changes. As in all realistic development processes, the steps tend not to be followed purely sequentially, but include both iteration (and recursion) and the opportunity to advance in the process on one part of the system while lagging behind on another. This design approach has been extensively used, though there is still little experience of what effect change exerts on the design process. Nevertheless, we do believe that components can and should be designed to accommodate change. Software Structure Real World Application

System

Component Types

Interface specification

+

Elaboration component composite structure program begin ....... end

Figure 12 - A diagrammatic representation of the Design Approach We also intend to provide a separate CASE tool for the design process based on ConicDraw, the tool currently used for graphical monitoring and change of the operational system [21]. This tool already provides the required graphical support, but would have to be enhanced to support the duality we preserve between textual and graphical configuration descriptions. In addition, it should include method advice based on heuristics (such as was used in the TARA project [20]), recording of the designer’s status and progress in the method, and calls to the component programming and configuration building tools for editing, compiling, constructing and loading the distributable components. In addition to the provision of a development method, there is a need to support distributed development of the system by teams of developers. At the simple pragmatic level, this requires that we support separate and independent compilation, construction and testing of components and subsystems which can be subsequently integrated into a system. From the design point of view, we need to support partitioning of the application system and even perhaps the use of different design representation schemes. The former can be supported using the viewpoints concept adopted from the requirements engineering technique, CORE [33]. A viewpoint identifies a role or responsibility performed by humans, hardware or software in the domain of the required system and its environment, such as that of the operator, the safety system, the data Configuration

Programming

12

19/8/91

acquisition system and so on. Rules can be provided for checking consistency between these viewpoints and for integrating them. Experience using viewpoints has shown that they provide a sensible and cohesive partitioning of a system. Our belief is that they provide a conveniently selective means for viewing system requirements, design information, system architecture and evolutionary changes. Figure 13 illustrates this point in a simplified form, since in actual fact the viewpoints may overlap or address some particular aspect of many components (eg. failure recovery). Furthermore, we intend to investigate the use of integrating different styles (representation techniques) and workplans (submethods) using viewpoints, thereby providing a partitioning of the application domain, the method steps and the representation formalisms [11].

Requirements:

VIEWPOINTS

elicitation, specification & analysis

Design:

Construction:

Reconfiguration:

Documentation:

Figure 13 - Using Viewpoints to partition the Application Domain Component Programming “Development of adequate linguistic support for the design of software components with a high degree of parallelism, especially exploiting the potential for asynchronous communication and real-time processing.” The many different diverse functions in distributable systems should be programmable in the appropriate style and programming language. Programmers should not be forced to use the same language for all aspects of the system. Hence systems should be constructible from components implemented in different programming languages, selected to suit the particular application function or architecture. This has been recently referred to as ‘multi-paradigm programming’ [45] but that description fails to indicate the need for (and benefits of) providing a configuration framework in which to integrate these heterogeneous components. Component interfaces should be specified in a common Interface Specification Language (ISL) which describes the component interaction points (cf. ports in Conic), the types of information generated and received, the synchronisation expected for interaction, exception handling and constraints. Component interconnection can then be checked and supported by transformations where necessary. In practice we envisage that only a small set of component implementation languages will actually be used. In order to support the different protocols and synchronisation that languages and application use, we are investigating the provision of simple communication primitives from which more complex protocols could be composed. This may require the grouping of primitive ports and use Configuration

Programming

13

19/8/91

of the communication primitives in a specified order according to the responses. This programmed interaction should be composable as a form of ‘protocol unit’ which could be compiled into code in the component language. At the configuration level, a composite port and associated protocol unit could linked to other compatible units of other components. Another programming facility which we intend to support is the ability of a component to initiate reconfiguration. The actual reconfiguration change would be specified at the configuration level using the configuration language. This is discussed below. Configuration and Dynamic Reconfiguration “The provision of graphical and linguistic support for the description, construction and evolution of systems based on the configuration and reconfiguration of software components, their interrelationships and allocation to the hardware configuration.” Our experience in using a separate configuration language in the Conic environment has been very positive, but has highlighted a number of issues which we hope to investigate in our future work. Three of the most important of these are as follows: 1. The configuration language is too closely tied to a progamming language in our use of Pascal message/port types and of particular communication primitives. 2. Although Conic supports externally initiated change, there is currently no language support for programmed reconfiguration changes: this is currently written using UNIX shell commands to form change scripts. 3. The role and use of hierarchy in the configuration language and the resultant system requires reconsideration. The first issue on language dependency could be solved by separating the configuration language from the component ISL (discussed above), thereby making the configuration language purely structural: for the specification of graphs. The arcs (connections) need not be directed since, although individual ports will be directed, the direction of groups of ports is less meaningful. For instance, the use of a protocol unit of port groups in two components to be connected would require that one of them has its direction reversed; this seems rather artificial. On the other hand, our experience designing Conic components has indicated that directionality at design time is useful in indicating the control relationship between components, and is needed in dynamic change. Practical experience has also shown that the suggested separation between configuration and component interface can lead to a proliferation of descriptions and files, such as one for the component code, one for the ISL interface, and one for the pure configuration structure. We hope to be able to cope with this separation by the provision of software tools which could combine and display associated separate descriptions, and additionally by permitting separable descriptions to be combined where maximum flexibility is not required (eg. all components are in the same programming language). Again, tools could be used to extract the information needed for interaction. ward new-patient new

new

bed[i] alarm bed

wardmanager

remove remove-patient

bed[i]

alarm[1] bed[1]

bed[3] bed[4] bed[5] nurse alarm[3] alarm[4] alarm[2] alarm[5] bed[2]

remove

component ward (N:integer=5; S:site=sun1); ports new-patient, remove-patient; use patient, nurse, manager; create wardmanager:manager at S; nurse: nurse(N) at S;

Configuration

Programming

14

19/8/91

change new (i:1..N; bedsite:site); create bed[i]:patient at bedsite; link bed[i].{alarm,bed} to nurse.{alarm[i],bed[i]}; end; change remove (i:1..N); remove bed[i]; end; link wardmanager.{new-patient, remove-patient} to {new-patient, remove-patient}; wardmanager.new to change.new; wardmanager.remove to change.remove; end.

Figure 14 - Patient Monitoring System with Programmed Changes Secondly, it should be possible to define programmed reconfiguration changes to a composite component at the same level (and scope) as the definition of the component. These could be defined in the configuration language, and invokable from components created at that level. In this way components could be programmed to invoke changes under any specific condition. For instance, figure 14 shows a possible description of a ward component in which a wardmanager can invoke changes to create or remove patients, either under some internally specified condition or as a result of a request initiated from outside the ward (ie. by a request to new-patient). An enhanced nurse component could also have been given the capability to invoke such changes. In distributed systems having both programmed and externally-initiated changes, managers such as the ward manager could act as the component which responds to change invocations and could serialise the changes to a component to prevent interference between concurrent changes. Distributed changes across multiple distributed components would still require some form of concurrency control. As in Conic, we would still like to support arbitrary changes initiated externally. One way to do this would be to consider it as the addition of a new change specification into the composite component. This concept and other possible changes to the change specifications requires further investigation. Where necessary, constraints could be specified to limit and prevent changes which would invalidate the operation of the component. Recording of change histories is yet another concern. Finally, we recognise that hierarchy is very useful in the design and description of components and systems. Conic utilises an hierarchical construction for group modules (which cannot be distributed) until the level of a logical node (which can). A logical node is a group module which includes a run-time executive. It is the unit of distribution, failure and change. The system is constructed as a flat structure of interconnected nodes. For efficiency, even the internal structure of nodes is flattened to remove any indirection overhead in the communication between ports. Thus the result is a compromise between a desire to use uniform hierarchical structuring and a pragmatic use of nodes for efficiency, distribution and change. Embedding change in components at any level seems attractive and more uniform, as does the notion of building node hierarchies. In large systems, management will require some form of domains [41] to group those distributed components to which a management policy applies. Components could then belong to more than one domain. Supporting such distributed hierarchies, especially with nested changes, is more complex than was provided in Conic, and will probably have to be controlled and restricted in some way in order to be practical. In general, we wish to retain the conceptual clarity of use and practical efficiency of the Conic configuration language and environment, but to reconsider the above issues and try to find some (compromise) solution which satisfies elegance, clarity and efficiency. REX Environment The development of tools to support software components and systems during runtime and to enable their configuration and reconfiguration, including monitoring tools for the extraction of data from the running system for timing analysis, load balancing, and reconfiguration.

Configuration

Programming

15

19/8/91

The provision of techniques and tools for the analysis and validation of the system behaviour and performance, based on the system specification, its realisation and simulation. The REX project expects to provide tool support in all areas of concern to the project. Our vision is of an environment that consists of a number of separate software tools which provide the offline compilation, analysis, verification and display facilities, and the run-time support for execution, reconfiguration, monitoring and so on. We intend to deliberately avoid constructing some large, uniform data base as a repository for all information, both because of a firm belief that it leads to an inflexible and unwieldy environment in which to construct and use tools, and because we are not involved in distributed data base research. We would rather advocate the use of translators and transformation tools to make information available to other tools and users. We will, of course, strive to provide a uniform user interface for all of these separate tools. Demonstrators The use of realistic examples and demonstrations from the industrial automation and telecommunications areas is integrated in the project, thus showing the feasibility of the REX approach and the practicability of the developed tools. The approach adopted in the development of the REX environment is a pragmatic one, involving early development of a prototype environment in the first phase (three years) of the project, and subsequent further development and evolution as a result of experience. Using the selected industrial applications of telecommunications and CIM as a focus, two laboratory demonstrators are being defined and constructed to act as the testbeds for the techniques, mechanisms and tools that are developed. For instance, the REX partners are already using a simple telephone exchange system as a case study for different specification techniques, design approaches and distributed implementations. Such a prototype has been designed and implemented in Conic. The result at the end of the first phase will be a demonstration of realistic applications together with an initial prototype version of the support environment.. The second phase of the project will extend and integrate the REX tools into a comprehensive environment and extend the demonstrators to real environments. The demonstrators will also serve as a base for the mutual transfer of technology and industrial requirements between research and industrial partners. 4. CONCLUSIONS The principles of configuration programming and its use of a configuration language provide the basis for the Conic environment for distributed programming. Its success has encouraged us to embark on a broader and more ambitious project, based on the same principles but incorporating many more of the aspects of the development process. This project, REX, includes work on formal specification techniques and tools for analysis and verification, on design methods and tools for recording and aiding the design process, and on configuration and dynamic reconfiguration and tools for performing the construction, reconfiguration and extension of distributed systems. Other important concerns include the support for multiple programming languages, support for distributed development and specification/analysis extensibility by the addition of component attributes each with associated software tools to manipulate that attribute. As mentioned, many others also adhere to some or all of the configuration programming principles, particularly in their use of a configuration language. A factor which has undoubtedly contributed to the success of our work has been the adoption of a single, uniform underlying model for all phases in the software development process. Unless essential to describe, analyse or construct some particular part or aspect of the system, it is certainly beneficial to retain this uniformity. In addition, we have always attempted to provide software tool support wherever feasible. We intend to continue working in this fruitful vein.

Acknowledgements

Configuration

Programming

16

19/8/91

Acknowledgement is made to my colleagues at Imperial College, Naranker Dulay, Anthony Finkelstein, Jeff Magee, Keng Ng, Morris Sloman and Kevin Twidle for their contribution to all the work described in this paper. I also acknowledge the work of our partners in the REX project, for their comments and discussions regarding the REX research. Finaly I gratefully acknowledge the SERC ACME Directorate under grant GE/E/62394, the SERC under grant GE/F/04605 and the CEC in the REX Project (2080) for their financial support. REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25]

R. Balzer, "A 15 Year Perspective on Automatic Programming", IEEE Transactions on Software Engineering, SE-11 (11), Nov. 1985 M.R.Barbacci, C.B.Weinstock, and J.M.Wing, "Programming at the Processor - Memory - Switch Level", Proc. of 10th IEEE Int. Conf. on Software Engineering, Singapore, April 1988. A. Black, N. Hutchison, E. Jul, H. Levy, L. Carter, “Distribution and abstract types in Emerald”, IEEE Trans. on Software Eng. SE-13(1), Jan. 1987, pp. 65-76. R.M.Burstall, J. Darlington, “A Transformation System for Developing Recursive Programs”, JACN, 24, Jan.1977. N. Carreiro, D. Gelernter, Technical Correspondence, Comm.ACM, Vol. 32, 10, Oct. 1989. J. Castro, J. Kramer, “Temporal-Causal System Specification”, Proc. of IEEE Int. Conf. on Computer Systems and Software Engineering (CompEuro 90), Israel, May 1990. M. Coulas, G. MacEwen, G. Marquis, “RNet: A Hard Real-Time Distributed Programming System”, IEEE Transactions on Computers, C-36 (8), August 1987. T. De Marco, "Structured Analysis and Structured Specifications", Prentice Hall 1979. F. DeRemer, H.H.Kron. "Programming-in-the-large Versus Programming-in-the-small, IEEE Trans. Software Engineering", Vol. SE-2, 2, June 1976. N. Dulay, “A Configuration Language for Distributed Programming”, Ph.D. Thesis, Imperial College, London University, 1990. A. Finkelstein, M. Goedicke, J. Kramer, C.Niskier, “Methods and Viewpoints in Requirements Engineering”, in 'Proceedings of 2nd METEOR Workshop on Methods for Formal Specification', to be published by LNCS, (Springer Verlag). J.A.Goguen. "Reusing and Interconnecting Software Components", IEEE Computer, (Designing for Adaptability), Vol. 19, 2, February 1986. H.Gomaa. "A Software Design Method fro Distributed Real Time Applications", Journal of Software Systems, Feb. 1989. D. Harel, H. Lachover, A. Naamad, A. Pnueli, M. Politi, R. Sherman, A. Shtul-Trauring, "STATEMATE: A Working Environment for the Development of Complex Reactive Systems", Proc. 10th. Int. Conf. on Software Eng., Singapore April 1988, pp. 396-406. C.A.R.Hoare, " The Axiomatic Basis of Computer Programming", Communications of the ACM 12, October 1969, pp. 576-583. M.A.Jackson, "System Development", Prentice Hall 1983. S. Kaplan, G. Kaiser, “Garp: Graph Abstractions for Concurrent Programming”, ESOP '88, Nancy, France, March 1988, Springer-Verlag, pp. 191-205. J.Kramer, J.Magee, "Dynamic Configuration for Distributed Systems", IEEE Transactions on Software Engineering, SE-11 (4), April 1985, pp. 424-436. J. Kramer, J. Magee, M. Sloman. The CONIC toolkit for building distributed systems. IEE Proceedings, Vol. 134, Pt. D, No.2, March 1987. J. Kramer, K. Ng, C. Potts, K. Whitehead, “Tool Support for Requirements Analysis”, IEE Software Engineering Journal, Vol. 3,3, May 1988. J. Kramer, J. Magee, K. Ng, "Graphical Configuration Programming" IEEE Computer, 22(10), October 1989, 53-65. J. Kramer, J. Magee, and M.Sloman, "Configuration Support for System Description, Construction and Evolution", Proc. of IEEE 5th Int. Workshop on Software Specification and Design, Pittsburgh, May 1989. J. Kramer, J. Magee, "The Evolving Philosophers Problem: Dynamic Change Management", to appear in IEEE Trans. on Software Eng. J. Kramer, J. Magee, A. Finkelstein, “A Constructive Approach to the Design of Distributed Systems”, to be presented at the 10th Int. Conf. on Distributed Computing Systems, May 1990. R.J. Leblanc and A.B. MacCabe, “The Design of a Programming Language based on a

Configuration

Programming

17

19/8/91

Connectivity Network”, Proc. 3rd Int. Conf. On Distributed Computing Systems, 1982. [26] T. LeBlanc and S. Friedberg. "HPC: A model of structure and change in distributed systems". IEEE Trans. on Computers, Vol. C-34, 12, December 1985. [27] I. Lee, N. Prywes, B. Szymanski, “Partitioning of Massive/Real-Time Programs for Parallel Processing”, in Advances in Computers, ed. M.C. Yovits, Vol.25, Academic Press 1986. [28] M. Lehman, V. Stenning, W. M. Turski, "Another Look at Software Design Methodology", Software Engineering Notes, Vol. 9, No. 2, April 1984, pp. 38-53. [29] J.Magee, J.Kramer, and M.Sloman, "Constructing Distributed Systems in Conic" IEEE Transactions on Software Engineering, SE-15 (6), June 1989. [30] R. Milner, “A Calculus of Communicating Systems”, LNCS Vol. 92, Springer Verlag, 1980. [31] J. Mitchell, W. Maybury, R. Sweet, “Mesa Language Manual, Version 5.0”, Xerox Parc Report CSL-79-3, Palo Alto Research Center, April 1979. [32] H. Muhlenbein, T. Scheider, S. Streitz, “Network Programming with MUPPET”, Journal of Parallel and Distributed Computing, Vol. 5, 1988. [33] G. Mullery, “CORE- A Method for Controlled Requirements Analysis and Design”, Proc. of 8th IEEE Int. Conf. on Software Engineering, Munich, Sept. 1979. [34] W.P. Myers, G.F. Myers and L.C. Constantine. "Structured design", IBM Syst. J., vol. 13, no. 2, pp. 115-139, 1974. [35] J. Nehmer, D. Haban, F. Mattern, D. Wybranietz, D. Rombach, “Key Concepts of the INCAS Multicomputer Project”, IEEE Transactions on Software Engineering, SE-13 (8), August 1987. [36] H. Ossher, “A Case Study in Structure Specification: A Grid Description of Scribe”, IEEE Transactions on Software Engineering, Vol. 15, 11, Nov. 1989. [37] D.E.Perry, "The Inscape Environment", Proc. of 11th IEEE Int. Conf. on Software Engineering, Pittsburgh, May 1989. [38] J. Purtilo, “A Software Interconnection Technology”, Computer Science Dept., University of Maryland, TR-2139, 1988. [39] REX Technical Annexe, ESPRIT Project 2080, European Economic Commission, March 1989. [40] C. Rich, R. Waters, “Automatic Programming: Myths and Prospects”, IEEE Computer, Vol. 21, 8, August 1988. [41] M. Sloman, J. Moffett, “Domain Management for Distributed Systems”, Integrated Network Management I, Proc. of IFIP 6.6 Symposium, Boston May 1989, ed. B. Meandzija, J. Westcott, North Holland. [42] M. Stovsky, B. Weide, “STILE: A Graphical Design and Development Environment”, Digest Compcon Spring 87, CS Press, California. [43] E.Yourdon, L.Constantine, "Structured Design", Yourdon Press, 1978. [44] P.Zave, "An Operational Approach to Requirements Specification for Embedded Systems", IEEE Trans. on Software Engineering, SE-8 (3), 1982. [45] P.Zave, "A Compositional Approach to Multi-Paradigm Programming", IEEE Software, September 1989.

Configuration

Programming

18

19/8/91