Simulating the emergent behavior of complex software-intensive organizations Alan M. Christie Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213, USA phone: 412-268-6324 email:
[email protected]
Abstract Organizations are composed of individuals and groups of individuals who interact with their local neighbors. These individuals and groups inherently have limited horizons both with whom they interact and in what information they can access. Because of this limited visibility no one individual can "see" the edges of the universe in which they live and in this sense the system can be considered unbounded. A major characteristic of unbounded systems is the fact that global behavior can emerge which cannot be predicted from local actions. These issues have direct relevance to the understanding of software-intensive organizations. However, performing experimental investigations to assess these organizational properties is virtually impossible. Thus we need a simulation language in which unbounded systems and emergent properties can be simulated and examined. Few, if any existing simulation languages explicitly provide the appropriate mechanisms and this has motivated the development of the Easel language, the topic of this paper.
Keywords Simulation, unbounded systems, emergent properties, software process, informal process
The simulation of large complex software engineering processes Recent advances in information technology are having a fundamental effect on the way companies now do business. Speed of communications, rapid access to information, and ease of interaction between different business entities have all been significant influences in this area. The resulting competitive climate has meant: great success for those who can adapt effectively, wrenching changes for others, and has put many out of business [3]. By
David A. Fisher Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213, USA phone: 412-268-7703 email:
[email protected] the nature of their work, software-intensive businesses have been on the leading edge of these changes. Software-intensive companies must deal with dynamically changing conditions (in terms of growth, personnel turnover, and product evolution etc.) and, to an increasing extent, have to succeed in a dynamically changing context. To address these issues, organizations have tried to "reinvent" themselves through reorganization, downsizing, mergers, changing cultural values etc. However, the consequences of such changes are quite difficult to predict. A significant contributor to this is that changes can result in complex series of interactions that have unforeseen consequences [12]. To make the above more concrete for softwareintensive organizations, the following questions help illustrate the issues 1 : • What are the consequences of using geographically distributed groups for software development (e.g., using collaborative technology)? • What is the impact of workflow automation on software development? • What is the influence of cultural and organizational factors (trust, authoritarianism, customer orientation etc) on software development? • What is the effect on software development when collaboration between culturally different organizations is involved? • What is the effect of the development organization's size or structure in successfully completing software projects? • Can a highly distributed organization produce hierarchically structured software? These questions imply the need to understand the effects of large numbers of actors, each of which 1
Note that these examples are illustrative - we do not claim to have addressed them.
interacts with local actors. Similarly, in these scenarios, actors do not have access to all relevant information, but only to a subset. These are the characteristics of "unbounded" systems, that is, systems in which each individual actor has a restricted view of information in their world. Thus, decisions that actors make may seem locally appropriate but can turn out to have unintended ramifications when seen globally. The resulting system properties are called emergent and are difficult to predict [2].
information. In other words, unbounded networks may be characterized by distributed administrative control without central authority, by limited visibility beyond the boundaries of local administration, and for any individual actor, by incomplete information about the network's topology and component functions.
Emergent properties and unboundedness Emergent properties are analogous to the properties of biological and social systems where each actor performs simple local behaviors involving interactions with other actors but without complete knowledge of who else is participating. In these systems, extremely complex global properties emerge from these simple behaviors and interactions in the absence of central control or administrative authority. These local behaviors form the basis of an emergent algorithm - a computation that achieves predictable global effects by communicating with a bounded number of immediate neighbors and without the use of central control or global visibility.
Figure 1 Actors with limited visibility These properties are certainly true in the software development domain, even in organizations that appear to have a clear authority structure and well-defined processes. However, this can be deceiving. In real organizations: • each individual has limited knowledge of the entire organization • chains of command do not always communicate effectively • software modules will not always be built consistent with their requirements. • orders may not be implemented as requested • inter-personal and inter-organizational rivalries are common (not invented here syndrome) • company-defined processes are not followed accurately In all of the above examples, the sum of local actions results in global effects, but no one can predict with certainty what these effects might be.
Examples include behavior in ant colonies [4], the culture created by people of a region, and the national economy. Internet examples include the use of chat rooms, the overall governance of the Internet, and the combining of independently developed search tools in ways unanticipated by their authors. Unbounded systems also include other human-created systems such as: • the electric power grid • the telephone system • the stock market • software development organizations In such domains, collections of simple entities can survive successfully even when these entities have quite limited knowledge of their environment. The finite horizon that can be experienced by each actor results in system properties that cannot be determined by any individual actor (in a sense actors cannot see the wood for the trees). Thus to any actor the system appears to be unbounded and the system-wide properties that arise cannot be predicted solely from local
As implied in the above discussion, this paper focuses little on traditional software engineering processes (e.g., emphasizing artifact flows). A major objective of the paper is to make the
2
software process community aware of the power of Easel's conceptual capabilities in addressing a broader class of problems that the classical approaches to simulation can address only with difficulty.
Specifically, we know of no complete simulation languages that explicitly support the modeling of unbounded systems as does Easel: • Easel’s actors are first-class citizens • Easel allows for dynamically changing neighbor relationships • Easel allows simulation to proceed in the presence of incomplete and imprecise information • Easel’s level of abstraction allows one to manipulate concepts in high-level, symbolic terms • Easel provides a threaded environment that has significant modeling and performance implications • Easel provides a mobile code facility that allows code modification and execution within a simulation • Easel is supported by a dynamic visualization facility
We believe that concepts of local action and visibility, unboundedness, and emergent properties are important factors in simulating systems where large numbers of loosely coupled actors are involved. To this end, we are developing a new general-purpose simulation language called Easel, and this is currently being implemented [7]. The rest of this paper reviews the concepts behind Easel, describes some of its unique properties and provides three examples of Easel's use. These small examples do not explicitly focus on software engineering, although the final example models a simple social interaction process. The examples are provided to illuminate Easel's principles in modeling actors in unbounded systems
A type is a type is a type...
The need for Easel
Easel is a property-based type language, that is, all modeled entities are considered to be types. A type is a description of some class of objects, while a description is a set of properties. An example of a type is any object that satisfies the type’s properties. Easel has a built-in family of types that can be extended to specify userdefined types. Figure 2 illustrates some of the important built-in Easel types and their relationships. The following are some characteristics of Easel types:
To the best of our knowledge, there are no simulation languages that are explicitly designed to address the types of problems discussed above. There are packages such as Swarm [11] and MAML [9] that provide libraries of functions which run on languages such as C++ and Java. There are also many simulations that have been hand-crafted using general purpose languages and whose aim is to demonstrate specific behaviors in such fields as artificial life [1]. The only other tool that is also a complete language is StarLogo [10]. However, it is limited in several respects such as its restriction to the spatial domain, its inability to model neighbor relationships, its restriction to relatively simple models (e.g. it cannot model data structures), and its grid-based graphic depiction. Traditional simulation tools used to model software engineering processes (such as Extend[6], and iThink[8]) provide powerful capability for models in which the topological relationships between the elements are fixed prior to the simulation's execution. However, they are not designed to perform simulations in which these relationships (for example between actors in the system) evolve. This characteristic is very important in the simulation of informal processes.
Types are built up by inheriting properties of parent types • All types inherit from the root type “all”. This includes all consistent (true) types and inconsistent types (false) • The type that contains all types is the false type - so named since types with inconsistent properties (e.g. 3 > 5) are also types - however, the false type has no examples • as one ascends the type hierarchy, types accumulate more and more examples (from their children) • as one descends the type hierarchy, types accumulate more and more properties inherited from their parents) • examples of types are - mutable - types whose properties are changeable - immutable - types whose properties are
3
•
unchangeable - prop- a physical "thing" that does not have behavior - actor - a physical "thing" with behavior and which is threaded - abstract - a type that can be completely described within computer memory - type- the set of named types is also a type (type “type”) an example of an abstract type is the set of all integers - even the number “5” is a type (albeit a degenerate type - a singleton) - it inherits properties from the types: positive integers and odd integers;
Easel's architecture is designed so that it can simulate very large numbers of independent "actors". Actors are simulated entities of the physical world (e.g., a system administrator, user, intruder, automobile, bird, or the moon), of the electronic world (e.g., a computer, router, or peripheral device), or of the software world (e.g., a software agent or task). Each actor has it own thread of control and this allows for a high degree of parallelism in Easel's execution. Actors can interact directly only with their near neighbors and only in ways prescribed by their neighbor relationships. Neighbor relationships are protocols of interaction and are defined as types that can be associated with any actor. Thus, in a simulation of birds in flight, a bird's near neighbors might be any bird or other object that the bird can see from its current position and heading. In an organizational simulation, an actor's near neighbors might be only those actors that are connected by formal organizational ties. In the latter example, neighbor operations might include sending and receiving messages.
Type manipulation within Easel The Easel type hierarchy allows one to build up and manipulate types. This is unlike other conventional programming (or simulation) language where one can only operate on examples of the type (e.g., through iteration). In the object oriented paradigm, the equivalent to types is classes. However, Easel's types can be manipulated in powerful symbolic ways that are not possible with classes. The program in Easel Example 1 illustrates some of the ways that types can be manipulated (note that this is a program not a simulation).
In summary, actors have • threads of control • have behavior • are “born” and can “die” • have significant performance advantage over non-threaded approaches Let us now look at a very simple Easel simulation - taken from the "ant" world. This example assumes that there is a centrally located nest of 10,000 ants, and that the ants move out in random directions, all at the same speed. (1.0 cm/sec). The emergent property that results from this is that, the ants generate a circle. This property is unknown to the individual ants. After a certain time, they all "die" leaving a circle of radius 1000 cm.
Simple types are first defined and other types are built up from the simpler inherited properties of these types. Thus hair has color and length, mammals are warm-blooded hairy things, and dogs are mammals. Specialized types of dog are then defined. We can then test the truth (or falsity) of relationships between the defined types. Thus we see that a poodle is a brown dog, and brown dogs are a subtype of the type mammal. Note that parameterized types such as hairy(t: type) allow sub-types to be defined in the same way that adjectives qualify nouns in English to select subtypes (e.g. hairy dogs”). This use of adjectives provides Easel with a powerful mechanism for defining sub-types. In a similar way, the use of quantifiers such as any, all, and some provides Easel with a means of selecting sub-types that have to be manipulated or tested
The ant properties are their position and direction. Each new ant that is created is given a random direction and sent on its way through the simulation loop for each time do. Note that • a circle is a sub-type of simulation type • an ant is a sub-type of the actor type, and • a generator is a sub-type of the facilitator type (which performs operations external to the simulation and in this case creates the ants).
Actors and neighbors
The simulation program can be divided into three parts: definition of attributes of ant types,
4
•
behavior of ant types, and generation of the ant examples. Note that easel programs are structured through indenting and "outdenting."
provides a graphical capability to display simulation results
A software engineering related example
Traditional discrete and continuous simulation languages have fixed node and link type of models. This is fine for processes that follow predefined topologies such as in the “manufacturing” paradigm. However, it is not applicable for systems illustrated by Example 2. Here we see a focus on actors and the dynamically changing relationships between actors (as reflected by their nearness to each other). These are the sorts of issues that are important in modeling informal processes In this regard, Easel introduces a “neighbor” relationship concept. What is a neighbor can be a function, not only of physical distance (as in the above example, but of other considerations such as: • network linkage • organizational dependencies • common interests
Easel can readily model the types of formal processes that are common to software development activity - such as the flow of artifacts through the development cycle. As mentioned above, there are the types of processes that have been simulated many times with commercial traditional simulation packages. However, Easel's strength is in the modeling of informal processes - that is processes where interactions between actors of the system have not been predefined. In this sense Easel can simulate less deterministic processes. These issues were also discussed earlier. What we will illustrate now is a simple model of such a process. This model, defined in Example 3, describes the interaction of individuals in a conference room. The assumptions of the model are: • actors have a set of software engineering interests (database, security, etc) • actors randomly wander around a circular room • if two actors meet and have at least one interest in common they form a group which remains at the location where they met. • a group’s interests consist of the intersection of the actors’ interests • actors who wander close to a group and have at least one interest in common with the group join that group.
Interacting with a simulation A simulation needs mechanisms through which it can interact with the person performing the simulation. Thus we need to be able to start the simulation, set up simulation parameters, change parameters while the simulation is proceeding, and observe output from the simulation. None of these functions are part of the simulation, they either send information to the simulation or retrieve it from the simulation. Thus Easel supports facilitators, observers, and graphical depiction. In summary these have the following attributes:
The example is relevant to software engineering process as it is representative of the types of informal interaction that will occur during software development. To date these types of social interactions have not been explored, in part because of a lack of tools that can adequately address them. In this example neighbors are individuals who come physically close to each other. The neighbor relationship is also expressed by the commonality of interests that two individuals have. If the model were executed, it would result in multiple commoninterest groups forming. The number of groups is an emergent property that is related to the distribution of interests across the actors. Note that this simulation example only shows the top level - this represents about 30 percent of the code.
Facilitators allow external control over the simulation. For example, they allow for • global control (in contrast, actors usually only "see" locally") • introduction of new examples • control of parameter values Observers allow one to extract values of the simulation parameters • to do statistical analysis • as input to graphical depiction Graphical depiction to display simulation output • provides a windowing facility for displaying output
5
to-use depiction facility that will help analysts visualize emergent properties and algorithms. The Easel language is now specified and a significant portion of the implementation is complete. We are currently working on several applications problems in the areas of computer network survivability, organizational dynamics, neural networks, and infrastructure assurance.
Conclusions Easel was developed to support the simulation of network system survivability. However, Easel is a general purpose language and has significant potential in addressing software engineering process problems.
It is hoped that this work will ultimately spark interest in using Easel within the software simulation community particularly in the context of simulating organizational behavior in competitive environments. Easel has many unique features that make it suitable for simulating formal and informal software development processes. Formal processes are characterized by models that have predefined topologies • nodes and links • “manufacturing” paradigm
In software-intensive organizations, planning decisions are made on the basis of currently available information that is inherently incomplete and imprecise. Such decisions can be consistent with the available information and its implications, but may result in unexpected consequences in the broader, more informed context of the real world in which they are applied. Specifically, in dynamic, rapidly evolving, and stressful environments, as exemplified by software-intensive organizations, actors • attempt to reach their goals with imperfect or biased information and with severe time constraints • have to deal with information that, while possibly accurate, may be insufficiently complete to expose critical issues • interact with neighboring actors, with varying degrees of trustworthiness when exchanging information. • may respond to different actors in different ways (e.g., as competitors or partners).
Informal processes are characterized by • local visibility and control • unboundedness • emergent properties Easel is designed to model • very large numbers of actors • neighbor relationships that change dynamically
Current status The Easel language is now stable but may show evolution as experience is gained in its use. An alpha version is now running on Macintosh platforms.
The primary purpose of the Easel language is to simulate unbounded systems. One such type of system is reflected by software-intensive organizations that, to succeed, are forced to rapidly adapt to a dynamically changing external environment. Key requirements for Easel (language and software system) include: an execution semantics consistent with unbounded networks, a one-to-one mapping between the simulation description and what is being simulated, the ability to support a broad spectrum of applications, and support for monitoring, data collection and analyses within a simulation. Another key requirement is an easy-
Many simulation languages can handle traditional “node and link” models. However, Easel’s strength is in modeling less formally defined processes such as those where social interactions take place. We encourage you to explore such applications with Easel when it becomes available. (Keep and eye on the Easel web page for status information - see [5]).
6
example inheritance more examples
all true =, /=
type (k5) applies to types, not examples of types union of props: type & type intersection of props: type | type negation of props: ¬ type
abstract type prop
mutable type
actor has thread of control
real *,/,^
immutable type enumeration
integer
(m1-m2) enum+enum enum-enum enum+int enum-int
float
int16
property inheritance more properties
types that do not change throughout the execution
false contains the properties of all types
Figure 2: A sub-set of the Easel Type Hierarchy
main is # build up inherited properties length: unit inches; color: type is enum(white, brown, gray, black); hair: type property length & color; hairy(t: type) type is property t & hair; brown_haired(t:type): type is property t.hair.color = brown; long_haired(t:type): type is property t.hair.length > 5 * inches; mammal: type is property hairy & warm_blooded; dog: type is property mammal; poodle: type is hairy dog; collie: type is long_haired dog; # the above statements allow all of the # following propositions to be true any poodle isa hairy dog; all brown_haired dog
