CONFERENCE O N SYSTEM S E N G IN E E R IN G R E S E A R C H
Paper #251
Towards Standardizing and Accelerating Systems Engineering Processes Larry J. Earnest
Azad M. Madni
Northrop Grumman Corporation One Hornet Way, M/S 9Q01/W6 El Segundo, California 90245-2804
[email protected]
Intelligent Systems Technology Inc. 12122 Victoria Avenue Los Angeles, CA 90066
[email protected]
Abstract Organizations engaged in the design of complex systems, are looking for ways to standardize and accelerate Systems Engineering (SE) processes and link them to project plans. Achieving the goal thus far has been more difficult than expected. To begin with capturing and making project-related process assets reusable has proven to be quite challenging. Also, reuse of process assets carries with it the risk of misapplication due to serious contextual differences in the author’s original context and the user’s current context. Beyond that, there are cultural issues that need to be dealt with. It is hard to break ingrained habits and overcome institutional resistance. Securing executive sponsorship, and making the requisite infrastructure investments pose additional challenges. This paper focuses on what it takes to capture and codify process knowledge in reusable form. It represents the key challenges in creating reusable process assets and offers recommendations for process-driven systems engineering.
SE Process Standardization and Cycle Time Reduction Problem The need for Systems Engineering (SE) is well-researched in virtually every industry involved with product development. Today SE been embraced as a preferred means to integrate enterprise processes throughout the development lifecycle and to achieve an effective balance between competing quality factors such as system performance, cost, schedule and risk. The DoD has issued several Instructions, memorandums and directives for use by the DoD and contractors, which require that a sound systems engineering capability be established for all systems development (DoD Directive). Systems engineering organizations, such as the International Council on Systems Engineering (INCOSE), which is supported by Government, Industry and Academia, has provided a well-defined set of processes and procedures to implement systems engineering practices in the development of complex systems. This development has resulted in a “cottage industry” comprising SE consultants that offer process and tool training to government and system developers as they strive to comply with these DoD issuances. After several years of attempting to comply with SE best practices, it is time to assess what is working well, what is not, and why. The International Organization for Standardization and other Management organizations have set standards for best business practices that require enterprises to be process-driven. Today SE processes provide references to applicable standards, CSER 2008, University of Southern California, ISBN - 978-0-9814980-0-3 PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
1
such as IEEE-1220, EIA-632, and ISO/IEC-15288. Additionally, the Capability Maturity Model Integration (CMMI) requires that enterprises establish standard processes to achieve consistency across their various organizations. Today, projects involving complex systems development are required to create defined processes by tailoring the organization’s core set of standard processes according to tailoring guidelines (CMMI). For example, each program/project is expected to create an instance of the core processes that defines the way in which the procedures will support the generation of new work products for a given project. A survey of such initiatives reveals mixed results in meeting these requirements. Research has shown that 80 to 85 percent of all corporate assets (including process assets) tend to be unstructured and not easily amenable to being reused, or tailored. Not surprisingly, a common practice on new programs is to merely reference the generic organizational processes. This practice of referencing generic processes on a new program is not the same as reuse; and it is most definitely not process tailoring as defined by CMMI. The consequences of this practice are that the final work products, their production cost, and the time it takes to create them cannot be traced to the processes that created them. For the lack of a better approach, most SE organizations today merely list their procedures in a Systems Engineering Management Plan (SEMP) to imply that they meet the CMMI requirement for working to a set of procedures. This practice does not provide the necessary rigor or mechanisms to validate that a work product was the result of following a referenced process/procedure. While there can be many possible explanations for why static processes are used this way and why processes are not being reused or tailored, the two most likely explanations are the lack of a common vocabulary standard representation of processes. These deficiencies make it different to rapidly access processes and employ them on programs. Figure 1 presents a conceptual approach to creating process assets (Madni, 2000; Madni, et. al., 1998). This approach is motivated by the recognition that the creation of complex systems takes too long to develop, costs too much and lacks repeatability from program to program (Madni, et. al., 2001) At the heart of this approach is a Systems Engineering (SE) process ontology (Madni, et. al., 1998). This ontology is key to both standardizing processes and providing key terms by which to tag and access processes. The processes based on this ontology and accessible by the terms contained in the ontology (and those supplied by users) comprise the contents of the process assets repository. The contents of the process assets repository can range from process models, and process instances, to process documents and reference material.
Need
Standardized SE Processes
Technologies
Integration
Cycle Time Reduction
Process Ontology Process Assets Repository
Achieve reuse of SE process assets
Impact
Key Terms
Standardized Process
Easily Accessible Reusable Process Assets
Figure 1. Conceptual Approach.
PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
2
Barriers to Process Standardization Ideally, when designing complex systems one would like to have standard processes and guidelines for applying them in a consistent manner. One would also like to capitalize on findings and lessons learned from earlier design efforts. However there are several barriers that need to be overcome before such a vision can become a reality. To begin with, the use of standardized processes is only beneficial when the processes used are well-defined. By process standardization, we simply mean a consistent way to describe and define processes for the user community. Merely, referencing ad hoc processes each time is of limited value. Therefore, the first step in process standardization is having well-defined processes. Even after processes have been defined, there is the danger of misapplication because of serious contextual differences in the author’s original context and the user’s context. Another key problem is the fact that existing documentation of SE processes tend to use process descriptions that are biased towards the author’s experience, (i.e. they are not sufficiently general) and, consequently, lack transferability to other projects. Yet another problem with process artifacts is that they are subject to capricious interpretation due to lack of standard semantics. This variability hinders process reuse and repeatability across programs. This is hardly surprising in that if the original process contains ambiguous task descriptions, then tailoring them becomes even more problematic, and usually results in a rewritten process rather than a tailored one. The foregoing suggests a pressing need to standardize - - we need a standard way to describe, model, index, store, and access processes.
It All Begins with Process Ontology Most organizations that develop complex systems have some type of process documentation. Those that do not are striving to become process-driven. However, having processes and ad hoc process documentation does not make an organization process-driven. Only well-defined processes can enable organizational transformation into one that is process-driven. The CMMI model defines a well-defined process as “A process that includes readiness criteria, inputs, standards and procedures for performing the work, verification mechanisms (such as peer reviews), outputs, and completion criteria.” The elements of a well defined process are presented in Table 1. Table 1: Definitions for a Well-defined process Describe individuals or collections of individuals participating in process activities (i.e. suppliers, customers, agents, reviewers, or verifiers of a practice). Entry crite- Describe the conditions under which an activity can start. The entry criteria can be a simple or compound predicate about the state of a work product, role, ria or activity. "TRL levels must be at a level 6 before they can be integrated into the system” is an example of entry criteria. Describe those items or work products produced by a prior activity and Inputs consumed by the current activity which adds incremental value. Describe what is being done. They may be directly associated with the Activities production of a product, a management function, or a service provided to help those directly involved to operate more effectively or efficiently. Outputs/work Describe those items that are a direct result of executing a process step. Work Roles
PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
3
products only exist if the process is executed. Describe the conditions under which an activity can be declared complete. The exit criteria can be a simple or compound predicate about the state of a work product, role, or activity. "Software requirements shall be reviewed by software managers and other affected groups," is an example of an exit criteria. A process ontology is what inter-relates the various elements of a process in a consistent, understandable fashion. The process ontology defines what exists (entities) in the process domain and defines all relevant relationships. Each relationship is in accord with a well-defined set of rules, constraints and semantic links that are ultimately intended to support process analysis and execution. products Exit criteria
Process Reuse is Key to Accelerating SE Process It goes without saying that no organization would deliberately duplicate the same design effort, solve the same problem more than once, and create a risk mitigation plan from scratch each time to mitigate the same risks, re-address identical integration problems from the beginning, or validate the exact same design twice. Yet this is exactly what occurs in organizations which do not have standard process assets that are calibrated, costed and indexed for reuse. So how does one achieve process reuse? To begin with, processes need to be defined and stored in standard form. They need to be indexed/tagged with appropriate terms from the process ontology and user context to facilitate their access from a repository of process assets. In this regard, process ontology plays a key role in process standardization, reuse, and cycle time reduction. It assures the use of a consistent, standardized terminology when describing, defining, designing, or modeling processes. It provides the key terms for indexing processes for rapid future retrieval. It enables the creation of reusable process assets - - the key to SE cycle time reduction. As importantly, it clearly conveys the inter-relationships within the various entities in the process descriptions.
Systems Engineering Process Assets Library A SE Process repository is a repository that contains process description documents, tailoring guidelines, templates, tools, examples, contextual information, and other related enterprise perspectives that contribute to determining if the work product is a good fit for reuse (Madni, 2000; Madni, et. al., 1998). A well-designed, well-maintained SE repository can facilitate process standardization, process reuse and improvement. Conversely, a poorly designed and maintained degenerates into a swamp of process confusion that discourages reuse and frustrates users. It is important to realize that the process assets in a SE repository can serve several purposes in the lifecycle of a system. Process assets can be reused, tailored, scaled, composed into a larger process, and, ultimately, transformed into integrated master plan/integrated master schedule (IMP/IMS) for complex systems program management (Madni, et. al., 2001). There are several other benefits to creating a SE-PAL. A well-defined process that is generated with the intent for reuse can, for example, be tagged with additional information that is useful for metric development and governance. For example, a well-defined process can be indexed with cost and schedule information. Processes that are indexed with such information, are key to bidding an effort, monitoring an ongoing effort, and collecting cost data during execution. For example, actual costs can be compared to predicted/estimated costs to refine the cost models used in the bidding process. The period of performance for each work product can PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
4
be monitored in similar fashion. Eventually, well-defined processes can include associated costs, as well as schedules and work products. They can also be continually monitored and validated for accuracy against the bid data. Figure 2 presents the various uses of a SE repository. IMP/IMS Creation (Project Management)
Cross-functional process definition (collaboration)
SE Repository • Indexable assets • Reusable assets • Composable assets • Tailorable assets
SE Process Improvement (CMMI)
SE Process Transformation (Lean, Agile)
Figure 2. Possible Uses of SE Repository. By augmenting the process model for a well-defined process with validated cost and schedule data can inform how SE work products are created.
Implications for Systems Engineering Reusable process libraries are an important capability for accelerating systems development and deployment cycle times. In the past, reuse has primarily been the result of chance or opportunistic success, where one program was able to take advantage of the efforts of another. Today, there needs to be a paradigm shift from current systems engineering practices to a systems engineering process where process asset reuse is an integral part of systems engineering and acquisition process. Most importantly, it is essential that an organizational infrastructure be implemented to “govern” domains, define product families and standards, establish ownership criteria, allocate investment resources, and direct the establishment of SE repositories. The potential benefits of a SE-PAL for SE is presented in Table 2. Table 2: Potential Benefits of a SE Repository 1) 2) 3) 4) 5) 6) 7) 8)
Reduced life cycle cost Higher productivity Improved product quality Faster time to market Consistent process application within a product family Continuous organization learning Pattern identification Add shareable
PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
5
Use of the Approach The ontology-enabled approach to process standardization and reuse was first employed by Intelligent Systems Technology Inc. (ISTI) in creating it’s ProcessEdge® Enterprise Suite product. This product is based on IDEON®, the enterprise process ontology developed by ISTI (Madni, 2000). The contents of the process repository are based on IDEON, an enterprise process ontology. The processes are readily accessible, tailorable, composable, and verifiable using the capabilities of the tool. A sample process assets repository screen is shown in Figure 3.
Figure 3. ProcessEdge® Repository. The standardized, pre-calibrated process fragments/snippets in this library can potentially support the full systems engineering lifecycle. The fact that the processes are tailorable, and composable means that they can be instantiated as Integrated Master Plans/Integrated Master Schedules (IMP/IMS). For complex system development programs in aerospace and defense, the fact that the processes can be pre-calibrated in terms of cycle-time, costs and required resources can dramatically lower program execution risks.
PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
6
Concluding Remarks It is important to realize that there will always be tradeoffs between the level of tailoring needed and the level of reuse desired as shown in Figure 4.
Figure 4. Tailoring - Reuse Tradeoffs. It is also important to realize that similar tradeoffs exist between Technology Readiness Levels (TRLs) and reuse. Some of the implications of such tradeoffs are: • use standard processes whenever possible to maximize reuse • minimize degree of tailoring to maximize reuse if reuse is key to cost/risk reduction • choose technologies with high TRL to maximize reuse • conform to systems engineering standards (i.e., IEEE-1220, ISO/IEC-15288, EIA-632) When it comes to addressing product families, the greater the similarity in the product families, the higher the likelihood of process reuse. Does this mean that product families need to be always sufficiently similar, to exploit reuse? Not necessarily. It simply means, to the extent possible, without violating a products requirements, systems architects should strive for reuse to lower lifecycle costs. In this regard, competitive organizations need to increasingly identify, acquire and compare processes from across their company. Most organizations are not aware that they continually create process silos throughout their critical business units. The approach presented in this paper focuses on process standardization, reuse and sharing. By adopting the practices presented here, organizations will be able to increase not only their bid competitiveness, but also their profitability, while also achieving risk mitigation on large-scale complex programs.
References DoD Directive 5000.1 and DoD Instruction 5000.2. Software Engineering Institute. Capability Maturity Model Integration, Carnegie Mellon Software Engineering Institute. Madni, A.M. Thriving on Change through Process Support: The Evolution of the ProcessEdge™ Enterprise Suite and TeamEdge™, in International Journal on Information ٠ Knowledge ٠ Systems Management, Special Issue Vol. 2, No. 1, Spring, 2000. Madni, A.M., Madni, C.C., and Lin, W. IDEON™/IPPD: An Ontology for Systems Engineering Process Design and Management, (Invited Paper) Proceedings of the 1998 IEEE International Conference on Systems, Man, and Cybernetics, San Diego, CA, October 11-14, 1998, pp. 2597-2602.
PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
7
Madni, A.M., Lin, W., and Madni, C.C. IDEON™: An Extensible Ontology for Designing, Integrating, and Managing Collaborative Distributed Enterprises in Systems Engineering: The Journal of the International Council on Systems Engineering, Vol. 4, No. 1, 2001.
Biography Larry Earnest, Ph.D. Candidate. Mr. Earnest is a senior systems engineer at Northrop Grumman where he has performed critical systems engineering work on the B2 Stealth Bomber, F/A-18 E/F, Global Hawk, Homeland Security and other complex systems. Larry is currently pursuing his doctorate in Systems Engineering at Stevens Institute of Technology in Hoboken, New Jersey. He earned his Masters of Business Administration from the University of Redlands—California, and a Bachelor of Science in Industrial Technology Electronics, from California State University—Long Beach. His research interests are systems architecting processes and tradeoff analysis. Azad M. Madni, Ph.D. Dr. Azad Madni is the CEO of Intelligent Systems Technology, Inc. His research interests are adaptive architectures, enterprise modeling and simulation, and human performance enhancement. He is the recipient of several prestigious, international and national awards including the 2006 C.V. Ramamoorthy Distinguished Scholar Award, the SBA’s National Tibbetts Award for California, the 2000 Blue Chip Enterprise Award for entrepreneurship, and the Developer of the Year Award from the Technology Council of Southern California in 2000 and 2004. He is an elected Fellow of the IEEE, INCOSE, SDPS, and IETE, and an Associate Fellow of AIAA. He received his Ph.D., M.S., and B.S. in Engineering from UCLA. He is also a graduate of the Executive Institute at Stanford University. He is listed in all the major “Who’s Who.”
PROCEEDINGS CSER 2008, April 4-5, Los Angeles, CA, USA
8
