Value-Based Software Engineering (VBSE) - CiteSeerX

Paper appeared in: Faulk, Harmon, and Raffo, "Value-Based Software Engineering (VBSE): A ValueDriven Approach to Product-Line Engineering", Submitted to the First International Conference on Software Product-Line Engineering, Held in Denver, Colorado, August 28, 2000

Value-Based Software Engineering (VBSE) A Value-Driven Approach to Product-Line Engineering Stuart R. Faulk', Robert R. Harmon", and David M. Raffo" 'University of Oregon & "Portland State University

Abstract:

We consider a set of programs a family when it pays to look at their common aspects before looking at their differences. For commercial software developers the implications are twofold: First, making rational decisions about product-line processes and products requires the ability to answer the question: “Does it pay?” Second, whether or not something pays is ultimately a business (rather than software engineering) question. In short, making sound software engineering decisions requires understanding the business implications of those decisions, and vice versa. This paper describes work in

does describe the motivation for developing programs as families. For developers of commercial software products seeking to apply product-line technology, the implications are twofold: First, the ability to make rational decisions over the course of product-line development depends on being able to answer the question: “Does it pay?” Does it pay to develop software product P as a product line? Does it pay to include feature F in the scope of the product line? Does it pay to automate production of work-product W for the product line? Indeed, does it pay to use product line technology at all in my company? Second, the issue of whether or not something pays is ultimately a business (rather than software engineering) question. To determine whether it pays to develop software product P as a product line, one must estimate how many members of the family will be developed and what the market is for each. To determine if it pays to include feature F in the scope of a product line, one must be able to compare the return on investment from a product that includes F with one that does not. To determine if it pays for a company to adopt product line technology, one must understand how a product-line approach addresses the company’s strategic business goals and be able to determine the expected return on adopting a product-line process. In short, making sound software engineering decisions about product lines requires understanding both the technical and business implications of those decisions. The advantage of a product-line approach is that it takes a strategic view of software development, i.e., addresses the question “what pays?” beyond the scope of a single product development. The consequence is that one cannot make good decisions without understanding the cost/benefit implications of

progress to develop a product-line process model and common value metric that adequately link product value drivers (what it pays) with the software engineering decisions that affect those drivers. In it, we describe a systematic approach to quantifying the return on investment for both product and process improvements based on common software engineering principles and a common value metric, customer value. Key words:

1.

Software product-line, product line economics, value driver, customer value, value-based software engineering

INTRODUCTION We consider a set of programs a family if they have so much in common that it pays to look at their common aspects before looking at the aspects that differentiate them. – David L. Parnas

For all its brevity, Parnas’ quote [Parnas 79] pinpoints the essential question that developers should answer before deploying or applying a product-line approach. While he notes that the definition does not tell us exactly what it pays, it 1

Stuart R Faulk, Robert R Harmon, and David M Raffo those decisions – i.e., how does my decision affect the next product, the ability to produce new products, and so on. Conversely, making sound business decisions related to product lines requires understanding the software engineering implications of those decisions. For example, if product P is developed as a product line, how does that affect our organizational structure and our ability to develop the next product, P++? If I add feature F to my product to make it more competitive, how will that affect the development cost, schedule, design flexibility, etc.? Thus, identifying and quantifying “what it pays” is necessary but not sufficient. The productline methods and processes employed must also support effective communication of this information between the business and technical sides of an organization. In our experience, such effective communication is the exception rather than the rule. Current software development models and prevailing organizations tend to inhibit such communication. In most companies, valuecreating units such as product management, marketing, and development are not only separated organizationally, they are separated by culture, language, and perception of overall goals. Information flow between organizational units is unreliable. These disconnects are reflected in the software development process [Conway 68]. Where a product specification written for a business audience (e.g., the concept document) will address business issues, by the time this is translated into a technical software specification (e.g., software requirements), the business rationale is lost. Similarly, technical decisions can have profound effects on business capabilities (e.g., architectural design decisions) but this is often neither adequately understood nor effectively communicated to the business side of the house.

Where software development processes provide little support for linking business objectives with strategic software design decisions, the result is often a mismatch between technical capabilities and business goals that cannot easily be corrected. One hears the following kinds of complaints as a result: – "We had the best product but by the time it shipped we'd missed the market window." – "Our customers wanted us to add a couple of simple features but found we couldn't do it without rewriting much of the code." – "We were behind schedule and wanted to deliver a scaled down version of the system but found that nothing would work until the system was completed." – “We planned to develop a whole line of products based on our original design but found we had to start over again.” This paper describes ongoing research by the authors to address such issues for commercial software developers, particularly small businesses developing end-user applications (e.g. shrinkwrapped software). Our goal is to provide a process framework that adequately links the delivered value of a software product line with the software engineering decisions that affect that value. The process framework encompasses both adoption and application of product line processes. Developers should be able to apply a common value metric to help make rational decisions about adopting components of product line technology as well as make decisions affecting a particular product line development. The project will develop model products and a case study supporting our educational efforts in product-line engineering (e.g., for the Oregon Master of Software Engineering program [Faulk 00]1).

1

2

See http://www.omse.org/

Value-Based Software Engineering (VBSE)

1.1

3

Approach

in market value analysis with Software Engineering research in process improvement and product lines. Our Customer Value Analysis approach is similar to Quality Function Deployment [Hauser 88] in seeking to reconcile customer needs with product design. It differs in that, first, it seeks to measure more directly and accurately customer’s perception of total delivered value, hence, what the customer will actually pay for a product. Second, it directly measures the difference between an organization’s internal perception of customer value and the customer’s actual perception. This provides a basis for aligning a company’s internal view of delivered value with market realities. Our approach to product-line engineering builds on research by the authors and others on product-line processes. Our model of product line development processes is based on work initially described by Campbell, Faulk, and Weiss ([Campbell 92], [Campbell 94]) and subsequently refined, elaborated, formally modeled and applied by Weiss, Lai, and colleagues ([Weiss 99], [CUA 98]). Using delivered value to guide product line development is similar to, and sympathetic with, DeBaud and Schmid’s [DeBaud 99] approach to scoping product lines. As in [DeBaud 99], we apply a value metric to help make decisions about product-line scope. The VBSE approach differs in the use, derivation, and qualities of the metric. [DeBaud 99] focuses primarily on measuring the relative benefit of including or excluding capabilities from a product line’s scope using a dimensionless measure of benefit. Such a metric does not allow arithmetic operations on the metric, relate directly to conventional measures of return on investment, or support comparisons outside a common quality matrix. In contrast, VBSE seeks to address a variety of decisions beyond product line scoping. This includes making cost/benefit decisions about adopting product line processes, applying product line technologies (e.g., building a compiler for a product line), and developing a particular product line. Thus, the VBSE approach seeks to provide a common framework and metric for making decisions that bring together business issues, process issues, and product issues. To do so, we

Central to our approach is a measure of delivered product value called Customer Value. The Customer Value metric is designed to measure what the customer will actually pay for the total value delivered by a product. We use the Customer Value metric to provide a common standard (“coin of the realm”) for measuring and communicating the relative value (i.e., “what it pays,”) of alternative choices in a software development process. As detailed in the following discussion, Customer Value provides: 1. A common language for communicating about the relationship between business issues and software engineering issues. 2. A basis for evaluating the relative worth of different decisions (e.g., different family architecture designs). 3. A common basis for evaluating decisions about products (e.g., product line scope) and decisions about process (e.g., whether to adopt a particular product line technology). Our long-term objective is to provide a lightweight process framework that unifies strategic business goals, process improvement, and the application of domain engineering to a software product line. Because it focuses on delivered value, we refer to our approach as Value-Based Software Engineering (VBSE). We are currently in the process of developing and evaluating the VBSE framework in collaboration with a small-business software developer, Timberline Software, Inc.2. The remainder of this paper describes the conceptual underpinnings of the VBSE approach and expected results of the work. We give an overview of Customer Value Analysis then discuss how the results can be used to drive both decisions about products in a product-line and decisions about processes in a family of software product-line development processes.

1.2

Related Work

The VBSE approach marries the author’s Business Administration and Marketing research 2

Timberline Software of Beaverton, OR. Timberline develops estimating and accounting software packages for the construction industry. (http://www.timberline.com)

3

Stuart R Faulk, Robert R Harmon, 4 and David M Raffo require a value metric or set of metrics that make sense in all of these contexts and support

2.

comparisons. provides incremental economic benefit. Our assumption is that small businesses often cannot afford to make significant investments in process changes unless they can recover the cost of those investments in the near term: specifically, within the time frame of the current product development. The implication is that we should be able to provide a process migration path with the property that the technologies adopted (e.g., design of a common family architecture) have a positive, near-term return on investment. For these reasons the VBSE approach seeks to develop a common measure of value for both process and product improvements. Similarly, the VBSE framework encompasses a process adoption strategy in addition to a product-line development strategy.

APPLYING PRODUCT-LINE ROCESSES IN SMALL BUSINESSES

Our work focuses on developing lightweight product-line processes that can be applied by small-businesses developing commercial software products. For small software developers, productline software engineering offers both great promise and substantial risk. The market for enduser and shrink-wrapped applications is characterized by intense competition for market share, rapid obsolescence, and constantly increasing demand for improved capabilities. Developing software as product lines potentially offers significant competitive advantage. The reduced development cost, decreased time to market, and rapid product adaptation offered by a mature product-line development process directly address the small-developer’s critical production issues. However, these advantages do not come without risk. Migrating from a more conventional (e.g., “waterfall”) development process to a product-line process requires significant changes in a company’s development processes, work products, and attendant organization [Bass 98]. Such changes can require a substantial portion of a small business’ labor, capital, and calendar resources. Subsequent domain engineering to develop any significant product-line infrastructure (i.e., an application engineering environment) can likewise place significant burdens on available resources. A practical product-line approach should address both the risks of adopting and the risks inherent in applying a product line process. In practice, this means that the developer must be able to assess the cost/benefit of software process changes in addition to software product changes, over the short or long term. Further, we should be able to show a process adoption strategy with the property that each step

3.

THE VBSE CONCEPTUAL STRUCTURES Figure 1 gives a notional layout of the process components addressed in the VBSE approach. The vertical columns correspond to different software development models with each box corresponding to a conceptual component of the development process. The horizontal dimension indicated by the Migration Path arrow is intended to illustrate a process improvement strategy migrating from a product-centered process to a production-centered process (i.e., one centered on developing the means of producing products). The vertical lines are intended to illustrate the flowdown and use of the Customer Value metric to determine what pays at each development stage. The overall layout of Figure 1 embodies two key ideas behind the VBSE approach. The first is that a common value metric can be applied, not only within a development process, but also over an evolving set of processes, to link business and product goals with the software engineering decisions that impact those goals.

4

Value-Based Software Engineering (VBSE)

5

Customer Value Analysis - Identify value drivers - Determine value

Requirements Specification - Expected changes - Fundamental assumptions - Required subsets

M odule/Object Structure - Design for change - Design f o r extension and contraction

Design Review - Analyze ease of change - Analyze subsetability - Analyze cost

Design for Change

Commonalty Analysis - Commonalties - V ariabilities

Domain Qualification - ROI for current process - ROI for P -L process - Economic model

Requirements Specification - Family member requirements

Domain Engineering - Analyze domain - Implement domain

Architectural De sign - Design common family architecture

Application Engineering - Specify application requirements - Produce application - Analyze application

Architectural Analysis - Analyze variation scenarios - Analyze cost

Family -Based Design

P r o d u c t-L ine Engineering

M igration Path

Figure 1. Relating Process and Product Development The second is the principle behind our overall approach to process acquisition and improvement: It pays us to view an organization’s software development processes, concurrently or over time, as a family of processes.

In particular, we are interested in processes that are in some effective sense on a “migration path” to a mature product-line engineering process. From this perspective, VBSE views an organization’s software processes as a family with the following attributes in common: 1. All of the processes use a common set of software engineering principles rooted in the notions of developing programs as families of systems [Parnas 76]. These include the principles of information hiding [Parnas 72], abstraction, and separation of concerns [Dijkstra 72].

2. All of the development paradigms in the family take a strategic view of software development. By this, we mean that the processes examine, and the work products document, goals and issues outside the scope of the system currently being developed. 3

3

5

We call this view of software development strategic software engineering [Faulk 00]. It is distinguished by principles and practices that forgo tactical advantage (e.g. take longer for the current development) to achieve strategic goals (e.g., faster development of subsequent systems). One can justifiably argue that this commonality

Stuart R Faulk, Robert R Harmon, 6 and David M Raffo 3. All of the processes use a common metric, Customer Value, to link business goals with software engineering decisions and evaluate alternatives (i.e., decide what pays). Family members vary in the degree to which each process focuses on, and invests in, work products that address strategic goals (i.e., are not necessary to produce the product at hand). At one end of the spectrum represented in Figure 1, a process categorized as Design for Change focuses on expected changes over a product’s life cycle. It invests in identifying both what is expected to change, and what is not expected to change (fundamental assumptions), and documenting these in the requirements. It subsequently invests in applying abstraction and information hiding to develop a design that easily accommodates the anticipated changes. For example, object-oriented development paradigms are consistent with Design for Change. At the other end of the spectrum, Product-Line processes focus on and invest heavily in strategic goals. In particular, the primary emphasis of a mature product-line process is on developing and maintaining the means of production for members of a software family. While this viewpoint is illustrated in Figure 1 as three distinct classes of processes, the reality more nearly approximates a continuum. Clearly, we can create a host of additional family members by defining processes that incorporate some aspect of a product-line process (e.g., variations where different parts of production are more or less automated). All such processes would qualify as members of our family.

3.1

capabilities and quality are typically value drivers. Cost of production is a value driver from the developer’s point of view. Customer Value Analysis seeks to quantify value drivers that affect a customer’s decision to buy a particular product. It seeks to identify and quantify all of the factors that affect a customer’s buying decision including tangible attributes like features and price as well as intangible ones like prestige of ownership. While other approaches to value analysis have been discussed in the literature (e.g., [Hauser 88] and [DeBaud 99]) we choose to use one based on customer perceptions for the following reasons: Directness: A measure of what the customer will pay for a product, based on the customer’s own perceptions, quantifies the essence of the question “what does it pay?” Understanding of what the customer wants is critical to designing successful software products and hence making sound software development decisions. This view is confirmed by marketing research showing that knowledge of customer-value expectations is critical to the success of software products. In a survey of over 8,000 projects, one research firm found user involvement was the primary determiner of project success [McConnell 96]. Others have found that easy access to customers is a critical success factor [Millington 95]. Conversely, misunderstanding of customer needs is a leading contributor to product failure [Urban 91]. Independence: The customer’s perception of value is independent of prevailing perceptions within the organization. We have pointed out that disconnects between the business functions of an organization and the software development functions can result in software products with properties that are inconsistent with business goals. Surveys by the author have shown that there are also disconnects between the customer’s perception of product value and the perceptions held by different groups within a development organization. In a recent customer-value study of three software firms by the authors, a severe disconnect was found between software developers, marketers and customer service personnel. Although all three groups had at least some contact with customers, the perceptions of their

Customer-Value Analysis (CVA)

A goal of the VBSE process is to provide a common metric for measuring the relative, expected value of product and process decisions. We have chosen an approach to value analysis based on the author’s work on Customer Value Analysis (CVA) [Harmon 97]. Briefly, we use the term value to denote a product’s perceived overall benefit relative to its cost. A value driver is any aspect of the product that contributes to its market value. Product is implicit in the principles listed under point 1; we prefer that the point be explicit.

6

Value-Based Software Engineering (VBSE)

7

company’s ability to create customer value varied widely by function. Software developers, who had the least customer contact, shared marketers optimism about the customer value that was being delivered. In contrast, customer-service personnel, who had to work with unhappy customers every day, took a much dimmer view of their company’s customer-value successes. In all these firms, software developers were convinced they were delivering good value to the customers. Their development process, however, failed to provide methods for acquiring and disseminating information about the customers’ value requirements to all of the organizational entities responsible for a product. Universality: A goal of VBSE is to provide a common basis for communicating among a company’s organizational entities about decisions that impact business or product goals. Since everyone is a customer for some product (including software), everyone in the organization has a basis for understanding issues expressed in terms of customer value. Customer value thus provides a basis for overcoming communication disconnects.

3.2

Map Internal Perceptions •Developers •Marketing •Product Management •Sales •Customer Service •Top Management

Talk with Customers Identify: •Requirements •Reactions to product concept •Competitive options

Reconcile Value Perceptions Multi-functional teams Include top management Figure 2. The CVA Process

The first step in the process is to capture internal perceptions of customer-value requirements that are held by the developers, product management, marketers, sales representatives, customer service representatives, and top management. These perceptions are often the ones that end up being designed into the product. It is important that they are captured first to calibrate the development team’s initial mindset. The second step is to survey customers. Participants are interviewed about their requirements, reactions to the product concept, competitive options, and price, among other issues. The third stage is to convene a team with representatives of each survey group to reconcile internal value perceptions with those of the customers. Since there is often significant divergence between what the design team has conceptualized and what the customers have articulated as their value expectations, this step is critical to aligning the development team’s goal with actual customer values. In our experience, the internal discussion of customer requirements most often focuses on product feature sets, performance specifications, and costs. Often, there is additional discussion concerning whether the product concept would enjoy competitive advantages over similar

The CVA Process

Determining customer-value requirements is a three-step process (Figure 2). It involves direct contact with customers and potential customers that fit relevant market segment profiles for the product in question. Customer-value requirements may be assessed in personal interviews, focus groups or by using customer panels. For our work with Timberline, we are using an internal panel composed of personnel from each relevant organizational area and an external panel composed of both users and buyers of Timberline software.

7

Stuart R Faulk, Robert R Harmon, 8 and David M Raffo products. Rarely do internal discussions focus on what benefit the customer will receive or what factors are the most important in the customer’s purchase decision. Customers on the other hand, tend to not be feature driven in their decision making. Competitors often rapidly “me too” the feature set. This makes product features a poor candidate for differentiating products. Customers generally are able to form perceptions about how a product may benefit them. In addition, customers are particularly good at identifying the factors that are important to them in deciding which product to buy. It is precisely this perspective that should guide a business’software development decisions.

Customers’ value drivers provide a means for retrieving and analyzing data that summarizes a customer’s attitudes about the product or supplier. They provide a platform for product development by facilitating the organization of information about features, advantages, and benefits into customer requirements. They are the emotional antecedents of purchasing behavior. In the following sections, we focus on the implications that understanding customer value driver information and resolving disconnects among key internal groups can have on software product development.

3.3

Customer value driver (CVD) information addresses the customer’s entire experience of acquiring and using a particular product and therefore cuts across many parts of an organization – marketing, advertising, customer service, distribution, and system development. Broad-spectrum business requirements (Figure 3) are derived using Customer Value Assessment. This part of the process identifies customer value drivers and a company’s strategic product goals. The focus in this process step is on identifying product requirements from the CVD information and communicating them to the appropriate areas of the company.

3.4

Customer-Value Drivers

Customer-value drivers denote the decisionrelated attributes that are perceived by the customer to be the most important factors in choosing a product. They are based on the buyer’s beliefs about the product, the company, the buying situation, and the buyer’s motivations. The customer’s perceptions of economic, performance, and supplier value are the embodiment of strongly held beliefs and attitudes that make up a brand’s image and influence the decision to buy. They are built on a foundation of features, advantages and benefits that interact with and influence the customer’s mind-set [Harmon 82].

Customer Value Assessment

Company Strategic Goals

Broad Spectrum Business Requirements System Development Hardware Software Other areas Marketing Customer Service Distribution

Product Strategy Development Economic evaluation and prioritization of Software Requirements

Business Requirements

Domain Engineering -Product line requirements -Value analysis

Application Engineering - Build family instance -Validate

Figure 3. Linking Customer Value Drivers

Customer value information is combined with company strategic intent to provide overall business requirements. These requirements provide input necessary for creating a

development strategy for successive products and product lines.

8

Value-Based Software Engineering (VBSE)

3.5

9

Product Strategy Development

4.

Once a set of software requirements has been identified, a decision is made regarding which requirements will be included in the product. Importance to the customer and contribution to company strategy are weighed against technical feasibility and cost. By understanding what is valuable to the customer, an accurate estimate of (1) the product modification needed (cost) and (2) the potential market acceptance can be assessed. These are then used to refine product requirements and create a product development strategy.

A SLICE OF VBSE

In this section, we illustrate the VBSE approach by considering how the results of Customer Value Analysis can be used to drive software architecture evaluation. In the following discussion, we assume that we are developing a common, reusable architecture for a family of systems as part of a product-line engineering process. We assume a product-line development process of the form described in [Campbell 90] or [Weiss 99]. Figure 4 illustrates where Customer Value Analysis is performed in our product-line process.

Domain Qualification - Scope domain - Economic Analysis - Customer Value Analysis

Commonalty Analysis - Identify commonalties and variabilities - Refine Customer Value Analysis

feedback

Domain Engineering

Domain Implementation - Design common architecture - Assess architectural qualities

Reusable Product Line Architecture

Model Application Key Build Family Instance Product Process

Applicatio

Figure 4. Using Customer Value in Architectural Analysis

9

Application Engineering

Our approach to architectural evaluation is a refinement of the Software Architectural Analysis Method (SAAM) [Bass 98]. Similarly, we drive the analysis of architectural qualities with scenarios. Since we are developing a common architecture for a family of systems, our goal is to design an architecture that encompasses all of the family’s common requirements, but can be easily adapted to produce any member of the family. This means that addressing the variations among family members should require no change, or very little change, to the common architecture. The purpose of our architectural evaluation process is both to assess the quality of a given architectural design relative to our design goals and to quantify that measure of quality so we can meaningfully compare different designs. We do this using the results of Domain Qualification, Commonality Analysis, and Customer Value Analysis as follows: Domain Qualification: Domain Qualification activities include 1. Scope domain: Creating a preliminary definition of the family in terms of commonalties and variabilities. 2. Economic analysis: Building an economic model of the product’s cost/return using first the company’s current software development process and then a product-line process. These models are used to determine the expected return from adopting a product-line approach. 3. Customer value analysis: Customer value analysis is first performed as part Domain Qualification. We apply CVA to establish a value for each of the possible variations in the potential scope of the family.4 The CVA results can be used in two ways. First, they can be used to help make decisions about the domain scope. As in [DeBaud 99], we can use the results of value analysis to calculate the expected return on including or excluding particular variations or sets of variations from the scope of the product line. These results can, in turn, be used to help build an overall model of the 4

expected return from developing the software as a product line. Second, we can use the results of CVA to help determine the costs and benefits of keeping the current architecture. In particular, we construct change scenarios based on how the product is expected to evolve to meet customer needs. CVA results are used both to identify which changes are most important (valuable) and to quantify the expected return on making the change. We play the change scenarios against the current architecture to determine the expected cost of evolving the system without using a product line approach. We can then compare the costs and benefits of each approach based on the value of the product changes the customer wants and the costs of making such changes under each development paradigm. Commonality Analysis: The goal of Commonality Analysis is to identify and document the commonalties and variabilities characterizing our software family. The second phase of our CVA is conducted as part of commonality analysis. In the second CVA phase, we go back to our customer panel and again develop value metrics, this time based on the more detailed definition of expected variations. This aligns the customer value data with the family requirements that will drive the Domain Engineering phase. Domain Implementation: For this example, we assume a compositional approach to Domain Implementation in which we develop a common, reusable architecture for the product line. If we wish to generate members of the family, we can do so by using adaptable, parameterized modules (e.g., as in the CelsiusTech architectural case study in [Bass 99]). As part of the Domain Implementation activity, we must evaluate our architecture against its quality requirements. In particular, we desire a detailed, quantitative evaluation of how well the architecture instantiates the commonalities and accommodates the variabilities that characterize our product family. We develop a detailed and quantitative analysis by 1) creating scenarios based on the

As discussed in DeBaud and Schmid [DeBaud 99], there are typically dependencies among the variabilities that can make the value function quite complex to calculate.

10

Value-Based Software Engineering (VBSE)

11

results of Commonality Analysis and 2) evaluating them based on the results of Customer Value Analysis. We create one or more scenarios for each commonality and each variability. We can adjust the cost and effort of the evaluation by using the value metrics to order the possible variabilities according to relative value and creating scenarios only for those variations above some threshold value. We play the scenarios against our architecture using qualified personnel to determine whether and to what extent the candidate architecture must change to accommodate each scenario. Where accommodating a scenario would require a change in the architecture, we estimate the expected cost to make that change. In general, scenarios developed from the commonalties should result in no architectural changes. Scenarios developed from variabilities within the product line scope should result in changes that can be accommodated by our generation mechanism (e.g., by parameterizing a module or substituting one module for another). Overall evaluation of the architecture is carried out using customer value data to assign weights to scenarios and scenario interactions. We can use this weighting both to evaluate one architectural design against another and (since the weights are based on expected return) to estimate the expected return on investment from deploying the architecture as part of an application engineering environment. Because we have used a measure of delivered value, these measures are detailed and correlate directly to the business goals associated with our product line.

5.

by the degree of strategic reuse. We apply this notion to 1. Define a migration path from processes with little strategic reuse to processes for which the primary focus is strategic reuse, and 2. Provide a basis for evaluating the costs and benefits of changing the development process employed from one member of the process family to another. In fact, just this approach was used in the economic modeling activity described in the previous section. There, we used the results of CVA to determine the return on investment if we develop our software product with domain engineering and compare it to the return on investment if we develop it without domain engineering. For evaluating the process change, we build off the process change impact analysis done by Raffo [Raffo 96, Raffo 99, and Raffo 00]. In this work, Raffo et al. used software process simulation models to quantitatively evaluate the impact of process improvements in terms of cost, quality, and schedule. We can apply similar analyses to scoped process changes in much the same manner. For example, we can ask what the expected return on investment will be for developing only a common architectural design, creating reusable code components, or developing a common set of reusable documentation. As part of the VBSE work, we plan to extend this approach to evaluate the cost/benefit of business-wide process change. We view the software development process as a value chain. In general, a value chain is any sequence of development activities that adds value to a product. In value-based business models, value chains are used to assess what each activity in the chain contributes to the total value of a product. In modeling a software development process as a value chain, we must estimate the value of different parts of the process that add to the customer value of the product (i.e., quantify how each process step affects the product’s value drivers). This problem remains an open area of research.

EVALUATING PROCESS CHANGE

The VBSE approach makes little conceptual distinction between evaluating product changes and evaluating process changes. As discussed, we view all the processes of interest as a family of processes, related by common principles, varying

11

Stuart R Faulk, Robert R Harmon, 12 and David M Raffo By extending Raffo’s approach, we hope to link process performance measures with customer value in evaluating both process changes and product changes.As a result, this approach could be used to define a process adoption strategy with an acceptable level of risk vs. reward. A goal of our validation activity is to see if we can define process adoption strategies for product line engineering with the property that each process change provides a positive return on investment on the first product developed.

6.

– Guide decisions about making process changes, particularly adopting product-line technology. Because this paper describes work in progress, much remains to be done to refine or validate the approach. At the time of this writing, we have finished an initial definition of the VBSE process and are beginning to apply it in collaboration with our colleagues at Timberline Software. The approach will be applied to a new software product line being developed at Timberline. As part of our work with Timberline we will develop a process model, process handbooks, and model products for each development phase. We will also develop metrics for analyzing and using customer value to drive software engineering decisions.

CONCLUSIONS

In adopting or applying product-line methods, developers are faced with the question, “Does it pay?” In this paper, we have outlined a systematic approach to quantifying the return on decisions about both product and process changes based on a common metric, Customer Value. We argue that the Customer Value metric is appropriate because it: – Quantifies decisions in terms of what the customer will actually pay for a product, thus measuring value as directly as possible although this is a subjective measurement (or just delete everything after “product”. – Helps overcome communication barriers by providing a common basis for evaluating both business and software engineering issues that can be understood by people in every part of an organization. The VBSE approach uses customer value to evaluate both process and product decisions. It provides a basis for linking business goals to the software engineering decisions that impact those. Customer value provides a common metric that can be used to: – Guide the software engineer in understanding the business consequences of his or her decisions and evaluating the costs and benefits of different alternatives (marginally true). – Communicate the effects of software engineering decisions to other organizational entities and to customers.

REFERENCES [Bass 72] F. M Bass and W. Talarzyk, , “Attitude Model for the Study of Brand Preference,” Journal of Marketing Research, 9 February, 1972, 93-9,. [Bass 98] L. Bass, P. Clements, and R. Kazman, Software Architecture in Practice, Addison Wesley, 1998. [Belk 74] R. Belk, “An Exploratory Assessment of Situational Effects in Buyer Behavior,” Journal of Marketing Research, 11 May, 1974, 156-162. [Belk 75] R. Belk, “Situational Variables and Consumer Behavior,” Journal of Consumer Research, 2(3), 1975, 157-64. [Campbell 90] G.R. Campbell, Jr., S. Faulk, and D. Weiss, Introduction to Synthesis, INTRO_SYNTHESIS_PROCESS-90019-N, Software Productivity Consortium, Herndon, VA, 1990. [Campbell 94] G.R. Campbell, Jr., J. O’Connor, C. Mansour, and J. Turner-Harris, “Reuse in Command and Control Systems, IEEE Software, September, 1994. [Conway 68] M. Conway, “How Do Committees Invent?,” Datamation, April, 1968, 37-45. [Cuka 98] D. Cuka and D. M. Weiss, “Engineering Domains: Executable Commands as an Example,” Proceedings of the 5th International Conference on Software Reuse, Victoria, Canada, June 1-5 ,1998, 26-34. [DeBaud 99] J. DeBaud and K. Schmid, “A Systematic Approach to Derive the Scope of Product Lines,” Proceedings: 21st International Conference on Software Engineering (ICSE 99), Los Angeles, CA, June 1999. [Dijkstra 72] E.W. Dijkstra, “Notes on Structured Programming,” In Structured Programming, O.J. Dahl, E.

12

Value-Based Software Engineering (VBSE)

13 [Parnas 76] D. L Parnas, “On the Design and Development of Program Families,” IEEE Transactions on Software Engineering, v. SE-2, No. 1, March, 1976, 1-9. [Parnas 79] D. L. Parnas, “Designing Software for Ease of Extension and Contractions,” IEEE Transactions on Software Engineering, v. SE-5, March, 1979, 128-138.. [Raffo 96] D. M. Raffo, “Modeling Software Processes Quantitatively and Assessing the Impact of Potential Process Changes on Process Performance”, Ph.D. Dissertation, Graduate School of Industrial Administration, Carnegie Mellon University, 1996. UMI #9622438. [Raffo 99] D. M. Raffo, M. Vandeville, and Martin, “Software Process Simulation to Achieve Higher CMM Levels,” Journal of Systems and Software, Vol. 46, No. 2/3 (15 April 1999). [Raffo 00] D. M. Raffo and M. I. Kellner, “Empirical Analysis in Software Process Simulation Modeling” Journal of Systems and Software, (Forthcoming 2000) [Urban 91] G. L. Urban and S. H. Star, Advanced Marketing Strategy: Phenomena, Analysis, and Decisions, Englewood Cliffs, NJ: Prentice Hall, 1991, 562 pp. [Weiss 99] D.M. Weiss and C.T.R. Lai, Software Product Line Engineering: A FamilyBased Software Development Process, Addison-Wesley, 1999.

W. Dijkstra, and C.A. Hoare, eds., Academic Press, London, 1972. [Faulk 00] S.R. Faulk, “Achieving Industrial Relevance with Academic Excellence: Lessons from the Oregon Master of Software Engineering,” Proceeding, 22nd International Conference on Software Engineering, Limerick, Ireland, June 2000. [Harmon 82] R.R. Harmon and K. A. Coney, “The Persuasive Effects of Source Credibility in Buy and Lease Situations,” Journal of Marketing Research, May, 1982: 255-60. [Harmon 97] R.R. Harmon and G. Laird, “Linking Marketing Strategy to Customer Value: Implications for Technology Marketers,” Proceedings of the Portland International Conference on Management of Engineering and Technology, July, 1997, 896-900. [Hauser 88] J. R Hauser and Don Clausing (1988), “The House of Quality,” Harvard Business Review, May-June, 63-73. [McConnell 96] McConnell, S. (1996), Rapid Development: Taming Wild Software Schedules, Redmond, WA: Microsoft Press, 647 pp. [Millington 95] D. Millington, and J. Stapleton, “Developing an R&D Standard,” IEEE Software, September, 1995 5455. [Parnas 72] D.L. Parnas, “On the Criteria to Be Used in Decomposing a System into Modules,” Comm. ACM, 15, Dec. 1972, 1053-1058.

13