Software Product Line Engineering for Consumer ...

Software Product Line Engineering for Consumer Electronics Keeping up with the speed of innovation Proefschrift ter verkrijging de graad van doctor aan de Rijksuniversiteit Groningen op gezag van de rector magnificus prof. dr. E. Sterken en volgens besluit van het College voor Promoties. De openbare verdediging zal plaatsvinden op vrijdag 9 oktober 2015 om 12.45 uur door Johan Herman Hartmann geboren op 28 september 1966 te Onstwedde

Promotor Prof. dr. ir. J. Bosch

Beoordelingscommissie Prof. dr. ir. P. Avgeriou Prof. dr. ir. I. Crnkovic Prof. dr. J.C. van Vliet

ISBN: 978-90-367-7991-3

Software Product Line Engineering for Consumer Electronics Keeping up with the speed of innovation

Herman Hartmann

Abstract During the last decade consumer electronics products have changed radically. Traditionally, these products were used for a few dedicated tasks, and were implemented through hardware. Nowadays, these products are used for a variety of tasks and most of the functionality is implemented through software. Furthermore there is an increasing amount of variants needed to serve the market and a continuous pressure on development cost, quality, and time-to-market. These trends have caused a significant impact on the way that software product lines for consumer electronics are being developed. This thesis starts in part I with an exploration of the problem space: namely the challenges that are caused by the changing industry structures. This analysis shows that due to the increasing amount of software and high speed of innovation it is unfeasible for an individual company to develop the software on its own. As a consequence, consumer electronics products are built using components from a large variety of specialized firms. A model of five industry structure types is introduced that describes the transition and the consequences for software architectures. It is shown that software supply chain is the dominant industry structure for developing the software that is embedded in these products. This is because the modularization of the architecture is limited and because resource constrained devices require variants of software that are optimized for different hardware configurations. To support applications from third parties, many consumer electronics firms offered a stable interface towards the 3rd party applications. In this way these firms transitioned from software product line engineering in an intra-organizational context to ecosystems for these 3rd party applications. Three types of ecosystems are identified that are currently used: vertically integrated hardware/software platforms, closed source software platforms and open source software platforms. A first step towards a decision support method is introduced that can determine which type of ecosystem is most suitable for a specific type of consumer electronics product from a software engineering perspective. Part II of this thesis presents solutions to the challenges for variability management that arise when the software is developed through a supply chain. When a supplier delivers products to multiple customers, this may lead to multiple product lines. In this thesis a modelling approach is introduced that captures the context as a separate tree of a feature model, which makes it possible to model multiple product lines in software supply chains. A company that is in the middle of a supply chain, has to integrate components from suppliers and offer (partly configured) products to its customers. To satisfy the variability requirements by each customer it may be necessary to use components from different suppliers, partly offering the same feature set, thus leading to overlapping feature models. This thesis introduces a supplier independent feature model. In this concept, dependency relations between the supplier independent feature model and the feature models of the individual suppliers are used to capture the variability of the combined product line. The use

v

of alternative components from suppliers partly may also lead to a product line with components which are using different mechanisms for interfacing, binding and variability. A model-driven approach is introduced for automating the integration between components. This approach reduces the development costs and time-to-market. As a result of the growing number of variation points, feature models often become too complex to deal with. This thesis evaluates a textual variability language which shows that the readability of models using a textual language is easier than with graphical languages, especially when many constraints between features are used, which is common for feature models in software supply chains. Part III of this thesis presents solutions to the challenge of ensuring a sufficient quality with a high pressure on time-to-market, as identified in the part I. This last part focusses on testing, the last phase in a development process. The consequence for testing is that due to the increasing amount of faults, of which many are caused by a mismatch between components from different suppliers, and the limited time for testing, it is not feasible to remove all faults before market introduction. In this thesis a statistical model is introduced that determines the benefits of using operation profiles, which is a method to increase the test efficiency based on the quantitative usage by the end users. The analysis shows that using operational profiles improves the test efficiency, however not when a high reliability is required. The analysis of risk based testing, which is a method to focus testing on those areas that have the biggest impact on end user and business, shows that when the usage frequency is treated as a separate dimension of the risk matrix, the efficiency can be further improved. To reduce the test effort caused by the high amount of variability, this thesis introduces risk based testing for software product line engineering, which extends risk based for single system engineering with a dimension that captures the percentage of product variants that use a particular development artefact. This method provides the means to guide the test effort for different test levels and set priorities during domain and application engineering. Because operational profiles cannot be directly applied to testing of highly innovative consumer electronics products, as it focuses primarily on technical reliability risks, this thesis proposes an enhanced framework to analyze unexpected user-product interaction. In this framework, product, user as well as environmental conditions are used to analyze userproduct interaction which will help reducing non-technical failures. This thesis has the following key contributions:  A classification of different industry structures and ecosystems for consumer electronics is introduced along with their architectural challenges.  Solutions for managing the variability in software supply chains have been created and for managing large and complex feature models.  Methods are introduced to improve the test efficiency for software product lines and for highly innovative consumer electronics products.

vi

Acknowledgements Doing a PhD is a journey that one cannot successfully bring to an end without the help of many. This is especially the case in which a PhD is pursued during a regular job. The list of persons that have helped me during the PhD is long, whether it is in the research itself, creating the right environment or giving personal support. Most likely I will have forgotten to mention many. First of all, I would like to thank Jan Bosch. Jan was an inspiring promotor and he understands what it is like to do a PhD while working in the industry. His encouragements and optimism were essential to keep me motivated. I learned from Jan how to write papers that were business and strategy oriented. With a few questions he helped me to focus and structure my papers. Some of those questions and remarks, I always keep in mind when writing a paper. Secondly, I would like to thank Tim Trew. Tim actively supported my ambition to do a PhD during my time at NXP Research and he taught me how to be thorough in writing scientific papers. His reviewing capacity is unparalleled and every time I review a paper, I try to do this as precise as Tim does. My ambition to do research and a PhD started when I was working at Philips CFT. During that period I worked closely together with the Technical University of Eindhoven and I was collaborating on research projects and making my first steps in doing research and writing my own papers. In particular, I want to thank: Elke den Ouden, Vincent Ronteltap, Aarnout Brombacher, Lu Yuan, Guillaume Stollman, Peter Sonnemans, and Wim Geudens. I joined NXP Research and pursuing a PhD was an important reason for that. Ad ten Berg was a very supportive department leader, despite the general business trends that were not in favor of scientific research. I also want to thank Clara Otero-Perez who continued this support as a department leader. Doing research, and especially writing papers, is most of the time a solitary activity. When you can write papers together with others this makes the work much more pleasurable, as well as allows you to exchange experience. Therefore, I want to thank the co-authors of my papers, i.e. the persons that have not been mentioned above: Aart Matsinger, Arnaud Hubaux, Aylin Koca, Frank van der Linden, Jarmila Bökkerink, Julia Rubin, Mila Keren, Patrick Heymans, Quentin Boucher, Raphaël Michel and Tali Yatzkar-Haham. Case studies are an essential pre-requisite for research in the engineering sciences, which often requires the help of persons that are taking part in these case studies, provide the required resources or give advice. Therefore, I would like to thank the following persons: Alan Hartman, Arjan den Hartog, Asaf Shaar, Bart Caerts, Gaëtan Delannay, Hans den Boer, Itay Maman, Jacques Flamand, Jean-Marie Trager, Jack Goossen, Jos Hegge, Michiel Mennen, Ofir Brukner, Peter van Loon, Rob van Ommering, Rob van Twist, Rob Wieringa,

vii

Shiri Kremer-Davidson, Somasundaram Venkatachalam, Uri Avraham and Wouter Batelaan. I would like to thank the reading committee for providing me useful feedback on my thesis: Paris Avgeriou, Ivica Crnkovic and Hans van Vliet. I also want to thank the many anonymous reviewers of the journals and conferences. Often I received very useful feedback which helped me to improve my publications. I would like to thank the many colleagues at Philips CFT, NXP Research and NXP R&D IT, who provided me a pleasant daily working environment without which it wouldn’t be possible to work many evenings and weekends on writing the papers. Some persons were specifically of importance for my PhD and during my career: Jos Gerards, Paul Meeuwissen and Thérèse Schoenmakers. Over the years, I have got acquainted with many fine people, during my career or during my personal life, some of those have become good friends. I can recall many conversations on my research. While not all of my friends were able to understand the details of my research, the fact that I was able to share my ideas, and explain the objectives, helped me to formulate the contributions more clearly and encouraged me to continue with the research. For these conversations and for being good friends or fine colleagues, I want to thank: Abhijit Kumar Deb, Caro Afman, Carolyne Trew, Liesbeth Steffens, Manvi Agarwal, Peter Meyer, Peter Reuvekamp, Razvan Dinu, Renee van Liempd and Yanja Dajsuren. I have always had the luck of being surrounded by a heartwarming family. Without their support, doing a PhD wouldn’t have been possible. I therefore want to thank: Martha, Toolsi, Bida, Marie, Maureen, Erik, Ben, Liesbeth, Asha, Johan, Annie, Surita, Lauran Heleen, Remke, Soeris, Hans, Monique. Every time they asked me about my work and research, I regained new energy. Finally I want to thank Amita, my wife, girlfriend and mate. Amita supported me all the way. She knew that doing a PhD was my longtime ambition. Many evenings and weekends I couldn’t be the husband that I wanted to be, but she understood why I was doing this and accepted it. While doing the PhD and not being able to spend much time with her, I again realized what the most important thing in life is.

viii

Contents Abstract ............................................................................................................................... v Acknowledgements ........................................................................................................... vii List of Figures ..................................................................................................................xiii List of Tables .................................................................................................................... xiv Chapter 1. Introduction .................................................................................................. 1 1.1

Market Trends ..................................................................................................... 1

1.2

Problem Statement .............................................................................................. 3

1.3

Research Objectives ............................................................................................ 3

1.4

Research Methodology ....................................................................................... 4

1.5

Research Questions ............................................................................................. 6

1.6

Thesis Outline and Article Overview ................................................................ 10

1.7

Applicability of the Research Results ............................................................... 14

1.8

Detailed Overview of Research Processes and Methods .................................. 14

1.9

Related Publications .......................................................................................... 18

1.10

Contributions to the Articles ............................................................................. 19

Part I:

Industry Structures for Software Development ........................................... 21

Chapter 2. The Changing Industry Structure of Software Development for Consumer Electronics ...................................................................................................... 23 2.1

Introduction ....................................................................................................... 23

2.2

Background ....................................................................................................... 25

2.3 Case Studies: Transition in the Development of Digital Televisions and Mobile Phones .......................................................................................................................... 29 2.4 Model of Industry Structures for Software Development in Consumer Electronics...................................................................................................................... 31 2.5

Transitions in the Development of Digital Televisions ..................................... 35

2.6

Transition in the Mobile Phone Industry .......................................................... 44

2.7

Related Work .................................................................................................... 50

ix

2.8

Conclusions and Future Research ..................................................................... 51

Chapter 3. Consumer Electronics Software Ecosystems ............................................ 55 3.1

Introduction ....................................................................................................... 55

3.2

Background on Consumer Electronics Products ............................................... 57

3.3

A Classification of Ecosystem Types ............................................................... 58

3.4

Towards a Multi-Criteria Decision Support Method ........................................ 60

3.5

Case Studies ...................................................................................................... 68

3.6

Comparison with Related Art ........................................................................... 78

3.7

Conclusions and Further Research .................................................................... 78

Part II:

Variability Management in Software Supply Chains .................................. 81

Chapter 4. Using Context Variability to Model Multiple Product Lines .................. 83 4.1

Introduction ....................................................................................................... 83

4.2

Related work ..................................................................................................... 85

4.3

Feature Models for Multiple Product Lines ...................................................... 85

4.4

Context Variability in Software Supply Chains ................................................ 90

4.5

Merging MPL-Feature models .......................................................................... 93

4.6

Comparison with Related Work ........................................................................ 96

4.7

Preliminary experimental results....................................................................... 97

4.8

Discussion and Further Research ...................................................................... 98

4.9

Conclusions ....................................................................................................... 98

Chapter 5. Supplier Independent Feature Modelling ................................................. 99

x

5.1

Introduction ....................................................................................................... 99

5.2

Background ..................................................................................................... 101

5.3

Integrating Feature Models ............................................................................. 101

5.4

Supplier Independent Feature modelling ........................................................ 105

5.5

ZigBee Case Study .......................................................................................... 110

5.6

Tool support .................................................................................................... 112

5.7

Discussion and Alternatives ............................................................................ 113

5.8

Related Work .................................................................................................. 114

5.9

Conclusions ..................................................................................................... 114

Chapter 6. Using MDA for Integration of Heterogeneous Components in Software Supply Chains ................................................................................................................. 117 6.1

Introduction ..................................................................................................... 117

6.2

Problem Description ....................................................................................... 119

6.3

ZigBee Case Study .......................................................................................... 123

6.4

MDA for the Integration of Heterogeneous Components ............................... 127

6.5

Development Roles ......................................................................................... 137

6.6

Discussion and Further Research .................................................................... 138

6.7

Evaluation ....................................................................................................... 141

6.8

Comparison with Related Art ......................................................................... 143

6.9

Conclusions ..................................................................................................... 144

Chapter 7. Evaluating a Textual Feature Modelling Language .............................. 145 7.1

Introduction ..................................................................................................... 145

7.2

Related Work .................................................................................................. 147

7.3

TVL................................................................................................................. 149

7.4

Research Method ............................................................................................ 150

7.5

Results ............................................................................................................. 156

7.6

Findings .......................................................................................................... 159

7.7

Threats to Validity .......................................................................................... 163

7.8

Conclusion ...................................................................................................... 164

Part III: Software Testing............................................................................................ 165 Chapter 8. A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing .................................................................................................................. 167 8.1

Introduction ..................................................................................................... 167

8.2

Background ..................................................................................................... 169

8.3

A Statistical Model to Estimate the Improvement on Test Efficiency ............ 172

8.4

Simulations of the Test Efficiency when using Operational Profiles .............. 174

8.5

Case Study: Philips HealthCare ...................................................................... 177

8.6

A Statistical Analysis of Risk Based Testing. ................................................. 180

8.7

Discussion and Further Research .................................................................... 184

xi

8.8


8.9

Conclusions ..................................................................................................... 186

Chapter 9. Risk Based Testing for Software Product Line Engineering ................ 187 9.1

Introduction ..................................................................................................... 187

9.2

Background ..................................................................................................... 189

9.3

Applying Risk Based Testing to Product Line Engineering ........................... 192

9.4

Case Study: Philips Healthcare ....................................................................... 195

9.5

Tool Support ................................................................................................... 198

9.6


9.7

Conclusions and Further Research .................................................................. 203

9.8

Appendix: Consistency Rules of Quantified Feature Models ......................... 204

Chapter 10. Interactions

Towards a More Systematic Analysis of Uncertain User-Product ................................................................................................................ 207

10.1

Introduction ..................................................................................................... 207

10.2

Issues when Applying Operational Profiles for CE Products ......................... 209

10.3

Modelling Unexpected User-Product Interaction: an Initial Step ................... 210

10.4

First implementation ....................................................................................... 213

10.5

Conclusion ...................................................................................................... 219

Chapter 11.

Conclusions ........................................................................................... 221

11.1

Research Questions and Answers ................................................................... 221

11.2

Threats to Validity .......................................................................................... 232

11.3

Generalization ................................................................................................. 233

11.4

Key Contributions of this Thesis..................................................................... 235

11.5

Further Research ............................................................................................. 237

Bibliography.................................................................................................................... 241 Samenvatting .................................................................................................................. 261 Curriculum Vitae ........................................................................................................... 263

xii

List of Figures Figure 1 Research process .................................................................................................... 5 Figure 2 Article overview and their main relations ............................................................ 11 Figure 3 Changing structure in the computer industry ....................................................... 24 Figure 4 Example of a software supply chain..................................................................... 26 Figure 5 Model of industry structure types, visualized ...................................................... 32 Figure 6 Forces of moving from one industry type to another ........................................... 34 Figure 7 Reference software architecture of a TV developed in a single company ............ 37 Figure 8 Reference software architecture of a TV developed in a supply chain ................ 39 Figure 9 Middleware and platform with glue layer ............................................................ 42 Figure 10 Split of glue software and platform .................................................................... 42 Figure 11 Anticipated mobile industry transition ............................................................... 46 Figure 12 Software architecture EMP ................................................................................ 47 Figure 13 System architecture of consumer electronics products that support 3rd party applications ......................................................................................................................... 58 Figure 14 Classification of ecosystem types with examples from smart phones ................ 59 Figure 15 Multi-Criteria Decision Support Method ........................................................... 61 Figure 16 Multiple product line feature diagram ................................................................ 86 Figure 17 Feature diagram car infotainment system .......................................................... 87 Figure 18 Context variability diagram of the car infotainment system .............................. 88 Figure 19 Multiple product line-feature diagram of the car infotainment system .............. 89 Figure 20 Generating specialized feature models ............................................................... 91 Figure 21 MPL-feature diagram for CarA budget .............................................................. 92 Figure 22 MPL-feature diagram of CarA with merged infotainment system ..................... 95 Figure 23 Car-infotainment suppliers ............................................................................... 100 Figure 24 Integrating functional areas .............................................................................. 102 Figure 25 Feature models from the suppliers ................................................................... 102 Figure 26 Alternative subtrees .......................................................................................... 103 Figure 27 Merged models with dependencies .................................................................. 104 Figure 28 Composed SIFM and SSFMs (CSFM) ............................................................. 106 Figure 29 ZigBee reference architecture. ......................................................................... 110 Figure 30 Infotainment high level component diagram. ................................................... 120 Figure 31 Homogeneous and heterogeneous component integration. .............................. 120 Figure 32 Example feature diagram with links to development artifacts, together with its configuration space. .......................................................................................................... 121 Figure 33 Layers in the ZigBee protocol with pofiles ...................................................... 124 Figure 34 Integration of Network and MAC layers .......................................................... 126 Figure 35 Model transformations for integration and configuration of components ........ 128

xiii

Figure 36 Process overview .............................................................................................. 129 Figure 37 SVP's of the case study .................................................................................... 130 Figure 38 Conceptual model of entities used ................................................................... 130 Figure 39 Meta-model for a glue component ................................................................... 133 Figure 40 Wizards for the interface map configuration process ....................................... 134 Figure 41 Snippet library .................................................................................................. 135 Figure 42 Example of the generated glue code for the ZigBee case ................................. 136 Figure 43 Component composition and configuration with reachability analysis ............ 139 Figure 44 Feature diagram of the PhoneMeeting voting component................................ 146 Figure 45 Interview protocol ............................................................................................ 155 Figure 46 Fault tree of software failures........................................................................... 169 Figure 47 Example of a risk matrix .................................................................................. 172 Figure 48 Risk matrix with 5 functions ............................................................................ 182 Figure 49 Quantified feature model of a digital television ............................................... 188 Figure 50 Example of a risk matrix .................................................................................. 191 Figure 51 Feature model with linked components ............................................................ 192 Figure 52 Risk matrix for reusable components ............................................................... 194 Figure 53 Philips Medical Workspot platform development ............................................ 196 Figure 54 Risk matrix for Philips Medical Workspot platform ........................................ 197 Figure 55 Screenshot of a feature a family model with quantification and consistency checking ........................................................................................................................... 200 Figure 56 The enhanced user-product interaction model ................................................. 212 Figure 57 ANOM results of the test data. ......................................................................... 217

List of Tables Table 1 Applicability of the research results ...................................................................... 14 Table 2 Overview of types of ecosystems and their strengths and challenges ................... 67 Table 3 Example of a decision support matrix ................................................................... 68 Table 4 Decision support matrix for gaming consoles ....................................................... 69 Table 5 Decision support matrix for digital photo cameras ................................................ 71 Table 6 Decision support matrix for digital televisions and set-top boxes ......................... 72 Table 7 Decision support matrix for smart watches ........................................................... 74 Table 8 Decision support matrix for smart phones ............................................................. 75 Table 9 Decision support matrix for tablets ....................................................................... 76 Table 10 Decision support matrix for personal computers ................................................. 77

xiv

Table 11 ZigBee suppliers ................................................................................................ 111 Table 12 Feature set of selected suppliers ........................................................................ 125 Table 13 Differences between the technologies used in the ZigBee case study ............... 126 Table 14 Quality criteria for the evaluation of TVL ......................................................... 151 Table 15 Profiles of the 5 participants .............................................................................. 153 Table 16 Results of the evaluation of TVL ...................................................................... 157 Table 17 Definitions in software reliability ...................................................................... 170 Table 18 Fictitious operational profile ............................................................................. 174 Table 19 Comparison of MTBF with uniform versus OP distribution ............................. 175 Table 20 Comparison with large amount of test cases ..................................................... 176 Table 21 Operational profile for the orthopedists ............................................................. 179 Table 22 Simulations of allocating test cases to different medium type risks .................. 183 Table 23 Comparison of risk analysis processes .............................................................. 195 Table 24 Collecting information related to user-product interaction ................................ 214 Table 25 Data collected from the consumer test. .............................................................. 215 Table 26 Results of ANOVA of the test data. .................................................................. 217

xv

xvi

Chapter 1. 1.1

Introduction

Market Trends

Since the end of the previous century, consumer electronics products have changed radically. Traditionally, consumer electronics products were stand-alone products and were used for a few dedicated tasks, such as watching televisions, listening to music or were used to watch a video. Nowadays, consumer electronics products are connected devices and are used for a variety of tasks, such as communicating with friends using a mobile phone, watching online video clips or used as a navigation device for getting driving directions. A large part of today’s functionality is implementation through software, whereas previously it was implemented though hardware. The consequence is that the software development effort now constitutes a significant, and in many cases largest, proportion of the total development effort. Therefore nowadays the success of a consumer electronics company is highly determined whether it can successfully develop the required software. Since the end of the previous century, the following major trends exist for the development of software for consumer electronics products: An increasing amount and complexity of software: The size of software has grown rapidly and has roughly followed the law of Moore [Moore 1965], i.e. the size of software is doubled every 2 years [Rooijmans 1996, Genugten 2007]. These developments have been made possible because Integrated Circuits (ICs) are available at low costs and because the performance of these ICs have grown rapidly over the last decades. Together with the size also the complexity of the software has grown. This relates to the complexity of developing software in large teams, possibly spread over the world [Jacobs 2007], as well as the complexity of the software architectures [Siewiorek 2004]. An increasing amount of variability: A wide range of variants are needed to serve the needs of different market segments, different geographical regions and different customer’s needs [Ommering 2004]. As an example look at the variety of digital cameras that are available in the market. Manufacturers offer a wide range of cameras with different lenses, size of screens, amount of pixels. A wide range of built-in software features are available in these digital cameras.. for instance to support the photographer in taking different types of pictures, such as macro, portrait or pictures at candle light and to add special effects such as using “classic” sepia colors, or to create 3 dimensional pictures. Another example is that Digital Televisions have to support different interfaces such as different standards for connecting video, e.g. Scart, DVI, HDMI, S-video, USB and have to support different encoding standards such as MPEG, DivX, H.264.

1

Chapter 1 In order to develop the required variability more efficiently and deliver products to market more rapidly, many companies have adopted the concepts of product line engineering [Pohl 2005]. This approach allows the variants of the products to be created efficiently through a systematic reuse of the development artifacts. Product line engineering has been successfully applied in the development of software for consumer electronics in many different firms [PHOF 2014]. The use of third party components Until the early nineties, companies like Sony, Philips and General Electric developed and manufactured the entire product, but nowadays large parts of a product is developed by specialized firms [Davenport 2005]. This trend is a result of the requirements described above: The first reason is that, due the huge amount of software, it is not possible for a single firm to employ sufficient software developers anymore [Raskin 2005]. Secondly for a single firm it is too difficult to amortize the investments, while a specialized firm could develop this software and sell this to a large number of firms, thereby creating a larger economy of scale [Porter 1985]. The third reason is that many products are now a combination of functionality that originates from other types of products. For instance mobile phones now have built in cameras and MP3 players, digital cameras are now equipped with a GPS. For a firm that wants to add a new feature to a product, it would take too much through-put-time to develop this software in house. The use of 3rd party software components has been facilitated by the use of software components. Since software components can be developed independently, the development can be subcontracted to third parties or can be purchased as existing, commercial of the shelf components [Wallnau 2002, Greenfield 2004]. An increasing speed of innovation and pressure on time-to-market: New functionality is introduced rapidly and older products become commercially obsolete within a few years [Minderhoud 2005]. For instance playing Vinyl records was the most used way to listen to music between 1930 and 1990. Vinyl records were largely replaced by CDs between 1990 and 2005 and CDs have become largely replaced by storage on flash memory on Mp3 players. In the last few years music is streamed from the internet, e.g. using Spotify, and no copies are stored locally anymore. Furthermore, functionality of products changes rapidly. This innovation is causing a high pressure on the time-tomarket because usually the first company that offers the functionality, earns most of the revenues, while firms that are lagging lose market share rapidly. Furthermore the time to amortize the development investments has reduced because a feature that is considered a unique selling point at one moment in time may become mainstream functionality within in a few months [Kano 1984]. A high pressure on product quality: Delivering products with sufficient quality is becoming an increasing problem and consumers are increasingly unsatisfied with the reliability of the products. The field call rate, which is the percentage of products that are returned to the shops and service centers, has dramatically increased [Brombacher 2004]. This is caused by two types of failures:

2

Introduction (1) Failures caused by faults in the software [Sieworek 2004], referred to as software failures [Lyu 1996, BSI 1998]. This is often caused by a mismatch between components from 3rd parties and in-house developed components [Trew 2006]. (2) Failures caused by a discrepancy between the user’s expectations and the functionality of a product, referred to as soft failures [Geudens 2005, Koca 2006]. This means that although the product functions as specified, the user is not satisfied with the functionality [Korhonen 2010], experiences usability problems or experiences problems that are caused by a poor interoperability with other devices [Norman 2002, Ouden 2006].

1.2

Problem Statement

The following problem statement is the starting point for this thesis: Consumer electronics products are characterized by a high degree of innovation and are containing an increasing amount of variability that is implemented through software. The consequences are that the development cost, throughput time and product quality are highly determined by software development. To reduce development costs and bring new functionality to the market faster, the recent trend is that the software is developed by multiple parties. As a result the success of a company nowadays largely depends on whether it can develop the required software together with multiple parties in a timely fashion, while ensuring sufficient reliable products.

1.3 Research Objectives Based on the problem statement in the previous section the following objectives for this thesis have been formulated:  Determine the consequences and identify the problems that are caused by the increasing amount of software, variability and involvement of multiple parties for the practice of software product line engineering in the consumer electronics industry.  Developed methods that can be used by the industry as part of their strategy, software development and test process to address the consequences and problems that have been identified, and determine the benefits and consequences of applying these methods for the software development process. Note: While the objectives for the research in this thesis find its origin in consumer electronics, some of the results are more widely applicable, i.e. to the domain of embedded systems or software engineering in general. Section 1.7 gives an overview of the applicability of the individual research results.

3

Chapter 1

1.4

Research Methodology

This section describes the research methodology that is used in this thesis. Determining an appropriate research methodology is considered an important element in a research study [Wedawatta 2011]. A research methodology is defined as “the overall approach to the entire process of the research study” [Collis 2009]. This includes the process that is followed and the methods that are used to answer the research questions [Saunders 2009]. This section describes the general process and gives and overview of the methods that are used. Section 1.8 presents in detail which type of research results, process and methods have been used to obtain the individual research questions. This section starts with the type of research results, since this determines which process and what type of methods have to be used [Runeson 2009, Saunders 2009].

1.4.1

Types of research results

Robson [Robson 2002] identifies four types of research results: (1) Exploratory, (2) Descriptive, (3) Explanatory, (4) Improving. In the literature of software engineering usually two different types of research results are identified [Lazaro 2005, Easterbrook 2008, Shaw 2002]: A. Research that studies the practice of software engineering, including the methods and techniques used. This type is empirical research and results in a descriptive or explanatory model, the identification of concerns, reporting of experiences, answer to a specific question, or taxonomy for a problem area. This type encompasses the first three types of research results as identified by Robson. B. Research that is aimed to solve a particular problem. The goal of this type of research is to find better ways in doing things. This type of research results in a solution, which can be a method or technique; a language or tool; or a specific solution that embodies a result. These solutions are aimed to improve the development of software e.g. to make the process more efficient, develop higher quality products, create architectures that are better maintainable etcetera. This type encompasses the last category of Robson, i.e.: improving. Long term research programs in software engineering usually involves a mix of these two types of research results since researchers investigate a specific problem and develop a solution that works best [Wieringa 2006]. The research for this thesis also consists of these two types of research, since the objectives are both to determine the consequences, as well as to create methods to improve the practice.

1.4.2

Research process

A scientific research process is defined as follows [Bailey 2012]: “Research is a systematic process based on the scientific method that facilitates the identification of relationships and determination of differences in order to answer a question. The scientific method is a process that uses an organized structure to formulate questions and determine answers in a research project”.

4

Introduction The research in this thesis follows a stepwise process and these steps are presented in Figure 1. Here the processes to obtain the research results that covers both the objectives are presented as connected tracks. On the left side of the figure the steps to obtain descriptive and explanatory results are shown and on the right hand side the steps to obtain the research results that are aimed to solve a particular problem.

Step 1: Observe current practice

Step 2: Describe current practice

Step 4: Formulate Theory

Step 3: Compare with related art

Step 5: Identify Problems

Step 6: Compare with Related art

Descriptive and explanatory

Step 7: Analyze Shortcomings

Step 8: Develop Improved method

Step 10: Determine Improvements

Step 9: Implement Improved method

Problem solving

Figure 1 Research process The first research track is aimed to study and describe the practice more closely and results in descriptive and/or explanatory research. Based on the observations, the current practice is compared with descriptions in related, i.e. steps 1 through 3. A new theory is formulated when the related art does not describe the practice sufficiently accurate, i.e. step 4. The new theory is validated by comparing the theory with current practices, which is step 1 again. In this thesis the validation is done by comparing the theory to one or more additional case studies. This cycle is performed a few times, until the newly formulated theory conforms to the studied cases. The second track is aimed to develop methods to solve problems in current practice. Based on the observations the problems are identified, i.e. step 5, and the related art is studied to determine whether solutions for these problems are already available, i.e. step 6. When there are shortcomings in the described method in related art, i.e. step 7, an improved method is developed, i.e. step 8. This method is then implemented and the improvements are determined and analyzed, i.e. step 9 and 10. This cycle is executed one or more times, until the developed method solves the problem sufficiently or is stopped when a better solution cannot be found. This overall process is applied several times in this thesis. In the first chapters of this thesis the research is primarily descriptive and explanatory and is focused on the identification of the problems and challenges. In the chapters that follow, the findings from the first chapters are used to develop and evaluate new methods to solve the identified problems.

1.4.3

Research methods

Software engineering science is a laboratory science which involves case studies to demonstrate the applicability of a certain method and to develop and prove or disprove theories [Basili 1996]. Easterbrook identified five classes of research methods that are most relevant in software engineering [Easterbrook 2008]: (1) controlled experiments, (2)

5

Chapter 1 exploratory case studies, (3) confirmatory case studies, (4) survey research, (5) ethnographies and (6) action research. In this thesis, the first 3 of these methods are used and are summarized below: (1) Controlled Experiments: A controlled experiment is used to test a hypothesis. It is an investigation of a testable hypothesis where one or more independent variables are manipulated to measure their effect on one or more dependent variables [Easterbrook 2008]. A precondition for using controlled experiments is a clear hypothesis, and the outcome of the experiment is whether the hypothesis is rejected or accepted. (2) Exploratory case studies: Exploratory case studies are used to explore a problem area, get an overall picture, describes current practice and to build theories [Easterbrook 2008, Eisenhardt 1989]. During theory building, a theory is created by identifying similarities and differences between different cases and therefore the resulting theory is derived from the studies [Robson 2002]. In general, case studies are preferred when the investigator has little control over the events and when the focus is on a contemporary phenomenon with some real-life context [Yin 2003]. A theory is more sound when multiple and different case studies and data collection methods are used [Eisenhardt 1989]. (3) Confirmatory case studies: Confirmatory case studies are used to test or validate an existing theory. The purpose of theory validation is to test whether a proposed theory is sound and to assess whether a method or technique actually works in practice [Easterbrook 2008]. Furthermore it is used to determine the benefits of a method. Shaw described the different validation techniques in software engineering [Shaw 2002]. She argued that a good research result requires evidence that the result is sound. In this classification, confirmatory case studies can have the form of a guided experiment, a pilot project, a toy or real life example, or by capturing the experiences of the involved persons that are using the newly developed method. In this thesis, these various types of case studies are used and different data collection methods. In most cases the theory building and validation is based on qualitative data since quantitative data was usually not available. Where possible the qualitative evaluation was supported by quantitative data.

1.5

Research Questions

The research questions in this thesis are based on a stepwise refinement of the research objectives and follows the process described in Figure 1. Since not all the identified problems could be addressed; the decision on which research questions have been answered is based on the importance of the research questions for the involved organizations and on the availability of case studies [Robson 2002]. These selection criteria have the advantage that the research is based on existing problems in the industry and that the proposed methods have been validated using concrete case studies. Part I is the starting point of this thesis. This part is focused on an exploration of the problem space and identification of the architectural challenges for the development of software for consumer electronics. The overall research question for this part is:

6

Introduction RQ 1

What transitions have taken place in the development of software for consumer electronics products and what are the consequences for building software product lines?

In Chapter 2 the transition in the development of software development for consumer electronics products between 1990 and 2010 is analyzed and the forces that govern these transitions are identified as well as the consequences for software architectures. The research questions that will be answered in Chapter 2 are: RQ 1.1 RQ 1.2 RQ 1.3

What transitions between industry structures have taken place? What were the forces and actors involved? What were the forces and actors involved? How were the architectural interfaces between the actors defined, and what were the consequences for modularity, variability and product integration?

One of the identified steps for further research in Chapter 2 is that an industry can adopt a hierarchical model where at different places in the architecture, different industry structures are used, thus leading to a more fine-grained model. Since 2010, several consumer electronics firms have create ecosystems by offering a stable interface towards 3 rd party application developers and an application store to support this, but do not have a modular architecture of the software that is embedded in the device. Consequently a model is adopted where an ecosystem is used to support 3rd party applications, but different industry structures are used for the device. Choosing an ecosystem type is a critical strategic question. When a company decides to use its own platform, it has to open ups its architecture and has to attract sufficient parties. When a company decides to use an existing ecosystem by integrating an existing middleware layer that serves as application framework, it has to be confident that the adopted ecosystem is sufficiently viable. Therefore the research questions that will be answered in Chapter 3 are: RQ 1.4 RQ 1.5

What types of ecosystems are used in consumer electronics products to support 3rd party applications? Which of the ecosystems types is most suitable for a specific product category, from a software engineering perspective?

One of the outcomes of the research presented in chapter 2 is that for software that is embedded in consumer electronics products, software supply chain is the dominant form of co-operation between companies. In a software supply chain a company buys a set of components from several suppliers, with their variability, and integrates them into a product, possibly combined with in-house developed components. For each of its customers, i.e. the next participants in the chain, a specialized version is created. Each of the participants adds new functionality and makes configuration choices based on their role and responsibility in the supply chain.

7

Chapter 1 This means that the product line now consists of different components with different sets of variability and possibly using different technologies to implement the components. Therefore the overall research question for Part II of this thesis is: RQ 2

What are the consequences for variability management, when applied in software supply chains, and what modelling approaches can support this?

Since suppliers of software develop a product line for multiple customers and multiple products, such as mobile phones, TVs and DVDs; the variability of these product lines have to be managed across different, or multiple, product lines. From this we derive the following research question which will be answered in chapter 4: RQ 2.1

How can feature models be constructed to model multiple product lines in software supply chains?

It frequently happens that software is purchased from alternative suppliers for similar functionality, for instance because these suppliers have components with a different feature set; a different price setting, quality etcetera. This gives the following research question which will be answered in chapter 5. RQ 2.2

How should feature models be constructed which capture the variability from the components from alternative suppliers, partly offering the same feature set?

Since software components from alternative suppliers often have different interfaces and are developed using different component technologies, also known as heterogeneous components, glue components are usually needed to bridge these components. In a product line where multiple suppliers are used, this may become a repetitive task for which an automated approach, such as Model Driven Architectures (MDA), could be suitable. This gives the following research question, which will be answered in chapter 6: RQ 2.3 RQ 2.4 RQ 2.5

What are the challenges that arise from combining components with nonmatching interfaces and what are the limitations of current practice? How can MDA be used to bridge mismatches between heterogeneous components in software supply chains? What are the consequences for the development process and the development roles?

One of the outcomes of the research in chapters 4 and 5 is that variability models that are used in software supply chains have become large in size and complex, especially due to the large amount of dependency relations. Therefore alternative languages should be considered to capture variability. Such an alternative language is TVL, a textual language which abbreviation stands for Textual Variability Language. This brings us the following research question that will be answered in chapter 7.

8

Introduction RQ 2.6 RQ 2.7

What are the benefits of TVL for modeling product line variability, as perceived by model designers? What are the product line variability modeling requirements that are not fulfilled by TVL?

One of the outcomes of part I of this thesis is that the amount of software faults is increasing which is caused, amongst other reasons, by an increasing amount and complexity of software and because of the use of components from 3rd parties, as shown in part II. Because of the pressure on time-to-market it is not possible anymore to remove all software faults prior to introduction. One of the methods to increase the test efficiency is to improve the effectiveness of the test process by focusing on those areas that have the largest impact on the reliability, as perceived by the end-user, and by focusing on the risks to the user and business. Therefore the overall research question for part III of this thesis is: RQ 3

How can test efficiency be improved when the risks and user product interaction are taken into account?

Operational profile driven testing and risk based testing are wide spread, and related, methods to improve the test efficiency. In operational profile driven testing the number of test cases is allocated to the different functions and operations according to the quantitative use in practice. Risk based testing is used to prioritize test effort to those functions that pose the greatest risk, which includes several aspects. However the benefits of these methods have not been established with statistical evidence. This gives the research questions that will be answered in chapter 8. RQ 3.1 RQ 3.2

What are the benefits for the test efficiency when using operational profiles? Which dimensions in risk based testing require most attention to reduce the overall risks?

One of the outcomes of chapter 8 is that risk based testing improves the test efficiency. However, this method does not take into account that different members of the product line contain different feature sets. Furthermore, in product line engineering the development and testing of software product lines is usually separated into testing during domain engineering and during application engineering. And, as shown in part II, product line engineering relies heavily on variability management tools. This gives the research questions which will be answered in chapter 9. RQ 3.3 RQ 3.4 RQ 3.5

How can risk based testing by applied in product line engineering? What are the consequences for testing during domain and application engineering? What tool support is required?

9

Chapter 1 The use of operational profiles is a widespread method for software products and the benefits have been determined in chapter 8. However, operational profiles are used for reducing the technical failures caused by faults in the software. For highly innovative consumer electronics products, failures often occur because an unexpected product use and because of a mismatch between the functionality of a product and the user’s expectations. This leads often to non-technical failures, also known as “soft” failures. Chapter 10 determines the suitability of operational profiles for highly innovative consumer electronics products and therefore the research questions that will be answered in this chapter are: RQ 3.6

RQ 3.7

1.6

To what extent can an operational profile be applied to consumer electronics products to model (unexpected) product use for highly innovative consumer electronics products? What adjustments need to be made?

Thesis Outline and Article Overview

The main body of this thesis consists of a number of articles that have been peer reviewed and have been published in major journals (chapters 2, 6, 10) and conferences (chapters 3, 4, 5, 7, 8, 9). The articles have been mainly left as is. The exceptions are chapter 3, 8 and 9 for which an extended version is included. In this thesis, the articles have been published in a logical order, rather than the order in which they have been published. The reasons for this are as follows:  Part I contains descriptive and explanatory research and describes the context and challenges, while the part II and III contain solutions for a selection of the challenges that are identified in the part I.  It is based on the order of activities in a business context as described in the BAPO (Business Architecture Process Organization) model for software product line engineering [Linden 2004]. Part I describes the choices of a firm, e.g. the role it wants to play in an ecosystem or supply chain. This is followed part II that covers the creation of a product, while part III contains solutions for the last phase in a development project, namely that of testing. The main relations between the articles is shown in the figure below, and follows the same structure as the stepwise refinement of the research questions:

10

Introduction Chapter 3: Consumer electronics software ecosystems

Chapter 2: The changing industry structure of software development for consumer electronics

Chapter 4: Using context variability to model multiple product lines

Chapter 8: A statistical analysis of operational profile driven testing and risk based testing

Chapter 5: Supplier independent feature modelling

Chapter 9: Risk based testing for software product line engineering

Chapter 6: Using MDA for integration of heterogeneous components in software supply chains Chapter 7: Evaluating a textual feature modelling language

Chapter 10: Towards a more systematic analysis of uncertain user-product interactions

Figure 2 Article overview and their main relations

1.6.1

Part I: Industry structures for software development

The article presented in Chapter 2 provides an analysis of the transition of software development for the consumer electronics industry between 1990 and 2010 using case studies from digital televisions and mobile phones. This article introduces a model consisting of five industry types, describes the forces that govern the transitions and describes the consequences for software architectures and software product line engineering. It is shown that the software supply chain is the dominant structures because the modularization of the architecture is limited. The model and forces presented serve the decision making process for companies that consider a transition to a different industry type. The article presented in Chapter 3 describes three types of ecosystems that are used for consumer electronics products that support 3 rd party applications: Vertically integrated hardware/software platforms, closed source software platforms and open source software platforms. A first step towards multi-criteria decision support method is introduced to select the most suitable ecosystem type for a product category from a software engineering perspective. This method is used to analyze traditional consumer electronics products, such as gaming consoles, digital cameras and digital televisions, as well as smart phones, tablets and personal computers, and newly introduced product categories, such as smart watches. Using the analysis method, the main challenges are identified and the article shows how the different players address these challenges.

11

Chapter 1 Chapter 2 H. Hartmann, T. Trew, J. Bosch: “The Changing Industry Structure of Software Development for Consumer Electronics and Its Consequences for Software Architectures”, In Journal of Systems and Software 85-1, (2012), 178-192. DOI 10.1016/j.jss.2011.08.007 Chapter 3 H. Hartmann, J. Bosch: “Towards a Multi-Criteria Decision Support Method for Consumer Electronics Software Ecosystems” This article is submitted and is an extended version of the article: H. Hartmann, J. Bosch: “Orchestrate Your Platform: Architectural Challenges for Different Types of Ecosystems for Mobile Devices”, ICSOB 2014: 163-178, DOI 10.1007/978-3-319-08738-2_12

1.6.2

Part II: Variability management in software supply chains

The article presented in Chapter 4 introduces the concept of context variability, which contains the primary drivers for feature selection, such as geographic regions or different customer types. The context variability model constrains the feature model which makes it possible to model multiple product lines. The article shows how this approach can be used in software supply chains and that the approach is intuitive and straightforward to use with existing feature modeling tools. The article presented in Chapter 5 introduces an approach to combine feature models from alternative suppliers, i.e. suppliers that offer partly the same functionality. The article shows that the introduced method facilitates an efficient selection of features and suppliers during application engineering, while preserving the links between the features and the development artifacts. The article presented in Chapter 6 describes the challenges that arise from combining components using different component technologies and with non-matching interfaces and in particular the challenges that arise from software supply chains and for developing resource constrained devices. An approach is introduced that uses Model Driven Architectures (MDA) to generate the required glue components efficiently and only then when they are required. The article addresses the implications on the development process and identifies the development roles that are associated with this approach. The article presented in Chapter 7 describes an evaluation of a textual feature modeling language using four industrial case studies. These case studies represent examples of large feature models which are cumbersome to model with the traditional graphical feature modeling languages, e.g. based on the FODA notation. The article shows that TVL, the textual feature modeling language that was evaluated, combines advantages of textual languages as well as graphical languages and that practitioners can benefit from TVL. Chapter 4 H. Hartmann, T. Trew: Using Feature Diagrams with Context Variability to Model Multiple Product Lines for Software Supply Chains. In: 12th International Software Product Line Conference (2008), DOI 10.1109/SPLC.2008.15 Chapter 5 H. Hartmann, T. Trew, Aart Matsinger: “Supplier Independent Feature Modeling”. In: 13th International Software Product Line Conference (2009) Chapter 6 H. Hartmann, M. Keren, A. Matsinger, J. Rubin, T. Trew, T.Yatzkar-Haham: “Using MDA for Integration of Heterogeneous Components in Software Supply Chains”, Sci. Comput. Program. 78(12): 2313-2330 (2013) DOI 10.1016/j.scico.2012.04.004

12

Introduction Chapter 7 Hubaux, Q. Boucher, H. Hartmann, R. Michel, P. Heymans:”Evaluating a Textual Feature Modelling Language: Four Industrial Case Studies”, SLE 2010, 3rd International Conference on Software Language Engineering, (2010). DOI 10.1007/978-3642-19440-5_23

1.6.3

Part III: Software testing

The article presented in chapter 8 gives a statistical analysis of the benefits of test time allocation based on an operational profile and test time allocation based on a the risks for the user and the business. An operational profile is a quantification of the operations that in its turn is derived from a quantification of functional profiles and user profiles. Risk based testing is used to prioritize test effort to those functions that pose the greatest risk. The analysis shows that using operational profiles improves the test efficiency, however not when a high reliability is required. For risk based testing a quantification is introduced, and the analysis shows that an efficiency improvement can be realized when the usage frequency is treated as a separate dimension of the risk matrix. The article presented in chapter 9 presents risk based testing for product line engineering. This methods is based on risk based testing for single system engineering which is extended with a dimension that captures the percentage of product variants that use a particular development artefact. Based on the risk of development artifacts, the priorities for testing during domain and application engineering are determined. It is shown that the basic ideas behind risk-based testing product lines are intuitive, pragmatic in nature, and provide the means for practitioners for guiding the test effort for different test levels. The article presented in Chapter 10 discusses the limitations of using operational profiles for highly innovative consumer electronics products since operational profiles are focused on reducing the technical risks while an increasing proportion of product returns is caused by non-technical failures, so called soft-failures. The article presents an enhanced userproduct interaction framework that takes into account the use-process, the environment and the different types of users. Chapter 8 H. Hartmann: “A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing”. This article is submitted and is an extended versions of: H. Hartmann, J. Bokkerink, V. Ronteltap: "How to reduce your test process with 30%; The application of operational Profiles at Philips Medical Systems”. In: Supplementary proceedings of the 17th IEEE International Symposium on Software Reliability Engineering. Chapter 9 H. Hartmann, F. van der Linden, J. Bosch: Risk based testing for software product line engineering. SPLC 2014: 227-231, DOI 10.1145/2648511.2648535 Chapter 10 Y. Lu, P.H. den Ouden, A.C. Brombacher, W.H.J.M. Geudens, H. Hartmann: “Towards a more systematic analysis of uncertain user-product interactions in product development: an enhanced user-product interaction framework”, Quality and Reliability Engineering International, 2007, DOI 10.1002/qre.820 This thesis concludes with Chapter 11. This chapter answers the research questions, gives recommendations for future research and describes the contributions of this thesis.

13

Chapter 1

1.7 Applicability of the Research Results While this thesis has found its starting point in consumer electronics, many of the results of these articles are applicable to a wider domain. Here we identify three levels (1) Consumer electronics (2) Embedded systems and (3) Generally applicable in software engineering. The table below presents this classification and includes the domain in which the case studies where executed. Table 1 Applicability of the research results Chapter 2: The changing industry structure of software development for consumer electronics 3: Consumer electronics software ecosystems 4: Using context variability to model multiple product lines 5: Supplier independent feature modelling 6: Using MDA for integration of heterogeneous components in software supply chains 7: Evaluating a textual feature modelling language 8: A statistical analysis of operational profile driven testing and risk based testing 9: Risk based testing for software product line engineering 10: Towards a more systematic analysis of uncertain user-product interactions

Domain Consumer Electronics

Case studies Consumer Electronics

Consumer Electronics


Embedded Systems


Embedded Systems


Embedded Systems


Generally applicable

4 different domains


Healthcare


Healthcare



1.8 Detailed Overview of Research Processes and Methods In Section 1.4 the overall research methodology was described. The current section provides details on how this was applied for each of the research questions that have been answered in the individual chapters. This section covers the type of research results, the process that was applied, in relation to the process shown in figure 1, and the research methods and

14

Introduction validation techniques. Here the types of research results, methods, and the validation techniques from Shaw [Shaw 2002] are highlighted using an italic font. Chapter 2: The changing industry structure of software development for consumer electronics Type of research result: A descriptive model that presents a taxonomy of different types of industry structures. Furthermore it contains an explanatory model that presents the forces that govern the transitions between these types. Research process: To arrive at the research results, steps 1 through 4 have been executed twice: At first the current practice from digital televisions was observed and described. This was compared with related art, i.e. the PC industry and others, and a theory was formulated. Then the practice of mobile phones, was observed and described. Based on these observations the model of industry structures was refined. Finally the problems and consequences, i.e. step 5, were identified and described. Research methods: Exploratory case studies have been used to build the theory, using case studies from two different product types. Multiple data sources are used, i.e. the experiences from the authors, data obtained through interviews, and information from literature. The empirical predictive model is evaluated using these case studies and information from literature. Chapter 3: Consumer electronics software ecosystems Type of research result: A descriptive model that presents the different types of ecosystems. Furthermore it contains a first towards a multi-criteria decision support method which is a method to select between the alternative types of ecosystems. This method is based on an explanatory model of the strengths and challenges of each ecosystem type. Research process: To arrive at the research results, step 1 through 4 have been executed a number of times. To obtain the descriptive and explanatory model, multiple cases from mobile devices were studied, which iteratively lead to these theories. This result was published as separate article. Then the practice of other consumer electronics products was studied and described, which confirmed the theory. The creation of the decision support method was started with the observations from the challenges, i.e. step 5, and the comparison with literature, i.e. step 6, did not reveal an existing method that can determine the most suitable type of ecosystem. The procedure to select between different types of ecosystem was then developed iteratively which started with an initial method, i.e. step 7, and was exercised and evaluated, i.e. steps 8 through 10, using the case studies from 7 different consumer electronics products. Research methods: Exploratory case studies have been used to build the theory. For validating the theory and the model, the case studies where used to confirm and refine the theories. The case studies are based on qualitative data from real life examples of a wide range of consumer electronics products, where literature and the experiences from the authors are the main data sources. The method to determine the most suitable ecosystem type was evaluated using real life examples from a wide range of products, using data from literature.

15

Chapter 1 Chapter 4: Using context variability to model multiple product lines Type of research result: This research contains a technique to improve the practice of variability modelling in software supply chains. Research process: This research started with step 5, a result of the research in chapter 2, that variability is managed in a supply chains. Furthermore, an existing example from NXP showed that a method was needed to model multiple product lines in a single feature model. The comparison with related art, i.e. step 6, revealed that a method did not exist that could be used within a supply chain, nor with standard feature modelling tools. The improved method was developed, i.e. step 8, and validated with cases from literature and the earlier mentioned example from NXP. This showed clear improvements, i.e. step 9, but also some shortcomings of the used tools, related to merging of feature models. Since merging was not regarded as primary objective, the method was not further improved. Research method: Exploratory case studies from the practice of NXP and in literature were used to identify the problem. An example is used to demonstrate the applicability of this technique, and experiences from several persons were used, using pilot projects, to validate whether this technique is easy to apply and can be used with existing variability management tools. Chapter 5: Supplier independent feature modelling Type of research result: This research contains a technique to improve the practice of variability modelling in software supply chains. Research process: This research started with the results from chapter 4, i.e. step 5, which identified the problem of merging overlapping feature models. This problem was confirmed by a concrete case study, i.e. that of ZigBee. The literature was studied, i.e. step 6, but revealed no existing solutions. To arrive at the improved solution, multiple alternatives where developed, compared and exercised, i.e. step 7 through 10. Research method: Case studies from the practice of NXP were used to identify the problem. A confirmatory case study consisted of using a pilot project that demonstrates the applicability of this technique. Experiences from several persons were used to validate whether this approach can be used with existing variability management tools. Chapter 6: Using MDA for integration of heterogeneous components in software supply chains Type of research result: This research contains a technique to improve the practice of creating glue components. Research process: This research started with the results from chapter 4, i.e. step 5, which identified the problem of combining heterogeneous components. This problem was confirmed by a concrete case study, i.e. that of ZigBee. The literature was studied, which revealed no existing efficient solutions. To arrive at the improved solution, multiple alternatives where considered, developed and exercised, and the improvements determined i.e. step 7 through 10. The final solution still revealed some shortcomings, but no additional improvement cycles where executed due to a lack of funding. Research method: Case studies from the practice of NXP were used to identify the problem and a confirmatory case study consisted of using a pilot project that demonstrates the

16

Introduction applicability of this technique. Experiences from several persons were used to validate whether this approach has efficiency gains in comparison with current practice. Chapter 7: Evaluating a textual feature modelling language Type of research result: This research provides an answer to the question what the benefits of TVL are and provides an answer to the requirements that are not fulfilled. Research process: This evaluation fulfils step 10: determine improvements, and step 7: analyze shortcomings, to validate the improved method of using a textual variability language. Research method: Confirmatory case studies were used to validate the research results. The data was obtained using a questionnaires and through interviews. In this way the experiences from several persons were used based on real life examples using TVL and these experiences were compared with using the tools that were used by the participants as part of their day-to-day work. Chapter 8: A statistical analysis of operational profile driven testing and risk based testing Type of research result: This research provides answers to the questions what the efficiency improvements of using operational profiles are and comparison of which dimensions in risk based testing require most attention. Furthermore, it contains an improved method for risk based testing. Research process: This evaluation fulfils step 10: determine improvements of using operational profiles and risk based testing. Based on the analysis the shortcoming were analyzed, step 7, and improved methods were proposed, step 8, namely to use operational profiles unless a very high reliability is required and for risk based testing to use three dimensions, rather than two. The improvements were determined using a statistical analysis. Research method: A quantitative analysis was done using a number of examples and a confirmatory case study which consisted of using a pilot project. Chapter 9: Risk based testing for software product line engineering Type of research result: This research contains a method to improve the practice of testing in software product line engineering. Research process: This research followed the steps 5 through 10: First the problems from current practice were identified that revealed that in current practice an improvement can be realized. The comparison with related art revealed no existing methods. Based on the shortcomings of current practice, an improved method was developed and the improvements were determined. Also the shortcoming of the improved method were analyzed, related to quantified feature modelling. A further improvement cycle has not been started but is left for further research. Research method: The method was developed using an exploratory case study from a real life project and interviews where used to obtain the experiences from the practitioners. An example was used to validate the use and modelling of quantified feature models.

17

Chapter 1 Chapter 10: Towards a more systematic analysis of uncertain user-product interactions Type of research result: This research contains a qualitative model and framework that describes which aspects are needed to analyze unexpected user product interaction. Research process: This research followed the steps 5 through 10: First the problems from current practice were identified that revealed that operational profiles cannot be used for detecting non-technical failures. The comparison with related art revealed that there are no existing models to analyze user product interaction and do not provide a framework that can be used for highly innovative consumer electronics. Based on the shortcomings of operational profiles, an improved framework was developed and the improvements were determined. The analysis revealed that the framework could not be fully validated, i.e. related to the environment. A further validation and improvement cycle has not been started but is left for further research. Research method: Exploratory case studies were used to develop the framework, and a controlled experiment was executed to test the hypotheses using a quantitative analysis.

1.9 Related Publications A few related publications are not included in this thesis. Earlier versions for chapter 6 have been published as a workshop and conference articles. On the relation between software reliability and soft reliability, a publication was made that describes different classifications that were found in the academia and the industry. This has led to a better understanding of the differences between Fault-not-Found and No-FaultFound and has provided supportive material for modeling different types of users and user product interaction as described in chapter 9. A publication was made on the application of TRIZ for software development. This publication discusses the increasing size and complexity of software architectures and argues that TRIZ might become useful for dealing with conflicting requirements in developing software architectures. These related publications are the following:   



18

H. Hartmann, M. Keren, A. Matsinger, J. Rubin, T. Trew, Tali Yatzkar Haham: “Using MDA for Integration of Heterogeneous Components in Software Supply Chains”, In 14th International Product Line Conference, 2010 H. Hartmann, M. Keren, A. Matsinger, J. Rubin, T. Trew, Tali Yatzkar Haham: “Integrating Heterogeneous Components in Software Supply Chains”, Proceedings of the 2010 ICSE Workshop on Product Line Approaches in Software Engineering, 2010 Koca, Y. Lu, A.C. Brombacher, J.H. Hartmann: “Towards establishing foundations for new classes of reliability problems concerning strongly innovative products”, In proceedings of the 2006 IEEE International Conference on Management of Innovation and Technology, (Vol. 2, pp. 581-584), 2006 H. Hartmann, A. Vermeulen, M. van Beers :”The application of TRIZ in Software Development”, TRIZ-Journal, September 2004.

Introduction

1.10 Contributions to the Articles This thesis consists of research that is published in articles for which multiple authors have contributed. This section describes my personal contributions to the research in these articles. Chapter 2: H. Hartmann, T. Trew, J. Bosch: “The Changing Industry Structure of Software Development for Consumer Electronics and Its Consequences for Software Architectures”, Journal of Systems and Software 85(1): 178-192 (2012): I did the research for this paper and wrote the paper. Tim Trew contributed with parts of the description of the case study of digital televisions and both Tim Trew and Jan Bosch acted as reviewer. Chapter 3: H. Hartmann, J. Bosch: “Orchestrate Your Platform: Architectural Challenges for Different Types of Ecosystems for Mobile Devices”, ICSOB 2014: 163-178 I did the research for this paper and wrote the paper. Jan Bosch acted as reviewer. This also holds for the extension that has been submitted and is included in this thesis. Chapter 4: H. Hartmann, T. Trew: “Using Feature Diagrams with Context Variability to Model Multiple Product Lines for Software Supply Chains”. SPLC 2008: 12-21 I did the research for this paper and wrote the paper. Tim Trew acted as reviewer. Chapter 5: H. Hartmann, T. Trew, Aart Matsinger: “Supplier independent feature modelling”. SPLC 2009: 191-200 I did the research for this paper and wrote the paper. Aart Matsinger analyzed the variation points of the case study. Both Aart Matsinger and Tim Trew acted as reviewer. Chapter 6: H. Hartmann, M. Keren, A. Matsinger, J. Rubin, T. Trew, T.Yatzkar-Haham: “Using MDA for integration of heterogeneous components in software supply chains. Sci. Comput. Program. 78(12): 2313-2330 (2013)” The research for this paper was a joint effort between the six authors over a long period of time. My primary contributions to the research were the analysis of the different approaches in current practice, the identification of the requirements from software supply chains, the overall process, the connection to variability management tooling, the analysis of the tasks of the different development roles and the evaluation of the approach. Furthermore I acted as corresponding author of this paper, including earlier versions for workshops and conferences, I wrote most parts of this paper and acted as editor for those parts that I didn’t write. Chapter 7: A. Hubaux, Q. Boucher, H. Hartmann, R. Michel, P. Heymans:” Evaluating a Textual Feature Modelling Language: Four Industrial Case Studies”, SLE 2010, 3rd International Conference on Software Language Engineering, (2010). The research for this paper was initiated by the University of Namur and I contributed as expert from the industry on complex feature modelling and my experience as researcher on user product interaction. I contributed by the identification of suitable cases studies and I

19

Chapter 1 analyzed and described these two case studies. Furthermore I identified the feedback process and contributed by analyzing the differences between the case studies. I acted as reviewer for the parts that I didn’t write. Chapter 8. H. Hartmann, J. Bökkerink, V. Ronteltap: "How to reduce your test process with 30%; The application of operational Profiles at Philips Medical Systems”. The 17th IEEE International Symposium on Software Reliability Engineering (2006) (Conference paper, Industry Practices Report) I did the research for this paper and wrote the paper. Jarmila Bökkerink was the industry partner at which the case study was performed. Vincent Ronteltap acted as reviewer. The extensions that has been submitted and is included in this thesis was done solely by myself. Chapter 9: H. Hartmann, F. van der Linden, J. Bosch: “Risk Based Testing for Software Product Line Engineering”, SPLC2014 I did the research for this paper and wrote the paper. Frank van der Linden identified the case study. Frank van der Linden and Jan Bosch acted as reviewer. Chapter 10: Y. Lu, P.H. den Ouden, A.C. Brombacher, W.H.J.M. Geudens, H. Hartmann: “Towards a more systematic analysis of uncertain user-product interactions in product development: an enhanced user-product interaction framework”. Quality and Reliability Eng. Int. 23(1): 19-29 (2007). This paper was the result of joint research over a long period of time. The starting point for this article was my research on operational profiles, described in chapter 8. I identified that operational profiles has less value benefits when applied to highly innovative products, by recognizing that (1) the use process of consumer electronics products is not captured in operational profiles; (2) the user profiles, as defined by software reliability engineering are different that those used for highly innovative products and (3) that the environment is not explicitly modeled. Based on this analysis I contributed to the developing of the proposed interaction framework. I contributed with setting up the analysis method, and executed two of the high contrast consumer tests and analyzed the results of these tests.

20

Part I:

Industry Structures for Software Development

“For when I construct lines and colour combinations on a flat surface, it is with the aim of portraying ‘universally beauty’ as consciously as possible. Nature inspires me, but I want to approach the truth as closely as possible, abstract everything until I come to the foundation of things.” Piet Mondriaan

21

22

Chapter 2.

The Changing Industry Structure of Software Development for Consumer Electronics

This chapter is published as: H. Hartmann, T. Trew, J. Bosch: “The Changing Industry Structure of Software Development for Consumer Electronics and Its Consequences for Software Architectures”, The Journal of Systems & Software 85 (2012) pp. 178-192. DOI 10.1016/j.jss.2011.08.007 Abstract During the last decade the structure of the consumer electronics industry has been changing profoundly. Current consumer electronics products are built using components from a large variety of specialized firms, whereas previously each product was developed by a single, vertically-integrated company. Taking a software development perspective, we analyze the transition in the consumer electronics industry using case studies from digital televisions and mobile phones. We introduce a model consisting of five industry structure types and describe the forces that govern the transition between types and we describe the consequences for software architectures. We conclude that, at this point in time, Software Supply Chains is the dominant industry structure for developing consumer electronics products. This is because the modularization of the architecture is limited, due to the lack of industry-wide standards and because resource constrained devices require variants of supplied software that are optimized for different hardware configurations. Due to these characteristics open ecosystems have not been widely adopted. The model and forces can serve the decision making process for individual companies that consider the transition to a different type of industry structure as well as provide a framework for researchers studying the software-intensive industries.

2.1

Introduction

The consumer electronics industry, i.e. TVs, set-top boxes (STBs), audio and video storage devices and mobile phones, has gone through substantial changes during the last 15 years. From its inception until around 2000, the industry was dominated by a small group of consumer electronics companies. Each company was responsible for the complete design of its products and developed many of the product components. For decades, the cost of designing a product was negligible, compared with the material and manufacturing costs. However, as the functionality increased as the capabilities of their integrated circuits followed Moore’s law; the relative design cost became a significant factor. Furthermore, as processors became fast enough to process video and audio in software, it became possible

23

Chapter 2 to serve a wider customer base with more product variants from a single hardware platform. This contributed to a shift in the balance of development costs from hardware to software. In order to amortize these development costs more widely, independent software and IC companies emerged, either as spin-offs from consumer electronics firms or as newly created companies, and these became major players. In this paper we analyze the transitions in the structure of the consumer electronics industry and we introduce a model that describes the different industry structures that we have identified. These structures are presented from the perspectives of software architecture and industry structure. Since the consumer electronics industry structure is still in flux, we draw analogies from the computer industry, which went through a similar evolution. Andy Grove described the changing structure that took place in the computer industry between 1980 and 1995 [Grove 1996]. In 1980 that industry was dominated by a small group of companies that behaved as vertical silos, meaning that they developed the entire system, from ICs to sales and distribution, see Figure 3, which is adapted from [Grove 1996].

Sls & Dist

Retail stores super strs dealers direct

Software

Word Excel Lotus etc

Operating System

DOS

Hardware

IBM

Chips

Intel IBM DEC

OS/2 Compaq

Motorola

MAC

UNIX

Dell

HP

Risc

Sperry Wang Univac

1980 – Vertical Silos

1999 – Modular Cluster

Figure 3 Changing structure in the computer industry By 1995 the vertical silos had been replaced by modular clusters, i.e. groups of companies that were dominant in certain parts, or layers, of the system. This transition was facilitated by the high degree of modularity that was created [Baldwin1997, Christensen 2004]. This transition resulted in an ecosystem, which is formulated by Moore follows: “An intentional community of economic actors whose individual business shares in some large measure the fate of the whole community” [Moore 2006]. Some authors, e.g. [Genugten 2007] and [Suoranta 2006], predicted that the consumer electronics industry would follow an evolution similar to the computer industry. So far, however, this state has not been achieved and in this paper we will show why this is not the case. To explain the current industry structure and its evolution, we have developed a model consists of five industry structure types: Type 1: Vertically Integrated Firms Type 2: System Integrators and Specialized Suppliers Type 3: Supply Chains

24

The changing industry structure of software development for consumer electronics Type 4: Closed Ecosystems Type 5: Open Ecosystems In this paper we show that the evolution in the consumer electronics industry followed a structured, logical pattern and, at the time of writing, a software supply chain is the dominant industry structure. The industry has not evolved to type 4 or 5 due to the characteristics of the products, i.e. the need to make optimal use of system resources, the high degree of variability and the high rate of innovation. The consequence of this is a lack of industrywide standards and hence a limited degree of modularity. The key contributions of this paper are twofold:  We introduce a model consisting of five industry structure types which captures the evolution of software development for consumer electronics. This model is developed using a case study research method, with case studies from digital televisions and mobile phones.  We describe the forces that govern the transition between types and the consequences for product line architectures. These forces can serve the decision making process for individual firms for selecting the most appropriate industry for their business. The remainder of the paper is structured as follows. In Section 2.2 we present background and related work, followed by a section concerning our research method. Section 2.3 presents a high level overview of the case studies of digital televisions and mobile phones, from which we build our Industry Structures model that is presented in Section 2.4. Section 2.5 and 2.6 provided an elaboration of the case studies, focusing on the rationale behind the transitions and the consequences for software architectures. This paper concludes with a comparison of related work, our conclusions and the identification of areas for further research.

2.2

Background

In this section we describe the characteristics of embedded and consumer electronics products, and related work on industry structures and transitions.

2.2.1

Characteristics of consumer electronics products

A consumer electronic product consists of a combination of hardware and software that is designed to perform specific functions, often with real-time performance requirements and constrained computing resources. The products include DVD-players, televisions, cameras and mobile phones. Since different market segments usually have different requirements, a company may develop a range of variants to serve different groups of customers [Ommering 2004]. Each group of customers receives products that serve their specific needs. The concept of product line engineering is widely applied in consumer electronics [PHOF2010]. This allows for an efficient creation of variants of product using reusable development artifacts [Pohl 2005A]. There is a high pressure on the cost price, so the integrated circuits (ICs) should have the smallest possible footprint. Furthermore, for handheld products, these ICs should use as little

25

Chapter 2 energy as possible, given the limited power sources and options for heat dissipation. In order to satisfy these requirements, the amount of software has to be minimized as much as possible and the software must operate as efficiently as possible. As a consequence, an IC usually only contains the software that is needed for the particular product, or a group of similar products, in which it is used.

2.2.2

Organizational networks and software supply chains

Michael Porter introduced the term Value Chain to describe the activities needed to make a product and the value that each of activities created for the end product [Porter 1980]. Later this method has been extended to analyze Value Chains in an inter-organizational context, denoted as the Value System [Porter 1980], including the activities of upstream and downstream participants in a supply chain. A supply chain is defined as: “A network that starts with raw material, transforms them into intermediate goods, and then into final product delivered to customers. Each participant consumes logical and/or physical products from one or more upstream suppliers, adds value, usually by incorporating them into more complex products, and supplies the results to downstream consumers” [Lee 1994]. In the software industry, companies have emerged that are specialized in a certain software product or service [Greenfield 2004]. As an example, consider the supply chain in Figure 4. There are four links, starting with an audio decoding software vendor and ending with a car manufacturer.

For media Player Create MP3 Decoders

Amplifier

Video Decoder

Create Media ICs

For MP3 player For DVD player

Navigation Connectivity

Create Car-Info System

For Aftermarket Car1

Engine

Create Car

Chassis

Car2 Car3

Figure 4 Example of a software supply chain In a supply chain, each of the participants uses components containing variability, combines them with in-house developed components, and delivers specialized components containing variability to its customers. For example, a manufacturer of car infotainment systems uses multiple suppliers for media processing, navigation, and connectivity to create a product line for different car manufacturers. Jansen et al. introduced the term Software Supply Networks (SSN) and identified that the software architecture is created based on products and services from other parties in the network [Jansen 2007]. Peppard and Rylander [Peppard2006] identified the strong cooperative behavior and relationships between the parties in such a network.

26

The changing industry structure of software development for consumer electronics

2.2.3

Industry de-verticalization and creation of software ecosystems

Moore formulated an ecosystem as follows. “An intentional community of economic actors whose individual business shares in some large measure the fate of the whole community.” [Moore 2006]. He described how an ecosystem is a network of interdependent niches that are occupied by organizations. These niches are more or less open, to the degree that they embrace alternative competitors. A general characteristic of an ecosystem is that most actors benefit from innovations that give value to the end-user. For instance Eclipse [Eclipse 2010], an open source multi-language software development environment, offers a user interface framework into which any company can add functionality though plug-ins, thus creating an overall product which serves a broad group of software developers [McGregor 2009]. For economic reasons, it is not feasible for one company to create all this functionality on its own. In earlier work [Bosch 2009], one of the co-authors identified that a company could use the notion of ecosystems to expand the variability of a product line by encouraging other companies or end-users to add functionality. In her analysis of the IBM System/360 mainframe computer, Baldwin observed that the de-verticalization of the industry and formation of an ecosystem was enabled by its high degree of modularity. Baldwin came to the conclusion that a company could increase its return on capital substantially by becoming a dominant leader in a cluster by setting industry standards and by using designs from other companies [Baldwin1997]. An important prerequisite for an industry to be organized as modular clusters is that the interfaces between the layers must be standardized. This makes it possible for a system integrator to integrate software and hardware from different suppliers easily. Also, it allows an application developer to easily distribute a software solution to a large group of platform users. These characteristics explain the huge commercial success of computers based on the Intel and Microsoft platform. Christensen et al. [Christensen 2002] analyzed several cases of de-verticalization, in their paper denoted as disintegration, and identified that a vertical industry structure was needed when the performance demands didn’t allow for a modular architecture. Once the performance delivered exceeded the customers’ expectations, modularization of the architecture was possible and a more horizontal industry structure emerged. Raskin and Mellquist identified the enablers for the de-verticalization of an industry, including standardization and modularity. They also identified stages of de-verticalization, which could start with outsourcing and identified that there are companies that recognize an opportunity to use a third-party supplier to gain an advantage over their competitors [Raskin 2005]. Messerschmitt and Szyperski [Messerschmitt 2004] identified the factors that govern ecosystems and the role of architecture since it defines the system, as well as the structure of the ecosystem. The typical layered architecture results in less efficient implementations, but in many cases Moore’s law allowed this form to emerge. Messerschmitt and Szyperski identified that the suppliers of middleware and spanning layers, i.e. layers that connect application layers to platforms, could become dominant players in the market.

27

Chapter 2 The concept of ecosystems has also emerged in E-business, here denoted Digital Business Ecosystems (DBE). The introduction of internet and modern ICT tools allows companies to interact more easily and create a network of companies that deliver shared products and/or services [Nachira 2007]. Similarly to the case of software ecosystems, technical standards are needed for a DBE to prosper. In the DBE domain the standards relate to data interchange between companies, shared databases [Dourmas 2009] and common reference architectures [Janjua 2009].

2.2.4

Research method

The research reported in this paper uses a case study research method as proposed by Yin [Yin 2003] and Eisenhardt [Eisenhardt 1989] and is based on two cases. The research questions for these case studies are based on the comparison with the transition in the computer industry [Grove 1996], the role that modularization and standardization plays in such a transition [Baldwin1997, Christensen 2004, Messerschmitt 2004, Raskin 2005] and the characteristics of embedded systems. The research questions are defined as follows:  What transitions between industry structures have taken place? What where the forces and actors involved?  How were the architectural interfaces between the actors defined, and what were the consequences for modularity, variability and product integration? As the first case study we selected the transition of the development of televisions for the following reasons: (1) It describes the transition of a mainstream consumer electronics product with a significant amount of software; (2) The transition took place over a long period of time and involves multiple geographic regions; (3) We have access to detailed information on this case, being members of the involved organizations. For this case study we used multiple sources: we acted as a participant-observer, we conducted interviews and we applied reviews on our findings within the involved organizations. We used information from internal presentations and technical descriptions as well as external literature. Much of this information is classified, and we therefore only make references to literature that was published outside the involved organizations. As a second case study we selected the development of Mobile Phones for the following reasons: (1) mobile phones contain a significant amount of software and the market was initially dominated by a small group of vertically integrated firms (2) the product category is relatively young and has gone through a rapid evolution in an overlapping, but different time period, (3) the type of product contrasts that of a television, since mobile phones have a different feature set and business model (4) a significant amount of literature is available for this industry, covering multiple players. The data for the case study on mobile phones is exclusively derived from descriptions in literature. By using significantly different cases and by using different data collection methods, our theory provides a strong basis for external validity [Eisenhardt 1989, Yin 2003].

28


2.3

Case Studies: Transition in the Development of Digital Televisions and Mobile Phones

This section presents an overview of the transitions in the development of software for digital televisions and that for mobile phones. In the Sections 2.5 and 2.6 we provide a more detailed description of each case study.

2.3.1

Transition in the development of digital televisions from 1990 until 2010

The examples relate to experiences at Philips Electronics and NXP Semiconductors, which were top-5 organizations, based on market share. At the time that the case study commenced in the early 1990s, Philips’ Product Divisions included Consumer Electronics and Semiconductors. Philips Semiconductors was spun out in 2006 to become an independent company, NXP Semiconductors. For reasons of clarity, these steps are described as discrete and sequential steps. In reality most of these steps have been gradual and sometimes overlapping. 1. Analog products for specific regions. Televisions were initially mainly analog devices and most functionality was determined by analog hardware components. Because of the different TV standards used in different regions, product development in, for instance, Europe and North America was independent. A modest portion of the functionality was implemented through software and a single team created the entire product. 2. Increasing software content and global development. As processing power became cheaper, TV software became more elaborate and the size of the software grew rapidly. It was no longer economic to support independent developments for different regions so an intra-organizational software product line approach was adopted. The systems were still developed by individual firms that created the entire product, but the development was distributed over multiple sites around the world and served products for each geographic region. This transition took place in the period 1995 to 1998. 3. Integration of connectivity features. COTS (commercial, off-the-shelf) components and software from independent software vendors were integrated in the system, e.g. for Wi-Fi connectivity and photo browsers photos in TV. The architecture was maintained by the integrator, i.e. the TV set maker, and was adapted to integrate the third party software. This transition took place around 2003. 4. Specialized manufacturers of Systems on Chips. The reception of digital broadcasts by TVs favored a move to processing video in software. This resulted in a transformation from a set of analog processing ICs to a single, digital System on Chip (SoC) solution. Around 2006 the television manufacturers spun out their IC manufacturing. As set makers have different requirements and software architectures, IC manufacturers created specialized software variants for each set maker. The overall software architecture was a combination of that of the set maker and the IC manufacturer. 5. Integrators and suppliers of middleware. Around 2008, independent software vendors (ISVs) and software integrators entered the TV market. These vendors

29

Chapter 2

6.

developed a substantial part of the software and determined a large part of the functionality of the product, e.g. middleware software. The overall architecture now had contributions from the IC manufacturer, the middleware supplier and the set maker, requiring a staged integration. Attempts to create industry standards. To reduce the time needed for software integration, to serve more customers and to avoid dependency on specific ISVs, set makers attempted to create industry-wide standards. Due to commercial interest, the need for optimal resource utilization and fast innovation speed, none of these attempts has been successful.

2.3.2

Transitions in the development of mobile phones from 1990 until 2010

This case study relates to the development of mobile phones and encompasses that of the major players in this industry, including the new entrants. As in the case of digital televisions, these transitions have not taken place in discrete and sequential steps. Furthermore, the companies that initially acted as the vertically integrated companies executed these steps differently. 1. Small group of vertically integrated companies. Initially mobile phones were developed by a small group of vertically integrated companies that developed the entire product and sold these to their customers, usually through the Mobile Network Operators. The functionality was limited to that of voice calls and SMS. 2. Introduction of functionality outside the traditional telephone area. New functionality was introduced to mobile phones, such as cameras, downloadable ringtones and MP3 playback. The vertically integrated companies used specialized suppliers for these components, mostly originating from consumer electronics firms, and integrated these components into their products. This transition happened around 1996. 3. Introduction of smartphones / creation of mobile platforms. Functionality that was originally used in Personal Digital Assistants (PDAs), such as touch screens, Wi-Fi connection and GPS was incorporated into mobile phones. This introduced the need for dedicated application processors and operating systems. Separate hardware and software platform suppliers were created, often through spin-offs of the vertically integrated companies. The overall software architecture was now controlled by a combination of the mobile phone manufacturers, the operating system vendors and the platform suppliers. This transition happened around 2002 and still exists today. Nokia, the company with the largest market share worldwide, acted much longer than other mobile phone manufacturers as a company that owned full control over the software architecture. Two firms, i.e. RIM and Apple still control the overall architecture and use specialized suppliers for specific parts of their products. 4. Connection to internet and third party application development. The availability of third Generation (3G) technology made it possible for faster connection to the internet. An API was created that enabled third parties to develop applications that the end-user could download and purchase through the internet, thus forming an ecosystem. This situation emerged around 2008 and still exists today.

30

The changing industry structure of software development for consumer electronics 5.

Attempts at industry standardization. Several attempts for industry standardization have been made since the year 2000 in order to create a modular architecture, allowing for easier product integration and multiple parties to contribute more easily. For instance several attempts have been made to define a common software platform. Most of these attempts failed to gain industry-wide attraction and where abandoned after a few years.

2.3.3

Analysis of these case studies

From these case studies we learn the following:  The transitions took place in several small steps. The initial steps of deverticalization started when functionality outside the traditional domain was introduced for which specialized suppliers where used.  The fast growth of the amount of software started when functionality was shifted from being implemented in hardware towards software. Product manufacturers had to focus their investments and spun out their IC business. Also, software integrators, middleware suppliers and operating systems vendors emerged. The overall software architecture was now controlled by several parties. Consequently products are created through a supply chain where each customer receives specialized components from its suppliers.  Standardization attempts for creating a modular architecture across the industry have failed due to different viewpoints of the involved parties and rapid changes of the technology.  For the development of downloadable applications, ecosystems have been created, currently mainly for mobile phones.  There is a great deal of similarity between the transitions in these two case studies, but the transition of mobile phones occurred at a much faster pace because the Mobile Network Operators enabled large investments. Furthermore some players act as more vertically integrated companies.

2.4

Model of Industry Structures for Software Development in Consumer Electronics

When analyzing the case studies presented in the previous section, we identified that various approaches are deployed by companies in the software development industry for consumer electronics. In this section we present a model that captures these types of industry structures and the rationale to move for an individual firm to move from one structure to another. We provide examples from the consumer electronics domain as well as examples from other software products to illustrate the different types. Our model is based on 5 “types”. Figure 5 gives a simplified graphical representation of how software architecture might look like for the different types of clusters. Here the different colored segments represent the relative size of the stack that different companies contribute to the system.

31

Chapter 2

Vertically Suppliers Integrated and Firms Integrators

1

2

Supply Chain

3

Open Closed Ecosystem Ecosystem

4

5

Figure 5 Model of industry structure types, visualized Type 1: Vertically Integrated firms The vertically integrated firm develops their entire product and delivers it to the market. An integrated firm fully owns the specification, architecture and develops the software based on its customers’ needs. Characteristics: Complete control over the architecture; close contact with the final customer; all functionality has to be developed by one company. Examples: Digital televisions and mobile phones during the early nineties; the computer software industry in the 1970s; products with small dedicated software such as coffee machines, shavers and pacemakers. Type 2: System integrators and specialized suppliers This structure consists of system integrators that, like the vertically integrated firms, are responsible for the end-solution. The system integrators use components from specialized suppliers to reduce their own development effort and/or when the knowledge is not available to develop that part of the product. The system integrator is responsible for the requirements of the product and the high-level software architecture. It delivers the product to the market. Characteristics: The system integrator has complete control over the overall architecture and has close contact with the final customer; a part of the functionality is developed by third parties; one party, typically the product manufacturer, acts as integrator, using one or more suppliers. Examples: Digital televisions between 2003 and 2006; the majority of mobile phones between 1996 and 2002 and Nokia until around 2008, manufacturers of PDAs or smart phones, such as Apple and RIM (see Section 2.6). Type 3: Supply Chains In this type the product is developed by a group of companies, each with their own specialty. The difference with type 2 is that the system integrator does not own the product architecture.

32

The changing industry structure of software development for consumer electronics Instead, the product architecture is shared by the players and might not be defined through rigorous industry-wide standards. The software is developed using supply chains, where the supplied software has to be tailored for each customer and integration may be done in stages. The software integration accounts for a substantial proportion of the development effort. Characteristics: The participants share control over the architecture; the functionality is developed by several parties; each party delivers specialized variants of products to the next party in the chain. Examples: Digital televisions, mainstream mobile phones; hard disk recorders, ZigBee wireless devices [Hartmann 2009]. Type 4: Closed Ecosystem In an ecosystem, the product is developed by a group of companies and the product architecture is defined through standards. In a closed ecosystem the ecosystem is formed around one specific vendor [Popp 2010]. This vendor opens up their platform, e.g. by offering an API or a domain specific language, so that other parties can add functionality for their own use. Other parties could be the customers, the end-users or third party software vendors. The advantage for the vendor is that it can offer a wider product line with less development effort and the customers or third parties can add functionality more easily [Bosch 2009]. Characteristics: Control over the architecture is owned by the party that develops the platform; third parties and customers can add functionality, based on close contact with the final customer. Examples: Microsoft Windows, some of Intuit’s applications, Apple iPhone applications, eBay, SAP. Type 5: Open Ecosystem In an open ecosystem the architecture is rigorously defined through industry standards and the several components are interchangeable. The standards are open, meaning that they are available for anybody to implement, well documented and unencumbered by intellectual property mechanisms [Messerschmitt 2004]. Open standards allow competition at the subsystem level, since suppliers can create competitive components. Hence the likelihood of one firm becoming dominant is reduced. The “openness” of an ecosystem is not an absolute because some of these properties can be relaxed making the standard less open but still not closed or proprietary [Messerschmit2004]. Characteristics: The architecture is defined through open industry standards. The integrator has close contact with the final customer and can create a product by combining interchangeable components; all parts of the system can developed by alternative parties and are interchangeable; Tailor-made solutions can be made by several parties for specific customer needs. Examples of open standards: The USB interface, JPEG, MPEG compression standards, HTTP and HTML web standards Examples of open ecosystems: Eclipse Development Environment, Web browsers and servers

33

Chapter 2

2.4.1

Rationale for transitions between industry structures

The types discussed categorize different approaches taken by companies, but over time the approach taken by a company or a company cluster evolves. Typically, an industry evolves from a vertically integrated to a more ecosystem-centric approach due to the increasing amount of functionality. However, there also are forces that restrain this evolution. In Figure 6 we show the forces that are driving companies to evolve over time. Vertically Integrated Firms

Suppliers and Integrators

Supply Chain

Closed Ecosystem

Open Ecosystem

Integrators view: Driving forces:

Driving Forces

(F4) Lower development costs & (F1) More efficient use of resources shorter time-to-market (F2) More freedom to innovate (F3) Better control over product quality (F5) More variability can be offered

Restraining forces (F6) Increasing time for integration (F7) Increased costs of interacting with suppliers (F8) A dominant firm may emerge

1

2

3

4

5

Figure 6 Forces of moving from one industry type to another The advantages of staying, or moving to a more vertically integrated approach can be summarized as follows:  Force (F1): Modular and layering introduces inefficiency in the implementation [Christensen 2002]. In a vertically integrated approach, direct control of the hardware is possible, thus allowing for better resource utilization.  Force (F2): In a supply chain or ecosystem centric approach there are fewer possibilities to change the system architecture and APIs since the industry structure depends on its stability. This may constrain innovation [Messerschmitt 2004] and impose the requirement for backwards compatibility. Two examples: the Eclipse development environment doesn’t support backwards compatibility, requiring that the whole ecosystem has to make changes for each major release to remain compatible; Android supports backward compatibility of the core API since that part of the implementation is controlled by Google [Halasz 2011]. One co-author, Bosch, identified that, in case of different evolving hardware configurations, compatibility of applications may easily be broken and could easily become a major source of inefficiency [Bosch 2009], which is a major concern for embedded

34

The changing industry structure of software development for consumer electronics devices. A company that uses a more vertically integrated approach has the freedom to innovate more freely.  Force (F3): The overall product quality depends on the combination of software from different vendors and failures often occur because of component interaction, unclearly documented APIs or unknown use of the system [Trew 2004, Trew 2006, Oberhauser2007]. A more vertically integrated company has control over the architecture and its constituent components and can therefore guarantee the product quality more easily. The advantages of moving to a more ecosystem-centric approach can be summarized as follows:  Force (F4): Specialized firms are used to develop part of the system. In this way the total development costs are reduced because an individual firm does not have to invest in developing the entire software stack [Genugten 2007].  Force (F5): Variability can be offered more easily to the customers by using components from different suppliers or by enabling third parties and customers to develop the functionality they need [Christensen 2002, Moore 2006, Bosch 2009].  The disadvantages, or consequences, of moving to a more ecosystem-centric approach can be summarized as follows:  Force (F6): Increased uncertainty and time needed for system integration [Bosch 2006, Underseth 2007]. This is especially the case in the Supply Chain, in which heterogeneous architectures have to be combined [Vitharana 2003].  Force (F7): Co-operating with suppliers leads to additional interaction costs [Raskin 2005].  Force (F8): When the industry adopts an ecosystem centric approach, some firm may be able to take a dominant position and create a monopoly [Moore 2006]. This is often possible for a firm that develops the middleware or the spanning layer [Messerschmitt 2004].

2.5

Transitions in the Development of Digital Televisions

In this section we present the rationales and consequences of the transition of software development for digital televisions using the historical perspective, as presented in Section 2.3 and present it from the viewpoint of Philips and NXP Semiconductors. We end this section with a summary.

2.5.1 2.5.1.1

Analog products for specific regions (vertically integrated) Forces and actors

Until the early 2000s, TVs were receiving analog TV signals and the processing also occurred in the analog domain. Televisions also supported broadcast data: closed captions in North America and Teletext in Europe. This was also primarily implemented in hardware. A small amount of software was required to provide a limited user interface and to set values

35

Chapter 2 in the hardware registers in response to user inputs. The cost of developing the software was insignificant compared with the manufacturing costs. Because of the substantial differences between the standards used in different regions, such as different standards for broadcasting signals, there was little commonality between the hardware used in the sets. The TVs were therefore developed locally for each region. The actors were the set makers that created the entire product, e.g. Philips, Sony and Zenith. 2.5.1.2

Software architectures

Most of the functionality and variability was determined by the selection of hardware components. Variability was expressed through different analog components on printed circuit boards. As more powerful control processors became cost-effective, a market developed which competed on software-intensive features. This led to an increase in code size, reaching 600KLOC by the end of the 1990s. Rapid prototyping was used, with code generation to compose “reusable components” [Jonkers 1993]. The method provided both the means to specify reusable components and the technology to bind them in different configurations. However, it did not provide a modular architecture nor a product line, with clearly defined commonality and variability. Individual development teams were located in different regions to serve their local markets, with no reuse between teams.

2.5.2

2.5.2.1

Increasing software content and global development (vertically integrated) Forces and actors

By the mid 1990s the amount of software in consumer products roughly followed Moore’s law [Ommering 2004] and, to contain development costs, a significantly higher level of reuse was required. Independent product development in each region became uneconomical and the, now shared, software architecture was expected to explicitly provision for the variability between regions. This need to support variability was reinforced in the early 2000s with the industry transitioning from CRT to Plasma, LCoS and LCD displays. To achieve optimal picture performance these displays require an optimized video processing chain under software control (F1). In addition, features from high-end products started to migrate down to the mid-range TVs, requiring sets in different ranges to have sufficient architectural commonality to facilitate the effective sharing of implementations. Finally, with a booming dot-com industry, hiring sufficient developers in one location become impossible and support for multi-site development became a priority. These trends led to more attention being paid to the development process and software architectures [Rooijmans 1996]. 2.5.2.2

Consequences for software architectures

While the previous TVs notionally had a layered architecture, the lack of dependency management and, in particular, explicit required interfaces, had caused architectural degradation [Linden 2000]. This led to two changes:

36

The changing industry structure of software development for consumer electronics 

Software product line engineering approaches were deployed [Pohl 2005A] using a new architecture, based on reusable, configurable sub-systems. Each sub-system could be allocated to a different location and could be developed with limited communication. The architecture supported variability through both variation and composition, allowing the large range of product line variations to be supported by exchanging components or sub-systems.  A component model, Koala, [Ommering 2004] was introduced. This supported variation and composition with minimal run-time overhead (F1), allowing software component technology to be used down to the mid-range TV segment. The general form of the TV architecture is shown in Figure 7. In this, “middleware” is the term used to denote the layer that provides services such as TV channel installation and selection and the electronic program guide.

Middleware A/V API

OS

Application Layer

Software Platform Hardware Figure 7 Reference software architecture of a TV developed in a single company Crucial to the success of this architecture was the definition of an Audio/Video API (A/V API), which was sufficiently abstracted from the hardware to be stable, despite the variation in the hardware. Interface stability was achieved by including all the logic for coordinating the hardware components in the platform. The result was a platform with a high degree of modularity with few constraints on its usage (F1).

2.5.3

2.5.3.1

Integration of connectivity features (transition to suppliers and integrators) Forces and actors

Around 2000 digital video broadcasting and connectivity features were maturing. Increasingly users started to watch TV via a PC. As a response to this threat, also known as a “battle for the living room” [Waters 2011], the TV manufactures incorporated new functionality in the TV, like USB connections, Wi-Fi and Internet on TV. Finally, the penetration of terrestrial digital video broadcasting had grown to the point that there was also consumer demand for integrated digital receivers in TVs. The competitive advantage moved from the quality and cost-effectiveness of the video processing to new interactive

37

Chapter 2 applications and conditional access for pay TV, which provided additional revenues. This functionality resided in the middleware layer, for which implementations had already matured in set-top boxes. As certain digital broadcasting features differ by region, in-house development of all variants started to become prohibitively expensive. Instead, the required functionality was delivered with lower development costs by integrating software from external suppliers (F4, F5). TVs were developed by the same small group of vertically integrated firms as in the previous stage, but specialized independent software vendors provided specific parts of the software. The TV manufacturers had to develop new skills, such as software vendor selection, while the engineers were faced with the technical challenges of integrating of 3rd party components (F6) [Trew 2006]. 2.5.3.2


The components providing the new features were of a relatively small size and the supplied software was integrated into the in-house developed system by using alternative components from different suppliers (F5) [Hartmann 2009]. During the evaluation process of the software, the variability offered by the suppliers was taken into account, as well the fit of the purchased software into the existing architecture (F7) [Pohl 2005A]. Few of these features had standard APIs and the decision was made to wrap the supplied software to form Koala components [Ommering 2004], so that the product integration and configuration was unchanged (F6). To avoid vendor lock-in, these wrappers might have to perform significant transformations to the native API, allowing alternative suppliers to be substituted easily. No changes to the layered, modular architecture, presented in Section 2.5.1.2, were required. Given the relatively small size of the wrapper software in comparison to the overall software stack this was an economically feasible approach.

2.5.4

2.5.4.1

Specialized manufacturers of system on chips (transition to supply chain) Forces and actors

The advent of the digital video broadcast reception in TVs favored a move to processing video, from both digital and analog broadcasts, from dedicated hardware to software executing on specialized digital signal processors. This change allowed the transition from a set of dedicated video processing ICs to a single system on chip (SoC) with peripheral ICs. The SoC contains several hardware building blocks on a common bus. These blocks are either general-purpose computing elements, such as DSPs or the control processor, or dedicated hardware functionality, such as MPEG decoding. Peripheral ICs are dedicated ICs such as tuners, analog video decoders and ICs for audio and video output. These large SoCs are able to cater for a wide range of functionality and with huge variability between the several products and differences in price (F5). These variations, for instance, related to the display of video (high definition, screen size, picture in picture), audio capabilities (Dolby surround sound, equalizers), user interface, and connections to other devices (USB, Ethernet, HDMI). The underlying differences between TV standards

38

The changing industry structure of software development for consumer electronics between geographical regions, different set makers and market segments are now largely accommodated through variation points in the software. The choice of peripheral ICs vary depending on the exact TV set requirements. In the new architecture, software now has time-critical requirements to be able to guarantee the smooth rendering of the processed video and resource management becomes more critical, requiring detailed insight into the intended use of SoCs (F1). While the new hardware architecture allowed the total manufacturing costs to be substantially lower, the development costs for hardware and software became significantly higher. In order to amortize such investments most consumer electronics firms separated, their semiconductor divisions into independent companies that could serve multiple customers (F4) [Nagrani 2007]. A few examples: Siemens spun-off Infineon, Mitsubishi and Hitachi together spun-off Renesas and Philips spun-off NXP Semiconductors. Note that this not only impacted DTVs but all kinds of consumer electronics products. As a result, three types of actors now constitute the supply chain of DTVs: 1. Set makers that created the middleware, application layer and offered the end product to the consumer market 2. Manufacturers of SoCs, i.e. the platform suppliers, who also developed the low-level software and are typically responsible for integrating it on the hardware. 3. Specialized suppliers of software components, as mentioned in the previous stage. 2.5.4.2


In the supply chain that has emerged, the products have a heterogeneous architecture consisting of sub-architectures from different parties. The typical split in responsibility between the platform provider and the set maker is shown in Figure 8. The layers above the API are the responsibility of the set maker and below the API, the responsibility lies with the platform provider.

Application Layer

Glue

OS

Middleware API

Software Platform Hardware Figure 8 Reference software architecture of a TV developed in a supply chain There are no widely agreed standards for the interfaces between the platform and the higher layers, so some form of glue layer is normally required (F6). In the remainder of this section

39

Chapter 2 we will describe two particular challenges, namely the definition of an audio/video API and managing the variability across layers owned by different parties. Definition of the audio/video API. The API that was defined earlier, see Section 2.5.2.2, allowed the platform provider to have full control of the system behavior (F1). This was achieved by forcing the customers to enumerate the audio/video stream graph configurations that they wished to use. The platform provider could determine how these stream graphs could be implemented most efficiently and could fully validate them, irrespective of the behavior of the higher-level software. However, when selling to a broader market, this exposes the platform to much more variability, since different manufacturers have highly diverse requirements on how a TV behaves. For instance, a particular manufacturer will have specific requirements on how a tuner searches for a station, and not simply on the post-conditions of operations. The combination of these issues has resulted in the level of the platform API being lowered. This has two advantages. Firstly, most of the sources of customer-specific variability can be excluded, so that the platform software can remain stable and the customer-specific variation points are under the control of a single party (F2). Secondly, it is easier to bridge the gap between components at different levels than to bridge between provided and required interfaces that are at the same level but which use different concepts (F6). As an example, rather than providing search functionality for the TV’s tuner, only the capabilities of setting a specific frequency and detecting the presence of signals are offered, with the customerspecific searching functionality being implemented at a higher level in terms of these primitive operations. The disadvantage of lowering the API is that the client can inadvertently configure the platform such that the performance requirements are not satisfied because of resource constraints. This requires greater cooperation between suppliers and customers (F6, F7). The issues encountered during an attempt to standardize the platform API are discussed further in Section 2.5.6. Managing the variability across layers. The split in responsibility between multiple parties now has consequences for managing the variability:  Because elements of features are implemented in each of the layers, a variation point in the application layer must be mapped down to platform and, in some cases, to the hardware configuration [Pohl 2005A]. Therefore, not only must the semantics of the diversity mechanisms be compatible across the layers, but also their implementation.  It is usually necessary to apply staged configuration [Czarnecki2005]. In the DTV case the platform supplier makes the early configuration steps, including setting variation points that were imported from suppliers earlier in the supply chain, leaving other configuration decisions to the set maker.  To protect its intellectual property, the platform supplier may be very reluctant to release source code. Consequently, the implementation of variation points through simple conditional compilation is rarely applicable. Since the Koala component model, which was used in the fully integrated company, requires that the final product integrator has access to all the source code this proved less appropriate in a software

40

The changing industry structure of software development for consumer electronics supply chain. Therefore greater use is made of run time binding, using a component model similar to Microsoft COM, but adapted for resource-constrained products. Furthermore, the use of Koala also dictated that customers used this technology and, since it was not a widespread technology, this was unacceptable.

2.5.5

2.5.5.1

Integrators and suppliers of middleware (transition to supply chain with more parties involved) Forces and actors

Over time, the functionality of the middleware layer continues to increase, e.g. with the ability to browse content, e.g. photographs, stored on a networked PC, and the management of video recordings on integrated hard disks. A group of specialized development and integration companies emerged, e.g. LogiWays, and Shanghai-BMC, offering a complete middleware layer, rather than just specific features. Therefore TV set makers no longer needed to invest effort in maintaining the middleware (F4) and, as a consequence, the supply chain described in Section 2.5.4.1 now also includes the middleware vendors. 2.5.5.2


In the newly created supply chain there are various parties that can be responsible for the software integration. The development of the different layers of the software stack can be allocated to suppliers in different ways (see Figure 8). Some set makers develop both the middleware and application layer, others only the application layer. Some SoC manufacturers develop both the platform and middleware layers, other only the platform layer. In principle, the integration of middleware and application layers could be performed by any of these players, including the middleware vendors. An advantage for a supplier, e.g. of the SoC or middleware, to do the integration is that they maintain customer intimacy, allowing this supplier to discuss future roadmaps with the set maker [Messerschmitt 2004] and to be able to respond to market demands promptly. In Figure 9, the glue layer connects the Middleware to the platform API. There are economic advantages for the middleware supplier if it owns responsibility for the glue layer because parts of their revenues are obtained for creating the glue. Like the middleware itself, the glue code is proprietary software, whose source code is not normally provided to the other parties.

41

Chapter 2

Middleware

SoC Glue

API Drivers for Front end Peripheral

NXP: SoC Platform

Drivers for Back end Peripherals

Figure 9 Middleware and platform with glue layer A platform provider will tend to prefer to work with a small number of middleware suppliers to reduce the number of architectural mismatches that must be identified and resolved (F5, F6, F7). Therefore the platform provider will prefer to promote existing integrators, but will not wish sales to new customers to be dependent on the availability of that integrator’s engineering effort. Middleware

Front end End Glue

SoC Glue

Back end Glue

Drivers for Front end Peripherals

NXP: SoC Platform

Drivers for Back end Peripherals

API

Figure 10 Split of glue software and platform While the platform provider will have a preferred selection of peripheral ICs, the set maker sometimes selects alternative ICs and the platform provider has to integrate these ICs and their corresponding software drivers into the platform. The peripheral ICs can be partitioned into Front End (FE), such as tuners or analog video decoders, and Back End (BE), such as audio and video output ICs. Therefore, the platform software should be modularized in the same way, as shown in Figure 10. However, to reduce the dependence on the middleware integrator, it is also necessary for the glue to be modularized, including the coordination logic, and for the source code of the FE and BE glue code to be available to the other parties. At this stage the longer integration time was noted [Raaijmakers 2009], causing delay in market introduction (F6). Had a standard platform API been established, this would have removed the need for glue code and enabled an open ecosystem. However, as will be discussed in Section 2.5.6, to date no standard has gained traction in the TV market.

42


2.5.6

Attempts at creating industry-wide standards

As the industry structure began to shift, the SoC suppliers and set makers would have the greatest opportunity to increase their market through the standardization of a platform API. This would enable a market for new middleware vendors, whose products could be readily hosted on any compliant platform, allowing set makers to select components from different suppliers and integrate them in a straightforward manner (counteracting F6). A group of semiconductor companies and consumer electronics companies started a forum to define an industry standard for consumer electronics devices, called the Universal Home API, or UHAPI [Uhapi 2010]. The objective was to define a hardware-independent industry standard API for audio/video streaming platforms. This API is orthogonal to most the kernel of most operating systems being used, such as VxWorks, pSOS and eCos, as illustrated in Figure 8. When developing such an API, it is crucial to agree on its level, and therefore the split in responsibilities between the middleware and the platform. Previous architectures had exported separate drivers for each of the audio and video processing functions. This had exposed the difficulties of configuring the platform for the different combinations of active features required, which was compounded by the reluctance of the customer to reveal their planned usage of the platform. The semiconductor suppliers however, wished to raise the level of the API to ensure that it did not dictate a specific hardware architecture and that it provided guaranteed system performance (preserving F1), as described in Section 2.5.4.2. Consequently the UHAPI was defined [Uhapi 2010]. While optimal from the platform provider’s perspective, this approach had a number of disadvantages:  Some set makers were reluctant to reveal their exact requirements; for fear that they would be disclosed to competitors (F8).  For some middleware vendors, the development of the glue code was their primary source of revenue, and this work would no longer be required.  The model of selecting an enumerated stream graph was a poor match with the streaming frameworks in the general-purpose operating systems, such as Linux and Windows Embedded CE, which were starting to be used in consumer products. Although this API was ultimately standardized as ISO/IEC 23004, it failed to gain traction in the market. Other industry-wide API standards in this application domain face other challenges, due to rapid changes in technology. For instance, the currently agreed version of “OpenMax” standard for A/V stream processing components [OpenMAX 2010], does not support multiprocessor architectures or quality of service management. The functionality in consumer electronics products still isn’t stable and therefore no industry standard can be expected in the near future (F2). Another, more speculative, reason why attempts for standardizations of APIs have failed is that the several actors are afraid that one of the actors, e.g. a middleware supplier, might become the dominant firm and take the majority of the revenues (F8).

2.5.7

Summary of the case study of digital televisions

Initially, the amount of software for TVs was small. When software replaced functionality previously implemented though hardware, the amount of software grew quickly and a

43

Chapter 2 layered architecture was introduced, as well as a formal approach to software product lines. Soon after this step, software was integrated from specialized suppliers to provide functionality that was already available in PCs and set-top boxes (F4, F5), thereby requiring small architectural changes. When the IC manufactures where separated from the TV set makers, a supply chain was created and the control over the architecture was shared between the participants. Attempts at standardization of the audio/video API failed because more control over the system resources was demanded by the next participants in the chain (F1, F2). As a result, it became increasingly difficult for platform providers to guarantee failure-free system operations (F3). Independent middleware vendors emerged, whose software was integrated into the TV. This again caused longer integration times (F6) due to the increased interaction between the participants (F7) and the need to combine heterogeneous architectures. The use of supply chains is likely to continue for the foreseeable future. It will remain difficult to establish stable modular interfaces, given the increasing demand for additional processing power to support increasing requirements for video processing, e.g. 3D and QuadHD video and interactive applications, e.g. YouTube on TV.

2.6

Transition in the Mobile Phone Industry

In this section we present the rationales and consequences of the transitions in software development in the mobile phone industry, using the historical perspective presented in Section 2.3. In contrast to the case study of digital televisions we will look at all the major players and focus on the strategy of the players in dealing with the consequences. We will conclude the section with a summary.

2.6.1

First mobile phones (vertically integrated)

The early-generation mobile phones supported mainly voice processing. These phones consisted of a discrete single CISC-based microcontroller (MCU) core controlling a number of analog circuits. The move from analog to digital technology drove the need for a digital signal processor (DSP) core to be added to the architecture [Freescale 2008]. The first manufacturers of mobile phones, e.g. Nokia, Ericsson, Motorola and Siemens, were large, vertically integrated companies that owned the entire development, manufacturing and sales process [Bruhl 2005]. During the 1990s end-users and mobile network operators were willing to pay for enhanced functionality and usability. To meet these demands, products were made with customized components since standardized components could not meet the high performance requirements (F1, F3) [Anderson2006]. Hence a vertically integrated firm was the most suited industry structure because only large companies could make these investments. These companies also developed their own software, e.g. hardware drivers, real time operating systems and user interfaces. Different products were made to serve different standards and market segments. When the amount software grew, product line engineering

44

The changing industry structure of software development for consumer electronics was applied to reduce development effort for creating variants of the products. An example of this is Nokia [PHOF2010].

2.6.2

Introduction of extra-domain functionality (transition to suppliers and integrators)

Over time, additional functionality, arising from other consumer products, was added to mobile phones, now called feature phones. To deliver this new functionality, such as cameras, and FM radios, to the markets quickly, components were used from specialized suppliers (F4). To name a few of these components: real time operating systems, Bluetooth, camera sensor and video playback, color displays, R.F. transceivers, power amplifiers [Anderson2006]. The specialized suppliers mostly originate from other consumer electronics manufacturers, e.g. Philips, Samsung, Hitachi, and software vendors, e.g. related to video and audio encoding. The vertically integrated companies now used specialized suppliers and integrated these components, both hardware and software, into their existing architecture. The baseband processor and real time operating system that were already used were still capable of supporting this additional functionality [Infineon 2010] and the changes to the software architecture where limited (F3).

2.6.3

Smartphones (transition to supply chains/suppliers and integrators )

With the introduction of data and multimedia services, a mobile phone must interface with E-mail systems, the Internet and be able to handle a wide range of non-voice content, such as music, videos and games. As a result, several new technologies must be incorporated, such as GPS, e-mail clients, web browsers etc. [Kornby 2005]. This development was encouraged by the Mobile Network Operators, who offered phones at low price with a subscription, because the new functionality required large amounts of data transfer which was a major source of revenue [Rassweiler 2009, Strat 2008]. This functionality was originally used in personal digital assistants (PDAs) and personal computers and introduced the need for dedicated application processors, operating systems and software applications [Freescale 2008]. As an example, the first smart phone developed by Nokia was made as a combination of a PDA by Hewlett Packard and a phone by Nokia (F5) [NokiaHP 1996]. The development investments became significantly higher, both for hardware and software, and the vertically integrated companies needed to decide where they could excel and choose their own focus in this value chain (F4). Separate platform suppliers were created through spin-offs of the vertically integrated companies and specialized IC vendors [Kornby 2005]. For instance, Qualcomm focused on being a hardware and software platform supplier, Motorola spun off Freescale that focuses on delivering a hardware platform with software drivers, leaving integration to third parties. Ericcson Mobile Platform focuses on using hardware and software components from other suppliers (see Section 2.6.3.1) and so does MediaTek [Wang 2009].

45

Chapter 2 As a result, the industry transitioned and the suppliers, who defined part of the product architecture, determined a significant part of the functionality (see Figure 11, [Suaranto2006]).

2010

2000 Sony-Ericsson, LG

Motorola, Samsung

Nokia

•modem platform •Wireless standards •Application platform •Peripherals •Value domain SW/HW •Core applications •Operating Systems •Integration/assembly

Qualcomm TI Nokia Intel Samsung Nokia Ericsson Docomo Qualcomm Intel STm TI Qualcomm ARM Renesas Epson Samsung Toshiba Philips Matsushita Sony Philips STm Microsoft Nokia Real ATI Apple Microsoft Nokia Qualcomm Google IBM SUN Symbian Microsoft Linux Tron Palm Flextronics Huawei Nokia Motorola Samsung

Figure 11 Anticipated mobile industry transition The industry structure that was established around 2005 made it possible for new handset manufacturers to enter the market since components for mobile devices became commodities and hardware platforms became available. Examples of new manufacturers are TCL and Ningbo Bird, who bought complete hardware platforms focused on price differentiation [Anderson2006]. Another example is Apple, on which case we will elaborate in Section 2.6.6. Software Platforms and Operating Systems A mobile software platform typically consists of an operating system, middleware and base applications [Limo2010, Maemo 2011, Halasz 2011]. The PC Industry showed that the OS vendor could obtain most of the revenue, but in current days not only through licenses but also through the sale of applications and advertisements, such as with Android [Pang 2011]. The choice of consumers has very much shifted from a choosing a specific brand or network operator to choosing a specific software platform since the platform determines the applications that are available for that phone, see Section 2.6.4. For the development of a these platform both new entrants and existing players tried to gain a dominant position (F8). Examples are Google Android, Palm, and Windows. Also market leaders such as Nokia supported the development of open source mobile platforms, e.g. Symbian and MeeGo, to reduce their development effort (F4) [Bosch 2009]. Comparable to the case of digital televisions for middleware, a mobile platform supplier or handset maker has to integrate different software platforms to serve different customers (F5). Some examples (March2011): Samsung supports: Bada, Android and Windows [Samsung 2011], Nokia supports Windows, Symbian, Maemo, MeeGo [Nokia2011]. This leads to additional challenges for integration as shown in the following case study (F6). 2.6.3.1

Challenges for integration, case study from Ericsson Mobile Platform

Ericcson Mobile Platform (EMP) was transformed from an in-house development group for Ericsson into a supplier for different handset manufacturers. EMP develops a complete

46

The changing industry structure of software development for consumer electronics platform of ICs and software components, licensed from third parties, by integrating these components and customizes them for each customer [Kornby 2005]. A platform API is offered to the customers for developing applications on top of that, see Figure 12 ([Kornby 2005] © 2005 Ericsson Review, used with permission).

Customer and operator applications

Customer Customerand andoperator operator applications applications

Middleware services (Platform API) Network Access

Data Communication

User interface and codecs

Application servers

Operation Services

Hardware abstraction layer GPRS, EDGE, WCDMA Hardware

Figure 12 Software architecture EMP EMP takes the requirements of network operators into account, e.g. that the phones they sell contain specific applications to increase their revenues (F7). The variability that EMP offers to its customers is therefore related to the supplied HW and SW components (F5) and the customization that is done on the reference platforms. Three different product lines are offered, for Entry Phones, Feature Phones and Smart Phones, and different software platforms may be integrated to cover a wide range of customers. To obtain the best performance while minimizing the resources required, system use cases are obtained from the handset maker and network operators and these are analyzed, e.g. imaging, multi-player games etc. (F1) [Kovacevic 2007]. Also simultaneous usage scenarios are taken into account, such as simultaneous streaming and voice call, since these scenarios often require use of the same hardware resources, such as the DSP and the CPU, for different features. The NoTA Platform [Kronlof 2006] follows a similar approach. EMP face the same challenges as did the platform supplier NXP in Section 2.5.4.2 and 2.5.5.2: How to incorporate components from different suppliers (F5) to serve a wide customer range and maintain a platform API that enables third parties to develop applications, while assuring optimal system utilization (F1). The longer time required for integration and testing became visible in this industry (F6) [Underseth 2007].

2.6.4

Internet connectivity (transition to ecosystems for third party applications)

Third Generation (3G) communications standards enabled faster connections to the Internet. By giving third parties the possibility to develop applications that the end-user could directly purchase and download, an ecosystem was cultivated. In this way, the handset makers can

47

Chapter 2 offer more functionality to their end-users without making the development effort themselves (F4, F5). Handset manufactures such as RIM and Apple use this business model with their proprietary platforms, as did the open-source platforms, such as Android and Symbian. The rapid commercial success of the Apple iPhone can be explained because it made use of two existing supply chains, namely that of the handset, described in the previous section, and the internet supply chain through their existing iTunes store [Wehlage 2008, West 2010]. The industry structure for the majority of mobile phone industry is now a combination of two different structures. The structure for third party applications and the industry structure for the mobile devices. Here a decoupling point exists between that part of the functionality that has hard performance requirements (F1), e.g. speech processing and music playback, and the applications with only soft performance requirements, which do not require the tight balance between system performance and resource utilization.

2.6.5

Attempts at creating industry-wide standards

For wireless telecommunication standards, e.g. Open Mobile Alliance [OMA 2010], there is a broad adoption since that enables a mobile device to function in a network, but does not dictate software architectures. For the standardization of the hardware and software there are a number of different strategies of the different players. For instance the MIPI alliance aims to establish standards for interfaces [MIPI] and provides a standard interface to the OS. However these standards do not cover software or system architecture. Another standardization body, Khronos, includes the OpenMAX standard [OpenMAX 2010]. This standard, which addresses the API for multimedia streaming, gained only limited industry adoption, due to the rapid change of technology, e.g. for quality of service and power management. Several attempts have been made to define a common software platform in an attempt to replicate the Wintel model and create an ecosystem, by separating the supply of key components from hardware sales [West 2010]. The number of competing mobile platforms that have entered the market is huge, e.g. Palm OS, Google Android, LiMo, Symbian, Windows Mobile, Maemo, MeeGo etc., resulting in increasing fragmentation of the market. Often these platforms are supported by major contributors to create a dominant position in the market (F8), e.g. Symbian by Nokia, MeeGo by Nokia and Intel, Android by Google. Companies such as Apple and RIM challenged the need for standardization by offering more integrated and closed systems for the iPhone and BlackBerry, including OS, hardware, builtapplications, and online services [West 2010]. Most of these platforms offer a stable interface for applications, since that allows the applications to be used by different phones (F5). If the standardization of the interfaces is strictly controlled, this constrains the changes that a handset maker might wish to introduce to create unique selling points (F2). However, these platforms usually do not dictate a standardized interface towards the hardware since an embedded system developer requires a high degree of flexibility over the software to make use for optimal utilization of the hardware (F1) [Halasz 2011]. Over time some of these platforms gained a significant market share but where abandoned later because new functionality entered the market, e.g. multitouch screens, which could not easily be incorporated into existing platforms (F2).

48

The changing industry structure of software development for consumer electronics There are several reasons for the lack of standardization and this is why we believe that, except for the end-user applications, the forces in the mobile phone industry do not currently favor an ecosystem centric approach:  The fear that a dominant firm takes most of the revenue (F8).  The convergence of additional functionality from other consumer products can be expected (F4), e.g. from gaming consoles [Digitaltrends 2010] and therefore the speed of innovation will hinder the definition of stable interfaces (F2).  A high degree of modularity cannot be achieved because this would cause inefficiency and optimal use of resources remains an important design objective (F1).

2.6.6

Optimal industry structure for different market segments

Other authors have discussed whether a more vertically integrated or a more horizontal structure would be most suitable for the mobile phone industry [Anderson2006, Constantinou2010]. Using our model of industry structures and the forces, these strategic choices can easily be verified:  Anderson identified that for the entry-level devices, which contain little variability, vertically integrated firms offers better possibilities to obtain the lowest silicon costs because increased resource efficiency (F1).  Anderson and Constantinou argued that a more integrated approach might be more suited for high-end and new-to-market products [Anderson2006, Constantinou2010]. For this product range, bringing novel functionality to the market is more important than the higher development costs since the sales price is much higher than the manufacturing costs. As a small case study, consider Apple’s mobile products. Initially, Apple and Motorola co-developed the ROKR, a mobile phone with MP3 playback and iTunes connectivity [Vogelstein 2008]. The resulting device lacked sufficient innovative features to attract many customers (F2). Then Apple developed the iPhone, now keeping control of the hardware architecture, although based on a combination of existing hardware components from various suppliers. Moreover, Apple providing the software, derived from the Macintosh OS X. This degree of control enabled Apple to create a very innovative product and immediately attracted many customers. The development costs could be amortized because of the large market share that Apple has obtained and a huge margin on the manufacturing costs [Rassweiler 2009]. Furthermore additional revenues are generated by selling additional applications developed by third parties. Note that for the handset, Apple’s range contains very little variability, with the amount of internal memory being the principal variation point (F5). Although Apple has become an Integrator that uses specialized suppliers (F4), they continue to rely heavily on the closed ecosystem approach for downloadable applications.

2.6.7

Summary of the case study of mobile phones

Initially the size of the software was limited, so it could be developed by vertically integrated companies, and enabled them to use the system resources most efficiently (F1). With the

49

Chapter 2 introduction of feature phones, the players used components from other consumer electronics firms (F4, F5) but no major architectural changes where required. A major shift occurred when functionality from PDAs was incorporated, thereby requiring general purpose operating systems, which required large development investments. Several software platforms entered the market with the aim to create an ecosystem for the mobile phone handset. However none of these attempts have gained industry wide adoption and no high degree of modularization was attained. This led to the challenge to use different suppliers while assuring optimal resource utilization and, furthermore, an increase in the time needed for software integration was noted (F3, F6, F7). The supply chain is the dominant industry structure at this moment in time, although some players use a more vertically integrated approach (F1, F2). The need to ensure optimal resource utilization (F1), the ability to serve multiple customers and the lack of industrywide standards prevents the development of handsets from becoming more ecosystemcentric. The supply chain industry structure is likely to remain the dominant structure since no industry wide standardization may be expected due to the speed of innovation (F2) and the fear that a dominant player may take most of the revenues (F8). For the development of downloadable applications, closed ecosystems have been created because this functionality has no hard performance requirements nor does optimal resource utilization need to be guaranteed. There is a great deal of similarity with the transition of Digital Televisions, but the transition of mobile phones occurred at a much faster pace. In the mobile phone industry the Mobile Network Operators play an important role in the definition of the functionality and encouraged large investments. A similar situation is present in the development of set-top boxes where the service providers determine a significant part of the functionality. For digital televisions, however, the functionality is mainly decided by the set makers. Another difference is that, for digital televisions, no ecosystems have been created for downloadable applications, since this type of applications are not (yet) common for Digital Televisions. However, initiatives in this field have been started, e.g. by Google and Yahoo.

2.7

Related Work

Although there is some related work on ecosystems and supply chains in the computer industry and ICT, as well as documented case studies, little work exists in the domain of consumer electronics with resource-constrained devices. In the general management literature, software platforms are studied from a business strategy perspective. The work on Value Chains [Porter 1980, Porter 1980], Value Networks [Peppard2006], Value Chain Modularity [Sturgeon 2003] and platform leadership [Gawer 2002], focus on the strategy for an individual firm to become successful in an existing industry structure. This literature does not present a model of industry structures nor the forces that govern these transitions. One co-author, Bosch, provides a taxonomy for different forms of (closed) ecosystems [Bosch 2009] and described the successful examples in the domain of desktop and online software. Jansen et al. [Jansen 2009] discuss different types of software ecosystems and discuss the difference with a supply network, but do not address embedded systems, or

50

The changing industry structure of software development for consumer electronics industry transitions. Popp and Meyer also discuss closed ecosystems in the ICT domain and their work focuses on business models. Messerschmitt and Szyperski [Messerschmitt 2004] focus on open ecosystems. Although they describe the consequences of transitions, also these authors do not address the embedded systems domain. Also, they neither classify different types of ecosystems, nor do they discuss the differences between ecosystems and software supply chains. In the literature of digital business ecosystems [Nachira 2007, Dourmas 2009, Janjua 2009] the emphasis is on creating the ICT technology that enable an easier communication between companies which is needed for an ecosystem to prosper. In this domain the challenges relate to data interchange and interoperability, different governmental regulation, different business processes and so forth. The literature in this field does not address the de-verticalization of industries nor models that describe industry structures. Bruhl and Stieglitz [Bruhl 2005] characterize industries that undergo a de-verticalization and identify the role that an individual company can play and also use examples from the mobile phone industry. Christensen et al. [Christensen 2002] discussed the drivers behind de-verticalization and the role of modularization. Raskin and Mellquist [Raskin 2005] recognize stages of de-verticalization and the enablers for this de-verticalization. None of these authors identified the transition steps nor identified different industry structures. Krafft [Krafft 2003] identified different degrees of vertical specialization in the infocommunications industry, identifying a layered industry structure for equipment provision, network operators, up to content provision. His work looks at this industry as a whole and is not directed at the mobile devices. Suoranta [Suoranta 2006] envisioned a disruption of the mobile phone industry with opportunities for open innovation, based on a modular architecture at different levels of abstractions. Although the disruption has taken place, this has not led to the creation of a modular architecture, as we showed in Section 2.6. Anderson and Jonsson describe the transition of the mobile industry [Anderson2006] until 2005 and draw analogies with the PC industry and the role of modularity. They recognized that different companies and for different market segments a more vertically of horizontal industry structure is appropriate. Since their analysis included the period until 2005 it does not include the emergence of ecosystems for downloadable applications. Constantinou analyzed the mobile phone industry and recognized the move between vertical and horizontal structures [Constantinou2010]. None of these authors provides a model for different industry structures and describe the consequences for software architectures.

2.8

Conclusions and Future Research

The software development of consumer electronics industry has gone through a substantial transition in the last 15 years, passing through a number of structural types. In this paper we captured these structural types in a model of industry structures that we developed, based on case studies from digital televisions and mobile phones. While related art has claimed that a high degree of modularization and standardization is a driver and pre-requisite for deverticalization, we showed that that the de-verticalization of the consumer electronics

51

Chapter 2 industry started when functionality outside the traditional domain was introduced, despite a low degree of modularity and standardization. We demonstrated that the challenges in developing software for consumer electronics lie in providing the right balance between delivering the required variability in a timely fashion, while retaining sufficient control over the systems architecture to obtain optimal resource utilization. These characteristics originate from the nature of embedded systems that are designed to perform specific tasks, in contrast with general-purpose computers that are designed to be flexible and meet a wide range of needs. Furthermore, efforts for standardization have failed because of the rapid changes in functionality and the fear that a dominant firm could emerge. As a consequence the transition of the software development for consumer electronics has not followed a similar path to that of the computer industry, as envisioned by other authors. In this paper we described the forces driving towards a more ecosystem-centric approach, a vertically integrated approach as well as those forces that restrain changes towards a more ecosystem-centric approach. We showed how these forces drove the individual players in the industry of digital televisions and mobile phones to change their position. Furthermore we described the consequences for the software architectures during the transition steps. The model of industry structures we presented can serve the decision making process for an individual company to enter or change into a different type of industry structure. The forces and consequences we described provide insight into the benefits and expected problems when moving from one type to another. We showed that our model and forces provides a better decision support than related art does. Since the choices that individual companies make in choosing the industry structure has a large impact on their profitability, we believe that the current research is important for practitioners as well as provide a framework for researchers studying the software intensive industries. Preliminary research showed that our model and the governing forces are also applicable to the wider domain of embedded systems, given the commonality in characteristics between these products. More speculatively, our model might also be applicable for software intensive system in general, and we provided some examples of those in Section 2.4. However, the forces might be different from the ones presented in this paper and therefore further research is needed. Another important topic for future research is that an industry can adopt a hierarchical model where at different places in the architecture a different industry structures are used, thus leading to a more fine-grained model of that industry’s transitions. Furthermore, additional case studies are required to enhance the model into a full-fledged decision framework for companies transitioning into a different type of industry structure. The research presented in this paper is one of the first of its kind. While earlier research has described de-verticalization of industries, little work exists that analyzed the consumer electronics or embedded systems industry. Furthermore, none of the earlier work neither provided a comprehensive overview of the forces governing the transitions nor identified different forms of industry structures for software development of embedded systems.

52


Acknowledgements The Authors would like to thank Rob van Twist, Jack Goossen and Clara Otero for their contributions to the case study of digital televisions and the reviewers of the Journal for their comments, which helped us to improve the paper.

53

54

Chapter 3.

Consumer Electronics Software Ecosystems

This chapter is an extended version of the article which is published as: H. Hartmann, J. Bosch: “Orchestrate Your Platform: Architectural Challenges for Different Types of Ecosystems for Mobile Devices”, ICSOB 2014: 163-178, DOI 10.1007/978-3-31908738-2_12. This extension is submitted as: H. Hartmann, J. Bosch: “Towards a MultiCriteria Decision Support Method for Consumer Electronics Software Ecosystems” Abstract Due to the increasing size of software and because consumer electronics products are more and more connected to the internet, many consumer electronics firms are adopting an ecosystem centric approach for supporting 3rd party applications. In a mature market where successful platforms are available, a firm will mostly choose an existing platform. In an emerging market, a consumer electronics firm may need to create a new ecosystem or adopt a newly developed platform, both which has significant commercial and technical implications. In this paper we identify three types of ecosystems that are used today: Vertically integrated hard-ware/software platforms, closed source software platforms and open source software platforms. We intro-duce a first step towards a multi-criteria decision support method. This method determines which of the three types of ecosystem is most likely to become successful for a specific product category. The method is based on the scope and technology design of the platform and takes a software engineering perspective. We use this method to analyze a wide range of consumer electronics products. We show that the vertically integrated platform type is most suitable for product with a high degree of innovation and that open source software platforms are more suitable when a large amount of variants are needed to serve the market. The closed source software platform type is less suitable for consumer electronics, especially for products which require an optimal use of system resources and have a large degree of innovation. Consumer electronics manufacturers can use this method to select the ecosystems type that they want to adopt or create. Firms that create software platforms can use this method to select the product types they want to target.

3.1

Introduction

Consumer electronics products have radically changed over the past two decades. Initially the functionality of these products, including products such as CD and DVD players, televisions and radios, was mainly implemented through hardware. Due to the reduced costs

55

Chapter 3 for integrated circuits (ICs) and increased capabilities of these ICs more and more of the functionality is now implemented through software [Hartmann 2012], furthermore many products are now connected to the internet and support the downloading of applications in addition to the applications that are embedded in the devices. Initially the products were entirely developed by individual firms, but nowadays 3rd parties are used for the development of software components. Specifically for downloadable applications, it is not feasible for an individual company to develop these applications themselves. As a result, many firms are adopting an ecosystem centric approach in which 3rd parties are supported to develop applications for their products. In a mature market where there are existing ecosystems, a consumer electronics firm will usually choose an existing platform and this choice is largely determined by the commercial aspects, e.g. the amount of 3rd party applications and the license fees, and functional requirements, e.g. whether the platform supports the specific features that the a firm wants to offer. In an emerging market, i.e. when there are no existing or no dominant ecosystems, a consumer electronics firm may decide to create a new ecosystem and software vendors may decide to create software platforms to serve a this market. As an example consider the smart phone market. Nowadays that there are a few dominant ecosystems, and manufacturers of devices usually choose an existing platform, which is typically Android, Windows or Tizen. In the early years new ecosystems where created by consumer electronics firms such as Nokia, BlackBerry and Apple, and software platform vendors entered the market, e.g. Microsoft and Google. Some of these ecosystems became successful while other disappeared after a few years. The adaption of ecosystem has significant implications, both commercially and technically. For instance, if a company decides to open its own platform for 3rd parties, it has to attract sufficient parties and it has to maintain its own ecosystem. When a company decides to adopt an existing software platform, changes are required to the architecture and a significant part of the revenues may go the platform owner. Furthermore it has to be confident that the adopted ecosystem will become successful. The success of an ecosystem relies on many aspects which includes the scope and design of the platform, how the platform owner co-operates with the complementors, how each of the participants is able to obtain revenues, how the sales channels are organized and how the platform is able to deal with changing requirements [Popp 2010, Gawer 2008, Axelsson 2014]. In this paper we classify the type of ecosystems that are used for consumer electronics and identify the strengths and challenges of each type, thus serving as a reference for emerging markets. Furthermore we analyze, giving the requirements from the market, which of the identified types is likely to become successful for a specific type of product from a software engineering perspective. This paper answers the following research questions:  What types of ecosystems are used in consumer electronics products to support 3rd party applications?  Which of the ecosystems types is most suitable for a specific product category from a software engineering perspective? To answer the first question a variety of consumer electronics domains are studied, resulting in a taxonomy. To answer the second research question, we introduce a multi-

56

Consumer Electronics Software Ecosystems criteria decision support method [Gella 2006]. The decisions support method analyzes the scope and technology design for a software platform [Gawyer 2002] w.r.t. it’s abilities to meet the functional and non-functional requirements. Our method finds its origin in methods for selecting software components and software vendors [Nazir 2014, Alpar 1998] and similarly uses multiple criteria which are based on an expert opinion and are using ordinal scales [Cooper 2011]. The criteria to evaluate types of ecosystems are general market requirements [Bolwijn 1990, Sheat 2010], requirements from the nature of consumer electronic devices [Henzinger 2006, Trew 2011], and requirements from the use of ecosystems [Messerschmitt 2004, Bosch 2010, Underseth 2007]. For each type of ecosystem we identify the strengths and challenges from a software engineering perspective. The decision support method is based on comparing the requirements for a product type with the strengths and challenges of the ecosystem types. Since our decision support method does not cover commercial and organizational aspects we do not claim that our method is complete and this method is therefore a first step towards a full-fledged decision support method. This paper is structured as follows. In Section 3.2 background is provided on consumer electronics. In Section 3.3 a classification of ecosystems is given. Section 3.4 describes the first steps towards a multi-criteria decision support method. Section 3.5 describes the case studies. This paper is concluded with a comparison with related art in Section 3.6, and our conclusions and a description of further research in Section 3.7.

3.2 3.2.1

Background on Consumer Electronics Products Trends and challenges

Current consumer electronics products have characteristics of embedded devices as well as general purpose computing devices [Hartmann 2012]. As an embedded device they are aimed to perform dedicated functions, such as making telephone calls, playing games or movies, which have real-time performance requirements. There is a high pressure on cost price and therefore the integrated circuits (ICs) should operate as efficient as possible and using ICs with the smallest footprint achievable. Consumer electronics devices have constrained computing resources and should use as little energy as possible because they often have few options for heat dissipation. For mobile devices, which operate on batteries, this is even more important. As a consequence a consumer electronic product contains as little hardware and software components as possible. These devices also have characteristics of general purpose computing devices because they are meant for a variety of tasks such as Internet browsing, reading and writing documents, accessing social media and playing games. The combination of these two characteristics leads to numerous architectural and commercial challenges. As an example, consider Flash, an Internet browser plugin for watching videos. Apple has not allowed the use of Flash on their smart phones and tablets because it causes a significant shortened battery life [Jobs 2011], but has promoted the use of the H.264 video encoding standard for which their device is optimized.

57

Chapter 3

3.2.2

System architecture

In Figure 13 a high level system architecture of consumer electronics products is presented that support 3rd party applications, showing some of the actors for different product types. e.g. Samsung, HTC, Apple, RIM, Sony

e.g. Apple, LG, RIM

e.g. Windows, Horizon, Ubuntu

App. Device makers

App.

App.

App.

“Middleware” / Application Framework

e.g. Facebook, YouTube

e.g. Android, Tizen, Ubuntu

OS-Kernel

e.g. linux

Hardware

e.g. ARM, NXP

Figure 13 System architecture of consumer electronics products that support 3rd party applications The architecture consists of a hardware platform an OS-kernel and middleware that offers a framework for application developers [Halasz 2011, Trew 2011]. The hardware layer consists of a System on Chip (SoC) with peripheral ICs. A SoC contains several dedicated building blocks on a single IC such as audio decoding, memory and a configured CPU. A dedicated SoC, which is common for embedded systems, is better able to meet the performance requirements at lower power consumption than using a general purpose CPU and separate ICs, as in the PC industry. The design of the software and the SoC are tightly coupled, e.g. for controlling the power consumption of separate hardware building blocks.

3.3

A Classification of Ecosystem Types

This section describes the move towards ecosystem and provides a classification of ecosystems that are used today.

3.3.1

Growing software size: move towards ecosystems

In an earlier paper we analyzed the transition in the consumer electronics industry from 1990 until 2010 [Hartmann 2012]. Initially these products were developed by vertically integrated companies that developed the hardware, firmware, applications as well as created the end products. Examples of companies that create traditional consumer electronics products such as Televisions, DVD players and Radios are: Philips, Sony and Matshushita. Examples of companies that created the first generation of mobile phones are: Nokia, Siemens, Ericsson and Motorola. Over time, an increasing proportion of the functionality was implemented through software and this introduced the need for dedicated application processors and operating systems. The development investments, both for hardware and software, became significantly higher and therefore the companies needed to focus their activities. Separate

58

Consumer Electronics Software Ecosystems hardware platform suppliers were created as spin-offs of the vertically integrated companies, such as Qualcomm [Qualcomm 2014], NXP and Freescale. When hardware platforms and other components became commodities it was possible for newcomers to enter the market without large investments. Examples of these newcomers are TCL, HTC and Apple in the market of mobile and smart phones and Dell, Apple and Microsoft in the market of digital televisions and set-top boxes. Several software vendors entered the market and created a common software platform in an attempt to replicate the Wintel model [Grove 1996, West 2011]. Similarly as the case with Microsoft Windows, a firm controlling the spanning layer can earn most of the revenues since it controls the interface towards the hardware as well as towards the applications [Messerschmitt 2004, Baldwin 1997]. The number of competing mobile platforms that entered the market was significant, e.g. WebOS, Android, LiMo, Symbian, Windows Mobile, MeeGo etc. Similarly in the domain of digital televisions and set-top boxes, multiple platforms entered the market, such as Windows, Horizon TV and OpenTV. Until 2009 most of these attempts didn’t gain industry wide adoption and the industry was highly fragmented. In the mobile domain, starting in 2010, many of the vertically integrated firms lost market share and Android, an open source software platform, gained a share rapidly. Furthermore the dominant player in the PC market, Microsoft, increased their efforts to gain market share. For other consumer electronics products, such as digital televisions, and digital cameras a similar transition occurs in which vertically integrated companies start adopting open source software platforms [Engadget 2014]. As a result there is a fierce battle between different ecosystems in the market of many consumer electronics products.

3.3.2

Classification of ecosystems with their complementors

This paper discusses ecosystems for consumer electronics that are aimed to support 3 rd party applications. For a comparison of these ecosystems we use a classification that is based on two properties: (1) is the software proprietary or open source, and (2) are the hardware and device included. Figure 14 shows this classification with examples from the domain of smart phones.

Software Platform only

(2) Windows WebOS (2010-2012)

(3) Android, Tizen, Firefox OS

Hardware and device included

(1) Apple, RIM Nokia (< 2013)

(4) Not present in the market

Proprietary closed Source

Open Source

Figure 14 Classification of ecosystem types with examples from smart phones

59

Chapter 3 This classification results in four possible ecosystems: 1: Vertically integrated proprietary hardware/software platform. This platform consists of the hardware, proprietary closed source software and includes the device. The complementors are the application (App) developers. Examples of platforms and their leaders for smart phones: Apple with the IPhone, Rim with Blackberry and Nokia with Asha. Examples for digital televisions and set-top boxes: LG with WebOS, Apple with AppleTV, Samsung with Smart TV. Examples for personal computers: Apple with OS X. 2: Proprietary, closed source software platform. This platform consists of a proprietary closed source software platform. The complementors are the suppliers of hardware platforms, system integrators, device manufacturers and application developers. Examples of such platforms for smart phones: Windows Phone and WebOS (2010-2012). Examples for digital televisions and set-top boxes: Horizon TV, Microsoft TV. Examples for personal computers: Microsoft Windows. 3: Open source software platform. This platform consists of an open source software platform. The open source software platform is based on the concept that multiple parties can contribute to share development effort and since the source is open, the device manufacturers can change, add or remove functionality. The complementors are the suppliers of hardware platforms, system integrators, device manufacturers and application developers. Examples for smart phones are Android, Tizen and Firefox OS. Examples for digital televisions and set-top boxes: Android, Meego, Ubuntu TV. Examples for personal computers: Chrome OS, Linux, FreeBSD. 4: Open source software and hardware platform. This platform would consist on an open source software platform including the hardware and device. The users of such a platform, or better product, would be able to change the code of the software platform and add their own functionality. We have excluded this type from this paper as this type, to our knowledge, is not available in the market. Note that in the past some handset makers of smart phones used an open source platform of which they acted as the platform leader, e.g. Nokia that used Symbian when it was open source and Motorola of Google that used Android. In this situation the handset makers are in-house complementors of the open source software platform and therefore the combination is not a separate type of ecosystem.

3.4

Towards a Multi-Criteria Decision Support Method

This section describes the first step towards a multi-criteria decision support method for selecting the most suitable ecosystem type for a certain product category. This section starts with an overview of the method, followed by the background. Then each step is explained in detail.

60


3.4.1

Overview of the method

This method is based on comparing the strength and challenges of each ecosystem type with the market requirements of a product category. As shown in Figure 15, the method consists of 3 steps: Step 1: For a product category the most important requirements are determined. Step 2: The strengths and challenges of the three types of ecosystem type are identified for these requirements. Step 3: A decision support matrix is created that shows a total score per ecosystem type, based on the importance of the requirements and the strengths and challenges of the ecosystem types. 1: Identification of important requirements for a product category 3: Decision support matrix 2: Strengths and challenges of an ecosystem type for each requirement

Figure 15 Multi-Criteria Decision Support Method This method uses similar steps and techniques as those that are used for the selection of software suppliers and software components [Nazir 2014]. In these methods similarly a multi-criteria analysis [Gella 2006] is used to compare multiple criteria, or requirements, with the ability of the supplier, or software component, to fulfill them [Alpar 2010]. In these methods a decision matrix is used to rank the options so that the decision can be made more objective [Pugh 1996], and ordinal scales [Cooper 2011] are used as part of these methods since quantitative information is mostly not available [Goulão 2007]. The decision support matrix is used in the same way as these method to support the selection of alternatives based on a multi-criteria analysis, and the importance and evaluation of the criteria are based on an expert opinion. Since ordinal scales are used for the requirements and for the total score, the values do not represent absolute numbers but are solely used as a scale to express the differences between the types of ecosystems. Section 3.5 presents case studies for different product categories. Step 2 of the decision support method is independent from the product category and therefore the strengths and challenges of the three identified ecosystems are described in Section 3.4.4 and then used in the case studies in Section 3.5.

61

Chapter 3

3.4.2 3.4.2.1

Background of the method Platform scope and technology design

Gawer and Cusumano designed a framework for a company to become a platform leader. A platform leader is driving the innovation in the industry, ensures the integrity of the overall system and creates opportunities for firms that create complementary products [Gawer 2002]. This framework includes (1) how a company defines the scope of the platform, i.e. what part of the product is developed in-house and what it leaves for complementary firms, (2) the technology design, e.g. to what degree the platform should be open to the complementors and how the interfaces are defined (3) how external relations are managed (4) how the internal organization is structured [Gawer 2008]. The scope and technology design highly determine the success of a platform. It should support the capabilities of the platform leader, allow for complementors to contribute and be able to respond to changing requirements. While all three ecosystem types provide a platform for 3rd party developers, they differ in the contributions the complementors can make to the final product and to what degree the complementors can alter in the platform. For the vertically integrated platform type the scope is very wide and the complementors have no options to contribute or alter the platform. The closed source platform types provides the system integrators and device manufacturers the possibility to add distinguishing functionality but leaves little to no options to alter the platform, e.g. to make changes to the API (Application Programming Interface) and interfaces towards the hardware layer. The open source software platform provide the complementors the options to contribute to the platform and make changes to the platform thus allowing a large degree of freedom. The latter also leads to risks because the platform owner may lose control over the platform [Iansito 2004]. Since the scope and technology design constitute the largest differences between the three identified types of ecosystems, this is the basis of the decision support method. 3.4.2.2

Evaluation criteria

In order to evaluate the best suitable scope and technology design for a platform, it is needed to define the criteria that determine this. Here we draw an analogy between the selection of an ecosystem type and the selection of software components [Nazir 2014] and the selection of a software vendor [Alpar 2010]. A selection of software component is based on whether it fulfills the functional and architectural requirements as well it is able to address nonfunctional requirements. This set includes: Costs of the components, quality, whether it fits the architecture, is modifiable, maintainable, portable and so on [Kontio 2010]. A supplier selection is typically based on the abilities of a supplier to fulfill the functional and non-functional requirements [Alpar 2010] and includes the maturity of an organization, its customer base, and the way it cooperates with its customers. Since we use the requirements from the market we will further use the term requirements to describe the criteria. The requirements that will be used in our decision support method, is a combination of those that are used for component and supplier selection. Similar as with a vendor selection,

62

Consumer Electronics Software Ecosystems the choice for an ecosystem is based on its abilities to fulfill the functional and nonfunctional requirements. Similar as with the selection of components, an ecosystem type has to be able to provide a platform which fulfills the architectural requirements. For evaluating the non-functional requirements we use the competitive forces of Bolwijn and Kumpe [Bolwijn 1990] which determine the success of a consumer electronics firms: Cost, Quality, Flexibility and Innovation. In a later analysis Sheate [Sheat 2010] identified Variability as a requirement, which is strongly related to flexibility. In this context cost is interpreted as the software development cost since reproduction costs for software are negligible. The key architectural requirements for embedded systems and consumer electronics products are: Efficient use of system resources and hard real time requirements [Henzinger 2006, Trew 2011]. Furthermore we use requirements that are specific for ecosystems, namely: Stability of the interface since complementors have to rely on a stable and, ideally, backwards compatible interface [Messerschmitt 2004, Jansen 2013], and Effort for system integration, since for the set-makers the integration of the hardware and software is a substantial and increasing proportion of the development effort [Underseth 2007]. This gives the following set of requirements: Development Costs, Quality, Variability, Speed of Innovation, System resources and hard real time requirements, Interface stability and Integration effort. In the following sections, these requirements are elaborated. Other criteria that determine the success of an ecosystem are not covered since this paper looks from a software engineering perspective.

3.4.3

Step 1: Identification of the important of each requirement for a product category

The requirements are evaluated based on the characteristics of the product and the market demands. For each requirement, it is determined whether it is “key”, meaning that it is an absolute necessity to support this, or of “high” importance.  Development cost: This is based on the estimated size of the software, the effort that is required to maintain the platform and develop new functionality. This requirement will be evaluated as of high, or key, importance when the amount is large and continues enhancements are needed.  Quality: This is based on the costs when a product or one of its functions fail which includes the useability of a product and the commercial impact of product failures.  Variability: This is based on the amount of variants that are needed to satisfy the different market segments and customers. This typically includes different features, prize setting, screen size, the support of different regional standards etc..  Speed of Innovation: This is based on the speed at which new products with new features are introduced and whether the new functionality is a reason for consumers to buy a (new) product. E.g. when a new product replaces an older version within one or two years, and customers replace their old product, the speed of innovation is determined as of high importance.

63

Chapter 3 

System resources and hard real time requirements: This based on the pressure of the cost price of the ICs, the need to ensure a good system performance, and the requirements for ensuring a long battery lifetime for mobile devices. So for small products, such as wearables, this is a very important requirement and also for products for which there is a high pressure on the cost price. When real time aspects are important, e.g. to play music and games without audible and visual interruptions, this requirement is also evaluated as of high importance.  Interface stability: This is based on whether the API has to be rigorously followed for an application to work correctly and whether a high amount of 3rd party applications is needed to gain sufficient traction in the market.  Integration effort: This is based on the development and test effort that is required to create an end-product using its software and hardware components. This is a result of the amount of hardware and software components that are used and whether the interfaces between them are stable, i.e. whether a modular architecture has been created. For innovative products that contain components with new functionality and changing interfaces the integration and test effort is high. Note1: These requirements are not completely independent. For instance a high degree of innovation will cause higher development costs and integration effort. Note2: We do not further distinguish between requirements that are of “Medium” or “Low” importance, since for such a fine-grained distinction our method is not sufficiently accurate and complete at this moment in time.

3.4.4

Step 2: Evaluation of the requirements for each ecosystem type

In this section we analyze each of the three types of eco-system and each requirement is “scored” on whether it is challenging or easier to address, i.e. a strength of an ecosystem type. Here + means: easier to address, 0 means: neutral, and - means: challenging. This analysis is based on the experience of the authors, the information in literature and the case studies that are presented in our earlier work [Hartmann 2014]. A: Development Cost. The larger the scope of the platform, the more development and maintenance effort is required by the platform owner. When multiple parties are involved these costs can be shared between the participants [Genuchten 2007]. Because hardware and software development is costly, this is the main reason why ecosystems became widely adopted. Firms that aim for low costs specialize on a few products so that tasks become routine [Bolwijn 1990]. Consequently products that are developed in consort by specialized firms, especially when interfaces are pre-defined, can be made at lower costs. For the vertically integrated type this requirement is challenging since they develop the entire platform including both hardware and software and can only amortize their investments over their own devices (hence score = -). The open source platform can more easily address this, especially when the complementors contribute to the platform and often individual developers develop code in their free time [Gawer 2008] (hence score = +). For the proprietary software platform we evaluate this as neutral (score = 0): Although this

64

Consumer Electronics Software Ecosystems platform owner has to develop the platform on its own, the costs can be amortized over multiple products from different device manufacturers. B: Control over Quality. The overall product quality depends on the combination of software from the different contributors and failures often occur because of component interaction, unclearly documented API or unknown use of the system [Trew 2006, Oberhauser 2007]. A firm that controls a wide scope of the architecture can guarantee the product quality more easily [Axelsson 2014]. In the situation where multiple firms are involved, and especially when the interfaces are not clearly defined, the quality can easily break down and externally developed code could access data in the system, causing malfunctioning or security problems. Furthermore, applications developed by complementors may not follow the UI standards, as set by the platform owner, thereby causing a reduced user experience [Bosch 2010, Jansen 2013]. The quality can be controlled easier by the vertically integrated platform type as they have full control over the end-product. Furthermore these firms are able to test the externally developed applications on their devices (hence score = +). For the proprietary platforms we evaluate this as neutral (score = 0): although they have no control over the hardware, they can control the API and, because of its closed nature, can easier avoid that code with defects is added to the platform. For the open source platform we also regard this as neutral (score = 0). Because of its open nature, faulty code can be added or code can be changed incorrectly by the complementors and applications may compromise the security. An advantage however, is that the software is developed and tested by a large variety of firms and open source developers. C: Variability: Specialized firms can often add variability to a platform more easily [Moore 2006], e.g. because they have the required knowledge, can reuse existing hardware and software components and they can have a more intimate contact to the end-users [Bosch 2009]. Flexibility and variability is easier to achieve through the open source platform type, since complementors can add or remove functionality without the need to involve the platform owner (hence score = +). For the proprietary platform type we evaluate this as neutral (score = 0): The device manufactures can create differentiating products with different hardware configuration; however this is limited to that which is supported by the software platform. As a comparison look at the PC industry where the different OEM suppliers of a Windows based PC can only compete on price, service and hardware quality, since the functionality is largely determined by the proprietary software platform. For the vertically integrated platform owner it is far more difficult to cover a wide range of products. In order for a firm to be flexible and respond quickly, it needs to focus on a number of core activities [Bolwijn 1990] and the development of all the required hardware and software components would simply be too costly. We therefore evaluate that this as challenging for this type (score = -). D: Freedom to innovate: The optimal definition of the boundaries depends on where the major innovation steps in the architecture take place. When innovation takes place across

65

Chapter 3 the boundaries of the platform the integrity of the platform is compromised and the complementors need to be involved thus slowing down the speed of innovation [Messerschmitt 2004]. Therefore, a wide scope allows for larger innovation steps more easily. The introduction of multi touch screens in smart phones is such an example. Due to this innovation specialized hardware was needed; the interface towards the user has changed and a new API towards the application developers was required. Such a large innovation step couldn’t be done through small changes to an existing platform. Large innovation steps are easier to establish by the vertically integrated platform type (score = +) because these firms control the entire architecture and complementors do not need to be involved. In the proprietary platform type the innovation is restricted because hardware is not part of the scope and the platform supplier has to involve hardware suppliers and device manufacturers for major steps (hence score = -). For the open source platform the hardware is also not part of the scope, however, the complementors may change the code and thus has the possibility for innovations, independently from the platform owner, for which the architecture does not have to be changed. Therefore we evaluate this as neutral (score = 0). E: Efficient use of system resources and hard real time requirements: Due to the need for optimal resource utilization and low power consumption a direct control of the hardware is required. An embedded device usually contains a System on Chip (SoC) and each component is controlled separately. For instance, for video decoding in a digital television a separate building block of the SoC is used which can operate with low power consumption and is only active when needed. Furthermore, by controlling each part of the IC separately it can be avoided that processes interrupt each other. Therefore the design of the hard- and software are developed in parallel and require close co-operation [Halasz 2011]. As a comparison, look at the Wintel framework; the dominant ecosystem in the PC industry [Grove 1996]. This is a modular architecture with stable interfaces between the hardware and the software layer. Both Microsoft and Intel can independently innovate on their part of the architecture. Such a modular interface is possible because most demands of the end user can easily be met by existing hardware. For mobile phones such a modular interface is not yet possible since there are still large innovation steps that involve changes to hardware and software together and a modular architecture would lead to less efficient resource utilization [Christensen 2002]. Since the vertically integrated platform type has both control of the hardware and software this can controlled more easily (score = +). In the open source type the complementors can also change the code to accommodate the hardware and vice-versa and therefore this can also be controlled (hence score = +). For a proprietary platform type this is challenging since changes to hardware may require changes to the proprietary platform and vice versa (hence score = -). F: Stability of the Interface: It is important for a platform to maintain a (sufficiently) stable API, thus avoiding that interoperability problems exist where applications do not work on the variety of devices based on different versions of a platform [Iansito 2004]. This

66

Consumer Electronics Software Ecosystems fragmentation is seen as the major challenge by the application developers [Gadhavi 2010, Gilson 2012]. Vertically integrated platforms can more easily avoid fragmentation since they have full control over the API (hence score = +). This also holds for the closed source proprietary platform, similarly as the case in the PC industry, which has proven to be the major advantages and success of this type of platform [Moore 2006] (hence score = +). For an open source platform this is challenging since fragmentation can easily occur because device manufacturers can change the API or the hardware (hence score = -). G: Effort for system integration: The time needed to integrate the hardware and software components is taking an increasing amount of time especially when components from different parties are used, e.g. with different technologies, non-matching interfaces or heterogeneous architectures [Henzinger 2006]. When a modular architecture exists with stable interfaces, the integration time would be substantially less [Christensen 2004]. Furthermore, when multiple parties are involved, the time for interaction, definition of requirements etc., is forming a substantial part of the development effort. For the vertically integrated platform type the integration is easier to manage since no complementors are involved and the integration is done with in-house development for which the interfaces can be defined (hence score = +). For the proprietary platform type this is challenging since this may require that code has to be altered or glue components have to be developed [Hartmann 2013] (hence score = -). In the open source platform type the device manufacturers or the system integrators can perform the hardware/software integration without (intensive) support of the platform supplier. Therefore we evaluate this as neutral (score = 0). Table 2 gives and overview of the strengths and challenges per ecosystem type. Table 2 Overview of types of ecosystems and their strengths and challenges E: F: System Interface resourc Stability es 1:Vertically Integrated + + + + 2:Proprietary Software 0 0 0 + 3:Open Source Software + 0 + 0 + In this overview: + means: easier to address, i. a strength, 0 means: neutral, - means: challenging Type / Requirement

3.4.5

A: Dev. cost

B: Quality

C: Variability

D: Innovation

G: Integr. effort + 0

Step 3: Creation of the decision support matrix

The determination, of which type of ecosystem is most suitable, is based on comparing the importance of each requirement with the strengths and challenges of the ecosystem types from Table 2. The decision support matrix of a product category shows the evaluation for each ecosystem type. An example is shown in Table 3. This matrix shows which of the requirements are of “high” importance and those that are “key”.

67

Chapter 3 Table 3 Example of a decision support matrix

Important requirements 1:Vertically Integrated 2:Proprietary Software 3:Open Source Software

A: Dev. cost High

B: Quality

C: Varia bility High

D: Innov ation

E: System resources Key

F: Interface Stability

G: Integr. effort High

Total Score

Most used type

-1

-1

+2

+1

+3

2

0

0

-2

-1

-3

n.a.

+1

+1

+2

0

+4

1

The total score is an addition of the individual scores for each requirement, i.e. ranging from -1 to +1. Here the scores of the key requirements are doubled, to expressive their relative higher importance, and are therefore ranging from -2 to 2. The final column show which of the ecosystem types are used most in the current market. In this example the open source platform type is used most, followed by the vertically integrated type, while the proprietary type is not available (n.a.).

3.5

Case Studies

The case studies in this section cover a wide range of consumer electronics products that support 3rd party applications: (1) Gaming Consoles, (2) Digital Cameras, (3) Digital Televisions and Set-top Boxes, (4) Smart Watches, (5) Smart Phones, (6) Tablets and (7) Personal Computers. This selection includes products that exist for a decade or longer as well as products that are recently introduced. Furthermore both large boxes and wearable devices are included. Each case study contains a brief description of the product, the firms that are active, a historical overview of the transitions, a description the ecosystems that are used and the ecosystems that are considered most suitable, using the multi-criteria decision support method, and those that are used most. These case studies are based on publicly available information. The determination of the importance of each requirement is done by the authors based on the products that are currently available in the market and supported by references when available.

3.5.1

Gaming Consoles

Product description and historical overview Gaming consoles have existed since the fifties and originally only contained games that were integrated in the consoles. Later it became possible to purchase games separately, initially as cartridges, later as CD-ROMS and today downloadable from the Internet. The development of games was initially done by in-house developers, the so-called 1st party developers. Later games were also developed by 2 nd party developers, i.e. game developers that worked on a contract base for the console manufacturers and later 3rd parties who independently developed games for a specific console. Hence the use of ecosystems was

68

Consumer Electronics Software Ecosystems introduced already early for these products. However, the sales of games is controlled by the publishers, i.e. the manufactures of the consoles. Gaming consoles are replaced with new versions every three to four years and consists of radical new functionality compared to the previous version [worldfinance 2014]. Consumers are willing to buy these new products, e.g. to support faster games, better graphics, 3D, motion detection and so on. Furthermore, consumers spend a great deal of money on new games, which provides additional revenues for the console manufacturers. Since the nineties, only three or four console manufactures had a dominant position [Vgchartz 2014], which resulted in a customer base that was large enough to amortize the investments. Manufactures do not create many variants of their products; only regional differences are supported. In this way the developed games can be played on all the devices, where backwards compatibility of older versions of consoles is not supported. The major manufacturers use a proprietary hardware/software platform and have done so since the beginning of this industry. In 2013 gaming consoles where introduced based on the open source software platform Android, such as Shield, Gamestick and Ouya. However none of these initiatives gained a large market share because they did not create the gaming experience that the traditional manufacturers offered [Trew 2014, Kain 2014, Hollister 2014]. A proprietary closed source software platform is not used in this market. Analysis of ecosystem type This product category is characterized in which almost all the requirements are evaluated as being of high importance with speed of innovation and interface stability as key requirements. The development costs are high since consoles consist of a large amount of software that need to support a variety of input devices and to support different screens. Also the requirements for system resources and hard real time requirements are high because performance and graphics quality determine the attractiveness of the system. The speed of innovation is key because gamers switch between systems frequently and want to play the latest games using the latest technology. A stable interface an absolute necessity because it has to be guaranteed that a purchased games run on a given console. Table 4 Decision support matrix for gaming consoles


A: Dev. cost High

B: Quality

C: Varia bility

D: Innovat ion Key

E: System resources High

F: Interface Stability Key


Total Score

Most used type

-1

+2

+1

+2

+1

+5

1

0

-2

-1

+2

-1

-5

-

+1

0

+1

-2

0

0

2

69

Chapter 3 Table 4 shows that the vertically integrated platform type scores substantially better than the other types. This is because the speed of innovation and hard real time requirements are supported the best in this ecosystem type while the variability is not an important requirement. Due to the speed of innovation it is not expected that a high degree of modularity can be obtained. The vertically integrated ecosystem type is by far used the most in this market and this is likely to remain the dominant type. The development costs are a main challenge. As a consequence only a few, 2 to 3, manufacturers have been successful in the market and newcomers are rarely successful. This is in sharp contrast with most consumer electronics products, e.g. televisions, audio players, smart phones etcetera, in which many firms are active and have a profitably business, and for which the development investments are substantially lower.

3.5.2

Digital Photo Cameras

Product description and historical overview Digital cameras have been introduced in the mid-nineties and rapidly replaced photographic films. Digital cameras contain dedicated hardware components and operating system that have hard real time requirements to ensure that the system response timely and store an image correctly. The size of this software has been relatively small and focuses on a few dedicated tasks. In later years software features where added such as facial recognition, however a general purpose operating system was not needed. Around 2010 cameras became connected via Wi-Fi, at first mainly to upload pictures. Due to the growing popularity of smart phones, and the reduced costs for hardware components, manufacturers started using android as a platform to add functionality that exists in smartphones. At the same time smart phones were equipped with higher quality lenses and sensors and cross-over product, such as the Samsung Galaxy K zoom were created. This cross-over is not surprising since Android was initially intended as platform for digital cameras [Berenaum 2013]. Lythro has created a camera that is focused on 3D and allows for focusing after the picture has been taken. Lythro offers a software platform, along with their device that is able to share pictures with social media [Lytro 2014]. Until know Lythro has not yet made their platform available for 3rd parties. In the early conception of digital cameras, i.e. until 2005 the speed of innovation was high resulting in consumer willing to purchase new cameras with better image quality and features created through software. In later years, e.g. since 2010, the speed of innovation is much lower and the innovation is limited. As a result the sales on new cameras dropped [Mintel 2014]. Analysis of ecosystem type The variability requirements are high because digital cameras are offered in large amounts of variants, e.g. ranging from compact cameras till single reflex cameras and a variety of types in between. Furthermore, different sensors, (touch screen) displays, flash lights etcetera are used. The uses of system resources and hard real time requirements is challenging because ICs with as small as possible footprint should be used because of the high pressure on cost price and because the response time when taking a picture is crucial. In contrast with gaming consoles, innovation is not a requirement of high importance

70

Consumer Electronics Software Ecosystems because digital photo cameras usually follow the functionality in smart phones. Also interface stability is less important because supporting 3rd party applications is not a key selling feature. Table 5 Decision support matrix for digital photo cameras A: Dev. cost Important requirements 1:Vertically Integrated 2:Proprietary Software 3:Open Source Software

B: Quality


D: Innov ation



G: Integr. effort

Total Score

Most used type

-1

+1

0

n.a.

0

-1

-1

n.a.

+1

+1

+2

1

Table 5 shows that the open source software platform has the highest total score, because the variability is high while the speed of innovation is very important for this type of product. A high degree of modularity is not likely to be obtained because of the need to support a large range of products while using ICs with a low cost price is essential. Therefore a proprietary software platform is not a suitable ecosystem type. In the current market only an open source platform is used.

3.5.3

Digital televisions and set-top boxes

Product description and historical overview Televisions, and later digital televisions and set-top boxes, where developed by specialized firms that developed and manufactured the entire product. Over time more and more software and hardware components from specialized suppliers where used [Hartmann 2012]. The last decade years, i.e. since 2007, the possibility to add 3 rd party applications is introduced so that consumers can use applications that they are familiar with from their personal computers and smartphones. Examples are YouTube, weather forecasts and sharing of pictures. For these types of TVs, the term “Smart TV” is used. Many firms have created an ecosystem based on a vertically integrated platform, such as Samsung with Smart TV [Samsungforum 2014], LG with WebOS [Onlinereport 2014] and Apple with AppleTV [Apple 2014], a set-top box. A large number of open source platforms have entered the market, including Android TV [Android 2014], Meego [Meego 2014], Ubuntu TV [UbuntuTV 2014] and XBMC [XMBC 2014]. Also vendors of proprietary platforms have entered the market, such as Horizon TV [Horizon 2014], Microsoft TV [TheVerge 2014A], and Yahoo! TV [YahooTV 2014]. Many of these platforms aim both digital televisions as well as set-top boxes since these devices largely contain the same hardware and software components. A variety of platforms is used, where none has gained a dominant position. Recently a number

71

Chapter 3 of firms, with a relatively small market share, have adopted the open source software platform Android, such as Philips, Sony and Sharp [Engadget 2014]. Analysis of ecosystem type The development cost are high because the software in these systems have to support a large range of broadcasting standards and require a high amount of software to efficiently create a good quality of pictures with high frame rate and high pixel density. The variability is high because digital televisions are developed with different screen size, price setting, should be able to support different regional standards and should be able to operate together with many other products. System resources and hard real time requirements are evaluated as main requirement because there is a large pressure on cost price and because power consumption has to remain low because cooling fans are not accepted in these products. Furthermore showing video and playing audio needs to be done without visual or audible interruptions. Finally, the integration effort is high since a large number of components from different suppliers need to be integrated and a high degree of modularity has not been obtained. Table 6 Decision support matrix for digital televisions and set-top boxes


A: Dev. cost High

B: Quality


D: Innov ation




Total Score

Most used type

-1

-1

+1

+1

0

1

0

0

-1

-1

-2

3

+1

+1

+1

0

+3

2

Table 6 shows that the open source software score best on most aspects, directly followed by the vertically integrated platform type. This analysis is in contrast with current practice in which the vertically integrated platform type is most used, where the two major players, Samsung and LG own 40% of the market share [Strategy 2014]. We believe that this is caused by commercial reasons, i.e. the manufacturers are reluctant to use an open source or proprietary platform with the fear that a dominant firm may take most of the revenues. Furthermore the investments to switch from an existing platform to a new one are significant and the major players have a large market share thus causing little commercial pressure. The main player in this market, i.e. Samsung [Strategy 2014], uses a proprietary platform which is based on the open source platform Tizen [Samsungforum 2014]. Consequently, Tizen may become an important candidate to dominate this market and a battle for dominance with Android may be expected [Information 2014A].

72


3.5.4

Smart Watches

Product description and historical overview Smart watches have been introduced in recent years but already exists in a wide variety. The functionality of most watches is, in addition to displaying the time, focused on showing notifications of incoming calls and messages [Theguardian 2014], while some watches are able to play music, take pictures and can be used for making telephone conversations, such as the Samsung Gear [Androidwatches 2015]. A selection of watches is focused on fitness and health by monitoring heart rate and physical activity, such as the Pebble and the Basis Peak [Pebble 2015, Mybasis 2015]. Apple introduced the Apple Watch that can be used for contactless payment [AppleWatch 2015], which was earlier introduced for smart phones [Trustedreviews 2014]. While the functions of smart watches are currently limited, in comparison with smart phones, a major architectural challenge is size and battery lifetime. As a consequence special operating systems have been introduced to accommodate this. At this moment in time there is no dominant platform used: Google has introduced Android Wear, which was first used by firms such Samsung, LG and Motorola [Androidwatches 2014]. Samsung also started using the open source platform Tizen [TheVerge 2014B] By using this platform, for which Samsung act as main contributor, it has more control over the platform and is not dependent on Google. Other firms, such as Apple, Sony and Pebble use a proprietary platform [Pebble 2014, AppleWatch 2014, Connectedly 2014]. Analysis of ecosystem type The variability is evaluated as key because with different functionality, size, price setting etcetera are needed to satisfy the different customer needs, and because watches are often a personal statement, which is why Android wear is not considered sufficiently modifiable by some vendor [Johnson 2015]. The speed of innovation is high because these products are new and it is uncertain which functionality will become successful in the market. The use of system resources and hard real time requirements are evaluated as key requirement, given the size of these devices and the increasing amount of ICs that are used in these products. Currently most smart watches need to be charged daily even when they are not used continuously.

73

Chapter 3 Table 7 Decision support matrix for smart watches A: Dev. Cost

B: Quality


C: Varia bility Key

D: Innov ation High

E: System resources Key

-2

+1

0 +2


G: Integr. effort

Total Score

Most used type

+2

+1

1

-

-2

-3

n.a.

0

+2

+4

1

Table 7 shows that the open source platform type is most suitable since this offers the consumer electronics and watch manufactures the best option to create distinguishable products. In the current market multiple open source platforms as well as multiple vertically integrated platforms are used, however little is known about the market shares. A proprietary software platform type is not available and based on this analysis this type is unlikely to become successful in this market especially because it is difficult for this type to support the requirements for optimal use of system resources.

3.5.5

Smart phones

Product description and historical overview A description of the transition in the industry is already given in section 3.3.1. This subsection is limited to the trends in functionality: Initially the functionality of these products was limited to making telephone calls and sending messages via SMS. Around 1996 the so called feature phones were introduced that supported downloadable ringtones, playing music and games. A major shift occurred when functionality was incorporated that was previously available in personal digital assistants (PDA’s), such as a Wi-fi connection, GPS and at a later stage, the introduction of touch screens. As a result, a mobile phone, now called smart phone, attracted a large group of users, which are using their devices for a variety of tasks such as making pictures, using social media and communication with friends and others using WhatsApp and Twitter. Analysis of ecosystem type The development costs and innovation are evaluated as of high importance given continues effort that is needed to introduce new functionality. The use of system resources and hard real time requirements: Evaluated as high, given the size of these devices and the need to have a long battery lifetime. The interface stability is regarded as an important requirement because application developers should be able to rely that their product functions on the different devices, to amortize their investments. The variability is evaluated as key because it is needed to support phones with different functionality, size, price setting, quality of the camera, design etcetera and users typically buy a product that exactly serves their needs.

74

Consumer Electronics Software Ecosystems Table 8 Decision support matrix for smart phones


A: Dev. cost High

B: Quality

C: Varia bility Key

D: Innov ation High


F: Interface Stability High

-1

-2

+1

+1

0

0

-1

+1

+2

0

G: Integr. effort

Total Score

Most used type

+1

0

2

-1

+1

-1

3

+1

-1

+3

1

Table 8 shows that for smartphones, the open source platform type is evaluated as most suitable. This analysis reflects the market share of 2013 [Gartner 2014] since the open source software has 79% of the market and the vertically integrated platforms 18% and 3% is of a proprietary platform, i.e. Microsoft. The overview also shows the challenges for each ecosystem type:  For the vertically integrated type, Development costs & variability are the main challenges. Apple, a key player, addresses this by limiting the variability and development costs by focusing on a particular market segments and by only supporting the latest versions of the handsets [Grobart 2013].  For the proprietary software type these challenges are: innovation & system resources. Windows addresses this by closely cooperating with a small group of HW platform suppliers [Duryee 2009] and acquired Nokia, a previously vertically integrated firm.  For the open source software type, interface stability & fragmentation are the main challenges. Google, the main open source supplier, addresses this by offering a compatibility test suite offered [AndroidComp 2014] and by making agreements with handset makers to use standard Google apps and user interface [Gannes 2014].

3.5.6

Tablets

Product description and historical overview Tablets have been introduced two decades ago by traditional computer manufacturers but did not gain much traction from the market in the first decade [Zdnet 2014]. With the introduction of the iPad by Apple, this product category become widely used. Soon other manufacturers followed that, similarly as Apple, used the same platform for tablets as for smart phones. Tablets, especially with larger screens, are used for different purposes than smart phones. Smart phones are primarily used, besides making telephone calls, for taking and sharing pictures, sending messages and so forth. Tablets are more used for answering mail, reading books and magazines, and watching and editing pictures. Therefore, tablets are increasingly used in schools and in business environments and often replace books and personal computers [Washingtonpost 2014, Information 2014B].

75

Chapter 3

Analysis of ecosystem type Development costs are evaluated as high given the continuous effort to introduce new functionality. The interface stability is also is evaluated as high because these products are starting to replace laptops and these products are used by schools and business applications. Variability is also an important requirement, since tablets are offered with different price settings, screen sizes, and so on. Table 9 Decision support matrix for tablets


A: Dev. cost High

B: Quality


D: Innov ation

E: System resources

F: Interface Stability High

G: Integr. effort

Total Score

Most used type

-1

-1

+1

+1

2

0

0

+1

+1

3

+1

+1

-1

+1

1

Table 9 shows that there is no difference in the total score between the three ecosystem types. In comparison with smart phones, the interface stability is regarded as also an important requirement, while system resources are less important because tablets have more options for heat dissipation and larger batteries. In the current market there is only one vertically integrated player with a sizeable market share, i.e. Apple. For this type the interface stability is a big advantage over the open source platform type which is visible because schools use the iPad rather than an Android device [Washingtonpost 2014]. As an example: To accommodate the use of iPad in schools and business environments, Microsoft has ported Microsoft Office for the iPad, rather than for Android which has a larger market share, because of the fragmentation of the Android platform [Appleinsider 2014]. The market share of tablets, in comparison with smart phones reflects the differences. In comparison with smart phones, both the vertically integrated platform, i.e. Apple and the proprietary software platform have a larger market share [Thenextweb 2014, Theregister 2014]. However, the share of the proprietary software platform is (still) very small. For the vendor of the proprietary platform, i.e. Windows, this can be a very viable market, much more than smart phones, since the interface stability is regarded as important, while the use of optimal system resources is less important because of the size of these products. Therefore targeting this market with a proprietary platform is expected to be more successful than targeting the market of smart phones as well, which is the current strategy of Microsoft [BBCNews 2014].

76


3.5.7

Personal Computers

Product description and historical overview Personal computers were originally developed by vertically integrated firms, but with the introduction of a Microsoft’s MS-DOS and a modularization of the PCs hardware an ecosystem was created, the so called Wintel platform [Baldwin 1997]. The innovation of PCs has been small since more than a decade and consumers only switch to a new product when the old system is becoming too slow or does not support the software that they need to use. This often leads to version of new operating systems that are not adopted by the business and only limited by consumers. For instance Windows Vista was hardly adopted in a business environment. In this market one proprietary software platform is present, i.e. Microsoft Windows, and only one vertically integrated platform type, i.e. Apple. Also some open source software vendors are present, e.g. Linux, Chrome OS and FreeBSD. Analysis of ecosystem type Development cost: Development costs are evaluated as high because PC’s should be able to support a wide range of products, require frequent updates for security reasons and so on. Furthermore, users only switch to a new version when it is really needed, thus making it difficult to amortize the investments. Quality is evaluated as high, since these products have to support business critical applications and interface stability: This is evaluated as an absolute must because a user or company has to rely that an application he purchases will function correctly, and that it keeps on functioning during a long period of time. Table 10 Decision support matrix for personal computers


A: Dev. cost High

B: Quality

C: Varia bility

D: Innov ation

E: System resources

High

F: Interface Stability Key

-1

+1

0 +1

G: Integr. effort

Total Score

Most used type

+2

+2

2

0

+2

+2

1

0

-2

-1

3

Table 10 shows that the proprietary software type scores higher than the vertically integrated platform and the proprietary software type. In the market Microsoft has the largest market of around 90%, with personal computers of Apple’s just below 10% [Theregister 2014]. Open source software platforms are marginally used. Although there are benefits of the open source ecosystem type, the disadvantage, i.e. the interface stability, is a key requirements and prevents that this type likely to gain a significant market share. The only open source software platforms that have some success is the Chromebook, which is completely focused

77

Chapter 3 on using a personal computer as client for web usage [Chrome 2015], and therefore relies on HTML as a stable interface. A native API is not offered.

3.6

Comparison with Related Art

In our previous work [Hartmann 2012] we analyzed the transition of mobile devices from 1990 until 2010, where the current paper analyses the situation of today. In [Hartmann 2012] we introduced a model consisting of 5 industry structure types, ranging from vertically integrated to closed and open ecosystems. The different types of ecosystems in the current paper is a refinement of the two ecosystem types described in that paper, where the vertically integrated hardware software platform consists is based on a vertically integrated platform, but has created a standardized interface for 3rd parties to contribute. In that earlier work the difference between a proprietary and open source platform types were not identified. Another previous work of the co-author [Bosch 2010] describes architectural challenges. Some of these challenges have been used in the current paper as criteria that determine the best suitable scope. However in that work the criteria have not been identified for mobile devices or embedded systems. In the work of Knauss and Hammouda [Knaus 2014] an assessment method is introduced that determines the suitability of a software architecture for an eco-system centric approach. In their work they addressed the tradeoff between the degree of openness of a software platform and reasons to limit the freedom. Their work did not include a comparison between different types of ecosystems for consumer electronics products. The work of Gawer and Cusumano [Gawer 2008] discusses the scope and technology design of a platform but does not discuss implications for software engineering nor does this work identify or compare different types of ecosystems for consumer electronics products. Other related work identified different types of ecosystems [Baldwin 1997, Jansen 2012, Bosch 2009] but did not compare ecosystems in a particular domain nor did it identify the challenges from a software engineering perspective. To our knowledge there is no related art that analyzes different types of ecosystem for consumer electronics and related work that contains a decision support model for different types of ecosystems; and therefore this paper is one of the first of its kind.

3.7

Conclusions and Further Research

In this paper we identified three different types of ecosystems for consumer electronics to support 3rd party applications and identified the criteria that determine the best suitable scope and technology design form a software engineering perspective. Based on this analysis we introduced a first step towards multi-criteria decision support method to determine the most suitable ecosystem type for consumer electronics products and used this method to analyze a wide range of products. Although our decision support method is not complete at this moment in time, and further research is needed, the analysis using this method already revealed essential difference between the types of ecosystems and their suitability for different product types.

78

Consumer Electronics Software Ecosystems This analysis shows that when the speed of innovation is a key requirement while variability is less important, the vertically integrate hardware/software type is more suitable. This is the case for gaming consoles. For product types that require a high degree of variability and when it is needed to share the development investment of the platform between multiple firms, the open source software type is more suitable. Examples of such products are digital photo cameras, digital televisions, smart watches and smart phones. For product types for which a stable interface is a key requirement, and development costs and quality are also important requirements, the proprietary software platform type is most suitable. This has been analyzed to be the case for personal computers. The proprietary software platform is less suitable for traditional consumer electronics products given the high pressure on cost price, which requires ICs with a small footprint and optimal use of system resources. On the other hand, open source software platforms are unlikely to gain a dominant position in the market of PC’s, since the requirements for interface stability are key, which is difficult to achieve for this ecosystem type. For tablets, all three ecosystem types are considered equally suitable since these products have characteristics of both smart phones and personal computers. While the choice of an ecosystem type is also, and possibly even more, determined by commercial and organizational aspect, in most cases the type that was considered most suitable using our decision support method, turned out to be the type that is most used in the market. This provides confidence that the introduced model is a sound first step. The case studies also showed how the individual requirements influence the market and the choices of consumer electronics firms and software vendors. For instance, the challenge of development cost for the gaming consoles results that just a few players are active. In the smart phone and tablet domain, Apple focusses on a limited variety of products, Microsoft took over an existing hardware firm to be able to act as a vertically integrated HW/SW ecosystem type, and Google is offering a compatibility test suite and makes agreements with the handset makers to use standard Android in order to avoid fragmentation. A primary topic for further research is to include business and organizational aspects in a decision support model. Further research may also reveal more relevant criteria. Furthermore an evaluation that is based on the expert opinion of more experts, and a more fine-grained analysis of the importance of the criteria will provide higher confidence in the results of the analysis. Due to these limitations, we consider the method introduced in this paper a first step towards a full-fledged decision support method. In the categorization and comparison of ecosystem types, a strict separation was made between open source and proprietary closed source. However, software vendors may use a mixed approach, in which part of the software is proprietary, e.g. to create a stable interface towards the 3rd party application developers, while other parts can be used without restrictions. Similarly a vendor may decide that a certain part of the closed source platform may be modified, e.g. to easier support different hardware configurations. These mixed approaches will lead to a combination of both types of ecosystems. More research is needed to identify what the consequences for each of these ecosystem types are. The key contributions of this paper are threefold:

79

Chapter 3 (1) Three types of ecosystems are identified that are used in consumer electronics to support 3rd party applications. (2) A first step towards multi-criteria decision support method is introduced to select the most suitable ecosystem type for a product categories, from a software engineering perspective. (3) Case studies of seven consumer electronics products are presented and a comparison is made between the suitability of the types that are used in traditional consumer electronics products with those that are used in mobile devices and personal computers. Firms that create consumer products and are considering to adopt an ecosystems centric approach can use the method presented in this paper to choose the type of ecosystem that will suit the market demands in the best way and address the challenges that arise as a result of this choice. Especially in an emerging market where no dominant ecosystems are available, such a choice is non-trivial and of crucial strategic importance. Firms that create software platforms can use this method to target the product types for which this is most suitable and pro-actively address the identified challenges to make their platform successful in the market.

80

Part II:

Variability Management in Software Supply Chains

“It is not necessary to change. Survival is not mandatory.”

W. Edwards Deming

81

82

Chapter 4.

Using Context Variability to Model Multiple Product Lines

This chapter is published as: H. Hartmann, T. Trew: “Using Feature Diagrams with Context Variability to Model Multiple Product Lines for Software Supply Chains.” In: 12th International Software Product Line Conference (2008), DOI 10.1109/SPLC.2008.15. Abstract Feature modeling is a key concept in Product Line Engineering. Due to the large number of different features and relations in practice, feature models often become too complex to deal with. One of these complexities is caused by the need to support multiple product lines. In this paper we introduce the concept of a Context Variability model, which contain the primary drivers for variation, e.g. different geographic regions. The context variability model constrains the feature model, which makes it possible to model multiple product lines supporting several dimensions in the context space. We will discuss how this concept can support software supply chains. Here it can be used to facilitate the process of merging feature models during staged configuration. Our initial experimental results indicate that the approach is intuitive and straightforward to use, and can be applied with commercially available variability management tools.

4.1

Introduction

In software product line engineering seeking the commonalities and managing the variability is one of the essences [Clements 2001]. Feature modeling is considered as a means to describe the variability in a product. It is used for requirements engineering and for configuring a product. Feature modeling originates from the FODA method [Kang 1990] and since then has been extended (e.g. [Czarnecki 2000, Czarnecki 2004A, Czarnecki2005]). Feature models can become large, for instance in the automotive industry [Thiel 2002, Weber 2002]. Adequate tools are important for an effective method deployment. Various variability management tools have found their way into the industry, many of them support feature modeling [Debbi 2007]. Reducing complexity in feature models has been a topic of research since their introduction. This complexity can be caused when different units of an organization are working together, amendments are made to feature models or because different levels of abstraction have to be maintained [Ye 2005, Reiser 2006, Buhne2004, Hamer 1996].

83

Chapter 4 The need to support multiple product lines is described in [Aoyama 2003]. A common example is that of two car types, a cabriolet and a station wagon [Czarnecki200, Buhne2004, Reiser 2005]. A cabriolet cannot be equipped with a rear wiper, but for a station wagon arear wiper is a mandatory feature. A cabriolet has a roof control, which can be manual or automatic. For a station wagon, a roof control is not available. These differences between the car types necessitate a separate feature model for each type. This causes redundancy and may lead to inconsistencies [Buhne2004, Reiser 2005]. The reuse of software components between different product lines in the consumer electronics domain is described in [Ommering 2002A]. In this domain, variability in requirements is driven by several different dimensions, e.g. different product types and different geographic regions [Czarnecki 2004A, Ommering 2002B]. In many industries Software Supply Chains will emerge to feed the demand for components [Greenfield 2004]. Common components will become commodities, which are developed by specialist suppliers. Software supply chains could occur in a wide variety of domains. A supply chain [Lee 1994] is: “A network that starts with raw material, transforms them into intermediate goods, and then into final product delivered to customers. Each participant consumes logical and/or physical products from one or more upstream suppliers, adds value, usually by incorporating them into more complex products, and supplies the results to downstream consumer”. For software this means that a company buys a set of components from (several) suppliers and integrates them into a product, possibly combined with its own software. For each of its customers, i.e. the next participants in the chain, a specialized version may be created. Each of the participants adds new functionality and makes configuration choices based on their role and responsibility in the supply chain. Overview: The remainder of this paper is structured as follows: We will start with giving an overview of earlier work. We will then introduce the novel concept of a Context Variability model, which, combined with a Feature model, is able to model multiple product lines supporting several dimensions in the context space. We will then describe how staged configurations can be applied to generate a specialized feature model. We will continue with describing how this concept facilitates the merging of feature models from different suppliers. We will then make a comparison with related work and we will end with a description of the results of a preliminary case study and describe topics for further research. Although the semiconductor industry contains a huge variety of different features and provides a challenging case, we will use examples from the automotive industry to illustrate our approach. The example is described in such an order that it matches that of the players of a software supply chain.

84


4.2 4.2.1

Related work Multiple product lines

Czarniecki and Eisenecker [Czarnecki 2000] suggest the use of “vertical” dependency rules between high level features and lower level features to deal with different feature models for different product types. Buhne et al. [Buhne2004] describe the problems related to the use of a single feature model and propose the use of a centralized variability model, which is later elaborated and described as the Orthogonal Variability Model [Pohl 2005A]. In a later paper, Buhne et al. [Buhne 2005] discuss various approaches and describe a way to model different product lines as part of the Orthogonal Variability Model. In their approach a solution is presented for modeling variation across different product types (e.g. DVD, MP3 and Hard-diskrecorder). Reiser and Weber [Reiser 2005] discuss several scenarios, including the use of a product tree with relations to a feature model. They came to the conclusion that this method is not scalable because it would lead to a lot of redundancy for large product families. Also the loss of the rationale for selection criteria is discussed. Reiser and Weber suggest the use of a product set to describe the constraints for different products and the tool support that is essential to make it practical.

4.2.2

Staged development and software supply chains

Pohl et al. [Pohl 2005A] described that there are different stakeholders that decide on the variability of a product and Czarniecki [Czarniecki2005A] used a similar notion of different parties and roles that eliminate variabilities. Czarniecki et al. [Czarniecki2004A] describe that a process of specifying family members may be performed in several stages, where each stage eliminates certain configuration choices. A transformation process is described that supports the specialization for refinement of feature models in different stages of development. In [Czarniecki2005A] this approach is extended to support software supply chains. The need for merging feature models, due to software supply chains, is also addressed here. In this paper the use of multilevel configuration is described to represent feature descriptions at different levels and use this for different stages. They suggest using a level-0 feature model describing the different market segments or geographic regions as a high level requirement.

4.3 4.3.1

Feature Models for Multiple Product Lines Context variability model

In this paper we will introduce the novel concept of Context Variability. Context variability is the variability of the environment in which a product resides. A Context Variability Model consists of general classifiers of the context in which the product is used. A general classifier stands for a set of requirements or constraints for that

85

Chapter 4 context. The Context Variability model captures the commonality and variability of the context. Examples: A general classifier of the context could be a geographic region. For a mobile phone the requirements for the European continent is to support for the 1800MHz band (a technical requirement) and SMS messaging (a commercial requirement). Both 1800MHz and SMS messaging are requirements that hold for all mobile phones in Europe. Another classifier of the context could be the type of user. Is it a type of user that likes all kinds of gadgets (also known as technology enthusiasts) or is it an elderly person that is reluctant to use new technology. Another example of classifier can be the type of product in which the software is used, e.g. a DVD-player, MP3 player or hard-disk-recorder. The Context Variability can be represented with the commonly used FODA diagrams, although the context classifiers are not features. It may include sub-“features” (meaning subclassifiers), cardinalities and dependencies between classifiers. For cardinality or group cardinality the minimum value is always 0 (e.g., [0..x]). In other words, a context variation point cannot be mandatory.

4.3.2

Multiple-Product-Line-Feature Model

The Context Variability Model is combined with a conventional feature model to create a new model, which will be called the Multiple Product Line Feature Model, or for short, the MPL-Feature model. Context Variability Model

Feature Model

Dependencies

Figure 16 Multiple product line feature diagram The feature model is used in the classical way, and describes the commonality and variability for a product range and the dependencies between different features. By making this separation, a feature model contains the possible features, with dependencies that capture the (technical) dependencies between the features. The dependencies related to the context are modeled as dependencies between the Context Variability model and the feature model, as shown in Figure 16. These dependencies are captured in the same way as dependencies within a conventional feature model.

86

Using Context Variability to Model Multiple Product Lines The types of dependencies between the Context Variability model and the Feature Model include “requires” and “excludes”, as is common with conventional Feature models. Besides these dependency relations we will introduce a new relation. This type of relation narrows the cardinality of either a feature or a group, as will be illustrated in Section 4.3.3. Furthermore, each dependency can be annotated with a rationale for the constraint imposed by the context. It is important that the rationale is captured, e.g. to assess the impact of requirements changes. For instance a purely “commercial” rationale can always be changed, but is valuable for a software supplier to record. In contrast, it may not be possible to change a dependency with “technical” rationale, if it were a property of the domain, or it might have a major impact on the product line architecture. Because of the size of the Feature Model for realistic product lines and the complexity of the constraints, it is therefore necessary to create a model, rather than relying on static diagrams, supported by appropriate tooling. Consequently, we will represent dependencies in textual form, and display filtered graphical views.

4.3.3

Example

To illustrate our concept we will use a fictitious example. Imagine a manufacturer of car infotainment systems, a tier-1 supplier in the automotive industry. Features: The software of the infotainment system consists of several optional features for information and entertainment purposes. For instance: a navigation system, a radio, MP3 player, DVD player, interface to external MP3 player dock, a USB connector and memory card slot, Bluetooth connections, etc. Several standards are support such as various DVDregions, broadcasting and communication protocols (FlexRay, ZigBee and Bluetooth). Figure 17 shows a subset of the infotainment system, using the FODA notation. Features

Connections

Maps

…….

EU

USA

…. Interface

Protocol CardSlot

Flexray

ZigBee

USB

BlueTooth

Figure 17 Feature diagram car infotainment system Context: The infotainment manufacturer creates products for different Car manufacturers: CarA, CarB, CarC and CarD, for different continents: USA, Asia and Europe, and has three

87

Chapter 4 product types: budget, mid-range, high-end. The budget, with a small feature set, fits into the space used for ordinary car radios. The mid-range type contains more features, is larger and therefore needs more cabin space. The high-end contains many possible features, is significantly larger than the mid-range, so needs space in the trunk. A diagram of the Context Variability model is shown in Figure 18. Context

Type

Continent

Car-Brand

Europe

ASIA

Budget

USA

Mid

High-end

CarA

CarB

CarC

CarD

Figure 18 Context variability diagram of the car infotainment system The dependencies between the context variability model and the feature model represent the specific requirement of each classifier. Some examples of dependencies and their rationales are presented below, using a textual notation. European Maps are part of the Navigation system and in Europe DVDs with region 2 must be supported: Continent.Europe European Maps {Rationale: Obvious} Continent.Europe DVD-region 2 {Rationale: Standard}

Manufacturer CarA may not support a specific communication protocol: Car-Brand.CarA FlexRay protocol

The budget (small) configuration can only support a USB interface or a Card Slot, not both, because the size of the front panel is too small. Type.Budget MP3 interface >> > [0..1] {Rationale: Technical; Too small size of front} Dependencies can also relate to different context classifiers: In the USA, CarA does not sell budget: Continent.USA AND car type.CarA budget {Rationale: Logistics}

Or become even more complex: In the Asian market, CarB only sells midrange without Memory-Card slot:

88

Using Context Variability to Model Multiple Product Lines Car Type.CarB AND Continent.Asia AND Type. Mid-range Memory-Card. {Rationale: Commercial}

Figure 19 shows a diagram of the MPL-Feature model with a graphical representation of the dependencies between the Context Model and the Feature Model (for reasons of clarity only a subset of the dependencies are shown). Features

Context

Connections

….

Maps

Type

EU

Europe

USA

…. Interface

Car-Brand

Continent Budget

Asia USA

Mid

High

CarA CarB CarC

Protocol CardSlot

CarD Flexray ZigBee

USB

BlueTooth

Figure 19 Multiple product line-feature diagram of the car infotainment system

4.3.4

Extendibility and scalability

In theory, all kinds of requirements can be captured. The car infotainment manufacturer could use this type of information, for instance, to exclude this combination from the test set or to analyze the commonality and variability in their product families. However, some of the dependencies shown in the example of Section 4.3.3 are more likely to be maintained and used by the car manufacturers, i.e. the downstream participant in the supply chain. It is their “role” in the supply chain to capture this and use it to generate different car types with an integrated infotainment system. In other words, the dependency relations “belong” to the participant of the supply chain that is responsible for it. In this way the MPL-feature models are reduced in size and complexity for each participant in the supply chain. This will be explained in Section 4.4.

4.3.5

Sub-classifiers as part of the context variability model

A sub “feature” as part of the context variability model can be used to capture specific requirements for this sub classifier. In our example we could use sub-classifiers for the continent Europe, such as The Netherlands, UK and Germany, which would capture specific requirements for these

89

Chapter 4 countries. In addition to this one could add sub-classifiers describing the use of the Euro as standard monetary unit, which holds for some countries of the EU. The dependency rules that are part of the sub-classifiers, i.e. the children, “overrule” that of their parents. In this way it is avoided that one ends up with many context classifiers, each representing a small set of requirements. All kinds of groupings and sub classifiers are possible to capture variability and commonality in the context space.

4.4 4.4.1

Context Variability in Software Supply Chains Requirements from software supply chains

In software supply chains, each of the participants creates software components that are specialized for their customers, i.e. the downstream participant. The customer integrates the components of the supplier into their own products and generates specialized products for their customers, i.e. the next downstream participant The supplier has to be able to capture the requirements for each customer to be able to generate a specialized product for that customer. In order to do this efficiently, the software supplier has to create a product family, or a set of reusable artifacts, and has to maintain the commonality and diversity of the different contexts, such as the different product types and customers. In this section we will describe how specialized feature models can be generated from the MPL-Feature model. We will apply the concepts of specialization and configuration that have been described by Czarnecki et al. [Czarnecki 2004B, Czarnecki2005]. In the next section we will discuss the subject of merging feature models.

4.4.2

Generating a specialized feature model

The concept of specialization and configuration can be applied to the feature model as well as the Context Variability model. A selection in the Context Variability model, followed by a transformation of the MPLFeature model, leads to a more specialized MPL-Feature model with fewer options.

90


MPL-Feature Model

Specialized for a single context

Specialized MPL-Feature Models

\

Figure 20 Generating specialized feature models A fully specialized Context Variability model results in a feature model for a single context i.e. that contains only those variation points which are available for that context. A partly configured Context Variability model results in an MPL-Feature model for a reduced set of product lines, with still one or more dimensions of freedom in the context space. See Figure 20. The decision on what and how far to specialize a feature model depends on what variability will be left to the customer. Some of the selections have to be made by the customer, like described in Section 4.3.4. Some other choices will be for the supplier, such as for which customer the product will be and the dependencies related to the hardware that will be delivered to that customer.

4.4.3

Example

Figure 21 shows a diagram of the specialized feature model for budget product for CarA of the car infotainment system that is obtained by selecting the budget and the CarA classifiers in the Context Variability model, followed by a transformation. Because of the dependency relations, FlexRay is no longer part of the features and the MP3 Interface has a group cardinality of [0..1].

91

Chapter 4

Context

Features

Connections

Continent Maps

….

Europe

Asia USA EU

USA

…. Interface

Protocol CardSlot ZigBee

USB

BlueTooth

Figure 21 MPL-feature diagram for CarA budget This MPL-Feature model can be supplied to CarA, which can select the region and select specific features to be included.

4.4.4

Specializing group cardinality in the context variability model

As described in Section 4.3.2 it is possible to use group cardinality in the Context Variability model. The following example shows the value of this in practice. We could have added a group cardinality of [0..3] in our example for the different continents instead of the current [0..1]. We then could have configured Europe and Asia and thus would have obtained a specialized feature model that represents the features for the infotainment system that is suitable for both Europe and Asia.

4.4.5

Extensions through the development lifecycle

More branches can be added to the MPL-feature model during later development phases. For instance a branch that models the different hardware platforms on which the software runs, i.e. the Product Line Architecture feature model. Once a small set of platforms have been defined to satisfy the required product line variability, these will impose additional constraints on how the product line requirements can evolve without an expensive redesign. Furthermore, the requirements for specific manufacturers could be added to support specialization of the feature model.

92


4.5 4.5.1

Merging MPL-Feature models Merging feature models

The next stage in a software supply chain involves the merging of feature models. These could be feature models from one or more suppliers, possibly with their own feature models. There are number of pitfalls that might occur when software components are integrated and feature models are merged. Merging feature models is a general problem in software supply chains and not only for multiple product lines. In this section we will start with describing the specific issues related to MPL-Feature models and will discuss the issue of overlapping feature models, which is generic for software supply chains. We will discuss the following possible scenarios:  Merging a single product line feature model into an MPL-Feature model.  Merging an MPL Feature model into another MPL-Feature model.  Merging models from different suppliers.  Merging overlapping feature models.

4.5.2

Merging a single product line feature model

Merging a single product line feature model into a MPL-feature model is a straightforward three step approach. First, the two feature models are merged as separate leaves, or one feature model becomes a leaf of the other. The second step is to add the dependencies between the features of the merged models. Finally, the dependencies between the context and the new leaf are added.

4.5.3

Merging two MPL-Feature models

When two MPL-Feature models are merged into one MPL-feature model, an extra step is necessary, which is the merging of the context variability models. It could be that the classifiers of the two models represent different contexts. In this case, the obtained context variability model is simply a combination of both models. In case the classifiers describe the same context, the dependency relation of one model has to be added to the other. The classifiers could also be describing the same context but are grouped in different ways. For instance one participant uses NAFTA as geographical regions because of the broadcasting standards, and the other uses USA. For Mexico the default language of the user interface could be Spanish and for the USA it could be English. In this case one could choose to create a new classifier in which the two countries end up as sub-classifiers, as described in Section 4.3.5. An alternative approach is to create two different contexts, capturing broadcasting standards and the other describing language preferences. In this way the two context classifiers describe more precisely what variability aspect is captured, but the disadvantage of this is that the amount of contexts is growing. It may be necessary to use a combination of all three options to merge Context Variability Models.

93

Chapter 4

4.5.4

Merging feature models from different suppliers.

Merging feature models from different suppliers, offering a similar feature set (e.g., components of two different infotainment solutions) a set of sub features can be used to represent the alternative choices. After the feature models have been merged, the dependencies related to the context variability can be added.

4.5.5

Example

The example given here uses the specialized feature model from the previous section, i.e. the budget car infotainment product for CarA, to be merged into the overall feature model of CarA. These are two MPL-Feature models of which the features do not overlap and the context classifiers, e.g. the different continents, describe the same context. CarA can still change the regional context and can configure the remaining features of the infotainment system. After the feature models have been merged, the dependency relations, between the features of the car and that of the infotainment systems have to be added. An example could be the following: the car infotainment has a feature that the outside temperature can be displayed. This feature depends on the availability of an outdoor temperature sensor of the car itself. After this step the context variability models are merged, meaning that the dependency relations related to the regions have to be combined. Finally there might be some new dependencies. For instance related to the type of the car in which the infotainment system is used e.g. the support of a memory card slot for certain car types or regions. Figure 22 shows a diagram of the merged MPL-Feature models.

94


Context

Features

….

Car Type

Engine

Continent

Europe Asia

Cabriolet Station Wagon

Limousine

Gas

Diesel Hybrid Sensors

USA

Infotainment

Rain

Temp

Connections

….

Maps

EU

USA

…. Interface

Protocol CardSlot ZigBee

USB

BlueTooth

Figure 22 MPL-feature diagram of CarA with merged infotainment system

4.5.6

Merging overlapping feature models

Two or more feature models could partly represent the same functionality. We will denote this as overlapping feature models. When features are overlapping it results in dependencies between the features. A feature, coming from one party cannot operate together with another feature, coming from another party, since they represent alternative implementations of the same functionality. Example: One supplier may offer software components that play various kinds of audio and video files, such as MP3, WMA, MPEG4, BluRay and DivX. Another supplier may offer software components that play various audio files and can also rip CDs. In this example part of the functionality is overlapping. The easiest way to avoid this would be to ask the suppliers to create specialized versions that do not contain overlapping features. In practice one might be faced with commercial off the shelf software for which this is not possible. Furthermore it might be possible that different suppliers are used for different contexts, e.g. one supplier for the Asian market and others for the European market. Another solution is to split the two feature models and create separate leaves, which represent different functionality. The last solution is to add dependency relations where one feature (group) excludes the other.

95

Chapter 4 In this section we have given an introduction to overlapping feature models, which is a generic problem for software supply chains. Merging feature models, especially when they are overlapping, requires a significant engineering activity. We believe that this can hardly be automated, but that the explicit representation of context assists the process of identifying and resolving overlap.

4.6

Comparison with Related Work

Several authors have addressed how to store the feature information for multiple product lines one model, thus avoiding the redundancy and inconsistencies due to changes, as discussed in [Buhne2004, Reiser 2005]. We discuss the differences between the approaches in this section. Reiser and Weber [Reiser 2005] also consider using of a product model in tree form with “includes” and “exclude” links to the feature model. They begin by exploring how constraints that depend on combinations of contexts could be represented by nesting contexts hierarchically in the product tree, so that only one leaf need be considered for each product. However, they identify the limitations of this approach and introduce the use of Product Sets, each of which is a subset of all the conceivable products. Product Sets are defined by the inclusion or exclusion of particular features from the final products. This cannot be applied in a software supply chain, where a supplier does not know all the configurations of the final product. In contrast, our approach only reasons about constraints on the features that we supply, maintaining them as logical expressions on the contexts. At successive integration stages, the constraints on features from different suppliers can be related. A variability management tool would combine the expressions to form composite constraints and create the appropriate graphical representations. Our approach therefore achieves the aims of Reiser and Weber, in storing the dependencies and their rationales together, thereby ensuring that no new source of inconsistency is introduced, but in a form that can be employed in a supply chain. Buhne et al. [Buhne 2005] describe an approach to model multiple product lines by using an extension of the OVM model [Pohl 2005A]. In their approach only one dimension of the context can be modeled, for instance the product type (DVD, MP3, Hard-disk-recorder), or the geographical region. In our approach several dimensions of the context variability can be expressed within one model. Czarnecki et al. [Czarneck2005B] suggest using a level-0 feature model describing the different market segments or geographic regions as a high level requirement. However, they do not address how to integrate the constraints from combinations of these classifiers. Buhne et al. [Buhne2004] and Reiser and Weber [Reiser 2006] describe the use of multiple levels of feature models to deal with different abstraction levels. Their research aimed at dealing with large feature models in large organizations.

96


4.7

Preliminary experimental results

A number of preliminary experiments were executed within NXP Semiconductors to validate our approach. NXP is a tier-1 supplier. It delivers products to various customers such as manufacturers of Televisions, DVD-recorders, Mobile Phones and the automotive industry. Products consist of both Hardware (ICs) and software. The customers integrate the hardware and software from NXP and other suppliers to create a product that is delivered to end-users. Part of the variability is configured by NXP and part by the customers. NXP buys software components, for instance operating systems that have to be configured for our products. Furthermore, within NXP there are organizational units that are specialized in creating reusable assets. These internal units are act as internal suppliers. The following experiments were executed:  Defining context classifiers and creating an MPL-Feature model.  Generating specialized MPL-Feature models.  Merging feature models. Five persons from different organizational units from NXP where involved. We used two commercial variability management tools that are based on conventional feature modeling. Creating an MPL-feature model, including a wide range of dependency relations, was straightforward and was well supported by the tools. There were several ways to present the features and relations in different graphical and textual styles and provide consistency checking. One of the tools was able to visualize the result of a choice in the context model on the feature model. This proved to be very essential when making configuration choices in large models, when a feature selection is found to be constrained unexpectedly. Neither of the tools we used could support the specialization of dependencies between features that specializes the cardinalities. However, there was no case where actual products needed this. For staged configurations a prototype was available in one tool. With this prototype we could generate specialized MPL-feature models, both for internal as well as external customers. For merging feature models it was possible to create a model in which the feature models where separate leaves. It was not possible to merge a feature model in such a way that one feature model is a leaf of another. For this, we used a manual process by copying and pasting parts of a feature model into a new feature model. We have not executed experiments with merging overlapping features, although we have identified cases from NXP’s practice in which this occurs. Although the tools could not fully support this way of working at this moment, the approach was considered as intuitive and straightforward. It has therefore been decided to adopt this approach at NXP.

97

Chapter 4

4.8

Discussion and Further Research

The definition of the context classifiers may not be easy in all situations. To differentiate between product types and customers is straightforward and seems to work for different domains. To define regions may not be so straightforward. For instance, in some countries traffic drives on the left side of the road: South Africa, the UK and Suriname. These three countries are located in different continents. It requires careful consideration to decide what aspect of the context is captured. We believe that requirements elicitation can benefit from this approach and may avoid problems like interoperability in consumer electronic product [Ouden 2006B]. Further research is needed. The dependency relation between a feature and the cardinality of another feature could be useful in conventional feature modeling as well. For instance the amount of processes that can run together depends on the amount of memory in the system.

4.9

Conclusions

The concept presented in this paper provides a solution for modeling multiple product lines by using the novel concept of Context Variability. The Context Variability Model provides an intuitive way to capture the variability and commonality in the requirements that originate from different contexts, such as different product types, geographic regions and customers. With this concept it is possible to capture several dimensions of variability in the context space in one model. The concept fits very well in staged development and supports software supply chains. Extensions to the approach are possible to support the development lifecycle, for instance by adding an extra branch that describes the hardware architecture. Our preliminary experiments showed that the approach is straightforward to use. Because our approach is based on conventional feature modeling, many variability management tools give the support needed for our approach. This makes it a very attractive candidate to be applied in the industry. NXP Semiconductors has decided to adopt this approach.

Acknowledgements The authors would like to thank Hans den Boer, Wouter Batelaan, Jean-Marie Trager, Somasundaram Venkatachalam and Aart Matsinger from NXP Semiconductors and Rob van Ommering from Philips for sharing experimental results, fruitful discussions and the challenging problems they have given us.

98

Chapter 5.

Supplier Independent Feature Modelling

This chapter is published as: H. Hartmann, T. Trew, Aart Matsinger: “Supplier Independent Feature Modeling”. In: 13th International Software Product Line Conference (2009).

Abstract The growing use of Software Supply Chains results in an increasing proportion of the functionality of a software product line (SPL) being determined by functionality of suppliers. In order to cover the whole product line, it is sometimes necessary to use several suppliers, offering partly the same functionality. This leads to overlapping feature models. This paper introduces a Supplier Independent Feature Model (SIFM). Through dependency relations between the SIFM and the feature models of the individual suppliers, the variability of the combined components is modelled. In this way the complexity of merging feature models is avoided and the relations with the development artefacts are maintained. The creation of an SIFM is elaborated through an example and a case study. These show that this is a straightforward process, and show that the SIFM facilitates the generation of product variants in an efficient manner.

5.1

Introduction

In software product line engineering an essential step identifies the commonalities and manages the variability [Clements 2001]. Feature modelling is a means to describe the variability in a product. Feature modelling originates from the FODA method [Kang 1990] and has subsequently been extended [Czarniecki2000]. It is used for requirements engineering and for automated product configuration [Czarniecki2000], i.e. creating an SPL variant. The process of creating a product line and creating individual products is usually separated into two phases, i.e. domain engineering and application engineering [Czarniecki2000,Pohl 2005A,Czarniecki2004B]. During domain engineering, the commonality and variability of a software product line are identified and captured using a variability model, e.g. a feature model, and the artefacts that make up the product line are developed. During application engineering a SPL variant is created through a selection or configuration of features. In many industries Software Supply Chains have emerged to feed the demand for components [Greenfield 2004]. Within a product line, the selected Commercial off the Shelf (COTS) components have to support the variability of the features in the product line that

99

Chapter 5 they implement. COVAR [Pohl 2005A,Pohl2001] describes a process to select Software Suppliers and COTS components. The supplier selection may result in a situation in which the requirements and variability of the product line cannot be fulfilled by a single supplier. Several suppliers have to be selected that, together, fulfil the set of requirements for a product line. When deriving a member of the product line, a choice is made between the alternative components from the different suppliers. By alternative components we mean that only one of the (set) of components can be used in a particular implementation. Example 1: A car manufacturer uses different Integrated Circuit (IC) Suppliers for alternative car-infotainment systems with different variability. Figure 23 shows a subset of their feature models in FODA notation [Kang 1990, Czarniecki2000].

SupplierA

Player

Shock

Single

CD

SupplierB

Radio

MP3

Player

Output

EQ

25W

CD

Radio

MP3

Output

25W

45W

Loader

Figure 23 Car-infotainment suppliers The ICs from SupplierA and SupplierB represent alternative components since it is not possible to combine functionality from both suppliers. So, for a particular configuration, one supplier has to be chosen that satisfies the requirements best. This is usually achieved through an iterative process, in which the principal features are selected first, then the most appropriate supplier, and finally the remaining features provided by that supplier. The challenge is to construct a feature model that integrates the variability of the components from the alternative suppliers into the overall feature model. The resulting feature model should describe the variability of the entire product line, i.e. including the required features and variability, and should be constrained by those provided by the suppliers. It should also contain the mapping from the features to the development artefacts. Furthermore, the feature model should provide a structure that supports the selection process of features and suppliers during application engineering. Overview: In Section 5.2 we will give some background and in Section 5.3 we will discuss alternative approaches. In Section 5.4 the Supplier Independent Feature Modelling (SIFM) is introduced. Section 5.5 gives an example from an existing Software Supply Chain, followed by tool experiences. We will conclude with a discussion, a comparison with related art and our conclusions.

100


5.2 5.2.1

Background Feature modelling and tool support

Weiss et al. [Weiss 1999] describe that the purpose of application engineering is to explore the space of requirements and to generate the application very quickly. Botterweck et al. [Botterweck 2008] observe that a product line can easily incorporate thousands of variation points and visualization techniques are needed to a productivity gain of SPL by making the application engineering process as efficient as possible. Czarnecki et al. [Czarniecki2000] describe that “A hierarchical decomposition for the primary structure of a feature configuration is important for reasons of understand-ability”. The tree structure of a feature diagram supports the selection process, e.g. a child feature only needs to be considered if its parent has been selected. Feature modelling is supported by commercial variability management tools [Debbi 2007]. These tools are used during domain engineering to create the feature model and to relate the features to the variation points in the product line architecture. These tools are used during application engineering to generate the product variant and they offer validation and consistency checking.

5.2.2

Supply chains and COTS selection

Wallnau et al. [Wallnau 2002] describe a growing influence of software supply chains and a change in the practice of component-based software design. In the past, a software product lines was developed, based on specifications from the stakeholders’ needs. Nowadays, more and more software is being developed from commercially available components. Consequently, a trade-off has to be made between the stakeholders’ needs and the desire to leverage components from the marketplace [Albert2002]. COVAR [Pohl 2005A,Pohl2001] describes a process to select Software Suppliers and COTS components for Software Product Line Engineering. This is positioned during domain engineering. COVAR introduces provided variability to describe the variability of the COTS components, which can be evaluated against the required variability of the product line.

5.3

Integrating Feature Models

The overall feature model of a product line contains several functional areas, e.g. a car contains engine management, chassis, infotainment, etc. With the adoption of Supply Chains, many of these functional areas are implemented by components from external suppliers. Therefore, a downstream participant in the supply chain uses a set of components from (several) suppliers and integrates them into a product, possibly combined with its own components [Czarniecki2005A, Hartmann 2008, see Chapter 4]. The feature models are integrated at two levels. First, for each functional area, the feature models of the alternative suppliers are combined. Subsequently, these are integrated into the overall feature model for the complete product, as shown in Figure 24. The second step may

101

Chapter 5 also add dependencies between the integrated feature models that arise from the product line’s requirements and technical dependencies. A

L B M C

D

N

O

F P

Q

E

R

S

G

H

I T

Overall Feature Model

J

K

Z

Functional Area

Figure 24 Integrating functional areas An example of this process is given in [Hartmann 2008, see Chapter 4]. In that paper the overall feature model consists of the feature model of a car-manufacturer merged with a feature model of a car-infotainment system.

5.3.1

Overlapping feature models

In an earlier paper [Hartmann 2008, see Chapter 4] we described that merging feature models is especially difficult in cases where there are overlapping features, i.e. features that represent alternative implementations. Supplier SUP1

Supplier SUP2

Supplier SUP3

S2

S3

S1

F1

F2

F5

F4

F6

F1

F2

F5

F3

F6

F1

F2

F5

F4

F6

Figure 25 Feature models from the suppliers Example: Before illustrating the approach with a case study, we will introduce the following example. We have 6 features: F1, F2… F6 and 3 different suppliers: Sup1, Sup2 and Sup3, which offer largely the same functionality (see Figure 25). The features F1..F6 of each supplier represent the same functionality and we will call them corresponding features. In

102

Supplier Independent Feature Modelling an actual situation the different suppliers might use different names for corresponding features.      

F1 is an optional feature, supported by all suppliers, but it is a mandatory feature for Sup2. F2 is a mandatory feature supported by all suppliers F3 is an optional feature, only supported by Sup2 F4 is an optional feature, not supported by Sup2 F5 and F6 are optional children of F2, For Sup1 they are alternatives. For Sup2 and Sup3 they are optional children of F2. For Sup3: F5 requires F1

The following sub-sections describe two possible approaches for integrating overlapping feature models and the limitations of each approach. Section 5.4 will describe a new approach that overcomes the limitations.

5.3.2

Possible approach 1: Alternative subtrees

A basic way to integrate feature models from suppliers is to create subtrees which represent the alternative feature sets from the suppliers, see Figure 26. To configure the feature model to generate an SPL variant, first a supplier has to be selected and only then the child features that belong to that supplier can be selected. The drawback of this approach is that it is not easy to compare the feature sets that are offered by these suppliers, especially not when the number of features becomes large. Features

S1

F1

F2

F5

S2

F4

F6

F1

F2

F5

S3

F3

F6

F1

F2

F5

F4

F6

Figure 26 Alternative subtrees However, in order to generate a product variant, a comparison is required to select the most suitable supplier for a product. This means that a part of the supplier selection, which was already done during domain engineering, has to be redone during application engineering. This is too time-consuming.

103

Chapter 5 We therefore conclude that this approach does not provide a structure to effectively support the feature selection process during application engineering.

5.3.3

Possible approach 2: Merging

Another approach is to merge the feature models from the alternative suppliers by creating a set of features which is the superset of the features from the different suppliers. We will call these Supplier Independent Features and, in our example, there would be six of these. A variation point is added to select between the alternative suppliers. This variation point is modelled as a mandatory feature with alternative child features, as shown in Figure 27.

Features Suppliers

Sup1

Sup2

Sup3

F1

F2

F3

F4

F5

F6

Figure 27 Merged models with dependencies By using constraints between the Suppliers variation point and the merged features, the differences between the suppliers can be captured. We would then get: Sup2 F4, Sup1 and Sup3 F3 and Sup2 F1. We also need to describe that, for Sup1, F5 and F6 are alternative features, and for Sup3, F5 requires F1. This is illustrated in Figure 27, which only shows two of these constraints. This approach has the advantage of permitting the selection of features first and, because of the dependency relations, the possible suppliers are determined. For instance, the selection of feature F3 will result in only Sup2 being suitable, and consequently F1 becomes a mandatory feature. Using the notation of [Czarniecki2005B], we can represent the reasoning as: excludes F3  (Sup3  Sup1)  xor-group

(Sup3  Sup1)  Sup2 

requires

Sup2  F1.

The approach also permits the selection of a supplier first, which then gives a specialized version of the supplier independent feature tree. However, this approach has two disadvantages.  The possible relations inside each of the suppliers feature models have to be modelled as dependency relations between the supplier variation point and the merged features. For instance, F5 and F6 are alternative features for Sup1, but

104




5.4 5.4.1

optional child features of F1 for the other suppliers. This leads to complex dependency relations between the supplier variation point and the supplier independent feature tree. The mapping between the feature models and the development artefacts is not kept intact because the feature models from the suppliers are replaced by a merged feature model. Czarnecki et al. argue that features in a feature model are merely symbols. The mapping of features to development artefacts, such as components, models and user documentation, gives the feature models semantics [Czarniecki2004B, Czarniecki2005B]. Pohl et al. [Pohl 2005A] describe how a development artefact can be related to one or more variants and that a variant must be related to at least one development artefact. The preservation of the mappings between the feature model and the artefacts is essential for automatic application derivation from the configuration of the feature model. The lack of traceability means that the selection and configuration of supplier-independent features is not sufficient to generate an SPL variant.

Supplier Independent Feature modelling Composition of models

To address the limitations described above, we retain the feature model from each supplier and create a separate feature model that composes them. This will be called the Supplier Independent Feature Model (SIFM). The SIFM contains the super-set of the features from all the suppliers, together with a variation point to distinguish between the suppliers. Through dependency relations, the SIFM is linked to the Supplier Specific Feature Models (SSFM) and consequently, linked with their development artefacts. This composition, which we will call Composite Supplier Feature Model (CSFM), models the variability of the alternative components.

5.4.2

Example

The SIFM, see Figure 28, consists of the Supplier Independent Features and a variation point to describe the alternative Suppliers, Sup1, Sup2 and Sup3. This is similar to that described in Section 5.3.3. However, we will not replace the feature models from the separate suppliers but use dependency relations between the features of the SIFM and the SSFMs to capture the differences between the suppliers and to maintain the relations to their artefacts. Unlike the dependency relations required in the merging approach in Section 5.3.3, the dependency relations between the SIFM and SSFMs are now uniform and independent of the dependency relations within the SSFMs. For reasons of clarity we have only shown a sub-set of the dependencies in Figure 28.

105

Chapter 5 SIFM Features Suppliers

Sup2

Sup1

F1

Sup3

F2

F3

F4

F5

F6

SSFMs

S1

F1

F2

F5

S2

F4

F6

F1

F2

F5

S3

F3

F6

F1

F2

F5

F4

F6

Figure 28 Composed SIFM and SSFMs (CSFM) The dependency relations are as follows. First we need to capture that a Supplier Independent Feature requires a Supplier Specific Feature and vice versa. F1S1.F1 XOR S2.F1 XOR S3.F1, F2S1.F2 XOR S2.F2 XOR S3.F2, F3>S2.F3 (Not for S1 and S3, since F3 is not supported by Sup1 and Sup3) F4S1.F4 XOR S3.F4, (Not for S2, because F4 is not supported by Sup2) F5S1.F5 XOR S2.F5 XOR S3.F5, F6S1.F6 XOR S2.F6 XOR S3.F6 S1.F1F1, S2.F1F1, S3.F1F1 S1.F2F2, S2.F2F2, S3.F2F2 S2.F3F3 S1.F4F4, S3.F4F4 S1.F5F5, S2.F5F5, S3.F5F5 S1.F6F6, S2.F6F6,S3.F6F6 We also need to express that the alternative suppliers require the corresponding supplier’s features, and vice versa so: Sup1S1, Sup2S2, Sup3S3, S1Sup1, S2Sup2, S3Sup3 Note 1: As described, the dependency relations between the SIFM and the SSFM are independent from the internal relations within the SSFMs, e.g. S1.F5 and S1.F6 are

106

Supplier Independent Feature Modelling alternative features for Sup1, but this does not need to be expressed in the dependency relations. Note 2: rather than merging the SSFMs into the SIFM’s supplier sub-tree, the SSFMs have been modelled as separate feature models, with dependencies with the SIFM. The split clarifies which features are explicitly selected by the application engineer, i.e. those of the SIFM. It also has practical benefits when maintaining the feature models, as described in Section 5.6.1.

5.4.3

Feature and supplier selection during application engineering

The SIFM gives the application engineer the possibility of starting by either selecting features, or selecting the supplier, or a combination of both. We will show two scenarios using the example. Scenario 1: Primary selection of features. Selection of F3 results in Sup2 becoming selected, since only Sup2 supports F3. Consequently F2 becomes a mandatory feature and F4 is not available anymore. requires F3  S2.F3  child-parent

S2.F3  S2 

requires

S2  Sup2 

xor-group

Sup2  (Sup1  Sup3) 

requires

Sup1  S1 

parent-child

S1  S1.F4 

requires

Sup3  S3 

parent-child

S3  S3.F4 

requires

(S1.F4  S3.F4)  F4

Scenario 2: Primary selection of supplier. Selection of Sup1 results in F3 not being available any more. xor-group Sup1  (Sup2  Sup3)  requires

Sup2  S2 

parent-child

S2  S2.F3 

requires

S2.F3

 F3

Selection of F5 results in F6 not being available anymore, because F5 and F6 are alternative features for Sup1. requires F5  S1.F5  xor-group

S1.F5  S1.F6 

parent-child

S2  S2.F6 

requires

Sup3  S3 

107

Chapter 5 parent-child

S3  S3.F6 

requires

(S1.F6  S2.F6  S3.F6)  F6

The selection of features of the SIFM can be used to fully configure a product. Since it contains a complete set of the features, the features in the SSFMs do not also need to be considered. Hence, this structure provides an efficient means to support feature selection and configuration during application engineering. The selection of features might easily lead to conflicting situations, which are difficult to resolve and requires studying the SSFMs. The solution proposed in [White 2008] provides a powerful approach for automatically identifying the source of such conflicts.

5.4.4

Resolving ambiguous supplier selection

The selection of features may result in situations in which more then one supplier can still be used. This means that the Supplier variation point is still undetermined. We propose to resolve this situation by using the preferred supplier for the remaining available suppliers. This can be achieved by introducing prioritized alternative features in a feature model. The prioritized preferred suppliers could be determined at COTS component selection, during domain engineering, by considering the lowest cost, highest quality, or any combination of non-functional requirements.

5.4.5

Structure of the SIFM and dependency relations with SSFMs

In the example, we have seen that F5 and F6 are child features of F2 in the SIFM. This is only possible because this was also the case for all the suppliers. If, for instance, F6 had been a child feature of F1 for Sup3, then F6 had to be modelled as an optional feature, directly below the root feature. In general, the more commonality there is in the structure of the SSFMs, the more it can be “copied” to the SIFM. Since a tree-like structure enhances the usability of a feature model during application engineering [Czarniecki2000], it is worthwhile to apply this structure to the SIFM. To identify common structural elements, both common parent-child relations and grouping of features might be used. If one feature requires another, it may be treated as a parent-child relation and features that are mutually exclusive might be treated as alternatives [Czarniecki2007]. However this also depends on other relations of the involved features. The dependency relations between the SIFM and SSFMs do not change when the structure of the SIFM is changed. These relations are independent of the internal structure of the SIFM and the SSFMs. These relations only depend on the availability of a feature. In our example, this was the availability of F3 and F4 for different suppliers.

5.4.6

Generic description and work process

The generic description of the CSFM is a generalization of the example. In order to create a CSFM and subsequently the overall feature model, the following work process is applicable. Step 1: Identify the correspondence between features from different suppliers. Step 2: Create a sub-tree to describe the supplier variability.

108

Supplier Independent Feature Modelling Step 3: Create a complete list of the corresponding features, which will become the leaves of the Supplier Independent Feature Model. Step 4: To create a tree-like structure, use the structure which is common in the referenced SSFMs, e.g. parent child relations and grouped features. Step 5: Add the dependency relations between the corresponding features in the SIFM and the SSFMs and the relations between the suppliers and their SSFMs. This combination forms the CSFM. Step 6: Integrate the CSFM, covering a particular functional area, into the overall feature model. Step 7: Add the constraints that arise from the product lines requirements and technical constraints to the SIFM.

5.4.7

Corresponding features

There are situations where it is difficult to define corresponding features. Two features might look similar, but do not represent identical functionality. For instance, two suppliers might offer JPEG enhancement (JE). However, these might give different image results and therefore cannot be called corresponding features. We will define them as JPEG1, which belongs to Supplier1 (Sup1), and JPEG2, which belongs to Supplier2 (Sup2). In this case a Supplier Independent feature can be defined "JE" with two alternative child features JPEG1 and JPEG2. Now the dependency relations are JESup1.JPEG1 XOR Sup2.JPEG2, just as if they were corresponding features. We also have to add: JPEG1Sup1.JPEG1,JPEG2Sup2.JPEG, Sup1.JPEG1JPEG1 and Sup2.JPEG2JPEG2. This gives the possibility to select JE alone, in cases where the type of enhancement is not important. The alternative choice of JPEG1 and JPEG2 might then be forced by the selection of other features that determine the supplier. Alternatively, an additional selection of one of the child features can be done to choose the type of enhancement explicitly. Another situation is where one feature defines a subset of the functionality of another feature or is specified more precisely. For instance, Audio compression versus MP3 and WMA. In this case, the finest level of granularity of the features has to be used as the Supplier Independent features, to allow a precise representation and explicit selection. Features with a coarser level of granularity have to be analyzed to determine their functionality. A similar approach to that of JPEG enhancement might be used to allow both explicit and implicit selection. A combination of both granularity and quality situations is also possible. Comparing features in general can be difficult, as this is a problem that is pervasive in COTS selection.

5.4.8

Separate components

Until now, we have described a situation in which the components from different suppliers cannot be combined, i.e. they are alternatives. One can imagine that a subset of the development artefacts of a supplier can be separated. For instance, in our example, we could imagine that, for feature F4, supplier Sup3 has a separate component that implements the feature and fits within the architecture. Now, this component can be treated as independent

109

Chapter 5 from Sup3. Only a constraint with feature S1.F4 is required to prevent these implementations from being combined. This separation is possible at every level of granularity. In the description of the ZigBee case, in Section 5.5, we will show an example where different suppliers are used for different layers of the software architecture. There may also be situations in which several components that relate to similar features may be part of the product configuration but may not be available together at run-time. In this case these components may be treated as independent as well but an extra restriction is needed to prevent them from being instantiated together at run-time.

5.5 5.5.1

ZigBee Case Study Functionality and products

ZigBee is an industry standard for wireless communication over short distances. In contrast to Bluetooth and Wifi, ZigBee focuses on low-power applications, such as the monitoring and control of devices for industrial control, medical data collection and home automation, see [Zigbee 2009].

5.5.2

Architecture and variation points

A ZigBee device consists of hardware and software. The hardware contains small microprocessors, memory, a RF transceiver and optional sensors. For the software, there is an OSI reference architecture that consists of 6 layers, as shown in Figure 29. Application Application Profiles Application Framework Network Layer Mac Layer -------------------------------

ZigBee Alliance IEEE 802.15.4

PHY Layer

Figure 29 ZigBee reference architecture. The specifications for the Physical (PHY) and MAC layers are defined by IEEE 802.15.42006. This specification contains a set of conformance tables that describe the mandatory and optional features in relation to different device types and describes dependencies between features. The Network (NWK) and Application (APL) layer are defined by the ZigBee alliance [Zigbee 2009]. This alliance has defined a conformance table for the NWK layer. This table defines different device types with their role in the network and also includes mandatory and optional features with dependencies. The following gives some examples of optional features. For the PHY and MAC layers there is an optional Beaconing feature and there are differences in device types. For the

110

Supplier Independent Feature Modelling NWK and APL layers there are optional features that involve the creation and separation of networks and the role of a device in a network, i.e. co-ordinator, router or end-device. Because the software relates to an industry standard that contains variation points, there is naturally a common interpretation of most of the features. However, suppliers provide other features that are outside the scope of the standard. Some examples of such features are the possibility to use external non-volatile memory and dynamic power saving.

5.5.3

Supply chain

The supply chain consists of IC vendors, software vendors and OEMs that create the final product. In the supply chain we found multiple parties for each link, partly offering the same functionality. In the example of this paper we take the position of an IC vendor that considers using software from specialized suppliers. In this chain, an IC vendor may offer hardware only, or may deliver hardware with software that for the PHY and MAC layers, or which may also include the NWK layers. For the software, we investigated three different suppliers. We found that some of the suppliers only contained the PHY and MAC layers and others also included the NWK and APL layers. We analyzed the available features using a similar approach to that described in [Pohl 2005A]. Table 11 below gives a subset of the features of the suppliers, which we will identify as A, B and C. In the table Man means Mandatory, Opt means Optional and No means not supported. Table 11 ZigBee suppliers Feature \ Suppliers NWK layer

A No

B Opt

C Opt

Beaconing GTS Security Mesh/Cluster Configuration ….. Outside Conformance table: Power saving Message fragmentation …..

Opt Opt No No

No No Opt Man

Opt Opt Opt No

No No

No No

Opt Man

Giving these differences, an IC-vendor can offer a wider product portfolio by using multiple suppliers. For instance, the power saving feature together with the beaconing feature gives the possibility to obtain battery powered nodes with a lifetime of 10 years. This is required in advanced e- metering infrastructure [Kistler 2008A] and, consequently, Supplier C is the only suitable supplier for this. Supplier B offers more capabilities in Security and Network configuration, which makes it more suitable for commercial buildings with strict

111

Chapter 5 authentication and authorization rules [Kistler 2008B]. Supplier A is an open source implementation, with a very small set of features and without a NWK layer. This supplier is most suited to form the basis for an in-house implementation that can be used for implementing ZigBee End Devices, which require the minimum set of functionality and the least amount of memory and, consequently, the lowest cost.

5.5.4

Creation of the SIFM

The creation of a SIFM was started by first modelling the features listed in the conformance tables. These features could be identified easily in the implementations from the different suppliers because very similar names were used. This made it easy to define the corresponding features and to find the variability in the implementations. Subsequently, we added the features outside the conformance tables. For these features, a careful comparison of their specifications was required to ascertain the full set of features and their correspondence across suppliers.

5.5.5

Different suppliers for different layers

It is feasible to combine implementations from different suppliers. For instance, a different supplier could be chosen for the MAC and PHY layers from that selected for the NWK and APL layers. These different supplier choices are represented by different variation points and, therefore, in this case, there are two variation points, selecting the MAC/PHY supplier and the NWK/APL supplier respectively.

5.6 5.6.1

Tool support Experiences

The CSFM approach was tested using a commercial variability management tool [Purevariants 2009]. We were able to generate the CSFM, as described in this paper. Some observations from this are listed below: 1. The SSFMs where modelled as separate projects within the Eclipse environment. The CSFM was created using a separate SIFM feature model, with the SSFMs as “referenced” projects. This allowed us to maintain the SSFMs and related family models as separate entities. When changes were made to the development artefacts or internal relations of the SSFMs, no changes need to be made to the CSFM. Furthermore, the SSFMs could be referenced by multiple projects for different customers. 2. Instead of using the “requires” relation from the SSFMs to the SIFM, e.g. S1.F1F1, we used F1S1.F1 etc. In this way we only needed to add relations to the SIFM and not to the SSFMs, which might have been obtained from suppliers. 3. During selection of features, the auto-resolve functionality of the variability management tool made it possible to immediately see the remaining choices after

112


4.

5.6.2

a feature selection, thus preventing an invalid selection being made. For instance, after a selection of Sup1 and F5, F6 was not selectable anymore. The auto-resolve functionality of the variability management tool did not show changes in the relations between features. For instance, when Sup1 was selected, F5 and F6 could have been visualised in the SIFM as alternative children of F2, because of the relations with the SSFM of Sup1. However, this was not supported. Additional visualisation [Botterweck 2008] might further increase the efficiency of the selection of features, caused by complex dependency relations. We will leave this for further research.

Automation

The creation of a CSFM could be further automated. When the corresponding features are defined and the SIFM has been constructed, the creation of the dependencies between the SIFM and the SSFMs could be done automatically. The creation of the SIFM may be automated by merging the SSFMs as described in [Segura 2007]. However the approach described in that paper assumes that features have the same name and the same parental relations. To be able to use more general structure of the SSFMs, as described in Section 5.4.5, a form of refactoring, as described in [Alves2006], is needed. Further research on these topics is needed.

5.7

Discussion and Alternatives

In this section we will describe how our approach can be used with the Orthogonal Variability Model, how it can be used during COTS selection and how it fits into staged configuration.

5.7.1

Application with the Orthogonal Variability Model (OVM)

The approach described in this paper can be applied to the OVM [Pohl 2005A] model in a similar fashion. A Supplier Independent OVM model can be constructed that consists of a variation point describing the alternative suppliers and a set of Supplier Independent variation points with dependency relations to the related variation points of the suppliers. Since the OVM model lacks the tree like structure commonly used in feature modelling, step 4 of the work process (see Section 5.4.6) can be omitted.

5.7.2

Using the CSFM during COTS selection

The selection process with more than one alternative component to serve the product line variability differs from the selection of a single component with variability [Pohl 2005A, Pohl2001]. Now the required functionality and variability has to be compared with the combined functionality and variability of the alternative suppliers. The CSFM can facilitate the selection of suppliers. By creating a CSFM, the variability of a combination of suppliers is described. This can be used to compare the required variability with the provided

113

Chapter 5 variability. With the CSFM, the same approach can be used as with the selection of one supplier. Because the CSFM describes the superset of available features, not all these features are available together. This makes the selection non-trivial. However, the same problem also occurs when one has to deal with only one supplier in which one feature excludes another.

5.7.3

Integration with context variability and staged configurations

In an earlier paper [Hartmann 2008, See Chapter 4] we introduced Context Variability to model multiple product lines. This approach can be combined with the CSFM. The Context Variability Model then specializes the SIFM in a similar fashion as the Context Variability model specializes a conventional feature model to create a feature model for multiple product lines. We have seen in some cases that the context determines the choice for the supplier. For instance, that a geographic region dictates standards that are only fulfilled by one of the suppliers. An example is an Electronic Program Guide for TVs and hard-diskrecorders, which are usually developed for a particular broadcasting standard. The CSFM can be combined with Staged Configuration. A specialized CSFM can be created for the next participant in the supply chain. This could either be a selection of the supplier, a selection of features, or both. When a feature model that contains multiple suppliers is transferred to a customer, a number of problems might occur. E.g. it might not be desirable, for commercial reasons, that the customer is aware of the different suppliers.

5.8

Related Work

The integration of variability from alternative components has not been discussed earlier in literature. Sugera et al. [Segura 2007] described a method for merging feature models. They assume that the merged features relate to the same artefacts. So their solution is not applicable for alternative components. Reiser and Weber introduce Multi-level Feature Trees [Reiser 2006]. A reference feature model is used as a template and guideline for referring models. Their approach aims to deal with separate organizational units that are responsible for a part of the overall feature model. Since their solution does not incorporate the relation to different artefacts, it does not provide a solution for alternative components.

5.9

Conclusions

Building a product line using alternative suppliers for similar functionality makes it possible to serve a wider group of customers and avoids a dependency on a single supplier. In this paper we demonstrated that merging feature models is not the most suitable approach when dealing with alternative components since it would sever the link to the development artefacts. We have described an approach in which an additional feature model

114

Supplier Independent Feature Modelling is created, the Supplier Independent Feature Model. With this model we have avoided the disadvantages of merging feature models. A particular advantage of our approach is that the original feature models have been kept “intact”, meaning that no changes have to be made to SSFMs. This means that that the relation between the feature models and the artefacts haven’t changed. Consequently, any changes made to the development artefacts do not influence the created Supplier Independent Feature Model. The ZigBee case demonstrated the need for using multiple suppliers and showed that creating the SIFM is easier in cases where there is a standard to which the suppliers adhere and the reference architecture facilitates the use of different suppliers for different layers. The experiences with a commercially available variability management tool showed that the method is straightforward to apply.

115

116

Chapter 6.

Using MDA for Integration of Heterogeneous Components in Software Supply Chains

This chapter is published as: H. Hartmann, M. Keren, A. Matsinger, J. Rubin, T. Trew, T.Yatzkar-Haham: “Using MDA for Integration of Heterogeneous Components in Software Supply Chains”, Sci. Comput. Program. 78(12): 2313-2330 (2013) DOI 10.1016/j.scico.2012.04.004 Abstract Software product lines are increasingly built using components from specialized suppliers. A company that is in the middle of a supply chain has to integrate components from its suppliers and offer (partially configured) products to its customers. To satisfy both the variability required by each customer and the variability required to satisfy different customers’ needs, it may be necessary for such a company to use components from different suppliers, partly offering the same feature set. This leads to a product line with alternative components, possibly using different mechanisms for interfacing, binding and variability, which commonly occurs in embedded software development. In this paper we describe the limitations of the current practice of combining heterogeneous components in a product line and describe the challenges that arise from software supply chains. We introduce a model-driven approach for automating the integration between components that can generate a partially or fully configured variant, including glue between mismatched components. We analyze the consequences of using this approach in an industrial context, using a case study derived from an existing supply chain and describe the process and roles associated with this approach.

6.1

Introduction

Software product line engineering (SPLE) aims to create a portfolio of similar software systems in an efficient manner by using a shared set of software artifacts. The software product line development process is usually separated into two phases: domain and application engineering. During the domain engineering phase, reusable artifacts that contain common and variable parts are created. These artifacts are configured during the application engineering phase to create specific variants. A variability model is used to capture the commonality and variability in a product line and to configure a variant [Pohl 2005A].

117

Chapter 6 In this paper we address products running embedded software, such as those in the automotive, aerospace & defense or consumer electronics domains. As the amount of software in these products grew, an increasing portion of a product line used commercial components [Wallnau 2002]. When such components first began to be incorporated, there was a clear separation of roles, with product integrators selecting components from suppliers. As the complexity of products increased further, a new market for pre-integrated subsystems emerged. Each sub-system might contain components from several suppliers, resulting in a software supply chain. In a supply chain, participants use components containing variability, combine them with in-house developed components, and deliver components containing variability to the next party in the supply chain [Greenfield 2004, Czarnecki 2005A, Hartmann 2008, Pohl 2005B, Kim 2005]. Many application areas employing embedded software have limited standardization of APIs and software component technology, so different suppliers may choose different mechanisms for interfacing, binding, and implementing variation points in their components [Ommering 2004, Park 2007, Blair 2009]. When a system is formed from components that do not conform to a common architecture, these components are usually referred to as heterogeneous components [Yang 1997]. When heterogeneous components have to be integrated, there might be mismatches between their interfaces, which have to be bridged by glue code. For instance, a set of interfaces might contain different numbers of methods (interface splitting), method parameters can be passed in different forms, e.g., as a struct vs. a list of separate parameters, methods having the same name might implement somewhat different functionality (functional splitting). Often, for a particular functional area, the variability required to satisfy all customers is too great to be implemented by a component from a single supplier. Therefore, there may be alternative components for each functional area, together with associated glue components [Hartmann 2009]. Where there are alternative components, specific components must be selected, configured and integrated during application engineering. In this paper we analyze the current practice for integration and configuration of components. We show that the traditional approach, in which the possible glue components are developed during the domain engineering phase, is not a scalable solution. The alternative approach, in which the required glue components are created during application engineering, requires too much throughput time and, finally, creation of a common standard interface would introduce unnecessarily complex glue components. We identify the challenges of developing embedded software that executes on resource-constrained devices, as well as the challenges that arise from developing software in a software supply chain. For resource constrained devices, it is important to minimize the code in the product. Therefore, some component technologies use a reachability analysis on the code to exclude components that are not required in a particular configuration [Ommering 2004]. This need for source code to be available during applications engineering conflicts with the desire of parties earlier in the chain to protect their Intellectual Property (IP). It may also be necessary to protect the IP of customers, e.g. to restrict the visibility of variation points. Furthermore, each specialized component technology may have a customized build environment, hampering building the overall product.

118

Using MDA for Integration of Heterogeneous Components in Software Supply Chains To address the aforementioned challenges, our work introduces a novel model-driven approach for automating the integration of heterogeneous components, covering syntactic mismatches, mismatches related to different component technologies and minor semantic mismatches. Our approach is based on a reference architectural model, from which modelbased transformations generate the artifacts required for a specific product variant. The selection of components is driven by a feature-modeling tool and wizards guide the application engineer, who may not be experienced in model-driven technology, through the creation of glue components, for which the code is generated automatically. Using this approach, glue components are generated efficiently and only when they are required. Staged configuration is supported, while we will identify a route by which, given further research, the reachability analysis to minimize code size can be applied across multiple component technologies. We exercised our approach on a case study that is derived from an existing supply chain and describe the process and roles that are associated with this approach. The paper addresses the following issues: 1. What are the challenges that arise from combining components with non-matching interfaces and what are the limitations of current practice? 2. What are the challenges that arise from developing a product line in a software supply chain? In particular, what are the challenges for developing resource constrained devices? 3. Can MDA be used to bridge mismatches between components from alternative suppliers? 4. Can support for staged configuration be provided? 5. What is the development process for the proposed approach and what level of MDA expertise is required by the engineers? The remainder of the paper is organized as follows: Section 6.2 elaborates on the integration and delivery problems, and the current industrial practices. Section 6.3 introduces a case study, while Section 6.4 describes our approach and how it is supported with MDA and variability management tools. The management of the expertise expected of engineers is presented in Section 6.5. Section 6.6 contains a discussion and identifies areas for future research. Section 6.7 gives an evaluation of our approach with Section 6.8 making a comparison with related art, followed by our conclusions in Section 6.9.

6.2 6.2.1

Problem Description Component integration

A product line must be able to satisfy the requirements of its potential customers. The key performance metric for suppliers in a software supply chain is its ability to be responsive to changing product requirements [Xia 2005]. When implementing a functional area using preexisting components, often no single supplier can cover the full range of variability needed, so it is frequently necessary to use components from several suppliers. This leads to a

119

Chapter 6 product line that contains alternative components, only one of which can be used in a particular implementation [Hartmann 2009], and a large number of glue components. For example, in the high level component diagram of the car infotainment system shown in Figure 30, the manufacturer uses four alternative suppliers for Navigation (with different displays and for different countries), three alternative suppliers for connectivity (Bluetooth, USB and/or 3G cellular), and three alternative suppliers for audio processing (with different equalizers, and amplifiers). To bridge between the interfaces of these components, up to 33 different glue components (A+B+C combinations) could be needed.

Navigation

B

A

Audio

Connectivity

C

Figure 30 Infotainment high level component diagram. Figure 31 shows an example of the structure of the glue that is required between a particular pair of components. The left-hand side shows the case where the component interfaces match in all respects and the interfaces can be bound directly. The right-hand side shows the introduction of a glue component, as already discussed, and a wrapper. Client

Client Notification

Notification Command Wrapper Glue Notification

Command

Command

Server

Server

Homogeneous Components

Heterogeneous Components

Figure 31 Homogeneous and heterogeneous component integration. The wrapper is required when the glue component uses a component technology, such as Koala [Ommering 2004] or nesC [Gay 2003] that binds components statically at build time, and requires a top-level component to specify the component composition. Each such

120

Using MDA for Integration of Heterogeneous Components in Software Supply Chains component technology requires a separate wrapper, containing all the components conforming to that component technology. Components of a particular component technology could be scattered over the system.

6.2.2

Current practice for integration and configuration of components

In current practice, a variability management tool is often used for configuring a product line, and creating a particular application out of the base components and glue components that were developed during domain engineering. The variability management tool allows the developer to select the required features, resulting in corresponding settings being made to the variability mechanisms of the development artifacts. Examples of development artifacts are source code and build scripts. Variability management tools include a feature model, which identifies the combinations of features that can be selected, and links to the development artifacts required to support each feature [Pohl 2005A, Xia 2005]. Additionally, these links can provide the rules for configuring the artifacts’ variability settings, (e.g., using a dedicated pre-processor). Figure 32 shows a diagram of a feature model, based on the feature-oriented domain analysis (FODA) notation [Kang 1990], with its links to example development artifacts, and the corresponding configuration space of a commercial variability management tool [Purevariants 2009]. The variability management tool can guide an engineer in the selection of the features to be supported by the application, by indicating which selections are no longer available according to constraints defined in the feature model to depict constraints arising from, for example, commercial policies or technical considerations [Hartmann 2008]. In the case of a software supply chain, the feature model can also be used to select the desired suppliers, as described in [Hartmann 2009]. Reusable Components

Feature Diagram F1

F2

F3

F4

Optional Mandatory

F5

F6

Alternative

Figure 32 Example feature diagram with links to development artifacts, together with its configuration space. When considering the integration of glue components, we will describe three of the different approaches that are taken in current practice. In the remainder of this subsection we will describe the characteristics and the limitations of each of these approaches.

121

Chapter 6 Approach 1: Creation of all possible glue components during domain engineering. In this approach all the possible glue components are created during domain engineering and configured during application engineering. The limitation of this approach is that these artifacts must exist prior to configuration. However, if glue components are only to be used between mismatched interfaces, the need for glue and wrapper components is only known when components from the different suppliers are being selected during application engineering. Furthermore, the links between the features and the glue components have to be created [Pohl 2005A, Xia 2005]. The presence of each glue component is based on the mismatched interfaces and is therefore based on a unique combination of the selected suppliers for a particular configuration. The manual creation of glue components and their links to the feature model introduces a substantial development effort into the product line development process, since the number of possible glue components can become considerable, as shown in the example from Section 6.2.1. For a scalable solution it is economically unfeasible to create all glue components and their links speculatively. Approach 2: Creation of the required glue component during application engineering. In this approach the variability management tool selects and configures the required base components during application engineering, but the required glue component must be added manually during application engineering. The advantage of this approach is that only the required glue components have to be developed. The disadvantage is that the glue is defined late in the development process and the throughput time during application engineering could become too large. Since the “purpose of application engineering is to explore the space of requirements, and to generate the application very quickly” [Weiss 1999], this approach decreases the benefits of product line engineering. Approach 3: Use of a common standard interface and component technology As an alternative to developing glue components wherever mismatched components are to be bound, standard interfaces and component technologies may be defined between each of the components. With this approach, any component not conforming to these interfaces is wrapped to achieve a match. This reduces the number of potential glue components and, since each of them is only associated with a single supplied component, they can be created during domain engineering. In the infotainment example of Figure 30, each glue component would then need two glue layers, but it reduces the total number of possible glue elements to 20. The use of standard interfaces has the important additional benefit that the selection of a glue element is only linked to the selection of the supplier of a single component. Consequently the links between the feature model and the artifacts can be maintained easily and the selection and configuration during application engineering can be done without needing to consider glue components. However, the glue elements that bridge to a standardized interface can become unnecessarily complex when there is a significant mismatch between the standard interfaces and those of the supplied components. This leads to undesirable additional development effort compared to the effort required to glue the two chosen components directly. In addition, different individuals often work on the two halves

122

Using MDA for Integration of Heterogeneous Components in Software Supply Chains of the glue, which can lead to more faults during integration than if a single individual had studied the interfaces of both supplied components.

6.2.3

Delivery of partially configured components

One of the outstanding challenges in a software supply chain is the ability to deliver partially-configured development artifacts to customers. A customer may not be the last participant in the supply chain, so they may leave configuration choices to their own customers, i.e. the next participants in the supply chain. The challenges related to this process have both commercial and technical aspects:  Some variation points may be implemented in the source code using pre-processor directives. This is a popular mechanism in embedded software because of its ability to minimize deployed software size and avoid performance overheads [Atkinson 2003]. However, to fully protect intellectual property, it may be that a customer should not receive any source code at all [Kim 2005, Sturgeon 2003].  From a commercial perspective it may be the case that one customer should not be aware of the existence of a variation point that is delivered to another customer, never mind the code that implements that feature. For instance when a particular feature is developed for a customer for whom this feature is a unique selling point, he may not wish to expose it prior to market introduction. There are also situations in which a particular choice of technology may not be revealed to competitors, thereby retaining market advantage.  Where source code is to be provided, there is a wide variety of build environments, particularly when specialized component technologies are being used. Even when no explicit component technology is employed, there are a variety of approaches used to propagate configuration settings to the variation points of the artifacts. This diversity makes it difficult to create an integrated build environment for a product line containing components from several suppliers.  Another challenge is to preserve the capabilities of existing technologies to exclude unneeded code. Some component technologies, such as Koala and nesC, target resource-constrained products where hardware costs are critical. These technologies use static binding and employ a reachability analysis prior to compilation, to exclude any code that will not be reachable in that configuration. As will be discussed in Section 6.6, this analysis may be frustrated if multiple component technologies are used in a product.

6.3

ZigBee Case Study

We will demonstrate the applicability of our approach to solving the issues introduced in Section 6.2 using a realistic case study. In order to limit the complexity of the first evaluation, the capabilities of the glue code in our tools are restricted to adapting between syntactic differences and the differences between component technologies. This caters for cases in which the semantics of interfaces are standardized, independently of the technology that

123

Chapter 6 may be used to implement them. As a case study, we chose a heterogeneous ZigBee stack. ZigBee is a wireless technology for low-cost, low-power wireless sensor and control networks, supporting applications such as home automation, health care and smart metering [ZigBee 2009]. It is designed for applications whose power consumption and hardware cost are lower than, for instance, those supported by Bluetooth. Consequently, many of the standard’s features are optional, so that code size can be reduced to the minimum required for any particular application. The ZigBee stack is defined as a layered protocol (see Figure 33). Smart Metering

Home Automation

Plant Monitoring

Application Objects

Application Framework Network Layer Medium Access Control (MAC) Layer Physical (PHY) Layer

Figure 33 Layers in the ZigBee protocol with pofiles The standard is platform-independent, being expressed in terms of service primitives, i.e. abstract, implementation-independent representations of interactions between the serviceuser and the service-provider [ANSI 2003]. In each layer of the ZigBee protocol stack, service primitives are grouped into Service Access Points (SAPs), which expose the layer’s services to its clients. This form of specification gives implementers the freedom to make their own choices for the exact form of the APIs between the layers. In this case study we focus on the lower layers, i.e. Physical, MAC, and Network layers. There is considerable variation between the Network layers for different application profiles, e.g. “Plant Monitoring”, “Home Automation” and “Smart Metering”, so a particular software supplier usually only support a few such profiles. The MAC layer is independent of the application profiles, but has to be configured for the particular integrated circuit (IC). Therefore, in order to serve a range of customers, an appropriate Network layer has to be integrated with the MAC layer for the selected IC. In this case study, we investigated the source code of three ZigBee stacks. Table 12 gives a subset of the features of these stacks, whose suppliers we will identify as A, B, and C. For a particular customer, the IC vendor selects the most suitable supplier. For instance, the power saving feature together with the beaconing feature allows a battery powered nodes to have a lifetime of 10 years. This is required in advanced e-metering infrastructure [Kistler 2008A] and, consequently, Supplier C is the only suitable supplier for this. Supplier B offers more capabilities in Security and Network configuration, which makes it more suitable for commercial buildings with strict authentication and authorization rules [Kistler 2008B].

124

Using MDA for Integration of Heterogeneous Components in Software Supply Chains Table 12 Feature set of selected suppliers Feature\ Suppliers Beaconing

Guaranteed Time Slot Security Mesh Configuration Power Saving Message fragmentation Implementation Technology

A Optional Optional Not implemented Not implemented Not implemented Not implemented nesC

B Not implemented Not implemented Optional Mandatory Not implemented Not implemented C

C Optional Optional Optional Not implemented Optional Optional C

Considering the structure of the supply chain, it consists of the IC vendor, alternative Network and MAC layer software suppliers and the final product developer. Because of the constraints on performance and power consumption, the IC vendor is usually responsible for the integration of the ZigBee Network and MAC layers, prior to delivery to the final product developer. During development, staged configuration may be required for each of the customers. Once the supplier has been selected, the remaining choices for optional features shown in the table can either be made in-house or by the customer. For example, when Supplier C is selected, the optional feature “Power Saving” can be selected in-house and the choice for “Security” can be handed over to the customer. In this case, the customer receives code in which “Security” is still a variation point but “Power Saving” is already configured. In the case study, an Open-ZB implementation [Cunha 2007], based on nesC technology [Gay 2003], was used for the MAC layer. To identify the range of capabilities that might be required of glue components, the differences between the technology choices of the Open-ZB implementation and those of the other suppliers were studied. In order to demonstrate how the code for these glue components could be generated, a dummy ZigBee Network layer was created, whose interfaces exhibited the union of the differences that had been found in the investigated software stacks, thereby covering all the observed differences. Here, we chose tmCom as the component technology, this being a precursor of that used in the MPEG Multimedia Middleware standard [ISO 2007B]. The main differences between the technologies used in this dummy Network component and the Open-ZB MAC are summarized in Table 13.

125

Chapter 6 Table 13 Differences between the technologies used in the ZigBee case study

Component technology Binding Interfaces Naming convention Calling convention Specific keywords

Network layer tmCom Dynamic Uni-directional tmINXP[Ntf] Referencing structure STDCALL

Open-ZB MAC layer nesC Static Bi-directional _ . Individual parameters command, event

Figure 34 Integration of Network and MAC layers Both components have two Service Access Points (for data and control), which are connected correspondingly. To bridge the technology differences, an additional glue component must be added, as shown in Figure 34. The role of this glue component is to exhibit tmCom style interfaces toward the Network component and nesC style interfaces toward the MAC component. The glue component translates each down-call (command) that it receives from the Network component into a corresponding call to the MAC component. In this translation, it deals with naming and parameter-passing conventions. For the up-calls (events) originating from the MAC component, nesC uses static binding, based on a configuration description, whereas the tmCom Network layer employs run-time binding, with a subscription pattern at the granularity of its interfaces [ISO 2007A]. Therefore the glue component implements the notification subscription management, which uses a table to map between the nesC functions called by the MAC and the tmCom functions in the Network layer. The glue code must provide the additional subscription management functions.

126


6.4

MDA for the Integration of Heterogeneous Components

The product line architecture contains components from alternative suppliers, where their commonalities are at a higher level of abstraction than the code. We therefore use Model Driven Architecture (MDA) to represent these commonalities since this technology supports abstract modeling and the automatic generation of concrete models and code for specific configurations [Stahl 2005, OMG 2011]. In this section, we first present a conceptual description of our MDA-based approach and the modeling constructs that we introduce. We then describe a high level development process that is to be followed according to our approach, and further elaborate on each step of the process.

6.4.1

Variability pattern and profile

Variability patterns [Gurp 2001] refer to the mechanism of introducing variability into a system, and the process of generating specific system out of many variations. In our modeldriven approach, we propose a new product line variability pattern, which extends the common variability patterns such as optional and alternative components, by supporting selection of supplier components and integration of heterogeneous components. Our pattern allows the replacement of a conceptual architecture component, which represents an architectural variation point and has no actual implementation, by one of several alternative supplier components. The selected supplier component substitutes the conceptual component while keeping all its external connections. Moreover, interfaces of the supplier component do not have to exactly match the interfaces of the corresponding conceptual component. This allows us to partially define the interfaces of the conceptual architecture component, according to high level specification or standard, and to refine the interfaces of the concrete supplier components, including method names and signatures. As part of the pattern, automatic detection of mismatched components, and their subsequent integration using the automatic insertion of glue components, is supported. The reference architectural model contains conceptual components and their composition, which are defined in terms of the dependencies between components’ untyped ports. The concrete components, some of which are supplier components and others are in-house components, are modeled with their internal structure and their own variation points. The ports of the concrete components must correspond to those of the conceptual components, but are typed by their actual interfaces. Consequently, the component dependencies can be maintained through their ports. The concrete components are associated with the conceptual components by using variability conditions. An overview of our approach follows, described with respect to the transformations illustrated in Figure 35.

127

Chapter 6

SI Client

SS Client

SS Client

Glue Makefile

SI Server

SICM

SS Server

SS Server

Library

Source

SSCM

Figure 35 Model transformations for integration and configuration of components Inspired by the OMG PIM/PSM terminology [OMG 2011], we term the initial model a supplier-independent component model (SICM). This model, which is represented in terms of a new UML profile, consists of supplier-independent (SI) components and their dependencies. When suppliers for the SI components are selected, a model-to-model transformation creates a new model, in which SI components have been substituted by supplier-specific (SS) components that contain interface descriptions for the corresponding development artifacts. Components of the new model are tagged with their component technology. Now, glue components are inserted by a second model-to-model transformation. This is done wherever a pair of components with connected ports is found that have either different technology tags or incompatible interfaces for these ports. Then, a combination of modelto-code transformations and reusable code snippets is used to generate the required glue code and other auxiliary files, such as build scripts. Generated code can be transferred to the next participant in the supply chain. The pattern enables the specification of the needed integration at the model level, which allows postponing some variability decisions to a later stage (done in-house or by a downstream customer), and postponing the actual generation of the glue code to the stage when all desired configuration decisions have been made. This also implies that at the feature model level, the downstream customer is only aware of the variability that is available to him and is not aware of any prior variability decisions, such as supplier selection.

6.4.2

Process overview

The process of using this approach is illustrated in Figure 36 and further elaborated in the following subsections. During the domain engineering phase the Feature Model and Reference Architecture are created. The required transformations are implemented as well. During the application engineering phase the suppliers are selected, and then the glue is generated. At the end, the component build is performed, followed by delivery to the next party in the chain (e.g., the customer), who can perform the final configuration.

128

Using MDA for Integration of Heterogeneous Components in Software Supply Chains Domain engineering activities Create Feature Model

Create Reference Architecture

Application engineering activities Configure for Supplier Selection

Identify Need for Glue

Configure and Generate Glue

Build Components

Deliver to Customer

Figure 36 Process overview To evaluate the approach, tool support for glue specification was implemented as an extension to the IBM Rational toolset [IBM 2011], including a new UML profile for modeling the supplier variability and glue specification, and model-to-code transformations for generating code artifacts. For feature modeling, a commercially available variability management tool was used [Purevariants 2009], for which no extensions where needed. Component modeling was done in the IBM Rational modeling tool. In what follows, we elaborate on the activities presented in Figure 36 and illustrate their use on the ZigBee case study.

6.4.3

Creation of a feature model and a reference architecture

As the first step in the process, as part of the domain engineering activities, the feature model that represents the product line variability is defined using a variability management tool. Following [Hartmann 2009], a distinction is made between variation points that relate to product features, denoted as functional variation points (FVP), and variation points that describe alternative suppliers, denoted as supplier variation points (SVP). In the ZigBee case, there are two supplier variation points - one for the MAC layer and one for the Network layer, and a list of functional variation points, a subset of which is shown in Table 1. Next, the SICM is created using the new UML profile. The SICM contains the SI components with their ports and variation points, as well as the dependencies between these components (left hand side of Figure 35). As potential component suppliers are identified during domain engineering, models of the SS components are defined. These components contain the interface descriptions for the supplied development artifacts. Subsequently, an implements connection with variability conditions is used to connect each SS component to its corresponding SI component, as shown in Figure 37 for the ZigBee case study.

129

Chapter 6

Figure 37 SVP's of the case study To complete the product line domain definition, the domain engineer activates validation rules to check that the SICM is legal and complete (e.g., each SI component must have a corresponding SS component to implement it). In the Zigbee case study the domain engineer modeled a subset of the ZigBee protocol architecture with two SI components, NWK_vp and MAC_vp, connected by two ports. Two alternative SS components are modeled for each SI component, Nwk and tmNwk for the NWK_vp SI component, and tmMac and Mac for the MAC_vp SI component (Figure 37).

Figure 38 Conceptual model of entities used The conceptual model of the entities used in this approach is shown in Figure 38. It shows how the SS Components implement the SI components and their relation to the glue components. It also shows the relations between the components and the variation points (FVP, SVP). The different variability conditions for a particular SI component represent the

130

Using MDA for Integration of Heterogeneous Components in Software Supply Chains SVP and are linked to the feature model in the variability management tool. The feature model is also linked to the FVPs of the SI components. Additionally, since different suppliers can implement the same FVP differently, and with different binding times, the SS components contain the configuration mappings of their FVPs to the FVPs of the SI components that they implement.

6.4.4

Configuration for supplier selection

During application engineering, an early step is the selection of the suppliers for each component, based on the different features that each supplier provides, as well as nonfunctional criteria, such as cost. The selection is made using a variability management tool, which assigns values to the supplier variation points (SVP). The assigned values determine, for every supplier independent (SI) component, which supplier specific (SS) component is chosen to implement it. Then, a model-to-model transformation creates a new model in which every SI component with an assigned SVP is replaced by the selected SS component. The process is depicted in Figure 35, where the model with SI components on the left is transformed to the model in the middle, in which SI components are replaced with SS components. These SS components are connected to the rest of the model in the same way that the SI components were previously connected. The transformation was implemented using the transformation mechanism of IBM’s Rational Software Architect (RSA) tool. This allows the creation of a set of mapping declarations that defines relationships between the elements in the models. These mapping declarations include complex expressions to identify elements in the input model and how they are transformed to the output model. The mapping rules are specified by referring to elements of a UML metamodel, extended through the UML profile described in Section 6.4.6. The variability constraints included in the model are evaluated during the transformation, in order to find the attached SS components that satisfy the user's selected suppliers. This model-to-model transformation is generic and can be applied to any supply chain application that has been defined in the tool. The transformation supports staged configuration without any other measures being required to keep the partially configured feature and component models synchronized, allowing suppliers to be selected in several steps, and allowing future configuration of FVPs. In addition, the input model is validated during the transformation against the UML profile.

6.4.5

Identification of the need for glue

When resolving supplier variation points, the conceptual components of the initial model are replaced by the selected specific components, possibly from different suppliers. Here we consider the case in which alternative SS components associated with the same SI component are implemented differently. Yet they should have similar functionality and equivalent ports. Glue components are required wherever connected ports have mismatched interfaces or where they have been implemented in different technologies. These conditions are detected automatically by validation of the UML model, based on the following criteria: 1. For each pair of SS components with connected ports, a required interface of an SS component does not match a provided interface, or any interface from which it

131

Chapter 6 inherits. Here, interface matching relates to the interfaces names and their method signatures (i.e. method names and the number, order and type of their parameters). 2. The components use different component technologies. The model component elements are checked for the conditions above. Wherever the validation fails, the skeleton of a glue component is created with ports and interfaces that match those of both the SS components. This glue component is inserted between the mismatched components by a second model-to-model transformation. This transformation is depicted in the right hand side of Figure 35, where the above criteria detected that a glue element needed to be added between the two SS components, as illustrated in the middle part of the figure. This is also a generic transformation that can be applied to any supply chain application. The inserted component provides a skeleton for the glue, with its implementation still to be generated, as will be described in Section 6.4.6. Some component technologies, such as Koala [Ommering 2004] and nesC [Gay 2003], require a wrapper to specify the composition of the components that use that technology, as illustrated in Figure 31. Where required, this top-level component is also generated by the transformation. When all SVPs have been resolved and all glue components have been added, we obtain the final, supplier-specific component model (SSCM).

6.4.6

Configure and generate glue components

To discuss how glue components are configured and generated, we first describe the glue components’ meta-model. We then proceed with a description of tool support for the application engineer, who can interactively generate the glue code without being exposed to mechanisms of code generation, i.e. the model-to-text transformations. Modeling glue components: A glue component should resolve mismatches between interfaces, methods, method parameters and other mismatches between the glued components. Furthermore, it may also supply additional functionality that some component technologies may require. For example, in the case study, the nesC MAC expects that its notification interfaces will be bound statically at build time, whereas the tmCom NWK expects that the server provides a dynamic subscription management facility, which must now be provided by the glue. Other incompatibilities between implementations from different suppliers that must be resolved within the glue component are initialization, debugging, logging, and power management. In order to create the glue components most efficiently during application engineering, their model is defined in a way that allows the implementation to be fully-generated from it. This is in contrast to generating the skeleton of the code and completing it manually. The model combines information from the SSCM with parameterized code fragments, called “snippets”, created during domain engineering and the model is configured interactively during application engineering using a set of wizards, which we elaborate later in this section. A meta-model for glue component models is defined in Figure 39. Each glue component is associated with the two SS components to be glued (the “TargetEnd” association); one of them may also be specified as “Wrapped End” for cases where a top-level wrapper component is needed. The glue component contains ports that correspond to the connected

132

Using MDA for Integration of Heterogeneous Components in Software Supply Chains ports of the replaced SI components. For each such port it holds a number of interface maps. In addition to these interface maps, the glue component has indicators for whether subscription management and initialization are to be included. When set, these indicators result in the instantiation of the corresponding snippets to generate the additional code.

Figure 39 Meta-model for a glue component The relationships between the provided and required interfaces of the glue components are addressed at three levels of component integration: interface maps, method maps and parameter maps. Each kind of map has its specific attributes and references to snippets. This arrangement is able to support component integration even when different suppliers group methods into interfaces in different ways. These relationships between interfaces are represented in the meta-model as follows. It contains an InterfaceMap for each interface provided by the glue. Each interface map contains a set of method maps that have one caller entity and a number of callee entities. This 1:n relationship caters for cases where the methods of the client and server are not matched, so that a single client call must result in a sequence of calls to the server. Each callee entity contains a set of parameter maps, which are used to control the transformations between those parameters that are passed in different forms by the caller and callee methods. The code that is generated from the glue model consists of a set of methods for each provided interface of the glue component. The core of the caller method's body is a sequence of calls to the methods in the CalleeMethodsInfo list, together with the necessary parameter transformations, defined by the corresponding snippet. This sequence is surrounded by prerun and postrun code snippets, which can support other functionalities, such as memory management for temporary parameter structures or logging. When the application engineer has populated all the maps, a model-to-code transformation is used to generate the software artifacts, such as glue code, the wrapper component, and build scripts. Tool support for glue code generation: To make the glue specification process faster, tool support for configuring glue, as well as for defining and using glue snippet templates was developed on top of IBM Rational Software Architect (RSA) [IBM 2011]. The implemented tool includes a set of wizards and dialogs to support the application engineer during glue configuration. These wizards hide

133

Chapter 6 the mechanics of code generation from the application engineer. For illustration, screenshots of the Interface Map Wizard, applied to the ZigBee case study, are shown in Figure 40.

Figure 40 Wizards for the interface map configuration process The Interface Map Wizard assists the user in specifying the mapping between provided and required inter-faces that need gluing. The central part of the wizard is the Interface Map Editor, which allows the selection of one or more callee methods for each caller method, and to specify the mapping between the parameters of the methods. The final model-to-text transformation creates glue code that converts between interface methods with similar meanings but of different naming conventions and technologies. Code generation requires the instantiation of parameterized code fragments, in which the parameters are replaced by elements from the model. Conventionally, this is implemented using transformation templates, such as JET used by RSA [Java 2011]. These template languages support constructs for model navigation and conditional code generation. Because hard-coding fragments of glue code in the transformation template would require widespread knowledge of the template language in the development organization, the fragments are stored as snippets. The snippets use a much simpler syntax, only supporting parameter substitution, and are straightforward for developers to create. For example, the debug information that is generated at the beginning of each function is specified by a snippet, rather than being hard-coded in the template. So, we can change the debug information, without changing the template. The code for a snippet can be entered by the user either explicitly or by selecting a predefined snippet from a snippet library. Figure 41 shows how the snippet library is populated. The snippet library allows snippets to be reused across different interface maps and glue components. The snippet text can also be parameterized by predefined parameters such as interface name, callee or caller method name, etc. Parameter values are referenced by the model-totext transformation templates, and are automatically substituted by the model attributes

134

Using MDA for Integration of Heterogeneous Components in Software Supply Chains taken from the SSCM. The use of such parameterized code fragments increases range of circumstances in which the templates can be reused unchanged.

Figure 41 Snippet library In addition, to reflect technology naming conventions, the tool allows usage of naming hints, which significantly automates the method and parameter name matching process. We characterize the name's structure as a combination of the part that identifies the specific method or parameter, and additional parts (prefix, suffix and delimiters), e.g. the identifier of the interface, that together comprise the full identifier. In this way the additional parts may be stored for each supplier or technology, and used during the glue configuration process. For the ZigBee case study we first created examples of glue methods manually, from which a library of parameterized code snippets was extracted. They are used later to configure numerous glue component maps in conformance with the meta-model described in Section 6.4.6. Code generation was implemented using the extensible JET-based Rational Software Architect transformation framework. A fragment of the code generated by our prototype is shown in Figure 42.

135

Chapter 6 Static void STDCALL _tmIMCPS_Data_Request (ptmThif thif, NXPDataRequest_t* pdata) { //BEGIN PRERUN SNIPPET DBG_PRINT( (dbgUnit, BG_INTERFACE_ENTER, MCPS_DATA.request) ); //END PRERUN SNIPPET uint8_t SrcAddrMode = pData.SrcAddrMode; uint8_t SrcPANId = pData.SrcPANId; uint32_t SrcAddr = pData.SrcAddr; uint8_t DstAddMode = pData.DstAddMode; uint16_t DstPANId = pData.DstPANId; uint32_t DstAddr = pData.DstAddr; uint8_t msduLength = pData. msduLength; uint8_t msduHandle = pData. msduHandle; uint16_t TXOptions = pData.TXOptions;

Caller signature

Parameter mappings

call MCPS_Data_Request (SrcAddrMode, SrcPANId, SrcAddr, DstAddMode, DstPANId, DstAddr, msduLength, msduHandle, TXOptions) //BEGIN POSTRUN SNIPPET DBG_PRINT( (dbgUnit, BG_INTERFACE_LEAVE, MCPS_DATA.request) ); //END POSTRUN SNIPPET

Callee signature

}

Figure 42 Example of the generated glue code for the ZigBee case

6.4.7

Building the components and delivery to the customer

Prior to the final build of the product, all the SVPs must have been configured, but we require flexibility in the configuration time for any FVP. The integrator makes an initial configuration, e.g. to protect the intellectual property of other customers and suppliers and each customer receives a specialized configuration space containing only the remaining unconfigured variation points. At the point that the code is validated and delivered, the mappings from the SI components’ unconfigured FVPs to the corresponding variation points in the development artifacts are added to the generated build script. Subsequently, this mapping is used to translate the customer’s configuration description. To address these two issues, we adopt a two-stage approach. For each programming language: 1. The components are passed through the early stages of the build process for their respective component technologies, to the point where standard language source and header files are generated. For example, Koala [Ommering 2004] identifies which source files will be required and generates macros to rename functions to permit static binding. 2. Having transformed all components to a standard form of source file for their language, build scripts are generated. These files include the FVP settings, such as the definition of pre-processor symbols used in the realization of variation points. Additional build scripts are generated for each glue component and a further, toplevel build script identifies all the required components and validation is performed. Where the customer only receives binary code to protect the suppliers’ IP, or where it is not possible to separate the two stages above, the final build is performed remotely on the supplier’s site. For instance, by exposing the complete configuration and build process as a web service, the build is performed based on the customer’s feature selection.

136


6.5

Development Roles

Given the small proportion of software developers who have experience with MDA technology, to be deployable in the short term, it is essential that only a few developers need to be familiar with the more esoteric aspects of MDA, such as defining UML profiles and model transformations [Stahl 2005]. This section describes the development roles involved in the approach and the different levels of knowledge that each role requires to perform the activities, as illustrated in Fig. 4 and described in Section 6.4. Tasks during Domain Engineering:  The task Create Feature Model is performed by the requirements manager and requires a working knowledge of feature modeling.  The task Create Reference Architecture is performed by the domain architect, who defines the product line architecture, represented by the SICM, and who also identifies the potential suppliers and creates the SS component models. This role requires a working knowledge of MDA. Tasks during Application Engineering:  The tasks Configure for Supplier Selection, Identify need for Glue and Configure and Generate Glue are performed by the COTS engineer. The COTS engineer is responsible for the integration of components from different suppliers and creating glue components. The COTS engineer should be familiar with the component technology and development environments used by the suppliers but he does not need specific knowledge of MDA since his tasks are assisted by the set of wizards that hide underlying MDA complexities.  The tasks Build Components and Deliver to Customer are performed by the customer support engineer. The customer support engineer liaises with customers and determines what configuration is required prior to delivery. He will use the variability management tool to define a specific product configuration and uses the MDA tool to create the SSCM and to perform the final export. These tasks do not require knowledge of MDA principles.  The final configuration is performed by the customer, being the next link in the supply chain. He uses the variability management tool to make the final configuration of the received product artifacts, but is not exposed to any MDA technology. We recognize additional tasks for our approach. These tasks correlate to the “meta-team”, identified by Aagedal and Solhein [Aagedal 2004]:  The definition of the meta-model and the model-to-model transformations are performed by the language designer [Stahl 2005]. This work requires an in-depth knowledge of MDA. Since the meta-model and transformation can be reused for any component composition, these activities would typically be done by the MDA tool vendor.

137

Chapter 6 



6.6 6.6.1

The creation of transformations for generating the glue code is performed by the transformation developer. The transformation developer requires very specific skills related to the transformation tooling. However, this only needs to be undertaken once because these transformations can be reused for any combination of component technologies. The COTS engineer is responsible for the maintenance of the snippet templates and creation of new snippets, e.g. when a new component technology is used. Should a new combination of component technologies ever be required for which the glue component cannot be generated from the current meta-model, the language designer must extend the meta-model and the transformation developer must adapt the modelto-text transformations to meet these new requirements. It is the task of the COTS engineer to identify these situations and discuss the required changes with the language designer and the transformation developer.

Discussion and Further Research Challenges solved by our approach

The approach described in this paper solves the challenges that are described in Section 6.2. It allows components from different suppliers to be integrated, despite syntactic differences in interfaces and semantic differences related to component technologies that are based on a common programming language. It also supports staged configuration, where some variation points are resolved by the next participant in the supply chain and the suppliers’ IP can be protected through a remote build process. The approach addresses the application engineering phase of SPLE by supporting the creation of glue components where they are required. It makes glue code generation as efficient as possible, without making speculative investments during domain engineering. Once the glue snippet templates have been created for a few exemplars they are reused for numerous glue components. The approach recognizes the limited experience of MDA available in the industry and restricts the number of development roles that need to be familiar with it. This is achieved by using a balance of reusable model-to-code transformations and parameterized code snippets, with a development environment that provides guidance to the applications engineer. Furthermore, our approach supports the export of standard programming languages and makefile technology, thereby avoiding exposure of the customer to unfamiliar technology. The approach retains the sophisticated feature modeling techniques and supporting variability management tools developed for SPLE. For instance, these support staged configuration through the ability to present the next party in the supply chain with a specialized configuration space, in which choices made at earlier stages are no longer visible, although the constraints resulting from these choices are still in effect. Controlling the visibility of variation points protects the IP of other customers. However, the links to the development artifacts and, in particular, the mapping to the different variability mechanisms used by different suppliers, is now passed to the component

138

Using MDA for Integration of Heterogeneous Components in Software Supply Chains models. The variability management tool is no longer required to determine where glue components will be required; this is now determined by a single model validation rule, providing a scalable solution. Hence, the variability model is now not directly connected to development artifacts [Pohl 2005A], or model transformations [Voelter 2007], but the choice of the SVP now, indirectly, may lead to the generation of a glue component.

6.6.2

Outstanding challenges

While our approach solves the challenge identified in Section 6.2, some other challenges remain for further research, e.g. concerning reachability analysis and bridging of greater semantic differences. Reachability analysis. One of these challenges is in the ability to deliver to customers partially-configured development artifacts while preserving the full capabilities of component technologies, such as nesC [Gay 2003] and Koala [Ommering 2004], which minimize the memory requirements of the code through their reachability analyses of their components. The current approach converts all components into standard source files and generates a uniform style of build script, which propagates the remaining FVPs, thereby shielding the customer from multiple build environments. The creation of standard source files uses the native build process for each component technology and their reachability analysis is utilized. However, the build process for some component models, such as nesC, only creates conventional C files once the C pre-processor has been run, by which time all variation points with design-time binding will have been instantiated.

Figure 43 Component composition and configuration with reachability analysis

139

Chapter 6 The current approach does not support the reachability analysis across different component technologies whose consequences are illustrated in the following scenario. Figure 43 gives an example of a client and a server with a glue component. The client contains an optional component C2, which requires the server to contain the component C4. If the client and server were implemented in the same technology then, omitting C2 would automatically exclude C4 from the build since it would have no clients. The complicating factor in a software supply chains is that the configuration is done in steps, and the presence of component C2 might only be resolved though a configuration choice of a participant later in the supply chain. Let’s assume that only the server is implemented with the Koala component technology, which uses reachability analysis. The server is to be bound to a client, implemented in some other technology. The diagram begins at the point at which one participant in the supply chain has selected their suppliers and identified the need for glue. One might imagine that the glue could be fully instantiated at this point and delivered to the next participant. Note that only glue components are implemented in two technologies; all other components are implemented in a single technology. In this example, Koala is only being used for the glue component’s required interfaces. The Koala reachability analysis will not see any provided interfaces for the sub-components of the glue, since these are implemented in the component technology of the client and the analysis will therefore assume that the glue is the top-level component, and that all its required interfaces are necessary. However, looking downstream, we see that, if C2 is deselected then C4 is no longer required, and should be omitted by the reachability analysis. In other words, if the glue component were to be instantiated as soon as its initial specification is available, then it would appear that all of the server components are always necessary, and the reachability analysis would have no effect. Note that, in a more generic situation, the glue would support both commands and notifications. This would result in a cyclic pattern of dependencies, making the reachability analysis even more complex than is shown here. The optimal solution would result in a situation in which the reachability analysis is only performed once the final specification is known, allowing all unnecessary code to be removed. Since, at this stage, we have components based on different technologies, together with generated glue components; no existing build environment supports the full analysis. Therefore, technology-neutral interface dependencies should be maintained and new build environments should be developed that are aware of variation points, so that a reachability analysis can be performed across all components. Further research is needed to achieve this optimal solution. Bridging of greater semantic differences. A principal area for further research is the bridging of greater semantic gaps between components. This would allow the approach to be applicable to a much larger range of mismatched components. The current approach supports moderate mismatches because of the ability to map one caller function to a sequence of callee functions. This is sufficient for the ZigBee case, whose standard defines the semantics of messages within the communication stack, but there are many standards for embedded products that only consider the interaction between the product and its environment, with no consideration for the APIs of the software within the product. Egyed

140

Using MDA for Integration of Heterogeneous Components in Software Supply Chains and Balzer [Egyed 2006] have proposed a reference architecture for stateful glue components, which may form the basis of more capable glue components for COTS integration. Here further research is required to extend the automated support to be able to guide creation of this style of wrapper. A related issue is that the current approach only inserts glue components between pairs of components. However, there are cases, such as the generation of code to map from the operating system abstraction layers (OSAL) of each supplier to the actual OS, that cannot be done in a pairwise manner, because of the need for common bookkeeping for shared OS resources. Therefore, a more general model of glue code must be developed for these cases.

6.7

Evaluation

This section discusses our approach using three evaluation criteria: Quality Improvement; Reduction in Development Effort and Required Level of Expertise. These criteria can help practitioners and industrial organizations in estimating the benefits that our approach would provide in their context. Below, we compare our MDA-based approach with the manual creation of glue code.

6.7.1

Quality improvement

In the manual approach, exemplar code is usually developed first. Then, the rest of the glue components are created through copying, pasting and editing this code for each function. When following the manual approach, it is essential that the developer makes all the changes necessary for each copied function, otherwise errors can be introduced, which can be timeconsuming to localize, given the limited information that is usually available for COTS components [Wallnau 2002]. In contrast, in the MDA approach, code snippets are created from the exemplar glue component. The snippets fall into two categories: those that will always be instantiated, with their parameters replaced by models elements, and those whose selection requires the developer’s judgment. Snippets in the first category are instantiated completely automatically, while the wizard guides the developer through the selection of those in the second category. In addition, in the manual approach the developer has to alternate between selecting the appropriate code fragment to copy and deciding how this fragment should be modified. On the other hand, in the MDA approach, the developer only has to consider those differences between the components that cannot readily be characterized. The wizard prompts the developer for each decision that has to be made. Consequently, the quality of the generated glue components is higher than for those created manually.

6.7.2

Reduction in development effort

The efficiency gain depends on the number of functions for which glue code is required and on the number of decisions that the applications engineer must make when running the code generation wizard (see Section 6.4.6). The use of code snippets allows a fixed set of transformation templates to be reused unchanged because the use of snippets avoids hard-

141

Chapter 6 coding any code fragments within the model-to-text transformations. These snippets can be edited by the COTS engineer without much effort. The efficiency increases when few decisions have to be made for each function, so that most of the code is generated automatically. Several different scenarios can be considered: 1. In the situation where the mismatches are solely based on the use of different component technologies, the glue components can be generated automatically without human intervention. However this was not the case for the ZigBee case study. 2. For syntactic mismatches, the transformations between the calling and called functions can be captured in the code snippets. This was the case for the ZigBee case study. The applications engineer only has to select the relevant snippets for each function. 3. For simple semantic mismatches, where there is not a 1:1 correspondence between the calling and called functions, the wizard maps a single function call to a sequence of calls. Here, more extensive testing is required to confirm that this mapping is correct, e.g. to verify whether the functions have been invoked in the correct order. We expect that an additional reduction of development effort will be achieved during maintenance activities, as earlier identified by one of the co-authors [Dubinsky 2011] and by other studies [Mohaghegi 2008]. This is because the structure of the glue code will be more consistent, despite being created and modified by different application engineers, due to the uniformity imposed by the code generator and the reduced number of errors that are expected. The efficiency gain is further increased if the initial investment can be amortized over many glue code functions.

6.7.3

Required level of expertise

To introduce our approach, a certain amount of training is required to perform the tasks described in Section 6.4. The domain architect needs to have a working knowledge of the UML component model and the use of our UML profile to represent the supplier variation points. The COTS engineer needs to become familiar with the creation and maintenance of snippets, e.g. when new component technologies are used, and the use of the wizards to create the glue components. For both tasks, no specific MDA knowledge is required and the engineer does not have to be familiar with the transformation language itself. The reference architecture is created when a new product line is started and when additional suppliers are used. This model is easy to develop and maintain, since component dependencies are only represented at the granularity of ports, rather than interfaces. Moreover, the meta-model, the model-to-model and model-to-text transformations, and the wizards, once developed, can be reused between different applications and domains since the meta-model and the transformations cover a very wide range of different component technologies.

142


6.8


From the perspective of staged configuration in software supply chains [Czarnecki 2005A], we address organizations in the middle of the chain, which must both integrate components from different sources and pass partially-configured artifacts on to downstream customers. The problem of the use of feature models for coordinating the configuration of artifacts using different variability mechanisms is addressed by Reiser et al. [Reiser 2007]. However, they do not consider the creation of glue components. We have previously discussed merging feature models from alternative suppliers for a particular feature area [ISO 2007A], but that paper did not consider how glue components would be addressed. Voelter and Groher describe the integration of a variability management tool with a model-based software development environment [Voelter 2007]. However, they address the links to transformations for in-house developed components, rather than the needs of a software supply chain. The Common Variability Language (CVL), described by Fleurey et al. [Fleurey 2009], supports a notation that is compatible with feature models and provides a mechanism for the substitution of model fragments. CVL could support the supplier selection and the first transformation from the SICM, in which supplier independent components are replaced by supplier specific components. However, it is neither able to synthesize the model fragments on-the-fly for the glue components nor to dynamically identify new boundary points where the glue components are to be inserted. The definition of the SICM for the ZigBee case study was straightforward, given the reference model in the standard. Where there is no pre-existing reference mode, the architectural reconciliation approach, proposed by Avgeriou et al. [Avgeriou 2005], to defining a COTS-based architecture can be used. However, while their approach aims to avoid architectures that require excessive amounts of glue code, they do not address how the essential glue code would be created efficiently. Zhao et al. [Zhao 2004] address the combinatorial explosion of potential glue components when bridging between different component technologies. They use a generic grammar to specify the implementation of glue, but they avoid having to handle hybrid build processes by using SOAP as a common communication format between all technology types. This approach is unacceptable for resource-constrained devices, in which code size and performance remain critical. Smeda et al. [Smeda 2008] address the creation of the specification of glue components from the composition of parameterized templates in the context of an architectural description language and address the creation of a modeling tool for this language. However, they do not address how automated support could be given to developers to assist in template composition. Stahl et al. [Stahl 2005], Krahn et al. [Krahn 2006] and Aagedal and Solhein [Java 2011] describe the different roles needed in MDA and the skills required. Where they provide a classification for the roles during domain engineering, we additionally provide the different roles and skills, associated with our approach, during application engineering and for the customer’s organization.

143

Chapter 6

6.9

Conclusions

In this paper we discussed the implications of developing product lines in a software supply chain. Since software components are developed by different parties in the supply chain, often using different interfaces and different component technologies, an increasing number of glue components are required to bridge those mismatches. The paper described the problems that arise from combining components with nonmatching interfaces. We particularly addressed the case where components from different suppliers are used to implement a single functional area. We showed that the traditional three approaches, that is, the glue components are developed during the domain engineering phase, during the application engineering phase or are mapped to a common standard interface; do not provide a scalable solution. We introduced a novel model-driven approach for automating the integration of heterogeneous components, covering syntactic mismatches and semantic mismatches related to different component technologies. We exercised our approach on a case study that is derived from an existing supply chain, for which we used a commercially available variability management tool [Purevariants 2009], and a prototype was implemented as an extension to an IBM Rational MDA tool [IBM 2011]. We described the process and roles that are associated with our approach. We showed that the approach has the following benefits compared to prior art and current practice:  Glue components are generated efficiently only when they are required, thereby avoiding unnecessary development effort during domain engineering.  Staged configuration is supported; enabling the next party in the chain to do further configuration while IP and commercial interests are protected.  The additional skills required to deploy MDA are localized in the organization by providing tool support for configuration and glue code generation, which ensures that only a limited group of developers are exposed to unfamiliar technology. Finally, we addressed areas for further research: To support reachability analysis across multiple component technologies to minimize code size, we identified that technologyneutral interface dependencies should be maintained and new build environments should be developed that are aware of variation points. For bridging greater semantic differences we identified that a more general model of glue code is required.

Acknowledgments The authors would like to thank Uri Avraham, Ofir Brukner, Alan Hartman, Shiri KremerDavidson, Itay Maman and Asaf Shaar from IBM Research and Yanja Dajsuren, Jos Hegge and Rob Wieringa from NXP Semiconductors for their contributions to this work.

144

Chapter 7.

Evaluating a Textual Feature Modelling Language

This chapter is published as: A. Hubaux, Q. Boucher, H. Hartmann, R. Michel, P. Heymans:” Evaluating a Textual Feature Modelling Language: Four Industrial Case Studies”, SLE 2010, 3rd International Conference on Software Language Engineering, (2010) , DOI 10.1007/978-3-642-19440-5_23. Abstract. Feature models are commonly used in software product line engineering as a means to document variability. Since their introduction, feature models have been extended and formalised in various ways. The majority of these extensions are variants of the original tree-based graphical notation. But over time, textual dialects have also been proposed. The textual variability language (TVL) was proposed to combine the advantages of both graphical and textual notations. However, its benefits and limitations have not been empirically evaluated up to now. In this paper, we evaluate TVL with four cases from companies of different sizes and application domains. The study shows that practitioners can benefit from TVL. The participants appreciated the notation, the advantages of a textual language and considered the learning curve to be gentle. The study also reveals some limitations of the current version of TVL.

7.1

Introduction

Feature models (FMs) were introduced as part of the FODA (Feature Oriented Domain Analysis) method 20 years ago [Kang 1990]. They are a graphical notation whose purpose is to document variability, most commonly in the context of software product line engineering (PLE) [Pohl 2005A]. Since their introduction, FMs have been extended and formalized in various ways (e.g. [Czarniecki2005A, Schobbens2006]) and tool support has been progressively developed [purevariants 2009]. The majority of these extensions are variants of FODA’s original tree-based graphical notation. Figure 44 shows an example of graphical tree-shaped FM that describes the variability of an eVoting component. The anddecomposition of the root feature (Voting) implies that all its sub-features have to be selected in all valid products, i.e. are mandatory features. Similarly, the or-decomposition of the Encoder feature means that at least one of its child features has to be selected, and the xordecomposition of the Default VoteValue feature means that one and only one child has to be selected. Cardinality-based decompositions can also be defined, like for VoteValues in the example. In this case, the decomposition type implies that at least two,

145

Chapter 7 and at most five sub-features of VoteValues have to be selected. Finally, two constraints impose that the feature corresponding to the default vote value (Yes or No) is part of the available vote values. Voting

Encoder

VoteValues

Default VoteValue [2..5]

Manager

Voter

Abstain

No

NotYetEncoded

Yes

Legend And-decomposition

Xor-decomposition

NotVote

No

Yes

Or-decomposition

[i..j]

Group cardinality

Figure 44 Feature diagram of the PhoneMeeting voting component Over time, textual FM dialects have also been proposed [Deursen 2002, Batory 2005, Boucher 2010, Czarnecki 2010], arguing that it is often difficult to navigate, edit and interpret large graphical FMs. The need for more expressiveness was a further motivation for textual FMs since adding constructs to a graphical language can quickly start harming its readability. Although advanced techniques have been suggested to improve the visualisation of graphical FMs (e.g. [Nestor2007,Cawley 2010]), these techniques remain tightly bound to particular modelling tools and are hard to integrate in heterogeneous tool chains [Dordowsky 2009]. Finally, our experience shows that editing functionalities offered by such tools are usually pretty limited. Based on these observations, we proposed TVL [Boucher 2010, Classen 2009], a textual FM dialect geared towards software architects and engineers. Its main advantages are that (1) it does not require a dedicated editor - any text editor can fit - (2) its C-like syntax makes it both intuitive and familiar, and (3) it offers first-class support for modularity. However, TVL is meant to complement rather than replace graphical notations. It was conceived to help designers during variability modelling and does not compete, for instance, with graphical representations used during product configuration. The problem is that empirical evidence showing the benefits and limitations of existing approaches, be they graphical or textual, is cruelly missing [Boucher 2010]. The goal of this paper is to collect evidence that

146

Evaluating a Textual Feature Modelling Language demonstrates whether TVL is actually fit for practice, which is translated into the following research questions: RQ1 What are the benefits of TVL for modelling product line (PL) variability, as perceived by model designers? RQ2 What are the PL variability modelling requirements that are not fulfilled by TVL? It is important to understand that the goal of this research is neither to compare TVL to other languages, nor to assess capabilities of TVL other than its ability to model variability, nor to compare graphical and textual approaches. Instead, this research aims at identifying the benefits and limitations of TVL. To answer these research questions, we conducted a controlled field experiment following a sequential explanatory strategy [Shull 2007]. It consists of a quantitative data analysis followed by a qualitative one. The quantitative analysis is meant to collect data while the qualitative analysis assists in explaining the outcomes of the quantitative analysis. Quantitative data on TVL is collected via evaluation forms based on a set of quality criteria inspired from the evaluation of programming languages. The main motivation for this is that TVL is in many respects similar to a declarative constraint programming language. TVL was evaluated by five participants working for four companies of different sizes (from 1 to 28 000 employees), domains (hardware and software) and business models (proprietary and free software). Furthermore, TVL was evaluated by participants with different backgrounds in modelling and programming languages. The interviews carried out during the qualitative analysis helped us (1) collect evidence that practitioners can benefit from TVL, (2) identify the types of stakeholders who can reap benefits from a language like TVL, and (3) elicit requirements that are not fulfilled by TVL. The remainder of the paper is structured as follows. Section 7.2 looks into related work on feature-based variability modelling. Section 7.3 recalls the essence of TVL. Section 7.4 describes the research method and the four cases, whilst Section 7.5 presents the results of the quantitative and qualitative analyses. Section 7.6 analyses these results and Section 7.7 discusses the threats to validity.

7.2

Related Work

This section studies related work respectively dedicated to graphical and textual approaches to feature modelling.

7.2.1

Graphical feature models

Most common FM languages are graphical notations based on FODA which was introduced by Kang et al. [Kang 1990] twenty years ago. Since this original proposal, several extensions have been proposed by various authors [Schobbens2006]. Most of these graphical notations are meant to be accessible to non-technical stakeholders. However, working with large-scale FMs can become challenging with such notations. Given

147

Chapter 7 that a FM is a tree on a two dimensional surface, there will inevitably be large physical distances between features, which makes it hard to navigate, search and interpret them. Several tools have been developed to help modellers [Antkiewicz 2004, Beuche 2008,Kastner 2009,Krueger 2007]. Most of them use directory tree-like representations of FMs to reduce physical distances between some features and provide collapse/expand functionalities. More advanced user interfaces and visualisation techniques have also been proposed to attenuate the aforementioned deficiencies (e.g. [Nestor2007,Cawley 2010]), but tools have their own drawbacks. First, building FMs can become time consuming as tools often require lots of clicks and drag-and-drops to create, place or edit elements. Second, they rely on software to visualise a model, meaning that without this software, like for instance on paper, blackboard or on a random computer, they will not work. Furthermore all those tools tend to have poor interoperability, which prevents effective collaboration. Besides all those considerations, some constructs like cross-tree constraints or attributes cannot be easily accommodated into those graphical representations.

7.2.2

Textual feature models

Various textual FM languages have been proposed for a number of purposes. Their claimed advantages over graphical notations are: they do not require dedicated modeling tools and well-established tools are available for text-based editing, transformation, versioning… Furthermore, textual information and textual models can be easily exchanged, for instance by email. To the best of our knowledge, FDL [Deursen 2002] was the first textual FM language. It is the only such language to have a formal semantics. It also supports basic requires and excludes constraints and is arguably user friendly, but it does not include attributes, cardinality-based decompositions and other advanced constructs. The AHEAD [Batory 2005] and FeatureIDE [Kastner 2009] tools use the GUIDSL syntax [Batory 2005], where FMs are represented through grammars. The syntax is aimed at the engineer and is thus easy to write, read and understand. However, it does not support decomposition cardinalities, attributes, hierarchical decomposition of FMs and has no formal semantics. The SXFM file format is used by SPLOT [Mendonca 2009A] and 4WhatReason [Mendonca 2009B]. While XML is used for metadata, FMs are entirely text-based. Its advantage over GUIDSL is that it makes the tree structure of the FM explicit through indentation. However, except for the hierarchy, it has the same deficiencies as GUIDSL. The VSL file format of the CVM framework [Reiser 2009,Abele 2010] supports many constructs. Attributes, however, cannot be used in constraints. The Feature Modelling Plugin [Antkiewicz 2004] as well as the FAMA framework [Benavides2004] use XML-based file formats to encode FMs. Tags make them hard to read and write by engineers. Furthermore, none of them proposes a formal semantics. The Concept Modelling Language (CML) [Czarnecki2009] has been recently proposed but to the best of our knowledge is still a prototype and is not yet fully defined or implemented.

148


7.3

TVL

Starting from the observation that graphical notations are not always convenient and that existing textual notations have limited expressiveness, formality and/or readability, we proposed TVL [Boucher 2010,Classen 2009], a textual alternative targeted to software architects and engineers. For conciseness, we can only recall here the basic principles of the language. More details about its syntax, semantics and reference implementation can be found in [Classen 2009]. The following model will be used to illustrate some TVL constructs. It is a translation of the graphical model presented in Figure 44, which is an excerpt of the complete FM we built for PloneMeeting, one of the case studies. It captures the variability in the voting system that governs the discussion of meeting items. Note that the default vote value is specified here as an attribute rather than a feature. 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15

root Voting { // define the root feature enum defaultVoteValue in {yes, no}; //attribute is either yes or no (defaultVoteValue == yes) -> Yes; //yes requires Yes in VoteValues (defaultVoteValue == no) -> No; // no requires No in VoteValues group allOf { // and-decomposition Encoder { group someOf {manager, voter} }, VoteValues group [2..*] { // cardinality Yes, No, Abstain NotYetEncoded, NotVote, } } }

TVL can represent FMs that are either trees or directed acyclic graphs. The language supports standard decomposition operators [1, 3]: or- , xor-, and and-decompositions. For example, the and-decomposition of the Voting is represented from lines 05 to 12 (group allOf {...}). The xor-decomposition of the Encoder is represented at line 06 (group oneOf {...}). Generic cardinality-based decompositions [Riebisch 2002] can also be defined using a similar syntax (see line 07). Five different types of feature attributes [Czarniecki2002] are supported: integer (int), real (real), Boolean (bool), structure (struct) and enumeration (enum). The domain of an attribute can be restricted to a

149

Chapter 7 predefined set of values using the in keyword. For instance, the set of available values of the enumerated attribute defaultVoteValue is restricted to yes and no (see line 02). Attributes can also be assigned fixed or calculated values using the is keyword. Furthermore, the value of an attribute can differ when the containing feature is selected (ifIn: keyword) or not selected (ifOut: keyword). Several standard operators are available for calculated attributes (e.g. arithmetic operations). Their value can also be computed using aggregation functions over lists of attributes. Calculated attributes are not illustrated in the example. In TVL, constraints are attached to features. They are Boolean expressions that can be added to the body of a feature. The same guards as for attributes are available for constraints. They allow to enable (resp. disable) a constraint depending on the selection (resp. nonselection) of its containing feature. Line 03 is an example of (unguarded) constraint where the assignment of the yes value to the defaultVoteValue attribute requires the selection of the Yes feature. TVL offers several mechanisms to reduce the size of models and modularise them. We only touch upon some of them here and do not illustrate them in the example. First, custom types can be defined and then used in the FM. This allows to factor out recurring types. It is also possible to define structured types to group attributes that are logically linked. Secondly, TVL supports constants that can be used inside expressions or cardinalities. Thirdly, include statements can be used to add elements (e.g. features or attributes) defined in an external file anywhere in the code. Modellers can thus structure the FM according to their preferences. The sole restriction is that the hierarchy of a feature can only be defined at one place (i.e. there is only one group structure for each feature). Finally, features can have the same name provided they are not siblings. Qualified feature names must be used to reference features whose name is not unique. Relative names like root, this and parent are also available to modellers.

7.4

Research Method

This section describes the settings of the evaluation, its goals and the four cases along with the profiles of the companies and participants involved in the study. The section ends with a description of the experiment’s protocol.

7.4.1

Objectives

The overall objective of this paper is to evaluate the ability of TVL to model the variability of a PL as perceived by software engineers. The criteria that we use to measure the quality of TVL are inspired and adapted from [Pratt1984, Holtz 1988]. Originally, these criteria were defined to evaluate the quality of programming languages. We have selected programming language criteria because TVL resembles a declarative constraint programming language whose constructs have been tailored for variability modelling. Finally, TVL should ultimately be integrated in development environments like eclipse or advanced text editors like emacs or vim. TVL is thus likely to be assimilated to a

150

Evaluating a Textual Feature Modelling Language programming language by developers. We outline the quality criteria relevant to our study in Table 14. Table 14 Quality criteria for the evaluation of TVL C1

Clarity of notation

C2

Simplicity of notation

C3

Conciseness of notation

C4

Modularisation

C5

Expressiveness

C6

Ease and cost of model portability Ease and cost of model creation

C7 C8 C9

Ease and cost of model translation Learning experience

The meaning of constructs should be unambiguous and easy to read for nonexperts. The number of different concepts should be minimum. The rules for their combinations should be as simple and regular as possible. The constructs should not be unnecessarily verbose. The language should support the decomposition into several modules The concepts covered by the language should be sufficient to express the problems it addresses. Proper syntactic sugar should also be provided to avoid convoluted expressions. The language and tool support should be platform independent. The elaboration of a model should not be overly human resource-expensive. The language should be reasonably easy to translate into other languages/formats. The learning curve of the language should be reasonable

No specific hypothesis was made, except that the participants had the appropriate expertise to answer our questions.

7.4.2

Cases

The evaluation of TVL was carried out with five participants coming from four distinct companies working in different fields (two in computer hardware manufacture, one in meeting management and one in document management). Table 15, on the next page, summarises the profiles of the five participants involved in the evaluation as well as the company and project they work for. For each participant, we indicate his position, years of experience in software engineering, his fields of expertise, the modelling and programming languages he used for the last 5 years, his experience with PLE and FMs, and the number of years he actively worked on the selected project. For the experience with languages, PLE and FMs, we also mention the frequency of use, i.e.

151

Chapter 7 intensive/ regular/casual/evaluation. By evaluation, we mean that the language or concept is currently being evaluated in the company. The four cases are described below as well as the motivations of the participants to use TVL. 7.4.2.1

PloneMeeting

Description: PloneGov [Plonegov2010] is an international Open Source (OS) initiative coordinating the development of eGovernment web applications. PloneGov gathers hundreds of public organizations worldwide. This context yields a significant diversity, which is the source of ubiquitous variability in the applications [Delannay 2007,Hubaux 2008,Unphon 2009].We focus here on PloneMeeting, PloneGov’s meeting management project developed by GeezTeem. PloneMeeting is built on top of Plone, a portal and content management system (CMS) written in Python. The current version of PloneMeeting extensively uses UML for code generation. However, the developer is not satisfied by the limited editing functionalities and flexibility offered by UML tools. To eliminate graphical UML models, he developed appy.gen [Appygen2010]. appy.gen allows to encode class and state diagrams, display parameters, portlet creation, and consistency rules in pure Python and enables the automated generation of full-blown Plone applications. PloneMeeting is currently being re-engineered with appy.gen. A major challenge is to extend appy.gen to explicitly capture variation points and provide systematic variability management. We collaborate with the developer to specify the FM of PloneMeeting. Motivation: The initial motivation of the developer to engage in the evaluation was to assess the opportunity of using FMs for code generation. He has not used FMs to that end so far because, in his words, “graphical editing functionalities (typically point-and-click) offered by feature modelling tools are cumbersome and counterproductive”. The textual representation of FMs is therefore more in line with his development practice than their graphical counterpart.

152

Evaluating a Textual Feature Modelling Language Table 15 Profiles of the 5 participants Criteria

Plone Meeting

Company

GeezTeem

#Employees

1

70

Location

Belgium

Belgium

Open Source

Type of software Project kickoff date Project stage reached Project version Number of features Position Years of experience in SE Fields of experience Modelling Languages

Programming Languages Experience with PLE Experience with FDs Project participation

CPU Calculation

OSGeneric

NXP

VirageLogic

28000 (Worldwide) The Netherlands

700 (Worldwide) The Netherlands

Proprietary

Proprietary

Proprietary

January 2007

May 2008

June 2009

2004

Production

Production

Development

Production

1.7 build 564

4.2.1

July6, 2009

September30 2009

193

152

19

60

Freelance

Product Line Manager

Senior Scientist

Senior Software Architect

Development Manager

12+

31+

20+

15+

20+

WA, CMS

PM

PM, SPI, SR

McS

ES, RT

UMl (casual)

UML (regular)

UML (regular) Python (intensive), Java-script (regular) 2 years (evaluation) 2 years (evaluation) 3 years

PRISMA prepare OSL Namur S.A.

1 year (casual) 1 year (casual)

UML (casual), DSCT (evaluation) C (casual), Prolog (regular), Java (evaluation) 3 years (intensive) 3 years (intensive)

2 years

1 year

None

C++ (casual)

4 years (regular) 4 years (casual)

C, C++, Visual Basic, Python (all casual), 4 years (intensive) 4 years (intensive)

6 years

2 years

C (intensive)

Legend: CMS - Content Management System, DSCT - Domain Specific Components Technologies, ES Embedded System, McS - Multi-core System, PM - Project Management, RPC - Remote Procedure Call, RT - Real Time, SE - Software Engineering, SPI - Software Process Improvement, SR - Software Reliability, WA - Web Applications

153

Chapter 7 7.4.2.2

PRISMAprepare

Description: Océ Software Laboratories S.A. (OSL) [OSL 2010], is a company specialized in document management for professional printers. One of their main PLs is Océ PRISMA, a family of products covering the creation, submission and delivery of printing jobs. Our collaboration focuses on one sub-line called PRISMAprepare, an all-in-one tool that guides document preparation and whose main features are the configuration of printing options and the preview of documents. In the current version of PRISMAprepare, mismatches between the preview and the actual output can occur, and, in rare cases, documents may not be printable on the selected printer. The reason is that some incompatibilities between document options and the selected printer can go undetected. For example, a prepared document can require to staple all sheets together while the target printer cannot staple more than 20 pages together, or does not support stapling at all. The root cause is that only some constraints imposed by the printers are implemented in the source code, mostly for time and complexity reasons. Consequently, OSL decided to enhance its PL by automatically generating configuration constraints from printing description files. The objective is twofold: (1) build complete constraint sets and (2) avoid the manual maintenance of cumbersome constraints. So far, we have designed a first FM of the variability of the printing properties. Motivation: Feedback was provided by the product line manager of PRISMAprepare. OSL is currently evaluating different modelling alternatives to express the variability of its new PL and generate the configuration GUI. The evaluation of TVL is thus part of their exploratory study of FM languages. Additionally, most of the software engineers dealing with FMs are developers, hence their interest for a textual language. 7.4.2.3

CPU calculation

Description: NXP Semiconductors [NXP 2010] is an international provider of Integrated Circuits (IC). ICs are used in a wide range of applications like automotive, infotainment and navigation systems. They are the fundamental pieces of hardware that enable data processing like video or audio streams. ICs typically embed several components like CPU, memory, input and output ports, which all impose constraints on their possible valid combinations. This study focuses on the FM that models the variability of a video processing unit and study the impact it has on the CPU load. The FM, which is still under development, is meant to be fed to a software configurator. Thereby, it will support the customer during the selection of features while ensuring that no hardware constraint is violated (e.g. excessive clock speed required by the features). The FM also allows the user to strike an optimal price/performance balance, where the price/performance ratio is computed from attributes attached to features and monitored within the configuration tool. Motivation: The evaluation was performed by the developer who originally participated in the creation of the FM with a commercial tool, before and independently from this experiment. Prolog was used for defining the constraints. The major problem is that the time needed to implement the calculation over attributes was deemed excessive compared to the

154

Evaluating a Textual Feature Modelling Language time needed to design the whole FM. This lead the company to consider TVL as an alternative. Furthermore, the developer’s knowledge of a declarative language like Prolog motivated him to test a textual constraint modelling language. 7.4.2.4

OSGeneric

Description: Virage Logic [Viragelogic 2010] is a supplier of configurable hardware and software to a broad variety of customers such as the Dolby Laboratories, Microsoft and AMD. Its variability-intensive products allows its customers to create a specific variant for the manufacturing of highly tailorable systems on chip (SoC). OSGeneric (Operating System Generic) is a PL of operating systems used on SoCs. The produced operating systems can include both proprietary and free software. Each SoC can embed a large variety of hardware and contain several processors from different manufacturers. Motivation: The evaluation was performed by two participants: the lead software architect of OSGeneric and the software development manager. Their PL is currently modelled with a commercial tool. The participants are now considering other variability modelling techniques. Their motivation for evaluating TVL lies in using a language that (1) is more suited for engineers with a C/C++ background, (2) has a lower learning curve than the commercial tool and (3) makes use of standard editors.

7.4.3

Experiment protocol

In this experiment, TVL was evaluated through interviews with the five participants of the four companies. Interviews were conducted independently from each other, except for VirageLogic where the two participants were interviewed together. Two researchers were in charge of the interviews, the synthesis of the results and their analysis. For each interview, they followed the protocol presented in Figure 45. Slides

TVL model

1 Introduction to TVL

2 Presentation of TVL model

Quality criteria

TVL documentation

3 Quantitative evaluation

4 Qualitative evaluation

Evaluation forms

Rationale

Legend Document

Action

Process Flow

Data flow

Figure 45 Interview protocol

155

Chapter 7 The protocol starts with a short introduction to TVL (circa 20 minutes) that aims at giving the participants an overview of the language. At this stage, the participants are not exposed to details of the language. The goal of the second step is to provide the participants with a real TVL model. To keep the effort of the participants moderate, the appointed researchers designed, for each company and prior to the interviews, TVL models that respectively correspond to the configuration menus of PloneMeeting and PRIMSAprepare, and the FMs of the CPU calculation and OSGeneric. The presentation of the TVL model was limited to 30 minutes to keep the participants focused on the understanding of the model and avoid untimely discussions about the quality of the language. During that step, the participants are exposed to more details of the language and discover how their PL can be modelled using TVL. Alternative design decisions are also discussed to demonstrate the expressiveness of TVL. During the third step, the participants fill out the evaluation form presented in Table 16. The evaluation scale proposed to the participants is:

++ + O -N=A

the participant is strongly satisfied; the participant is rather satisfied; the participant is neither satisfied nor dissatisfied; the participant is rather dissatisfied; the participant is completely dissatisfied; the participant is not able to evaluate the criterion.

The results of the evaluation are then discussed during the fourth step. The qualitative information collected during this last phase is obtained by asking, for each criteria, the rationale that lead the participant to give his mark. On average, these two last steps lasted two hours in total.

7.5

Results

Table 16 synthesises the evaluation of TVL performed by the participants of GeezTeem, OSL, NXP and Virage Logic. Note that we kept the evaluations of the two Virage Logic participants separate, has indicated by the two columns under OSGeneric. To facilitate the explanation, we group the criterion into five categories: notation, modularisation, expressiveness, ease and cost, and learning experience. Note that the collaborations with OSL, NXP and VirageLogic are protected by non-disclosure agreements. Therefore, specific details of the models are not disclosed.

156

Evaluating a Textual Feature Modelling Language Table 16 Results of the evaluation of TVL Criterion

Plone Meeting

PRISMA prepare

CPU calculation

O

C1

Clarity of notation

C2 C3

Simplicity of notation Conciseness of notation

+ + ++

C4

Modularisation

O

+ ++ ++ +

C5

Expressiveness Ease and cost of model portability Ease and cost of model creation Ease and cost of model translation Learning experience

-

C6 C7 C8 C9

OSGeneric

O

+ + + ++

++ ++ ++ + ++

+ + ++ + ++

+

++

++

++

++

+

+

++

O

O

+

+

+

O

+

+

++

+

+

+

Notation [C1-C3]. The participants unanimously appreciated the notation and the advantages of text in facilitating editing (creating, modifying and copy/pasting model elements). The NXP and VirageLogic participants liked the compactness of attributes and constraints and the fact that attributes were explicitly part of the language rather than an add-on to a graphical notation. The GeezTeem participant appreciated the ability of the language to express constraints very concisely. He reports that appy.gen, his website generator, offers two major ways of specifying constraints. First, guards can be used to make the value of an attribute depend on the value of another attribute. Secondly, Python methods can be used to express arbitrary constraints. These mechanisms can rapidly lead to convoluted constraints that are hard to maintain and understand. Additionally, developers struggle to maintain these constraints across web pages. The participant reports that at least 90% of the constraints (both within and across pages) implemented in classical appy.gen applications could me more efficiently expressed in TVL. The OSL participant was particularly satisfied to see that TVL is not based on XML. He reported that their previous attempts to create XML-based languages were not very satisfactory because of the difficulty to write, read and maintain them. He also reported that the model represented in the language is much more compact than anything he could have produced with existing graphical representations. The NXP participant was concerned about the scalability of the nested structure, i.e. the tree-shaped model, offered by TVL. He also reports that people used to graphical notations who already know FMs might prefer classical decomposition operators (and, or, xor) rather than their TVL counterparts (allOf, someOf, oneOf). Finally, the participants from NXP and Virage Logic were confused by the fact that the -> symbol can always replace

157

Chapter 7 requires but not the other way around. In their opinion, a language should not offer more than one means to express the same thing. One of the Virage Logic participants reports that attributes might be hard to discern in large models. He suggested to declare them in an Interface Description Language (IDL) style by prefixing the attribute declaration with the attribute keyword. Modularisation [C4]. The ability to define a feature at one place and extend it further in the code was seen as an undeniable advantage as it allows to distribute the FM among developers. The Virage Logic participants both discussed the difference between the TVL include mechanism and an import mechanism that would allow to specify exactly what parts of an external TVL model can be imported but also what parts of a model can be exported. In their opinion, it would improve FM modularisation and module reuse since developers are already used to import mechanisms. Apart from the include mechanism, TVL does not support model specialization and abstraction (as opposed to constructs, e.g. class and specialisation). In contrast, the developer of appy.gen considers them as fundamental. Their absence is one of the reasons that lead them to drop UML tools. Along the same lines, the OSL participant argued that the include should be augmented to allow macro definitions. By macro, the participant meant parametrized models similar to parametrized types, e.g. Java generics. A typical use case of that mechanism would be to handle common variability modelling patterns. Expressiveness [C5]. The GeezTeem participant expressed that TVL is sufficiently expressive to model variability in most cases. However, he identified several constructs missed by TVL that would be needed to model PloneMeeting. First, TVL does not offer validators. In his terms, a validator is a general-purpose constraint mechanism that can constrain the formatting of a field. For instance, validators are used to specify the elements that populate a select list, to check that an email address is properly formatted or that a string is not typed in where the system expects an integer. Secondly, he makes intensive use of the specialisation and abstraction mechanisms available in Python, which have no TVL equivalents. These mechanisms are typically used to refine already existing variation points (e.g. add an attribute to a meeting item) or to specify abstract variation points that have to be instantiated and extended when generating the configuration menu (e.g. an abstract meeting attendee profile is built and it has to be instantiated before being available under the Encoder feature in Figure 44). Thirdly, multiplicities are used to specify the number of instances, i.e. clones, of a given element. Cloning is a fundamental aspect of appy.gen as many elements can be cloned and configured differently in Plone applications. This corresponds to feature cardinalities (as opposed to group cardinalities), which have already been introduced in FM [3], but are currently not supported in TVL. Besides offering more attributes types, appy.gen also allows developers to add parameters to attributes, e.g., to specify whether a field can be edited or requires specific read/write permissions. Type parameters are mandatory in appy.gen to support complete code generation. Finally, in order to be able to display web pages in different languages, i18n labels are attached to elements. i18n stands for internationalisation and is part of Plone’s built-in translation management service. Translations are stored in key/value pairs. A key is a label in the code identifying a

158

Evaluating a Textual Feature Modelling Language translatable string; the value is its translation. For instance, the meeting-item-i18n element will be mapped to Meeting Item (English) and Point de discussion (French). In most cases, several labels are attached to an element (e.g. a human-readable name and a description). The OSL participant also pointed out some missing constructs in TVL. First, default values which are useful in their projects for things like page orientation or paper dimensions. Secondly, feature cloning is missing. In PRISMAprepare, a document is normally composed of multiple sheets, where sheets can be configured differently and independently from one another. Thirdly, optionality of attributes should be available. For instance, in TVL, the binding margin of a page was specified as an attribute determiningits size. If the document does not have to be bound, the binding margin attribute should not be available for selection. The NXP and VirageLogic participants also recognized that feature cloning and default features were missing in the language. Additionally, they miss the specification of error, warning and information messages directly within the TVL model. These messages are not simple comments attached to features but rather have to be considered as guidance provided to the user that is based on the current state of the configuration. For instance, in the NXP case, if the selected video codec consumes most of the CPU resources, the configurator should issue a warning advising the user to select another CPU or select a codec that is less resource-demanding. Since they are a needed input for a configurator, they argued that a corresponding construct should exist in TVL. Ease and cost [C6-C8]. The OSL participant reports that improvements in terms of model creation are, in his words, very impressive compared to the graphical notation that was used initially [Czarniecki2005C]. And since TVL is formally defined, he does not foresee major obstacles to its translation into other formalisms. The NXP and VirageLogic participants report that, no matter how good the language is, the process of model creation is intrinsically very complex. This means that the cost of model creation is always high for real models. Nevertheless, they observed that the mechanisms offered by TVL facilitate the transition from variability elicitation to a formal specification, hence the neutral score. Learning experience [C9]. All the participants agreed that the learning curve for software engineers with a good knowledge of programming languages was rather gentle. Software engineers who use only graphical models might need a little more time to feel comfortable with the syntax. In fact, the NXP and VirageLogic participants believe that people in their teams well versed in programming languages would give a ++ whereas those used to modelling languages would give a , hence their average + score.

7.6

Findings

This section builds upon the analysis by looking at the results from three different angles: (1) the constructs missed by TVL, (2) the impact of stakeholder profiles on the use of TVL, and (3) the tool support that is provided and still to provide.

159

Chapter 7

7.6.1

Language constructs

The analysis revealed that extensions to the catalogue of constructs provided by TVL would be appreciated by the participants. We summarise below those that neither make the language more complex nor introduce extraneous information (such as, e.g., behavioural specifications) into the language.  Attribute cardinality Traditionally, the cardinality of an attribute is assumed to be [1::1] i (oneOf), i.e. one and only one element in its domain can be selected. However, the translation of select lists in PloneMeeting would have required enumerations with cardinalities. For instance, in Figure 44, the vote encoders would typically be captured in a select list, which should be translated into an enumeration with cardinality [1::2]. The absence of cardinality for enumerations forced us to translate them has features. Yet, these select lists typically allow multiple selections, i.e. they require a cardinality like [1::n] (someOf). Additionally, optional attributes, like the binding margin, would require a [0::1] (opt) cardinality. Technically, arbitrary cardinalities for attributes are a simple extension of the decomposition operators defined for features. Their addition to TVL will thus be straightforward.  Cloning All the participants expressed a need for cloning in FMs. They have not been introduced in TVL because of the theoretical problems they yield, viz. reasoning about potentially infinite configuration spaces and managing clonespecific constraints. Feature cardinalities will thus be proposed in TVL when all the reasoning issues implied by cloning will be solved. This is work in progress.  Default values The main advantage of default values is to speed up product configuration by pre-defining values (e.g. the default page orientation). The participant from OSL argued that, if their applications were to include a TVLbased configuration engine, then TVL should contain default values. This would avoid having default values scattered in the source code, thereby limiting the maintenance effort.  Extended type set Far more types are needed in PloneMeeting than TVL offers. Our experience and discussions with the developers show that only some of them should be built in the language. For this reason, only the String, Date and File types will be added to TVL.  Import mechanism In addition to the include mechanism, the participants requested a more fine grained import mechanism. This will require to add scoping to TVL.  Labels and messages In the PloneMeeting case, the participant showed that labels have to be attached to features to provide the user with human-readable feature names and description fields, and also to propose multilingual content. Both the NXP and VirageLogic participants use messages to guide the

160




configuration of their products. Constraints on features determine the messages that are displayed based on the current configuration status. Specialisation In two cases, specialisation mechanisms appeared to be desirable constructs. They would typically allow to overload or override an existing FM, for example, by adding or removing elements from previously defined enum, struct and feature groups, or by refining cardinalities, e.g. from [0::*] to [2::3].

The extensions that could have been added to TVL but that, we thought, would make it deviate too much from its purpose are the following.  Abstraction Although particularly useful for programming languages (e.g. abstract classes in object-oriented programming), we do not see the interest of using abstract FMs. FMs are, by definition, not meant to be instantiated but to be configured.  Method definition TVL provides a static description of the variability of a PL. Methods, even declarative, would embed behavioural specifications within the language. This is neither an intended purpose nor a desired property of the language. Any method manipulating the FM should be defined externally.  Typed parameters In PloneMeeting, typed parameters are extensively used. However, many of these parameters are very technical and case-specific, e.g. editable or searchable fields. Unless we come across more use cases, we will not add parameters to attributes. For cases like PloneMeeting, these parameters can be encoded with the data construct.

7.6.2

Stakeholder profiles

As shown in Table 15, the participants had fairly different profiles. Our population consists of two developers, one designer and two project managers. Their experience with PLs and FMs also differ. Two participants are intensive users, one is a regular and the other two are still in the transition phase, i.e. moving from traditional to PL engineering. Interestingly, these profiles did not have a noticeable influence on the marks given to the notation (C1-C3), ease and cost (C6-C8), and learning experience (C9). They all preferred and attribute grammar-like syntax to a markup-based language like XML, usually considered too verbose, difficult to read and tricky to edit. Furthermore, the Clike syntax was deemed to preserve many programming habits—like code layout, the development environment, etc. Deviations appear at the level of modularisation (C4) and expressiveness (C5). One way to interpret it is that OSL and PloneMeeting are still in the transition phase. This means that they are not yet confronted to variability modelling tools in their daily work. They are more familiar with traditional modelling languages like UML and programming languages like C++ and Python. Compared to these languages, FMs, and TVL in particular, are more

161

Chapter 7 specific and thus far less expressive. Furthermore, in the Plone- Meeting case, the participant developed its own all-in-one website configuration and generation tool, which embeds a domain specific language for statechart and class diagram creation. These observations lead us to conclude that:  stakeholder profiles do no impact the evaluation of the notation. Developers clearly appreciated the textual alternative proposed by TVL. The ease and efficiency of use are the main reasons underlying that preference. Thanks to their knowledge of programming languages, developers naturally prefer to write code than draw lines and boxes. This is usually seen as a time-consuming and clerical task, even with proper tool support. Surprisingly, the participants who were used to graphical models also positively evaluated TVL. They not only liked the language but also the convenience offered by the textual edition interface.  stakeholder profiles influence the preference for the configuration model. The developers looked for ways to reuse the TVL model for configuration purposes. Their suggestion was to remove the unnecessary elements from the TVL model (unselected features and attributes) and directly use it as a product specification. The advantage of this approach is that comparison between products could be immediately achieved with a simple diff. In contrast, the designers were in favour of a graphical model, e.g. a tree-shaped FM, or more elaborate configuration interfaces like configuration wizards or tables.

7.6.3

Tool support

At the moment, TVL only comes with a parser that checks syntactic and type correctness, as well as a configuration engine that supports decision propagation limited to Boolean values. We have also developed plugins for tree editors, namelly NotePad++ (Windows), Smultron (MacOS) and TextMate (MacOS). These plugins provide basic syntax highlighting and collapse/expand mechanisms to hide/show pieces of code. Besides textual editors, out-of-the-box versioning tools like CVS or Subversion already support the collaborative editing of TVL models as any other text file, as reported by the OSL and Virage Logic participants. The interpretation of a change made to a TVL line is as easy as it is for programming code. By simply looking at the log, one can immediately see who changed what and when. In contrast, graphical models usually require dedicated tools with their own encoding, which makes interoperability and collaboration difficult. The configuration capabilities of TVL have recently been applied to re-engineer the configuration menu of PloneMeeting. This resulted in a prototype that demonstrates how it is possible to use an application-specific web interface as frontend for a generic TVL-based configurator. Although very limited in functionality, the prototype gave the participant a better overview of the benefits of TVL. Surprisingly, the PloneMeeting participant was not interested in generating appy.gen code from a TVL model because of the Python code that would still have to be edited after generation. However, generating a TVL model from appy.gen code would greatly simplify constraint specification and validation. Tedious and

162

Evaluating a Textual Feature Modelling Language error-prone Python code would no longer have to be maintained manually, and most of the constraints that are only available in the head of developers would become trivial to implement. Put simply, TVL would be used here as a domain-specific constraint language. We could not produce similar prototypes for the other cases because configuration interfaces were not available and/or access to the code was not granted. A functionality not provided by the TVL tools but requested by the participants is code completion of language constructs, feature names and attributes. Another important functionality would be the verification of the scope of features in constraints. Since a constraint can contain any feature in the FM, it might rapidly become hard to identify whether the referenced feature is unique or if a relative path to it has to be given. The onthe-fly suggestion of alternative features by the editor would facilitate constraint definition and make it less error-prone. By extension, the on-the-fly checking of the satisfiability of the model would avoid wasting time later on debugging it. The downside of such checks is that they can be resource-intensive and should thus be carefully scheduled and optimized.

7.7

Threats to Validity

The evaluation was performed with four PLs and five participants, providing a diversity of domains and profiles. Yet, their diversity did not have a significant influence on the results since we observed a substantial overlap between their comments. Therefore, we believe that the results are valid for wide a range of organizations and products [Yin2002]. The TVL models were prepared in advance by the two researchers and later checked by the participants. Consequently, the expertise of the researchers might have influenced the models and the evaluation of the participants. In order to assess this potential bias more precisely, we will have to compare models designed by participants to models designed by the two researchers. However, TVL is arguably simpler than most programming languages and the modelling task was felt to be rather straightforward. As as consequence, we do not expect this to be a problem for our evaluation. Furthermore, when the participants questioned the design decisions, alternative solutions were discussed based on the constructs available in the language—even those not disclosed during the presentation. The limited hands-on experience with TVL might have negatively influenced the evaluation of the expressiveness, notation and modularisation of the language, and positively impacted the evaluation of the ease and cost and learning experience. That situation resembles the setting of an out-of-box experience [Sangwan 2004]. This gives valuable insight as to how software engineers perceive TVL after a one-hour training and how fast they can reach a good understanding of the fundamental concepts. A more specific problem was the unavailability of proper documentation and the limited access granted to the codebase in the case of OSL, NXP and Virage Logic. This made the modelling of those cases more difficult. In the case of OSL, the development team is still in the SPL adoption phase. This could be a threat as the participant has only been exposed to FMs for reviewing. Therefore, he might have focused on comparing the textual and graphical approaches rather than evaluating the language itself. Along the same lines, the PloneMeeting participant was already reluctant to use graphical FMs and might have

163

Chapter 7 evaluated the textual approach rather than TVL itself. In any case, we believe that the feedback received was more shaped by the expectations and requirements of the participants than by the preference for a textual representation over a graphical one. More generally, one limitation of our study is the relatively small size of the subsystems we could deal with during the experiment.

7.8

Conclusion

Effective representations of FMs are an open research question. Textual alternatives have been proposed, including TVL, a textual variability modelling language meant to overcome a number of known deficiencies observed in other languages. Yet, evidence of the suitability of TVL in practice was missing. In this paper, we systematically evaluated the language by conducting an empirical evaluation on four industrial product lines. Our evaluation of TVL showed that practitioners positively evaluated the notation and that efficiency gains are envisioned in terms of model comprehension, design and learning curve. However, they suggested some extensions like attribute cardinalities, feature cloning, default values and guidance messages that can be used during product configuration. In the future we will focus on integrating the recommended extensions into TVL. Furthermore, the prototype implementation of the TVL parser and reasoner needs to be extended to better support on-the-fly verification of model consistency. To assess these new extensions, live evaluations through modelling sessions are envisaged. To better assess the pros and cons of variability modelling languages, comparative evaluations are planned, too.

Acknowledgements We would like to thank Gaetan Delannay from GeezTeem, Jacques Flamand from OSL, Hans den Boer and Jos Hegge from Virage Logic for the time they dedicated to our evaluation, and Bart Caerts from NXP Semiconductors for his contribution to the development of the CPU-load calculation variability model. This work is sponsored by theInteruniversity Attraction Poles Programme of the Belgian State, Belgian Science Policy, under the MoVES project, the Walloon Region under the NAPLES and the FIRST DOC.A programmes.

164

Part III:

Software Testing

“When we had no computers, we had no programming problem either. When we had a few computers, we had a mild programming problem. Confronted with machines a million times as powerful, we are faced with a gigantic programming problem. “ Edsger Dijkstra

165

166

Chapter 8.

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing

This chapter is an extended version of the paper that was published as: H. Hartmann, J. Bokkerink, V. Ronteltap: "How to reduce your test process with 30%; The application of operational Profiles at Philips Medical Systems”. In: Supplementary proceedings of the 17th IEEE International Symposium on Software Reliability Engineering. This extension is submitted as: H. Hartmann: “A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing”. Abstract. Due to the increasing size and complexity of software the amount of faults is constantly increasing, thereby leading to a reduced reliability. Due to the pressure of time-to-market, it is not feasible to remove all faults, or even the majority of them, before market introduction. It is therefore crucial to focus testing on finding those faults that have the largest impact. Within many organizations, operational profiles are used to focus the test activities on those areas that are most used by the end-user. Similarly risk based testing is used to focus testing on those areas that pose the largest risk to the user and the business. In this paper a statistical model is introduced that captures the relation between faults and failures. Using the model the efficiency improvements of using operational profiles are simulated under different conditions. This analysis shows that using operational profiles improves the test efficiency, however not when a high reliability is required. For risk based testing a quantification is introduced and the analysis shows that an efficiency improvement can be realized when the usage frequency is treated as a separate dimension of the risk matrix, rather than making the usage frequency part of the impact dimension. Because the model that is introduced is based on easy to understand probability theory, it has great explanatory power in illustrating the effects of allocating different amounts of test cases. A case study at Philips Healthcare is presented that shows the applications of the model and the experiences from using it. The results of this paper can be used by practitioners to improve their test planning and can be used by researchers to further develop statistical models for software reliability.

8.1

Introduction

Due to the increasing size and complexity of software products, the number of faults, also known as defects, is constantly increasing [Siewiorek 2004]. This is observed during the test phase and after product release. Due to the pressure on time-to-market it is not feasible to

167

Chapter 8 remove all faults or even the majority of them before market introduction. This results in a product with insufficient reliability [Ulrich 2004], and therefore an increasing field call rate for consumer electronics products [Ouden 2006A] or increasing maintenance efforts [Jacobs 2003]. The costs related to this, can become considerable. It is therefore imperative to make the test activities more effective and achieve the highest possible reliability within the available time for testing. Testing within a project is usually defined by a test plan [Veenendaal 2002, McGregor 2010, Jorgensen 2013]. This plan describes what tests need to be executed and what coverage is required, which is a measure of how thorough each test should be executed. The challenge in defining the right coverage is to find the optimal balance between time-tomarket and costs versus the required reliability [Veenendaal 2014]. The concept of operational profiles was introduced to guide the test planning process [Musa 1993]. In this approach the tests are allocated to different operations according to the operational profile, which is a set of operations along with the probability with which they occur. In many situations different operations will have a different criticality and therefore different reliability requirements. In these situations the operations are classified according to their criticality and operational profiles are generated for each class [Musa 1999]. The notion of different criticality is the rational for risk based testing [Amland 2000, Alam 2013, Schaefer 2014]. Risk based testing is used to prioritize test effort to those development artifacts that pose the greatest risk. These risks are usually based on two dimensions [Veenendaal 2014, Amland 2000, Schaefer 2014]: (1) the probability that a function or feature contains faults and (2) the impact that a failure causes, where the impact is a combination of the damage, or costs, of a failure and the usage frequency of each function or operation. This paper focusses on test efficiency and answers the following research questions:  What are the benefits for the test efficiency when using operational profiles?  Which dimensions in risk based testing require most attention to reduce the overall risks? In this paper a statistical model is introduced which estimates the efficiency improvements for testing when using operational profiles, and which analyzes the dimensions in risk based testing. The inputs for the model are the amount of faults that are expected in a function or operation, the chance that a fault leads to a failure and the operational profile. From this input the expected mean time between failures is calculated and the amount of faults that are found during testing. A case study from Philips Healthcare describes the experiences using this model to derive the efficiency improvements for the test process. This paper is structured as follows. Section 8.2 presents background and in Section 8.3 the statistical model is presented followed by Section 8.4 that shows simulations of using operational profiles. Section 8.5 describes the experiences at Philips Healthcare, while Section 8.6 analyses risk based testing. Section 8.7 provides a discussion and areas for further research and Section 8.8 gives a comparison with related art. The paper finishes with the conclusions in Section 8.9 and acknowledgments.

168


8.2

Background

This section provides background on software reliability and testing.

8.2.1

Software reliability, faults and failures

Reliability is defined as [Lyu 1996]: “The probability that a system functions in a specified environment without failure, for a specified time or a number of natural units”. Example: The reliability of a car can be expressed in the number of years it drives without problems, or the number of kilometers. The most used measure for reliability is Mean Time Between Failures (MTBF), which is the average time between two failures. Person makes zero to many

Environment Input

Mistakes

OR Potentially leads to zero or many

Can be attributed to one or many

Faults Can be attributed to one or many

AND

Revealing mechanism Leads to zero or many

Errors Leads to zero or many

Can be attributed to one or many

Failures

Figure 46 Fault tree of software failures Figure 46 shows a fault tree of software failures [BSI 1998]. A failure originates from a person making a mistake. This can be a mistake made during specification, design or coding. Such a mistake may result in a fault in the software. A fault may lead to an internal error, depending on the way the system is used, i.e. the revealing mechanism. So, a fault in a part of the software that is not executed will not lead to an internal error and failure, and therefore has no influence on the reliability. In some cases, the software is fault tolerant meaning that it recognizes that an error occurs and the system chooses another way to perform the operation. However, an internal error in most cases leads to a failure which is experienced by the user (see Table 17 for the definitions of the used terminology).

169

Chapter 8 Table 17 Definitions in software reliability Mistake Fault Error Failure Failure severity Failure intensity

Incorrect or missing action by a person or persons, that may cause a fault in a program Defect in a system that may cause a failure when executed. In this paper the terms defect and fault will be used interchangeably Discrepancy between the internal state of a system and its desired state Departure of system behavior from the requirements Severity of a failure, i.e. the consequences of a failure for end-user or business Failures per natural or time unit. A way of expressing reliability, commonly as MTBF.

In other words: A software fault is a defect in the code, caused by a human mistake during the creation of the architecture, the design or during coding, and is developer oriented. A software failure occurs during execution, and is user oriented. Therefore, the purpose of testing should be to obtain a product which has the lowest failures intensity, i.e. highest MTBF, rather than focusing on the lowest amount of faults. There are alternative ways to create reliable software [Lyu 1996]. The primary way is to prevent that the software contains faults, e.g. by using formal methods. Secondly it is possible to remove the faults through testing and it is possible to prevent faults becoming failures during execution, known as fault tolerance [Pullum 1999]. This paper focuses on testing as one way to reduce the failure intensity.

8.2.2

Test planning

In many organizations the verification and test process starts with a planning phase in which test activities are defined, including test techniques and coverage [Veenendaal 2002, McGregor 2010, Jorgensen 2013]. Verification and test techniques may include code reviews, static analysis and dynamic testing, such as white- and black box testing. Test coverage is a measure of how thorough each test is executed. The test coverage criterion determines how many test cases need to be created. Since this is the most intensive part of test activities [Pohl 2005A], the test coverage largely determines the required effort. The challenge in test planning is to find the right balance between time-to-market and quality [Veenendaal 2014]. For safety critical systems, such as nuclear power plants, the reliability is essential thus requiring a high coverage, while for consumer products the time-to market is more important and less coverage is needed [Musa 1999].

8.2.3

Software reliability engineering and operational profiles

Software Reliability Engineering (SRE) is defined as: “A practice for quantitatively planning and guiding software development and tests, with the emphasis on reliability and availability. It provides the solution to estimate, predict and measure the rate of failure occurrences in software [Musa 1999]”. This is done by determining when the software is

170

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing good enough to release, minimizing the risks of releasing with serious problems thereby avoiding excessive time-to-market due to over testing. Within SRE, two types of tests are used: the certification test and the reliability growth test: The certification test is meant to establish that an application works the way it should. This test requires a stable system and does not involve debugging. The decision to be made is whether to accept or reject the software, based on the measured MTBF versus the target that is set. The reliability growth test is a test aiming to improve the product’s reliability over time. It has the purpose of finding and removing faults. It includes feature, load and regression tests. Both these tests use operational profiles to guide the test. An operational profile is a quantitative characterization of how a system is used. The operation profile is a set of software operations, along with the probability with which they occur. For obtaining the operational profile, a number of steps have to be taken. First the customer profiles have to be identified, the users and the system modes, i.e. the mode in which the system operates. For each user the functions he/she uses have to be identified. The functions are usually further separated into operations. For each of these profiles, the occurrence probabilities have to be determined and are combined to obtain the overall operational profile [Musa 1993]. Musa claims that when the tests are allocated to the most-used operations, the reliability level will be the maximum practically achievable when the software needs to be shipped due to imperative schedule constraints before all the software faults can be removed [Musa 1993]. In many situations different functions and operations will have a different severity when this function or operation fails [Musa 1993]. In these situations the operations are classified according to their criticality and operational profiles are generated for each criticality class. Test time is allocated to each category, where most test time and resources are allocated to the categories with the highest criticality.

8.2.4

Risk based testing

Risk based testing is a widely adopted practice for setting priorities in testing [Amland 2000, Alam 2013]. Risk based testing is based on the assumption that not all functions and operations should be tested equally rigorously, but that (only) the most important faults should be found and removed. For areas with high risks, the tests are executed earlier, using more comprehensive test techniques and higher coverage targets are set [Schaefer 2014], while for non-critical areas lower coverage targets are set [daMotaSilveira 2011]. Usually a matrix visualizes the risks, which consists of two dimensions: The probability dimension captures the likelihood that a function fails and the impact dimension captures the usage frequency and consequence that a function fails [Schaefer 2014]. The values for probability and impact are determined by the various stakeholders in a project, e.g. the product managers, developers and testers. These values are based on expert opinion, rather than rigorous statistics. Through a guided discussion the participants score the factors that determine the values [Veenendaal 2014].

171

Chapter 8 For the probability the following factors are commonly used [Jorgensen 2013, Veenendaal 2014, Schaefer 2014]: Complex or changed areas, new technology and tools, number of people involved, experience of the developers, whether the software is developed by developers located in different geographical locations, time pressure and areas that revealed many defects in the past. For the impact the following factors are commonly used [Jorgensen 2013, Veenendaal 2014, Schaefer 2014]: visibility of the function to the user, the impact on the business when a function fails and the usage frequency. The values of the individual factors are added, sometimes using a weighted sum [Veenendaal 2014], and visualized in a matrix, of which an example is shown in Figure 47. 50

F3

F4

Impact

F1

High F2

F5

0

low 1

Medium Probability

30

Figure 47 Example of a risk matrix In this example the functions F1 and F3 are regarded as having a high risk and should be tested intensely, while less test effort should be allocated to function F5. The risk matrix is used to plan the test which includes the test techniques, coverage and the order in which functions should be tested.

8.3

A Statistical Model to Estimate the Improvement on Test Efficiency

In this section a statistical model is introduced that captures the relation between faults and failures, and which calculates the reliability improvement of allocating different amount of test cases to each function and operation.

8.3.1

Model elements and assumptions

The statistical model is based on the definition of faults and failures as shown in Figure 1 and defined in Table 1. It uses the amount of faults as input as well as the usage frequency. Furthermore each fault has a chance of leading to a failure, later referred to as failure probability of a fault, which depends on the revealing mechanism. Mistakes and errors are not modelled. The model uses the following assumptions:  Each fault can be assigned to exactly one operation.

172

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing  Each fault has probability that it manifests as a failure when this operation is used. The following assumptions about the test process are made [Oshana 1994, [Walto 1995]:  Testing is a stochastic process and a test case is a random selection of the input space as in black-box testing.  Faults are repaired directly and correctly, no new faults are introduced and no new functionality is added during the test process.

8.3.2

Definitions:

The model uses the following definitions:  A product consists of different operations Oj, with j = 1, 2, 3, etc.  The software that implements an operation can contain a number of faults. Because not each fault has the same failure probability, groups of faults are used that have different probabilities of leading to a failure. So there are fault groups with i = 1, 2, 3….. and in each group i every fault has the same failure probability.  Pi,j is the failure probability, which is the probability that a fault from fault group i of operation j manifests during execution.  Ni,j is the number of faults of group i in operation j.  P(Oj) is the chance that operation Oj fails.  Rtot is the total number of test cases that are executed during a test cycle.  Rj is the amount of test cases allocated to operation j.  N(Oj) is the number of faults that are found with Rj test cases.  Tj is the time that an operation j takes to execute.  OPj is the probability of occurrence of operation j as part of the operational profile, also referred to as usage frequency.

8.3.3

Calculation of the MTBF

The probability that an operation fails is the probability that one or more of its faults manifests as a failure. The chance that an operation Oj fails then is: P (O j )  1   (1  Pi , j ) i

Ni , j

.

(8.1)

The MTBF of a single operation is determined by the time to execute an operation divided by the chance that an operation fails: MTBF( j )  T j / P (O j )

(8.2)

The MTBF of the product is the weighted average of the times the operations take to execute, divided by the weighted average that these operations fail. The weighted averages are determined by the operational profile: MTBF  (  OPj * T j ) /( OPj * P(O j )) (8.3) j

j

173

Chapter 8

8.3.4

Calculation of the number of faults found during testing:

The number of faults that are found during a test for operation Oj is the sum of the faults that are found for each fault groups i. The number of faults that are found for each group i is the probability that such a fault has occurred during this test multiplied by the number of faults from that group: N (Oj)   (1  (1  Pi , j )

i

Rj

) * Ni, j

(8.4)

With these formulas the MTBF before testing can be calculated, the number of faults that are found for each fault group and operation and the MTBF can be calculated after the faults have been removed.

8.4 Simulations of the Test Efficiency when using Operational Profiles This section presents the effect on the test efficiency of using operational profiles with the following scenarios:  Test allocation based on operational profiles versus a uniform test distribution.  The effect when a large number of test cases are executed to obtain very high reliability.

8.4.1

Simulation of test time allocation based on operational profile versus a uniform distribution

For this example a fictitious product with 4 operations is used as shown in Table 18. Table 18 Fictitious operational profile Operation

Percentage of use

A

50

B C D

30 10 10

Each operation contains a total number of initial faults, being 100, which are equally divided over three fault groups. The first group has a failure probability 0.01, the second group of 0.005 and the third group of 0.001. By using multiple fault groups, rather than one, it is avoided that the simulations show extreme results. For instance when a single group is used with a high probability of failure then a few test cases will already find many faults and not many test cases are needed to find all faults. On the other hand, when faults have a low failure probability, then the MTBF is small and many tests are needed to reveal faults. The time that a test case runs is set to 1, in this example without dimension, so could be seconds, minutes, or hours.

174

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing Table 19 shows a comparison between three scenarios. 1. The first scenario is a uniform distribution, i.e. to each operation the same amount of test cases is allocated, using a total of 250 test cases. 2. In the second scenario the amount of test cases is also 250 but is allocated according to the operational profile, so to operation A and B more test cases are allocated than to operations C and D. 3. In the third scenario only 70% of the total 250 test cases is used but also allocated according to the operational profile. Table 19 Comparison of MTBF with uniform versus OP distribution Uniform distribution of test cases Operation A

Faults 100

Tests 62.5

Faults found 27

remaining Faults 73

B C D

100 100 100 400

62.5 62.5 62.5 250

27 27 27 108

73 73 73 292

MTBF

2.4

MTBF After

3.5

Test case distribution using OP Operation Faults Tests Faults found A 100 125 43 B 100 75 31 C 100 25 12 D 100 25 12 400 250 98

Remaining faults 57 69 88 88 302

MTBF

4.1

2.4

MTBF After

Test case distribution using OP with 70% of test cases Operation Faults Tests Faults found Remaining faults A 100 87.5 34 66 B 100 52.5 23 77 C 100 17.5 9 91 D 100 17.5 9 91 400 175 75 325 MTBF

2.4

MTBF After

3.5

Table 19 shows the initial amount of faults and the amount that have been found during testing. This example shows that the MTBF after the fault removal with the number of test cases allocated according to the operational profile (OP) is higher than when allocated according to a uniform distribution, i.e. 4.1 versus 3.5 while fewer faults need to be repaired.

175

Chapter 8 The third scenario shows the results using OP with only 70% of the test cases, which results in the same MTBF and significantly less faults need to be repaired, i.e. 75 versus 108. In other words, when using operational profiles the test effort could be reduced with 30% for this example. The result of this example cannot be generalized, since they depend on the number of faults, the amount of test cases and failure probabilities. The operational profile in this example has a moderate ratio between the occurrences probabilities of the operations with the highest occurrence probability versus the lowest, i.e. only a ratio of 1 to 5. In reality the ratio is much larger, e.g. 10 times or even 100 times as much. With such operational profiles the benefits will be higher as well when the same amount of faults per operation is assumed.

8.4.2

A large amount of test cases to obtain a high MTBF

In the simulations in the previous section still a significant proportion of faults remained in the system. In practice it might be needed to remove more faults to obtain a higher MTBF. In Table 20 a comparison is shown of the MTBF of the uniform distribution versus an allocation according to the operational profiles, using different amount of test cases. The MTBF is shown after the test and removal of faults, and the efficiency improvement. The efficiency improvement is defined by the difference in percentage of test cases that is needed to obtain the same MTBF. E.g. an efficiency improvement of 23% means that 77 % of the test cases according to the operational profiles results in the same MTBF as the uniform distribution. In this example, the same operational profile, number of faults, fault groups and failure probability are used as in the previous example, in Section 8.4.1. Table 20 Comparison with large amount of test cases

Uniform MTBF After Remaining faults Operation A Operation B Operation C Operation D Total remaining faults OP MTBF After Remaining Faults Operation A Operation C Operation C Operation D Total remaining faults Efficiency improvement in %

176

Number of test cases 250 500 1000

5000

10000

3.5

5.2

10.4

102

366

73 73 73 73 292

57 57 57 57 128

38 38 38 38 156

10 10 10 10 40

3 3 3 3 12

4.1

6.3

11.7

93

306

57 69 88 88 302 30

38 52 78 78 246 23

23 34 63 63 183 12

3 8 23 23 55 -5

0 2 13 13 28 -12

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing This example shows that when the amount of test cases increases, the benefits of using operational profiles are smaller in situations with many test cases, i.e. when a high reliability is required, and using operational profiles is even less efficient than using a uniform distribution. The reason is that testing according the operational profiles leaves defects in the system for those parts that have a low occurrence probability, operation C and D, while for the operations with higher occurrence probability, A and B further testing will hardly reveal any additional faults and these operations are basically over tested. This effect differs per situation, however, with a growing number of test cases for some operations no faults will be present anymore, to start with for those to which many test cases are allocated, i.e. have a higher occurrence frequency. Applying more tests to these operations will not have an effect on the overall reliability anymore. Therefore, for high and ultrahigh reliability the use of operational profiles is less efficient for reliability growth testing. To obtain a very high MTBF the tests should be aimed at finding ALL faults most efficiently rather than allocating more test cases to operations that are most used, since this lead to over testing. Therefore a better strategy to obtain very high reliability is to allocate more test time to operations that contain many defects and/or are hard to find, i.e. have a lower failure probability.

8.5

Case Study: Philips HealthCare

This section describes the experiences at Philips Healthcare with operational profiles and using the statistical model.

8.5.1

Problem description and objective

Philips Healthcare manufactures a wide range of systems with a large portfolio of imaging equipment, i.e. systems for image acquisition, such as Magnetic Resonance Imaging (MRI) and Computer Tomography (CT). Other products are used to retrieve and visualize the medical images which are used by medical experts to create a diagnosis. Healthcare Informatics is one of the business groups of Philips Healthcare. Healthcare Informatics is responsible for the development and marketing of software solutions for visualization, processing, archiving and distribution of medical information. Delivering software products with high quality is essential for Healthcare Informatics to remain competitive in this market. Despite ample testing activities of Healthcare Informatics software products, still too many field problems are encountered and consequently too much effort has to be spent on maintenance activities. To reduce maintenance effort, more relevant faults, i.e. faults that lead to failures during the usage by the end-user, must be found during the test phase where the effort for testing cannot be extended since new product features need to be delivered to the market in a timely fashion. The application of the software reliability engineering method at Healthcare Informatics had the aim to define the process of establishing and applying operational profiles and to investigate the benefits and consequences.

177

Chapter 8 As part of this project, the operational profile has been determined for two products. The product MIP DICOM Viewer was used as a pilot project to gain experience. After the pilot project was finished, the operational profile has been determined and applied in the development of a new product, EasyWeb 4.1. For reasons of clarity, the experiences from MIP DICOM Viewer are described rather than for EasyWeb 4.1.

8.5.2

Obtaining the operational profile

Since there were no end-users for MIP DICOM Viewer available for determining the profiles, the operational profile was determined with the help of applicants from Healthcare Informatics. The applicants represent end-users, are often former radiologists and have close contacts with the end users in hospitals and are therefore knowledgeable with the way the Healthcare Informatics products are used. To determine the occurrence rates of all functions and operations, all steps that are executed by the applicants while using the product, were recorded. For MIP DICOM Viewer, a record-and-playback tool, originally meant for the creation of automated tests, has been used to register the steps. Every executed step has been recorded and a snapshot of the screen has been taken. These snapshots provided the usage frequencies of the different functions and operations for the defined user types. The user profile consists of the following group:  Radiologists, e.g. orthopedists.  General Practitioners, who are receiving results from specialists.  Two types of Cardiologists.  Patients, who are getting the possibility to view a selection of the images. In addition to the user profile frequency and usage frequency also the user weights have been determined to capture the relative importance of different user types [Aggarwal 1993]since the reliability for some users, especially the orthopedists is more important than for other users. The reliability for the patients is regarded as much less important. Table 21 Show a subset of the operational profile of MIP DICOM Viewer for the Orthopedist. This gives a good representation, i.e. the final operational profile of the different user and user weights hardly differs.

178

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing Table 21 Operational profile for the orthopedists Function Operation File View Preferences Help Right Mouse Button Scroll Zoom Pan Gray Level …….. Vertical Task Bar Play/Stop Movie Increase to Frame Rate Decrease to Frame Rate Image Information ……… Other Switch from Series in File

% 9.1 1.4 1.4 0.0 53.2 2.8 7.4 5.8 10.5 27.4 6.4 1.0 1.9 2.8 7.7

The profiles show that there are functions and operations that were hardly used while others were more heavily used, i.e. some operations are used more than 10 times than other operations. Some operations are not used at all.

8.5.3

Estimation of benefits for test efficiency

Since data of the number of faults and failures from Philips Healthcare are confidential, only the method to determine the benefits can be shared in this paper.  To use the model, described in Section 8.3, the following data is required:  The initial number of faults Ni,j.  The number of tests executed Rtot.  The failure probability Pi,j of a fault and the fault groups with the number of faults per group.  The operational profile.  The time than an operation j takes to execute Tj. The MTBF and the number of faults found during the test can be calculated from the above data. The operational profile is obtained and the time to execute an operation can be measured. Finally the number of test cases can be defined. So the initial number of faults Ni,j and Pi,j the chance a fault manifests need to be estimated. Data from MIP DICOM viewer was used to estimate the benefits for EasyWeb 4.1; the product under development. Since these products have similar functionality and are based on the same technology the fault density and failure probability of a fault are expected to be similar.

179

Chapter 8 The initial number of faults was estimated based on comparing the code size of the EasyWeb 4.1 in relation to that of MIP DICOM Viewer. A uniform distribution of faults across the operations and an average failure probabilities Pi,j for the faults was assumed, so only one fault group was used. For MIP DICOM viewer the number of faults were estimated based on the problem reports that were entered in the various stages of the project, furthermore the amount of test cases were known. So using formula (8.4) only Pi,j was the remaining unknown and could be determined by simulation. The benefits could then be estimated by using the total amount of test cases and comparing a uniform test case distribution versus a distribution according to the operational profile. In this analysis a separation was made between operations that belong to different criticality classes.

8.5.4

Presentation of the results

It was experienced that, when explaining the use of operational profiles to an audience that is unfamiliar with this method, the model turned out to be very useful for demonstrating the effects. Because the simulations not only show the outcome, but also show the amount of faults and faults found, it was regarded as very convincing even though there was a general agreement that the results of the simulations should not be regarded as highly accurate. The difference between faults and failures was made clear through this model and the effect on the reliability. Whereas, in the past, test architects were typically aiming to find as many faults as possible, the test architects realized that the tests should be focused on finding the most relevant faults. As a result of the presentation of the results, Philips Healthcare decided to adopt the use of operational profiles to guide the test effort.

8.6

A Statistical Analysis of Risk Based Testing.

In this section the statistical model will be used to evaluate the priority setting of test activities using risk based testing. This section starts with experiences from the Philips Healthcare, a comparison between risk based testing and operational profile driven testing and a quantification of risks in risk based testing.

8.6.1

Different reliability requirements in healthcare products

While reliability is extremely important for medical devices, there is a large difference for each function or operation. For instance, when an image acquisition is used during medical interventions, e.g. surgery, the reliability requirements are now extremely high because it is immediately life threatening when a system breaks down. For other systems, such as the retrieval and storing of data the reliability requirements are lower and differ per operation. For instance it is crucial that the correct image is stored and retrieved because this might lead to a wrong diagnosis. For operations such as printing or zooming, the reliability requirements are lower. Therefore Philips Healthcare has adopted risk based testing for planning the test cases and resources.

180


8.6.2

A comparison of risk based and operational driven testing

In software reliability engineering the functions and operation are classified according to their severity and operational profiles are created for each class. For certification tests, the reliability targets are used to accept or reject the software. For growth testing most tests are allocated to the highest severity classes and the operational profiles are used to allocate test cases within those classes. In risk based testing the priorities are based on expert opinion, which includes an analysis of areas where many faults are expected, i.e. the probability dimension. For each function it is then defined how much test time should be allocated as part of reliability growth testing where the functions with the highest risk obtain most test cases. Risk based testing does not include reliability certification tests. So the main differences are, that in operational profile driven testing the prioritization does not include an estimation of the faults, while risk based testing is based on expert opinion and does not include reliability certification tests.

8.6.3

Quantification of the risks

For a quantification of the risks and application of the statistical model a quantification of the risk dimensions and factors is introduced. 1. The probability dimension is the area where most problems are expected [Veenendaal 2014]. These areas are estimated based on a number of factors [Veenendaal 2002, Schaefer 2014]: Complex areas, changed area, new technology, time pressure and so on. These factors are also used for defect density predictions [Neufelder 2000]. In the statistical model these will be represented by the number of faults from which, together with the failure probability of the faults, the failure probability per operation P(Oj), is calculated. 2. The impact dimension consists of two parts: The frequency of use and the damage [Veenendaal 2002, Schaefer 2014]:  The frequency of use will be represented by the occurrence probability of the operations as part of the operational profile [Jorgensen 2013].  The damage describes the impact to the business or end user when a function fails [Amland 2000, Schaefer 2014] also referred to a cost and consequences of a failure [Veenendaal 2014]. In general this is very difficult to quantify and in [Veenendaal 2014] it is suggested to use a logarithmic scale to represent the costs using values for costs of 1, 3 and 10. In the statistical model this scale will be used. Using these quantifications for probability and impact, the risk per operation can be quantified. Here R(Oj) is defined as the risk for operation Oj. The risk per operation is defined as the multiplication of the chance that an operation fails, P(Oj), with the change that it is executed, OPj, and the damage that occurs when an operation fails D(Oj): R (O j )  D(O j ) * P (O j ) * OPj

(8.5) 181

Chapter 8 The failure chance P(Oj), is defined in formula (8.1) while OPj and D(Oj) are given. Note that in [Jorgensen 2013] a formula for risk is given that only consist of the operation profile and the damage, and in [Amland 2000] a formula is given that only includes costs and the chance that a function contains a fault. So both this related art miss a factor in the formula for risk.

8.6.4

Simulation of different scenarios

An estimation of the benefits of using risk based testing is obvious and will therefore be omitted. What is more worthwhile is to determine what the impact of test effort is on the different risk dimensions. For evaluating effects of the factors in risk based testing, three different functions, i.e. set of operations, are compared that are all three classified as having medium risk. Using the statistical model the effect of adding extra test cases to each of these functions, F2, F3 and F4, from the risk matrix shown in Figure 48 Risk matrix with 5 functions, is analyzed and compared. Here we assume that one operation is assigned to one function only as in [Jorgensen 2013]. High F4

Impact

F1

Medium Risk F3

High Risk

F5 F2

Low

Low risk Low

Probability

High

Figure 48 Risk matrix with 5 functions The Function F2 has a high probability, i.e. high expected number of faults, while the impact is regarded as low. Function F3 has medium probability and medium impact, while Function F4 has low probability and high impact.

182

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing Table 22 Simulations of allocating test cases to different medium type risks Function Number of Faults D(Oj): Damage per failure OPj: Usage frequency Risk type Test cases

F2 500 (high) 1 (low) 0.05 (low) medium 100 500

F3 200 (med) 3 (med) 0.2 (med) medium 100 500

F4 50 (low) 10 (high) 0.5 (high) medium 100 500

Faults Found

187

384

75

154

18

34

P(Oj): Failure Chance after tests

0.72

0.16

0.40

0.07

0.12

0.02

R(Oj): Risk per operation

0.04

0.01

0.24

0.04

0.60

0.09

Risk Reduction

0.03

0.20

0.51

Table 22 shows the results of the simulations. Again the same fault groups have been used as in the examples in Section 8.4 and the time for a test case to run is 1. In the simulation 500 versus 100 test cases are executed and the faults have been removed. The table shows the risk reduction, which is the reduction in risk, of allocating a larger number of test cases to a function, versus a lower amount of test cases, i.e. 500 versus 100. This table shows that by allocating 500 test cases instead of 100 to function F4 reduces the risk by 0.51 per time unit. For the Functions F3 and F2, which have more faults but lower damage per failure and lower usage frequency, the risk reduction is substantially smaller, namely 0.20 and 0.03. In other words: although all three functions are classified as having medium risk, which would mean that for each of them about the same test cases would be allocated, the simulations show that having more test cases for function F4 leads to a much higher risk reduction than allocating this amount of test cases to F3 and F2. The result of these simulations can be explained for two reasons:  The impact consist of two parameters that both have a linear effect on the damage per unit of time, because these are based on the quantification of the risks as given in formula (8.5).  The reduction of faults, due to the number of test cases, is nonlinear because increasing the number of test cases, e.g. by two times, will not reduce the number of faults two time, since the same faults may be found in two different test cases. Basically when a function contains many faults, few cases will already reveal a significant amount of faults. The example shows that 100 test cases reveal almost half of the amount of faults as 500 test cases. Different values for damage, number of faults and usage frequency will give another value for risk reduction as results. In this example there is a ratio of 10 on usage frequency, damage and also number of faults, which is regarded as realistic. Since the influence of damage and usage frequency on the reliability will be similar for other products, it is likely that the observed effect is applicable to many projects.

183

Chapter 8

8.6.5

Implications for the application of risk based testing

The current practice of risk based testing would be improved, i.e. result in products with higher reliability, when the test cases are allocated using three dimensions: 1. Probability; this is the same as currently used and estimates the amount of faults. 2. Frequency of use, also the same as currently used, but now as separate dimension. For this dimension the operational profile can be used. 3. Damage or Costs of a failure. Again this is also the same as currently used but now also as separate dimension. This means that the highest amount of test cases should be allocated to the product of these three dimensions. This new approach can be easily incorporated in current practice, since the required information is already gathered; only the prioritization will be set differently. The effects will differ per situation. For instance, in the situation when the use frequency is close to uniform, this new approach will not lead to large differences in comparison with the current approach. However this new approach is more fine-grained, and based on the quantification of risks, and therefore lead to a better allocation of the amount test cases in most situations, and can never lead to a worse allocation of the amount of test cases. While the proposed new practice will allocate fewer test cases to operations with high amount of faults with lower usage frequency and damage, these test will still reveal a higher amount of faults. So the time needed to resolve this higher amount of faults will be higher and needs to be planned a priori.

8.6.6

Conclusions on risk based testing

From the analysis the following conclusions are drawn:  The two dimensional representation of risks, in which frequency of use is part of the impact of a failure, does not provide good guidance because there is a too strong focus on removing faults, rather than reducing the amount of (severe) failures.  A representation using three dimensions, i.e. Number of Faults, Frequency of use and Damage gives a better representation.  Incorporating this improved analysis in current practice will not require a large change in way of working since the required information is already available.

8.7 8.7.1

Discussion and Further Research Accuracy and extensions to the statistical model

The model introduced in this paper is based on the assumption that a fault can be allocated to one function or operation, rather than assigning a fault to a development artifact, such as software code, that is used by multiple operations. This means that when testing one operation, this may also reveal faults in other operations and thus would reduce the effect of testing based on usage frequency. For a more precise model the design of the system is needed so that a fault can be allocated to different operations and thus manifests during the tests of these operations. In the case of Philips Healthcare a product line approach is adopted [Pohl 2005A] in which software components are allocated to functions (see [Linden 2007]

184

A Statistical Analysis of Operational Profile Driven Testing and Risk Based Testing Chapter 13) and therefore such mapping would be possible between different functions and the expected number of faults that are shared. This type of modeling can be added to the statistical model by introducing fault groups that are shared by multiple operations. The accuracy of the predictions also relies on the input to the model. For the number of faults a large amount of research is available, e.g. [Neufelder 2000], and also on the creation of operational profiles. The remaining input is Pi,j the chances that a fault becomes a failure. In the case study this was based on a past project and only an average value was estimated. More research is needed to derive more accurate values. It may be expected that these values will differ per application area; e.g. for user interface applications or database applications, a fault may reveal itself quickly while for embedded systems, where a fault may only occur in very specific circumstances, e.g. related to timing aspects and different hardware configurations the failure chance for many faults will be much smaller [Gray 1985]. While the accuracy and the model can be improved, the author believes that the conclusions drawn on the use of operational profiles and risk based testing are valid since improvements to the model are not likely to results in large differences. Furthermore, the simulations with the current model show clear effects that for which the reasons behind these effects could be clearly argued.

8.7.2

Use of the model in practice

In this paper the estimated efficiency improvements are based on black box testing. Similarly as operational profiles and risk based testing are also used to plan the test activities for other test techniques [Veenendaal 2002], [Musa 1999], the planning of test activities in general and setting the priorities for other types of tests can also be based on simulations with the model. Here further research is needed. The model in its current form is only able to give a rough estimate and provides general trends and conclusions. Model enhancements are identified in this paper which could make this model suitable for further improving the planning of test activities. For instance, when the values for the probability and impact in risk based testing are estimated more precisely, e.g. based on a fault density prediction and market analysis, the statistical model can be used for optimizing the distribution of test cases, in relation to the available test time. From the experiences in demonstrating the simulations to experts as well as non-experts on testing, the author believes that the model may also be very suitable for educational purpose. From simulations with the model the difference between faults and failures, failure intensity and so on become clear and choices made during test planning become very tangible. Furthermore the model uses basic probability theory and can easily be implemented using spreadsheets.

8.8


The benefits of using operational profiles have been discussed in earlier work, e.g. by [Musa 1993], but this work focused on the benefits of having reliability targets and an improvement of the engineering process. In prior art it has been stated that using operational profiles also would lead to a higher efficiency by finding more relevant faults but none has

185

Chapter 8 provided an estimation, nor did that prior art recognize that for products that requires very high reliability, using operational profiles is less efficient. In the work on decision support, e.g. [Rus 1999] different strategies are discussed, but no quantification for test efficiency for risk based testing and using operational profiles is given. In none of the prior art on risk based testing a quantification of the benefits is given, nor has prior art compared the priorities for the different dimensions in risk based testing. Therefore prior art did not identify that the current practice of combining damage and usage frequency may lead to too much focus on reducing the number of faults rather than the risks. The use of different targets for MTBF has been discussed [Musa 1999] but did not address the number of faults to prioritize the test cases. In the work on weighted operational profiles [Aggarwal 1993] it is suggested to allocate more test cases to functions and operations that are regarded as more important, but also this related art does not provide quantification.

8.9

Conclusions

In this paper a statistical model is introduced that estimates the benefits of using operational profiles and risk based testing. The simulations and case study gave two results:  Guiding tests according to operational profiles improves the test efficiency however not when very high reliability is required. When a very high reliability is required, test cases should focus more on finding faults in operations that contain many faults.  The priority allocation in risk based testing has a tendency of focusing too much on reducing the amount of faults rather than reducing the failure intensity. An efficiency improvement can be realized when the usage frequency is used as a separate dimension of the risk matrix rather than making the usage frequency part of the impact dimension. This improvement can easily be incorporated in current practice. The case study at Philips Healthcare showed that the model can be applied in practice and resulted in the adoption of using operational profiles to allocate test priorities. Because the model is easy to understand, it has great explanatory power in illustrating the influence of different amounts of test cases on the products reliability. The results of this paper can be used by practitioners to evaluate and improve the prioritization of test time allocation and can be used by researchers as a basis to further develop statistical models on test efficiency. The model in its current form is only able to give a rough estimate and provide general trends and conclusions. Further improvements are identified in this paper which could make this model suitable for a more precise planning and optimization of test activities.

Acknowledgments The author thanks Vincent Ronteltap, Jarmila Bökkerink and Arjan den Hartog from Philips Healthcare for their contributions to the case study and Peter Sonnemans from the Technical University of Eindhoven for his help on the statistical model.

186

Chapter 9.

Risk Based Testing for Software Product Line Engineering

This chapter is an extended version of the paper that was published as: H. Hartmann, F. van der Linden, J. Bosch: Risk based testing for software product line engineering. SPLC 2014: 227-231, DOI 10.1145/2648511.2648535. Abstract. The variability of product lines increases over time thereby leading to an increasing effort for testing. Since the available time for test activities is limited an efficiency improvement is needed to ensure that products have sufficient quality. This paper introduces risk-based testing for software product lines. Our approach is based on risk based testing for single system engineering which is extended with a dimension that captures the percentage of product variants that use a particular development artefact. Based on the risk of development artifacts, the priorities for testing during domain and application engineering are determined. We demonstrate our approach using a case study from an existing product line and discuss the tool support that is needed. We conclude that the basic ideas behind risk-based testing product lines are intuitive, pragmatic in nature, and provide the means for practitioners for guiding the test effort for different test levels.

9.1

Introduction

Software Product line engineering improves the efficiency of developing variants of software products by using a set of common artifacts [Pohl 2005A]. The amount of test effort as a percentage of the total development effort has dramatically increased [Siewiorek 2004]. The testing costs for product lines are high and effort reduction strategies can significantly improve profitability and productivity [daMotaSilveira 2011]. In most organizations the time for testing is limited and therefore many products cannot be tested thoroughly, resulting in lower product quality, a loss of market share or financial loss [Tassey 2002]. It is therefore essential to focus the test activities to those areas where the biggest risks are. The development and testing of software product lines is usually separated into domain engineering and application engineering. The main focus of testing during domain engineering is on unit and subsystem testing of the reusable development artifacts while that of testing during application engineering is on integration and system testing [McGregor 2010]. Testing during domain engineering has the limitation that the large combination of variation points and combination of components leads to a combinatorial explosion of possible configuration which, given the limited time that is available for testing

187

Chapter 9 is unfeasible. Furthermore, for many of the components and subsystems of a product line it is not known how many of the derived products use a specific component and what their quality requirements are. As a result it is unclear how the limited test effort should be allocated to the different components. Risk Based Testing (RBT) is a widely adopted practice in single system engineering [Amland 2000, Alam 2013]. It is used to prioritize test effort to those development artifacts that pose the greatest risk. This risk is based on two dimensions (1) the probability that a function or feature contains defects and (2) the impact that a failure causes. The risks are identified by the different stakeholders and visualized in a risk matrix [Amland 2000, Veenendaal 2014, Schaefer 2014]. The approach described in this paper extends risk based testing to software product line engineering (SPLE) in the following ways:  For each development artifact it is decided, based on its percentage of use in the set of variants, whether it should be tested during DE or that test should be deferred to AE. The percentage of use forms an additional dimension in RBT.  The percentage of use of development artifacts and impact when it fails are derived from the features they implement. To capture the risks and identify the most critical components, this paper uses feature models [Kang 1990] in which each feature is associated with a percentage of the use of a feature as well as the impact of a feature. As an example, Figure 49 shows a diagram of a quantified feature model of a subset of the variability of a digital television. DTV I=5

Analog Reception

NetTV

Size

(70 %) I=5

(20%) I=2

(100%) I=4

3D (10%) I=1

PAL

NTSC

28

32

48

(20%) I=5

(50%) I=3

(35%) I=2

(40%) I=3

(25%) I=4

Figure 49 Quantified feature model of a digital television This diagram shows that 70% of the variants contain Analog Reception of which 20% supports PAL and the other 50% NTSC, etcetera. Furthermore, in this example, PAL is a feature that has high impact on the user or the business (value=5) while 3D isn’t (value = 1). This paper shows that these feature models can be used to quantify the use of development artifacts in the different configurations, since development artifacts are linked to features. The risk matrix is derived by including the impact of a feature and the probability of a component causing a failure. In general a feature model with quantification, which will be denoted as quantified feature models (QFM), has to adhere to consistency rules. In the example in figure 1 we see that the

188

Risk Based Testing for Software Product Line Engineering percentage of PAL and NTSC together are equal to the percentage of their parent feature. These consistency rules can become complex thereby leading to inconsistent QFMs. Therefore this paper also discusses the modeling aspects and the tool support that is needed to support the risk assessment process. In this paper we answer the following research questions:  How can risk based testing by applied in Product line Engineering?  What are the consequences for testing during domain and application engineering?  What tool support is required? This paper is structured as follows: Section 9.2 gives a background on product lines testing and risk based testing. Section 9.3 describes risk-based testing for software product lines, while Section 9.4 presents a case study and Section 9.5 discusses tool support. This paper concludes with a comparison with related art, our conclusions and a description of further research.

9.2

Background

This section provides the background on testing of product lines and risk based testing and provides the problem description.

9.2.1

Strategies for product line testing

In software product line engineering tests are executed during domain engineering (DE) and during application engineering (AE) [Pohl 2005A, McGregor 2010]. Usually the testing is separated into different test levels:  A unit test verifies a component or software unit against its specification.  An integration test verifies the interaction between integrated software units and components.  A system test verifies the whole system against the specification of the product. Sometimes sub systems, or compound component are used thus introducing an additional level of testing. In dividing the test activities between DE and AE the following strategies are known [Pohl 2005A, daMotaSilveira 2011, Kang 1990]: Brute force: All test levels and combinations of variation points and components are performed during DE. Given the combinatorial explosion of the amount of features and combination of components this is unfeasible in most situations. Testing product by product: All test levels are performed during AE. The advantage of this strategy is that only the components and variation points are tested that are actually used. The disadvantages of this strategy are that defects are found late in the development process and, since some components may be used in many products, many tests are redundant as the same defects may be found in different products. This leads to additional effort to correct the defects in the various configurations. Division of responsibilities: In this strategy the unit tests are performed on the reusable components during DE, while the integration and system test are performed during AE. The

189

Chapter 9 advantage of this strategy is that no additional effort is needed during DE to test the interaction of components. A disadvantage is that still a large, possibly huge, number of tests of reusable components is needed and that failures that are caused by the interaction of components cannot be found early. Test assets for reuse: In this strategy the test assets, containing variability, are created during DE. During DE the common parts are tested while the application specific parts are tested during AE using the reusable test assets. Compared to the Division of Responsibility strategy, in this strategy the tests can be performed during AE and only for those variation points that are actually used. Sample application strategy: A few sample applications are selected, based on a selection of components and variation points, and all levels of testing are performed. During AE each application needs to be retested with its specific components and bound variation points. The advantage of this strategy is that it shows that an application can be created from the reusable components and that a selection of the components is also tested on their interfaces between them. A disadvantage of this strategy is that the creation of a sample application causes overhead since this application will not be delivered to customers. In practice the Division of responsibility is, to our knowledge, the most applied strategy and some case studies are described [McGregor 2010, Ganesan 2007], the Brute force and Testing product by product strategies are not advised [Pohl 2005A] and are not further discussed in this paper.

9.2.2

Defining test techniques and coverage

The test plan defines which test techniques and test coverage are required for each test level [McGregor 2010]. The test techniques may include reviews, static analysis and dynamic testing, such as white- and black box testing. Test coverage is a measure of how thorough each test, for each test level, needs to be executed. This defines what percentage of the code is tested, e.g. the function or statement coverage and what percentage of the requirements are tested. The test coverage determines how many test cases need to be created, which is the most labor intensive part of testing [Pohl 2005A]. The challenge in defining the required test techniques and coverage is to find the right balance between the optimal time-to-market and costs versus the required quality [Veenendaal 2014]. For safety critical systems, such as healthcare equipment, the reliability is essential thus requiring a high coverage, while for consumer products the time-to market is more important and less coverage is needed [Musa 1999]. For instance for some of the components test may include code reviews and dynamic testing with a 90% coverage, while for other components, where less quality is required, only dynamic testing with a 50% coverage is sufficient.

9.2.3

Risk based testing

In many companies the concept of RBT is used to support the planning of test activities for system testing [Amland 2000]. RBT is based on the notion that not everything can be tested and that (only) the most important defects should be found and removed. For areas with high risks the tests are executed earlier, using more comprehensive test techniques and higher

190

Risk Based Testing for Software Product Line Engineering coverage targets are set [Schaefer 2014], while for non-critical areas lower coverage are set [daMotaSilveira 2011]. Usually a matrix visualizes the risks, which consists of two dimensions: The probability dimension captures the likelihood that a function or feature fails and the impact dimensions capture the consequence that a feature or function fails. The values for probability and impact are determined by the stakeholders in a project, e.g. the product managers, developers and testers. These values are based on expert opinion, rather than rigorous statistics. Through a guided discussion each of the participants scores the factors that determine the values [Veenendaal 2014]. For the probability the following factors are typically used: complex or changed areas, new technology, number of people involved, time pressure and areas that revealed many defects in the past. For the impact typically the following factors are used: cost and consequences of a failure, visibility to the end user and usage frequency [Schaefer 2014]. The values of the individual aspects are added, often using a weighted sum, and this gives the risk matrix, of which an example is shown in Figure 50. 50

F3

F4

Impact

F1

High F2

F5

0

low 1

Medium Probability

30

Figure 50 Example of a risk matrix In this example the features F1 and F3 have a high risk and should be tested intensely, meaning that more comprehensive test techniques should be used and higher coverage targets should be set, while for feature F5 less test effort should be allocated.

9.2.4

Problem description

Given the limited time that is available for test activities it is needed to test those components and variation points that have the biggest impact on the product quality and support the strategies most efficiently. More specifically the following questions need to be answered:  Division of responsibilities: Which components and which variation points should be tested during DE and for which the tests can be deferred to AE?  Test assets for reuse: For which components and variation points should reusable test assets be created during DE?

191

Chapter 9 

Sample application strategy: Which components and variation points should be selected for a sample application?

9.3

Applying Risk Based Testing to Product Line Engineering

This section describes how RBT can be applied to product line engineering.

9.3.1

Approach

Our approach is based on the following additions to RBT in single system engineering:  The risks at system level are translated into risks to individual development artifacts, such as software components, units or sub-systems.  The development artifacts are identified which pose the largest risk to the product line as a whole.  A selection is made of which artifacts need to be tested during DE and for which testing can be deferred to AE.

9.3.2

Obtaining the risks at component level

In order to obtain the risk and set priorities for components and subsystems, the risks that are identified at system level have to be allocated to those components and subsystem that implement the features. Since development artifacts are linked to the features [Pohl 2005A], the quantification of components follows from the quantification of features. Look at the following example, presented in Figure 51. This example is simplified for reasons of clarity. The figure shows a feature model with the components that implement the features, of which part of the links between them are shown. FF I=5 C1 Pr. = 6

FE I=5

FD (20%) I=1

C2 Pr. = 10 C3 Pr. = 8

FA (10%) I=2

FB (20%) I=3

FC (70%) I=5

C4 Pr. = 2

C5 Pr. = 8 C6 Pr. = 5

C7 Pr. = 1

Figure 51 Feature model with linked components The components are linked to the features, with the following dependency rules:  C1 → FD i.e. Component C1 is present in a configuration when feature FD is present in the configuration,

192

Risk Based Testing for Software Product Line Engineering 

C2 → FA, C3 → FB, C4 → FC AND FD, i.e. Component C4 is present when both features FC and FD are present in the configuration.  C5 → FC AND ¬FD, i.e. Component C5 is present when both FC is present but not feature FD.  C6 → FC OR FD, i.e. Component C6 is present when either features FC or FD is present in the configuration,  C7 → FF i.e. Component C7 is present in every configuration. When defining the risks of an individual component, three dimensions are relevant. The first two dimensions are equal to those used in RBT for single system engineering, namely the probability and the impact. The third dimensions originates from Product Line Engineering, namely what percentage of product variants use a specific feature, and therefore component. The probability of a component: The probability that a component fails is shown in the figure, as attribute of a component. This is defined as a number between 1 and 10. The impact of a component: The impact of a component is derived from the features it implements and is a number between 1 and 5. Since the impact of a compact depends on the impact of the highest impact of the linked features, we take the maximum value of the features it implements, rather than e.g. an average value. Thus we get: I(C1) = I(FD) = 1, I(C2) = I(FA) = 2, I(C3) = I(FB) = 3, I(C4) = Max(I(FC), I(FD)) = 5, I(C5) = I(FC) = 5, I(C6) = Max(I(FA), I(FD)) = 5, I(C7) = I(FF) = 5. The percentage of component usage: The percentage for the components, P(C1), …P(C6) follow from the percentages of the features. Here we assume that the percentages of the features are independent from each other. So we get: P (C1) = P(FD) = 20 %, P (C2) = P(FA) = 20 %, P (C3) = P(FB) = 20 %, P (C4) = P(FC AND FD) = 14 %, P (C5) = P(FC AND ¬FD ) = 56 %, P (C6) = P(FA OR FD ) = 76 %. Note1: In this example we have used a direct link between a component and a feature. In a more generic case a component, or other development artifact, may be indirectly linked to a feature. For instance a component may depend on the presence of another component. For instance an additional component C8 is present when component C1 or C3 is present. Furthermore a component may contain software units, which presence in a configuration also depends on a variation point. Note2: As stated in this example it assumed that feature percentages are independent, e.g. the choice for feature FC and FD are independent. However, the choice of one feature may influence the choice for another In Section 9.5 tool support and quantified feature models are discussed in more detail.

9.3.3

Risk matrix for reusable components

The results of the scores can be presented in a matrix. Since we have three dimensions rather than two, the scores for impact and probability have been combined into one axis.

193

Chapter 9 100

Percentage

C7

C6 C5

C1

C2

C3 C4

0 1

Probability X Impact

50

Figure 52 Risk matrix for reusable components

9.3.4

Risk based testing during domain engineering

The risk matrix is the basis for making choices during DE depending on the different strategies: Division of responsibilities: For this strategy the components and variation points that are used in many products should be tested as part of DE, especially those that have a high probability and implement features with high impact. The components that are used in a small percentage of products and have little probability and impact should not be tested during DE. Test assets for reuse: Since creating reusable test assets is work intensive a selection can be made. This selection is similar to the division of responsibility strategy. Sample application strategy: Again here we see that the components and variation points that are used in many products should be included in the sample application, while the others should not be. During the testing of the sample application more test effort should be devoted to the components that implement features with high impact and have high probability of causing a failure. A matrix such as shown in Figure 52 provides clear guidance. Some examples: Component C7 is used in all products and therefore should be tested during DE although no intensive testing is required as the quality is already considered as sufficient. Component C5 has high risk and is used in the majority of products and should be tested intensively. Component C3 and C4 are not used in many products and therefore testing should be done during AE only, while Components C1 and C2 are hardly used and because they pose little risks do not need to be tested in separation anymore. The selection of components based on their risks for the product line addresses the disadvantages of the different strategies. For the Division of responsibilities strategy it is avoided that components that are not used heavily are tested, for the Test assets for reuse strategy only a limited set of reusable test artifacts need to be created while for the sample application strategy a combination of components is selected for which the probability is high that they will be used in an actual product. In this way it is avoided that effort is spend during DE which cannot be justified.

194


9.3.5

Risk based testing during application engineering

During this phase risk based testing can be applied very much in the same way as risk based testing for single system engineering and a risk matrix can be applied to capture the risks of the features for a specific application. An important input to the risk table, related to the probability dimension, is whether components have already been tested during DE. For instance, those components that have already been tested intensively during DE, and with the variation points used for that application, should not be tested heavily anymore. AE can focus on application specific components and on the interaction between components.

9.3.6

Comparison of risk analysis processes

The process of RBT for product lines differs from that of single system engineering, see e.g. [Veenendaal 2014]. Table 23 shows the differences. Table 23 Comparison of risk analysis processes Step

Single System

Product Lines

Estimate impact of a feature

Yes

Yes

Estimate percentage of a feature in the product line

Not applicable

Yes

Calculate percentage for each component

Not applicable

Yes

Estimate probability per component

Yes

Yes

Visualize the risks in a risk matrix

Yes

Yes

Choose which components should be test during DE and which during AE. Prioritize test activities.

Not applicable

Yes

Yes

Yes

9.4

Case Study: Philips Healthcare

This section describes a case study of Philips Healthcare that uses RBT as part of their software product line testing.

9.4.1

Introduction to the case study

Philips Healthcare manufactures a wide range of medical imaging equipment, including systems for image acquisition such as X-ray, Magnetic Resonance Imaging (MRI) and Computer Tomography (CT). Philips uses a product line approach for the Philips Medical Workspot (PMW) which is a product line of system used by radiologists to analyze the images that are created by the different medical imaging equipment [Linden 2007]. The

195

Chapter 9 PMW supports storing, retrieving and exchanging of medical images as well as applications that assist the radiologists to determine a diagnosis. The product line is based on a reference architecture and reusable components, see Figure 53. DE and AE are strictly separated in different organizational units. The platform development group is responsible for creating and testing the components as well as the platform. The product groups, i.e. the groups that create the PMW for the CT, MRI X-ray systems etc., are responsible for creating and testing the end-product and bringing it to the market. The platform for the PMW consists of components that are common for all products as well as optional component and software units that are used by more than one product group. This platform is a semi-final product meaning that the platform can be configured by the product groups. The product groups can add components from the set of reusable components; modify these and add product specific components. External regulations: Because healthcare equipment requires high quality the approval of legal authorities is needed to sell this equipment. This includes the identification and resolving of risks. For the PMW this means that for features that can cause a safety or security risks additional measures have to be taken, which includes a validation of the test cases. These features include the analyses and reporting of medical images. For other features, such as printing of cleaning up databases less rigorous testing is required. Generic components with optional parts

Platform

WorkSpots (Products)

Reusable components for multiple products

Domain Engineering by platform group

Application Engineering by product groups

Figure 53 Philips Medical Workspot platform development

9.4.2

Testing and priorities

Testing is performed at three levels: (1) Unit and Component testing (2) Testing of the Platform and (3) Testing of the End Products:

196

Risk Based Testing for Software Product Line Engineering (1) Unit and Component Testing: The test of the reusable component is performed by the DE teams, while the product specific units and components are tested by the product groups. For all components test automation is applied where each component has 100 % requirement coverage and at least 80 % line coverage. For a selected set of components a test script validation procedure is in place which checks the intended use of the test script against its test design and outputs. These components are: (A) Components that implement Safety and Security Features. (B) Components that are used by multiple customers and for which it is decided that the responsibility for the quality of these components lies with the DE team. This means that these internal customers do not separately test these components during AE. (2) Testing of the platform: The platform tests are executed by the DE Team. This team uses RBT where the process of Veenendaal [Veenendaal 2014] is used and the factors from Schaefer [Schaefer 2014]. The probability dimension is based on 10 factors such as the complexity of the component, whether it is new or implemented with 3rd party components and whether the components contained many defects in the past. The Impact dimension is based on 3 factors: The damage a feature might give; the usage frequency and the commercial importance. An example is given in Figure 54 in which, for reasons of confidentiality, the names of the features have been removed:

Figure 54 Risk matrix for Philips Medical Workspot platform In this analysis an exception is made for features that can cause safety or security risks. Even if the analysis of probability and impact shows that the feature is not of high risk, it will be treated as such. (3) Testing of the end product: The product groups perform testing at system level and testing of product specific components. For system tests the standard method of RBT is applied, but now including the features that have been added by the product team. Furthermore the product teams obtain information from the usage of features at the customer’s site and in-house measurement. This information is used as input for the risk matrix [Hartmann 2006, and chapter 8] and for performing reliability and performance testing.

197

Chapter 9

9.4.3

Selection of components for domain engineering

For each component it is identified which of the products use these components and thus the amount, and percentage, of products that use a component is identified. During meetings with the various stakeholders, such as architects and product managers, it is decided for which components the testing should be done by the DE team and which components should become part of the platform. Furthermore the risks are identified for each component. In this way the most critical and most used components are not only tested in isolation but also tested in combination with other components, as part of the platform.

9.4.4

Tool support

The variability is managed hierarchically, using different mechanisms and different solutions at many levels of the hierarchy (see [Linden 2007], Chapter 13). At the top level component selection is the main mechanism. The relation between the features and the components that implement them is a simple 1-to-n relation and therefore no feature and family modelling tools, that represent these relations, are used. In other words, it is clear to the stakeholders which components implement which features and therefore the discussion is about the components and their criticality rather than the criticality of features.

9.4.5

Analysis and summary of this case study

The test activities for this product line is a combination of two strategies: (1) Division of responsibility: The DE group is responsible for the testing of the reusable components and has to ensure that these components function properly in the platform. The product groups are responsible for testing application specific components and the end product. (2) Sample application: The semi-final product serves both as a platform as well as a sample application to test the reusable components and their interaction. RBT, using a risk table, is applied for the platform as part of DE and during AE by the product groups. This case study only validates a part of our approach presented in Section 9.3:  The selection of components to be tested by the DE team and which is part of the sample application is based on which and how many, i.e. percentage, of the products use a component.  Feature models are not used. Therefore the use of QFMs to derive the quantification of component usage and their impact is not validated by this case study.

9.5

Tool Support

In current practice spreadsheets are used for RBT in single system engineering and in the case study the risks and percentage of use where directly captured at component level and also do not require special tool support. However, when QFMs are used, as shown in Section 9.3, new requirements arise because spreadsheets do not provide a scalable solution.

198

Risk Based Testing for Software Product Line Engineering Therefore, this section describes the modeling requirements and evaluates prior art to serve as a basis for further research.

9.5.1

Requirements

Tool support for RBT should support the process of creating the risk matrix. Here we focus on the steps which are specific for RBT in Software Product Line Engineering, see Section 9.3.6: Estimate percentage of a feature in a product line: This estimation is done by the product managers of the different product groups as part of the commonality variability analysis. This can be based on a prediction of products that still need to be developed together with information from products that are already delivered to the market. These estimations need to be combined and captured in a tool. This tool should support the use of Quantified Feature Model, which will be defined in the next subsection. Calculate percentage for each component: It should be able to model the dependencies between a feature model and the development artefacts, where the percentage of a development artefact, such as a component, can be derived from the features it implements.

9.5.2

Definition of quantified feature models

A QFM is a feature model in which each feature Fi is associated with a percentage P(Fi). This percentage is calculated as follows: We define the complete set of configurations as Ω. We then define: S(Fi). This is the set of configurations in which Fi is part of the configuration. We then define C(S(Fi)), as the number of elements, or Count, of the set S(Fi). Consequently C(Ω) is the total amount of configurations. The percentage is then P(Fi) = C(S(Fi))/C(Ω) * 100%. Since features may have relations between them, e.g. parent child, requires relations and OR-groups, their associated percentages are also related. A QFM is valid when the feature model and the associated percentages are consistent. Some consistency rules are listed below (a more comprehensive set and their proofs is presented in Section 9.8:  A mandatory feature Fm is present in every configuration so has a percentage of 100%.  An optional feature Fo can have a percentage: 0 ≤ P(Fo) ≤ 100  Parent-child relation: The percentage of a child feature Fc cannot be greater than that of the parent feature Fp.: P(Fc) ≤ P(Fp).  Requires relation: The percentage of the feature Fi cannot be higher than a feature Fj it requires: Fi → Fj then P(Fi) ≤ P(Fj).  Excludes relation: When one feature Fi excludes another Fj, then the sum of the percentages of these features cannot be more than 100%: Fi → Fj then P(Fi) + P (Fj) ) ≤ 100%.

199

Chapter 9

9.5.3

Extended feature models

The quantification of a feature can be captured by using extended feature models which are feature models that contain attributes and may include constraints among attributes and features 0. Existing commercially available tools such as pure::variants [pure-variants 2006] and Gears [Biglever 2014] support this functionality. Figure 55 shows a feature model of the example from Figure 49 modeled in pure::variants.

Figure 55 Screenshot of a feature a family model with quantification and consistency checking The figure shows that each feature contains an attribute and a constraint that captures the consistency rule for the alternative child features PAL and NTSC of the optional feature Analog Reception. This consistency rule provides an error message when inconsistent percentages are entered. The figure also shows a family model, which contains the reusable components with the percentage and impact derived from the feature model and the probability that is related to the component. As in the feature model, the family model also supports dependency relations with other development artefacts. Furthermore the percentage and impact for each component can be automatically derived from the feature models. However, while existing tools support the basic functionality, they have two types of practical limitations.  In situations where multiple features are involved complex consistency rules may occur: As an example consider the following scenarios: (1) Suppose we have two features (Fa, Fb) that exclude each other: Fa → ¬Fb and both features require another feature Fc: Fa → Fc, Fb → Fc. Then the percentage of

200

Risk Based Testing for Software Product Line Engineering Fc is larger than the sum of the percentages of Fa and Fb together: P(Fc) ≥ P (Fa) + P (Fb) (2) Suppose we have three features (Fa, Fb, Fc, Fd). Feature Fc and Fd are alternative features. Feature Fc requires Fa and excludes Fb; Feature Fd excludes Fa and requires Fb. As a consequence Fa and Fb exclude each other and we get the same consistency rule for Fa and Fb as described above. To recognize and capture this type of relation in a model, becomes complex since the entire f model has to be analyzed.  The percentages of features may not be independent as in the example of Section 9.3.2 where components C4 implements the combination of feature FC and FD. This would introduce additional percentages that capture these combinations. Note that in the work of Benavides et. al [Benavides 2005] a feature commonality is defined as the percentage of products containing a feature, also using extended feature models. However, this definition and the method described to obtain the percentages are based on the percentage of a feature as part of the possible set of configurations, rather than the expected or actually created set.

9.5.4

Probabilistic feature models

An alternative to the use of extended feature models this section discusses Probabilistic Feature Models (PFMs) [Czarniecki 2008]. PFMs are feature models with soft constraints that express relations between features with a certain probability. PFMs can be used as belief measures where the probabilities can also be viewed as a relative frequency of seeing a particular outcome of an experiment in a large number of trials. In the case of QFMs these large number of trials represent all configurations and the percentage that is associated with each feature represent the probability that a particular feature is selected. QFMs can be modeled with PFMs by adding a soft constraint to each dependency relation instead rather than using a feature attribute. For instance a parent with child features will have a soft constraint that describes the likelihood that a child feature is selected when the parent is selected. The percentages of each feature are automatically derived by the PFM through choice propagation, which in its initial state gives the percentage of each feature without a specific selection. Since PFMs are based on belief measures they do not provide consistency checking for the hard dependency relations, such as requires or excludes relations, since belief measures introduce a chance, rather than a hard constraint. An advantage is that PFMs can be used to derive the percentages of features combinations, as show in the example in Section 9.3, i.e. feature FC and FD.

9.5.5

Further research on tool support

The current tools that are used for RBT, i.e. spreadsheet, are widely available and are easy to use, but do not provide a solution for modelling QFMs and the relation to development artifacts. Current variability management tools support extended feature models including modelling of development artifacts, however complex relations between features and their

201

Chapter 9 quantification require an intensive analysis of the feature model. Probabilistic feature models support a quantification using soft constraints, but do not provide the consistency checking. Further research is needed to determine whether the current modelling approaches can be extended to support QFMs or that a new approach is required. Furthermore it needs to be evaluated what type of tools will be practically applicable in the industry. It should be realized that RBT is based on estimations rather than rigorous statistics and therefore a precise consistency checking may not be needed. Since our case study didn’t include QFMs this needs to be further analyzed. We believe that QFMs will not only benefit RBT, but may also benefit the commonality variability analysis. An estimate of feature usage can support the decision process on what features should be included in the product line and which should become application specific. Furthermore quantification can help in deciding which features should become optional and which should become mandatory, e.g. when used in a large percentage of the product line.

9.6


To our knowledge there is no prior art that discusses RBT for software product lines, and recent surveys confirm this [Lamancha 2013, Lee 2012, doCarmo 2012,]. We compare our research with methods to test efficiency, the selection of the test cases and test case generation.

9.6.1

Related art on test efficiency

Ganesan et.al [Ganesan 2007] discusses different test strategies, and uses a financial model to obtain the best strategy. In their work they didn’t include risk as a factor and therefore didn’t address individual quality requirements of software components. Ensan et.al [Ensan 2011] reduce the test space by selecting the most desirable features and also capture this as part of a feature model. Their work is similar to ours by focusing on a set of features, but in their work they didn’t address risks and quality requirements nor distinguished between testing during DE and AE.

9.6.2

Related art on selection of test cases

Several related art exists to limit the amount of test cases by reducing the combinatorial explosion, such as Kim et al. 0. While this analysis maybe used to limit the amount of test cases, this would still lead to a huge amount of test cases which remains unfeasible in most situations. However this related art can be combined with the work in this paper. RBT can be used to preselect the development artifacts while the work on test case selection can further reduce the test space.

202


9.6.3

Related art on test case generation

A number of papers discuss the automation of test cases to handle the amount of variability, e.g. Segura et. al [Segura2010], Henard et.al [Henard 2012]. In the work of Devroey et. al [Devroey 2014] the likelihood of execution of a feature is used to generate test cases based on the usage model. In our work we have introduced the percentage of variants that use a feature rather than its usage. In RBT the usage is part of the impact of a feature. The percentage of use can be used to decide between testing during DE and AE while usage frequency is used to determine the required level of testing. However, the main difference between this related art and our work, is that RBT does not generate test cases, but is the basis for test planning and to support the test strategy. Therefore our work and this related art can very well be combined as they are complementary contributions. Another, related, difference is that our work is independent from the chosen technology and does not require a model or detailed knowledge of the behavior of a system. Furthermore, none of this related art discusses the risks as basis to prioritize tests. Regarding the automated creation of test cases; it should be realized that in many cases visual inspection may be required to validate the outcome of a test [Veenendaal 2002]]and for these situations automated tests have limited value. For these situations, since verifying the outcome of such tests is very time consuming, reducing the amount of test cases while ensuring quality is crucial.

9.7

Conclusions and Further Research

In this paper we showed that the concept of risk based testing fits very well in the theory and practice of software product lines. We introduced a third dimension to risk based testing, i.e. the percentage of use of a feature in the variants of the product line. The three dimensions provide the criticality for an individual development artifact, such as a component or software unit, which is used to set the priorities for domain and application engineering. The approach can be applied with different test strategies and addresses the advantages and disadvantages of these strategies. We discussed the required tool support as a basis for further research. We identified the limitations of using spreadsheets, which is the current practice in risk based testing for single system engineering, the limitation of extended feature models and that of probabilistic feature models. The case study showed that risk based testing is used in practice at various test levels of the test process, during domain as well as application engineering. In the case study the quantification of features and component was not formally captured as part of a feature model but spreadsheets were used to capture and visualize the risks that were identified at the level of components and software units. We showed that our approach can be combined with related art, especially on the selection of test cases and test automation, since these contributions are complementary. The contributions of this paper are three fold:

203

Chapter 9 

It provides an approach for practitioners to apply risk based testing in software product line engineering.  It provides academia a view on how practitioners set the priorities for test activities in product line engineering which enables further research on test efficiency.  It provides the basis for further research, especially on the topic of quantified feature modelling. We are certain that risk based testing will further find its way to the industry because of its intuitive nature and because it fits very well within existing practices. While our case study showed the application of risk based testing, it didn’t validate the use of quantified feature models. Therefore we plan further case studies.

Acknowledgements The authors thank Peter van Loon for his contributions to the case study of the Philips Healthcare Medical Workspot.

9.8

Appendix: Consistency Rules of Quantified Feature Models

This appendix contains the proofs of the consistency rules from Section 9.5.2. A mandatory feature Fm is present in every configuration so has a percentage of 100%. Proof: S(Fm) = Ω => C(S(Fm)) = C(Ω) => P(Fm) = P(Ω) = 100%. An optional feature Fo can have a percentage: 0≤P(Fo)≤100. Proof: ∅⊆S(Fo) => C(∅)≤ C(S(Fo)) => 0≤ P(Fi) ; S(Fo)⊆Ω => C(S(Fo))≤C(Ω) => P(Fo)≤P(Ω) => P(Fo)≤100% Parent-child relation: The percentage of a child feature Fc cannot be greater than that of the parent feature Fp. Proof: In each configuration where the child feature Fc is present, then also the parent Fp is present S(Fc)⊆S(Fp) => C(S(Fc))≤C(S(Fp)) => P(Fc) ≤ P(Fp). Requires relation: The percentage of the feature Fi cannot be higher than a feature Fj it requires: Fi → Fj then P(Fi) ≤ P(Fj). Proof: This is the same as for a parent-child relation since a parent child relation is a special type of requires relation. Excludes relation: When one feature Fi excludes another Fj, then the sum of the percentages of these features cannot be more than 100%: Fi → ¬Fj then P(Fi) + P (Fj) ) ≤ 100%. Proof: S(Fi) and S(Fj) are disjoint sets since an individual configuration cannot contain both features: S(Fi)∩S(Fj)=∅ Furthermore, the conjunction of the sets cannot be larger than Ω: S(Fi)∪S(Fi)⊆Ω then follows: C(S(Fi)+S(Fj))=C(S(Fi))+C(S(Fj))≤C(Ω ) => P(Fi)+P(Fj)≤100%.

204

Risk Based Testing for Software Product Line Engineering Exclusive OR group (also known as alternatives): In such a group the sum of the percentages of the child features, Fc1, Fc2,..Fcn, equals the percentage of the parent feature Fp: ∑P(Fi)=P(Fp). Proof: We have two sets of relations that govern this rule (1) Each of the alternative features exclude the others (2) Exactly one of the features is selected in a configuration. Following (1) S(Fi), for (all i,j =1..n) are disjoint sets: S(Fi)∩S(Fj)=∅ and following (2) The union of all the sets together exactly matches the set of the parent features: ⋃S(Fi)=S(Fp). We then get ∑C(S(Fj))=C(S(Fp)) => ∑P(Fj)=P(Fp). OR Groups with cardinality: In these OR groups the number of features selected is defined through a range with a lower- and upper-bound. For instance between 2 and 5 features must be selected from a group of e.g. 10 features. For instance 2 features are selected in one configuration and 4 features in another. There is a consistency rule for the lower-bound, Lb, and a consistency rule for the upper-bound, Ub. Lower-bound consistency rule: The sum of the percentages of the n child features, Fc1, Fc2, ..Fcn, is larger than or equal to the percentage of the parent feature Fp multiplied by the lower-bound: ∑P(Fci)≥P(Fp)*Lb. Upper-bound consistency rule: The sum of the percentages of the n child features, Fc1, Fc2, ..Fcn, is smaller than or equal to the percentage of the parent feature Fp multiplied by the upper-bound: ∑P(Fci)≤P(Fp)*Ub. Note: In an exclusive OR group the lower-bound and upper-bound are both equal to 1 and fits this formula. Also the rule for optional features fits since there is only 1 feature in the group, the lower-bound equals 0 and upper-bound equals 1. Proof: Note that now the sets S(Fci) are not necessarily disjoint since in a configuration more than 1 of the child features may be selected together. For our proof, first we make a few definitions: Let Vj be a configuration in which Fp, the parent of the group, is selected. Hence Vj∈S(Fp).We define T = C(S(Fp)), i.e. the total number of elements for which Fp is selected. Now we define S(Vj). This is the set of features Fci which are selected as part of the configuration Vj. Then C(S(Vj)) is the total number of elements of S(Vj). From the cardinality of the group it follows that for each of the configuration Vj ,∀ j∈(1..T) at least Lb features from the cardinality group are selected and at most Ub. Therefore: Lb≤C(S(Vj))≤Ub,∀ j∈(1..T) Now we take the sum of the counts of the groups S(Vj): ∑C(S(Vj)). Since Lb≤C(S(Vj)),∀j∈(1..T), It follows: T*Lb=C(Fp)*Lb≤ ∑C(S(Vj)). Since C(S(Vj))≤Ub,∀ j∈(1..T) It follows: ∑C(S(Vj))≤T*Ub = C(Fp)*Ub. Because P(Fp) = C(Fp)/ C(Ω) *100%, it only remains to be proven that: ∑C(S(Vj))=∑C(S(Fci)). Let’s take a closer look at the set S(Vj). As defined this is the set of features Fc which are selected as part of the configuration Vj. So e.g. S(V2) contains the elements e.g. (Fc1, Fc3, Fn). The set S(Fci) is the set of configurations in which Fci is selected. E.g. S(Fc3) contains the elements e.g. (V1, V2, V5). Now each element of S(Vj) corresponds to exactly one element of S(Fci) and vice versa. In our example the member Fc3 of the set S(V2), corresponds to member V2 of S(Fc3), by definition. Therefore there is a bidirectional relation between all the members of the

205

Chapter 9 combined group of sets S(Vj) ,∀ j∈(1..T) and that of the combined group of sets S(Fci) ,∀ i∈(1..n) and hence the number of elements of these two groups of sets is the same. Feature cardinality: A feature that has cardinality can have multiple instances and each can have a percentage associated with it. So there is a percentage of a having one instance in the set of configurations, another percentage for the second instance etc. Features with cardinality can also have child features that consequently have multiple instances as well and have a percentage associated with it.

206

Chapter 10.

Towards a More Systematic Analysis of Uncertain UserProduct Interactions

This chapter is published as: Y. Lu, P.H. den Ouden, A.C. Brombacher, W.H.J.M. Geudens, H. Hartmann: “Towards a more systematic analysis of uncertain user-product interactions in product development: an enhanced user-product interaction framework”. Quality and Reliability Engineering International, 2007, 23: 19-29, DOI 10.1002/qre.820. Abstract For highly innovative products the actual product use is often highly uncertain. However, the uncertainty/variability in product use and its impact on product reliability have been explored very limitedly. Mismatch between anticipated product use and actual product use can lead to considerable unexpected technical and non-technical reliability risks. However, traditional reliability management techniques focus primarily on managing technical reliability risks. This situation results in an increasing amount of no-fault-found (NFF) or fault-not-found (FNF) reports in high innovative consumer electronics (CE) industry. Under the time-to-market pressure, it is increasingly important to take into consideration the significant factors that determine product use in the early product development process. In Software Reliability Engineering (SRE) framework, a top-down approach (operational profile) is used to analyse the product use. However, operational profile cannot be directly applied to analyse product use for CE products due to the differences between reliability of software products and that of CE products. This paper proposes an enhanced framework to structurally analyze unexpected userproduct interaction for highly innovative CE products. In this framework, product, user as well as environmental conditions are used to analyze user-product interaction. This framework has been implemented to analyse user-product interaction by using data from a consumer test of a highly innovative CE product.

10.1 Introduction The importance of customer satisfaction on future business success is widely accepted. Understanding and anticipating what customers want of a certain product is crucial to achieving customer satisfaction. Product reliability has been recognised as an important measure to achieve customer satisfaction. Traditional reliability approaches consider product reliability as a predominantly technical issue. Most of the quality and reliability programs/methods such as Six Sigma, DFM, QFD, FMEA, robust design, etc. assume that all customer expectations and requirements could be

207

Chapter 10 correctly collected so that they can help translate the customer requirements into product specifications and help implement the specifications into reliable products [Schmidt1997, Booker 2003, Lewis 1996]. However, under increasing product innovation and increasing customer demands, it is often extremely difficult not only for the manufacturers to fully understand the customer requirements but also for the customers themselves to know exactly what their expectations are before really using them. Even worse, customers, who purchase the product, are not always the users. The users are actually those using the products. Both of customers and users are often disconnected from the manufacturers by third parties in modern high-volume CE industry [Petkova2003]. As a result, the product specifications may not be complete and correct in the first place for these products and the users can have very different requirements and many unexpected ways to use the products. In particular, Brombacher et. al [Bombacher2005] has shown that the number of ‘no fault found’ (NFF) or 'fault-not-found' (FNF) failures in modern high-volume CE industry has been rising to a level of nearly 50% in 2004. NFF refers to the situations where the product still performs according to the technical specifications but it is still rejected because it does not meet the customer expectations while FNF refers to the situations where a technical failure of the product is reported, but this failure cannot be reproduced by the manufacturer. To cover these types of failures, the concept of product reliability has been extended to include failures due to unexpected customers’ expectations and unexpected product use [Ouden 2005]. Instead of focusing on the technical perspective in product development from a manufacturer's point of view alone, it is also necessary to look into the users' perspective when determining the reliability of the product. Therefore it is very important for the manufacturer to understand the unexpected user-product interaction so as to prevent NFN and FNF in the field. It is also important to mention that not all unexpected user-product interactions will lead to customer complaints in practice, but this paper takes these interactions as sources for potential customer complaints. A reliability engineering methodology exists in the software reliability. Using operational profiles in software reliability engineering [Musa 1987] has been proven to be useful in incorporating actual product usage in reliability management for professional software products. It is important to know that the amount of software in CE products has increased significantly [Kramer 1999]. Although operational profile is specifically tailored for use in a software development environment, it has been argued to be useful for electronic hardware devices as well [Pant 1994]. Interesting questions arise:  To what extent, can operational profile be applied for CE products to model (unexpected) products use for highly innovative CE products?  What adjustment needs to be made? In order to answer these questions, Section 10.2 analyses theoretically the feasibility of applying operational profile in modelling and analysing unexpected product use for highly innovative CE products. Based on the findings in Sections 10.2, an enhanced framework is proposed in Section 10.3. The proposed framework is implemented to model user-product interaction using consumer test data of a highly innovative CE product in Section 10.4. Conclusions are given in Section 10.5.

208

Towards a More Systematic Analysis of Uncertain User-Product Interactions

10.2 Issues when Applying Operational Profiles for CE Products Musa [Musa 1993, Musa 1996] describes how operational profiles can be built. One of our co-authors [Hartmann 2005A, Hartmann 2006] estimated that software growth testing could be conducted more effectively when developing operational profiles for a specific professional product, see Chapter 8. He also observed that developing operational profiles significantly improved the requirement engineering process. Operational profile is originally developed for reliability analysis for software only. Before applying operational profiles to model and analyse unexpected user-product interactions for highly innovative CE products, it is necessary to understand the characteristics of software reliability and their impacts on constructing operational profiles. The differences between software reliability and reliability of highly innovative CE products need to be also taken into account. Firstly, software does not break or wear-out over time [Boyd 1995]. Software failures occur without warning and they occur the first time when the defective sequence is processed. Although, the software could be used for weeks or months before the failures occur, software failures are often considered as time-independent [Rees 1994]. From the way that the operational profiles are constructed, it can be also concluded that time in use is irrelevant in this process. On the contrary, failures of highly innovative CE products can occur due to 0-hour failures, early wear-out, random failures, as well as degradation and wear-out effect, i.e., they can be time-dependent and can relate to physics of failure [Brombacher 1992, Wong 1988]. Therefore, it is necessary to take into account the time in use, i.e. the use process when modelling and analysing unexpected user-product interactions for these products. Secondly, a fault tree analysis of software failures reveals that software failures are mainly due to software development defects [Boyd 1995]. Failures of highly innovative CE products are far more complex. Increasing product complexity and increasing customer demands has led to a situation where both hard failures and soft failures exist for CE products [Wong1998]. Soft failures are often due to unexpected user-product interaction [Geudens 2005]. The fault tree analysis mentioned above aims at identifying hard software failures only. Ouden et. al [Ouden 2005] suggests an extended reliability definition to include both hard failures and soft failures in reliability analysis to reach customer satisfaction for highly innovative CE products. When constructing user-product interaction in operational profile for software, certain functionality in the product are used to relate to certain users [Lyu1995] so that corresponding defects can be identified and removed. For example, password-limited functions and different user-accounts are among the measures that can be taken to explicitly give certain users limited access to the product. With highly innovative CE products this is practically impossible. A DVD recorder will be used by the entire family that consists of quite often different types of users and it is unrealistic for the development team to determine upfront that which levels of access should be given to different types of users. The diversity and differences in users is much higher there

209

Chapter 10 [Petkova2003]. In short, to model and analyse unexpected user-product interaction, the diversity in users and their different use patterns needs to be structurally taken into account. Last but not least, software operating-environments are extremely diverse and complex although a simple distribution of inputs is often used to describe the situation in operational profiles [Whittaker 2000]. There is an increasing need to model the complete operating environment for software products [Whittaker 2000]. A similar situation also occurs with highly innovative CE products as well. As the connectivity with other products becomes more and more important in the world of CE products [Kramer1993], it is necessary to take the use environment into consideration when analysing unexpected user-product interaction. In conclusion, to model and analyze unexpected user-product interaction, three elements need to be addressed better than they were in the operational profile: the (early) use process, the users and the environment.

10.3 Modelling Unexpected User-Product Interaction: an Initial Step In this section, an enhanced framework is proposed to model and analyse unexpected userproduct interaction in terms of the users, the use process and the environment as an initial step. How the users, the use process and the environment can be defined in this model is discussed in detail below.

10.3.1 Users Rogers [Rogers 1995] classifies customers into five groups based on how customers adopt a new product technology: Innovators, Early adopters, Early majority, Later majority, Laggards. However, this categorization, based on technology adoption, has furthermore received widespread criticism due to measurement problems of innovativeness and personal characteristic differences across different type of technologies. Rogers' adopter segment has been largely researched using personal characteristics. A person belonging to each category is said to have certain characteristics, which are only vaguely described, such as ‘likes to experiment’, ‘seeks challenges’ and ‘sociable’ for innovators. These traits can hardly be measured and are therefore not very fit for categorizing a group of people. Moreover, in certain cases these characteristics have been found to be incorrect. Dickerson and Gentry [Dickerson 1983] found that early adopters of home computer to be “logical introverts” as opposed to the sociable trait associated with innovators in Rogers’ classification. Furthermore, while attitudes towards adopting new products and technology may in fact be related to the five categories, it is doubtful whether there is a fixed classification of individual customer. For one product, the customer can only be positioned in one of the categories. For other product, he may belong to other categories. For example, while a customer may be an adopter when it comes to photography equipment, the same customer may be a laggard when it comes to ironing equipment. Last but not least, the customers may not be the users. An innovator can purchase the very first plasma multi-media TV. However, his family members, who can belong to other customer categories, may use the TV as well. In other words, this classification is based on technology adoption measurement via

210

Towards a More Systematic Analysis of Uncertain User-Product Interactions purchasing intension but not based on using experiences and user expectations. An alternative to Rogers’ classification has been proposed by Saaksjarvi [Saaksjarvi2003]. The new approach presented in this paper by Saaksjvari deals with segmenting user groups based on knowledge and compatibility. She uses compatibility [Rogers 1995, Dickerson 1983, Holak 1988] to highlight the likelihood of the purchasing intention of certain users: the higher the compatibility, the more willingness to buy the products. Furthermore Sjaaksjarvi uses knowledge to distinguish the use behaviour of different users during the use process. In this way, not only the buying intension but also the using experiences are used to distinguish the users. In order to understand unexpected user-product interaction, it is important for this research to gain more insight in users and the way they use the products. Therefore this paper considers the classification from Saaksjarvi more applicable to differentiate different types of users. In addition, since the focus here is more on users than buyers, we will use only the knowledge dimension from Saaksjarvi in this paper to classify users into four groups by assuming that the CE product is already purchased: Technovators (experts in both core and supplemental area), Supplemental experts, Novices (novice in both core and supplemental area), and Core experts.

10.3.2 Use process The user-product interaction is time dependent. A user uses the product differently at the different moment in use. Let us take a Media Centre PC as an example. A Media Centre PC is a so-called "all-in-one PC" and home entertainment centre. It is the same as any other Windows XP PC, but it can do more. One of the attractive functions is the time shift function. Using such a function enables the users enjoying any live TV programs without a break even when they need to do some other things during the programs. However they cannot immediately use such a function in a Media Centre PC until the PC is installed and television broadcasters are detected and installed. It is therefore important to include the use process when modelling and analyzing user-product interaction. Also, along the use process, different types of failures (0-hour, early wear-out, random failures and systematic wear-out) will be encountered. Clearly distinguishing between these types of failures with respect to product use is important as the relevance of the failure types varies with the type of product [Brombacher 2005]. The use process may be divided into five generic phases: unpacking and installation, configuration, first use, extended use and end of use. It should be noted that not every user goes through all these phases consecutively. The first few steps up to first use may have been done by the some users only while other different users may not use the product until the first use phase. Also the manufacturer might include installation and configuration in the total product. Brombacher et. al [Brombacher 2005] stresses that only early reliability problems that occur at the early product use process are most relevant for highly innovative CE products because of their shortened product economic lifecycles. In addition, based a field data of a highly innovative CE product, Boersma [Boersma 2001] showed that in the first weeks after purchase, the product rejection rate by customers is high, while after those initial weeks the rejection rate becomes lower. It implies that unexpected user-product interactions at the early use process are most relevant to study. The early use

211

Chapter 10 process then consists of the following phases: unpack and installation, configuration and first use. Extended use phase as well as end of use phase is considered irrelevant for highly innovative CE products because of their reduced economic lifecycle.

10.3.3 Environment As stated in any reliability definitions, environment plays a very important role in determining the product reliability. Environment naturally consists of generic elements of physics such as temperature, pressure, magnetic fields and humidity. In addition, due to increasing interaction between different consumer electronics products, the peripheral equipment environment in which the product is supposed to operate is also becoming more important. The SL300i Multimedia Link by Philips for example, is a device that can interact with a TV, stereo set, PC and the Internet. For the Internet connection, even more possibilities exist; the unit can be connected wirelessly or wired and the connection can be set up directly or through a gateway. Under this increasing environmental complexity and interaction, environment may have a significant impact on user-product interactions. Indicative environmental variables in this context may also include the number, age and brand of peripheral equipment. Typically, environmental reliability issues should surface during the installation and configuration phase as the first interactions between the product and its environment takes place.

10.3.4 The enhanced framework Based on the discussions in the previous sections, an enhanced model (Figure 56) for structurally modelling user-product interaction for highly innovative CE products is developesectiond. For any given products, (unexpected) user-product interactions can be modelled by different users, during the different moment in the use process and in the environment where the product is used.

Environment

Use process Installation

Configuration

First use

Users

Unpacking

Figure 56 The enhanced user-product interaction model

212

Towards a More Systematic Analysis of Uncertain User-Product Interactions

10.4 First implementation This section aims at demonstrating that how the enhanced framework models the (unexpected) user-product interaction using data from a highly innovative CE product in a field case study. The core question is: can the unexpected user-product interactions be modelled by users, the use process and the environment? The data used is explained first, followed by the analysis method, and the case results.

10.4.1 Data To model user-product interaction using the proposed framework, detailed user-use information is needed. The first logic information source should come from the service centre. However, Petkova [Petkova2003] showed that most product performance information collected at the service centres in CE industry is logistics oriented and it is hardly possible to derive any root-cause information. Through a field case study, Boersma et. al [Boersma 2005A] came to the similar conclusion that the origins and root causes of non-technical customer complaints are largely unknown based on field feedback data from the service centre or from the call centres. In other words, user-use information behind those field returns is largely unknown. Boersma et. al [Boersma 2005B] proposed a new process methodology to improve the quality of the field feedback information from the service centre and call centre in a multinational company actively operating in high-volume consumer electronics industry. However, to apply such methodology in real field case to collect more detailed information related to product use requires an organizational change as well process change. It is not practical for this project to first implement this method and then to collect the required data. Evans and Lindsay [Evans199] described other methods to gather user information with different sophistication levels are described. These methods and their capability in collecting user-product interaction information are summarised in Table 24. It can be observed in Table 24 that consumer test is the most potential method that can generate user-product interaction information. In particular, the High Contrast Consumer Test (HCCT) [Boersma 2003] developed jointly by Design Technology Institute and Technical University of Eindhoven can provide information on user-product interaction. By testing the product with carefully selected users, the HCCT-tests analyse user behaviour over a few use phases (unpacking and installation, configuration and part of first use) in the natural environment of the user himself (for example, in the living room at home). Several HCCTtests have already been performed and they provide results for a wide range of products and a wide range of users and environments in which the product is used. Therefore, HCCT data from a highly innovative CE product are used here to demonstrate user-product interaction.

213

Chapter 10 Table 24 Collecting information related to user-product interaction Methods Comment cards and formal surveys

Focus groups

Field intelligence

Direct consumer contacts Monitor the internet

Consumer tests

Remarks Concentrating on consumer satisfaction. Used to retrieve general impressions of product acceptation, no possible root cause of product problems are collected, i.e., there is lack of user-product interaction information. Careful selected panel answers questions on for example how their experiences with the product match their expectations. However, it is not representative for non-focus groups who may initiate unexpected user-product interaction. Obtaining field information directly from employees who are in direct contact with customers. However, customers may not cover all the users. For example top managers watching consumers use the product and gathering information on what the consumers like and dislike. However, who should be watched is not well defined. Monitoring the internet on product problems in for example forums can give valuable insights that might be translated into creative product improvements. However, the user's information behind those complaints is often unavailable. Basically combinations of the previous described methods. These systems in these studies were able to provide early feedback & give more and direct insight on user-product interaction.

The data set used in this analysis comes from HCCT performed by DTI with a multi-national company operating in the CE industry. The product tested is a prototype of new and to be launched digital CE product. A full factorial design of experiment approach is used to analyse data from the consumer test and to identify the effect of the three dimensions: users, use process and environment on user-product interactions. Four groups of users (4-level), Technovators, Supplemental experts, Core experts and Novice users, were selected based on their core knowledge and supplemental knowledge about this digital CE product. In total 16 participants were recruited for the test, which consists of 4 technovators, 4 supplemental experts, 4 core experts and 4 novices (4 replications). During the test, test participants were asked to install, configure and first use these new digital products (3-level use process). Home was chosen as the test environment in order to collect realistic user-product interaction information in a natural environment of the test participants. Environment was specified as the peripheral equipment at home (other CE products that were connected with this digital CE product during the test). Environment factor was set at 2-level: when the product is used alone (without connections with other products) and when there were connections made with other products. Unexpected userproduct interactions were identified by encouraging the test participants to think aloud

214

Towards a More Systematic Analysis of Uncertain User-Product Interactions during the test process. The test results consists of the following information: consumer complaints related unexpected user-product interactions, types of users, environment and also the moment when the interactions occur during the use process. The observations are summarised below and the experiment data is listed in Table 25.  Technovator-product interactions. All supplemental experts unpack and install the product successfully with/without the information provided by the user manual. However, when connecting complex peripheral equipment to this product they acted unexpectedly and two problems were reported. During the first use phase, all main functions were used, but no advanced functions were used and no complaints were made.  Supplemental expert-product interactions. All supplemental experts unpack and install the product successfully with/without the information provided by the user manual. However, when connecting complex peripheral equipment to this product they acted unexpectedly and two problems were reported. During the first use phase, all main functions were used, but no advanced functions were used and no complaints were made.  Core expert-product interactions. Two Core expects could not install the product successfully by referring to the user manual intensively. Two problems were reported during unpack & installation. Configuration was very limited performed and no complaints were reported. The first use was limited to checking two new features by three of the users and only one user tried other functions. Two of them explicitly mentioned that they would not use some advanced functions at all. No problems were reported during the first use phase.  Novice user-product interactions. Although almost all Novice users made use of the manual extensively, they all encountered difficulties during installation. There were lack of much knowledge in the area of installation and connecting this product with other CE products at home. In total, 16 complaints were reported during the installation phase. Configuration and connecting peripheral equipment was done with the help from friends, but not the participants themselves. Only very basic functions of the product were tried out during the first use. Table 25 Data collected from the consumer test. Exp. No.

User

Use process

Environment

1 2 3 4

1 2

Technovator Technovator

unpack & install unpack & install

0 0 0 0 0 0 0 0

3 4

Technovator Technovator

configuration configuration

5

Technovator

first use

used alone connected with others used alone connected with others used alone

0 0 0 0 0 0 0 0 0 0 0 0

215

Chapter 10 Exp. No.

User

Use process

Environment

1 2 3 4

6

Technovator

first use

0 0 0 0

7 8

Supplement Expert Supplement Expert


9 10



11 12


first use first use

13 14

Core expert Core expert


15 16



17 18


first use first use

19 20

Novice user Novice user


21 22



23 24


first use first use

connected with others used alone connected with others used alone connected with others used alone connected with others used alone connected with others used alone connected with others used alone connected with others used alone connected with others used alone connected with others used alone connected with others

0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 4 4 4 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

10.4.2 Analysis method In this study, the main interest is to analyse the effects of the three dimensions of the enhanced framework on the unexpected user-product interactions. Three hypotheses are formulated.  Hypothesis 1: differences in users do not cause unexpected user-product interactions  Hypothesis 2: differences in use process do not cause unexpected user-product interactions  Hypothesis 3: differences in environment do not cause unexpected user-product interactions

216

Towards a More Systematic Analysis of Uncertain User-Product Interactions The experimental design data presented in Figure 57 ANOM results of the test data. Figure 57 and Table 26 will be used to test these three hypotheses using Analysis of Mean (ANOM) and Analysis of Variance (ANOVA). Main Effects Plot (data means) for No of unexpected interactions Mean of No of unexpected interactions

User

Use process

0.60 0.45 0.30 0.15 0.00 1

2 3 Environment

4

1

2

3

0.60 0.45 0.30 0.15 0.00 1

2

Figure 57 ANOM results of the test data. Table 26 Results of ANOVA of the test data. Source

Sum of squares 6.8333

Mean Square 2.2778

F- statistic

P-value

User

Degrees of freedom 3

9.52

0.000

Use process

2

8.3333

4.1667

17.41

0.000

Environment

1

2.6667

2.6667

11.14

0.001

User*Use process

6

13.6667

2.778

9.52

0.000

User*Environment

3

8.3333

2.7778

11.61

0.000

Use process*Environment

2

5.3333

2.6667

11.14

0.000

Error

78

18.6667

0.2393

Total

95

63.8333

217

Chapter 10

10.4.3 Results and analysis The data analysis results using ANOM and ANOVA are listed in Figure 57 and Table 26. Statistically it can be concluded that  The occurrence of complaints related unexpected user-product interaction depended very much on the type of users. Especially, novice users have higher probability to use the product unexpectedly. For example, the way that the manual was used was very different. Technovator used no manual at all; Supplemental experts used manual for troubleshooting and encountered no problems; Core experts used manual for the same purpose but encountered some problems; Novices used manual intensively but encountered many problems. It is then interesting to mention that Moreau et. al [Moreau 2001] discovered that regarding high innovation, the adopter were likely to be users with extensive knowledge in supplemental product category or those in both core and supplemental product category.  The occurrence of complaints related unexpected user-product interaction depended very much on the use process. There is a higher probability to have more unexpected user-product interaction in the early phases (mostly occur in unpack & installation). All 20 unexpected product-user interactions occurred during the first two phases of the use process. This conclusion is also in line with the observations made by Boersma [Boersma 2001] where most of NFF/FNF was reported during the early use phases.  There is a higher probability to have more unexpected user-product interaction when the product is used alone;  It is not yet clear whether the environment factor influences the occurrence of complaints related unexpected user-product interaction. However, the case results did show that there are some interactions between the type of users, the use process and the associated environmental conditions. Connecting peripheral equipment (environment) had only impact on the user-product interaction of the Supplemental users. It had no impact on that of the Technovators and Core experts, neither on that of the Novices because this task was often left to their friends. On the other hand, the data also showed that product is more likely to be used unexpectedly when it is used alone during the early use process by the novice users These quantitative conclusions, however, cannot be fully accepted after reviewing the test results. Especially, the conclusion related to the effect of environment factor and its interactions with user and use process on the number of unexpected user-product interaction is not valid based on the statistically analysis results alone. During the tests, it was observed that the core experts hardly connect the product to other equipments and the novice users did not connected their products to other peripheral equipment alone. They were helped by their friends, who may belong to other groups of users. Moreover, the data collected during the tests were very spars due to the limited number of test participants and the predefined en simplified factor levels for the environment factor. In practice, environment factors can be further classified by the number of peripheral equipment, their age, brand, etc. Last not least, not all functions are used for different types of users. Only the Technovators have tried all functions. The reason behind this situation is not clear to us. In practice, it may be not always

218

Towards a More Systematic Analysis of Uncertain User-Product Interactions the case since people would not always act completely naturally during a test, even if they are in their natural home environment. This, in a way, also limited the number of unexpected user-product interactions. In short, the test results do show the significance impact of users and use process on the unexpected user-product interactions. However, it is not clear how environment factor relates to the occurrence of the unexpected user-product interactions. More structured test and test data and more objective test results are needed to further verify the enhanced framework. A consumer test lab has been built at DTI Eindhoven in order to create the opportunity to collect the required data and carry out the related research projects further.

10.5 Conclusion In order to ultimately prevent NFF and FNF problems in future product development, it is important to understand how users interact with the product (unexpectedly) when these problems occur. This paper proposed an enhanced user-product interaction framework to systematically model and analyse unexpected user-product interaction for highly innovative CE products. This framework aims at modelling user-product interaction through three aspects: the users, the use process and the environment. Further analysis of user-product interaction data from a field case study has concluded that user and use process did show significant influence on the occurrence of unexpected user-product interactions. However, due to the limited number of test data and constraints in a consumer test, it is still not yet clear how environment factor works in this framework. Further research, more structured data and test structure is needed.

Acknowledgements The authors wish to acknowledge that this paper is based on the MSc project of Mr. Michiel Mennen who graduated on the results of this project at Eindhoven University of Technology in December 2004.

219

220

Chapter 11.

Conclusions

This section contains the conclusions of this thesis and is structured as follows: First the answers to the research questions are given, which is followed by a discussion on the threats to validity. Then hypotheses are presented on the applicability of this research to other domains. This section ends with the key contributions of this thesis and a description of further research.

11.1 Research Questions and Answers This section describes the research questions and their answers. For each of three parts, first the overall research question is answered, which summarizes the main findings, and this is followed by detailed answers to each research question. Part I is the starting point of this thesis as it analyzed the transitions and the consequences. The overall research question for this part was as follows: RQ 1: What transitions have taken place in the development of software for consumer electronics products and what are the consequences for building software product lines? Answers: The software development for consumer electronics has transitioned in the following ways: 1. The software that is embedded in the device is in most cases no longer developed by a single firm, but is developed through a supply chain in which multiple parties develop specialized components for the downstream participants in the chain. The consequences for product line engineering are that components from different suppliers, with their variability, have to be integrated. This causes an increasing effort for system integration and testing; especially because no high degree of modularity has been obtained. 2. Ecosystems are used to support downloadable applications from 3 rd parties. In this way organizations transitioned from software product line engineering in an intraorganizational context to an inter-organizational context. Because a successful ecosystem relies on attracting sufficient 3rd party developers, choosing the most suitable ecosystem type and software platform can have major commercial and strategic consequences. The consequences depend heavily on the choice of the ecosystem type.

221

Chapter 11 The research questions that have been addressed in Chapter 2 were: RQ 1.1: What transitions between industry structures have taken place? Answer: The transition took place in a number of steps: Initially the functionality and the required variability were implemented through hardware components. The second step was that an increasing part of the functionality and variability was implemented through software, but still by vertically integrated firms, meaning that these firms developed the entire product, i.e. both the hardware and software. The third step is that the vertically integrated firms used ICs and software components from specialized suppliers but remained responsible for the overall architecture. The fourth step is that a supply chain approach is formed in which the product is developed by a group of companies and the responsibility of the architecture is shared by the participants. The fifth step is that for supporting 3rd party applications a stable API (Application Programming Interface) has been created and an application store to enable an ecosystem. RQ 1.2: What were the forces and actors involved? Answer: The research identified 3 types of forces: (A) Forces that are driving, or staying, in a more vertically integrated approach, (B) Forces that favor a more ecosystem-centric approach and (C) Forces, or consequences, that restrain the transition to a more ecosystem centric approach. The forces of type A are: (F1) Less efficient use of system resources and therefore higher costs because modularity and a layered architecture introduce inefficiency because no direct control of the hardware is possible. (F2) Reduced options for large innovation steps, because an eco-system centric approach and, to a lesser degree, a supply chain approach, relies on stable interfaces. Large innovation steps often require changes of the high level architecture, which is easier for a vertically integrated company. (F3) The overall quality is more difficult to guarantee because it relies on the quality of components from several suppliers. An individual firm can control this more easily. The forces of type B are: (F4) Lower development costs and shorter time-to-market, because the development effort can be shared between the participants and because using third party components takes less throughput time then developing these components inhouse. (F5) More variability because components from different suppliers can be used and the next party in the supply chain can develop the required functionality more easily. The forces of type C are: (F6) Increased time for system integration, especially when components using different component technologies, i.e. heterogeneous components, are used. (F7) Higher interaction costs with the component suppliers (F8) The fear that a dominant firm creates a monopoly and takes most of the revenues. Initially there was only one actor involved, namely a vertically integrated firm that developed and manufactured the ICs, the required software and created the device. After the transitions the following actors are involved: IC manufacturers, middleware vendors, vendors of specialized software components, vendors of operating systems, set makers and application developers.

222

Conclusions RQ 1.3: How were the architectural interfaces between the actors defined, and what were the consequences for modularity, variability and product integration? Answer: The architectural interfaces where initially defined by the vertically integrated firms, including the phase in which integrators used specialized suppliers but defined the overall architecture. After the transition, i.e. de-verticalization, to the supply chain industry structure, the interfaces are defined by the participants together. The research showed that a major challenges lies in providing the right balance between delivering the required variability within the available time-to-market, while retaining sufficient control over the systems architecture to obtain optimal resource utilization. This challenge originates from the nature of consumer electronics that are designed to perform specific tasks using ICs with a low cost price. The consequences of these transitions are that the effort for product integration has substantially increased because no high degree of modularity was obtained. This relates to the effort to match the interfaces between the supplied and in-house developed components, the effort to remove faults in the software of the supplied components and the effort to remove faults that are caused by unexpected interactions between with the inhouse developed components and the supplied components. For managing the variability, the transition means that components, including their variability from several suppliers and in-house developed components should be integrated, possibly using different component technologies and non-matching interfaces. The research questions that have been addressed in Chapter 3 were: RQ 1.4: What types of ecosystems are used in consumer electronics products to support 3rd party applications? Answer: Three types of ecosystems are identified that are currently used in the industry to support 3rd party applications. These types are (1) the Vertically integrated proprietary hardware/software platform that consists of the hardware, proprietary closed source software and includes the device. (2) the Proprietary, closed source software platform that consists of a proprietary closed source software platform but does not include the hardware nor the device and (3) the Open source software platform that consists of an open source software platform and also does not include the hardware and the device. This classification is based on two properties: (A) whether the software is proprietary or open source, and (B) whether the hardware and device are included. Therefore these ecosystems differ in the scope of the platform and the extent to which complementors can modify the software and contribute to the platform. RQ 1.5: Which of the ecosystems types is most suitable for a specific product category from a software engineering perspective? Answer: The most suitable type of ecosystem depends on a number of factors that have been identified: Development costs, quality, variability, speed of innovation, system

223

Chapter 11 resources, hard real time requirements, stability of the interface and effort for system integration. Note that these factors are very similar to the forces that cause a transition as described in chapter 2. The most suitable type differs per product type and is based on comparing the strength and challenges of each ecosystem type with the market requirements of a product type:  When the speed of innovation is high and a low degree is variability is required, the vertically integrated hardware/software type is most suitable. This has been analyzed to be the case for gaming consoles.  When a high degree of variability is required and it is needed to share the development investment of the software platform between multiple firms, the open source software type is most suitable. This has been analyzed to be the case for digital photo cameras, digital televisions, smart phones and smart watches  When stability of the interface is a key requirement, while development costs and quality are also important factors, then the proprietary platform type is most suitable. This has been analyzed to be the case for personal computers. For tablets, all three ecosystem types are considered equally suitable since these products have characteristics of both smart phone as well as personal computers. The overall research question for part II is derived from the findings in part I as it identified that supply chains are the dominant structure for software development for consumer electronics and that the time for system integration has increased substantially. Therefore the overall research question for this part was: RQ 2: What are the consequences for variability management, when applied in software supply chains, and what modelling approaches can support this? Answers:  Variability models that are used in software supply chains should be able to support multiple product lines, staged configuration and should support combining the variability from different suppliers. This can be modelled by capturing the variability of the context of a product in a separate tree of the feature model.  The description of the variability of the components from several suppliers, e.g. through variability models, have to be combined. This can be modelled by using a separate feature model that facilitates the configurations process and separate feature models of the suppliers. In this way the links between the feature models of the supplier’s components and their development artifacts are retained.  Components from several suppliers and in-house developed components have to be integrated which often have non-matching interfaces and are possibly implemented using different component technologies. In current practice the manual creation of glue components requires a large development effort and frequently leads to faults. An approach using model driven architectures can generate the glue components in an efficient manner while it supports staged configuration. This approach also avoids that a large group of customers and developers need to learn the complexities of model driven architectures.

224

Conclusions 

Variability models in software supply chains often lead to large models with complex and many dependency relations. A textual variability language can support this more easily than a graphical variability language because when using textual languages, these models can be created and edited more efficiently.

The research question that has been addressed in Chapter 4 was: RQ 2.1: How can feature models be constructed to model multiple product lines in software supply chains? Answer: Feature models can be used by adding a separate tree that describes the context in which a product is used. This context can be a geographical region, a product type or a customer. This method allows for staged configuration in which certain configuration choices are made by an upstream participant in a supply chain, resulting in a partly configured feature model, while the next participant in the chain may perform additional configuration choices. Furthermore this method allows the merging of feature models by the next participant in the supply chain. This can be done in a stepwise manner by first merging the feature trees, then the context variability trees and finally by adding the constraints that arise from the combination of the features and context variability. The method can be used with commercially available variability management tools and can be applied by the industry without further research needed. The research question that has been addressed in Chapter 5 was: RQ 2.2: How should feature models be constructed which capture the variability from the components from alternative suppliers, partly offering the same feature set? Answer: When components are used from different suppliers with their variability, this leads to an overall feature model that combines the variability from the different suppliers in addition to the in-house developed variability. When the suppliers offer partly the same feature set this leads to overlapping features, i.e. features that represent alternative implementations. When this is modelled by using alternative subtrees in the overall feature model, this would lead to a time consuming configuration process to select the desired feature set. An alternative is to merge the feature sets and use a context variability tree to make a selection based on a supplier. A blocking issue here is that feature models do not stand on their own, but are linked to the development artifacts to instantiate a product variant, and therefore merging feature models will not retain the links. To address the limitations above, a so called Composite Supplier Feature Model can be used. This Composite Supplier Feature Model consist of a Supplier Independent Feature Model which is linked through dependency relation to the separate Supplier Specific Feature Models. This modelling technique allows for an easy configuration process, while the links between the feature models and the development artifacts are maintained. This method, similarly as the method described in chapter 4, can be used with commercially

225

Chapter 11 available variability management tools and can be applied by the industry without further research needed. The research questions that have been addressed in Chapter 6 were: RQ 2.3: What are the challenges that arise from combining components with nonmatching interfaces and what are the limitations of current practice? Answer: The challenge is to create glue components that are able to bridge the difference between the components of different suppliers and in-house developed components. These differences may be caused by the use of different component technologies, which may include different mechanisms for interfacing, binding and implementation of variation points, and by syntactic differences, such as different names of interfaces, methods and parameters. The challenges that arise from the use of software supply chains are: (1) To protect intellectual property rights it may be necessary to avoid that the next parties in the chain receives source code. (2) For commercial reasons, variation points that are offered to one customer may not be revealed to other customers (3) The various build environments that are used by the suppliers must be supported. (4) The capabilities should be preserved of component technologies that support reachability analysis to exclude unneeded code. The latter is especially important for resource constraint devices, such as consumer electronics. In current practice three approaches exists for the creation of glue components:  The first approach is that all possible glue components are created during domain engineering and configured during application engineering. The downside is that this leads to a large development effort during domain engineering and because of the large number of glue components, also a large number of links between the variation points and the glue components are required.  The second approach is to create the required glue components during application engineering. The downside of this approach is that this will lead to a long throughput time during application engineering.  The third approach is to use a common standard interface and component technology. The downside is that each pair of components requires two glue components and that these glue components would be very complex since they have to use a common component technology that encompasses all the component technologies that are considered. The creation of such glue components could lead to faults easily. RQ 2.4: How can MDA be used to bridge mismatches between heterogeneous components in software supply chains? Answer: The model driven approach (MDA) introduced in this thesis is based on using a supplier independent component model to describe the reference architecture. This reference architecture contains conceptual components, which are components that

226

Conclusions contain no implementation and are independent from the component technologies that are used by the several suppliers. Furthermore, it describes the composition between these components in terms of the dependencies between the components as untyped ports, because the actual interfaces are determined by the concrete components of the suppliers. When suppliers are selected, a model-to-model transformation creates a new model, in which the supplier independent components are substituted by supplier-specific components. This is called the supplier specific component model. At this point, the components of the supplier specific model are tagged with the component technology that is used to implement them. A second model-to-model transformation generates skeleton glue components. This is done automatically, and only when needed, based on one of the following criteria: (1) when a pair of components, that have connected ports, are tagged with different component technologies (2) when a required interface does not match the provided interface, e.g. when different method names are used or different order and type of their parameters. Following this step, a combination of model-to-code transformations and reusable code snippets are used to create the required glue code to complete the skeleton glue components. After this step, the build scripts are created. Then the variation points can be configured and the code is created. Following this step, the code can be transferred to the customer, where it remains possible that some of the variation points are configured by the customer or the next participant in the supply chain. The advantage of this method is that the architecture can be defined independent from the selected suppliers and the selection of the suppliers and the creation of glue can be postponed until the desired decision have to be made, i.e. during application engineering. RQ 2.5: What are the consequences for the development process and the development roles? Answer: The process consists of the following main steps: During domain engineering the feature model is created by the requirements manager and the reference architecture by the domain architect. The feature model contains both the functional variation points as well as the so called supplier variation points, which represent the different suppliers. This feature model can be constructed as described in chapter 5. During application engineering first the supplier is selected by the COTS engineer. Then the glue components are created and configured based on model to text transformations. In this step the COTS engineer is supported by a set of wizards that guides the engineer in specifying the mapping between the provided and required interfaces of the components that need gluing, i.e. for which a skeleton glue component has been created. Here the engineer can select the required snippets from a snippet library or by entering the code manually, which is needed when a new component technology is used. Then the customer support engineer liaises with the customer to determine what configuration is required and generate the build scripts. The partly configured product is delivered to the customer who can make additional configuration choices using a variability management tool.

227

Chapter 11 For our approach a number of different roles are identified and the specific skills that they require:  There is the so called meta-team, which consists of experts in model driven architectures that create the UML meta-model, the transformations and maintain the templates for the transformations.  The domain architect requires working knowledge of MDA to create the reference architecture.  The requirements manager requires working knowledge of feature modelling to create the feature model.  The COTS engineer uses the MDA tool with the wizards and snippets library to create the glue components. For these tasks no knowledge of MDA is required, but knowledge of different component technologies is required.  The customer support engineer uses the MDA tool to generate the components for the customer and uses a variability management tool to make configuration choices. Also the customer support engineer does not require knowledge of MDA.  The customer can make the final configuration choices using a variability management tool. The research questions that have been addressed in Chapter 7 were: RQ 2.6: What are the benefits of TVL for modeling product line variability, as perceived by model designers? Answer: TVL has the following advantages: (1) It is easier to interpret large models in comparison with graphical languages. (2) The learning curve is moderate, especially for developers who are familiar with textual languages such as C. C++ or Java. (3) Editing and creating a model is easier as it involves copying and pasting of plain ASCII text, whereas with a graphical language often many mouse clicks, drop-down menus etc. have to be used. (4) Constraints between features can be captured more easily than with a graphical language and the feature model remains easy to interpret with a large number of constraints. (5) Feature attributes are an explicit part of the model instead of an add-on. As a result, constraints involving attributes can be described easier. RQ 2.7: What are the product line variability modeling requirements that are not fulfilled by TVL? Answer: The following capabilities were missing in TVL: (1) Attribute cardinalities. (2) Feature cloning. (3) The possibility to capture the default value, meaning that during configuration many choices have to be made, although many may be standard choices. (4) The possibility to capture error and guidance messages that can be used during product configuration. The overall research question for part II is derived from the findings in part I and part II as it is identified that there is an increasing amount of faults in the software and an increasing

228

Conclusions pressure on time-to-market which results that there is less time available for testing. Therefore the overall research question for this part was: RQ 3: How can test efficiency be improved when the risks and user product interaction are taken into account? Answers:  The test efficiency can be improved when operational profiles are used to guide the test effort. When a very high reliability is required, meaning that the aim is to remove all faults, it is more efficient to focus more test effort to those development artifacts that contain the most faults.  Risk based testing can be improved by allocating more test cases to operations that have a high frequency of use, which can be identified by using three dimension to capture the risks, instead of two. This is an improvement of current practice in which relatively too much test time is given to operations that have a higher probability of causing a failure.  Risk based testing can be applied in software product line engineering by capturing the percentage of use of the development artifact in the set of variants. Based on the percentage of use and the criticality of each development artefact, a selection is made of which development artefacts should be tested during domain engineering and for which testing should be deferred to application engineering.  For reducing non-technical failures, caused by an uncertain user-product interaction, the tests should include the differences in types of users, their use-process and the environment in which a product is used. The research questions that have been addressed in Chapter 8 were: RQ 3.1: What are the benefits for the test efficiency when using operational profiles? Answer: The reliability of a product is a function of the number of faults in the development artifacts that implement the operations and the chance that these faults lead to a failure. When operational profiles are used, most test cases are allocated to the operations that are used most frequently. As a consequence most faults will be found in the development artifacts that are used most frequently and therefore the reliability increases most of those operations that are implemented by these development artifacts. Since the overall reliability is a weighted sum of the reliability of the individual operations, the reliability will increase more when the most used operations are tested more heavily. However, this effect disappears when an artefacts that implements an operation does not contain faults anymore. Using more test cases for these operations will not increase the total reliability anymore. In that case, which means that the system contains hardly any remaining faults and therefore has an extremely high reliability, it is better to allocate more test time to those operations that still cause failures. Therefore the conclusion is, which is supported by a statistical analysis, that operational profiles increase the test efficiency, however not when a very high reliability is required.

229

Chapter 11

RQ 3.2: Which dimensions in risk based testing require most attention to reduce the overall risks? Answer: In the current definition and practice of risk based testing, a risk matrix is used which consists of two dimensions: The probability dimension and the impact dimension. The probability dimension is based on the technological complexity; whether the software is develop by inexperienced developers and whether it contained many faults in the past. These aspects are also used in defect (fault) density predictions and therefore the probability dimension indicates how many faults are expected in a certain area. The impact dimension is based on a number of aspects, such as the visibility of the operation to the end-users, the impact on the business when it fails and the frequency of use of an operation. When the risks are captured in a formula, this will be a product of three aspects, namely: The chance that an operation fails multiplied by the change that an operation is executed multiplied by the damage that a failure might cause. In other words, the impact dimension should be separated in two dimensions, namely the change that an operation is executed, which is determined by the frequency of use, i.e. the operation profile, and the damage that a failure causes. It can therefore be concluded, which is supported by a statistical analysis, that most attention should be paid to the impact dimension by making the frequency of use a separate dimension, which will lead to a risk matrix consisting of three dimensions. The research questions that have been addressed in Chapter 9 were: RQ 3.3: How can risk based testing by applied in software product line engineering? Answer: Risk based testing can be applied in software product line engineering by identifying which development artefacts should be tested during domain engineering and which during application engineering. The identification of these development artefacts is based on the percentage of use of the features that these development artefacts implement and the criticality of the development artefacts. RQ 3.4: What are the consequences for testing during domain and application engineering? Answer: The consequences for testing during domain engineering depend on the testing strategy that is chosen:  For the division of responsibilities strategy, the components and variation points that are used in many products should be tested during domain engineering, especially those that have a high probability and implement features with high impact. The components that are used in a small percentage of products and have little probability and impact should not be tested during domain engineering.  For the test assets for reuse strategy, only for those components and variation points that are used in many products should reusable test assets be created.

230

Conclusions 

For the sample application strategy, the components and variation points that are used in many products should be included in the sample application. For testing during application engineering this approach has little consequences. It should only be identified which components are already tested during domain engineering and to what degree. For instance, those components that have already been tested intensively during domain engineering should not be tested heavily anymore during application engineering. RQ 3.5: What tool support is required? Answer: To capture the risks and identify the most critical components, the novel concept of quantified feature models has been introduced, which are feature models in which each feature is associated with a percentage of use in the set of variants, and the impact when a feature fails. Furthermore it should be able to model the dependencies between a feature model and the development artefacts, where the percentage of a development artefact, such as a component, can be derived from the features it implements. A complexity in quantified feature models is that the percentage of a feature usage is related to the percentage of usage of another feature. For instance when one feature requires another feature, than the feature that is required should have a larger percentage than the feature that requires the other feature. Such relations may lead to an inconsistent quantified feature model, for which there are no solutions know in literature that covers the wide range of possible dependency relations and therefore requires further research. Further research is also needed to identify whether such consistency checking is really required, since risk based testing is based on estimations rather than rigorous statistics and therefore a precise consistency checking may not provide additional value to practitioners. The research questions that have been addressed in Chapter 10 were: RQ 3.6: To what extent can an operational profile be applied to consumer electronics products to model (unexpected) product use for highly innovative consumer electronics products? Answer: The operational profile has two purposes: First, it guides the software test effort and secondly it is used to determine the failure intensity of a product that is caused by technical software failures, i.e. failures where the behavior of the system deviates from the specified behavior. For highly innovative consumer electronics products, the nontechnical failures are increasingly becoming the major source of reliability problems. Non-technical failures are caused a by a mismatch between the requirements of the product and the customer’s expectation and because the product is used in different ways than anticipated by the manufacturer. These non-technical failures are not addressed when tests are executed according to the operational profiles. Therefore an operational profile can be used to guide the test effort for software testing, but cannot be used to determine the overall reliability of a highly innovative consumer electronics product and to identify non-technical failures.

231

Chapter 11 RQ 3.7: What adjustments need to be made? Answer: To identify the main causes of the increasing reliability problems, the experiments have shown that the following factors have a significant impact on nontechnical failures and should be included when the reliability of a product is determined:  Users: Different users may have different technical knowledge and experience with using innovative products.  Use process: A product is used in a different manner at different moments of time. Typically the following phases can be recognized: Unpacking, installation, configuration, first use, extended use, end of use. It has been identified that in the early phases most reliability problems are encountered. A third factor was analyzed, namely the Environment: A consumer electronics products is often connected to other products, for instance a multimedia player is connected to a television, the internet and an amplifier. Often failures occur when a device cannot be connected properly to another device, for instance because the connections are not available. However the experiments did not show a clear effect of the influence of the environment on the product’s reliability.

11.2 Threats to Validity In this section the threats to validity of the results are discussed using the validity threat types that are proposed in [Feldt 2010]. Since each chapter has its own research results, the threats are discussed for each chapter separately. A threat to external validity of chapter 2 is whether the conclusions, based on two case studies, are generally applicable to consumer electronics industry. These threats were mitigated by using two different case studies and two different data collection methods. A specific threat is that the results may not be applicable to consumer electronics that have a much smaller size of software or for products for which there is less pressure on cost price and time-to-market. For this chapter, the author sees few treats to internal validity: For the case study of digital televisions multiple sources of data were used and the authors had detailed insight as they were involved in this product for many years. For the case study of mobile phones, also the authors had experience with the software development for these products types and many studies are available in related art. The case studies in chapter 3 cover a wide range of consumer electronics products which gives a high confidence to external validity. A threat to internal validity is that the importance of the factors for each product type are determined by the authors, and not based on a detailed analysis of the market. A threat to external validity is that the strengths and challenges of each ecosystem type are based on qualitative data, mainly from the domain of mobile devices. More quantitative data, preferably from more product types would give stronger evidence. A threat to construct validity is that the factors that determine the most suitable type of ecosystem are based on the strengths and weaknesses form a software engineering perspective and do not include commercial and financial aspects. Therefore the conclusion that the ecosystem type that is used in practice is, in most cases, the same as that which follows from the analysis, could be caused by these factors as well.

232

Conclusions For chapter 4 and 5, the author sees few threats to both internal and external validity, since the proposed techniques of creating feature models have been exercised on concrete cases and demonstrated with a number of scenarios. An internal threat to validity is that merging of feature models may reveal unexpected problems, because this could not be exercised with the available tools. For chapter 6 the author sees few threats to internal validity, since the method to use MDA has been implemented and exercised using a concrete case study. A threat to external validity is whether the proposed method is applicable to other component technologies as well, since the case study was only exercised with two component technologies. A second threat to external validity is whether the proposed method does indeed provide the efficiency gains as discussed, since a quantitative comparison was not made between the proposed new approach and current practices. For the evaluation of TVL in chapter 7, an internal threat to validity is that the models were created by the researches and not by the participants, which may influence the evaluation of ease of use and costs of model creation. Another threat to internal validity is that the models that have been used were relatively small in size. In general, the wide range of case studies and difference in backgrounds of the participants, give a high confidence in the validity of the results. An internal threat to validity of the statistical analysis in chapter 8 is that the conclusions are based on data from a single case study and the experiments were not based on real life data. Therefore the conclusions might not be generalizable. The mitigation strategy was to exercise the statistical analysis on a wide range of variables to verify the conclusions. Furthermore, the main conclusions are supported by a reasoning that explains why the statistical experiments shows the results as they are. Another external threat to validity is that the claim that the model is easy to understand, is only supported by a single case study. A threat to external validity on the use of risk based testing in chapter 9 is that it has been evaluated using a single case study. A specific concern is that the organization, in which this case study has been applied, uses a strict separation between domain and application engineering, which may not be the case in other organizations. A threat to external validity of the evaluation of the user-product interaction framework is that the tests have been executed on a single product and therefore may not be generalizable for other consumer electronics products. A threat to construct validity is that other factors, besides users, use-process and environment, may influence the results.

11.3 Generalization Most of the research in this thesis was focused on consumer electronics, where some results are more widely applicable, as shown in Section 1.7. In this section it is discussed whether the results, classified in that overview as applicable to consumer electronics or embedded systems, are also more widely applicable. Consumer electronics are characterized by a fast increasing amount of software, a high degree of innovation, the lack of standardization and the need to optimally make use of system resources. These are the main reasons why the transitions have occurred as described

233

Chapter 11 in chapter 2 and 3. Many of these characteristics can be found in other domains as well, especially for embedded devices. It may therefore be expected that the research results and the identified challenges are also applicable to the wider domain of embedded systems. It has already been identified that most of the research results of part II are applicable to embedded system, which is in line with this observation. The automotive domain is an example in which the functionality, previously implemented by mechanical and electrical components, is being replaced by software. Furthermore new functionality, such as driving assistance and ultimately, self-driving cars, are largely implemented through software. Another example is the medical domain. In this domain increasingly healthcare products are used outside the hospital and are being used in our private homes, which results in cross-over products between consumer electronics and healthcare products. A difference however between these domains and the consumer electronics domain is that failures may cause health and safety risks and therefore the reliability requirements are much higher. Consequently more time has to be spent on ensuring the quality which includes rigorous testing. The methods that were developed for feature modelling in supply chains, in chapter 4 and 5, were dedicated to be used in embedded systems, because in embedded systems only those features that are required by the market are usually embedded in a product. However these approaches might be also applicable to software product line engineering for software supply chains in general as more and more users get overloaded with features and therefore reducing the feature set may become an explicit goal. Furthermore, this method may also be applicable outside the area of software engineering and possible also for electrical engineering, hardware development and mechanical engineering, since feature modelling itself may also be applicable to these domains. An example was already shown in chapter 7, in which the use of a textual variability language was evaluated using an example from hardware development. A second example is mechanical development in the automotive domain, in which there are also a large number of variation points that have to be managed during the design and manufacturing process. The method described in chapter 5 for combining overlapping feature models showed the limitations of merging feature models because the links to the development artifacts would be severed and therefore a merged feature models cannot be used to configure a variant. In the method described in this chapter a separate feature model is constructed that allows for an easy configuration with links to the original feature models. This method may therefore also be applicable to merging feature models in general. The approach in chapter 6 to generate glue components was specifically aimed at embedded systems, however part of this method may also be used in other domains where glue components are required, especially where different component technologies are used. A specific goal of the research in chapter 6 also was to develop methods that would not expose unfamiliar technology to a wide group of developers. This analysis method may very well be applicable to other technologies as well, since getting familiar with new languages and technologies may take a lot of time and resources, which does not directly benefit the profitability of a company. In the evaluation of the MDA, we identified the roles and their tasks. These roles and their tasks may very well be applicable to the deployment of MDA in general.

234

Conclusions Quantified feature models, as introduced in chapter 9, could potentially also be used during commonality/variability analysis to reduce the amount of variation points. Since software product line can easily incorporate thousands of features and variation points, the effort for developing, testing and maintaining a product line becomes problematic in terms of investments [Savolainen 2009]. By identifying how much each feature is actually going to be used, a better informed decision can be taken. For instance features that are never used, or hardly ever, can be removed, thereby reducing maintenance costs. Features that are used in a large majority of the products may become a mandatory feature which makes the configuration process easier and the code can be simplified. The enhanced framework for user product interaction for consumer electronics product, as introduced in chapter 10, may also be applicable to other domains where non-technical failures are increasing. In many products that contain a large amount of software, the installation and use of a product is becoming more and more complicated and therefore also for these domains the users, use-process and environment are likely to have an effect on the reliability.

11.4 Key Contributions of this Thesis This thesis showed that the increasing amount of variability and speed of innovation, together with a high pressure on development costs and product quality has a significant impact on many aspects on software product line engineering for consumer electronics. To start with, the impact on industry structures was analyzed, which in his turn had impact on variability management and the transitions created additional pressure on the available time for software testing. For each of these areas this thesis has contributed through a better understanding of the challenges and provided solutions for some of these challenges. The specific contributions of this thesis for industry structures of software development for consumer electronics are:  A model is introduced which captures the evolution of software development for consumer electronics and the forces that govern this transition. Detailed case studies are described that can serve as lessons learned for other domains.  The transition on the consumer electronics industry was compared with that on the computer industry in which the de-verticalization was enabled by a high degree of modularization. This thesis showed that the de-verticalization of the consumer electronics industry started when functionality outside the traditional domain was introduced, despite a low degree of modularity.  A first step towards decision support method is introduced that determines which type of ecosystem is most suitable to support 3rd party applications for consumer electronics product from a software engineering perspective.  The architectural challenges are identified that arise when a company considers a different industry structure and when it decides to adopt an ecosystem type to enable the use of 3rd party applications.

235

Chapter 11 The specific contributions of this thesis for managing the variability in a software supply chain are:  The concept of context variability was introduced which provides a solution for modeling multiple product lines in software supply chains and this method supports staged configuration. Because our approach is intuitive and based on conventional feature modeling this approach can be easily applied in the industry.  It was shown that merging feature models is not a most suitable approach to combine variability since it would sever the link to the development artefacts.  A method was introduced to model the variability of alternative components with overlapping features. An advantage of this approach is that the original feature models have been kept “intact”, meaning that no changes have to be made to the feature models of the supplier’s components. Furthermore, this approach can be used with existing variability management tools and the case study showed that this approach is straightforward to use.  A model-driven approach is introduced for automating the integration of heterogeneous components, covering syntactic mismatches and mismatches related to different component technologies. In this approach glue components are generated efficiently only when they are required, staged configuration is supported, and the additional skills required to deploy MDA are localized in the organization which ensures that only a limited group of developers are exposed to unfamiliar technology.  A textual variability language, TVL, was evaluated through an empirical evaluation on four product lines and developers from the industry. This showed that using TVL can lead to an efficiency gain, especially when using large feature models with complex constraints and feature attributes. The evaluation also revealed that some capabilities were missing, such as attribute cardinality, default values and guidance messages for product configuration. The specific contributions of this thesis for software testing are:  A statistical model is introduced that captures the relation between faults and failures. Using the model the efficiency improvements of using operational profiles are determined and an improved method for risk based testing is presented. The model is easy to understand which gives it great explanatory power in illustrating the influence of different amounts of test cases on the product’s reliability.  An approach is introduced to apply risk based testing in software product line engineering. This approach can be applied with different test strategies and addresses the disadvantages of these strategies.  Quantified feature modelling is introduced, which can support risk based testing for software product line engineering and which may, given further research, also lead to an improved commonality/variability analysis.  The limitations are identified of using operational profiles for highly innovative consumer electronics products. An enhanced framework is developed to structurally analyze unexpected user–product interactions for highly innovative consumer electronics products. This framework can be used during the test process to reduce both technical and non-technical failures.

236

Conclusions The results in this thesis can be used by the industry as part of their strategy, development and test process as follows:  The models of industry structures and ecosystem types can serve the decision making process for individual firms for selecting the most appropriate industry structure for their business and ecosystem type when a transition is considered. Furthermore these firms can proactively address the challenges that are identified.  The methods that are described for variability management can be used to improve the practice of software product line engineering when applied in a software supply chain. These methods are widely applicable to the domain of embedded software and, for some of the described methods, to software product line engineering in general.  The methods introduced in this thesis to improve the test efficiency and to identify non-technical failures can be used by the industry to improve the reliability of their products while preventing a longer time-to-market.

11.5 Further Research In Section 11.3 the generalization of the research was presented which mostly result in further research. In this section further research is discussed which is not covered in that section. This section covers further research that are directly related to the individual research results as well as identifies topics for further research for software product line engineering for consumer electronics in general. Evolving industry structures The model of industry structures shown in chapter 2, captured the system as a whole and captured the role that modularity plays in the transitions. When systems become larger, it may be expected that for some parts of the system a high degree of modularity can be obtained, e.g. between the middleware layer and the application layer, while for other parts of the system, e.g. between the middleware and firmware layer, this might not be obtained. This leads to a more fine-grained, possibly hierarchical model. Further research in this area is needed to identify in which parts of the system, interfaces can or cannot be kept stable and what the consequences are for other parts of the system. A principal area of further research is what the consequences are for the architectures and development process when an existing application framework is integrated that is used for a stable interface for 3rd party applications. As shown in chapter 2, integrating existing application frameworks, sometimes referred to as middleware, is already common practice. However in that situation the applications were integrated into the device and the combination could be tested prior to delivery to the customers. In the situation when an ecosystem centric approach is adopted, it is not a priori known which applications are downloaded by the end-users and what the consequences for system utilization are, which makes it more difficult to guarantee the correct functioning and performance of the product. Also more research is needed to identify the benefits of a mixed model of ecosystems for consumer electronics in which part of the software platform is open source, while another

237

Chapter 11 part is closed proprietary. Such an approach might provide the benefits of both these ecosystem type, e.g. by offering a stable interface towards the applications, based on proprietary standards, while an open source approach with open standards are used to support different hardware configurations. Deployment of MDA in software supply chains One of the areas of further research to create glue components, as described in chapter 6, is to support the capabilities of some component technologies that support reachable analysis to avoid the use of unneeded code. Also further research is needed to generate glue components that bridge larger semantic differences, which requires a more general model of glue code. The research on MDA showed that different participants in the supply chain use different incompatible component technologies. To transfer the software to the next participant in the supply chain, the proposed solution in this thesis is to transfer standard C code and build environments. Furthermore, also feature models are delivered to the customer, which assumes that these models can also be used by the customer. As another example consider MDA to create code, i.e. not only glue code. To optimally use this technology, both the supplier and customer have use it, so that models can be transferred, rather than code. The receiving party could then integrate and test the models and generate the code once the final configuration choices are made. These examples reveal a general problem in software supply chains, namely that different participants use different technologies and different tools, which may be incompatible. The consequence is that the introduction of new technology is restrained because it can only be optimally used when the other participant in the chain have also introduced this new technology. Further research is needed how technologies can be standardized across a supply chain, or how to deal with incompatibilities in technologies and tools. Ensuring sufficient quality in a supply chains and ecosystem centric approach Currently, multiple ecosystems are used for similar products, e.g. mobile phones or digital televisions. For application developers this means that they have to develop and test their applications on a number of different platforms. This is particularly challenging when open source software platforms are used as this often this leads to fragmentation. The results of this thesis may support the application developers in three ways: (1) Context variability, as presented in this thesis in chapter 4, can be used to capture what the variability in the platforms is for which the application is going to be developed. (2) A quantification of the context that captures the market share, similar as presented in chapter 9, can be used to determine which platforms should be targeted first (3) This quantification can be used to set priorities for testing. Another problem that requires more research is that components or applications from 3rd parties may use different user interface styles. A concrete example from a decade ago was a product that combined a DVD-player with a video recorder. This product contained two styles of user interfaces. Because the product was created from components from different

238

Conclusions suppliers and the user interface was tightly coupled with the middleware, it was not possible within a reasonable period of time, to create a product with a consistent user interface. A similar problem occurs when applications are used from different suppliers. It frequently happens in mobile devices that one application uses a physical “settings” button, another application uses settings from within the application, while a third uses a separate application to control settings. When risk based testing is applied in software product lines, an upstream participant may not know the criticality and the percentage of use of a development artefact because that is typically only known at system level. Therefore an upstream participant is not able to determine how rigorous each development artefact needs to be tested. This may result in situation in which to some artefacts too much test time is allocated, and worse, to some artefacts too little test time. An approach needs to be developed in which the upstream participant receives this information from the downstream participants.

239

240

Bibliography [Aagedal 2004]

J. O. Aagedal, I. Solheim, New Roles in Model-Driven Development, in: Proceedings of Second European Workshop on Model Driven Architecture, 2004, pp.109-115. [Abele 2010] A. Abele, R. Johansson, H. Lo, Y. Papadopoulos, M.O. Reiser, D. Servat, M. Torngren, M. Weber: “The cvm framework - a prototype tool for compositional variability management”. In: Proceedings of the Fourth International Workshop on Variability Modelling of Software intensive Systems (VaMoS’10), University of Duisburg-Essen (January 2010) 101– 105 [Accenture 2008] Accenture Communications & High Tech Solutions: “Big Trouble with “No Trouble Found” Returns”. Accenture Report, 12457694, 2008 [Aggarwal1993] K.K. Aggarwal, "Towards a Weighted Operational Profile" , ACM Sigsoft Software Engineering Notes vol. 18 no. 1, 1993 [Alam 2013] M. Alam, A.I. Khan: “Risk-based Testing Techniques: A Perspective Study”, International Journal of Computer Applications, 2013. [Albert2002] C. Albert, L. Brownsword, “Evolutionary Process for Integrating COTSBased Systems (EPIC)”. SEI, Pittsburgh, 2002. [Alionscience2004] http://rac.alionscience.com/pdf/rgh.pdf, accessed 2004. [Alves2006] V. Alves, R. Gheyi, T. Massoni, U. Kulesza, P. Borba and C. Lucena, “Refactoring Product Lines”, GPCE2006. [Amland 2000] S. Amland: “Risk-based testing: Risk analysis fundamentals and metrics for software testing including a financial application case study”, The Journal of Systems and Software 53 (2000) 287-295 [Anderson2006] J. Anderson, M. Jonsson: “The Mobile Handset Industry in Transition: The PC Industry Revisited?”, European School of Management and Technology, Berlin 2006. [Android 2014] http://www.android.com/tv/, accessed March 2015. [AndroidComp 2014] Android compatibility test suite, http://source.android.com/compatibility/cts-intro.html, accessed January 2014. [Androidwatches 2014] http://www.android.gs/best-smartwatches-in-august-2014/, accessed March 2015. [ANSI2003] American National Standards Institute, Information Retrieval (Z39.50): Application Service Definition and Protocol Specification, ANSI/NISO Z39.50-2003 ISSN: 1041-5653, National Information Standards Organization , 2003. [Accenture 2008] Accenture Communications & High Tech Solutions: “Big Trouble with “No Trouble Found” Returns”. Accenture Report, 12457694, 2008

241

Bibliography [Alpar 2010]

[Antkiewicz 2004] [Aoyama 2003]

[Apple 2014] [Appleinsider 2014] [AppleWatch 2014] [Appygen2010] [Atkinson 2002] [Avgeriou 2005]

[Axelsson 2014]

[Bailey 2012] [Baldwin1997] [Basili 1996] [Batory 2005] [BBCNews 2014] [Benavides 2005] [Benavides 2007]

[Benavides 2010]

[Berenbaum 2013]

[Beuche 2008]

242

F. Z. Alpar, H. Lucian-Viorel: "A Supplier Selection Model For Software Development Outsourcing," Annals of Faculty of Economics, University of Oradea, Faculty of Economics, vol. 1(2), pages 1190-1195, December 2010 M. Antkiewicz, K., Czarnecki: “Featureplugin: feature modeling plug-in for eclipse”. In:OOPSLA’04. (2004) 67–72 M. Aoyama, K. Watanabe, Y. Nishio, and Y. Moriwaki: “Embracing Requirements Variety for e-Governments Based on Multiple Product-Lines Frameworks”. Proceedings of the 11th IEEE International Requirements Engineering Conference, 2003. https://www.apple.com/support/appletv/, accessed March 2015. http://appleinsider.com/articles/14/03/29/why-did-microsoft-port-office-toapples-ios-ipad-before-android, accessed March 2015. https://www.apple.com/watch/, accessed March 2015. http://appyframework.org/gen.html, accessed June2010. C. Atkinson: “Component Based Product Line Engineering with UML”, Addison-Wesley, 2002. P. Avgeriou, N. Guelfi: “Resolving Architectural Mismatches of COTS through Architectural Reconciliation”, in: Proceedings of ICCBSS, 2005, LNCS vol. 3412, pp. 248-257. J. Axelsson, E. Papatheocharous, J. Andersson:”Characteristics of software ecosystems for Federated Em-bedded Systems”: A case study. Information & Software Technology 56(11): 1457-1475 (2014) S. Bailey, D. Handu: “Introduction to Epidemiologic Research Methods in Public Health Practice”, Jones and Bartlett Publishers, 2012 C.Y. Baldwin, K. B. Clark. “Managing in the Age of Modularity,” Harvard Business Review 1997, Sept/Oct: 81-93. V. Basili:”The Role of Experimentation: Past, Present and Future”, Proceedings of ICSE-18, Berlin, March 27-29, 1996 Batory, D.S.: “Feature Models, Grammars, and Propositional Formulas”. In: SPLC’05. (2005) 7–20 http://www.bbc.com/news/technology-29431412, accessed March 2015. D. Benavides, P. T. Martín-Arroyo, A. Ruiz Cortés: Automated Reasoning on Feature Models. CAiSE 2005: 491-503 D. Benavides, S. Segura, P. Trinidad, A.R., Cortes: “Fama: Tooling a framework for the automated analysis of feature models”. In: Proceedings of VaMoS’07. (2007) 129–134 D. Benavides, S. Segura , A. Ruiz-Cortés:”Automated analyses of feature models 20 years later: A literature review”, Information Systems, Volume 35, issue 6, 2010. S. Berenbaum:“Forget phones, Android was almost a “smart camera” operating system”, http://www.digitaltrends.com/mobile/android-originallywas-smart-camera-os/#ixzz3AdnI6KDx, April 16, 2013 D. Beuche: “Modeling and building software product lines with pure: :variants”. In: SPLC ’08: Proceedings of the 2008 12th International Software Product Line Conference, Washington, DC, USA, IEEE Computer Society (2008) 358

Bibliography [biglever 2014] [Blair 2009]

[Boersma 2001] [Boersma 2003]

[Boersma 2005A]

[Boersma 2005B]

[Bolwijn 1990]

[Bosch 2006] [Bosch 2009] [Bosch 2010] [Booker 2003] [Botterweck 2008]

[Boucher 2010]

[Boyd 1995]

[Brombacher 1992] [Brombacher 2005]

[Bruhl 2005]

http://www.biglever.com/solution/product.html, retrieved April 2014 G. Blair, T. Coupaye J. Stefani: “Component-based architecture: the Fractal initiative, Annals of Telecommunications”, Volume 64, Numbers 12, Springer, 2009, pp. 1-4 J. Boersma: “How to improve fast customer feedback in a product development process?” MSc. Thesis, TU/e, 2001. J. Boersma, G. Loke, Y. Lu, A. C. Brombacher, H. T. Loh: “Reducing product rejection via a High Contrast Consumer Test”, European Safety and Reliability Conference, 191-194, Maastricht, The Netherlands, 15-18 June 2003. J. Boersma, G. Loke, V.T. Petkova, P.C. Sander, A.C. Brombacher: “Quality of Information Flow in the Backend of a Product Development Process: a Case Study”, Quality and Reliability Engineering International, 20, 255–263, 2005. J. Boersma, G. Loke, V.T. Petkova, P.C. Sander, A.C. Brombacher: “Improving the Quality of Information Flows in the Backend of a Product Development Process: a Case Study”, Quality and Reliability Engineering International, 21,105–114, 2005. P.T. Bolwijn, T. and Kumpe: “Manufacturing in the 1990s—Productivity, flexibility and innovation”. Long Range Planning, 23 (4). pp. 44-57. ISSN 0024-6301 J. Bosch: “Expanding the scope of software product families”, lecture notes in computer science, Springer 2006. J. Bosch: “From Software Product Lines to Software Ecosystems”, SPLC2009,Sheridan Printing, 2009. J. Bosch: “Architecture challenges for software ecosystems”, ECSA 2010: 93-95, 2010 J. D. Booker: “Industrial practice in designing for quality”, International journal of quality and reliability management, 20(3), 288-303, 2003. G. Botterweck, S. Thiel, D. Nestor, S. bin Abid, and C. Cawley: “Visual Tool Support for Configuring and Understanding Software Product Lines”, SPLC 2008. Q. Boucher, A. Classen, P. Faber, P. Heymans: “Introducing TVL, a textbased feature modelling language”. In: VaMoS’10, University of DuisburgEssen (January 2010) 159–162 M.A. Boyd, C. M. Monahan: “Developing integrated hardware-software reliability models: difficulties and issues for digital avionics”, 14th Digital Avionics Systems Conference, 193 – 198, 5-9 Nov. 1995. A. C. Brombacher: “Reliability by Design”, John Wiley & Sons, Chichester (UK), 1992. A. C. Brombacher, P. C. Sander, P. J. M. Sonnemans, J. L. Rouvroye: “Managing product reliability in business processes ‘under pressure’”, Journal of Reliability Engineering and System Safety, 88(2),137-146, 2005. K. Brühl, N. Stieglitz: “Deep Impact: Technological Change and the Vertical Boundaries of the Firm in Deconstructing Industries”, Paper prepared for the Annual Conference on Corporate Strategy (ACCS), Vallendar, 2005

243

Bibliography [BSI 1998]

[Buhne 2004A]

[Buhne 2004B]

[Buhne 2005]

[Calder 2003]

[Capilla 2013] [Cawley 2010]

[Christensen 2002]

[Christensen 2004] [Chrome 2015] [Clements 2001] [Classen 2009]

[Connectedly 2014] [Constaninou 2010] [Collis 2009]

[Cooper 1999].

[Cooper 2011 [Cunha 2007]

244

British Standard: 5760-8: Reliability of Systems, Equipments and Components: “Guide to Assessment of Reliability of Systems Containing Software”, British Standards Institution, 1998 S. Buhne, G. Halmans K. Pohl M. Weber, H. Kleinwechter T. Wierczoch: “Defining Requirements at different levels of abstractions”, 12th IEEE International Requirements Engineering Conference 2004. S. Buhne S., K. Lauenroth K. Pohl: “Why is it not Sufficient to Model Requirements Variability with Feature Models”. Proceedings of Workshop: Automotive Requirements Engineering (AURE04), Nazan University, Nagoya, Japan, 2004, pp5-12. S. Buhne S., K. Lauenroth K. Pohl: “Modeling requirements Variability across Product Lines”, Proceedings of 2005 13th IEEE International Conference on Requirements Engineering. M. Calder, M. Kolberg, M.H. Magill, S. Reiff-Marganiec: “Feature Interaction – A Critical Review and Considered Forecast”. Elsevier: Computer Networks, Volume 41/1, 2003. S. 115-141 R. Capilla et al (eds):”Systems and Software Variability Management”Springer-Verlag Berling Heidelberg 2013. Cawley, C., Healy, P., Botterweck, G., Thiel, S.: Research tool to support feature configuration in software product lines. In: VaMoS’10, University of Duisburg-Essen (January 2010) 179–182 Christensen, C, M Verlinden and G. Westerman : “Disruption, disintegration and the dissipation of differentiability”, Industrial and Corporate Change 2002, Volume 11, Number 5, pp 955-993 C. Christensen, S. Anthony, E. Roth: “Seeing what’s next”, Harvard Business School Press, 2004. https://www.google.com/chrome/devices/, accessed March 2015. P. Clements, L. Northrop: “Software Product Lines: Practices and Patterns”, Addison-Wesley, Boston, MA 2001. A. Classen, Q. Boucher, P. Faber, P. Heymans: “Syntax and semantics of TVL, a comprehensive text-based feature modelling language”. Technical report, PReCISE Research Centre, Univ. of Namur (2009) http://www.connectedly.com/sony-wont-be-using-android-wear-theirsmartwatches, accessed March 2015. A. Constantinou: “Mobile Megatrends 2010”, Vision Mobile Research, www.visionmobile.com/megatrends version 4 May 2010 J. Collis, R. Hussey, R. (2009): “Business Research: A practical guide for undergraduate and postgraduate students”, 3rd edition, New York, Palgrave Macmillan, 2009 A. Cooper:”The inmates are running the asylum: Why high-tech products drive us crazy and how to restore the sanity”, Indianapoli, Sams publishing, 1999 D. R. Cooper, P. S. Schindler: “Business Research Methods”, 2011 McGraw-Hill Education, 2011. A. Cunha, A. Koubaa, R. Severino, M. Alves, Open-ZB: “an opensource implementation of the IEEE 802.15.4/ZigBee protocol stack on TinyOS”,

Bibliography in: Proceedings of the IEEE Internatonal Conference on of Mobile Adhoc and Sensor Systems, 2007, pp. 1–12. [Czarnecki 2000] K. Czarnecki, U.W. Eisenecker: “Generative Programming: Methods, Tools, and Applications”, Addison-Wesley 2000. [Czarnecki 2002] K. Czarnecki, T. Bednasch, P. Unger, U.W. Eisenecker: “Generative programming for embedded software: An industrial experience report”. In: GPCE’02, London, UK, Springer-Verlag (2002) 156–172 [Czarnecki 2004A] K. Czarnecki, S. Helsen, U. Eisenecker: “Staged Configuration through Specialization and Multi-Level Configuration of Feature Models”, Software Process Improvement and Practice 10, (2005) 143-169. [Czarnecki 2004B] K. Czarnecki, “Overview of Generative Software Development”, Unconventional Programming Paradigms (UPP), 2004. [Czarnecki 2005A] K. Czarnecki, S. Helsen, U. Eisenecker: “Staged Configuration through specialization and Multi-Level Configuration of Feature Models”. Software Process Improvement and Practice, 10(2), 2005. [Czarnecki 2005B] K. Czarnecki and M. Antkiewicz, “Mapping features to models: A Template Approach Based on Superimposed Variants”, Generative Programming and Component Engineering, 2005. [Czarnecki 2005C] K. Czarnecki, S. Helsen, U.W. Eisenecker: “Formalizing cardinality-based feature models and their specialization”. Software Process: Improvement and Practice 10(1) (2005) 7–29 [Czarnecki 2007] K. Czarnecki, A. Wasowski, “Feature Diagrams and Logics: There and Back Again”, SPLC 2007. [Czarnecki 2008] K. Czarnecki, S. She, A. Wasowski:”Sample Spaces and Feature Models: There and Back Again”, SPLC'2008. [Czarnecki 2010] K. Czarnecki: “From feature to concept modelling”. In: VaMoS’10, University of Duisburg-Essen (January 2010) 11 Keynote. [daMotaSilveira 2011] P. A. da Mota Silveira Neto, P. Runeson, I. do Carmo Machado, E. S. de Almeida, S. R. de Lemos Meira, E. Engström, "Testing Software Product Lines," IEEE Software, vol. 28, no. 5, pp. 16-20, Sept.-Oct. 2011. [Davenport 2005] T.H. Davenport: “The Coming Commoditization of Processes”, Harvard Business Review, Vol 83, 100-108, 2005 [Debbi 2007] O. Debbi, C. Salinese, G. Fanmuy: “Industry Survey of Product Lines Management Tools: Requirements, Qualities and open Issues”. 15th IEEE International Requirements Engineering Conference, 2007 IEEE. [Deelstra 2008] S. Deelstra, M. Sinnema: “Managing the Complexity of Variability in Software Product Families”. PhD thesis, Institute of Mathematics and Computing Science, University of Groningen, September 2008. [Delannay 2007] G. Delannay, K. Mens, P. Heymans, P.Y. Schobbens, J.M. Zeippen: “Plonegov as an open source product line”. In: OSSPL’07, collocated with SPLC’07. (2007) [Denney 2005] R. Denney: “Succeeding with Use Cases”, Addison Wesley, ISBN 0-32131643-6, 2005. [Deursen 2002] A. van Deursen, P. Klint: “Domain-specific language design requires feature descriptions”. Journal of Computing and Information Technology 10 (2002) 2002

245

Bibliography [Devroey 2014]

[Dickerson 1983]

[Digitaltrends 2010] [doCarmo 2012]

[Dordowsky 2009]

[Dourmas 2009]

[Dubinsky 2011]

[Duryee 2014] [Eclipse 2010] [Easterbrook 2009]

[Egyed 2006]

[Eisenhardt 1989] [Engadget 2014] [Ensan 2011]

[Evans 1999] [Feldt 2010] [Fleurey 2009] [Freescale 2008]

246

X. Devroey, G. Perrouin, M. Cordy, P. Schobbens, A. Legay, P. Heymans: “Towards statistical prioritization for software product lines testing”. VaMoS 2014: 10 M. D. Dickerson, J. W.Gentry: “Characteristics of adopters and nonadopters of home computers”, Journal of Consumer Research, 10, 225-235, 1983. http://www.digitaltrends.com/gaming/hardware-gaming/sony-adoptsandroid-3-0-for-upcoming-playstation-phone/ , accessed Februari 2011 I. do Carmo Machado, J. D. McGregor, E. S. de Almeida. Strategies for testing products in software product lines”. SIGSOFT Software Engineering Notes , 37:1–8, 2012 F. Dordowsky, W. Hipp: “Adopting software product line principles to manage software variants in a complex avionics system”. In: SPLC’09, San Francisco, CA, USA (2009) 265–274 G. Dourmas, N. Nikitakos: “ A New Approach to the Digital Maritime Economy”, In: Proceedings of the European and Mediterranean Conference on Information Systems (EMCIS2009), June 2009. Y. Dubinsky, A. Hartman, M. Keren: “Modelware – 511731 - D5.3: Industrial ROI, Assessment, and Feedback.”https://www.research.ibm.com/haifa/dept/services/papers/D5.3_ Assesment_feedback_report.pdf, accessed August 9 2011. T. Duryee: “We Learned Just How Great Of A Partner HTC Is To Microsoft”, Feb. 17, 2009, http://paidcontent.org/, accessed March 2015. http://www.eclipse.org/, accessed January2011. S Easterbrook, J. Slagter,M. Storey, D. Damian: “Selecting Empirical Methods fot Software Engineering Research”, in F. Shull et al. (eds), Guide to Advanced Empirical Software Engineering, Springer 2008. A. Egyed, R. Balzer: “Integrating COTS Software into Systems through Instrumentation and Reasoning”, Automated Software Engineering 13 (2006) 41-64. K. Eisenhardt: "Building theories from case study research", Academy of Management Review 1989, Vol. 14 No.4, pp.532-50. http://www.engadget.com/2014/06/25/android-tv-is-coming-to-sony-sharpand-philips-tvs-next-year/, accessed March 2015. A. Ensan, E. Bagheri, M. Asadi,D. Gasevic, Y. Biletskiy, "Goal Oriented Test Case Selection and Prioritization for Product Line Feature Models". In Proc. Of the 8th Int. Conf. on Information Technology: New Generations, USA, p. 291 J. R. Evans, W. M. Lindsay: “ The management & control of quality (4th Ed)”,International Thomson Publishing, 1999 R. Feldt, A. Magazinius: Validity Threats in Empirical Software Engineering Research - An Initial Survey. SEKE 2010: 374-379 F. Fleurey, Ø. Haugen, B. Møller-Pedersen, G. Olsen, A. Xiaorui Zhang: “A Generic Language and Tool for Variability Modeling”, SINTEF, 2009. Freescale Semiconductors Inc: “Mobile Extreme Convergence: A Streamlined Architecture to Deliver Mass-Market Converged Mobile Devices”, White Paper, 2008

Bibliography [Gadhavi 2010] [Gannes 2014]

[Ganesan 2007]

[Gartner 2014] [Gawer 2002] [Gawer 2008] [Gay 2003]

[Gella 2006] [Genugten 2007] [Geudens 2005]

[Gilson 2012] [Goulão 2007]

[Gray 1985] [Greenfield 2004]. [Grobart 2013] [Grove 1996]

[Gurp 2001]

[Halasz 2011]

B. Gadhavi, K. Shah: “Analysis of the emerging android market”. San Jose State University, 2010. L. Gannes, I. Fried:“After Google Pressure, Samsung Will Dial Back Android Tweaks, Homegrown Apps”, January 29 2014, http:// http://recode.net/2014/01/29. D. Ganesan, J. Knodel, R. Kolb, U. Haury, G. Meier:”Comparing Costs and Benefits of Different Test Strategies for a Software Product Line: A Study from Testo AG”. SPLC 2007. Gartner reports on smartphone market shares 2007 – 2013, e.g. http://www.gartner.com/newsroom/id/2623415, accessed March 2015. A. Gawer, M. Cusumano: “Platform leadership”. Harvard Business School Press 2002. A. Gawer, M. Cusumano: “‘How companies become platform leaders’’, MIT Sloan Management Review, 2008 D. Gay, P. Levis, R. von Behren, M. Welsh, E. Brewer, D. Culler: “The nesC Language: A Holistic Approach to Networked Embedded Systems”, in: Proceedings of the Conference on Programming Language Design and Implementation 2003, pp. 1-11. G. Della Riccia,D. Dubois,R. Kruse,H. Lenz:” Decision Theory and MultiAgent Planning”, Springer Science & Business Media, 2006 M. van Genuchten: “The Impact of Software Growth on the Electronics Industry”. IEEE Computer, 2007 W. H. J. M. Geudens, P. J. M. Sonnemans, V. T. Petkova, A. C. Brombacher: “Soft reliability, a new class of problems for innovative products: "how to approach them"”, Proceedings Annual Reliability and Maintainability Symposium, 374 – 378, Jan. 24-27, 2005. D. Gilson: “The History of Symbian's Secret Fragmentation” http://www.allaboutsymbian.com/, March 12th, 2012. M. Goulão, F. Brito e Abreu: "An overview of metrics-based approaches to support software components reusability assessment" in Software Quality Measurement: Concepts and Approaches, ICFAI Books, 2007. J.Gray:“Why do computers stop and what can be done about them?”, Tandem TR 85.7, 1985 J. Greenfield, K. Short: “Software Factories: Assembling Applications with Patterns, Models, Frameworks, and Tools”, Wiley, 2004. S. Grobart: “Apple Chiefs Discuss Strategy, Market Share and the New iPhones”. http://www.businessweek.com, September 19, 2013. A.S Grove: “Only the Paranoid Survive: How to exploit the crisis points that challenge every company and career”. Currency Doubleday, October 1996. J. Van Gurp, J. Bosch, M. Svahnberg: “On the Notion of Variability in Software Product Lines”, in: Proceedings of the Working IEEE/IFIP Conference on Software Architecture (WICSA 2001), 2001, pp. 45-55. M. Halasz:” Menu expands at the OS diner”, http://embeddedcomputing.com/menu-expands-the-diner-2, February 14th, 2011

247

Bibliography [Hamer 1996]

[Hartmann 2005A]

[Hartmann 2005B] [Hartmann 2006]

[Hartmann 2008]

[Hartmann 2009] [Hartmann 2012]

[Hartmann 2013]

[Hartmann 2014A]

[Hartmann 2014B]

[Henard 2012]

[Henzinger 2006]

[Holak 1988]

[Hollister 2014] [Holtz 1988]

[Horizon 2014]

248

P. van den Hamer, K. Lepoeter: “Managing design data: the five dimensions of CAD frameworks, configuration management, and product data management”. Proceedings of the IEEE, Volume 84, Issue 1, Jan 1996 pp: 42 – 56. H. Hartmann, V. Ronteltap, J. Bökkerink-Scheerová: “How to reduce your test process with 30%; The use of operational profiles in practice”, Internal Philips Software Conference, Veldhoven, The Netherlands, April 2005. H. Hartmann: “FCR Prediction, discussion paper”, Philips Internal Presentation, Philips Applied Technologies, May 2005. H. Hartmann, J. Bokkerink, V. Ronteltap: "How to reduce your test process with 30%; The application of operational Profiles at Philips Medical Systems”. The 17th IEEE International Symposium on Software Reliability Engineering (2006) H. Hartmann, T. Trew: “Using Feature Diagrams with Context Variability to Model Multiple Product Lines for Software Supply Chains”, in: proceedings of SPLC08, pp. 12-21. H. Hartmann, T. Trew, A. Matsinger: “Supplier Independent Feature Modeling”, in: Proceedings of SPLC09, pp. 361-376. H. Hartmann, T. Trew, J. Bosch: “The changing industry structure of software development for consumer electronics and its consequences for software architectures”. The Journal of Systems & Software 85 (2012), pp. 178-192 H. Hartmann, M. Keren, A. Matsinger, J. Rubin, T. Trew, T. YatzkarHaham: “Using MDA for integration of heterogeneous components in software supply chains”, Science of Computer Programming, Volume 78, Issue 12, 1 December 2013, Pages 2313–2330 H. Hartmann, J. Bosch: “Orchestrate Your Platform: Architectural Challenges for Different Types of Ecosystems for Mobile Devices”. ICSOB 2014: 163-178. H. Hartmann, F. van der Linden, J. Bosch: Risk based testing for software product line engineering. SPLC 2014: 227-231, DOI 10.1145/2648511.2648535 C. Henard, M. Papadakis, G. Perrouin, J. Klein, P. Heymans, Y. Le Traon: Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-wise Test Suites for Large Software Product Lines. CoRR abs/1211.5451 (2012) T. Henzinger, J. Sifakis. "The embedded systems design challenge”, Proceedings of the 14th International Symposium on Formal Methods (FM), August, 2006. S. L. Holak: “Determinants of Innovative durable adoption: an empirical study with implications for early product screening”, Journal of Product Innovation Management, 7, 50-69, 1988. S. Hollister: “The indie game console may already be dead”, http://www.theverge.com, march 7, 2014. N. Holtz, W. Rasdorf: “An evaluation of programming languages and language features for engineering software development”, Engineering with Computers 3 (1988) 183–199 https://www.horizon.tv/nl_nl/, accessed March 2015.

Bibliography [Hubaux 2008]

[Humphrey 1989] [Husk 2007] [Iansito 2004] [IBM 2011]

[Incentmedia 2011] [Infineon 2010] [Information 2014A] [Information 2014B] [ISO 2007A] [ISO 2007B]

[Jacobs 2003]

[Jacobs 2007].

[Janjua 2009]

[Jansen 2007]

[Jansen 2009]

[Jansen 2012]

[Jansen 2013] [Java 2011]

A. Hubaux, P. Heymans, D. Benavides: “Variability modelling challenges from the trenches of an open source product line re-engineering project”. In: SPLC’08, Limerick, Ireland (2008) 55–64 W. Humphrey: “Managing the Software Process”, Addison Wesley 1989. S. Husk: “Less can Mean More”, Appliance Magazine, August 2007, ApplianceMagazine.com, 2007. M. Iansiti, R. Levien: “The keystone advantage”, Harvard Business School Press, 2004 IBM Rational Software Architect for WebSphere Software. http://www-01.ibm.com/software/awdtools/swarchitect/websphere/, accessed August 9 2011. http://incentmedia.wordpress.com/2011/02/24/androids-advertisingrevenue-to-reach-1-3-billion-in-2012/ February 24 2011 http://www.infineon.com, Accessed July 2010 http://www.informationweek.com/mobile/mobile-business/samsung-talkstizen-os-of-everything/d/d-id/1269399, accessed March 2015. http://www.informationweek.com/mobile/the-good-and-bad-of-tablets-atwork/d/d-id/1110346? , accessed March 2015. ISO/IEC 23004-1:2007, Information Technology – Multimedia Middleware – Part 1: Architecture. International Organization for Standardization, 2007. ISO/IEC 23004-3:2007, Information Technology – Multimedia Middleware – Part 3: Component Model. International Organization for Standardization, 2007. J. Jacobs, J. H. van Moll, P. J. Krause, R. J. Kusters, J. J. M. Trienekens: Effects of Virtual Development on Product Quality: Exploring Defect Causes. STEP 2003: 6-15 J. Jacobs, J. van Mol:”Effects of Virtual Product Development on Product Quality and their Influencing Factors”, PhD Thesis, Technical University of Eindhoven, 2007 N.K. Janjua, M. Hussain, M. Afzal, H.H. Ahmad: “Digital health care ecosystem: SOA compliant HL7 based health care information interchange“,3rd IEEE International Conference on Digital Ecosystems and Technologies, 2009. S. Jansen, S. Brinkkemper, A. Finkelstein: “Providing Transparency In The Business Of Software: A Modeling Technique For Software Supply Networks”. Virtual Enterprises and Collaborative Networks 2007: 677-686 S. Jansen, S. Brinkkemper, A. Finkelstein: “A Sense of Community: A Research Agenda for Software Ecosystems”. 31st International Conference on Software Engineering, New and Emerging Research Track 2009. S. Jansen, M. A. Cusumano:”Defining Software Ecosystems: A Survey of Software Platforms and Business Network Governance”. IWSECO@ICSOB 2012: 40-58 S. Jansen: “How quality attributes of software platform architectures influence software ecosystems.”, WEA 2013: 6-10. Java Emitter Templates: http://www.eclipse.org/emft/projects/jet, accessed August 9 2011.

249

Bibliography [Jobs 2010] [Johnson 2015]

[Jonkers 1993] [Jorgensen 2013] [Kain 2014]

[Kang 1990]

[Kano 1984]

[Karlsson 2006]

[Kastner 2009]

[Kim 2005]

[Kim 2011]

[Kistler 2008A] [Kistler 2008B] [Knaus 2014]

[Koca 2006]

250

S. Jobs: “Thought on Flash”, http://www.apple.com/hotnews/thoughts-onflash/, April 2010, accessed November 24 2011. L. Johnson:”Huawei expresses wish for more customisable Android Wear platform”” http://www.trustedreviews.com/news/huawei-calls-on-googleto-allow-freedom-to-customise-androidwear#7e33DpstJ7JywDuX.99.,March 17 2015. H. B. M. Jonkers, “An overview of the SPRINT method”, FME '93: Industrial-Strength Formal Methods. Springer. P. C. Jorgensen:”Software Testing: A Craftsman’s Approach”, Auerbach Publications; 4 edition (October 18, 2013) E. Kain: “The Ouya Is Basically Dead”, http://www.forbes.com/sites/erikkain/2014/03/07/hit-kickstarter-videogame-console-ouya-is-basically-dead/, March 7, 2014. K. Kang, S.Cohen, J. Hess, W. Nowak, S. Peterson: “Feature-oriented domain analysis (FODA) feasibility study”, Technical Report CMU/SEI90-TR-21, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, 1990 N. Kano, N. Seraku, F. Takahashi, and S. Tsuji: “Attractive quality and must be quality”, The Journal of the Japanese Society for Quality Control, 14(2):39–48, 1984. L. Karlsson, M. Höst, B. Regnell: “Evaluating the practical use of different measurement scales in requirements prioritisation”, ISESE '06 Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering, 2006. C. Kastner, T. Thum, G. Saake, L. Feigenspan, T. Leich, F. Wielgorz, S. Apel: “FeatureIDE: A tool framework for feature-oriented software development”. In: Proceedings of ICSE’09. (2009) 311–320 S. Kim, H. Min, S.Y. Rhew: “Variability Design and Customization Mechanisms for COTS Components”, in: Proceedings of International Conference on Computational Science and its Applications, 2005, pp. 57 – 66. C.H.P.Kim, D.S. Batory. S. Khurshid:”Reducing combinatorics in testing product lines”, in Proc. Of the 10th Inlt. Conference on Aspect-oriented Software Development, AOSD, ACM 2011. R. Kistler, S. Knauth and A. Klapproth, “EnerBee - An Example of an Advanced Metering Infrastructure Based on ZigBee”, EuZDC 2008. R. Kistler, S. Knauth and A. Klapproth, “An Adaptive Network Architecture for Home- and Building Environments”, ETFA 2008. E. Knaus, I. Hammouda: “EAM: Ecosystemability Assessment Method”, Proceedings of 22nd International Requirements Engineering Conference, 2014. A. Koca, L. Yuan, A.C. Brombacher, and J.H. Hartmann: “Towards establishing foundations for new classes of reliability problems concerning strongly innovative products”. In 2006 IEEE International Conference on Management of Innovation and Technology (Singapore), volume 2, pages 581–584. IEEE, 2006.

Bibliography [Kontio 1996]

[Korhonen 2010]

[Kornby 2005] [Kovacevic 2007] [Krafft 2003]

[Krahn 2006]

[Kramer 1999] [Kronlof 2006] [Krueger 2007]

[Lamancha 2013]

[Lazaro 2005]

[Lee 1994]

[Lee 2012] [Lewis 1996] [Limo 2011] [Linden 2000]

[Linden 2004]

J. Kontio, G. Caldiera, V.R. Basili:”Defining Factors, Goals and Criteria for Reusable Component Evalua-tion.” CASCON’96 Conference, Toronto, Canada, 1996. H. Korhonen, J. Arrasvuori. K.Väänänen-Vainio-Mattila: “Analysing User Experience of Personal Mobile Products through Contextual Factors”, International Conference on Mobile and Ubiquitous Multimedia (MUM), ACM Press (2010), pp 1-10. M. Kornby: “The EMP story”, Ericsson Mobile Platform, ERICSSON REVIEW, NO. 01, 2005. M. A. Kovacevic: “Key Success Factors for Ericcson Mobile Platforms using Porter’s Value Chain model”, Blekinge Institute of Technology, 2007. J. Krafft:” Vertical structure of the industry and competition: an analysis of the evolution of the info-communications industry”, Elsevier Science Direct, 2003. H. Krahn, B. Rumpe, S. Völkel: “ Roles in Software Development using Domain Specific Modeling”, in: Proceedings of the 6th OOPSLA Workshop on Domain-Specific Modeling (DSM' 06), Technical Report TR37, Jyväskylä University, 2006. R. Kramer: “Consumer electronics as silicon engine”, IEEE, IEDM Technical Digest. International, 3-7, 1999. K. Kronlöf, S. Kontinen: “Use Case Driven Platform Development”. MARTES Finnish Partners’ Workshop Tampere, May 17, 2006. C.W. Krueger: “Biglever software gears and the 3-tiered spl methodology”. In:OOPSLA’07: Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion, New York, NY, USA, ACM (2007) 844–845 B. P. Lamancha, M. Polo & M. Piattini (2013):”Systematic Review on Software Product Line Testing”. Software and Data Technologies, Springer Berlin Heidelberg. M. Lázaro, E. Marcos: “Research in Software Engineering: Paradigms and methods”. Workshop on philosophical Foundations of Information Systems Engineering, CAISE, Oporto, (2005). H. Lee, C. Billington: “Designing Products and Processes for Postponement”. In S. Dasu and C. Eastman (eds) Management of Design: Engineering and Management Perspectives, pp105-122, Kluwer, 1994. J. Lee, S. Kang, D., Lee:”A survey on software product line testing”. In: 16th International Software Product Line Conference. (2012) E.E. Lewis: “Introduction to Reliability Engineering”, 2nd Edition, John Wiley & Son, 1996. www.limofoundation.org, accessed March 2011. F. van der Linden: “Experiences with Family Architectures for Mass Electronics”. In M. Jazayeri, A. Ran and F. van der Linden, Software Architecture for Product Families: Principles and Practice: Addison Wesley, 2000. Frank van der Linden, Jan Bosch, Erik Kamsties, Kari Känsälä, J. Henk Obbink: Software Product Family Evaluation. SPLC 2004: 110-129

251

Bibliography [Linden 2007]

[Lytro 2014]

[Lyu 1996] [Maemo 2011] [McGregor 2009] [McGregor 2010]

[Meego 2014] [Mendonca 2009A] [Mendonca 2009B] [Messerschmitt 2004] [Minderhoud 2005]

[Mintel 2014]

[MIPI 2010] [Mohagheghi 2008]

[Moore 1965] [Moore 2006] [Moreau 2001]

[Musa 1987] [Musa 1993]

252

F. van der Linden, K. Schmid, E. Rommes: Software Product Lines in Action: “The Best Industrial Practice in Product Line Engineering”, Springer 2007. https://www.lytro.com/press/releases/2014-04-22_lytro-unveils-worldsfirst-light-field-camera-and-designed-for-creative-pioneers/, accessed March 2015. M.R. Lyu (Ed.) “Handbook of Software Reliability Engineering”, McGraw-Hill 1996. ISBN 0-07-039400-8 www.maemo.org. Accessed March 2011. J. McGregor: “Ecosystems”, Journal of Object Technology, vol. 8, no. 6, September-October 2009. J.D. McGregor: “Testing a Software Product Line” in Testing Techniques in Software Engineering, Lecture Notes in Computer Science, 2010, Volume 6153/2010, pp 104-140. http://www.allaboutmeego.com/, accessed March 2015. M. Mendonca, M. Branco, D. Cowan: “ S.p.l.o.t. - software product lines online tools”. In: Proceedings of OOPSLA’09. (2009) 761–762 M. Mendonca: “Efficient Reasoning Techniques for Large Scale Feature Models”. PhD thesis, University of Waterloo (2009) D. G. Messerschmitt, C. Szyperski: “Software Ecosystem“, MIT Press, 2004 S. Minderhoud. P. Fraser: “Shifting paradigms of product development in fast and dynamic markets”, Reliability Engineering and System Safety, 88(20: 127-135, 2005 http://www.mintel.com/press-centre/technology-press-centre/sales-ofdigital-cameras-decline-as-consumers-snap-up-smartphones, accessed March 2015. The Mobile Industry Processor Interface Alliance, http://www.mipi.org/, accessed July 2010. P. Mohagheghi, V. Dehlen: “Where Is the Proof? - A Review of Experiences from Applying MDE in Industry”, in: Proceedings of the 4th European conference on Model Driven Architecture: Foundations, 2008, pp. 432 – 443. G.E. Moore: “Cramming more components onto integrated circuits”, Electronics, Vol 38, p114-117, 1965. J. F. Moore: “Businesses ecosystems and the view from the firm”: The Antitrust Bulletin, March 2006, American Antitrust Institute, 2006. C. P. Moreau, D. R. Lehmann, A. B. Markman: “Entrenched knowledge structures and consumer response to new products”, Journal of Marketing Research, 38, 14-29, 2001. J. D. Musa, A. Iannino, K. Okumoto: “Software reliability: Measurement, prediction, application”, McGraw-Hill, New York, 1987. J. D. Musa:“Operational profiles in software reliability engineering”, IEEE Software, 10(2), 14-32, 1993.

Bibliography [Musa 1996]

[Musa 1999] [Mybasis 2014] [Nachira 2007]

[Nagrani 2007]

[Nazir 2014]

[Nestor2007] [Neufelder 2000]

[Nokia 2011] [NokiaHP 1996] [Norman 2002] [NXP 2010] [Oberhauser 2007]

[OMA 2010] [OMG 2011] [Ommering 2002A]

[Ommering 2002B]

[Ommering 2004] [Onlinereport 2014]

J. D. Musa: “The operational profile, Reliability and Maintenance of Complex Systems”, NATO ASI Series, F154, 332–343, Springer, Berlin, 1996. J. D. Musa: “Software Reliability Engineering”, McGraw-Hill 1999. ISBN 0-07-913271-5, 2nd Edition Author House, 2004, ISBN 1-4184-9388-0. http://www.mybasis.com/, accessed March 2015. F. Nachira, A. Nicolai, P. Dini, M. Le Louarn, L. Rivera Leon (Eds.):”Digital Business Ecosystems”, European Commission Publications Office, 2007. S. Nagrani, “Digital television chip market to double by 2011". EDN, 2 February 2007. http://www.edn.com/article/CA6438551.html. Accessed February 2011. S. Nazir, S. Anwar, S.A. Khan, S. Shahzad, M. Ali, R. Amin, M. Nawaz, P. Lazaridis, J. Cosmas: “Software component selection based on quality criteria using the analytic network process”, August 2014, Hindawi Publishing Corporation D. Nestor, L. O’Malley, E. Sikora, S.Thiel:“Visualisation of variability in software productline engineering”. In: VaMoS’07. (2007) A.M. Neufelder:“How to measure the impact of specific development practices on defect density”. In Proceedings. 11th International Symposium on Software Reliability Engineering (ISSRE), 2000. www.nokia.com , accessed March 2011. http://betterthaniphone.com/smartphone-2/bbg-communications-londonsmartphone/ D. A. Norman:”Home Theatre: Not ready for prime time”, Computer, 35(6), 100-102. NXP Semiconductors. http://www.nxp.com/ , Accessed June 2010 R. Oberhauser, R. Schmidt: “Improving the Integration of the Software Supply Chain via the Semantic Web”, International Conference on Software Engineering Advances, 2007. ICSEA 2007. Open Mobile Allicance, http://www.openmobilealliance.org/, accessed July 2010. OMG Model Driven Architecture, http://www.omg.org/mda/. Accessed August 9 2011. R. van Ommering: “Building Product Populations with Software Components”, Proceedings of 24th International Conference on Software Engineering, Orlando, Florida, September 2002, pp255-265. R. van Ommering, T. Trew: “Building Product Populations with Software Components Symposium and the consequences for requirements” on: Recycling Requirements for Systems and Product Families. London, April 24th, 2002 R. van Ommering: “Building Product Populations with Software Components”. PhD Rijks Universiteit Groningen, 2004. http://www.onlinereporter.com/2014/07/02/lg-opens-up-its-smart-tvswebos-to-developers/, accessed March 2015.

253

Bibliography [OpenMAX 2010] [Oshana 1994] [OSL 2010] [Ouden 2005]

[Ouden 2006A]

[Ouden 2006B]

[Pang 2011] [Pant 1994]

[Park2007]

[Pebble 2014] [Peppard2006] [Petkova2003]

[PHOF2014] [Plonegov2010] [Pohl2001]

[Pohl 2005A] [Pohl 2005B]

[Popp 2010] [Porter 1980] [Porter 1980] [Pratt 1994]

254

“OpenMAX - The Standard for Media Library Portability”, http://www.khronos.org/openmax/. Accessed July 2010 R. Oshana: "And automated testing environment to support operational profiles for software intensive systems" , Raython Systems 1994. O.S. Laboratories, http://www.osl.be/, Accessed June 2010 E. Den Ouden, Y. Lu, N. Ganesh, W. Geudens: “A Customer-Focused Reliability Approach”. Annual Reliability and Maintainability Symposium 2005, Alexandria, VA USA, 24-27 Jan 2005, 207 – 213 E. Den Ouden, Y. Lu, P. J.M. Sonnemans, A. C. Brombacher:”Quality and reliability problems from a consumers perspective: an increasing problem overlooked by businesses?” Quality and Reliability Engineering International. 2006; 22: 821-838. E. Den Ouden: “Development of a Design Analysis Model for Consumer Complaints”. PhD. Thesis, Technical University of Eindhoven, Eindhoven 2006. B. C. Pang, “Android’s advertising revenue to reach $1.3 billion in 2012” H. Pant, P. Franklin, W. Everett, A structured approach to improving software-reliability using operational profiles, Proceedings Annual Reliability and Maintainability Symposium, 142-146, 24-27 Jan. 1994. C. Park, S. Hong, K. Son, J. Kwon: “A Component Model Supporting Decomposition and Composition of Consumer Electronics Software Product Lines”, in: Proceedings of SPLC'07, pp. 181-192. https://getpebble.com/pebble, accessed March 2015. J. Peppard, A. Rylander: “From Value Chain to Value Network: Insights for Mobile Operators”, European Management Journal, Vol. 24, Issue 2, 2006. V. T. Petkova: “An Analysis of Field Feedback in Consumer Electronics Industry”, PhD Thesis, Technische Universiteit Eindhoven, Eindhoven, 2003. Product Line Hall of Fame: http://splc.net/fame.html accessed December 2014 PloneGov. http://www.plonegov.org/ , accessed June 2010 K. Pohl, A. Reuys: “Considering Variabilities during Component Selection in Product Family Development”, Lecture Notes in Computer Science, Volume 3412/2005, 2005, pp. 223-235. K. Pohl, G. Bockle, F. van der Linden: “Software Product Line Engineering”. Springer 2005. K. Pohl, A. Reuys, Considering Variabilities during Component Selection in Product Family Development, Lecture Notes in Computer Science, Volume 3412/2005, 2005, pp. 223-235. K. Popp, R.Meyer: “Profit from Software Ecosystems”, Synomic 2010. M. Porter: “Competitive strategy: Techniques for Analyzing Industries and Competitors”. The Free Press, New York, 1980 M. Porter: “Competitive Advantage Creating and Sustaining Superior Performance”, New York: The Free Press, 1985 T.W. Pratt: “Programming Languages : Design and Implementation”. Second edition edn. Prentice Hall (1984) 604 pages.

Bibliography [Pugh 1996] [Pullum 1999] [purevariants 2006]

[purevariants 2009] [Qualcomm 2014] [Raaijmakers 2009] [Raskin 2005] [Rassweiler 2009]

[Rees 1994] [Reiser 2005] [Reiser 2006]

[Reiser 2007]

[Reiser 2009] [Riebisch 2002] [Robson 2002] [Rogers 1995] [Rooijmans 1996] [Runeson 2009]

[Rus 1999]

S. Pugh, D. Clausing, R. Andrade: “Creating Innovative Products Using Total Design”. Addison Wesley Longman, 1996. I. Pullum: “Software Fault Tolerance”, Annual reliability and maintainability symposium, 1999. pure-systems GmbH: Variant management with pure::variants. http://www.puresystems. com/fileadmin/downloads/pv-whitepaper-en04.pdf (2006) Technical White Paper. pure::variants, Software Product Line Development with Feature Modelling, http://www.pure-systems.com. Accessed August 9 2011. http://www.qualcomm.com/solutions/operating-systems, accessed March 2015. R. Raaijmakers: “Focus NXP goed voor ecosystem” (translated: Focus NXP good for ecosystem). Bits&Chips, October 2009. A. Raskin, N. Mellquist: “The new industrial revolution: De-verticalization on a global scale”, AllianceBernstein 2005. A. Rassweiler: “iPhone 3G S Carries $178.96 BOM and Manufacturing Cost”, http://www.isuppli.com/Teardowns/News/Pages/iPhone-3G-SCarries-178-96-BOM-and-Manufacturing-Cost-iSuppli-TeardownReveals.aspx, June 24, 2009. R. A. Rees: “Detectability of Software Failure”, Reliability Review, 14(4), 10-30, December 1994. M. Reiser, M. Weber:“Using product sets to Define Complex Product Decisions”, SPLC 2005 Springer. M. Reiser, M. Weber: “Managing Highly Complex Product Families with Multi-Level Feature Trees”, 14th IEEE International Requirements Engineering Conference (RE'06) pp. 149-158 M. Reiser, R. Tavakoli Kolagari, M. Weber, Unified Feature Modeling as a Basis for Managing Complex System Families, in: Proceedings of the 1st International Workshop on Variability Modeling of Software-intensive Systems, 2007, pp. 19-87. M.O. Reiser: “Core concepts of the compositional variability management framework (cvm). Technical report”, Technische Universit¨at Berlin (2009) M. Riebisch, K. Bollert, D. Streitferdt, I. Philippow: “Extending feature diagrams with uml multiplicities”. In: IDPT’02. (2002) C. Robson: “Real world research”, Blackwell publishing, 2002. E.M. Rogers: “Diffusion of Innovations”, 4th ed., The Free Press, New York, NY, 1995 J. Rooijmans, H. Aerts, M. van Genuchten: “Software Quality in Consumer Electronics Products”. IEEE Software 1996. P. Runeson, M Host: "Guideliness for conducting and reporting case study research in software engineering”, Emperi. Software Eng 14: 131-164, Springer, 2009. I. Rus, J.S. Collofello:“A decision support system for software reliability engineering strategy selection”, Computer Software and Applications Conference, 1999. COMPSAC '99.

255

Bibliography R.T. Rust, D.V. Thompsosn, R.W. Hamilton:”Defeating feature fatique”, Harvard Business Review, 84 (2), 98-107 [Schmidt1997] R. Schmidt The implementation of simultaneous engineering in the stage of product concept development in the stage of product concept development: A process orientated improvement of quality function deployment, European Journal of Operational Research, 100, 293-314, 1997. [Samsungforum 2014] http://www.samsungdforum.com/Devtools/index, accessed March 2015. [Saunders 2009] M. Saunders, P. Lewis, A. Thornhill: “Research methods for business students”, 5th ed., Harlow, Pearson Education, 2009. [Savolainen 2009] J. Savolainen, J. Bosch, J. Kuusela, T. Männistö:”Default values for improved product line management”. SPLC 2009: 51-60 [Schaefer 2014] H. Schaefer:“Test Management is Risk management, Risk Based Testing”, Software Test Consulting, Reigstad, http://home.c2i.net/schaefer/testing.html, accessed april 2014. [Sheat 2010] W.R. Sheate: “Tools, Techniques and Approaches for Sustainability”, World Scientific Publishing Company (September 30, 2010) [Schobbens 2006A] P.Y. Schobbens, P. Heymans, J.C. Trigaux, Y. Bontemps: “Generic semantics of feature diagrams. Computer Networks”, doi:10.1016/j.comnet.2006.08.008, special issue on feature interactions in emerging application domains (2006) 38 [Schobbens 2006B] P.Y. Schobbens, P. Heymans, J.C. Trigaux, Y. Bontemps: “Feature Diagrams: A Survey and A Formal Semantics”. In: RE’06. (September 2006) 139–148 [Segura 2007] S. Segura, D. Benavides, A. Ruiz-Cortés, P. Trinidad: “Automated Merging of Feature Models Using Graph Transformations “, GTTSE 2007. [Segura 2010] S. Segura, R. M. Hierons, Ds Benavides, A. Ruiz Cortés: Automated Test Data Generation on the Analyses of Feature Models: A Metamorphic Testing Approach. ICST 2010. [Shaw 2002] M. Shaw: “What Makes Good Research in Software Engineering ?”, International Journal of Software Tools for Technology Transfer, 2002, vol 4, no1, pp 1-7. [Shull 2007] F. Shull, J. Singer, D.I.K. Sjøberg: “Guide to Advanced Empirical Software Engineering”. Springer-Verlag New York, Inc., Secaucus, NJ, USA (2007) [Siewiorek 2004] D. P. Siewiorek, R. Chillarege, Z. Kalbarczyk: “Reflection on Industry Trends and Experimental Research in Dependability”, IEEE Transactions on Dependable and Secure Computing, 1 (2), pp.109-127, 2004. [Sjaaksjarvi 2003] M. Saaksjarvi: “Consumer adoption of technological innovations”, European Journal of Innovation Management, 6(2), 90-100, 2003 [Smeda 2008] A. Smeda, M. Oussalah, A. ElHouni, E-B. Fgee: “COSABuilder: an Extensible Tool for Architectural Description”, in: Proceedings of the 3rd International Conference on Information and Communication Technologies: From Theory to Applications, ICTTA 2008, pp. 1-6. [Stahl 2005] T. Stahl, M. Voelter: “Model-Driven Software Development”, Wiley, 2005. [Strat 2008] Strategy Analytics:”Understanding the Mobile Ecosystem”, Adobe Systems, 2008. [Strategy 2014] http://www.strategyanalytics.com/ [Rust 2005]

256

Bibliography default.aspx?mod=pressreleaseviewer&a0=5475, accessed March 2015. T. J. Sturgeon: “Exploring the Risks of Value Chain Modularity, Electronics Outsourcing During the Industry Cycle of 1992-2002”, MIT Press, 2003 [Samsung 2011] www.samsung.com, accessed March2011. [Sangwan 2004] Sangwan, S., Hian, C.K.: “User-centered design: marketing implications from initial experience in technology supported products”. In Press, I.C.S., ed.: Engineering Management Conference. Volume 3. (2004) 1042– 046 [Steelrace 2004] http://www.steeltrace.com/products_lp_rup_general.htm, accessed 2004 [Sturgeon 2003] T. J. Sturgeon: “Exploring the Risks of Value Chain. Modularity: Electronics Outsourcing. During the Industry Cycle of 1992-2002”, MIT, 2003 [Suoranta 2006] R. Suoranta: “New Directions in Mobile Device Architectures", in 9th EUROMICRO Conference on Digital System Design: Architectures, Methods and Tools, pp. 17-26, 2006. [Tassey 2002] G. Tassey: “The economic impacts of inadequate infrastructure for software testing”. Report 02, National Institute for Standards and Technology (NIST), May 2002. [Theguardian 2014] http://www.theguardian.com/technology/2014/aug/14/smartwatches-applejackdaw-android-wear-itine, accessed March 2015. [Thenextweb 2014] http://thenextweb.com/insider/2014/04/28/strategy-analytics-android-tabletshipments-65-8-q4-2014-ios-fell-28-4-windows-secured-5-8/, accessed March 2015. [Theregister 2014] http://www.theregister.co.uk/2014/03/15/windows_desktop_and_ laptop_market_share_dips_below_90_per_cent/, accessed March 2015. [TheVerge 2014A] http://www.theverge.com/2013/7/7/4500074/microsoft-shuts-down-msn-tva-pioneering-set-top-box-that-couldnt, accessed March 2015. [TheVerge 2014B] http://www.theverge.com/2014/9/3/6098227/samsung-gear-s-wearing-themost-powerful-smartwatch-yet, accessed March 2015. [Tizen 2014] https://www.tizen.org/, accessed March 2015. [Thiel 2002] S. Thiel, A. Hein: “Modeling and Using Product Line Variability in Automotive Systems”, IEEE Software vol19, no 4. 2002. [Trew 2004] T. Trew: “What Design Policies Must Testers Demand from Product Line Architects?”, In Proceedings of the International Workshop on Software Product Line Testing (SPLiT 2004), August 2004. [Trew 2006] T. Trew, G. Soepenberg:“Identifying Technical Risks in Third-Party Software for Embedded Products”, Fifth International IEEE Conference on Commercial-off-the-Shelf (COTS)-Based Software Systems, 2006. [Trew 2011] T. Trew, G. Botterweck, B. Nuseibeh: “A Reference Architecture for Consumer Electronics Products and its Application in Requirements Engineering”, Relating Software Requirements and Architectures 2011, pp 203-231, Springer [Trew 2014] J. Trew: http://www.engadget.com/2013/11/11/gamestick-review/, accessed March 2015. [Trustedreviews 2014] http://www.trustedreviews.com/opinions/smartphones-the-new-worldorder_Page-2, accessed March 2015. [Sturgeon 2003]

257

Bibliography [UbuntuTV 2014] [Uhapi 2010] [Ulrich 2004]

http://www.ubuntu.com/tv, accessed March 2015. http://www.uhapi.org/, Accessed July 2010 K.T. Ulrich, S.D. Eppinger:“Product Design and Development”, McGrawHill Inc. New York, 2004 [Underseth 2007] M. Underseth: “Verifying Embedded Software Supply Chains”, EETimesIndia, April 2007. [Unphon 2009] H. Unphon, Y. Dittrich, A. Hubaux: “Taking care of cooperation when evolving socially embedded systems: The plonemeeting case”. In: CHASE’09, collocated with ICSE’09. (May 2009) [Veenendaal 2002] E. van Veenendaal: “The Testing Practitioner”, Tutein Nothenius, 2002 [Veenendaal 2014] E. Veenendaal:”Practical Risk-Based testing: The PRISMA Method”, Improve quality Services , www.improveqs.nl, accessed April 2014. [Vgchartz 2014] http://www.vgchartz.com/article/250982/2013-year-on-year-sales-andmarket-share-update-to-may-18th/, accessed March 2015. [Viragelogic 2010] Virage Logic. http://www.viragelogic.com/ (June 2010) [Vitharana 2003] P. Vitharana: “Risks and challenges of component-based software development”. Commun. ACM 46, 8 (Aug.2003), 67-72. [Voelter 2007] M. Voelter, I. Groher:“Handling Variability in Model Transformations and Generators”, in: 7th OOPSLA Workshop on Domain-Specific Modeling, 2007. [Vogelstein 2008] F. Vogelstein: “The Untold Story: How The iPhone Blew Up The Wireless Industry”, Wired Magazine 16(2), February 2008. [Wedawatta 2011] G.Wedawatta, B. Ingirige, D. Amaratunga: “Case study as a research strategy: Investigating extreme weather resilience of construction SMEs in the UK”, University of Salford, UK, 2011. [Wallnau 2002] K. Wallnau, S. Hissam, R. Seacord: “Building Systems from Commercial Components”, Addison-Wesley, 2002. [Walto 1995] G.H. Walto, J.H. Poore, C.J. Trammell:"Statistical testing of software based on a usage model". Softw: Pract. Exper., 25: 97–108, 1995. [Walton 1998] G. H. Walton, J. H. Poore, C. J. Trammell: “Statistical testing of software based on a usage based model”, Software: Practice and Experience, 25(1): 97-108, January 1998 [Wang 2009] L. Wang, P. Jaring : “Coupling the value chain of mobile telephony – a case study of business model innovation”, VTT, Technical Research Centre of Finland, 2009 [Washingtonpost 2014] http://www.washingtonpost.com/local/education/tablets-proliferate-innations-classrooms-and-take-a-swipe-at-the-status-quo/2014/05/17/, accessed March 2015. [Waters 2011] D. Waters, “Battle over the digital living room”, http://news.bbc.co.uk/2/hi/technology/6250711.stm (2007), Accessed February 2011. [Webb 2005] K. Webb :”Consumer Behaviour”, McGraw Hill, Australia, 2005 [Weber 2002] M. Weber, J. Weisbrod: “Requirements Engineering in Automotive Development – Experiences and Challenges”, IEEE Software. vol. 20, no 1. 2002.

258

Bibliography [Wehlage 2008] [Weiss 1999] [West 2003] [West 2010]

[West 2011]

[Wieringa 2006]

[White 2008]

[Whittaker 2000] [Woit 1994] [Wong 1988] [Worldfinance 2014] [Xia 2005]

[XMBC 2014] [YahooTV 2014] [Yang 1997]

[Ye 2005] [Yin 2003] [Zdnet 2014] [Zigbee 2009] [Zhao 2004]

C. Wehlage: “How the Digital Supply Chain Made Apple No. 1 on the Supply Chain Top 25 “, AMR Research, 2008 D.M. Weiss, C.T.R. Lau: “Software Product-Line Engineering”. Addison Wesley, 1999. J. West: “How Open is Open Enough? Melding Proprietary and open source platform strategies”. Research Policy 32 (7), (2003), pp. 1259–1285. J.West, M. Mace: “Browsing as the killerapp: Explaining the rapid success of Apple’s iPhone”. Telecommunications Policy (2010), doi:10.1016/j.telpol.2009.12.002, Accessed February 2011. J. West, D. Wood: “Tradeoffs of Open Innovation Platform Leadership: The Rise and Fall of Symbian Ltd.”, Stanford Institute for Economic Policy Research (SIEPR), Social Science and Technology Seminar Series 20102011. R.J. Wierimnga, J.M.G. Heerkens: “The Methodological Soundness of Requirements Engineering Papers: A Conceptual Framework and Two Case Studies”, Requirements Engineering Journal, 11: 295-307, 2006. J. White, D. Benavides, D. Schmidt, P. Trinidad, A. Ruiz-Cortés: “Automated Diagnosis of Product-line Configuration Errors in Feature Models”, SPLC 2008. J. A. Whittaker, J. Voas: “Toward a more reliable theory of software reliability”, Computer, 33(12), 36 – 42, 2000. D. Woit: “Operational profile specification, test case generation, and reliability estimation for modules”, Queens University Canada, 1994. K. L. Wong: “Off the bathtub onto the roller-coaster curve”, Proceedings Annual Reliability and Maintainability Symposium, 356-363, 1988 http://www.worldfinance.com/home/featured/giants-of-the-video-gameindustry, accessed March 2015. L.X.X. Xia, W. Lee, C.L. Sing, L. Zhengping: “Performance Metrics Design Framework for Software Focused Supply Chain”, in Proceedings of 3rd IEEE International Conference on Industrial Informatics (INDIN), 2005, pp. 176-180. http://xbmc.org/, accessed March 2015. https://smarttv.yahoo.com/, accessed March 2015. F. Yang, H. Mei, Q. Wu, B. Zhu: “An Approach to Software Development Based on Heterogeneous Component Reuse and its Supporting System”, Science in China Series E: Technological Sciences Volume 40, Number 4, 1997, pp. 405-413. Ye, H. Liu, H.: “Approach to modeling feature variability and dependencies in software product lines”. IEEE proceedings Softw., vol152, June 2005. R.K. Yin: “Case Study Research: Design and Methods” Sage Publications, 2003. http://www.zdnet.com/the-history-of-tablet-computers-a-timeline7000026555/, accessed March 2015. ZigBee Alliance, http://www.zigbee.org, Accessed August 9 2011. W. Zhao, B. R. Bryant, R. R. Raje, M. Auguston, C. C. Burt, A. M. Olson: “Automated Glue/Wrapper Code Generation in Integration of Distributed

259

Bibliography and Heterogeneous Software Components”, in: Proceedings of the 8th IEEE International Enterprise Distributed Object Computing Conference (EDOC2004), IEEE Press, 2004, pp. 275-285.

260

Samenvatting Titel De ontwikkeling van software productlijnen voor consumenten elektronica Subtitel: Het bijhouden van de hoge graad van innovatie Consumenten producten zijn radicaal veranderd de afgelopen 20 jaar. Oorspronkelijk werden deze producten gebruikt voor slechts enkele taken en bestonden grotendeels uit hardware componenten. Tegenwoordig worden deze producten voor een grote diversiteit aan taken gebruikt en wordt de functionaliteit voornamelijk door software bepaald. Bovendien zijn er steeds meer varianten nodig en is er een voortdurende druk op ontwikkelkosten, kwaliteit en levertijd. Deze ontwikkelingen hebben een grote invloed op de manier waarop software product lijnen voor consumenten producten worden ontwikkeld. Dit proefschrift begint in deel I met een beschrijving van de gevolgen van de veranderende structuur van deze industrie. De analyse laat zien dat vanwege een toenemende hoeveelheid software en een hoge graad van innovatie, het niet meer mogelijk is dat een bedrijf de software alleen ontwikkelt. Hierdoor zijn huidige producten samengesteld uit componenten van gespecialiseerde bedrijven. Een model wordt geïntroduceerd, bestaande uit vijf structuren van de industrie, die de transities en de gevolgen voor software architecturen beschrijft. Aangetoond wordt dat een leveringsketen de meest voorkomende structuur is voor de software die in de producten is bevat vanwege een lage graad van modulariteit en omdat de software geoptimaliseerd moet kunnen worden voor de verschillende samenstellingen van hardware componenten. Om het gebruik van applicaties van derde partijen te faciliteren wordt een stabiele applicatie interface ontwikkeld. Op deze manier vindt een transitie plaats naar een ecosysteem benadering. Drie types ecosystemen zijn geïdentificeerd in dit proefschrift: Verticaal geïntegreerde hardware/software platformen, gesloten-bron software platformen en open-bron software platformen. Een eerste stap richting een beslissingsmethode is geïntroduceerd die bepaald welk type ecosysteem het meest geschikt is voor verschillende types producten, vanuit het oogpunt van software ontwikkeling. In deel II van dit proefschrift worden oplossingen gepresenteerd om de software variabiliteit binnen een leveringsketen eenvoudiger te kunnen ontwikkelen. Als een toeleverancier verschillende producten aan diverse klanten levert, leidt dat vaak tot gecombineerde productlijnen. In dit proefschrift wordt een methode beschreven waarin de variabiliteit van de context apart wordt gemodelleerd, waardoor het mogelijk is om gecombineerde productlijnen in een leveringsketen vast te leggen. Een bedrijf in het midden van een leveringsketen, moet vaak componenten van leveranciers integreren en deels geconfigureerde producten leveren. Om aan de variabiliteitseisen van de klanten te voldoen is het soms nodig om vergelijkbare componenten van meerdere leveranciers te gebruiken, wat kan leiden tot overlappende

261

variabiliteitsmodellen. In dit proefschrift wordt een leveranciers-onafhankelijk model geïntroduceerd waarmee de gecombineerde variabiliteit kan worden vastgelegd. Het gebruik van componenten van verschillende leveranciers kan ook leiden tot componenten waarin verschillende mechanismen voor variabiliteit en het verbinden van componenten worden gebruikt. Een model gedreven methode wordt beschreven die het maken van verbindingscomponenten automatiseert. Door het grote aantal variatiepunten worden modellen vaak onhandelbaar, in het bijzonder als er complexe afhankelijkheidsrelaties tussen variatiepunten bestaan, wat voorkomt bij variabiliteitsmodellen in een leveringsketen. In dit proefschrift wordt een tekstuele taal geëvalueerd die handiger in gebruik blijkt te zijn dan de gangbare grafische talen. In deel III van dit proefschrift wordt het probleem geadresseerd om producten met voldoende kwaliteit te leveren. Dit deel richt zich op het testen van software, de laatste fase in een ontwikkelproces. Door een toenemende hoeveelheid fouten in de software, is het niet meer mogelijk om deze allemaal te verwijderen voorafgaand aan de marktintroductie. Een statistisch model wordt gebruikt om de voordelen te bepalen van het toekennen van de hoeveelheid testen op basis van de hoeveelheid van gebruik. Deze analyse toont aan dat deze aanpak voordelen heeft, tenzij een zeer hoge mate van betrouwbaarheid geëist wordt. Een analyse van testen op basis van de verwachte risico’s, toont aan dat een hogere efficiëntie bereikt kan worden als de gebruiksfrequentie als aparte dimensie meegenomen wordt. Om de inspanning voor het testen, die veroorzaakt wordt door de grote hoeveelheid variabiliteit, te verminderen, introduceert dit proefschrift een methode waarin het percentage van gebruik van een ontwikkelartefact vastgelegd wordt. Op basis van dit percentage worden de prioriteiten bepaald voor de verschillende fasen van een testproces en wordt de kans verminderd dat sommige artefacten te veel en andere te weinig getest worden. Als de hoeveelheid testen alleen gebaseerd is op de hoeveelheid gebruik door de eindgebruiker, wordt er onvoldoende aandacht gericht op gebruiksproblemen, wat in toenemende mate de klantenontevredenheid bepaalt voor sterk innovatieve consumentenproducten. In dit proefschrift wordt een raamwerk beschreven die het gebruik beter analyseert. Dit raamwerk bevat de verschillen tussen gebruikers, de omgeving waarin een product gebruikt wordt en de stappen in een gebruiksproces. Met dit raamwerk kan de betrouwbaarheid van een product beter bepaald en daarmee op voorhand verbeterd worden. Dit proefschrift heeft de volgende bijdragen:  Een classificatie van de structuren voor softwareontwikkeling in de consumenten elektronica-industrie is geïntroduceerd, een classificatie van ecosystemen en daarbij de uitdagingen voor het ontwikkelen van software architecturen.  Oplossing zijn beschreven voor het eenvoudiger ontwikkelen van variabiliteit in een leveringsketen en het omgaan met grote en complexe variabiliteitsmodellen.  Methodes zijn geïntroduceerd die de test efficiëntie verbeteren voor productlijnen met veel variabiliteit en voor sterk innovatieve consumenten producten.

262

Curriculum Vitae

Herman Hartmann received his M.Sc. degree in applied Mathematics at the University of Groningen in 1990. He started working at Philips Numerical Control in 1990 as a software engineer and developed motion control algorithms and robotics software. In 1992 he joined Philips CFT and worked as a project leader and consultant on the introduction of finite element analysis tools in several organizational units of Philips. Later he introduced product lifecycle management tools, knowledge management tools and collaborative engineering in Philips. During this period he also deployed CMMI for software development in the Computer Aided Development group and later also in other organizational units of Philips. He continued his career in 2000, as a researcher and consultant on software reliability. During this period he worked on the reduction of field call rates for consumer electronics and did research on software reliability and soft reliability for consumer electronics and healthcare products. In 2006 he joined NXP Semiconductors and did research on software product lines, model driven development, variability management and software reliability for systems on chips for mobile phones, digital televisions and set-top boxes. Currently he is a software architect in the NXP R&D IT department and focuses on design data management and the systematic reuse of software and hardware intellectual property. His research interests include software development processes, software product line engineering, ecosystems, software architectures and product quality.

263

The cover of this thesis is based on Composition No. IV / Compositie 6, 1914 by Piet Mondriaan.

264