For many quality attributes, different customer needs can be addressed with the ... To study the motivation to vary performance in a software product line, and the ...
Performance variability in software product lines: A case study in the telecommunication domain Varvana Myllärniemi, Aalto University, Finland Juha Savolainen, Danfoss Power Electronics A/S, Denmark Tomi Männistö, University of Helsinki, Finland
Motivation • Software product lines (SPLs) do not often intentionally vary quality attributes, such as performance, security, and reliability – For many quality attributes, different customer needs can be addressed with the same product variant • E.g., a product with 500ms response time will satisfy two customer segments that require 500ms and 750ms
– All explicitly managed variability adds to the complexity and cost • Quality attribute variability may imply costly architecture-wide variation
• There is little direct empirical evidence on industrial cases of quality attribute variability in SPLs (Myllärniemi et al., 2012) • Contribution: to present a case study (Yin, 1994) of performance variability within an industrial telecom SPL
Research problem • To study the motivation to vary performance in a software product line, and the realization of performance variability in the architecture design and design decisions • RQ1: Which characteristic of performance is decided to be varying? • RQ2: Why is performance decided to be varied? • RQ3: What is the strategy to realize performance variability within the product line architecture? • RQ4: Why is the strategy chosen?
The case description • The case study company: NSN • The case study SPL: IP-BTS – Base stations – 3rd generation mobile phone technology, All-IP RAN – Customers: telecom operators
Research method • A descriptive case study (Yin, 1994) – Thus, studies a contemporary phenomenon in a real-life context
• Research process: sampling, data collection, data analysis, validation • Post mortem case – The case product line was designed approx. 10 years ago, but discontinued before it was released • The promise of All-IP radio access networks did not take off
– However, the results in regard to the unit of analysis (performance variability) are generalizable beyond this single case
Results to RQ1: Which characteristic of performance is decided to be varying? • Capacity – Typically, defined in terms of maximum throughput or workload – In the case study, defined as the maximum number of phone calls a base station can serve at a time
• Operators select the capacity of the base station to match the estimated usage – Thus, differences in capacity between the base stations
Results to RQ2: Why is performance decided to be varied? • From the customer point of view: differences in needs – Differences in the initial capacity planning – Evolution of the needs and long-lived products • Eventually, the growing usage exceeds the capacity of a single installed product • Rebinding variability: base station capacity needs to be upgraded
• From the vendor point of view: motivation to invest – Differentiating variability • Different capacity levels are of different business value to the customers (and the business value could be estimated via ROI) • The customers are willing to pay premium price for premium capacity (especially if it is reboundable later on)
Results to RQ3: What is the strategy to realize performance variability within the product line architecture? A combination of: • Hardware variability strategy – Different installed hardware yields different capacity
• Software variability strategy – Licenses to control capacity – Downgrading to vary capacity with software: restricting capacity-related resources, e.g., channel elements • (But even this was dependent on hardware)
Results to RQ4: Why is the strategy chosen? Factors that motivate the decisions between the software and hardware variability strategy BOM (Bill of Material)
++ HW strategy
Ease of design and implementation
+ HW strategy + HW-dependent SW strategy
Ease of testing
+ HW strategy + HW-dependent SW strategy
Cost of variant rebinding
++ SW strategy
Cost of supporting variant rebinding
+ SW strategy
The time it takes to rebind a variant
+ SW strategy
Flexibility of variant rebinding
+ SW strategy
Lessons learned: motivation to vary • Also from the point of generalizing to other domains and quality attributes… • The evolution of the customer needs may motivate to vary performance – With short-lived products, this is not that relevant – Do needs / expectations evolve over time for all quality attributes?
• Price differentiation is a motivator to the vendor – But is easier when the quality attribute is tied to the mechanisms that bring revenue to the customer, and the customer is able to recognize this link in precise terms – Compare to many consumer product domains: the customer may not even understand the exact differences between the quality variants
Lessons learned: realization with software and hardware variability strategy • BOM typically favours the use of HW strategy – Perhaps there needs to be a specific reason to utilize SW strategy
• The SW strategy can be kept simple by downgrading • In contrast to these findings, the existing literature often focuses on feature impact management – Quality is varied by varying features and managing the impact of features to the overall product quality – Mostly about software features, often relies on having functionally similar but quality-wise different alternative features – Is more suited for optimization, trade-offs, or when the quality variation is not intentional
• In the case, the performance was guaranteed, price differentiated, and reboundable without affecting other functionality -> a simple and separate approach suited better
Conclusions • A descriptive, post mortem case study of performance variability in the domain of mobile base stations • The motivation to vary stemmed from price differentiation and from the evolution of customer needs • The realization strategy was a combination of software and hardware variability strategy – Rebinding flexibility posed a speficic need to use SW strategy
• But further empirical research, in particular case studies, would shed more light to the nature of quality attribute variability in industrial software product lines
Thank you!
… The slides will be available @ researchgate.net
