Towards A Software Failure Cost Impact Model For ... - PROMISE data

2 downloads 190 Views 2MB Size Report
Towards A Software Failure Cost Impact Model for the Customer. 2. Cost. Usage. Behavior. Failure. Research Question. What is the cost impact of software.
Ralf Gitzel (ABB Corporate Research) Simone Krug (University of Mannheim, Germany) Manuel Brhel (University of Mannheim, Germany)

Research Question

What is the cost impact of software failures on the customer?

PROMISE 2010

Failure

Usage Behavior

Cost

Towards A Software Failure Cost Impact Model for the Customer

2

Software Failure Impact Developer

Fault (Bug)

Error

Failures Here:

Customer

PROMISE 2010

Cost

• loss of bookmarks • freeze • program crash

Towards A Software Failure Cost Impact Model for the Customer

3

Our Data Source Closed Source

Open Source

 Bugzilla : Firefox bug repository

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

4

Identifying Relevant Entries 78 000 Bugs 7.5% Relevant for the user

Blocker / Critical Bugs

198 Bug Reports

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

5

Modeling Customers’ Failure Cost 1.

Identify cost drivers

2.

Model failure occurrence by effect

3.

Model user behavior

4.

Calculate expected cost for the customer

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

6

1. Identify Cost Drivers Direct impact Defective goods

Material for Repair

Resolution

Indirect impact

Etc.

Application / business specific PROMISE 2010

User dissatisfaction

Legal contracts

Etc.

Hard to capture

Towards A Software Failure Cost Impact Model for the Customer

7

Main Cost Drivers

Rework

(Usage Preferences) (Capability of the user)

PROMISE 2010

Solution Search Resolution Time Labor Time Lost

Towards A Software Failure Cost Impact Model for the Customer

8

2. Model failure occurrence by effect

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

9

Fault Trees Failure Impact

EFFECT (= FAILURE)

OR Gate

AND Gate

REPORTED BUG (= FAULT)

TECHNICAL CAUSE

ROOT CAUSE

PROMISE 2010

VERSION

...

USER ACTION

...

Events not further elaborated

Towards A Software Failure Cost Impact Model for the Customer

10

Firefox Failure Analysis Freeze (Halt)

Crash

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

11

3. Model User Behavior 

A User Profile contains: oFeatures of a system and their purpose oFrequency of execution



Each User Profile corresponds to a job specification

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

12

User Behavior Example

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

13

Time Lost Per Week Expected Frequency

Time Lost

PROMISE 2010

Length

Expected Resolution Time

Estimated Failure Probability

Towards A Software Failure Cost Impact Model for the Customer

14

Expected Cost for the Customer Expected Cost for the Customer

Repair Cost

PROMISE 2010

Staff Expenditures

Towards A Software Failure Cost Impact Model for the Customer

15

4. Calculate expected cost for the customer

User

T

Procurement Worker

12 min

Secretary

8.3 min

Marketing

9.5 min

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

16

Assumptions and Limitations  Producer-Consumer

gap leads to limited empirical data on failureinduced cost

 Cost

drivers are very domain specific

 Legal

PROMISE 2010

implications

Towards A Software Failure Cost Impact Model for the Customer

17

Lessons Learned and Future Work Most high impact bugs are eliminated prior to release However, minor failure effects accumulate over time!

Include software defect prediction techniques Industrial case study

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

18

Thank you for your attention! Questions / Comments?

PROMISE 2010

Towards A Software Failure Cost Impact Model for the Customer

19

Suggest Documents