Measurement in Software Engineering

Measurement in Software Engineering: Engineering Practices or Placebos? Alain Abran Irbid, Jordan April 3, 2018

© Copyrights Abran 2018

Presenter background - Alain Abran 20 years

+ 20 years

45 PhD  Development  Maintenance  Process Improvement © Copyrights Abran 2018

ISO: 19761, 9126, 25000, 15939, 14143, 19759

2

Agenda 1. Software Engineering & Measurement in the SWEBOK 2. Measurement in Civilizations – The Masters from the Past 3. Software Metrics and … placebos! 4. The future for software measurement.


3


 What is Engineering?  “The application of a systematic, disciplined, quantifiable approach to structures, machines, products, systems or processes.” (IEEE/ISO/IEC 24765, 2010)



What is software engineering? 

“The application of a systematic, disciplined, quantifiable approach to the development, operation, and maintenance of software; that is, the application of engineering to software.” (IEEE/ISO/IEC 24765, 2010)

© Copyrights 5 Abran 2018

Guide to the Software Engineering Body of Knowledge 2004 Version

Software Requirements

Software Design

Software Construction

Software Requirements Fundamentals

Software Design Fundamentals

Basic Concepts of Construction

Requirements Process

Key Issues in Software Design

Managing Construction

Requirements Elicitation

Software Structure and Architecture

Practical Considerations

Requirements Analysis

Software Design Quality Analysis and Evaluation

Test Related Measures

Requirements Specification

Software Design Notations

Test Process

Requirements Validation

Software Design Strategies and Methods

Software Testing

Sofware Testing Fundamentals

Test Levels

Software Maintenance

Software Maintenance Fundamentals Key Issues in Software Maintenance Maintenance Process

Test Techniques



Techniques for Maintenance

Guide to the Software Engineering Body of Knowledge (2004 Version)

Software Configuration Management

Software Engineering Management

Software Configuration Management Fundamentals

Initiation and Scope Definition

Keys Issues in SCM Software Configuration Control Software Configuration Status Accounting

Process Implementation and Change

Software Project Planning

Process Definition

Software Project Enactment

Process Assessment

Software Engineering Tools and Methods

Software Tools Software Requirements Tools Software Design Tools Software Construction Tools Software Testing Tools

Software Quality

Related Disciplines

Software Quality Fundamentals

Computer Engineering

Software Quality Management Processes

Computer Science


Management

Software Maintenance Tools

Review and Evaluation

Process and Product Measurement

Software Engineering Process Tools

Closure

Software Engineering Management Tools Infrastructure Support Tools

SW Engineering Measurement

Mathematics

Software Quality Tools Software Configuration Management Tools

Software Configuration Auditing Software Release Management and Delivery

Software Engineering Process

Miscellaneous Tool Issues

Project management Quality management

Software Ergonomics

Software Methods Heuristic Methods

Systems engineering

Formal Methods Prototyping Methods Miscellaneous Method Issues


7

Software Requirements

Software Requirements Fundamentals

Requirements Process

Requirements Elicitation

Requirements Analysis

Requirements Specification

Requirements Validation

Practical Consideration

Definition of Software Requirement

Process Models

Requirements Sources

Requirements Classification

System Definition Document

Requirements Reviews

Iterative Nature of Requirements Process

Product and Process Requirements

Process Actors

Elicitation Techniques

Conceptual Modeling

Systems Requirements Specification

Prototyping

Change Management

Functional and Non-functional Requirements

Process Support and Management

Architectural Design and Requirements Allocation

Software Requirements Specification

Model Validation

Requirements Attributes

Emergent Properties

Process Quality and Improvement

Requirements Negotiation

Acceptance Tests

Requirements Tracing

Quantifiable Requirements

Measuring Requirements

System Requirements and Software Requirements


Selected Usage Examples • Pyster, Lasfer, Turner, Bernstein, Henry, Master’s Degrees in Software Engineering: An Analysis of 28 University Programs, , IEEE Software, Vol. 26, no. 5, Sept.-Oct. 2009 pp.94 – 101. • “Radial plots of SWEBOK coverage in 20 of the 28 programs. The circumferential axes correspond to the SWEBOK knowledge areas.”


9



10

Quantifying ≠ Measuring Maths ≠ Engineering

Measurement scale types and admissible transformations


‘Measurement and quantification

are not the same: ISO 15939 and ISO 9126’, Abran, Desharnais, Cuadrado, Journal of Software: Evolution and Process, Wiley, 2012, Vol. 24: pp. 585–601.


Metrology


Metrology



Lack of universally accepted references: Impact 1,000 feet variations

Across cities close to Geneva:  314 to 350 meters Across European countries:  283 o 354 meters


16

Masters from the Past The ‘meter’ as the universal étalon: A product of the French Revolution (1789)!

- A consensual definition: A fraction of the Meridian at the Equator - A practical étalon: it took 7 years to measure the Meridian at the Equator..! - After 200 years, there are still some countries which have not adopted it..


17

Key Lessons from the Masters 1. Evolutionary societal & consensual understanding of measurement concepts •

Perfection is not expected first: the search for precision – when necessary only!

2. Development of Measuring Instruments 3. Establishment of measurement ‘etalons’ 4. Specialized measurement training & certification: • • • • •

Land surveyors Accountants, Engineers, Testing labs, etc.


18

Measurement is a Technology • Technology is defined as: • the set of methods & materials used to achieve industrial or commercial objectives.

• Not limited to materials alone: • It also includes processes & the knowledge related to them, referred to as “know-how”.


19

Measurement Masters 1. 2. 3. 4. 5. 6. 7. 8. 9.

National metrology Traceability & uncertainties Legal units of measurement Transparency of metrological information. Legal metrology Application of the Law Offences Responsibilities & duties Conformity assessment procedures


20

Time & Distance


Egyptian Measurement & Tools

New Kingdom Dynasty XVIII-XX (1550-1070 B.C.)

New Kingdom

Architect Kha

Amenhotec II


22

Architect Kha: Measurement Tools Golden Etalon - New Kingdom Dynasty XVIII-XX (1550-1070 B.C.)


23

Architect Kha & Measurement Tools Folding étalon - New Kingdom Dynasty XVIII-XX (1550-1070 B.C.)

© Fondazione Museo Antichità Egizie di Torino – used with permission © Copyrights Abran 2018

24

Software Measurement The dominant view: • Measurement is fine for physical products • Software is an intellectual product and …..

cannot be measured!


Measurement of Time Evolution of Perceptions of Time & Masurement Concepts & Tools


26

The communal local time

Sun shadow The house sun dial


27

Communal Time – XVIII Century - the ‘horloge’

© ©Copyrights Copyrights Abran Abran 2012 2018

28

Personal Time: Watches & Minituarization

Mechanical

Atomic

Quartz


29

Organizational Reference Context

Data Analysis

Decision Criteria: Target Values Evaluation Scale

Information Product

Reference Values for contexts

Indicator

Standard Model

Generic/ Accepted Model of Information Needs

Interpretation

Formal/Informal Individual Relationships With (Base/Derived) Measures Decision Criteria

Analysis Model

Algorithm: Implied criteria




31

What are our base measurement units in Software?


Enduring Software Measurement Myths: • Software is different: • It is an intellectual product & it is not material

• Software metrics = an algorithm • Foundation = measurement theory! (limited to maths)

• Innovation in metrics = again, another proposal... Impact - Examples: • +100 proposals on software complexity • The vast majority of software metrics from academia not used in industry..... © Copyrights Abran 2018

33

A typical new software metrics: • A modified algorithm • Additions of conditions to the algorithms • As many metrics as can be extracted automatically from files (codes, models,..)

• Typical empirical analyses: • No hypothesis to be tested! • Highest correlation with something else! • Little verification of the relevance to the concept to be measured!


34

Software Industry & Measurement Out of the +1,000 of software metrics proposed in the literature, which ones have reached some maturity with respect to industry needs?

• How do you recognize maturity in measurement? • Of the +100 Software Quality metrics, which ones would you consider mature?


35

Example 1: Halstead: ‘Software Science’ Metrics A look at of the design of Halstead’s metrics, including: entities and attributes measured, scale types measurement units A number of authors have adopted the structure of the Halstead’s metrics as the basis for their own proposed measures: • Function Points: based on the Halstead’s ‘volume’ metrics [Albrecht 1984] • UseCase Points, • ….


Objectives of Halstead’s metrics: A) Measure the attributes of a program: • • • •

length vocabulary volume level, difficulty, level estimator, intelligence content.

B) Measure something quite different, that is: • programming effort • required programming time  the measurement of entities of the development process, rather than attributes of the software


Halstead’s Metrics: • A computer program: an implementation of an algorithm considered to be a collection of tokens classified as either operators or operands • All Halstead’s metrics are functions of the counts of: • • • •

n1: Number of distinct operators. n2: Number of distinct operands. N1: Total number of occurrences of operators. N2: Total number of occurrences of operands.


Halstead’s Metrics

• The length (N) of a program P is:

• The vocabulary (n) of a program P is:

•

The volume (V) of a program P is defined as: a) the size of any implementation of any algorithm b) the number of mental comparisons required to generate a program • • • •

n1: Number of distinct operators. n2: Number of distinct operands. N1: Total number of occurrences of operators. N2: Total number of occurrences of operands.


Halstead’s Metrics • Programming effort (E): total number of elementary mental discriminations required to generate a program:


Halstead’s Metrics Scale types issues • Identification of the corresponding scale type is highly challenging: • For instance, Fenton [1997] notes that: • program Volume is of the ratio scale type, • program Level is of the ordinal scale type.

• By contrast, Zuse [1998] maintains that: • Length is of the ratio scale type, and • Vocabulary, Volume, Effort are of the ordinal scale type.

• Moreover, it is not clear to which scale other metrics belong!


Halstead’s Metrics Issues in measurement units • Program length (N) is calculated by adding the total number of occurrences of operators to the total number of occurrences of operands. • However, since their units are different, operators and operands cannot be directly added


Halstead’s Metrics • Program volume (V) has been interpreted with 2 different units of measurement: • ‘the number of bits required to code the program’ [Hamer 82] and • ‘the number of mental comparisons needed to write the program’ [Menzies 02] on the left-hand side of the equation:

• Thus, there is no relationship between: • the measurement unit on the left-hand side, and • the measurement units on the right-hand side of this equation  (occurences of token x distinct tokens).


Halstead’s Metrics •

Effort (E) required to generate a program:

E

elementary mental discrimina tions

occurrences of operands

distinct operators



n1

N2

N 2

potential operators

occurrences of tokens

distinct operands

log 2 n

distinct tokens

.

n2

 there is no relationship between the units of measurement on the lefthand and right-hand sides of the equation.




E



distinct operators



n1

N2

N 2

potential operators


distinct operands

n2

log 2 n

distinct tokens

.

 No relationship between the units of measurement on the left-hand and right-hand sides of the equation.




E



distinct operators



n1

N2

N 2

potential operators


distinct operands

n2

log 2 n

distinct tokens

.

 No relationship between the units of measurement on the left-hand and right-hand sides of the equation.


Example 2: Use Case Points Use Cases: Developed by Jacobson [1987] while at Ericsson in the 1960s: • A use case is a technique for capturing the requirements of a software system. • Each use case is composed of a number of scenarios, written in an informal manner in the language of the business domain to describe the interactions between the actors and a system.

In 1993, Gustav Karner proposed the UCP measurement method, an adaptation of Function Points to size the Use Cases


Use Case Points Design of UCP Takes 3 aspects of a software project into account: 1. Use cases - UUCP 2. Technical qualities - TCF 3. Development resources – Environment Factors - EF 1) Unadjusted Use Case Points – UUCP • Each actor & each use case is: • classified at a complexity level (simple, average, or complex) & • assigned a corresponding weight  from 1 to 3 points for the actors, and  from 5 to 15 for the use cases.


Use Case Points 2) Technical qualities - TCF • 13 technical qualities, each with a specific weight, combined into a single factor. • each technical quality, evaluated on a scale from 0 to 5 (where 0 is ‘not applicable’ and 5 is ‘essential’).

• these weights based on the constants & weights of the 1979 Function Points Value Adjustment Factors. 3) Development resources

• 8 Environment Factors (EF): • somebody must assess each factor and classify it on a scale from 0 to 5 • (0 meaning ‘very weak’; 5 meaning ‘very strong’). • The selection of the weights is balanced such that a value of 3 for each factor will produce an EF of 1. 4) Total UCP size • The final size in terms of the number of UCP is the product of these three components: Total UCP size = UUCP x TCTP x EF


Use Case Points The measurand: the entities and the attributes measured 5 distinct types of entities: 1. Actor 2. Use case 3. Specification of requirements (functional and non functional) 4. Development team (a project-related entity) 5. Programming language (a project-related entity) 11 different types of attributes are taken into account and measured by the UCP method described in [Carroll 1993] – see Figure 1 and Table 1.


UCP Design

Table 1: Entities, Attributes, and Measurement Rules

Entity

Attribute

Measurement rule

Complexity (of actor)

The type of complexity (simple, average, or complex) of the interaction between the actor and the system

Use case

Complexity (of use case)

The type of complexity (simple, average, or complex) measured in the number of transactions

Specification of requirements

Relevance of the technical quality requirements

The level of relevance (from 0 to 5) of each of the 13 known non-functional qualities

Stability of the requirements

The level of stability (from 0 to 5) of the functional and non-functional requirements

Familiarity with the methodology

The level (from 0 to 5) of skills and knowledge of the development methodology in use for the project.

Part-time status

The level (from 0 to 5) of part-time staff on the team

Analysis capability

The level (from 0 to 5) of analysis capabilities of the development team with respect to project needs

Application experience

The level (from 0 to 5) of team experience with the application domain of the system

Object-oriented experience

The level (from 0 to 5) of team experience with objectoriented design

Motivation

The level (from 0 to 5) of team motivation

Difficulty

The level (from 0 to 5) of programming difficulty

Actor

Development team

Programming language

Development Language -Difficulty Development Team -Familiarity with methodology -Part-time status -Capacity for analysis -Experience with application -Object-Oriented experience -Motivation

System under development

Requirements specification


-Technical constraints -Stability of requirements

Use Case -Complexity

Actor -Complexity

Figure 1: The 5 types of entities & 11 types of attributes measured in UCP

Use Case Points Inadmissible numerical operations The formula to calculate the number of UCP is: UCP = UUCP * TCF * EF • UUCP value is calculated by multiplying the number of use cases & actors of each type of complexity by their weights. • In doing so, the ordinal-scale values (categories of complexity) are transformed into interval-scale values. • These values on an interval-scale type are next multiplied, resulting in a value on a ratio scale, another algebraically inadmissible mathematical operation.

• TCF and EF: technical & resource factors are also all evaluated through a categorization process with ‘labels’ from 0 to 5: • Do not represent numerical values on a ratio scale, but merely a category on an ordinal scale type: • that is, they are merely ordered labels and not numbers.


Use Case Points The UCP measurement method:  assigns numbers to several entities (actors, use cases, requirements, etc.) and attributes (complexity, difficulty, etc.).  calculations are based on several algebraically inadmissible scale type transformations.  Combining many concepts at once, as this method does, makes it a challenge to figure out what the end-result is from a measurement perspective.


Example 3: Story Points in Agile Requirements described into Story Cards 1. Group Discussion & Planning Poker cards 1. Fibonacci suite to size a Story Card

2. A ‘dimensionless number’ …. • In practice, used as ‘estimated effort’ for estimation purposes and for calculating ‘velocity’ ………


Criteria for a ‘good’ measure • Repeatability: - different individuals, in different contexts, at different times, & following the same measurement procedures will obtain the same measurement results. • Measurement results: - obtained with minimal judgment. - results auditable.


55

Testing the design of Story Points: • Repeatability: - different individuals, in different contexts, at different times, & following the same measurement procedures will NOT obtain the same measurement results. • Measurement results: - obtained with minimal judgment. - results auditable.


56

Story Points


Story Points

• Unaccountability…..!




59

Software Measurement & Industry For industry, measurement has little to do with maths! • Measurement is not maths, but a technology with considerable consensual knowledge on: • the concepts to be measured, • credible references for measurement, & • expected measurement errors


60

Software Industry & Measurement How do you recognize maturity in measurement? • Measurement Maturity = Standardization Which ‘software metrics’ are recognized as ISO standards?


61

Functional Size Measurement is an exception in software meaurement Out of the +1,000 of software metrics proposed in the literature, only 5 Functional Size Measurement (FSM) methods have been adopted as ISO standards! • What has been done differently?


62

Software Measurement ISO 14143 meta-standard on Functional Size Measurement: Part 1: Definitions & Concepts Part 2: Conformity Evaluation of FSM methods Part 3: Verification Guide of FSM methods Part 4: Reference Model (but only samples of FUR!) Part 5: Determination of Functional Domains for FSM methods Part 6: Guide for the use of ISO 14143!


63

Software Functional Size Measurement 5 distinct methods • • • • •

ISO 20926 : IFPUG ISO 24570 : NESMA ISO 20968 : MRKII ISO 29881 : FISMA ISO 19761 : COSMIC

What is common in their design & design process?


64

ISO ‘FSM’ Standard 14143

1st generation

MkII FPA Full FP’s v.1 Feature Points

Allan Albrecht FPA

2nd generation

MkII FPA v.1.3

3-D FP’s

COSMIC FFP v. 2.0

IFPUG 4.1

COSMIC v. 4.0.1

IFPUG 4.3

IFPUG 4.0 1980

1985

1990

1995

2000

2017 © Copyrights Abran 2018

65

1st Generation FSM Key Contributions User Groups built the essential measurement infrastructure: • Procedural Measurement Manuals • Central technical authority: • Measurement Practice Committee • Certification criteria & exams • Case studies for reference materials

• Outside international recognition: ISO


66

1st Generation Key Weaknesses Weights-like methods: • A structure with weights to integrate multi variables: • An end number with a symbol (FP) but without a well defined measurement unit!

• A number of structural weaknesses pointed out in the literature over the past 30 years


67

1st Generation of Function Points = 68

Complexity tables & Weights

Inputs - Matrix

Output & Enquiries – Shared Matrix


Transactions: weights in FP (Function Points)

Function Points weights = Step functions

Key limitations: - Only 3 values - Limited ranges (min,max) - No single measurement unit of 1 FP!

6 FP

3 FP

4 FP 3-step size range for the IFPUG External Input Transactions


69

st 1 Generation

of Function Points

Function Points (FP)

6 FP

3 FP

4 FP =? © Copyrights Abran 2018

70

The types of scale in the measurement of the Data in FP


Weaknesses of the Function Points Design The types of scale in the measurement of the Data in FP


Input

Step

Output

Available Software Documentation

V1 Identify appropriate Characteristics

One / Several Characteristics

Definition of each of the 14 Characteristics (GSC)

V2 Assign a score

Ordered Ranking for each GSC

V3 Assign a Degree of Influence

Assigned Degree of Influence

Characteristics Criteria Degree of Influence = 0.1 V4 Add All Formula for the Degree Influence 0.65+0.01*N

V5 Adjustment Algorithm

Total Degree of Influence (N) Value Adjustment Factor

Figure: Model of the Value Adjustment Factor


Value Adjustment Factor (VAF): 14 General System Characteristics (GSC) of the software •

Rules for classifying the software being measured into one of the 5 classifications

 The values of 0, 1, 2, 3, 4, and 5 for a characteristic represent ordered labels •

The same degree of influence value of 0.1 is assigned to each of the irregular ordered intervals within a GSC: 

•

Such an equality (i.e. impact = 0.1 for each interval) across irregular intervals has no theoretically verified justification and is not based on empirical evidence.

Every thing is then added together…..!!!


Category

IFUG Updated SNAP Points S- Software N- Non-functional A- Assessment P- Process

Data Entry Validation Logical operations Mathematical operations Data formatting Internal data movements Functionality by data config. UI Changes Help methods Multiple input methods Multiple output methods Multiple platforms Database technologies System configuration Batch processing System critical (real-time) Component based software Design complexity

Sub-concepts for the classification Nesting level complexity

2,3,4 * number of DETs

Control flow complexity




Transformation complexity Internal boundaries crossed DET transferred Complexity

2,3,5 * DETs 5* (# of internal boundaries crossed)+2*(#DETs) 3,4,6 * Number of records

UI types complexity Help types Media types

2,-,4 * DETs 1,2,3 * number of Help items 3,4,6 * number of controls

Media types

3,4,6 * number of controls

No. of platforms to operate

8 * Number of platforms

Level & type of normalization of the physical schema

1,3,4,5,7 * levels of normalization SP=(middleware config.)+2*(# backend config.) +3*(# interface config.) 2*(number of batches or transactions) 5,10,15 * number of critical transactions 4,8 * number of unique components 8,16,24 *# of COTS applications + 12,24,36 *#nonCOTS applications

Number of batches or transactions - Type of transactions - No. critical trans. Type of components (In-house reuse or 3rd party component Interface complexity


SNAP weights basis

Category

IFUG Updated SNAP Points

Data Entry Validation Logical operations Mathematical operations Data formatting Internal data movements Functionality by data config. UI Changes Help methods Multiple input methods Multiple output methods Multiple platforms Database technologies System configuration Batch processing System critical (real-time) Component based software Design complexity

Sub-concepts for the classification Nesting level complexity






Transformation complexity Internal boundaries crossed DET transferred Complexity

2,3,5 * DETs 5* (# of internal boundaries crossed)+2*(#DETs) 3,4,6 * Number of records

UI types complexity Help types Media types

2,-,4 * DETs 1,2,3 * number of Help items 3,4,6 * number of controls

Media types

3,4,6 * number of controls

No. of platforms to operate

8 * Number of platforms

Level & type of normalization of the physical schema

1,3,4,5,7 * levels of normalization SP=(middleware config.)+2*(# backend config.) +3*(# interface config.) 2*(number of batches or transactions) 5,10,15 * number of critical transactions 4,8 * number of unique components 8,16,24 *# of COTS applications + 12,24,36 *#nonCOTS applications

Number of batches or transactions - Type of transactions - No. critical trans. Type of components (In-house reuse or 3rd party component Interface complexity


SNAP weights basis

Summary of weaknesses in the measurement designs of 1st Generation FP • Several different scale types are used in the various steps. • On a strictly mathematical basis, the results of many of the steps and sub-steps of the measurement designs are based on inappropriate use of mathematical properties of corresponding scale types.

• The consequences: • It fails primary school maths!


2nd Generation criteria Correction of all known structural weaknesses: • adopting a clear & unique ‘measurement unit’: • ‘a data movement of a single data group’ to which a size unit of 1 is assigned, together with the 1CFP as its measurement symbol.

• Making sure that it did not included any invalid mathematical operations.


78

nd 2

Generation of Function Points

Every software is different, but ….. what is common across all software: In different types of software? In very small software? In very large software? In distinct software domains? In various countries?


79

2nd Generation: COSMIC Function Points All software does this: Boundary

Functional Users types: Entries 1. 2. 3.

Humans Hardware devices Other software

Exits Reads

The ‘Data Movement of 1 data group’ is the unit of measurement: 1 CFP (1 CFP = 1 COSMIC Function Point)


COSMIC view of software

Software being measured Writes Persistent storage

80

nd 2

Generation with COSMIC No abitrary max 11 10 9 Largest observed 8

COSMIC Function Points (CFP)

7 6 5

3

1

2

4

functional processes: In avionics >100 CFP In banking > 70 CFP

A single CFP exists & is well defined


81

Example 1: Intruder Alarm System - Requirements Input devices (functional users)

Software Boundary

Keypad Power voltage detector

Output devices (functional users) External alarm

The embedded alarm software

Front door sensor

Internal alarm 2 x LED’s

Movement detectors

Persistent storage


82

Functional process: Possible intruder detected. Triggering event: Door opens whilst alarm system is activated.

Input devices (functional users)

Software Boundary

Keypad Power voltage detector

Output devices (functional users) External alarm

The embedded alarm software

Front door sensor

Internal alarm

Data Movem ent Entry Read Exit Exit Exit Entry

Functional User

-

Front-door sensor / Occupant Green LED Red LED Internal siren Keypad

2 x LED’s

Movement detectors

Persistent storage

* * Exit Exit

Green LED Red LED Internal siren External siren

Exit

External siren

Data Group

‘Door open’ message (triggering Entry) PIN (from persistent storage) Switch ‘off’ command Switch ‘on’ command Start noise command PIN (If the wrong code is entered, the user may enter the PIN two more times but the process is always the same so it is only measured once.) Switch ‘on’ command (after successful entry of PIN) Switch ‘off’ command Stop noise command (after successful entry of PIN) Start noise command (after three unsuccessful PIN entries, or if the PIN is not entered in time) Stop noise command (after 20 minutes, a legal requirement)

Size = 9 CFP (COSMIC Function Points)


83

ISO/IEC 19761:2002 COSMIC- A functional size measurement method Method Overview

Measurement Manual

FMS Measurement Instrumentation

Advanced & Related Topics

Beginners Practitioners Experienced Practitioners Guidelines

Case Studies

• Business V1.0

• Business

• Real-time

• Real-time

DOMAIN-SPECIFIC SUPPORT DOCUMENTS


84

Guidelines by Application Domains • Business applications • Real-time software • Data Warehouse software • SOA software (SOA: Service Oriented Architecture) • Mobile apps • Agile Development

The COSMIC Functional Size Measurement Method Version 4.0.1

Guideline for Sizing Business Application Software

VERSION 1.3a Febuary 2016


85

COSMIC Case Studies • Real-time: • Rice cooker • Automatic line switching • Valve control

• Business: • • • •

Course registration (distributed) Restaurant management (web & mobile phone) Banking web advice module Car hire (existing legacy app.)


86

COSMIC - at any level of software requirements App 1

Application Layer

App ‘n’

App 2

Middleware Layer (Utilities, etc) Database Management System Layer

DBMS 1

DBMS 2

Operating System Layer

Hardware

Keyboard Driver

Screen Driver

Print Driver

Disk Driver

Keyboard

VDU Screen

Printer

Hard Disk Drive


Central Processor

87

COSMIC data from Industry COSMIC method in Automotive embedded software By: Sophie Stern

Renault


88

Data from Renault - 2012

© Copyrights Renault 2012 © Copyrights Abran 2018

89

Renault: Estimation & Negociations

© Copyrights Renault 2012 © Copyrights Abran 2018

90

Renault - Remarkable cost estimation accuracy from its ECU software specifications Cost vs size (CFP)

© Copyrights Renault 2012

Memory size vs software size (CFP) © Copyrights Abran 2018

91

AUTOMATION ACCURACY REACHED WITH COSMIC Steer-by-wire case study Steer-by-Wire Runnable

Steer_Run_Acquisition Steer_Run_Sensor Steer_Run_Command Steer_InterECU_Wheel Steer_Run_Actuator Wheel_Run_Acquistion Wheel_Run_Sensor Wheel_Run_Command Wheel_InterECU_Steer Wheel _Run_Actuator Total

Functional size obtained by the manual FSM procedure (CFP)

Functional size obtained by the automated FSM procedure (CFP)

3 4 7 3 2 3 4 7 3 2 38

3 4 7 3 2 3 4 7 3 2 38

Automation in Industry Total

Total Size

Total Size

Difference

Number of

obtained

obtained

(%)

Models

manually

using the

(CFP)

prototype

Accuracy

tool (CFP) 76

fault-

1,729

1,739

Less than 1%

>99%

1,758

1,791

1.8%

>98%

free models All

77

models

Ref. : Hassan Soubra, Alain Abran, A. R. Cherif, ‘Verifying the Accuracy of Automation Tools for the Measurement of Software with COSMIC – ISO 19761 including an AUTOSAR-based Example and a Case Study,’ Joint 24rd International Workshop on Software Measurement & 9th MENSURA Conference, Rotterdam (The Netherlands), Oct. 6-8, 2014, IEEE CS Press, pp. 23-31.


92

Industry Data – Example 2 25 industrial Web applications

1000 500

Work-hour Residuals

Median

0 -500

No abitrary max 11 10 9 8

COSMIC Function Points (CFP)

6 FP

7 6

-1000

4 FP

5 3

1

2

4

3 FP

A single CFP exists & is well defined

CFP

Conclusions: ‘The results of the … study revealed that COSMIC outperformed Function Points as indicator of development effort by providing signiﬁcantly better estimations’

FP Ref.: ‘Web Effort Estimation: Function Point Analysis vs. COSMIC By Di Martino, Ferrucci, Gravino, Sarro, Information and Software Technology 72 (2016) 90–109


93

Industry Data – Example 3: Security & surveillance software systems Context:  Scrum method  Teams estimate tasks within each iteration in Story Points  Measurements of 24 tasks in 9 iterations  Each task estimated in Story Points  Task actual effort recorded  Each task also measured in CFP Ref. ‘Effort Estimation with Story Points and COSMIC Function Points - An Industry Case Study’, C. Commeyne, A. Abran, R. Djouab. Obtainable from www.cosmic-sizing.org ‘Software Measurement News’. Vol 21, No. 1, 2016


94

Industry Data – Example 3: Security & surveillance software systems 200

Effort = 0.47 x Story Points + 17.6 hours

Actual Effort (hours)

180

and R2 = 0.33)

160 140 120 100 80 60 40 20 0 0

20

40

60

Story Points

80

100

120

140

160

180

200

Estimated Effort (Hours)


95

Industry Data – Example 3: Security & surveillance software systems and R2 = 0.977)

Y = 2.35 x CFP - 0.08hrs 200

200

Effort = 0.47 x Story Points + 17.6 hours

and R2 = 0.33)

180 160

160

Actual Effor (Hours)

Actual Effort (hours)

180

140 120 100 80 60

140 120 100 80 60

40

40

20

20

0

0

0

20

40

60

80

100

120

140

160

180

200

0

10

20

30

40

50

60

70

Functional Size in CFP

Estimated Effort (Hours)

COSMIC

Story Points


96

80



97


Challenge…

When will we get our base unit & golden measurement étalons for software design & control?


99

For software: add a new Base Unit – the 8th one!


Entreprise Architecture & Balanced Scorecard perspectives


GOAL/OBJECTIVE

COSMIC-based measures: Process Perspective in IT organizations

DRIVER

PROCESS (PR) Application Development Size & Maintenance Effort Productivity Support

Defectability& Test

Reuse

INDICATOR ·

FSunit – Functional Size unit,

· · ·

PS – Portfolio Size WE – Work Effort PDR – Project Delivery Rate

·

EP – Enterprise Productivity

·

ASR – Application Support Rate

·

DDR – Duration Delivery Rate

· ·

AMPL – Application Maintenance Load per Person RCR – Repair Cost Ratio

· · ·

SR – Stability Ratio DR – Defect Ratio TPR – Testing Proficiency Ratio

·

MTTR – Mean Time To Repair ratio

·

AR – Application Reliability

·

DER – Defect Detection Ratio

·

# defects / 100 FSunit according to user acceptance

·

FR – Functional Reuse %

·

TR – Technical Reuse %


COSMIC-based measures: Financial Perspective in IT organizations

GOAL/OBJECTIVE

DRIVER

INDICATOR

FINANCIAL (F) Asset Management

Existing asset utilization

·

Total Assets (FSAV) / # employees ($)

·

FSAV – FSunits Asset Value

·

PS – Portfolio Size

·

Revenues / FSAV (%)

·

Revenues from new customers / Total Revenues (%)

· ·

Profits / FSAV (%) Investments in IT

·

PCFS – Project Cost per FSunit

·

ECFS – Enterprise Cost per FSunit

·

AMCFS – Application Maintenance Cost per FSunit

Revenue Profitability

Financial Management

& Revenue Growth

Profitability Point

Organizational Investments Project Investments


For software: COSMIC as a candidate to an 8th base unit


When will software architects & software engineers bring their measurement instruments with them when they pass away …?


105

Thank you for your attention

? [email protected]


You want to know more?


107

Other sources of COSMIC examples with industry data • COSMIC web site at: www.cosmic-sizing.org


108

Measurement in Software Engineering

Measurement in Software Engineering

Suggest Documents

Practical Software Measurement - Software Engineering Institute

Property-based Software Engineering Measurement

The Measurement Service in Software Engineering ...

Measurement based continuous assessment of software engineering ...

software-engineering measurement for logistics and ...

17-635 Software Measurement v.4 - Software Engineering Masters ...

Lack of Consensus on Measurement in Software Engineering ...

Software measurement

Patterns in Software Engineering

Liability in Software Engineering

Fundamental Issues in Software Measurement

Software Engineering Techniques - AG Software Engineering - Uni ...

Software Engineering Research What Is Software Engineering ...

In systems engineering and software engineering, requirements ...

requirement engineering process in software engineering pdf ...

Personality types in software engineering - Western Engineering

In systems engineering and software engineering, requirements

Agile Software Development in Global Software Engineering

Software Engineering

Software Engineering

Software Engineering

Software Engineering

Software Engineering

Software Engineering