Practical experiences of Function Points estimation and ... processes that are too interrelated with others who can not be easily ... Software as a market good.
Practical experiences of Function Points estimation and measurement in a complex outsourcing context of an Italian Public Administration. Roberto Meli – Franco Perna (DPO – Italy)
Agenda
Outsourcing software services in a Public Administration context A real case study from an Italian PA Software contracts governance by metrics
N. 2
Let’s start from here…
Forrester Research 2008 in EMEA area: “… Extraordinary performance in the second quarter of 2007 for the main outsourcer in the area, which have closed 84 contracts for a total of more than 5.4 billion euros (compared to the same period in 2006, the increase is 1 billion in value and 19 in number of contracts).” Good news ? Bad news ? N. 3
Outsourcing problems…
Dun & Bradstreet reported in 2006 that approximately 20-25% of all outsourcing relationships fail within two years.
According to a research from Everest, the main reasons why these projects fail can be attributed to four categories:
not well defined objectives : setting unrealistic and not achievable targets, not aligned with the objectives and corporate culture, or even based on an incorrect definition of the baseline cost Wrong definition of Outsourcing Scope: for example, outsourcing processes that are too interrelated with others who can not be easily managed outside Errors in planning and managing the transition phase, for example with too much hesitation in the transition from internal management to the Outsourcer. Wrong definition of the contract or its governance, in particular for measuring achievements, diagnostics and troubleshooting, reporting and process escalation. N. 4
Software as a market good Market Transactions External Productivity
Requirements
Equity Perception
Constraints Solution (TCQ2)
Equity Perception
Negotiation
Customer
Internal Productivity
Requirements Constraints Quantity Quality Effort Time Staff Unitary staff cost Discount Total cost
Contract
Supplier
Internal Productivity
N. 5
Is software different ?
How many industrial managers would be willing to base the acquisition of their supplies on a performance that they are not able to quantify? N. 6
As a matter of fact… The discipline and practice for software contracts is not as mature as in other industrial sectors.
Legal expert Buyer Software manager Account Administrative
No “measurement culture” for software procurement
N. 7
Usually
AR T TIME EX OWN KN
N W NO QUANTITY K UN US O I
N W NO COST K +/ -
R QUALITY TE
S Y M
Maximize
Minimize RISKS
N. 8
And what about the P.A. ?
Much more constraints than in a private market
Untouchable budgets Compliance of assignments with hundreds of rules No personal judgment Personal responsibility Sometimes absence of “technical” culture External rulers and monitoring bodies ….
N. 9
Problem statement current software development trends show an increasing relevance of the outsourcing option
Too many Litigations
pure time&material contracts are not preferred anymore by customers
software procurement practices and organizations are not mature enough software measurement discipline is overlooked by customers & suppliers
High nbr of inadequate SW Contracts
Too many Wasted Resources
Too many Low Quality Systems
N. 10
Equity perception vs Equity demonstration
Equity “demonstration” is a “must” for a Public Organization but…
Quite often, there is the suspect that the main goal of a public administrator, in a software procurement case, is not to buy the best product with the minimum and “fair” amount of resources but with any amount which could be referenced to an administrative formal precedent An equity demonstration which is based on the sole size (FSU) component (i.e. price per FP) is catastrophic in many cases! Public Procurement Processes tend to be “conformist” aligning the future to the past N. 11
The public sw acquirer’s dilemma The main risk of a public software acquirer is to be charged with being arbitrary in the determination of the economical aspects of the supply. So the question becomes: is there any other precedent to be referred in order to justify the cost of my next acquisition ? The fixed price per Function Points of other contracts (especially when it is not well known what exactly the FP are..) seems to be perfect to be used: simple and popular !
N. 12
Self-fulfilling Prophecy Customer & Supplier want (over) simplified procurement models
The equity model is based only on the Cost per FP parameter
The search for a Formal Administrative Precedent becomes essential
The need for a formal demonstration dominates the need for a substantial appreciation
A new contract will be considered “fair” if its “Cost per FP” is confirmed by past contracts
Any new benchmarking “point” will be aligned to the previous ones
Benchmarking database validity decreases as time passes by
“Real” performances might be not correlated to the formal equity Potential litigations, wasted resources or low quality systems N. 13
Backfiring from costs So… let me see how much this project should cost…
Well, using the ISBSG equations, some COCOMO adjustment factors, a little bit of analogy... The cost is €120’000, let’s call the procurement department…
I need a FP count within tomorrow
Sorry, Joe but that supplier has signed an agreement with us for a unitary price of €200/FP this year
Hey Boss, the size of the application is 400 UFP
OK, let us see… the required FP size should be: 120’000/200 …. Oh yes this application must be 600 FP !
N. 14
A solution… is to substitute a formal, abstract, economical validation process based on a single variable, with a specific, situational, articulated and flexible process based on a sufficiently large number of variables even if this might add some more “subjectivity” of judgment. This requires time and competencies by the parties involved in the negotiation but, unfortunately there are no valid shortcuts.
N. 15
A Real Case Study
N. 16
An Italian important P.A.
Long term ICT contract (48 months) with many suppliers One Program, 40 Projects, 220 sub projects Large components reuse for software production IFPUG FP 4.1.1 & E&QFP 2.0 adopted Payments based on released FP More than 150’000 FP to be supplied 480 FP based activities (1 estimation + 2 counts) Fixed FP price (weighted unitary price from average situations) FP Measurement under supplier responsibility External Monitoring Contract (starting after 1 year from the program launching) N. 17
Invoicing steps
N. 18
Measurement Program Plan
Time span of 2 years Initial supplier measurement process assessment (6 months) Statistical analysis of data Complete re-validation of all measurements already done in the past Regular on going FP validation Customer-supplier validation sessions Dashboard representation of FP based indicators (estimates and counts) Software asset documentation (Sfera)
N. 19
Measurements Data Base
3 4 Realizzazione sistema C
2
-380
-38,00%
-433
-38,42%
729
1,14
831
512
584
VEVP-002-1.0
-217
-30,00%
-247
-29,72%
1218 950
1,14 1,14
1388 1083
1000 889
1140 1013
VEVP-003-1.0 VEVP-004-1.0
-218 -61
-18,00% -6,00%
-248 -70
-17,87% -6,46%
676
1,14
771
700
798
VEVP-005-1.0
24
4,00%
27
3,50%
scarto FP (%)
NA
VEVP-001-1.0
scarto FP
1
694
scarto UFP (%)
4
6
609
scarto UFP
NA 1 NA 2 1 3
1127
verbale di verifica FP
2 2 2
1,14
FP proposti dal Fornitore
2 2 2
989
VAF proposto dal Fornitore
XX XX XX
UFP proposti dal Fornitore
NA 2
Fase
3
lotto
2
confronto misure monitore-fornitore
FP validati dal Monitore
2
XX
misura monitore UFP validati dal Monitore
Realizzazione sistema B
s-progetto
1
progetto
Realizzazione sistema A
misura fornitore
contratto
Id. del campione
riferimenti progetto nome WBS
XX
tot UFP verificati
4.562
tot FP verificati
5.200
tot UFP tot FP validati validati
3.710
4.229
tot scarto UFP
% scarto tot UFP
-852 -18,68%
tot scarto FP
-971
%scarto tot FP
-18,67%
N. 20
Supplier measurements analysis 43,59%
38,42%
29,72% 27,66%
28,30%
26,73% 24,16%
19,21%
19,09%
19,09% 17,70%
15,23%
15,32%
15,32% 13,65% 11,23%
10,61%
9,93% 6,46%
6,50% 4,68%
1,95%
1,14% 2
7
15
18
1
3
6
0,72% 8
9
10
16
19
23
4
5
11
12
13
17
20
21
22
24
14
10
6 5
1
-50÷-40
1
1
-40÷-30
-30÷-20
-20÷-10
-10÷0
0÷10
0
0
0
0
10÷20
20÷30
30÷40
40÷50
N. 21
17
P 06
13
P 05
15
P 04
7
P 03
0 P 02
2 P 01
2 G 6
3 G 5
1 G 4
G 3
G 2
1
G 1
7
S .F 2
29
S .F 1
10
S .T 3
S .T 2
88
S .T 1 .1
S .T 1
3
S .D 3
S .D 2
S .D 1. 1
3
S .D 1
0
S .C 2
2 S .C 1
E 3
E 2
E 1
Distribution of classified anomalies
480
217
157
75 63 96
28 16 0
N. 22
Final results
Counting supplier process regularly improved after initial assessment The customer paid the actual size at the contractual cost 15’000 FP of requirements could be recovered for realization spending the original budgeted money Less litigation over time Adequate financial anticipation of money for E&QFP better estimation. N. 23
Software contracts governance by metrics
N. 24
Why measuring software ?
Cognitive aspect (internal goal)
To improve the level of knowledge of a phenomenon (product, process)
Social aspect (contractual goal)
Giving evidence to the stakeholders of the acquired knowledge
N. 25
Available resources GUFPI-ISMA:
Linee Guida per l’uso contrattuale dei Function Points (2006) http://www.gufpi-isma.org/ CNIPA: Linee guida sulla qualità dei beni e dei servizi ICT per la definizione ed il governo dei contratti della Pubblica Amministrazione http://www.cnipa.gov.it/site/it-IT/
N. 26
What items should be considered in a software contract?
“Base” contractual scheme Type of supply (DEV, Fun. Enhancement, Non Fun. Enhancement, Corrective…) Process models and measurement points Scope of work Classification and level of detail of software requirement On going change request policy Economical appraisal of a cancelled project Factors impacting prices Non FP proportional work appreciation N. 27
Software contract governance
N. 28
Base contractual schemes
Fixed Cost for Specified Product (also called turnkey supply), Variable Cost for Time & Material Variable Cost for Released Product.
N. 29
Process models and measurement points
INTEGRATION
INTEGRATION
iterazione fasi discipline
milestone minore discipline principali milestone maggiore
discipline di supporto
Lifecycle Objectives
Lifecycle Architecture
Initial Operational Capability
Product Release
iterazioni
release
release
release
release
release
release release release finale
N. 30
“Scope” of the supply
It does not influence size It does influence unitary prices
N. 31
Classification and level of detail of software requirements SW Application
Macro Functional Process
Functional measurement (FUR) Technical measurement (non functional/technical requirements) Quality measurement (non functional/quality requirements)
General Functional Process
Multiple Logical Data Group
…
General Functional Process
Typical Functional Process
…
Logical Data Group
Base Functional Process
Logical Data Group
Base Functional Process
Base Functional Process
Logical Data Group
Base Functional Process
Base Functional Process
SW Application
Macro Functional Process
General Functional Process
…
General Functional Process
Typical Functional Process
Multiple Logical Data Group
…
Logical Data Group
Base Functional Process
Logical Data Group
Base Functional Process
Base Functional Process
Logical Data Group
Base Functional Process
Base Functional Process
N. 32
On going CR management policy CRFP = ADD + CFP + Σi (CHGAi * CLi * LCPi) + Σi (DELi * LCPi)
we can assess each FP Change Request as if it was an enhancement maintenance, correcting, however, measurements with some adjustment factors that take into account the level of reuse applicable and / or waste of effort due to the requirements changed or abandoned. These factors are named respectively as CLi (Change Level) and LCPi (Life Cycle Progress)
N. 33
Economical appraisal of a cancelled project
Riduced price = FP x (FP unitary price x %passed phases) N. 34
Factors impacting prices
Grado di impatto dei diversi fattori Molto Basso
Basso
Normale
Alto
Molto Alto Estremamente Alto
0,75
0,88
0,89
0,95
1 1 1
1,15 1,15 1,06
1,39 1,3 1,13
0,75
0,88
1
1,15
1,3
1,66
1
1,15
1,29
1,46
Fattori di qualità del prodotto Grado di affidabilità del software Incidenza dei test Accuratezza della documentazione Fattori tecnici del prodotto Vincoli tecnologici Fattori relativi al progetto Volatilità delle Specifiche
N. 35
In the years to come…
New System market or internal components (SOA)
Software reuse as a competitive factor N. 36
Two different “reuse” classes
Technical reuse
low
“Code” savings
No savings
high
Global savings
Analysis savings
high
low
Functional reuse N. 37
An example: 10 UFP/pm
500 UFP
50 pm
AVG(Productivity) from database
250 UFP by scratch 28 pm
100 UFP
150 UFP
reused modules
market of components
UFP (almost) for free! N. 38
The need for replication…
Same functionalities (EI,EO,EQ, ILF,EIF) Different architectures
N. 39
Contractual Functional Measurement
MFC = FPreleased - FPriused + FPreplicated
N. 40
Cost development model
N. 41
Non FP proportional work appreciation It makes no commercial and technical sense to "spread" the fixed or non FP proportional costs of the project to the price of the functional component.
For example: the cost of installing an application does not depend on how big it is in terms of FP, but on how many times you must do it and in what logistics situations.
N. 42
Measurement Process dashboard Ripartizione verifiche per fase CVS
0,6%
9,6%
8,4%
1 2 3
81,4%
4
N. 43
Thanks for your attention!
N. 44
