Application Lifecycle Management and process ... - WordPress.com

Application Lifecycle Management and process monitoring through an integrated and low-cost solution, mainly based on Open Source Software Products

1°International Conference on IT Data collection, Analysis and Benchmarking Rio de Janeiro (Brazil) - October 3, 2013

N. Bertazzo, D. Gagliardi, S. Oltolina, G. Ruffatti

Insert here a picture

Engineering Group

http://itconfidence2013.wordpress.com

ALM and…

Presentation Goals

G1. Measuring company performances through an effective SPI program G2. Adopting a multi-dimensional performance model deployed mostly with 3D instances G3. Implementing an integrated and low cost – OSS based – solution for the measurement and governance of software product & process quality

IT Confidence 2013 – October 3, 2013


2

The problem: Managers’ needs

Introduction

Which is users’ & customers’ level of satisfaction?

How productive is my organization?

Which is the quality level of my product?

How can I improve performance?

How can I compare different labs?

Which are corporate audits results?

Top Manager

Quality Manager

Project Manager

Is there REALLY a way to measure performance? IT Confidence 2013 – October 3, 2013

How can I improve the development process?

Is my project on track?


3

Engineering

At a glance

www.eng.it IT Confidence 2013 – October 3, 2013


4

Engineering production organization Managers’ needs

Background Worldwide Customers

Business Units (BUs) for different market sectors Business Analysis

Business Competence Center

Account Managers

Project Managers

Service Desk

Sales Managers

Technical, Innovation & Research Division Engineering’s Software Labs (ESL) Resource management PRODUCTION ESL1-2: Project development

RFPs technical support Architectural design ESL3: Application Management

Research & Development

Competence Centers

MANAGED OPERATIONS Infrastructures & System Services



5

Background

Compliance of SPI to quality systems

• Continuous Quality Improvement in Engineering's projects • Unified Infrastructure supporting quality processes granting flexibility and adaptability • CMMI-DEV and ISO certifications, as independent method to validate the compliance of processes and infrastructures with quality standards • Set-up of Engineering Software Labs (ESLs) to enhance and measure productivity and improve quality practices

…



6

Interaction & Information sources (quant. & qual. data)



7

QEST model: 3 dimensions

•

Three dimensions of analysis: 1. 2.

Economical (E) Social (S)

3.

Technical (T)

•Performance values for each dimension allow to identify process areas that need improvements



8

Productivity Intelligence

The model • QEST nD model, a conceptual framework for measuring process performance based on multiple analysis dimensions http://www.semq.eu/leng/modtechqlm.htm

The tool • Spago4Q, the open source platform to measure, analyze and monitor quality of products, processes and services http://www.spago4q.org



9

The model (1): QEST

Method: Performance is expressed as the combination of the specific ratios selected for each of the 3 dimensions of the quantitative assessment (Productivity - PR) and the perceived product quality level of the qualitative assessment (Quality - Q)

Performance = PR + Q Model: QEST (Quality factor + Economic, Social & Technical dimensions) is a “structured shell” to be filled according to management objectives in relation to a specific project Such a model has the ability to handle independent sets of dimensions without predefined ratios and weights - referred to as an open model



10

The model (2): geometrical indicators

• Target: measuring project performance (p) using 3 distinct viewpoints • Input Data: list of weighted ratios for each dimension and quality questionnaires • Output Data: an integrated normalized value of performance It is possible to measure performance considering at least 3 distinct geometrical concepts: • Distance between the tetrahedron base center of gravity and the center of the plane section along the tetrahedron height – the greater the distance from 0, the higher the performance level; • Area of the sloped plane section – the smaller the area, the higher the performance level; • Volume of the lowest part of the truncated tetrahedron – the greater the volume, the higher the performance level. IT Confidence 2013 – October 3, 2013


11

The model (3): key features

• Integrated quantitative and qualitative evaluation from 3 concurrent organisational viewpoints • A 3D geometrical representation at a single project phase (usually after the project is completed) •Use of de facto and de jure standards (e.g. ISO/IEC 9126 for the Quality Factor  now ISO/IEC 25010:2011) • Performance Measurement Model to use for consolidating Balanced Scorecard (BSC) measurement outcomes • Extension of the original 3D model to n possible dimensionsperspectives  QEST nD through the simplex as the mechanism to solve the problem from the 4th dimension on



12

The tool: Spago4Q



13

The integrated environment: QEST & Spago4Q

• QEST model is fully supported by Spago4Q • The procedure is coherent with the PMAI (Plan-Measure-AssessImprove) cycle:  PLAN, defining a set of metrics, based on the GQM approach, and possible dimensions of analysis (perspectives) characterizing the analysis  MEASURE, including the collection of data, and the computation of metric values and global performance value  ASSESS, presenting results through dashboards and reports  IMPROVE, analyzing in detail each value that is less than the expected thresholds, in order to find possible problems or bottlenecks from a process-based viewpoint



14

3D analysis: main goals

The ESL model selected goals for each analysis dimension: 1. Economical (E) E.G1 E.G2 E.G3 E.G4 E.G5

Reduce the effort of corrective maintenance (corrective + preventive,  ISO/IEC14764:2006) Improve ESL resource/assets allocation Reduce effort due to hardware system unavailability (‘downtime’) Reduce rework (Analysis/Design SLC phases) Improve productivity (note: different ‘sizing’ units)

2. Social (S) S.G1 S.G2 S.G3 S.G4 S.G5

Reduce the number of non-conformity issues (QA inspection) Improve artifacts reuse (functional reuse) Evaluate training skills for organizational resources Improve customer satisfaction (e.g. Customers/Prospects, Business Units, Developers) Improve knowledge sharing (“social 2.0”, communities)

3.Technical (T) T.G1 T.G2 T.G3 T.G4 T.G5

Reduce the resolution time for defects and technical issues Reduce the number of pre-delivery defects (as in ODC analysis) Improve delivery time for deliverables Improve code quality Improve the testing process (e.g. coverages, # req’s, # tests, ...) IT Confidence 2013 – October 3, 2013


15

3D analysis: metrics

Metric ID

Metric Desc

Formula

Source

E.M1.1

Incidence of corrective maintenance effort

Corrective Maintenance Effort/Development Effort

ALM & prj registry

E.M2.1

Allocation of ESL resources

Nr. of Res (hours) allocated on prj/Tot of Res (hours)

ALM & Corp. Systems

E.M3.1

Hardware System Availability

Percentage System Availability

System Monitoring

E.M4.1

Incidence of rework

Rework Effort / Development Effort

ALM & prj registry

E.M5.1

Development capability

FP/Effort

ALM & prj registry

S.M1.1

n. Of Non Conformity issue

% of NC for project

ALM & QA Registry

S.M2.1

Incidence of artifact reuse

Nr downloads/tot nr of artifacts stored

Component repo

S.M3.1

Skill improvement

% new (or modify) skills for resource

Skill management tool

S.M4.1

Customer Satisfaction

Results of survey

Survey tool

S.M5.1

Knowledge sharing improvement

% of interaction with collab. tools

Collaboration tools

T.M1.1

Incidence of defects

% nr. of defects (errors + defects) for project

ALM

T.M2.1

Defects Mean Resolution Time

Tot. resolution time/Tot. defects

ALM

T.M3.1

Incidence of delayed deliverables

% nr. delayed deliv. / Tot. deliverables

ALM

T.M4.1

Code Quality

Results of automatic static test

Code analysis tool

T.M5.1

Testing process improvement

Test coverage

ALM



16

3D view: drill-down through the organizational levels Top Manager (TM) ESL

Level 1

Engineering’s Software Labs

ESL Chief Manager

ESL 1

Level 2 ESL Lab Manager

Project development

PRJ 1 Level 3

ESL 2

ESL 3

Project development

PRJ 1

PRJ 1 PRJ n

Application maintenance

PRJ n

PRJ n

Project Manager (PM) IT Confidence 2013 – October 3, 2013


17

3D view: Level 1 - TM dashboard – sample #1  Unified view on Engineering Software Labs  Global performance indicator  Performance comparison (time, labs)



18

3D view: Level 1 - TM dashboard – sample #2



19

A single dimension view: Level 3 - PM dashboard  Detailed view  Tracking and trends

Risks Tasks & Issues

Reqs & Bugs

Docs


Metrics http://itconfidence2013.wordpress.com

20

A single dimension view: Level 3 - PM ALM tool



21

A single dimension view: Levels 1/2 - QA audits



22

Conclusions

Finally I know how productive my organization is!

Users & customers feedbacks are now integrated with corporate data

Productivity Intelligence enables performance improvement

Now I can compare Labs performance!

Through audit dashboards, corporate QA is under control

I can monitor the quality level of my product

Top Manager

Quality Manager

Project Manager

Finally we can REALLY measure performance! IT Confidence 2013 – October 3, 2013

The development process is under control and I can improve it!

I know if my project is on track & I can identify issues


23

Next steps

• Improve reports and KPIs  Introduce/improve reports for new/modified information needs  Dynamical update for thresholds

• Integrate tools to collect soft factors measures and indicators from two categories of ESL customers using on-line surveys:  “external customer”: feedback on the perceived product quality  “internal customer”: feedback on the actual process quality from project managers of the various Business Units

• Build new KPIs on data coming from the Infrastructure Enhancing Project that collects issues and suggestions by the ALM users  e.g. in the BSC ‘Learning & Growth’ perspective



24

References



Buglione L., Misurare il Software. Quantità, qualità, standards e miglioramento di processo nell’Information & Communication Technology , FrancoAngeli, 3/ed, 2008, ISBN 978-88-464-9271-5



Buglione L. & Abran A., QEST nD: n-dimensional extension and generalisation of a Software Performance Measurement Model , International Journal of Advances in Engineering Software, Elsevier Science Publisher, Vol. 33, No. 1, January 2002, pp.1-7



Spago4Q website and resources: www.spago4q.org



Contacts & Info: [email protected]



25

25

ALM and...

Q&A

Muito obrigado pela vossa atençao! Thanks for your attention! IT Confidence 2013 – October 3, 2013


26

ALM and...

Nicola Bertazzo

[email protected]

Our Contact Data

Daniele Gagliardi

Sergio Oltolina

Gabriele Ruffatti

[email protected] [email protected] [email protected]



27

Application Lifecycle Management and process ... - WordPress.com

Application Lifecycle Management and process ... - WordPress.com

Suggest Documents