Department of Software and IT Engineering
Systems y and Software Engineering g g Lifecycle Standards for Very Small Entities Professor Claude Y Laporte, P. Eng., Ph.D. Project Editor of ISO/IEC 29110
Berlin, November 16th 2016
Content • Introduction • Needs for Standards and Guides for Very Small Entities (VSEs) • Development p of ISO/IEC 29110 for Software Engineering g g • Pilot Projects • Development of ISO/IEC 29110 for Systems Engineering • Pilot Project • Next Steps VSEs = Very Small Entities are enterprises, organizations, projects or departments having up to 25 people.
2
Product Development - The Challenges
A
3
The Importance of VSEs • An Electronic Manufacturer selling millions of products all over the world VSE
Manufacturer Prime ((60)) Suppliers (600) Producers (~6,000)
A software defect from one of the producers p went into a product and resulted in a loss of over $200 million by the manufacturer
A
Adapted from (Shintani 2006)
4
Size of Enterprises • European Union*
Micro-enterprises
1 ‐ 9
≤ 2 million
Number of enterprises (% of overall) (% of 92.2 %
Small enterprises
10 ‐ 49
≤ 10 million
6.5 %
1 358 000
50 – 249 50
≤ 50 million ≤ 50 million
11% 1.1 %
228 000 000
99.8 %
21 544 000*
0.2 %
43 000
Type of enterprise
Medium enterprises SMEs, total Large enterprises Large enterprises, total
Number of employees
Annual turnover (EUR)
87 100 000 > 250 42 900 000
Number of enterprises 19 968 000
> 50 million
* Independent companies only, excluding legally independent companies that are part of large enterprises.
• Micro-enterprises account for 70% to 90% of enterprises in OECD** countries (about 95% in USA***) * Moll, R., Being prepared – A bird’s eye view of SMEs and risk management, ISO Focus, February 2013 ** OECD: Organisation for Economic Co-operation and Development *** Statistics About Business Size (including Small Business). US Census Bureau (www.census.gov/econsmallbus.html).
5
Development of International Standards and Guides for VSEs
• • • • • •
Phase 1 - Recognition g of Needs and Problems ((2004)) Phase 2 - Basic and Applied Research (2005-2006) Phase 3 - Development (2006-2010) Phase 4 - Commercialization (2010) Phase 5 - Diffusion and Adoption (2006- ) Phase 6 - Consequences C (2011 - )
(Adapted from (Rogers, 2003)
6
International Organization for Standardization Joint Committee for IT
Sub committee ((SC)) 7
Standardization of processes, supporting tools and supporting technologies for the engineering of software products and systems.
Working Group (WG) 24
A
7
Survey of VSEs
2. Research
• Objectives • Identify VSEs' utilization of standards • Identify Id if problems bl andd potential i l solutions l i to help h l VSEs VSE apply l standards and become more capable and competitive.
• Method • Web-based Survey • Questionnaire translated in 9 languages • E English, li h French, F h German, G Korean, K P Portuguese, R Russian, i Spanish, S ih Thai and Turkish.
• Invitation to participate in survey widely broadcasted via: • WG 24 Network of contacts • Centers and initiatives focused on SMEs/VSEs – e.g., SIPA (Thailand), CETIC (Belgium), Parquesoft (Colombia)
A
8
2. Research
Requests from VSEs
• Certification and Recognition • Only 18% are certified • Over 53% of larger companies are certified
• Over 74% indicated that it was important to be either recognized or certified • ISO certification requested by 40%. • Market recognition requested by 28% • Only 4% are interested in a national certification
• Needs Regarding Documentation • 62% are asking ki for f more guidance id andd examples l • 55% are requiring 'lightweight' standards that are easy to understand and apply and come with templates. A
9
3. Development
The Strategy of WG 24 To develop standards and guidelines for VSEs
• Use the notion of ‘Profile’ to develop a roadmap, standards and guides to meet the needs of VSEs. – A profile is an ‘assemblage’ from one or more base standards to accomplish a particular function. function – A Profile Group (PG) • A collection of pprofiles which are related either byy composition of processes (i.e. activities, tasks), or by capability level, or both. • Focus F fi firstt on VSEs VSE developing d l i Generic G i Product P d t development d l t (Generic Profile Group) • i.e. VSEs that do not developp critical pproducts. A
10
3. Development
The Strategy of WG 24 To develop standards and guidelines for VSEs • Use two types of standards, as the input, for the development off standards t d d for f VSEs: VSE – Process standards, such as ISO/IEC 15288 and ISO/IEC 12207,, that define the activities required q to achieve identified objectives or outcomes; – Product standards, such as ISO 15289, that define the structure t t andd content t t off artefacts t f t produced d d by b the th processes; – e.g. Specification, report • Develop a set of documents, targeted by audience, to describe and specify the profiles.
A
11
3. Development
The Generic Profile Group for Developers of System/Software •
• •
Advanced Intermediate Basic Entry
A
•
Entry - Targets VSEs typically developing 6 person-month person month projects or start-ups Basic – Targets VSEs developing a single product by a single work team Intermediate – Targets VSEs involved in the development of more than one project in parallel with more than one work team Advanced – Targets VSEs which want to sustain and grow as an independent competitive system or software development business
ISO/IEC 29110
12
Spectrum of Development Approaches
29110
A
Adapted from (Kroll 2003)
13
Set of 29110 Documents Targeted by Audience 29110 Overview (TR 29110-1) 29110 1)
For VSEs and customers
29110 Profiles (IS) Framework and Taxonomy (IS 29110-2) Specifications of VSE Profiles (IS 29110-4)
For Standard producers, tool vendors, methodology vendors
Specification - VSE Profile Group m (IS 29110-4-m)
List the Requirements i.e. ‘What to do’ 29110 Guides (IS/TR) Assessment Guide (IS/TR 29110-3)
For Assessors, customers and VSEs
M Management t and d Engineering E i i Guide G id (TR 29110-5) Management and Engineering Guide VSE Profile m m-n n
For VSEs F VSE and customers
‘How to do’
(TR 29110-5-m-n)
A
TRs are available from ISO at no cost http://standards.iso.org/ittf/PubliclyAvailableStandards/index.html
ISO/IEC 29110
14
ISO/IEC 29110 Software Management and Engineering Guide Customer
Implementation Process Implementation Initiation
Statement of Work
Software g Configuration
Project Management Process Planning
Evaluation
Execution
Closure
VSE’s Management
A
Requirements Analysis Architecture and Design Construction Integration and Tests Product Delivery
ISO/IEC 29110 is not intended to preclude the use of different lifecycles such as waterfall, iterative, incremental, evolutionary or agile.
15
Management and Engineering Guide Table of Contents Foreword Introduction 1 Scope 1. S 2. Normative references 3. Terms and definitions 4. Conventions and abbreviated terms 5. Overview 6. 7. 8. 9. 10.
Project Management (PM) process Software Implementation (SI) process Roles (all roles) Product description (all products) Software tools requirements
Annex A (informative) – Deployment Package Bibliography A
ISO/IEC 29110
16
Process Structure Description and Notation 1. 1 2. 3. 4. 5. 6 6. 7. 8. 9.
A H
Name Purpose Objectives Input Products Output Products I t Internal l Products P d t Process Roles involved Activities g Process Diagram Tasks Activity Description – Task - Description of the tasks to be performed. – Role - Abbreviation of roles involved in the task execution. – Input Products - Products needed to execute the task. – Output Products - Products created or modified by the execution of the task. ISO/IEC 29110
17
Software Implementation Process One Task of the Requirement Analysis Activity Role CUS
Task SI.2.4 Validate and obtain approval h Requirements Specification S f off the
AN Validate that Requirements Specification satisfies needs and agreed upon expectations, including the user interface usability. The results found are documented in a Validation Results and corrections are made until the document is pp by y the CUS. approved A
Input p Product Requirements S Specification f [verified]
Output p Products Validation Results Requirements Specification [validated]
ISO/IEC 29110
18
ISO/IEC 29110 - Document Content • Change Request Name
Description
Source Customer
Identifies a Software, or documentation problem or Change desired improvement, and requests modifications. R Request t It may have the following characteristics:
Project Management
- Identifies purpose of change - Identifies request status - Identifies requester contact information - Impacted system(s) - Impact to operations of existing system(s) defined - Impact I t to t associated i t d documentation d t ti defined d fi d - Criticality of the request, date needed
Software Implementation l i
The applicable pp statuses are: initiated, evaluated, and accepted p
A
ISO/IEC 29110
19
Certification Process •
Will be b published bli h d as an ISO 29110 Document D –
•
Based on ISO Standards on Conformity Assessment –
•
Systems and Software engineering — Lifecycle profiles for Very Small Entities (VSEs) — Part 3-2: Conformity Certification Scheme e.g. ISO/IEC 17065 – Requirements for bodies certifying products, processes and services p
Four-Stage Certification Process Initial Certification
Expiry of Certification
Three-year Certification Cycle On-going Surveillance Activities Application for Certification
A H
Initial Certification
Surveillance Audits
Recertification
(ISO/IEC 29110-3-2)
20
5. Diffusion
Deployment Packages (DPs)
• A Deployment Package (DP) is a set of artifacts developed to facilitate the implementation of a set of practices, of the selected framework, in a VSE. – Deployment packages are not intended to preclude or discourage the use of additional guidelines that VSEs find useful. • By B deploying d l i andd implementing i l i a Deployment D l Package, P k a VSE can see its concrete step to achieve or demonstrate coverage to Part 5. • Deployment Packages are designed such that a VSE can implement its content, without having to implement the complete framework at the same time. • Each DP is reviewed and edited by at least 2 persons
A
21
Deployment Packages for the Software Basic Profile Construction and U it ttesting Unit ti
Project Management
Requirements R i t Analysis
Verification and Validation
Integration and Tests
Architecture and d Detailed Design
Version Control
Product Delivery
Self-Assessment Self Assessment
Freely available on Internet A
22
Content • Introduction • Needs for Standards and Guides for Very Small Entities (VSEs) • Development p of ISO/IEC 29110 for Software Engineering g g • Pilot Projects • Development of ISO/IEC 29110 for Systems Engineering • Pilot Project • Next Steps
23
Overview of some ISO 29110 Implementations Description p of organization IT Start-up collocated in Canada and Tunisia
P j t Description Project D i ti • Created in 2013 • Develops Web solutions, mobiles applications • Their military customers require CMMI level 2 • Implemented ISO 29110 Basic Profile • 20 employees in 2016 (18 in Tunisia) • Will be ISO 29110 audited in Nov 2016 (Third (Third-party party audit)
Large financial institution
• Software team of 6 people • Develop tools for traders p ISO 29110 Basic-Agile g pprocess • Implemented • Number and impact of incidents was considerably reduced
Large public utility provider
• IT division of 1,950 people supporting 2,100 applications • Implemented 12 Process Areas of CMMI level 2 and 3 • Team of 11 people develops web applications • Implemented ISO 29110 Basic profile – Scrum
Manufacturer M f off electrical power trains
• Enterprise of 140 employees (14 software developers) • Develops D l embedded b dd d software f to controll power trains i • Completed a pilot project with ISO 29110 in 2015 • New projects use ISO 29110-based process
24
ISO/IEC 29110 in Perú • Start-up of 4 people • Created in 2012 byy two ggraduates of the Software Engineering g g program of the Peruvian University of Applied Sciences (UPC) • Specializes in providing software development services and a tomation of business automation b siness processes with ith information system s stem solutions. p ISO/IEC 29110 • Decided to implement – Project selected was the Legal Consultation System for an insurance company
• Implemented I l t d ISO/IEC 29110 using i an Agile A il approachh – The project had 6 sprints of 1 week each – Total effort of the project: 882 hours
• In 2016, the company has 23 people http://bitperfect.pe
(Garcia et al. 2015)
25
ISO/IEC 29110 in Perú Task Environment installation (Windows azure, management tools, development environment and project repository)
Prevention
Execution
Review
Correction off
(hours)
(hours)
(hours)
defects (hours)
14 1 15
Project plan development Project plan execution and project assessment and control
3
7
28
58
108
Project j pplan execution ((sprint p planningg and execution)) p
90
Project assessment and control: sprint review, sprint retrospective
18 107
Specification development Statement of work
12
3
7
Specifying user stories and product backlog
95
25
51
Architecture development
35
10
14
Test plan development
45
8
11
253
70
62
14
5
7
124
26 159
Code development and code testing User guide and maintenance document development Software product deployment
6
Project closure
2
A
Total (Hours)
14
585
An IT Start-Up St t off Initiative Start I iti ti
% 45of Total Project Cost 40
41% Cost of Non Conformance (Rework)
35
Appraisal & Prevention Costs
30
26%
25
20
IT Start-up
18%
18 %
15
10
11% 5
0
1 1988
CMM Maturity Level A
2
6% 3
1990
5%
4
1992 1994
1996 Adapted from (Haley et al., 1995)
27
ISO/IEC 29110 Certification •
Based on ISO Standards on Conformity Assessment –
•
e.g. ISO/IEC 17065 – Requirements for bodies certifying products, processes and d services i
Four-Stage Certification Process Initial Certification
Expiry of Certification
Three-year Certification Cycle On-going Surveillance Activities Application for Certification
•
Surveillance Audits
Recertification
Certificate is recognized by countries members of IAF –
A
Initial Certification
Over 66 countries http://www.iaf.nu
28
ISO/IEC 29110 Certification in Perú • Bit Perfect Solutions • First Fi t Peruvian P i VSE tto bbe awarded an ISO/IEC 29110 certificate of conformity – July 2014
• VSE audited by a Brazilian auditor in accordance with the Brazilian Normalization Organization (ABNT)
http://bitperfect.pe
29
ISO/IEC 29110 in Perú • Implementation and Certification of ISO/IEC 29110 in an IT Startup in Peru, • Software Quality Professional Journal,, ASQ, Q, vol. 17,, no. 2,, 2015, pp. 16-29.
http://profs.etsmtl.ca/claporte/Publications/Public ations/SES_2015.pdf
(Garcia et al. 2015)
30
Project Management in a Division of a Large Engineering Firm Advanced d a ced Intermediate Basic Entry
31
Background • Transportation and Electricity Distribution Division • A Canadian division of a large American engineering company, • Company p y was established 10 yyears ago, g , • Over 500 employees spread over 10 offices in Canada, • The company was already using a robust project management process for their large-scale projects, • A project was launched to document small and medium scale project management processes. processes
A
Adapted from (Laporte et al. 2013)
32
Pilot Project in a Large Engineering Firm • Classified their projects in 3 categories and developed 3 project management processes, checklists and templates Small Project
Medium project
Duration of p project j
Less than 2 months
Between 2 and 8 months th
More than 8 months
Size of team
Equal or less than 4 people
Between 4 and 8 people
More than 8 people
Number of engineering specialties i lti involved i l d
One specialty p y
Engineering fees
Between 5,000$ and 70,000$
More than one specialty i lt Between 50,000$ and 350,000$
Percentage of projects
70%
25%
• Small projects used ISO/IEC 29110 Entry Profile • Medium projects used ISO/IEC 29110 Basic Profile • Large projects used the Guide to the project management body of knowledge (PMBOK® Guide) of PMI A
Large project
Manyy specialties p Over 350,000$
5% Advanced Intermediate Basic E t Entry
Adapted from (Laporte et al. 2013)
33
Pilot Project in a Large Engineering Firm • Business objectives targeted for the improvement project Identification Number
Description
O-11 O
Facilitate the integration of new project managers. managers
O-2
Reach an overall customer satisfaction level of 80%
O-3
On average projects should reach cost and schedule targets within 5%
O-4
Reduce overload of staff by 10%
O-5
Reduce schedule slippage to less than one week and 5% of initial cost for mismanaged risks of projects
O-6
Reduce rework by 10%
O-7
Reduce non billable hours by 10%
Adapted from (Laporte et al. 2013)
34
Pilot Project in a Large Engineering Firm • Project Management process for small projects was evaluated against the 17 tasks of ISO 29110 Entry Profile Project Planning (11 tasks) Project Plan Execution (2 tasks) % of tasks % of tacks performed performed
Project Assessment and Control (3 tasks) Project Closure (2 tasks) 0%
20%
40%
60%
Adapted from (Laporte et al. 2013)
35
Pilot Project in a Large Engineering Firm • Cost analysis using the ISO method to evaluate the Economic Benefits off Standards • Value chain
• Costs and Benefits
A
Y Year 1
Y Year 2
Y Year 3
T t l Total
Cost to implement and maintain
59 600$
50 100$
50 100$
159 800$
Net Benefits
255 500$
265 000$
265 000$
785 500$
Adapted from (Laporte et al. 2013)
36
5. Diffusion
Pilot Project in a Large Engineering Firm
• INCOSE International Symposium, i • Seattle, July 2015 • 20-page article
http://profs.etsmtl.ca/claporte/ Publications/Publications/INC OSE%202015_Tetra_Tech.pdf 37
5. Diffusion
Translations of Software ISO 29110 • Spanish (Peru, Uruguay) •
http://bvirtual indecopi gob pe/normas/29110 5 1 2 pdf http://bvirtual.indecopi.gob.pe/normas/29110-5-1-2.pdf
• Portuguese (Brazil) •
http://www.abntcatalogo.com.br/norma.aspx?ID=90169
• Japanese •
http://www.jisa.or.jp/e
• French (Canada) •
http://profs.logti.etsmtl.ca/claporte/English/VSE/index.html
• Check
A committee of 5 Spanish countries is developing a set of ISO 29110 documents that will be published officially by ISO in Spanish A
38
Content • Introduction • Needs for Standards and Guides for Very Small Entities (VSEs) • Development p of ISO/IEC 29110 for Software Engineering g g • Pilot Projects • Development of ISO/IEC 29110 for Systems Engineering • Pilot Project • Next Steps
In the context of ISO/IEC 29110, systems are composed of hardware and software components 39
Development of Systems Engineering (SE) Profiles and DPs •
Project initiated under sponsorship of INCOSE/AFIS – –
•
International Council on Systems Engineering (INCOSE) Association Française d’ingénierie système (AFIS)
Goals – –
–
To improve p or make pproduct development p efficient byy using Systems Engineering methodology To elaborate tailored practical guidance to apply to VSEs in the context of p prime or subcontractor,, of commercial products To contribute to standardization
40
The SE Basic Profile System Definition and Realization Process
Acquirer
Statement of Work
Product
System Requirements E i Engineering i
Project Management Process Project Planning Project Plan Execution
Project Assessment and Control
System Architectural Design System Construction
Project Closure
VSE’s Management A
System Definition and Realization Initiation
System Integration, V ifi ti and Verification d Validation Product Delivery
ISO/IEC 29110 is not intended to preclude the use of different lifecycles such as waterfall, iterative, incremental, evolutionary or agile
41
Deployment Packages For the Systems Engineering Basic Profile Verification Basic Profile Interface Management
& V lid ti Validation
Project Management
Functional & Physical Architecture
Requirements Engineering
Configuration C fi ti Management
I Integration i
Product Deployment
Change Management
Available on INCOSE VSE WG site and on Internet 42
Polarsys Autonomous Rover • Goal – Develop an Autonomous Rover capable to carry a sensor payload into a confined theatre to: • Map the zone • Collect sensor data
• Uses Dagu g Electronics Rover 5 chassis – 2 DC Motors with encoders
• Multi-specialty development exposure: – – – – –
Electronics Microcontrollers Printed Circuit Board Mechanical Software
• Supports training activities for all Deployment Packages Laporte, C.Y., Houde, R., Open Source Systems Engineering Guides, Deployment Packages and Support Tools for Very Small Enterprises – A Case Study, 25th Annual International Symposium of INCOSE (International Council on Systems Engineering), Seattle, US, July 13-16, 2015.
H
Houde, R., Laporte, C., Blondelle, G., ISO/IEC 29110 Deployment Packages and Case Study for Systems Engineering: The "Not-So-Secret" Ingredients That Power the Standard, 26th Annual International Symposium of INCOSE, July, 2016
43
Overview • Public transportation customers often require a CMMI® maturity level for system and sub sub-system system suppliers – e.g. CMMI Level 2
• In 2012, CSiT was composed of 4 people (10 people presently) – IImplementing l i the h CMMI® Level L l 2 Process P Areas A was too demanding d di at that time.
• Strategy – Implement the draft version of Systems Engineering ISO 29110 Basic profile as a foundation • Used other frameworks to complete process descriptions – e.g. INCOSE Handbook, PMBOK® Guide (PMI) and CMMI®
– Perform a g gap p analysis y between CMMI® level 2 and the SE Basic Profile – Implement practices needed for a successful CMMI® level 2 assessment. A
http://csit.co
44
Classification of Processes Light Process Type of Project
Standard Process
Full Process
Prooff off C P Concept, t Prototype
T i l Project Typical P j t
Concept validation or Product Deployment at Customer Site
Product intended to be Product Testing or installed at Customer Site Product Deployment at Customer Site
Small Project
Medium Project
ISO/IEC TR 29110-5-6Framework 1 - Entry Profile to be used + CMMI - Supplier Agreement Management
Project P j t when h CMMI level 2 is required by a Customer
Large Project
ISO/IEC TR 29110-5-6-2 CMMI (Level 2) - Basic Profile + CMMI - Supplier Agreement Management
http://csit.co
45
Verification, Validation and Acceptance of Work Products • Peer reviews for each internal work products and deliverables – Personal, Desk-check, Walk-through, Inspection
• Tests and acceptance reviews VERIFICATION
Deliverables and Internal Work Products
Description Technical – System Requirement Specification Technical – Software Requirement Specification Technical – System Architecture Design Technical – Software Architecture Design Technical – Interface Control Document Technical – Customer Requirements Specification Technical – Factory Acceptance Test Technical – Site Acceptance Test T h i l Drawing Technical – D i … Project Management – Project Management Plan Project Management – Risk Register Project Management – Project Schedule … User documentation – User Manual Installation User documentation – User Manual Operation User documentation – User Manual Maintenance … Product ‐ Piece of software Product ‐ Piece of hardware Product ‐ System
Peer Review
Peer Review ? ((Y=Yes, , N=No)
Type of Review (P=Personal, D=Desk‐Check, W=Walk‐through, g , I=Inspection)
Tests
Output Documents ANN = Annotations RR = Review Report MoM = Minutes of Meetingg CHKL = Checklist
Type of Test U = Unit I = Integration g S = System
Output Document UTR = Unit Test Report ITR = Integration Test Report g p STR = System Test Report
Type of Test F = Factoryy S = On Site
VALIDATION
ACCEPTANCE AND SIGNATURE ACCEPTANCE AND SIGNATURE
Tests
Acceptance
Output Document FTR = Factory Test Report y p STR = Site Test Report
Internal Approval ((signature) g ) (Y=Yes, N=No)
Delivered to Acceptance required Customer? form Customer? (Y=Yes, N=No) (Y=Yes, N=No)
Y
D, W
RR then MoM
N/A
N/A
N/A
N/A
Y
Y
O
Y
D, W
RR then MoM
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D, W
RR then MoM
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D, W
RR then MoM
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D
RR
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D, W
RR then MoM
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D
RR
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D
RR
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D
RR
N/A
N/A
N/A
N/A
N/Y
N/Y
N/N
Y
D
ANN and/or MoM
N/A
N/A
N/A
N/A
Y, First version
N/Y it depends
N
Y
D
ANN and/or MoM
N/A
N/A
N/A
N/A
N
N
N
Y
W
MoM
N/A
N/A
N/A
N/A
N
N/Y
N/Y It depends
Y
D
RR
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D
RR
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D
RR
N/A
N/A
N/A
N/A
Y
Y
Y
Y
D
CHKL
U, I
UTR, ITR
N/A
N/A
N
N
N
Y
D
CHKL
I
ITR
N/A
N/A
N
N
N
Y
W, I
MoM
I, S
ITS, STR
F, S
FTR, STR
Y for UTR and STR
Y
Y
http://csit.co
46
Selection and Definition of Measures - 1 • “Goal - Question – Metric” Approach Goal
Question
Metric Number of defects per requirement
Improve the level of quality
What is the level of quality ?
What is the progress of the project ? Reduce delivery What is the level of time workload of employee ? Optimize the What is the number of development hours of each phase of time i d l development ?
Number of defects per development phase Effort (hour) to implement, monitor, prevent and correct defects of a product Earned Value (CPI, SPI) Workload per employee as a percentage Number of hours worked per development phase
http://csit.co
47
Selection and Definition of Measures - 2 Measure ID
Measures
Reasons
MET 01 MET-01
Number N b off errors detected d t t d by b document d t type and by phase of the development cycle
To know T k the th overall ll quality lit off eachh work product
MET-02
Number of hours worked for each phase of the system development cycle
MET-03
The cost of each project
MET-04
MET 05 MET-05
MET-06
The attributes of each project: • Number of change requests; • Level of risk; • Predominance hardware/software. Di t ib ti off effort Distribution ff t related l t d to t the th production, review and correction of deliverables Resources spent p versus those that were planned in the project plan
To be able to use the performance of past projects to estimate new projects
To be T b able bl to t analyze l the th efficiency ffi i of processes on product quality To be able to analyze y if the project p j is successful, to identify gaps and take the necessary remedial action 48
Supplier Management Process • • • • •
CSiT is a system integrator having to purchase components from suppliers Basic Profile has only a few tasks about the ‘make make or buy buy’ decisions and follow-up actions (e.g. document, review and issue a purchase order). Basic profile does not describe a supplier management process. – This process is covered in the Intermediate profile The process put in place contains a detailed description about planning and managing acquisitions from suppliers. Addi i l templates Additional l were createdd – Request for proposal, supplier selection matrix, purchase order and purchase agreement (i.e. a contract).
•
N sections New ti have h been b added dd d tto th the ISO 29110 project j t plan l – A list of acquisitions and potential suppliers – An acquisition plan/strategy – A supplier li management plan l Adapté de (Tremblay et col, 2015)
49
Mappings between Frameworks • Mappings between ISO/IEC 29110 and CMMI-DEV
http://csit.co
50
Mappings between Frameworks • Approximate coverage of CSiT processes to CMMI-DEV CMMI-DEV Level 2 Process Areas
Percentage of Coverage
Configuration Management Measurement and Analysis Project Monitoring and Control Project Planning
50 70% 50-70% 20-40% 70-90% 70-90%
Process and Product Quality Assurance
45-65%
Requirements Management
90 100% 90-100%
Supplier Agreement Management
70-90%
http://csit.co
51
ISO/IEC 29110 3rd Party Audit • An audit was requested to obtain Letter of Conformity with the Systems Engineering Basic Profile of ISO/IEC 29110 • Audit Steps 1.
Initiation of audit and analysis of CSIT's process documentation was done by the Auditor Written agreement between the Auditor and CSiT outlined:
2.
• • • • •
3. 4.
Scope of audit Offi i l contacts Official t t Costs Planned dates for filing the documents and the audit itself W di off some elements Wording l t off the th Letter L tt off compliance, li etc. t
Confidentiality Agreement was signed by the auditor Documentation submitted were reviewed by the auditor •
Documentation was complete and satisfactory 52
ISO/IEC 29110 3rd Party Audit •
Audit Steps 5 5.
An on-site on site audit was performed to review work objective evidence •
duration approximately 8 hours
6. Auditor produced an Audit Report with findings and recommendations 7. Report was submitted to a Review Committee for verification •
Review committee examined the documents produced by the auditor
8. A letter was issued by the organization that manages the compliance program attesting the conformity based on ISO/IEC 29110 documents audited •
Letter of compliance subject to period of validity (typically one year)
9 Audit successfully completed in May 2016 9. 53
Benefits to CSiT • Day-to-day Benefits – – – – – – –
Standardized work and consistent deliverables across projects Avoids ‘reinventing the wheel’ for each new project Work is done in a systematic and disciplined way Better quality of internal and external work products Better project management and project monitoring Reduction of project risks Better ette co communication u cat o within t the t e tea team
• Business Benefits – Better credibility to bid on tenders – Access to markets that require demonstration of compliance to a process standard – Better recognition of the quality of work and products – Better trust from customers and business partners – An important step towards a CMMI level 2 A
54
Young Transportation Enterprise • 2016 INCOSE International Symposium • 16-page paper • Edinburgh, July 16-21, 2016
http://profs.etsmtl.ca/claporte/Publications/Pu blications/Setting%20up%20process%20at%2 0CSiT_INCOSE_2016.pdf
55
Translations of Systems Engineering Guides for VSEs • French • Basic Profile translated by Michel Galinier of AFIS and approved by AFNOR • Entry profile is being translated • https://www.afis.fr/ • German • Basic Profile translated by Martin Geisreiter of the German INCOSE Chapter GfSE • http://www.gfse.de http://www gfse de • Basic Profile should be published in 2017 • Arabic • In progress 56
Next Steps – For Systems Engineering • Develop/finalize the remaining systems profiles
Advanced
Intermediate – Intermediate: Management of more than one project Basic – Advanced: Business management and portfolio management practices. Entry
• Develop Profile Groups for other domains – Critical systems: e.g. medical, aerospace
• Development of self-learning course modules to support DPs • Development of plug-in modules (e.g. Eclipse) to support DPs
A
57
A Public Web Site • Members of WG • Introduction • Survey of VSEs • Network of Centers • Generic Profiles Systems engineering Software engineering • Deployment Packages • Pilot Projects • Education DPs • Publications • Certification • Service S i Delivery D li
http://profs.logti.etsmtl.ca/claporte/English/VSE/index.html 58
Summary • ISO 29110 has been specifically developed for VSEs (company, organization, project, department) developing systems and/or software products • ISO 29110 is intended to helpp VSEs who have neither the expertise, nor the budget or the time to adapt existing standards (e.g. ISO 15288, ISO 12207) to their needs • ISO and INCOSE WGs have worked together to develop a set of Systems engineering Management and Engineering Guides and Deployment Packages to help VSEs • ISO 29110 brings g manyy benefits to VSEs,, their clients and their business partners A
59
“As iinnovation “A ti fuels f l economies, i standards smooth the ride” S. Joe Bhatia, President and CEO American National Standards Institute (ANSI)
ISO focus, May–June 2015 H
60
Contact Information • Claude Y Laporte – Voice: V i + 1 514 396 8956 – E-Mail:
[email protected] – Web: http://profs.etsmtl.ca/claporte/English/index.html htt // f t tl / l t /E li h/i d ht l
• Public P bli site it off WG 24 – Free access to Deployment Packages, presentation material and articles: • http://profs.logti.etsmtl.ca/claporte/English/VSE/index.html
62
References •
• • • • • • • • •
•
H
Fanmuy, G., L L’Ingénierie Ingénierie et Management des Systèmes pour les PME/TPE et petits projets, Association Française d'Ingénierie Système (AFIS)/International Council on Systems Engineering (INCOSE), May 24th, 2011, Paris, France. ISO/IEC JTC1/SC7 N3288, New Work Item Proposal – Software Life Cycles for Very Small Enterprises, Mai 2005. ISO/IEC 12207:2008, Information technology – Software life cycle processes, International Organization for Standardization/ International Electrotechnical Commission: Geneva, Switzerland. ISO/IEC 29110 - Lifecycle Profiles for Very Small Entities (VSEs) – Part 1: Overview. International Organization for Standardization/International Electrotechnical Commission: Geneva, Switzerland. ISO/IEC 15289:2006 - Systems and software engineering - Content of systems and software life cycle process information products (Documentation) Kabli, S., Conception, réalisation et mise a l’essai de trousses de déploiement pour faciliter et accélérer l’implémentation de la norme ISO/CEI 20000 par les très petites structures, ÉTS, 2009. K ll P., Kroll, P Kruchten, K h P 2003. P., 2003 The Th Rational R i l Unified U ifi d Process P Made M d Easy: E A Practitioner's P ii ' Guide G id to the h RUP. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA Laporte, C.Y., Alexandre, S., O’Connor, R., A Software Engineering Lifecycle Standard for Very Small Enterprises, in R.V. O’Connor et al. (Eds.): EuroSPI 2008, CCIS 16, pp. 129–141. M ll R., Moll, R Being B i preparedd – A bi bird’s d’ eye view i off SMEs SME and d risk i k management, t ISO F Focus + +, International Organization for Standardization, Geneva, Switzerland, February 2013. Shintani K., « Empowered Engineers are key players in process improvements », Proceedings of the First International Research Workshop for Process Improvement in Small Settings, CMU/SEI-2006Special Report-001, Report 001, p. 115-116, 115 116, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, United States, January 2006. Rogers, Everett M., Diffusion of Innovations, fifth edition, Free Press, New York, 2003.
63
