Design, Development and Qualification of Class IB. Software: A Case Study. Yogesh Nirgude, Ashutosh Kabra and Gopinath Karmakar. Reactor Control Division ...
2015 International Conference
on Industrial Instrumentation and Control (ICIC) Col/ege ofEngineering Pune, India. May28-30,2015
Design, Development and Qualification of Class IB Software: A Case Study Yogesh Nirgude, Ashutosh Kabra and Gopinath Karmakar Reactor Control Division, Bhabha Atomic Research Centre, Mumbai, India {ynirgude, kabra, gkarma}@barc.gov.in Abstractdevelopment
In
this
paper,
we
present
of software for c1ass-m
a
case
study
systems of a
fulfill those objectives. This work is an extension to our recent work [3] aimed at formulating such an approach for qualification. In this paper we present a case study to show how this approach is realised in a real-world application.
on
nuclear
reactor. International standards and engineering procedures do exist for development of software for c1ass-m instrumentation and control (I&C) systems. Conformance to these standards is the primary requirement for software to be subjected to c1ass-m
B. ERM Hardware
qualification. However, there is a gap between these standards
ERM was built around microcontroller (89V51RD2) based custom embedded hardware with 32 KB memory. ERM has membrane keypad for entering configurable parameters and alphanumeric LCD display to show positions of control rod, shut-off rod drop times and drop time history of shut-off rods.
and software development process. For any software developer, clearly defined objectives and a step b y step procedure to fulfIll those objectives to meet the c1ass-m qualification is required. The case study presented in this paper narrates a systematic and reviewable engineering process, which meets the identified objectives necessary for claiming a software worthy of c1ass-m systems of nuclear reactors.
n.
Keywords- SDLC, drop time, software development, class-IB I.
INTRODUCTION
Electronic Rod Monitor (ERM) is a microcontroller based system for calculating and displaying position of control rods in a research reactor. In case of unwarranted movements of rods, ERM generates outputs for the rod drive logic system required for necessary safety interlocks. In addition, ERM also measures drop time of shut-off rods (drops under gravity for reactor trip) and drive down time of control rods necessary for detecting undesirable conditions like stuck rod. ERM is identified as category-B/class-IE system as per the categorisation criteria of IEC-61226 [1]. A class-IE system is also called a safety related system, because failure of it leads to invocation of safety system (category-A/class-IA). ERM helps preventing unrestricted withdrawal of control rods, failure of which can cause actuation of reactor protection system(s), which is a safety system.
TABLE
I.
COMPARlSON OF IE C STANDARDS
Clauses
IEC-62138
Selection
Says
and
preferable to use a
use
that
it
is tool
IEC-60880
Remarks
It
Elaboration
provides
guidelines
of
well-known
software
with
tools
operational
of
experience.
requirements
extensive
of
specification
detailed for selection tools,
of
tools,
configuration of tools, etc.
A. Motivation
Testing
Development of class-IE Computer Based System requires strict adherence to software engineering procedure and AERB SG D-25 guidelines [2] applicable for such systems. There exist international standards and engineering procedures for software development of class-IE I&C systems. Conformance to these standards is the primary requirement for software to be subjected to class-IE qualification. However, there is a gap between these standards and software development process. From a developer's perspective what is required is i) a set of well-defined objectives, fulfilment of which can be claimed for its worthiness in class-IE computer based system of Indian Nuclear Power Plants (NPP) ii) a step by step procedure to
978-1-4799-7165-7/15/$31.00 ©2015 IEEE
SOFrWARE DEVELOPMENT
IEC-62138 [4] is the applicable standard for class-IE software. But we find that IEC-62138 can be considered as a subset of IEC-60880 [5], the standard for class-IA software development. This is because in most of the cases, either i) IEC-60880 elaborates the guidelines also specified in IEC62138 or, ii) it requires documentary evidence of a software development activity that is otherwise also required in development of class-IE software. In support of this claim, only a few examples are given in Table-I due to space constraint.
Does
not
It
specifies
i) Black box testing must for any
specifically mention
the
is a
about software unit
requirements
software.
of black box
ii) Any dependable
test and integration testing.
But
specifies
the
requirement system
it of
validation
well
white testing.
as box
software development demands white box testing
for
IEC-60880
ensuring compliance
as
of
integrated software.
which requires
documentary evidence.
Therefore, we followed IEC-60880 only for development of this class-IE software. We identified the various objectives and deliverables in each software development phase along
800
with the respective applicable working standards as shown in the table-II. TABLE II.
SDLe Phase
Guidelines. SQAP was prepared based on standard IEEE 730 [6] identifying the standards, practices, conventions to be used for software quality control in all Software Development Life Cycle (SDLC) phases. SCMP was prepared based on IEEE 828 [7] to provide guidelines for establishing and implementing configuration management for the ERM software development. SVVP describing the verification and validation (V&V) process for the SDLC was prepared based on standard IEEE 1012 [8]. MISRA-C [10] programming guideline (PG) is followed for this software. This is because, i) it is a safe subset of C followed in safety critical software development in automotive industries and ii) MISRA-C rules checkers are available from multiple vendors as a part of static analysis tools. It may be noted here that without automated tools, it is highly difficult and time consuming to check compliance with PG. We met all the objectives of planning phase identified in table-II by preparing all deliverables viz. SQAP (wherein MISRA-C was specified as PG), SCMP, and SVVP documents as per their respective working standards.
OBJECTIVES AND APPLICABLE WORKING STANDARDS IN SDLC PHASES Sr. No 1
Objectives/Deliverables Quality
Software
Working Standards
Assurance
IEEE STD 730
Configuration
IEEE STD 828
Plan (SQAP) 2
Software
Management Plan (SCMP) 3 Planning
Software
and
Verification
IEEE STDIOl2
Validation Plan (SVYP) 4
Programming
Example: MISRA-C,
Guidelines (PG)
2004
is
acceptable
in
Indian
nuclear
domain.
S
Requirements
Software
IEEE STD 830
Specifications (SRS) 6
RequirementAnalysis
on
Depends
B. Requirements and Requirements Analysis
selected specification
Requirements
language.
(this
work
used
UML ) 7
Traceability
Part of SRS.
toUserlSystemRequirements 8
Software Architectural Design
IEEE
(SAD)
1016
STD can
be for
referred architectural viewpoints.
Design 9
Detailed
Software
Design
(SOD)
IEEE
STD
1016
Design of dynamic behaviour Traceability to SRS 10
As
Compliance to PG
PG
per
(MISRA-C) 11
As
Source Code
per
the
Figure l.Use Case Diagram of ERM software
language lmplementati on
selected 12
Static
analysis
compliance
for
As
quality
PG
report
with
metrics
specified
(e.g.,
acceptable
in
SQAP
The main goal of this phase is the generation of software requirements and carrying out requirement analysis using formal or application oriented language with traceability to user requirements. In this section we present how the use case analysis helped us in finding incomplete and infeasible requirements. AERB/SG/D-25 recommends use of computer aided software engineering tools in SDLC phases. We used CASE Tool Rhapsody version 6.1 for UML modelling [9] and requirement analysis. After studying the requirements, we carried out functional grouping of the requirements and developed the use case diagram as shown in Fig. l. ERM has four use cases viz., ERM PROC, UI, POST and DIAGNOSTICS as follows. i) ERM_PROC: It receives encoder inputs from rod drive mechanism and computes the positions of control rods. In case of unwarranted movements of rods, it sends output, to the rod drive mechanism for necessary interlocking actions. In addition, it also measures drive down times of control rods and drop times of shut off rods.
specifiedin
and/orSQAP
nesting
depth, complexity, comments per lines of code etc.) 13
Software U nit Test Plan and
IEEE
Report (S UTP/R)
1008 and IEEE STD 829
Traceability to SOD Statement Coverage,
STD
Branch
Coverage Testing
14
Software Integration Test Plan
IEEE STD 829
and Report (SITP/R) Functional Testing Traceability to SOD Code
Coverage,
Branch
Coverage
A. Software Development Planning Phase
The objectives in planning phase are preparation of i) Software Quality Assurance Plan (SQAP) ii) Software Configuration Management Plan (SCMP) iii) Software Verification & Validation Plan (SVVP) and iv) Programming
801
Further, administrative decision of deploying software on a newly built hardware demanded assessment of the capability and performance of hardware. In order to meet the performance requirements like i) position computation of control rod moving at the speed of 2mm/second and ii) drop time of shut off rods with a resolution :s 10 ms, we find that prototype development approach is most suitable for this work.
ii) UI: This use case takes input from user through the keypad and sends the demanded information to ERM_PROC for configuration and to the LCD for displaying the computed results. iii) POST: It performs power on self-test. iv) DIGNOSTICS: It carries out diagnostic tests for both hardware and software. ERM has three actors viz.,field_io,led and keypad. Actor keypad and led interacts with UI use case, whereas actor field_io interacts with ERM PROC, UI, POST and DIAGNOSTICS. Requirements document of ERM not only includes functional requirements but also provide performance requirements quantifying response time, drop time resolution and display update time as a result of prototyping. Software requirements specifications (SRS) of ERM were documented as per IEEE-830 standard [12].
2) Prototyping: This first ERM software prototype was developed for position computation of control rods. Position computation algorithm acquires encoder pulse inputs in real time and computes current position by comparing with previous encoder inputs. Integrated Testing of prototype ERM was carried out in two steps. Since encoder pulses finally generate a set of contact inputs corresponding to rod position, we simulated encoder pulse with toggle switch contacts. Other inputs like bottom limit positions, RAISE, LOWER, TEST commands and TR IP status were also simulated by toggle switches. After prototype test of position computation, we integrated it with a test control rod facility where encoder inputs of actual rod were connected to ERM prototype. We carried out multiple test runs (raising and lowering of rod) of prototype with the test control rod. To our surprise, we observed that position indicated by ERM prototype was lesser than actual rod travel. Our investigation led to the conclusion that ERM prototype was missing a few encoder pulses as discussed below.
1) Incomplete and Infeasible User Requirements: Here we present a few examples of incomplete and infeasible user requirements revealed during requirements analysis. Though implementation is usually not discussed during requirement analysis phase, we find that feasibility of user requirement should be considered during this phase itself as discussed below. i) The initialisation requirement pertaining to interaction between the actor field_io and the use case ERM_PROC was not specified by the user. ERM is supposed to compute the rod position based on its previous position and encoder pulses, but it was not clear that what position ERM will show on power up. After clarification from user, rod positions are initialized to a value corresponding to their respective bottom limit switch (BLS) positions on power up. ii) There was a user requirement of tolerating failure of one out of four encoder switches and still provide correct rod position. On study of this requirement, it was found that this was not feasible. We could do away with this fancy user requirement at the requirement analysis phase before getting into design. Subsequently, scope of this requirement was made limited only to detection of the fault in encoder switch. iii) User wanted uploading of drop time profile to PC. But, it was found that the hardware does not have provision of communication facility. Therefore, requirement of drop time profile upload to PC was eliminated from this release of ERM software. Some grey areas in the user requirements were also revealed as follows. i) ERM acquires contact inputs generated by magnetic encoders, which measure positions of the control rods with a resolution of 0.48 mm. But, rate of change of these inputs was not known. Control rod speed is limited to 2 mm/sec. It was necessary to know the rate of change of encoder inputs to decide the input sampling frequency so that distance travelled by the rod at maximum speed can be measured correctly. ii) Contact de-bouncing time is necessary to avoid spurious inputs. Since input de-bouncing time was not known, the requirement of input validation could not be specified quantitatively. The above findings during requirement analysis led us to go for a prototyping approach for this software development.
3) Benefits derived from Prototyping: ERM software receives encoder inputs only after it is converted to equi valent set of contact inputs. Even for fastest moving rod, encoder inputs (pulses) persists at least for 20 ms. But due to settling delay of relays, it may be possible to miss one pulse with scan cycle of 10 ms and de-bouncing check for 2 scan. Therefore, position computation algorithm should account for the same. Due to practical limitations, replacement of encoders and its contact generation module was not feasible. It led to change in position computation algorithm in ERM prototype software to take care of missing pulses so that ERM calculates correct position. We also improved the simulation technique in order to i) simulate missing pulses ii) movement of control rods at their actual speed of 2 mm/sec. Simulator was also programmed to provide configurability in drop time simulation with a resolution of 10 ms. In this section, we showed that how prototyping helps even in improving the software requirements itself before design. C. Design Phase
Objectives in this phase include producing software architectural design and software detailed design. Identifying objects in ERM software is required as first step towards software design. A sequence flow diagram (SFD) is an instance of a use case showing a particular path of behavior and it helps in identifying objects. An SFD consists of a set of objects and it shows a sequence of messages exchanged between the objects to meet a particular system requirement. For example, stuck
802
rod detection is a functional requirement of ERM software pertaining to use case ERM PROC. Sequence flow diagram of stuck rod detection helped us to identify the objects ERM]ROC, FIELD_IO and UI as shown in fig. 2. For stuck rod check, object ERM_PROC interacts with FIELD_IO to get the current encoder position. Stuck rod state is identified by a scenario where there is no change in rod position within predefined time interval in presence of up/down command. ERM_
acquire inputsl send
0 utputs
read keys
UI
PROC
Figure 3 Object Model Diagram of ERM software stuck
iv) LCD: It displays messages on LCD. KEYPAD: It scans keypad to detect if any key is v) pressed. We realized that objects like ERM_PROC, UI are acti ve objects. An active object is an object that owns a process or thread and can initiate control activity. Having more than one active objects call for multiple threads in implementation which requires operating system for its temporal management. We will discuss about how we managed multiple threads without RTOS during implementation. Fig. 4 shows statechart diagram of ERM software which models the dynamic behaviour of a system. Various states mentioned in the state chart diagram are as follows. 1) INIT_LCD: This is the initial state of ERM-ENG when LCD is initialized. 2) POST: This state performs Power on S elf Test of ERM. 3) INIT_ERM: This state initializes all configurable parameters of ERM to their default values. 4) UI: This state periodically reads keypad (user input) and display information (based on user request) on LCD. 5) DIAGNOS: This state performs diagnostics periodically to detect failures of the system. 6) ERM_PROC: This state i) validates and acquires field inputs, ii) Calculate current position of all control rods, iii) actuate interlocking outputs based on position or Maximum Allowable Mismatch (MAM) between any two rods, iv) compute drop times of all shutoff rods and drive down time of control rods and v) report error conditions. 7) HALT: This state display error message on LCD and stops the processing of ERM. This state is reached on failure condition in POST or DIAGNOS states as shown in fig.3.
Figure 2 Sequence Diagram of Stuck Rod Check
Sequence diagrams helped us to achieve complete functional test coverage during integrated testing. For example, in fig. 2 various possible test scenarios which could be derived are i) if rod moves within predefined interval after raise or lower command ii) rod moves after stuck rod condition is declared iii) raise/lower command withdrawn before stuck rod time interval is expired but rod did not move. Various objects identified with the help of SFDs and their dependencies are presented in fig. 3. Object Model Diagram representing structural design of ERM software have following objects. i) ERM_PROC: It receives encoder inputs from rod drive mechanism and compute position of rods. In case of unwarranted movements of rods, it sends necessary output that is fed to the rod drive mechanism for necessary interlocking actions. In addition, it also measures drive down time of control rods and drop time of shut off rods. ii) UI: This component takes input from user (using keypad) and send demanded information to LCD component. Otherwise, it continuously displays position of control rods. iii) FIELD_IO: It reads the field inputs and sends field outputs.
803
D. Implementation Phase
Deli verables of this phase include source code, MISRA-C compliance report of source code and static analysis report for compliance with quality metrices specified in SQAP. MISRA C guidelines [10] were followed for the implementation of ERM software using Keil compiler under I1Vision3 V3.33 Integrated Development Environment. While discussing sequence diagram, we observed that this software has more than one active objects. Note that ERM is a microcontroller based system with limited memory and processing capability. Therefore, deploying RTOS was not a practical solution to deal with parallel execution of two acti ve objects. Therefore, we followed timed interrupt based approach to implement parallel executions. In our case, functionality of ERM_PROC object which is very time critical (response time :S 30 ms) was implemented in foreground (i.e. within interrupt service routine of timer). UI object being less time critical (since display refresh time was specified as 500 ms) was implemented in background. We incorporated a timer interrupt of 10 ms out of which 6 ms were taken by processing of ERM_PROC object in interrupt service routine. Thus 4 ms were available out of each 10 ms slot for the functionality of UI object which requires around 200 ms. We used static analysis tool ' LDRA Testbed' to check for MISRA-C compliance. It also computed various software quality metrices including cyclomatic complexity mentioned in SQAP. MISRA-C violations reported by tool were examined. We corrected the necessary MISRA-C violations pointed out by the tool. However, 100% MISRA-C compliance could not be achieved since compiler meant for microcontroller family has some language extensions beyond ANSI-C that led to the violations. Therefore, we produced a report justifying all the unavoidable violations of MISRA-C, the adopted PG. Use of static analyser tool helped us to detect all those instances of variables which were used before initialisation. Since our compiler did not provide error or warning in such cases, their presence in the code could have led to unpredictable results. During implementation we observed that floating point calculations were taking very large time for processing. Therefore, in order to meet the response time requirement, floating point computations were minimised without violating the other requirements specifications. We met all the objectives of implementation phase mentioned in table-II by producing source code and its static analysis report as well as MISRA-C compliance report.
Figure 4 State Chart Diagram of ERM_ENG object
When a functional requirement is to be achieved by multiple steps with conditional dependencies, activity diagram is used. We have shown an example of activity diagram of ELOCK functionality in ERM software. ELOCK means generating output which prevents further upward movement of control rod if it surpasses a pre-set value (called ELOCK
[eG currentpositio its ELOCK positi nJ
queued in message list
>=
G currentposition ElOCKposition]
< IS
RST
queued in
Figure 5.Activity Diagram of ELOCK checking As shown in fig. 5, when position of control rod is equal to or above ELOCK position and elock_flag is not set before then elock_flag is set and the corresponding output is sent with ELOCK message displayed. If ELOCK condition already exists, it is reset when rod position is below ELOCK position and elock_flag is already set. When ELOCK is reset, it reset corresponding output sent and display ELOCK reset message. Increased readability of functional requirements helped in reduction in mistakes during implementation. Thus we met all the objectives of design phase mentioned in table-II by preparing software detailed design document (which also includes architectural design) as per IEEE 1016 standard [14] including traceability to software requirements as deliverables.
£.
Testing Phase
Objecti ve of this phase is to carry out software unit testing and system integration testing including traceability to design and to provide statistics of testing in the form of code coverage. Unit testing of ERM software was carried out and the test plan, procedures & reports were documented based on IEEE 1008 [13]. Unit test was prepared for each entity, which is part of SDD. Each test case was traceable to SDD. It may be noted that automated checking of code coverage is preferred. But code coverage was manually confirmed due to unavailability of communication port to integrate with automated test tool.
804
software change authorisation was generated including impact analysis of software change, staff and time required to incorporate designated software change. Software change was carried out and list of changed files and procedures were documented in software modification report. Modified software was tested in lab and test report confirming correct incorporation of software change was generated. All the documentation and source code was submitted to review committee. After clearance of review committee, the same software change was incorporated at site. A good design made these software changes easier due to faster analysis of ripple effect and review of software changes.
System integration test plan was prepared to test all the requirements specified in SRS. Plan and procedure for integration tests were documented based on IEEE 829 [11]. System integrated testing of ERM software was carried out in the development lab using PLC based input simulator. As mentioned earlier, use case diagram helped us in deriving test cases, since they depict how the field and user would interact with the system to accomplish various functions. Each test case represented one specific scenario of interaction with the system, within the context of one of the essential use cases. We produced software unit test plan/report and system integrated test plan/report as per applicable standards. Thus, all the objectives of this phase mentioned in table-II were met.
V. CONCLUSION We presented the case study of a class-IE software development from scratch. We identified 14 key objectives and proposed that fulfilment of these minimum number of objectives followed by review at each phase of the SDLC process can be considered for claiming a software worthy of class-IE reactor systems. We established that when requirements are not clearly known, prototyping helps generating precise requirements. We also showed that when new hardware is used, it is preferable to prototype to bring out performance parameters correctly and precisely. As a result of prototyping, overall development time of ERM software was reduced. Further we showed that manual code review is also a necessity to find out likely grey areas in the software which are not otherwise revealed by testing.
SOFTWARE REVIEW An independent team not related to development activities of ERM was appointed for review of the software. ill.
A. Key Review Comments and Discussions
Following important code changes were carried out based on review comments. i) ERM software allows changing of configurable parameters in non-volatile memory (flash) so that those values are retained even after power off. Reviewers pointed out that the section of code performing flash write should be done atomically due to the following reason. If interrupt occurs in midway of write operation, parameter values are likely to get corrupted. Therefore, writing data into flash was protected by disabling interrupts. ii) During the process of integration of various modules of ERM software, a few pieces of dead codes were left in the ERM software. During manual code inspection, review team found the dead codes which could not have revealed even after testing. Such dead codes were removed.
References [I]
control important to safety - Classification of instrumentation and control Functions. International Electro technical commission, (2009-07) [2]
IV. A.
IEC 61226 (Ed. 3.0): Nuclear Power Plants - Instrumentation and
COMMISSIONING AND MAINTENANCE OF ERM
AERB Safety Guide for Computer based Systems of Pressurised Heavy Water Reactor No. AERBISG/D-25, January 2010
[3]
Commissioning Experience
AERB SG 0-25 and IEC 60880 for certification of Software in Safety Systems of Indian NPP;
Gopinath Karmakar and
Yogesh Nirgude;
Proceedings of International Conference on VLSl, Communication,
After clearance from review committee, ERM was commissioned at the research reactor site. Unexpectedly, nothing worked in the first run and as it happens, the first suspect was the software. But troubleshooting revealed that the sequence of encoder pulses connected was not correct due to wrong wiring in the field. This led to invalid positions reported by the ERM. After wiring rectification, control rods were raised and lowered to upper and lower limits at its rated speed and ERM was found to correctly showing position. It shows that following good practices produces good software.
Advanced Devices, Signals
& Systems and Networking (VCASAN-
2013), Volume 258, 2013, pp 309-316. [4]
IEC-62138, Software aspects for computer based systems performing category B or C functions, first edition, 2004-01
[5]
IEC-60880, Software aspects for computer based systems performing category A functions, second edition, 2006-05
[6]
IEEE Standard for Software Quality Assurance Plans, IEEE Std 730,
[7]
IEEE Standard for Software Configuration Management Plans, IEEE Std
(2002). 828, (1998). [8]
IEEE Standard for Software Verification and Validation, IEEE Std 1012, (1998).
B. Requirements Change based on Preliminary Experience
[9]
UML 2.0: Infrastructure
& Superstructure: Object Management Group,
(2005)
Based on user's feedback ERM software was required to be changed. Software change was initiated because of calibration of rod assembly and its effect on encoder resolution. Another change was initiated due to change in shut off rod drop time range which was kept with lesser margin.
[12] IEEE Recommended Practice for Software Requirements Specifications,
e. Incorporating Requirements Change Requests
[13] IEEE Standard for Software Unit Testing, IEEE Std 1008-1987 (RI993),
[10]
MISRA-C:
Guidelines for the Use of the C Language in Critical
Systems. The Motor Industry Software Reliability Association, UK, (2004) [II] IEEE Standard for Software Test Documentation, IEEE Std 829, (1998). IEEE Std 830, (1998). (1993).
Software requirement changes in ERM were carried out as per procedure specified in SCMP. All Requirements Change Requests (RCR) were formally generated. Based on RCR,
[14] IEEE Recommended Practice for Software Design Descriptions, IEEE Std 1016, (1998).
805
