The System Safety Assessment by the Use of Programming Tools during the Licensing Process
S. A. Vilkomir, Ph.D.; State Center on Nuclear and Radiation Safety; Kharkov, Ukraine
V. S. Kharchenko, Prof.; State Center on Nuclear and Radiation Safety; Kharkov, Ukraine
A. S. Ponomaryev, Prof.; Kharkov Polytechnic University; Kharkov, Ukraine
A. L. Gorda; Kharkov Polytechnic University, Kharkov; Ukraine
Keywords: safety assessment, control system, software, licensing, tools
Abstract
Programming tools for a software safety assessment support during licensing of digital systems are addressed. Expediency and necessity of the use of such tools, its functions, and methodology of the software safety assessment are considered. The paper is based on the experience with the ExpertPro tool, developed by Ukrainian State Scientific and Technical Center on Nuclear and Radiation Safety for the review of digital control systems at nuclear power plants (NPPs).
Introduction
For safety related digital control systems in critical areas, the process of licensing and the safety system assessment is usually carried out by state regulatory bodies or independent experts. One of the basic parts of this process is the assessment of software safety. It is a complex labour-consuming task and it is expediently to use the special programming tools for its decision.
It is possible to divide such tools into two classes. The tools for the decision of specific tasks at separate stages of the assessment concern to one class. There are, for example, tools of the software static and dynamic analysis, for example, MALPAS (ref. 1), LDRA Testbed (ref. 2), and programs of reliability estimation of
digital systems (ref. 3). Within the framework of solving tasks, such tools have the universal character and can be used by the various companies with licensing various systems.
The tools which automate and support the software review at all stages of its life cycle concern to other class. Such tools do not have the universal character and are developed by licensing organizations for the internal use. The cause is that the tasks and functions of such tools directly depend on used methodology of the assessment. As the example, it is possible to mention the database named ISSUES-7, which was developed by Czech Republic State Office for Nuclear Safety for licensing of NPP Temelin safety related digital systems (ref. 4).
The present report addresses the basic principles of construction and use of the tools of the second class, i.e. tools of the automated software safety assessment support (SSAS tools) during licensing and review. The preliminary version of such system, named ExpertPro, has been developed by Ukrainian State Scientific and Technical Center on Nuclear and Radiation Safety (SSTC NRS) for the software safety assessment support of digital control systems at nuclear power plants. Basing on the this experience, the report addresses the following: - substantiation of necessity of the SSAS tools use; - functions of SSAS tools; - features of methodology of the software safety assessment used in ExpertPro;
- features and formalized models of a number of tasks of ExpertPro.
life cycle, the kind of software etc. The use of SSAS tools could allow to automate separate operations with the review, in particular, tracing of performance of the requirements to software.
Necessity of the SSAS Tools Use Functions of SSAS Tools The following reasons cause expediency and necessity of the SSAS tools use for the important for safety systems software assessment:
1. The documentation on each system usually has a large volume. The specific requirements are imposed upon sets of the documents at each stage of the software life cycle (specification, design documentation, plans and reports on verification). The use of SSAS tools could allow to formalize process of the assessment and to transform it into a step-by-step procedure of a "question - answer" type.
2. The process of licensing and the safety assessment is extended in time and is carried out in parallel with software development. The use of SSAS tools could allow to receive information about the current condition of the carried out assessment at any moment (lists by the outstanding requirements to software, unprovided at present documentation or information, current assessments by separate criteria etc.). In turn, such opportunity could supply, if necessary, the inclusion of new experts in the current process of the assessment.
3. The experts participate in realization of the review of various systems in various time. Some of such assessments can be interconnected, for example, for systems constructed on uniform hardware units or for systems which fulfil identical functions. Therefore, expert has necessity to come back to the received results from time to time. The use of SSAS tools could allow to accumulate the information received with the safety assessment of various systems and to use accumulated experience in future.
4. Manually tracing of performance of all software requirements at all stages of the development cycle is a labour-consuming task. One and the same requirement can be reflected in many documents depending on the stage of the
In view of specified reasons, SSAS tools functions can be divided into: - functions of information support; - analytical functions (functions of the analysis); - functions of planning and documenting; - functions of the assessment of results.
The functions of information support are: - functions of the storage and updating of information on the normative base of the safety assessment (standards, norms, rules, instructions etc); - functions of the systematization and storage of information on the materials, submitted on the review; - functions of the storage of information on the fulfilment and current results of the assessment.
The second group of SSAS tools functions consists of functions of the analytical character for checking of software conformity to established criteria and requirements. For example, there are the following functions: - the direct and reverse examination of conformity of results received during the assessment at all stages of the software life cycle, from the software requirements up to the reports on software testing (verification); - analysis of completeness and quality of realization of the software requirements etc.
The functions of the third group fulfil: - the development of the general and individual plans of the software safety assessment; - distribution of resources (temporary, personnel, tools) on the set of the assessment objects (on stages of the software life cycle, on analyzed materials etc.); - development and the issue of intermediate and final reports on carried out assessments.
The functions of the fourth group support the fulfilment of particular and final safety assessments. The particular assessments are formed on the basis of the analysis of information about conformity of the system to the separate requirements, and the final assessment is formed on the basis of generalization of particular assessments according to the established criteria.
developer's documentation on all stages of the development and verification correspond to the requirements of the standards, norms and rules, and also there is a necessary operational documentation. The criteria of documentation and completeness are interconnected. Thus, by the criterion of completeness the substantial aspect, and by criterion of documentation the formal aspect are estimated.
Methodology of the Software Safety Assessment
Software satisfies the criterion of intelligibility, if the documentation on the software development and verification is stated in the form, clear and intelligible to the experts who are not directly participating in their realization. In addition, the traceability of the performance of the requirements to software at the various life cycle stages should be provided.
The realization of the considered above functions in each specific SSAS tool depends on used methodology of the safety assessment. The methodology, used in SSTC NRS, is based on the requirements of the international and Ukrainian standards and normative documents (ref. 5), and also on the experience of carried out by SSTC NRS safety assessments of digital control systems at Ukrainian NPPs (ref. 6). Among the international standards, the standard IEC-880-86 (ref. 7) is used wide. The rules of a number of documents ISO, IEEE, IAEA are used also. Among the valid in Ukraine standards, GOST 29075-91 (ref. 8) and GOST 24.104-85 (ref. 9) are used. These standards contain separate sections with the requirements to software of digital control systems.
For the software safety assessment, the set of the following criteria is used: completeness, documentation, intelligibility, independence and conformity.
Software satisfies the criterion of completeness, if: - its specifications completely correspond to the specifications of the system; - all functional requirements to software are reflected in the project; - software corresponds to the general requirements, common to software of all important for safety systems, including the requirements to designing and verification; - the performance of all software functions is checked up with testing.
Software satisfies the criterion of documentation, if the composition and structure of the
Software satisfies the criterion of independence, if a degree of the independence of software verification corresponds to a safety class of the system. For the most critical systems, verification should be carried out by the group of the experts (organization), administratively and financially independent on the experts (organization), developing software. For less critical systems, the realization of the development and verification by the different experts is recommended, however the administrative and financial independence is not required.
Software satisfies the criterion of conformity, if verification has been successfully completed before putting the system into operation, i.e. if all found out defects have been analysed and are eliminated by this moment (or the reasonable decision on their further elimination has been accepted).
One or several criteria can be applied with the safety assessment at each stage of the software life cycle (development of the specifications, designing, coding, verification). The application of criteria also depends on a safety class of a system - the more influence of the system on safety, the more detailed assessment should be carried out, i.e. more criteria should be used at each stage.
The detailed elaboration of the software safety assessment is achieved by division of each criterion into a number of subcriteria, and then each subcriterion into the set of the particular requirements. The separate expert assessment should be carried out under each requirement. All remarks of the experts are transferred to the software developer for operative elimination (updating of the documentation, realization of additional tests etc.).
possible erroneous actions of an expert (wrong data input etc.).
The realization of some functions of ExpertPro has required the formal description (modeling) of the process of the software safety assessment. The function of the completeness analysis and the function of formation of the final generalized assessment are the most representative in this sense.
Feature of the Expert Pro Tool
Described above methodology is a basis of the preliminary version of SSAS tool named ExpertPro, developed in SSTC NRS. The ratio "criterion - stage of life cycle" is realized in ExpertPro by the set of the pop-up menu. When the current stage of the assessment and the safety class of the system are chosen, the menu opens access only to the allowable set of criteria.
The ratio "criterion - subcriterion - requirement" is realized by the set of the display forms. The display form on each subctiterion contains its formulation, references on normative documents, the field for the expert conclusion and the list for a choice of a formal assessment on the threemark system (satisfied - satisfied partially - not satisfied). This formal assessment is automatically entered in the table reflected the current condition of the all process of the review.
The possibility to call the list of the requirements included in the cubcriterion is provided. The data prepared by the expert at one stage are automatically transferred in the display forms of the next stage for their assessment. For example, the list of software functions prepared according to project documents becomes a basis for the assessment at the stage of the software verification review.
One of the basic purpose of program realization of ExpertPro has been the creation of the most convenient and friendly interface. This interface should on the one hand allow an expert to carry out a safety assessment in any order, convenient for him, and on the other hand be steady against
The model of the completeness analysis function is based on representation of the process both results of software development and verification as the multilevel graph, which nodes correspond to checked elements (requirement to software, set of software functions, testing techniques, verification reports etc.), and its edges specify the relations between these elements. The relation between elements can be ambiguous, i.e. one element on the previous evaluation stage can correspond with a few elements at the subsequent stage and on the contrary. Such kind of relation is taken into account for the following pairs of estimated elements: 1. requirement to system - requirement to software; 2. requirement to software - software function; 3. software function - testing plant; 4. testing plan - testing report.
The direct and reverse tracing of chains from entrance elements (requirement to system) up to target elements (reports on verification) is carried out with the use of this model. The subset of target elements appropriate to each entrance element is established with direct tracing. The subset of entrance elements appropriate to each target element is established with reverse tracing. Besides, this model allows to check up completeness of performance of the system requirements by integration of results of the direct tracing on all entrance elements.
The model of the function of formation of the final assessment is represented by the multilevel graph of tree-type, the number of levels is determined by hierarchy of estimated elements. There are minimum three levels - criterion, subcriterion, requirement. This model also sets rules of formation of the assessment for each
element on a basis earlier established assessments of the lower level elements, connected to it. Consistently applying these rules, the general (final) software assessment is formed.
will allow to carry out the review of software of digital systems at Ukrainian NPPs on a higher scientific and methodical level.
References Conclusions
Considered in the present report functions of the tools of the software safety assessment support can be realized in each specific SSAS tool in view of used methodology of the safety assessment. Developed on the basis of methodology used at Ukrainian State Scientific and Technical Center on Nuclear and Radiation Safety, the ExpertPro tool helps the expert to carry out the assessment at all stages of the software life cycle.
The first experience of the use of ExpertPro has been received at SSTC NRS by the realization of the software safety assessment of Safety Parameters Display System (SPDS) of Westinghouse (USA) at Zaporozhskaya-5 NPP and Khmelnitskaya-1 NPP and Turbine Control System of Shevchenko Plant (Ukraine) for Zaporozhye-1 NPP.
The received experience has shown the certain difficulty of the use of the tool on the initial stages of review connected with necessity to input of the initial information about the reviewed documents. At the subsequent stages, the application ExpertPro considerably facilitated realization of the assessment. The application of the ExpertPro is more expedient for more complex system, when the big volume of safety assessment is necessary. The greatest difficulties by development of anyone SSAS tool are caused because the absence of uniformity in development stages, used methods, and issued documents during software designing for various important for safety digital systems. It is especially characteristic for systems developed in various countries, as the development is conducted on a different normative basis. The further use of ExpertPro with the software safety assessment for various systems will allow to take into account the accumulated experience during development of the next version of this tool. It
1. TACS/1019/N7. User Guide for MALPAS Release 6.0. TA Consultancy Services Limited. The Barbican, East Street, Farnham, Surrey GU9 7TB: December 1992. 2. M. A. Hennell, D. Hedley, I. J. Riddell. The LDRA Software Testbeds: Their Roles and Capabilities. Proceedings of IEEE Soft-Fair' 83 Conference, Arlington, Virginia, July 1983. IEEE catalog no. 83CH1919-0. 3. S. Vilkomir, S. Vinogradskaya and M. Yastrebenetsky. Software Package for Control Systems of Power-Stations Reliability Estimation. Thermal Engineering (Teploenergetica), num. 2, 1993, pp. 35-37. (In Russian). 4. C. Karpeta. Licensing Aspects of the NPP Temelin I&C Replacement Project. Proceedings of the International Topical Meeting on VVER Instrumentation and Control, April 21-24, 1997, Congress Centre, Prague, Czech Republic, pp. 137 - 144. 5. G. Zhidok, M. Yastrebenetsky, and S. Vilkomir. Legislative Policy and Standards Preparedness for Licensing Process of NPP's I&C Systems. Proceedings of the International Topical Meeting on VVER Instrumentation and Control, April 21-24, 1997, Congress Centre, Prague, Czech Republic, pp. 113-122. 6. S. Vilkomir, G. Zhidok. Experience of Licensing of Software for Digital Safety Related Systems in Ukraine. Project Control for 2000 and Beyond. Proceedings of ESCOM-ENCRESS 98, 27-29 May 1998, Rome, Italy, pp. 328-331. 7. IEC Std. 880. Software for Computers in the Safety Systems of Nuclear Power Stations: 1986. 8. GOST-29075. Nuclear instrumentation systems for nuclear power stations. General requirements. Moscow: 1991. (In Russian). 9. GOST-24.104. Unified system of standards of computer control systems. Computer control systems. General requirements. Moscow: 1985. (In Russian).
Biographies
S. A. Vilkomir, Ph.D., Leading Researcher, State Scientific and Technical Center on Nuclear and Radiation Safety, 17 Artema St., Kharkov, 310002, Ukraine, telephone - (380-572) 471700, facsimile - (380-572) 471-700, e-mail
[email protected].
Dr. Vilkomir is a state expert on safety and reliability of computer control systems at Ukrainian NPPs. His research interests include safety analysis of digital control systems, software reliability and quality assessment.
V. S. Kharchenko, Prof., Leading Researcher, State Scientific and Technical Center on Nuclear and Radiation Safety, 17 Artema St., Kharkov, 310002, Ukraine, telephone - (380-572) 471700, facsimile - (380-572) 471-700, e-mail
[email protected].
Prof. Kharchenko is also a chief of department of Kharkov Military University. His research interests include software reliability and quality assessment, fault tolerant systems, based on multiversion design technologies.
A. S. Ponomaryev, Prof., Kharkov Polytechnic University, 21 Frunze St., Kharkov, 310002, Ukraine, telephone - (380-572) 400-474.
Prof. Ponomaryev is a head of the nuclear systems specialty of Control System department. His research interests include power system analysis. A. L. Gorda, graduand, Kharkov Polytechnic University, 21 Frunze St., Kharkov, 310002, Ukraine, telephone - (380-572) 400-474.
Mr. Gorda received the MS in Software Engineering from Kharkov Polytechnic University in 1999. His areas of interest are programming and development of digital systems.
