Data flow in automated testing of the complex automotive electronic control units Velibor Ilić, Srđan Popić, Milan Kovačić
Abstract - Process of development complex electronic control units (ECUs) is usually complex and requires several iterations (releases). After each iteration, it is necessary to perform detailed testing and verification of all components in a testing environment that is as similar as possible to the real system. For this kind of testing it is recommended to use the automated testing environment that generates detailed reports about each segment of tested ECU. This paper describes techniques for secure and automated data flow in process of electronic control units testing. Keywords — Electronic Control Unit (ECU), Framework for Integrated Testing (FIT), Hardware in the Loop (HIL), ECU Test Environment, testing automatization, content management systems (CMS), workgroup support systems (WSS)
I. INTRODUCTION Modern ECUs are capable to simultaneously run several software components. The development process of those ECUs is complex and usually requires several iterations (releases) until it reaches the final version. After each release is conducted detailed testing and verification of all hardware and software components in testing environment that should be as similar as much as possible to the real system in which the ECU will be used. To ensure the proper functionality of complex systems in all possible situations is not simple task. Many of these electronic control units operate in situations where is necessary to take into account the safety of the user and therefore special attention must be paid to the testing and verification of such products [1]. The basic safety standard for the development of the ECU is defined in the document IEC 61508 (International Electrotechnical Commission), the electronic control units used in cars applies to standard ISO 26262 [2,3]. This paper describes methods and techniques that ensure correct data flow in the process without any information duplication or data corruption. Testing electronic control units represents the process that is used to determine the correctness, completeness and regularity in the functioning (functional correctness), and thus ensure the quality of the final product, while the verification procedure includes the systematic review, analysis and testing at all stages of the development cycle. The goal of testing is to detect existing faults in all stages of development[4].
Hardware-in-the-loop (HIL) simulation, or HWIL, is a technique that is used in the development and test of complex real-time embedded systems (ECUs). The main idea behind this concept is that HIL simulation provides an effective platform by adding the complexity of the system within which ECU works, in this case car, under control to the test platform. The ECU to be tested interacts with this “virtual car” in real time.[5,6,7] II. TESTING ENVIRONMENT Testing environment contains several groups of test to evaluate all specific hardware and software features. Unit tests are applied to each software component (module) in a controlled environment to independently test each segment of the system. Such tests should include checking whether each of the components to be tested properly respond to all the expected types of inputs, performance with boundary values for the input and output of data, logic verification, and verification of internal structure at the output data. Integration tests are used to check whether the system components work together as defined in the system design documentation, for example to checks whether the interfaces between the components are properly defined and implemented. Functional testing is a part of quality assurance (QA), those tests are used to evaluate functionality of the whole system and to confirm that the whole system really works as it is defined in the specification required. Performance testing represents group of tests developed to evaluate tested ECUs performs in terms of responsiveness and stability under a particular workload. This group of test also used to investigate, measure, validate and verify system reliability and resource usage. Acceptance testing represent group of tests perform to verify whether the ECUs meet all the necessary requirements defined in the project documentation. Looking from the process perspective there are four processes:
Velibor Ilić, RT-RK Institute for Computer Based Systems, Novi Sad, Serbia (e-mail:
[email protected]). Srđan Popić, RT-RK Institute for Computer Based Systems, Novi Sad, Serbia (e-mail:
[email protected]). Milan Kovačić, RT-RK Institute for Computer Based Systems, Novi Sad, Serbia (e-mail:
[email protected]).
Application Acceptance Test (AAT), which is Acceptance testing, Application Integration Test (AIT) evaluate performance of all software components (SWCs) that running at ECU, System Integration Test (SIT) that does all other testing with all SWCs on ECU. Finally, Integration Report (IR) is non-testing process that aggregates and integrates all test results into one Report.
Figure 1 shows how data is flowing through the Testing environment.
The process of automatic testing one of releases contains following steps: Preliminary phase: • Downloading software components (SWCs) that are tested on the ECU subsystem to deliver the software, • Update the accompanying test tools, configuration files and scripts from this subsystem for version tracking code, • Configure the input/output parameters based on a database to automate the process, • load ECU software on the ECU to be tested. Testing and diagnostics: • Acceptance test – Are neccesary requirements met? • Application Integration Test – Compare testing values against the expected on the one SWC level. • System Integration Test – Compare testing values against the expected on the ECU level. • Aggregation and integration of all test results.
Fig. 1. Testing process
III. ECU VALIDATION ENVIRONMENT Figure 2 shows structure of the system and the hardware assembly for testing electronic control units[8,9,10,11]. This environment allows: • diagnostics of electronic control units, • reproducibility of tests [12], • management and control of the flow testing, • storage and analysis of test results, • generation of reports and interpretation of test results.
Reports and results analysis: • Reports are created in XML and PDF formats. Report in PDF format is simple copy of the XML report. Source of all relevant information is XML report. This report is computer readable. • Problems detected during active test are recorded in the monitoring subsystem. • Send notification to engineers responsible for the development of software components where the problems detected. • Test results are statistically analyzed and generate summary reports such as the review of integrated software components, a review of test results, graphs allocated resources and results measurement. Final destination of all test data is database, but during the process of testing data is traveling through the XML reports. XML format is suitable for conversion in any other document format, and on the other hand, it is computer readable[13,14]. This ensures test data to be correct and uncorrupt.
Fig. 2. ECU validation environment
Environment for the validation of electronic control units, are shown at Figure 2, consists of: • • •
• • •
ECU that is subject of testing, Variant-a switch that allows connection of multiple ECUs with the HIL, HIL devices that simulate the system within which the ECU and intended to be installed and working (working) after validation, Logger-whose role is to capture traffic between HIL and the ECU. Test your computer is responsible for carrying out tests. Databases whose role is storage and organization of the results of testing, but an integral part of the lists of the current problems, as well as information pertaining to the results of the tests.
IV. CONCLUSION Automation of the testing process is highly recommended because that reduces human error during testing process, automation of testing is achieved by using programming languages or scripts. Information shared in a testing environment must be unambiguous, and easy-toread for computer as well as for human. Documents in XML format are suitable to be used as pipeline for data flow in complex process of ECU testing. XML format is convenient because it is easy to write/read by any script or programming languages. Another advantage of using XML documents is that they can be easily converted into reports in more human readable formats such as HTML, PDF, CSV or TXT. ACKNOWLEDGMENTS This work was partially supported by the Ministry of Education, Science and Technological Development of the Republic of Serbia, under grant number: III44009-1.
REFERENCES [1] [2]
[3]
[4] [5] [6] [7] [8]
[9]
[10] [11]
[12]
[13] [14]
Marco A.A. Sanvido, Vaclav Cechticky, Walter Schaufelberger (2002) "Testing embedded control systems using hardware in the loop simulation and temporal logic" Alberico D., Bozarth J., Brown M., Gill J., Mattern S., McKinlay A., (1999), “Software System Safety Handbook - A Technical Managerial Team Approach”, Joint Services Computer Resources Management Group. Dezfuli H., Benjamin A., Everett C., Smith C., Stamatelatos M, Youngblood R, (2011) “NASA System Safety Handbook, Volume 1, System Safety Framework and Concepts for Implementation”, National Aeronautics and Space Administration NASA Headquarters, Washington, D.C. 20546 Matijević K. (2013), “Neke metode za automatizovano testiranje softvera”, Master Rad, Matematički Fakultet, Beograd National Instruments (2014), "Building Flexible, Cost-Effective ECU Test Systems", http://www.ni.com/white-paper/3064/en/ Ebert C., Jones C., (2009) "Embedded Software: Facts, Figures and Future", Published by the IEEE Computer Society Wahlstrom M., Falcone F., Nelson D., (2010), “Integrating Hardware-In-The-Loop Into University Automotive Engineering Programs”, American Society for Engineering Education Pap M., Vujanović V., Ilić V., Dimitrijević A., (2015) "Okruženje za validaciju ispravnog rada elektronskih upravljačkih jedinica", 59. ETRAN, Konferencija za Elektroniku, Telekumunikacije, Računarstvo, Automatiku i Nuklearnu Tehniku, Srbija, Srebrno Jezero Tešić V., Pap M., Ilić V., Nedeljković M. (2015) "Automatizacija testiranja elektronskih upravljačkih jedinica u HIL namenskom okruženju", 23nd Telecommunications Forum (TELFOR 2015), Beograd, Srbija Tischer M., Widmann D., (2012) "Model based testing and Hardware in the Loop simulation of embedded CANopen control devices", CAN in Automation, Cordemans P., Landschoot S., Boydens J., Steegmans E. (2014) "Test-Driven Development as a Reliable Embedded Software Engineering Practice", M. A. Khan et al. (eds.), Embedded and Real Time System Development: A Software Engineering Perspective, Studies in Computational Intelligence 520, ©Springer-Verlag Berlin Heidelberg Sawant A. A., Bari P. H., Chawan P. M. (2012) "Software Testing Techniques and Strategies", International Journal of Engineering Research and Applications (IJERA), Vol. 2, Issue 3, May, ISSN: 2248 H. S. Thompson, D. Beech, M. Maloney, and N. Mendelsohn, "XML schema part 1: Structures second edition," W3C Recommendation, Oct. 2004. T. Bray, J. Paoli, C. M. Sperberg-McQueen, E. Maler and F. Yergeau, “Extensible markup language (XML) 1.0”, Fifth Edition, November, 2008.
