On Building Testable Software Components Co-authors: Jerry Gao, Ph.D., Kamal Gupta, Shalini Gupta, and Simon Shim, Ph.D.
Presented by: Jerry Gao, Ph.D.
San Jose State University One Washington Square San Jose, CA 95192-0180 Email:
[email protected] URL: http://www.engr.sjsu.edu/gaojerry
Feb. 5, 2002
ICCBSS’2002
1
Topic: On Building Testable Components
Presentation Outline • Introduction • Understanding component testability • Evaluation of component testing capability • Mechanisms to increase component testability • What is a testable bean? • Why do we need testable beans? • Design for testable beans • Advantages and issues • Technology used and status • Conclusion
Feb. 5, 2002
ICCBSS’2002
2
Topic: On Building Testable Software Components
Introduction Observations: –The quality of a component-based software highly depends on the quality of its components, specially reusable software components. –The testability of component-based software highly depends on the testability of software components. ---> testing software components is very important ---> component quality assurance and certification is very important
Problems: –Lack of research results on software component testing such as test models, adequate testing criteria, test methods –Lack of research and experience on design for component testability –Lack of tools for supporting test automation of software components - Lack of research and experience on quality control of components
Feb. 5, 2002
ICCBSS’2002
3
Topic: On Building Testable Software Components
Introduction Our major focus ---> how to build testable software components. We focus our study to answer the following questions: –What is a testable software component? –How to evaluate component testability? –How to increase component testability during design phase? –How to construct testable software components systematically? –How to facilitate component testing in a systematic way?
Component engineering needs to concern testing and maintenance issues in the following areas: –Component model –Component-based infrastructure –Component-based framework
Feb. 5, 2002
ICCBSS’2002
4
Topic: On Building Testable Software Components
Understanding Component Testability Basic concept: In general, testability is a program’s property. A software with high testability suggests that faults could be easily revealed during testing.
•Jeffrey Voas: “Testability is an important indicator for component quality and reliability”
•Roy S. Freedman consider two perspectives of testability: a) Observability and Traceability b) Controllability He defines domain testability as follows: “Domain testability is defined by specifying the semantics of expression evaluation and command execution”
Feb. 5, 2002
ICCBSS’2002
5
Topic: On Building Testable Software Components
Understanding Component Testability Our perspectives of component testability: –understandability –observability and traceability –controllability In our view: component testability indicates how well a component is structured to facilitate software testing, including component auto-test, component integration, and component-based software testing. The major issue on component testability in current component engineering practices: –Poor component testability
Feb. 5, 2002
ICCBSS’2002
6
Topic: On Building Testable Components
Evaluation of Component Testing Capability Level 1: Initial - Constructed with ad-hoc testing mechanism, test formats & functions - More time in understanding behaviors, debugging & testing Level 2: Standardized - Built to support pre-defined testing mechanism, test format - Reduces cost on debugging & testing - Extra programming overhead Level 3: Systematic - Designed with a set of systematic testing mechanism - Easy to monitor and test the components - Reduced programming overhead Level 4: Customizable - Designed to facilitate the support of testing functions & customization - Helps to set up testing for component based software
Feb. 5, 2002
ICCBSS’2002
7
Topic: On Building Testable Components
Mechanisms to Increase Component Testability Three Mechanisms to construct testable components: - Framework-based testing facility - well-defined framework (such as class library) to add test code - simple and flexible to use - needs component source code - Built-in tests - needs well-defined built-in mechanism to add test code - high programming overhead during component development - no external support needed - Automatic component wrapping for testing - component wrapped inside program for testing - low programming overhead - well-defined test framework to interact with test tools
Feb. 5, 2002
ICCBSS’2002
8
Topic: On Building Testable Components
Testability enhancement mechanisms Testing P erspectives Source C ode Testing C ode Separation D evelopment O verhead for Testing C omplexity U sage F lexibility A pplicable C omponents
Feb. 5, 2002
.
F rameworkB ased Testing F acility N eeded
B uilt-in Tests
Automatic C omponent W rapping for Testing
N eeded
Not needed
No
No
Yes
H igh
H igh
L ow
L ow
V ery H igh
H igh
H igh
L ow
L ow
In-house components
In-house components
In-hou se compo nents and C O TS
ICCBSS’2002
9
Topic: On Building Testable Components
What is a Testable Bean? •What is a testable bean? –should be deployable and executable - must be traceable by supporting basic component tracking capability - must provide a consistent, well-defined and built-in interfaces - standardized mechanism to include built-in test code •Design criteria while constructing testable beans: – minimize programming overhead for developers – standardize the test interface for testable beans – prevent adding detailed testing functions inside beans – separate the functional code from built-in test code – should not include detailed tests inside a test bean
Feb. 5, 2002
ICCBSS’2002
10
Topic: On Building Testable Components
Why Testable Beans? •Why do we need testable beans?
- reduce test costs and enhance software quality - automate component tests •Objectives to design and construct testable beans
- provide traceable, reusable, executable, deployable, testable beans - provide plug-in-and-test environment - standardized the test interface for testable beans - achieve the component test automation - construct software testable components with good architecture
Feb. 5, 2002
ICCBSS’2002
11
Topic: On Building Testable Components
Challenges in Building Testable Beans –Design and implementation in a systematic way? –Define component architecture and test interface? –Create Testing framework to support test automation? –Minimize programming efforts and system overhead? –Provide standardized interactions between Bean and its supporting environment?
Feb. 5, 2002
ICCBSS’2002
12
Topic: On Building Testable Components
Design for Testable Beans •Our solution includes: –a new architecture model –a well-structured test interface –a supporting test environment
•The architecture model: –GUI interface supporting user interaction (optional) –Built-in test interface –Built-in program layer to facilitate testing –Built-in tracking interface for tracking capabilities –Built-in tracking program layer to facilitate tracking
Feb. 5, 2002
ICCBSS’2002
13
Topic: On Building Testable Components
Testable Bean Architecture
Testing Interface Built-in testing code Component Built-in tracking code Tracking Interface
Feb. 5, 2002
ICCBSS’2002
14
Topic: On Building Testable Components
Test Interface •Test interface supports three basic functions:
- set up a test (test case and/or data) - exercise a test for a specified function - validate and report the test results •We define a structured test interface as follows: –setParameters, to set up the parameters for the functions –runMethod, to execute a given method –validateTestResult, to validate the test results
Feb. 5, 2002
ICCBSS’2002
15
Topic: On Building Testable Components
Testable Component Architectures
Built-in Test
F1
Tm
(a) Component with Built-in Test
Feb. 5, 2002
Test Interface
F1
T1 Component
Fn
Component Test Driver
Component
Test Interface Fn
(b) Component Interacting with A Test Driver
ICCBSS’2002
16
Topic: On Building Testable Components
Component Testing Environment A supporting test environment includes five parts: •Testable beans (testable components) •Test agent (along with a testing package) •Component test repository •Tracking agent (along with a tracking package) •Component tracking repository Features of a component testing support environment:
- a test management tool for tests - a component test bed to set-up and exercise test - a component test generation tool to generate test cases
Feb. 5, 2002
ICCBSS’2002
17
Topic: On Building Testable Components
Testing and Support Environment
Testable Bean
Component-based Program
RMI
Network
Test Agent
Tracking Agent Component Test Database
Component Test Tool
Feb. 5, 2002
ICCBSS’2002
Component Trace Repository
18
Topic: On Building Testable Components
Transaction Sequence Testable Bean
RMI Naming Service
Test Agent (BeanTest Agent) Register with the naming service
Bind to Test Agent
Proxy Sever
Remote Tester Interface tester.setParameter (……) Remote call setParameter (……) Return Object[ ] Remote call runMethod (……) tester.runMethod (……) Return Run Status Remote call runMethod (……)
tester.validateTestResult(……)
Return Validated Result
Transaction Sequence between a Test Agent and a Testable Bean
Feb. 5, 2002
ICCBSS’2002
19
Topic: On Building Testable Components
Advantages and Issues •Advantages: - Reduces software test cost and enhances software quality - Minimizes testing cost and efforts - Provides a scaleable model - Flexible to control testing capabilities - Provides a well structured testing package - Monitors external behavior of third-party components •Other issues: –Component testability analysis and fault analysis –Component integration –Software maintenance and support for component-based software –Component behavior analysis and performance evaluation
Feb. 5, 2002
ICCBSS’2002
20
Topic: On Building Testable Components
Technology used and Status
•Technology Used: - JDK 1.2 for Testing Client and Testing Agent - RMI for the communication •Implementation Status: -A simple prototype tool has partially built, some class-level examples have been verified the suggested model. •Future Work: –Come up with a complete component test-bed technology, applicable to support diverse components based on different languages, technologies, and platforms.
Feb. 5, 2002
ICCBSS’2002
21
Topic: On Building Testable Components
Conclusion What presented? –Introduce the concept of testable beans for component design and testing. –Provide an idea and present approaches to build testable beans. –Present our idea and implementation for a test environment for testable components. –Apply the examples using our prototype environment. •Future work: – Build a reusable component test bed to support script-based component tests at the black-box level. – Design for component integration concerning component model, component infrastructure.
Feb. 5, 2002
ICCBSS’2002
22
Topic: Design for Testability of Software Components
References [1] Elaine J Weyuker, "Testing Component-Based Software: A Cautionary Tale", IEEE Software, September/October 1998. [2] Jerry Z. Gao, "Testing Component-Based Software", STAR’99. [3] Jerry Z. Gao, "Perspectives of Software Component Testability", submitted for publication in 1999. [4] Jerry Gao, et al, "Monitoring Behaviors of Software Components", accepted by Journal of Object-Oriented Programming. [5] Voas, J. M. and Miller, Keith W.," Software Testability: The New Verification", IEEE Software, Vol. 12, No. 3: May 1995, pp. 17-28. [6] Roy S. Freedman, “Testability of Software Components”, IEEE Transactions on Software Engineering, Vol. 17, No. 6, June 1991. [7] David S. Rosenblum, “Adequate Testing of Component-Based Software”, Department of Information and Computer Science, University of California, Irvine, Technical Report UCI-ICS-97-34, Aug. 1997. [8] William T. Councill, "Third-Party Testing and the Quality of Software Components", IEEE Software, Vol. 16, No. 4, pp. 55-57, July/August 1999. [9] Yingxu Wang, Graham King, and Hakan Wickburg, “A Method for Built-in Tests in Component-based Software Maintenance”, Proceedings of the Third European Conference on Software Maintenance and Reengineering, 1998.
Feb. 5, 2002
ICCBSS’2002
23
Topic: Design for Testability of Software Components
Q&A
Contact Information: Jerry Gao, Ph.D. Dept. of Computer Engineering San Jose State University Email:
[email protected] URL: http://www.engr.sjsu.edu/gaojerry Feb. 5, 2002
ICCBSS’2002
24
