Code Coverage in Eclipse - Eclipse Wiki

0 downloads 333 Views 316KB Size Report
Sep 10, 2010 - How complete are your tests? Code Coverage in Eclipse ... Code Coverage Tools ... Integration into CI Ser
How complete are your tests?

Code Coverage in Eclipse Joachim Hofer

09/10/10

Code Coverage in Eclipse

1 / 10

Outline ●

Motivation



Types of Code Coverage



Case Study:



09/10/10



Testing TestBench



Tool Requirements

Code Coverage Tools

Code Coverage in Eclipse

2 / 10

Why Code Coverage? (1) ●

Tests need goals! ●

Black Box: – –



White Box: – –

09/10/10

Requirement Coverage System Tests

Code Coverage Integration/Unit Tests

Code Coverage in Eclipse

3 / 10

Why Code Coverage? (2) ●



Not for optimizing yet another code metric! ●

Easy to increase in all the wrong ways...



False sense of security

What can go wrong with developer tests? ●

… don't test what they are intended to



… won't get executed



… can be elaborate in the wrong places

→ Analyze their coverage! 09/10/10

Code Coverage in Eclipse

4 / 10

Types of Code Coverage ●



Coverage Types ●

Statement Coverage



Branch Coverage



Condition Coverage



Path Coverage

Not good enough!

Often good enough... Difficult...

Usually impractical.

Instrumentation Types ● ●

09/10/10

Based on Source Code Based on Bytecode

Usually recommended, but...

JVM multi-language support!

Code Coverage in Eclipse

5 / 10

Case Study: Testing TestBench ●

System Test (QF-Test, TestBench) ●

~16k tests run every night



Takes 10 hours



Video capturing (due to popular request: done via vnc und vnc2flv)



Integration/Unit Test (TestNG) ●

> 1000 tests run after each commit



Takes 10 minutes



Complemented by FindBugs

09/10/10

Code Coverage in Eclipse

6 / 10

Case Study: Tool Requirements ●

Measures Branch Coverage



Available via Command Line for Build



Integration into CI Server (Hudson et al)



Eclipse Plug-in Available



Based on Bytecode



Free if possible...

09/10/10

Code Coverage in Eclipse

7 / 10

Code Coverage Tools looks good



CodeCover



Emma / EclEmma



Clover



Cobertura

looks good

not bytecode-based

no branch coverage

not free

no Eclipse plug-in

→ eCobertura Eclipse plug-in

09/10/10

no CI integration

Code Coverage in Eclipse

yay!

still pretty basic

8 / 10

Links ●



Eclipse Plug-ins ●

eCobertura - http://ecobertura.johoop.de



EclEmma - http://eclemma.org/



CodeCover - http://codecover.org/



Clover - http://www.atlassian.com/software/clover/

Command Line / Ant ●

Cobertura - http://cobertura.sourceforge.net/



Emma - http://emma.sourceforge.net/



JaCoCo - http://www.eclemma.org/jacoco/

09/10/10

Code Coverage in Eclipse

9 / 10

Thanks!

Any questions?

09/10/10

Code Coverage in Eclipse

10 / 10