Test Suite Evolution - ICST 2012

4 downloads 208 Views 828KB Size Report
Test Suite Evolution. S.T.A.R. @. Faculty of Informatics. University of Lugano. Switzerland. Supporting through Test Case Adaptation. Fabrizio. Pastore. Mehdi.
S.T.A.R. @ Faculty of Informatics University of Lugano Switzerland

Supporting

Test Suite Evolution through Test Case Adaptation

Mehdi Mirzaaghaei

Fabrizio Pastore

Mauro Pezzè

public final class DateTimeField{ .. public final class CopticChronology{ .. }

Test Evolution

} public class Report{ .. addRule(int line, String file) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); }

JodaTime r1.62

public final class DateTimeField{ .. public final class CopticChronology{ .. }

Test Evolution

} public class Report{ .. addRule(int line, Context String file) ctx) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); }

JodaTime r1.62

public final class DateTimeField{ .. public final class CopticChronology{ .. }

Test Evolution

} public class Report{ .. addRule(int line, Context String file) ctx) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); ReportTest{ test(){ Report r = new Report(); r.addRule(5, new Context( file ) ); assertFalse(r.isEmpty());

}

}

JodaTime r1.62

public final class DateTimeField{ .. public final class CopticChronology{ .. }

Test Evolution

public final class EthiopicChronology{ ..

}

public class Report{ } .. addRule(int line, Context String file) ctx) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); ReportTest{

EthipicChronologyTest{ .. }

test(){ Report r = new Report(); r.addRule(5, new Context( file ) ); assertFalse(r.isEmpty());

}

}

JodaTime r1.62

public final class DateTimeField{ .. public final class CopticChronology{ .. } }

Test Evolution

public final class EthiopicChronology{ .. public class Report{ } .. addRule(int line, Context String file) ctx) }

CopticChronologyTest{ ReportTest{ EthipicChronologyTest{ Chronology COPTIC_UTC test(){ = CopticChronology.getInstanceUTC(); .. Report r =1,new DateTime epoch = new DateTime(1, 1, Report(); 0, 0, 0, 0, COPTIC_UTC); r.addRule(5, file ); long millis = epoch.getMillis(); } assertFalse(r.isEmpty()); ReportTest{ long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis(); test(){ DateTimeField monthOfYear = COPTIC_UTC.monthOfYear(); } Report r = new Report(); r.addRule(5, new Context( file ) ); while (millis < end) { assertFalse(r.isEmpty()); int monthValue = monthOfYear.get(millis); if (monthValue < 1 || monthValue > 13) fail("Bad month: }" + millis); }

JodaTime r1.62

public final class DateTimeField{ .. public final class CopticChronology{ .. } }

Test Evolution

Reuse Information

public final class EthiopicChronology{ ..

public class Report{ } .. addRule(int line, Context String file) ctx) }

available in

Existing Test Cases

CopticChronologyTest{ ReportTest{ EthipicChronologyTest{ Chronology COPTIC_UTC test(){ = CopticChronology.getInstanceUTC(); .. Report r =1,new DateTime epoch = new DateTime(1, 1, Report(); 0, 0, 0, 0, COPTIC_UTC); r.addRule(5, file ); long millis = epoch.getMillis(); } assertFalse(r.isEmpty()); ReportTest{ long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis();

to

Evolve Test Suites

test(){ DateTimeField monthOfYear = COPTIC_UTC.monthOfYear(); } Report r = new Report(); r.addRule(5, new Context( file ) ); while (millis < end) { assertFalse(r.isEmpty()); int monthValue = monthOfYear.get(millis); if (monthValue < 1 || monthValue > 13) fail("Bad month: }" + millis); }

JodaTime r1.62

EthiopicChronologyTest{

Test Evolution

public final class DateTimeField{ Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); .. DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); public long final class =CopticChronology{ millis epoch.getMillis(); .. } long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis(); DateTimeField monthOfYear = ETHIOPIC_UTC.monthOfYear(); public final class EthiopicChronology{ } .. while (millis < end) { int monthValue public class Report{= monthOfYear.get( new Date(millis) ); } if (monthValue < 1 || monthValue > 13) .. fail("Bad month: " + millis); addRule(int line, Context String file) ctx) } } CopticChronologyTest{ ReportTest{ EthipicChronologyTest{ Chronology COPTIC_UTC test(){ = CopticChronology.getInstanceUTC(); .. Report r =1,new DateTime epoch = new DateTime(1, 1, Report(); 0, 0, 0, 0, COPTIC_UTC); r.addRule(5, file ); long millis = epoch.getMillis(); } assertFalse(r.isEmpty()); ReportTest{ long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis(); test(){ DateTimeField monthOfYear = COPTIC_UTC.monthOfYear(); } Report r = new Report(); r.addRule(5, new Context( file ) ); while (millis < end) { assertFalse(r.isEmpty()); int monthValue = monthOfYear.get(millis); if (monthValue < 1 || monthValue > 13) fail("Bad month: }" + millis); }

JodaTime r1.62

Test Care Assistant (TCA) Add Test Cases for new Modules

Repair Compilation Errors due to Signature Changes

new Class in Hierarchy

new Interface Implementation

Parameter Addition

Parameter Type Change

New Overloaded Method

New Overridden Method

Parameter Removal

Return Type Change

Test Care Assistant (TCA) Add Test Cases for new Modules

Repair Compilation Errors due to Signature Changes

new Class in Hierarchy

new Interface Implementation

Parameter Addition

Parameter Type Change

New Overloaded Method

New Overridden Method

Parameter Removal

Return Type Change

Test Care Assistant Add Test Cases for new Modules

Repair Compilation Errors in Signature Changes

new Class in Hierarchy

new Interface Implementation

Parameter Addition

Parameter Type Change

New Overloaded Method

New Overridden Method

Parameter Removal

Return Type Change

Generate Test Cases for Classes added to a Hierarchy Identify & Copy Candidate Test Cases Adapt Test Cases Update References to Class Under Test Repair Compilation Errors Repair Runtime Errors

Remove Redundant Test Cases

Generate Test Cases for Classes added to a Hierarchy Identify & Copy Candidate Test Cases Adapt Test Cases Update References to Class Under Test Repair Compilation Errors Repair Runtime Errors

Remove Redundant Test Cases

Identify & Copy Candidate Test Cases Chronology BasicChrono GJChrono Julian

Gregorian

BuddhistChrono

FixedMonthChrono Coptic...

Ethiopic...

JulianTest

GregorianTest

CopticTest

BuddhistTest

testCalendar()

testCalendar() testMax()

testCalendar() testEpoch()

testCalendar() testEpoch() JodaTime r1.62

Identify & Copy Candidate Test Cases Chronology BasicChrono GJChrono Julian

Gregorian

BuddhistChrono

FixedMonthChrono Coptic...

Ethiopic...

JulianTest

GregorianTest

CopticTest

BuddhistTest

testCalendar()

testCalendar() testMax()

testCalendar() testEpoch()

testCalendar() testEpoch() JodaTime r1.62

Identify & Copy Candidate Test Cases Chronology BasicChrono GJChrono Julian

Gregorian

EthiopicTest

BuddhistChrono

FixedMonthChrono Coptic...

Ethiopic...

testCalendar_1() testCalendar_2() testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()

JulianTest

GregorianTest

CopticTest

BuddhistTest

testCalendar()

testCalendar() testMax()

testCalendar() testEpoch()

testCalendar() testEpoch() JodaTime r1.62

Identify & Copy Candidate Test Cases Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian

Gregorian

FixedMonthChrono Coptic...

Ethiopic...

testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()

JulianTest

GregorianTest

CopticTest

BuddhistTest

testCalendar()

testCalendar() testMax()

testCalendar() testEpoch()

testCalendar() testEpoch() JodaTime r1.62

Update References to Class Under Test Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); 31111111 assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian

JulianTest

Gregorian

GregorianTest

FixedMonthChrono Coptic...

Ethiopic...

CopticTest

testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()

BuddhistTest

Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); testCalendar() testCalendar() testCalendar() testCalendar() DateTime epochtestMax() = new DateTime(1, 1, 1, 0, 0, 0, testEpoch() 0, COPTIC_UTC); testEpoch() assertEquals( CopticChronology.AM, epoch.getEra() ) ... JodaTime r1.62

Update References to Class Under Test Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); 31111111 assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian

Gregorian

FixedMonthChrono Coptic...

Ethiopic...

testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()

EthiopicChronology CopticTest BuddhistTest Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); testCalendar() testCalendar() testCalendar() testCalendar() DateTime epochtestMax() = new DateTime(1, 1, 1, 0, 0, 0, testEpoch() 0, COPTIC_UTC); testEpoch() assertEquals( CopticChronology.AM, epoch.getEra() ) EthiopicChronology ... JodaTime r1.62 JulianTest

GregorianTest

Update References to Class Under Test Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); 31111111 assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian

Gregorian

FixedMonthChrono Coptic...

Ethiopic...

testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()

ETHIOPIC CopticTest GregorianTest BuddhistTest Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); EthiopicChronology.getInstanceUTC(); ETHIOPIC testCalendar() testCalendar() testCalendar() testCalendar() DateTime epochtestMax() = new DateTime(1, 1, 1, 0, 0, 0, testEpoch() 0, COPTIC_UTC); testEpoch() assertEquals( EthiopicChronology.AM, CopticChronology.AM, epoch.getEra() epoch.getEra()) ) ... JodaTime r1.62 JulianTest

Adapt Candidate Test Cases

Undefined Fields

Missing Constructors

Missing Methods

Adapting Undefined Fields Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); assertEquals( CopticChronology.AM, epoch.getEra() ) ... FixedMonthChrono CopticChronology +AM

EthiopicChronology +EE

Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )

Adapting Undefined Fields Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); assertEquals( CopticChronology.AM, epoch.getEra() ) ... FixedMonthChrono CopticChronology +AM

EthiopicChronology +EE

Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )

Adapting Undefined Fields

class CopticChronology {

class EthiopicChronology {

/** Serialization lock */ long serialVersionUID = -59L;

/** Serialization lock */ long serialVersionUID = -59L;

Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;

DateTimeConstants.CE;

/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");

+AM

+EE

Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )

Adapting Undefined Fields

class CopticChronology {

class EthiopicChronology {

/** Serialization lock */ long serialVersionUID = -59L;

/** Serialization lock */ long serialVersionUID = -59L;

Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;

DateTimeConstants.CE;

/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");

+AM

+EE

Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )

Adapting Undefined Fields

class CopticChronology {

class EthiopicChronology {

/** Serialization lock */ long serialVersionUID = -59L;

/** Serialization lock */ long serialVersionUID = -59L;

Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;

DateTimeConstants.CE;

/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");

+AM

+EE

Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )

Adapting Undefined Fields

class CopticChronology {

class EthiopicChronology {

/** Serialization lock */ long serialVersionUID = -59L;

/** Serialization lock */ long serialVersionUID = -59L;

Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;

DateTimeConstants.CE;

/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");

+AM

+EE

Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )

EE

Adapting Undefined Constructors Output GraphicOutput public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }

SVGOutput public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1

Adapting Undefined Constructors Output GraphicOutput

SVGOutput

public void testDefaultFont() { public void testDefaultFont() { g = new GraphicsMock(); g = new GraphicsMock(); output = new SVGOutput( output = new GraphicsOutput( g, g, DefaultEnv.DEFAULT_FONT, DefaultEnv.DEFAULT_FONT, fgColour, fgColour, bgColour); bgColour); [..] [..] assertEquals("graphicsOutput", assertEquals("graphicsOutput", Compilation error: output.toString()); output.toString()); The constructor } } SVGOutput(Graphics2D, Font,

Color, Color) is undefined Barbecue 1.5b1

Adapting Undefined Constructors Output GraphicOutput public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, GraphicOutput( fgColour, Graphics2D, bgColour); Font font, [..] Color fgColor, assertEquals("BLA", Color bgColor) output.toString()); }

SVGOutput SVGOutput(

public void testDefaultFont() { Writer writer, g = new GraphicsMock(); font, outputFont = new SVGOutput( g,Color fgColor, Color bgColor, DefaultEnv.DEFAULT_FONT, double scalar, fgColour, String units ) bgColour); [..] assertEquals("BLA", output.toString()); SVGOutput( } Writer writer,

Font fonts)

Adapting Undefined Constructors Output GraphicOutput public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, GraphicOutput( fgColour, Graphics2D, bgColour); Font font, [..] Color fgColor, assertEquals("BLA", Color bgColor) output.toString()); }

SVGOutput SVGOutput(

public void testDefaultFont() { Writer writer, g = new GraphicsMock(); font, outputFont = new SVGOutput( g,Color fgColor, Color bgColor, DefaultEnv.DEFAULT_FONT, double scalar, fgColour, String units ) bgColour); [..] assertEquals("BLA", output.toString()); SVGOutput( } Writer writer,

Font fonts)

Adapting Undefined Constructors Output GraphicOutput

SVGOutput SVGOutput(

public void testDefaultFont() { public void testDefaultFont() { Writer writer, g = new GraphicsMock(); g = new GraphicsMock(); +2 font, outputFont = new SVGOutput( output = new GraphicsOutput( g,Color fgColor, +2 g, Score=6 Color bgColor,+2 DefaultEnv.DEFAULT_FONT, DefaultEnv.DEFAULT_FONT, GraphicOutput( double scalar, fgColour, fgColour, Graphics2D, String units ) bgColour); bgColour); Font font, [..] [..] Color fgColor, assertEquals("BLA", assertEquals("BLA", Color bgColor) Score=2 output.toString()); SVGOutput( output.toString()); } } Writer writer,

Font fonts) +2

Adapting UndefinedSVGOutput( Constructors Writer writer, Font font, Output Color fgColor, Color bgColor, double scalar, GraphicOutput SVGOutput String units ) public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }

public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( ...., DefaultEnv.DEFAULT_FONT, fgColour, bgColour, ..., ...; [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1

Adapting UndefinedSVGOutput( Constructors Writer writer, Font font, Output Color fgColor, Color bgColor, double scalar, GraphicOutput SVGOutput String units ) public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }

public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( ...., new StringWriter( EasyMock(Writer) ... ) ), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, ..., ...; [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1

Adapting UndefinedSVGOutput( Constructors Writer writer, Font font, Output Color fgColor, Color bgColor, double scalar, GraphicOutput SVGOutput String units ) public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }

public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( ...., new StringWriter( EasyMock(Writer) ... ) ), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, 0..., , “”...; ); [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1

Adapting Runtime Failures Test Failures: apply ReAssert

Daniel et. al.[ASE09,ISSTA10]

public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( new StringWriter(EasyMock(Writer)), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, 0, “”; [..] assertEquals("graphicsOutput", output.toString()); }

Adapting Runtime Failures Test Failures: apply ReAssert

Daniel et. al.[ASE09,ISSTA10]

public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( junit.framework.AssertionFailedError: new StringWriter(EasyMock(Writer)), expected: but DefaultEnv.DEFAULT_FONT, was: fgColour, bgColour, 0, “”; [..] assertEquals("graphicsOutput", output.toString()); }

Adapting Runtime Failures Test Failures: apply ReAssert

Daniel et. al.[ASE09,ISSTA10]

public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( new StringWriter(EasyMock(Writer)), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, 0, “”; "SVGOutput" [..] assertEquals("graphicsOutput", output.toString()); }

Adapting Runtime Failures Uncaught Exceptions: involve users

Add Test Cases Identify & Copy Candidate Test Cases Adapt Test Cases Update References to Class Under Test Repair Compilation Errors Repair Runtime Errors

Remove Redundant Test Cases

Test Care Assistant Add Test Cases for new Modules

Repair Compilation Errors in Signature Changes

new Class in Hierarchy

new Interface Implementation

Parameter Addition

Parameter Type Change

New Overloaded Method

New Overridden Method

Parameter Removal

Return Type Change

Repair Signature Changes BankAccount deposit(int cents)

BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());

BankAccount deposit(Money cents)

Repair Signature Changes BankAccount deposit(int cents)

BankAccount deposit(Money cents) Compilation error: Method deposit(Money) in the type BankAccount is not applicable for the arguments (int)

BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());

Repair Signature Changes BankAccount deposit(int cents)

BankAccount account = new BankAccount(); int amount = 500; Money money = new Money(500); account.deposit(amount); account.deposit(money); assertEquals( 500, account.getBalance());

BankAccount deposit(Money cents)

Repair Signature Changes Analyze the Change

Determine Initialization Values

Repair the Error

Repair Signature Changes Analyze the Change Change Type

Determine Initialization Values

Repair the Error

Parameter Add Parameter Type Change Parameter Removal Return Type Change

Repair Signature Changes BankAccount

Analyze the Change

deposit(int cents)

Change Type

deposit(Money cents)

Determine Initialization Values

Repair the Error

Parameter Add Parameter Type Change Parameter Removal Return Type Change

Repair Signature Changes BankAccount

Analyze the Change

deposit(int cents)

Change Type

deposit(Money cents)

Parameter Add Parameter Type Change Parameter Removal Return Type Change

Determine Initialization Values

pairs

Repair the Error

Repair Signature Changes BankAccount

Analyze the Change

deposit(int cents)

Change Type

deposit(Money cents)

Parameter Add Parameter Type Change Parameter Removal Return Type Change

Determine Initialization Values

pairs

Repair the Error Money money = new Money(500); account.deposit(money);

EVALUATION

Q1: APPLICABILITY Q2: EFFECTIVENESS

Test Care Assistant Add Test Cases for new Modules

Repair Compilation Errors due to Signature Changes

new Class in Hierarchy

new Interface Implementation

Parameter Addition

Parameter Type Change

New Overloaded Method

New Overridden Method

Parameter Removal

Return Type Change

Repair Signature Changes Applicability Subject # versions Cassandra 19 Geronimo 4 POI 6 Santuario 4 Shindig 6 Velocity 15 Wicket 7 Xindice 9 XMLBeans 7

Repair Signature Changes Applicability 100

75

50

% applicability

90.1

69.3

54.0

Average= 45.9

50.6

46.9 41.7 35.2 25 18.7

0

6.9 Cassandra Geronimo

POI

Santuario

Shindig

Velocity

Wicket

Xindice

XMLBeans

Repair Signature Changes Effectiveness

Project PMD JodaTime JFreeChart

# Versions # Tests 13 2 6

99 19 21

21

138

Repair Signature Changes Effectiveness 138

# of tests

104

138

128 105

69

35

0

Fixed Errors Valid Test Cases Same as Developers

Test Care Assistant Add Test Cases for new Modules

Repair Compilation Errors in Signature Changes

new Class in Hierarchy

new Interface Implementation

Parameter Addition

Parameter Type Change

New Overloaded Method

New Overridden Method

Parameter Removal

Return Type Change

Applicability 74

Software version 275

JFreeChart

0.96

JodaTime Barbecue Xstream PMD

1.62 1.5 1.31 4.2

KLOC 28

120

classes 315

217

471

64 9 25 65

99 55 218 483

Applicability 80

74 % applicability

60

Average= 60.9%

275 28

40

315

120

20

0

JFreeChart JodaTime Barbecue Xstream 275/471

74/99

28/55

120/218

PMD 315/483

% classes for which TCA can generate tests

Effectiveness TCA

Developer’s

Test Generation Tools

Randoop

• random test generation • run for 50 sec on each class

• •

industrial, coverage based run for each classes

• •

search-based testing run with assertion generation

Subjects

classes

Xstream 1.31 PMD 4.2 Barbecue 1.5b1 JodaTime 1.62 JfreeChart 1.013 Total

36 56 12 32 204 340

Coverage Results % Instructions Covered

100

75

68.9 50

61.7

60.5 51.3 36.3

25

0

TCA

Developers Randoop

CodePro

EvoSuite

TCA and Evosuite are Complementary EvoSuite 6,208 Common 12,055 TCA 6,370

Covered Statements

Test Evolution

Test Care Assistant

Test Reuse Algorithms

Evaluation

FUTURE WORK

Test Evolution

Test Reuse Algorithms

•Evaluate other Test Reuse Algorithms •User study on readability of generated tests Test Care Assistant

Evaluation