Test Suite Evolution. S.T.A.R. @. Faculty of Informatics. University of Lugano.
Switzerland. Supporting through Test Case Adaptation. Fabrizio. Pastore. Mehdi.
S.T.A.R. @ Faculty of Informatics University of Lugano Switzerland
Supporting
Test Suite Evolution through Test Case Adaptation
Mehdi Mirzaaghaei
Fabrizio Pastore
Mauro Pezzè
public final class DateTimeField{ .. public final class CopticChronology{ .. }
Test Evolution
} public class Report{ .. addRule(int line, String file) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); }
JodaTime r1.62
public final class DateTimeField{ .. public final class CopticChronology{ .. }
Test Evolution
} public class Report{ .. addRule(int line, Context String file) ctx) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); }
JodaTime r1.62
public final class DateTimeField{ .. public final class CopticChronology{ .. }
Test Evolution
} public class Report{ .. addRule(int line, Context String file) ctx) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); ReportTest{ test(){ Report r = new Report(); r.addRule(5, new Context( file ) ); assertFalse(r.isEmpty());
}
}
JodaTime r1.62
public final class DateTimeField{ .. public final class CopticChronology{ .. }
Test Evolution
public final class EthiopicChronology{ ..
}
public class Report{ } .. addRule(int line, Context String file) ctx) } ReportTest{ test(){ Report r = new Report(); r.addRule(5, file ); assertFalse(r.isEmpty()); ReportTest{
EthipicChronologyTest{ .. }
test(){ Report r = new Report(); r.addRule(5, new Context( file ) ); assertFalse(r.isEmpty());
}
}
JodaTime r1.62
public final class DateTimeField{ .. public final class CopticChronology{ .. } }
Test Evolution
public final class EthiopicChronology{ .. public class Report{ } .. addRule(int line, Context String file) ctx) }
CopticChronologyTest{ ReportTest{ EthipicChronologyTest{ Chronology COPTIC_UTC test(){ = CopticChronology.getInstanceUTC(); .. Report r =1,new DateTime epoch = new DateTime(1, 1, Report(); 0, 0, 0, 0, COPTIC_UTC); r.addRule(5, file ); long millis = epoch.getMillis(); } assertFalse(r.isEmpty()); ReportTest{ long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis(); test(){ DateTimeField monthOfYear = COPTIC_UTC.monthOfYear(); } Report r = new Report(); r.addRule(5, new Context( file ) ); while (millis < end) { assertFalse(r.isEmpty()); int monthValue = monthOfYear.get(millis); if (monthValue < 1 || monthValue > 13) fail("Bad month: }" + millis); }
JodaTime r1.62
public final class DateTimeField{ .. public final class CopticChronology{ .. } }
Test Evolution
Reuse Information
public final class EthiopicChronology{ ..
public class Report{ } .. addRule(int line, Context String file) ctx) }
available in
Existing Test Cases
CopticChronologyTest{ ReportTest{ EthipicChronologyTest{ Chronology COPTIC_UTC test(){ = CopticChronology.getInstanceUTC(); .. Report r =1,new DateTime epoch = new DateTime(1, 1, Report(); 0, 0, 0, 0, COPTIC_UTC); r.addRule(5, file ); long millis = epoch.getMillis(); } assertFalse(r.isEmpty()); ReportTest{ long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis();
to
Evolve Test Suites
test(){ DateTimeField monthOfYear = COPTIC_UTC.monthOfYear(); } Report r = new Report(); r.addRule(5, new Context( file ) ); while (millis < end) { assertFalse(r.isEmpty()); int monthValue = monthOfYear.get(millis); if (monthValue < 1 || monthValue > 13) fail("Bad month: }" + millis); }
JodaTime r1.62
EthiopicChronologyTest{
Test Evolution
public final class DateTimeField{ Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); .. DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); public long final class =CopticChronology{ millis epoch.getMillis(); .. } long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis(); DateTimeField monthOfYear = ETHIOPIC_UTC.monthOfYear(); public final class EthiopicChronology{ } .. while (millis < end) { int monthValue public class Report{= monthOfYear.get( new Date(millis) ); } if (monthValue < 1 || monthValue > 13) .. fail("Bad month: " + millis); addRule(int line, Context String file) ctx) } } CopticChronologyTest{ ReportTest{ EthipicChronologyTest{ Chronology COPTIC_UTC test(){ = CopticChronology.getInstanceUTC(); .. Report r =1,new DateTime epoch = new DateTime(1, 1, Report(); 0, 0, 0, 0, COPTIC_UTC); r.addRule(5, file ); long millis = epoch.getMillis(); } assertFalse(r.isEmpty()); ReportTest{ long end = new DateTime(3000, 1, 1, 0, 0, 0, 0, ISO_UTC).getMillis(); test(){ DateTimeField monthOfYear = COPTIC_UTC.monthOfYear(); } Report r = new Report(); r.addRule(5, new Context( file ) ); while (millis < end) { assertFalse(r.isEmpty()); int monthValue = monthOfYear.get(millis); if (monthValue < 1 || monthValue > 13) fail("Bad month: }" + millis); }
JodaTime r1.62
Test Care Assistant (TCA) Add Test Cases for new Modules
Repair Compilation Errors due to Signature Changes
new Class in Hierarchy
new Interface Implementation
Parameter Addition
Parameter Type Change
New Overloaded Method
New Overridden Method
Parameter Removal
Return Type Change
Test Care Assistant (TCA) Add Test Cases for new Modules
Repair Compilation Errors due to Signature Changes
new Class in Hierarchy
new Interface Implementation
Parameter Addition
Parameter Type Change
New Overloaded Method
New Overridden Method
Parameter Removal
Return Type Change
Test Care Assistant Add Test Cases for new Modules
Repair Compilation Errors in Signature Changes
new Class in Hierarchy
new Interface Implementation
Parameter Addition
Parameter Type Change
New Overloaded Method
New Overridden Method
Parameter Removal
Return Type Change
Generate Test Cases for Classes added to a Hierarchy Identify & Copy Candidate Test Cases Adapt Test Cases Update References to Class Under Test Repair Compilation Errors Repair Runtime Errors
Remove Redundant Test Cases
Generate Test Cases for Classes added to a Hierarchy Identify & Copy Candidate Test Cases Adapt Test Cases Update References to Class Under Test Repair Compilation Errors Repair Runtime Errors
Remove Redundant Test Cases
Identify & Copy Candidate Test Cases Chronology BasicChrono GJChrono Julian
Gregorian
BuddhistChrono
FixedMonthChrono Coptic...
Ethiopic...
JulianTest
GregorianTest
CopticTest
BuddhistTest
testCalendar()
testCalendar() testMax()
testCalendar() testEpoch()
testCalendar() testEpoch() JodaTime r1.62
Identify & Copy Candidate Test Cases Chronology BasicChrono GJChrono Julian
Gregorian
BuddhistChrono
FixedMonthChrono Coptic...
Ethiopic...
JulianTest
GregorianTest
CopticTest
BuddhistTest
testCalendar()
testCalendar() testMax()
testCalendar() testEpoch()
testCalendar() testEpoch() JodaTime r1.62
Identify & Copy Candidate Test Cases Chronology BasicChrono GJChrono Julian
Gregorian
EthiopicTest
BuddhistChrono
FixedMonthChrono Coptic...
Ethiopic...
testCalendar_1() testCalendar_2() testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()
JulianTest
GregorianTest
CopticTest
BuddhistTest
testCalendar()
testCalendar() testMax()
testCalendar() testEpoch()
testCalendar() testEpoch() JodaTime r1.62
Identify & Copy Candidate Test Cases Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian
Gregorian
FixedMonthChrono Coptic...
Ethiopic...
testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()
JulianTest
GregorianTest
CopticTest
BuddhistTest
testCalendar()
testCalendar() testMax()
testCalendar() testEpoch()
testCalendar() testEpoch() JodaTime r1.62
Update References to Class Under Test Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); 31111111 assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian
JulianTest
Gregorian
GregorianTest
FixedMonthChrono Coptic...
Ethiopic...
CopticTest
testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()
BuddhistTest
Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); testCalendar() testCalendar() testCalendar() testCalendar() DateTime epochtestMax() = new DateTime(1, 1, 1, 0, 0, 0, testEpoch() 0, COPTIC_UTC); testEpoch() assertEquals( CopticChronology.AM, epoch.getEra() ) ... JodaTime r1.62
Update References to Class Under Test Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); 31111111 assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian
Gregorian
FixedMonthChrono Coptic...
Ethiopic...
testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()
EthiopicChronology CopticTest BuddhistTest Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); testCalendar() testCalendar() testCalendar() testCalendar() DateTime epochtestMax() = new DateTime(1, 1, 1, 0, 0, 0, testEpoch() 0, COPTIC_UTC); testEpoch() assertEquals( CopticChronology.AM, epoch.getEra() ) EthiopicChronology ... JodaTime r1.62 JulianTest
GregorianTest
Update References to Class Under Test Chronology COPTIC_UTC =Chronology CopticChronology.getInstanceUTC(); EthiopicTest DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); 31111111 assertEquals( CopticChronology.AM, epoch.getEra() ) testCalendar_1() BasicChrono BuddhistChrono testCalendar_2() ... GJChrono Julian
Gregorian
FixedMonthChrono Coptic...
Ethiopic...
testCalendar_3() testCalendar_4() testEpoch_1() testEpoch_2() testMax()
ETHIOPIC CopticTest GregorianTest BuddhistTest Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); EthiopicChronology.getInstanceUTC(); ETHIOPIC testCalendar() testCalendar() testCalendar() testCalendar() DateTime epochtestMax() = new DateTime(1, 1, 1, 0, 0, 0, testEpoch() 0, COPTIC_UTC); testEpoch() assertEquals( EthiopicChronology.AM, CopticChronology.AM, epoch.getEra() epoch.getEra()) ) ... JodaTime r1.62 JulianTest
Adapt Candidate Test Cases
Undefined Fields
Missing Constructors
Missing Methods
Adapting Undefined Fields Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); assertEquals( CopticChronology.AM, epoch.getEra() ) ... FixedMonthChrono CopticChronology +AM
EthiopicChronology +EE
Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )
Adapting Undefined Fields Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); assertEquals( CopticChronology.AM, epoch.getEra() ) ... FixedMonthChrono CopticChronology +AM
EthiopicChronology +EE
Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )
Adapting Undefined Fields
class CopticChronology {
class EthiopicChronology {
/** Serialization lock */ long serialVersionUID = -59L;
/** Serialization lock */ long serialVersionUID = -59L;
Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;
DateTimeConstants.CE;
/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");
+AM
+EE
Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )
Adapting Undefined Fields
class CopticChronology {
class EthiopicChronology {
/** Serialization lock */ long serialVersionUID = -59L;
/** Serialization lock */ long serialVersionUID = -59L;
Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;
DateTimeConstants.CE;
/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");
+AM
+EE
Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )
Adapting Undefined Fields
class CopticChronology {
class EthiopicChronology {
/** Serialization lock */ long serialVersionUID = -59L;
/** Serialization lock */ long serialVersionUID = -59L;
Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;
DateTimeConstants.CE;
/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");
+AM
+EE
Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )
Adapting Undefined Fields
class CopticChronology {
class EthiopicChronology {
/** Serialization lock */ long serialVersionUID = -59L;
/** Serialization lock */ long serialVersionUID = -59L;
Chronology COPTIC_UTC = CopticChronology.getInstanceUTC(); /** /** DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, COPTIC_UTC); * Constant value for ... * Constant value for ... assertEquals( CopticChronology.AM, epoch.getEra() ) */ */ ...public static final int AM = public static final int EE = DateTimeConstants.CE;
DateTimeConstants.CE;
/** A singleton era field. */ /** A singleton era field. */ FixedMonthChrono private DateTimeField private ateTimeField ERA_FIELD = new ERA_FIELD = new CopticChronology EthiopicChronology SingleEraDateTimeField("AM"); SingleEraDateTimeField("EE");
+AM
+EE
Chronology ETHIOPIC_UTC = EthiopicChronology.getInstanceUTC(); DateTime epoch = new DateTime(1, 1, 1, 0, 0, 0, 0, ETHIOPIC_UTC); assertEquals( EthiopicChronology.AM, epoch.getEra() )
EE
Adapting Undefined Constructors Output GraphicOutput public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }
SVGOutput public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1
Adapting Undefined Constructors Output GraphicOutput
SVGOutput
public void testDefaultFont() { public void testDefaultFont() { g = new GraphicsMock(); g = new GraphicsMock(); output = new SVGOutput( output = new GraphicsOutput( g, g, DefaultEnv.DEFAULT_FONT, DefaultEnv.DEFAULT_FONT, fgColour, fgColour, bgColour); bgColour); [..] [..] assertEquals("graphicsOutput", assertEquals("graphicsOutput", Compilation error: output.toString()); output.toString()); The constructor } } SVGOutput(Graphics2D, Font,
Color, Color) is undefined Barbecue 1.5b1
Adapting Undefined Constructors Output GraphicOutput public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, GraphicOutput( fgColour, Graphics2D, bgColour); Font font, [..] Color fgColor, assertEquals("BLA", Color bgColor) output.toString()); }
SVGOutput SVGOutput(
public void testDefaultFont() { Writer writer, g = new GraphicsMock(); font, outputFont = new SVGOutput( g,Color fgColor, Color bgColor, DefaultEnv.DEFAULT_FONT, double scalar, fgColour, String units ) bgColour); [..] assertEquals("BLA", output.toString()); SVGOutput( } Writer writer,
Font fonts)
Adapting Undefined Constructors Output GraphicOutput public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, GraphicOutput( fgColour, Graphics2D, bgColour); Font font, [..] Color fgColor, assertEquals("BLA", Color bgColor) output.toString()); }
SVGOutput SVGOutput(
public void testDefaultFont() { Writer writer, g = new GraphicsMock(); font, outputFont = new SVGOutput( g,Color fgColor, Color bgColor, DefaultEnv.DEFAULT_FONT, double scalar, fgColour, String units ) bgColour); [..] assertEquals("BLA", output.toString()); SVGOutput( } Writer writer,
Font fonts)
Adapting Undefined Constructors Output GraphicOutput
SVGOutput SVGOutput(
public void testDefaultFont() { public void testDefaultFont() { Writer writer, g = new GraphicsMock(); g = new GraphicsMock(); +2 font, outputFont = new SVGOutput( output = new GraphicsOutput( g,Color fgColor, +2 g, Score=6 Color bgColor,+2 DefaultEnv.DEFAULT_FONT, DefaultEnv.DEFAULT_FONT, GraphicOutput( double scalar, fgColour, fgColour, Graphics2D, String units ) bgColour); bgColour); Font font, [..] [..] Color fgColor, assertEquals("BLA", assertEquals("BLA", Color bgColor) Score=2 output.toString()); SVGOutput( output.toString()); } } Writer writer,
Font fonts) +2
Adapting UndefinedSVGOutput( Constructors Writer writer, Font font, Output Color fgColor, Color bgColor, double scalar, GraphicOutput SVGOutput String units ) public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }
public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( ...., DefaultEnv.DEFAULT_FONT, fgColour, bgColour, ..., ...; [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1
Adapting UndefinedSVGOutput( Constructors Writer writer, Font font, Output Color fgColor, Color bgColor, double scalar, GraphicOutput SVGOutput String units ) public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }
public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( ...., new StringWriter( EasyMock(Writer) ... ) ), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, ..., ...; [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1
Adapting UndefinedSVGOutput( Constructors Writer writer, Font font, Output Color fgColor, Color bgColor, double scalar, GraphicOutput SVGOutput String units ) public void testDefaultFont() { g = new GraphicsMock(); output = new GraphicsOutput( g, DefaultEnv.DEFAULT_FONT, fgColour, bgColour); [..] assertEquals("graphicsOutput", output.toString()); }
public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( ...., new StringWriter( EasyMock(Writer) ... ) ), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, 0..., , “”...; ); [..] assertEquals("graphicsOutput", output.toString()); } Barbecue 1.5b1
Adapting Runtime Failures Test Failures: apply ReAssert
Daniel et. al.[ASE09,ISSTA10]
public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( new StringWriter(EasyMock(Writer)), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, 0, “”; [..] assertEquals("graphicsOutput", output.toString()); }
Adapting Runtime Failures Test Failures: apply ReAssert
Daniel et. al.[ASE09,ISSTA10]
public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( junit.framework.AssertionFailedError: new StringWriter(EasyMock(Writer)), expected: but DefaultEnv.DEFAULT_FONT, was: fgColour, bgColour, 0, “”; [..] assertEquals("graphicsOutput", output.toString()); }
Adapting Runtime Failures Test Failures: apply ReAssert
Daniel et. al.[ASE09,ISSTA10]
public void testDefaultFont() { g = new GraphicsMock(); output = new SVGOutput( new StringWriter(EasyMock(Writer)), DefaultEnv.DEFAULT_FONT, fgColour, bgColour, 0, “”; "SVGOutput" [..] assertEquals("graphicsOutput", output.toString()); }
Adapting Runtime Failures Uncaught Exceptions: involve users
Add Test Cases Identify & Copy Candidate Test Cases Adapt Test Cases Update References to Class Under Test Repair Compilation Errors Repair Runtime Errors
Remove Redundant Test Cases
Test Care Assistant Add Test Cases for new Modules
Repair Compilation Errors in Signature Changes
new Class in Hierarchy
new Interface Implementation
Parameter Addition
Parameter Type Change
New Overloaded Method
New Overridden Method
Parameter Removal
Return Type Change
Repair Signature Changes BankAccount deposit(int cents)
BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());
BankAccount deposit(Money cents)
Repair Signature Changes BankAccount deposit(int cents)
BankAccount deposit(Money cents) Compilation error: Method deposit(Money) in the type BankAccount is not applicable for the arguments (int)
BankAccount account = new BankAccount(); int amount = 500; account.deposit(amount); assertEquals( 500, account.getBalance());
Repair Signature Changes BankAccount deposit(int cents)
BankAccount account = new BankAccount(); int amount = 500; Money money = new Money(500); account.deposit(amount); account.deposit(money); assertEquals( 500, account.getBalance());
BankAccount deposit(Money cents)
Repair Signature Changes Analyze the Change
Determine Initialization Values
Repair the Error
Repair Signature Changes Analyze the Change Change Type
Determine Initialization Values
Repair the Error
Parameter Add Parameter Type Change Parameter Removal Return Type Change
Repair Signature Changes BankAccount
Analyze the Change
deposit(int cents)
Change Type
deposit(Money cents)
Determine Initialization Values
Repair the Error
Parameter Add Parameter Type Change Parameter Removal Return Type Change
Repair Signature Changes BankAccount
Analyze the Change
deposit(int cents)
Change Type
deposit(Money cents)
Parameter Add Parameter Type Change Parameter Removal Return Type Change
Determine Initialization Values
pairs
Repair the Error
Repair Signature Changes BankAccount
Analyze the Change
deposit(int cents)
Change Type
deposit(Money cents)
Parameter Add Parameter Type Change Parameter Removal Return Type Change
Determine Initialization Values
pairs
Repair the Error Money money = new Money(500); account.deposit(money);
EVALUATION
Q1: APPLICABILITY Q2: EFFECTIVENESS
Test Care Assistant Add Test Cases for new Modules
Repair Compilation Errors due to Signature Changes
new Class in Hierarchy
new Interface Implementation
Parameter Addition
Parameter Type Change
New Overloaded Method
New Overridden Method
Parameter Removal
Return Type Change
Repair Signature Changes Applicability Subject # versions Cassandra 19 Geronimo 4 POI 6 Santuario 4 Shindig 6 Velocity 15 Wicket 7 Xindice 9 XMLBeans 7
Repair Signature Changes Applicability 100
75
50
% applicability
90.1
69.3
54.0
Average= 45.9
50.6
46.9 41.7 35.2 25 18.7
0
6.9 Cassandra Geronimo
POI
Santuario
Shindig
Velocity
Wicket
Xindice
XMLBeans
Repair Signature Changes Effectiveness
Project PMD JodaTime JFreeChart
# Versions # Tests 13 2 6
99 19 21
21
138
Repair Signature Changes Effectiveness 138
# of tests
104
138
128 105
69
35
0
Fixed Errors Valid Test Cases Same as Developers
Test Care Assistant Add Test Cases for new Modules
Repair Compilation Errors in Signature Changes
new Class in Hierarchy
new Interface Implementation
Parameter Addition
Parameter Type Change
New Overloaded Method
New Overridden Method
Parameter Removal
Return Type Change
Applicability 74
Software version 275
JFreeChart
0.96
JodaTime Barbecue Xstream PMD
1.62 1.5 1.31 4.2
KLOC 28
120
classes 315
217
471
64 9 25 65
99 55 218 483
Applicability 80
74 % applicability
60
Average= 60.9%
275 28
40
315
120
20
0
JFreeChart JodaTime Barbecue Xstream 275/471
74/99
28/55
120/218
PMD 315/483
% classes for which TCA can generate tests
Effectiveness TCA
Developer’s
Test Generation Tools
Randoop
• random test generation • run for 50 sec on each class
• •
industrial, coverage based run for each classes
• •
search-based testing run with assertion generation
Subjects
classes
Xstream 1.31 PMD 4.2 Barbecue 1.5b1 JodaTime 1.62 JfreeChart 1.013 Total
36 56 12 32 204 340
Coverage Results % Instructions Covered
100
75
68.9 50
61.7
60.5 51.3 36.3
25
0
TCA
Developers Randoop
CodePro
EvoSuite
TCA and Evosuite are Complementary EvoSuite 6,208 Common 12,055 TCA 6,370
Covered Statements
Test Evolution
Test Care Assistant
Test Reuse Algorithms
Evaluation
FUTURE WORK
Test Evolution
Test Reuse Algorithms
•Evaluate other Test Reuse Algorithms •User study on readability of generated tests Test Care Assistant
Evaluation