Continuous Integration: Improving Software Quality and Reducing Risk

Continuous Integration: Improving Software Quality and Reducing Risk

Preetam Palwe Aftek Limited

One more title  Do you love bugs ? Or Are you in love with QC members?

…

[Courtesy: Smita N]

Agenda  

  

Motivation Practices Advantages The road ahead Q&A

Motivation 



“Enter laptops, enter boards and put together with code, all teams have come together in one abode. Integration in progress.” … taken from an article published in AForce March 2008 Admin Utility  User now can be associated with the profile and action like send sms, call to a specific person or play welcome message on entering in the home.  The UI part for above feature is implemented by XYZ. And backend part is implemented by PQR. Integration of the same is

pending and will integrate on next week.

… taken from an weekly status report sent in 1st week of March 2008 

So ???

Motivation 

Integration is an EVENT !!!



Can we have something like following?

Image Courtesy: Book on CI by PMD

What is CI? 

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. … taken from Martin Fowlers article on CI

Agile manifesto 

Individuals and interactions processes and tools



Working software comprehensive documentation



Customer collaboration contract negotiation



Responding to change following a plan That is, while there is value in the items on the right, we value the items on the left more.

Agile principles 1. 2.

3.

4. 5.

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter time scale. Business people and developers must work together daily throughout the project. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Agile principles (contd) 6.

7. 8. 9. 10.

11. 12. 13.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Working software is the primary measure of progress. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely. Continuous attention to technical excellence and good design enhances agility. Simplicity—the art of maximizing the amount of work not done—is essential. The best architectures, requirements, and designs emerge from self-organizing teams. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Adaptive (agile) vs Predictive (plan-driven) 

Agile home ground   

 



Low criticality Senior developers Requirements change very often Small number of developers Culture that thrives on chaos

Plan-driven home ground 

   

High criticality Junior developers Requirements don't change too often Large number of developers Culture that demands order

Extreme programming 

Form of agile software development 



Values and/or principles    





Traditional software engineering practices taken to so-called "extreme" levels—leads to a development process that is more responsive to customer needs ("agile") than traditional methods, while creating software of better quality.

Communication Simplicity Feedback Courage Respect

Activities    

Coding Testing Listening Designing

XP practices 

Fine scale feedback    



Continuous process   



Continuous Integration

Refactoring or Design Improvement Small Releases

Shared understanding    



Pair programming Planning Game Test driven development Whole team

Coding Standards Collective Code Ownership Simple Design System Metaphor

Programmer welfare 

Sustainable pace

Coming back to CI 

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly. … taken from Martin Fowlers article on CI

Practices 

Maintain a single source repository  



Keep everything needed for build in CVS E.g. code, database schema, IDE configuration, test scripts etc but not application server and jdk. Even no build artifacts! Keep branches to minimum (in size and in lifetime)

Practices 

Automate the build  

From compilation to deployment Configure the build:   

Master build on server Developers build from IDE Build with test, without test or different set of tests!

Processes 

Make your build self-testing 

  



High degrees of automated tests suits to check larger code base Though TDD not mandatory but its helpful Test case failed: build failed! XUnit tools like JUnit and/or end-to-end testing tools like selenium Imperfect tests, run frequently, are much better than perfect tests that are never written at all.

Processes 

Everyone commits every day  



Every commit shall succeed the build Frequent commits: small work packets for developer Mentoring and practice is the key 

Processes 

Every commit should build the mainline on an integration machine 

 





You shouldn’t go home until the mainline build has passed with any commits you’ve added late in the day! Don’t check in untested / broken code Don’t check in when build is broken The whole point of CI is to detect integration errors as soon as possible Peers pressure within team ensures policies are followed 

Processes 

Keep the build fast  

XP guideline: 10 mins build! Staged build (build pipeline)   





Commit build Secondary build Performance build

If secondary build fails move the test case to commit build Parallel builds

Processes 

Test in a clone of production environment 





Every environmental difference results in a risk: what happens under test wont happen in production E.g. Same IP, port, machine, hardware, os etc Virtualization can help

Processes 

Make it easy for anyone to get the latest executable 



Its very hard to specify what you want in advance and be correct; people find it much easier to see something that’s not quite right and say how it needs to be changed Many executables in a day so not in CVS

Processes 

Everyone can see what's happening  





CI is all about communication Build status indicators: lava lamps, colored labels, hooters, mails Build history and build status reports indicates the project “health” Matrices are helpful to guide project management

Processes 

Automate deployment  

Use deployment pipeline Automated rollback

Advantages 

Reduces risk  



Makes dramatically easier to find and remove bugs 





Integration is predictable now We know where we are, what works, what doesn't work, how many outstanding bugs are there

Bugs - these are the nasty things that destroy confidence and mess up schedules and reputations. Bugs in deployed software make users angry with you. Bugs in work in progress get in your way, making it harder to get the rest of the software working correctly. Broken windows syndrome

Break Barriers between customer and development 

Add more features rapidly, give rapid feedback

The road ahead     



Just give it a try Start with commit build Add some automated test Use deployment pipeline Speed up the build Integrate with anything and everything available like project management tools, bug trackers, code analysis tools etc

Deployment pipeline

Image Courtesy: Paper on Deployment Pipeline by Dave Farley

Cruise control 

Open source CI toolkit from ThoughtWorks 





Support for different plugins like email, source control, builder, ant etc Support of web ui for build status Integrate with many code analysis tools, scm tools, build tools, reporting tools etc

Cruise control

Image Courtesy: CC documentation on cc website

References    

http://martinfowler.com/articles/continuousIntegration.html http://en.wikipedia.org/wiki/Agile_software_development http://en.wikipedia.org/wiki/Extreme_Programming http://cruisecontrol.sourceforge.net/

Q&A

Thanks!