Agile Development 1

Agile Development 1 MSE Spring 2009: Software Engineering and Architectures Andreas Meier Dipl.Informatik‐Ing.ETH [email protected]

Zürcher Fachhochschule

Literature [Beck] Extreme Programming Explained, Second Edition: Embrace change, 2004 Kent Beck with Cynthia Andres, ISBN 0321‐27865‐8 (aka “Second White Book”) [Beck02] Test‐Driven Development: By Example, 2002, ISBN 0321146530 [Cockburn] Agile Software Development, The Cooperative Game, Second Edition, Alistair Cockburn, 2007, ISBN 0321‐48275‐1 [Schwaber] Agile Software Development with Scrum, Ken Schwaber, Mike Beedle, 2002, ISBN 0‐13‐207489‐3 [Cohn06] Agile Estimating and Planning, Mike Cohn, 2006, ISBN 0‐13‐147941‐5 [Cohn04] User Stories Applied, For Agile Software Development, Mike Cohn, 2004, ISBN 0‐321‐20568‐5 [Highsmith04] Agile Project Management, Creating Innovative Products. Jim Highsmith, 2004, ISBN 0‐321‐21977‐5 Zürcher Fachhochschule

© 2009 InIT / School of Engineering

4

Literature [Highsmith 02] Agile Software Development Ecosystems, Jim Highsmith, 2002, ISBN 0‐201‐76043‐6 [Poppendieck] Implementing Lean Software Development, From Concept to Cash, Mary and Tom Poppendieck, 2007, ISBN 0‐321‐43738‐1 [Shore] The Art of Agile Development, James Shore, Shane Warden, 2008, ISBN 978‐0‐596‐52767‐9 [Rothman] Manage It! Your Guide to Modern, Pragmatic Project Management, Johanna Rothman, 2007, ISBN 0‐9787392‐4‐8 [Eckstein] Agile Softwareentwicklung im Grossen, Ein Eintauchen in die Untiefen erfolgreicher Projekte, Jutta Eckstein, 2004, ISBN 3‐89864‐250 [Martin] Agile Software Development, Principles, Patterns, and Practices, Robert C.Martin, 2003, ISBN 0‐13‐597444‐5



5

Overview Motivation What is an agile methodology? Agile manifesto Examples of agile methodologies

Extreme Programming (this lecture) Scrum (next lecture)



6

Project Failure Rate is Too High Despite the application of (heavyweight) methodologies, technology projects still regularly fail more than would be acceptable in any other industry. The evolution of technology (hardware) and tools (IDEs etc.) has outstripped methodologies by an order of magnitude.



7

Reasons For Failure

The Methodology didn’t work


The Solution didn’t work


8

Costly Functionality is Often Not Used



9

Response to regular project failure? Methodologies to make software development process more disciplined and predictive

More planning

Greater attention on analysis of requirements (Æ Requirements Engineering)

Tie down scope and sign‐off

Detailed and documented design before coding

Strict change control to suppress change

¾

And the result ?



10

Project success rates haven’t really improved Methodologies can prove bureaucratic and slow to deliver Hard for the business to completely conceptualize all requirements in one pass Even harder to capture all the requirements in a document in a completely detailed and unambiguous form

Developers “Interpret” requirements

Businesses constantly change ‐ the longer the project the greater the risk

Requirements and Priorities change

Its hard to completely design a system in one exercise

And not of great value if the requirements change anyway

If change is successfully suppressed, the business gets software they can’t use



11

Evolution: Agile Methodologies A number of methodologies have evolved to offer alternative approaches to building software that are more adaptive rather than predictive. Definition

Methodology:

Method:


A series of related methods and techniques [Merriam‐Webster] Systematic procedure, technique


12

What is an Agile Methodology? Agile proponents (Agilists) believe Current software development processes are too heavyweight or cumbersome

Current software development is too rigid

Waste: Too many things are done that are not directly related to software product being produced Difficulty with incomplete or changing requirements Short development cycles (e.g. Internet applications)

More active customer involvement needed


CMM (Capability Maturity Model) focuses on process


13

What is an Agile Methodology? Agile methodologies are considered

Lightweight People‐based rather than plan‐based

Several agile methodologies

No single agile methodology XP/Scrum probably most popular

No single definition Agile Manifesto closest to a definition

Set of principles Developed by Agile Alliance (http://agilemanifesto.org)



14

Agile Alliance Manifesto We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. Zürcher Fachhochschule


15

Agile Alliance Manifesto “We are uncovering better ways of developing software by doing it and helping others do it.” software development practitioners, not makers of rules uncovered, not invented!!! telling and helping



16

Agile Alliance Manifesto “Through this work we have come to value:” outcome of experience and reflection on that experience



17

Agile Alliance Manifesto “Individuals and interactions over processes and tools” processes and tools needed But people and interaction more important!



18

Agile Alliance Manifesto “Working software over comprehensive documentation” Working software is the primary goal of software development Æ Next slide Documents are hints & aids Only of value together with running software "just enough", "barely sufficient"



20

Sufficiency for the primary goal Alistair Cockburn [Cockburn] on comprehensive documentation Intermediate work products are not important as models of reality, nor do they have intrinsic value. They have a value only as they help the team make a move in the game. Thus, there is no idea to measure intermediate work products for completeness or perfection. An intermediate work product is to be measured for sufficiency: Is it sufficient to remind or inspire the involved group?



21

Agile Alliance Manifesto “Customer collaboration over contract negotiation” no "us" and "them", only "us" community, amicability, joint decision‐making, rapidity of communication, ... collaboration wins



22

Agile Alliance Manifesto “Responding to change over following a plan” plans are useful and necessary plans must be adopted to change



23

Agile Alliance Manifesto “We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more.



25

Principles behind the Agile Manifesto We follow these principles: 1. 2. 3. 4. 5. 6.

Our highest priority is to satisfy the costumer through early and continuous delivery of valuable software Welcome changing requirements, even late in development. Agile process harness change for the customer's competitive advantage Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale Business people and developers must work together daily throughout the project Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done The most efficient and effective method of conveying information to and within a development team is face‐to‐face conversation



26

Principles behind the Agile Manifesto 7. Working software is the primary measure of progress 8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely 9. Continuous attention to technical excellence and good design enhances agility 10. Simplicity – the art of maximizing the amount of work not done – is essential 11. The best architectures, requirements, and designs emerge from self‐ organizing teams 12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly



27

Agile Methodologies Scrum Ken Schwaber, Mike Beedle, Jeff Sutherland Dynamic Systems Development Model (DSDM) Arie van Bennekum, Dane Falkner, Jennifer Stapleton Crystal Methodologies Alistair Cockburn Feature Driven Development (FDD) Jeff De Luca, Peter Coad Lean Development Bob Charette Extreme Programming (XP) Kent Beck, Martin Fowler, Ron Jeffries, Bob Martin, ... Adaptive Software Development (ASD) Jim Highsmith Pragmatic Programmer Andy Hunt, Dave Thomas Zürcher Fachhochschule


28

Dilbert on Agile Programming



29

XP‐eXtreme Programming Three Core Contributors: Kent Beck Ward Cunningham Ron Jefferies Other notables:

Robert C.Martin Martin Fowler

“Second White Book” [Beck] Extreme Programming Explained, Second Edition: Embrace change, Kent Beck with Cynthia Andres, ISBN 0321‐27865‐8



30

Version 1 vs. Version 2 Version 1 (2000)

Version 2 (2004)

4 Core Values:

5 Core Values:

Communication Simplicity Feedback Courage

12 Practices

Communication Simplicity Feedback Courage Respect

13 Primary Practices 11 Corollary Practices (11 principles)



31

The Values Communication (V. 1 & 2) Simplicity (V. 1 & 2) Feedback (V. 1 & 2) Courage (V. 1 & 2) Respect (V. 2)



32

The Values: Communication Communication (V. 1 & 2) Simplicity (V. 1 & 2) ... person‐to‐person mutual understanding of the problem environment Feedback (V. 1 & 2) through minimal formal Courage (V. 1 & 2) documentation and Respect (V. 2) maximum face‐to‐face interaction ...



33

The Values: Simplicity Communication (V. 1 & 2) Simplicity (V. 1 & 2) Feedback (V. 1 & 2) ... the simplest thing that could possibly work ... Courage (V. 1 & 2) make it simple today, and Respect (V. 2) create an environment in which the cost of change tomorrow is low ...



34

The Values: Feedback Communication (V. 1 & 2) Simplicity (V. 1 & 2) Feedback (V. 1 & 2) Courage (V. 1 & 2) optimism is an occupational hazard to programming Respect (V. 2) feedback is the treatment ... hourly builds, frequent functionality testing with customers ... embrace change by constant feedback ...



35

The Values: Courage Communication (V. 1 & 2) Simplicity (V. 1 & 2) Feedback (V. 1 & 2) Courage (V. 1 & 2) Respect (V. 2) discipline requires courage doing what's right, even when pressured ... ... alignment of team's values creates conviction ...



36

The Values: Respect Communication (V. 1 & 2) Simplicity (V. 1 & 2) Feedback (V. 1 & 2) Courage (V. 1 & 2) Respect (V. 2) XP teams care about each other project I am important and so are you.



37

12 Practices (V. 1) Planning Game

Pair Programming

Small Releases

Collective Ownership

Metaphor

Continuous Integration

Simple Design

40‐Hour Week

Refactoring

On‐Site Customer

Testing

Coding Standards



38

12 Practices: Planning Game Planning Game

Pair Programming

Small Releases


customer stories = required feature Metaphor Continuous Integration Planning poker Simple Design 40‐Hour Week developers estimate customer prioritize Refactoring On‐Site Customer release plan = stories per release Testing Coding Standards plan = speculation plan updated often



39

12 Practices: Small Releases Planning Game

Pair Programming

Small Releases


Metaphor as small as possible


Simple Design 40‐Hour Week most valuable business requirements

sense of accomplishment On‐Site Customer Æ Refactoring Releasable = not necessarily released

Testing


Coding Standards


40

12 Practices: Metaphor Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design

40‐Hour Week

broad view of the project's goal Refactoring On‐Site Customer overall coherent theme broad sweep of the project Coding Standards Testing ≈ "architecture", but understandable by customers



41

12 Practices: Simple Design Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design

40‐Hour Week

Refactoring

On‐Site Customer

needed functionality Testing not potential functionality Coding Standards simplest design needed for functionality = best design Æ emergent, growing design = no big design upfront



42

12 Practices: Refactoring Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design

40‐Hour Week

Refactoring

On‐Site Customer

Testing simplest design first

Coding Standards

then refactor using design patterns before adding new functionality refactor existing code „Keep the code clean“ © 2009 InIT / School of Engineering


43

12 Practices: Testing Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design test‐driven development 40‐Hour Week Refactoring Testing


[Beck02] On‐Site Customer write test first Coding Standards unit tests, using e.g. JUnit re‐test continuously Development Cycle: Æ listen (requirements) Æ test (write first) Æ code (simplest) Æ design (refactor) © 2009 InIT / School of Engineering

44

12 Practices: Pair Programming Planning Game

Pair Programming

Small Releases


Metaphor

Continuous Integration continuous code inspection

Simple Design

pairs frequently change (e.g. daily) 40‐Hour Week

Refactoring

On‐Site Customer

Testing

Coding Standards


2 persons, 1 workstation


45

12 Practices: Collective Ownership Planning Game

Pair Programming

Small Releases


Metaphor

Continuous Integration anybody can change any code

Simple Design

40‐Hour Week Beware: only feasible when

Refactoring

unit tests in place On‐Site Customer

Testing

Coding Standards



46

12 Practices: Continuous Integration Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design

40‐Hour Week

Refactoring

On‐Site Customer Æ test case

Testing

Æ codeStandards Coding


minimum daily builds Æ integrate immediately


47

12 Practices: 40‐Hour Week Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design

40‐Hour Week

Refactoring

On‐Site Customer

Testing

Coding Standards avoid overtime

sustainable development extended overtime is a productivity‐reducing technique later: "Sustainable pace"



48

12 Practices: On‐Site Customer Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design

40‐Hour Week

Refactoring

On‐Site Customer

Testing

Coding Standards


real customer in team available for questions sets priorities


49

12 Practices: Coding Standards Planning Game

Pair Programming

Small Releases


Metaphor


Simple Design

40‐Hour Week

Refactoring

On‐Site Customer

Testing

Coding Standards

strict coding standards needed for clear, readable code collective ownership refactoring Zürcher Fachhochschule


50

XP Version 1 emergent design, no big up‐front design collective ownership continuous refactoring all 12 practices tightly coupled Æ all 12 practices strictly required for small, collocated teams



51

13 Primary Practices (V. 2) Sit Together

Quarterly Cycle

Whole Team

Slack

Informative Workspace

Ten‐Minute Build

Energized Work


Pair Programming

Test‐First Programming

Stories

Incremental Design

Weekly Cycle



52

13 Primary Practices: Sit Together Sit Together

Quarterly Cycle

Whole Team

Slack whole team together in open space privacy, "owned" space Informative Workspace nearby Ten‐Minute Build Energized Work


Pair Programming


Stories

Incremental Design

Weekly Cycle



53

13 Primary Practices: Whole Team Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

all skills and perspectives necessary Energized Work cross‐functional team Pair Programming sense of "team": Stories Together Weekly Cycle support each other … dynamic …


Continuous Integration Test‐First Programming Incremental Design


54

13 Primary Practices: Informative Workspace Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work

Continuous integration

workspace informative about work, e.g.: Pair Programming story cards on wall Stories big, visible charts Weekly Cycle provide for needs, e.g.: water, snacks cleanliness, order


Test‐First Programming Incremental Design


55

13 Primary Practices: Energized Work Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work


Pair Programming ex. 40‐hour‐week


Stories ex. sustainable pace

Incremental Design

Weekly Cycle

only as many hours as you can be productive, as you can sustain no burn‐out



56

13 Primary Practices: Pair Programming Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work


Pair Programming Stories Weekly Cycle


Test‐First Programming 2 persons, 1 workstation Incremental Design continuous code inspection pairs frequently change (e.g. daily) companionship and privacy if necessary work alone bring idea back reimplement as pair © 2009 InIT / School of Engineering

57

Pair programming All code is written by two people at one machine. What pair programming is not:

One writing and one watching for syntax errors. A tutorial session.

What pair programming is:

code and design review. A chance Continuous for communication. A way to stay honest.



58

13 Primary Practices: Stories Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work


Pair Programming


Stories

Incremental Design

Weekly Cycle plan with user stories Æ

customer‐visible functionality name and short prose or graphical description on index cards Zürcher Fachhochschule


59

Excursion: Agile Planning Business people decide

scope priority of features composition of releases date of releases

Development team decides

estimates technical consequences process detailed scheduling



60

13 Primary Practices: Weekly Cycle Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work


Pair Programming


Stories

Incremental Design

Weekly Cycle

review progress customer picks stories for this week stories Æ tasks Æ persons Zürcher Fachhochschule


61

13 Primary Practices: Quarterly Cycle Sit Together

Quarterly Cycle

Whole Team

Slack bottlenecks


Ten‐Minute Build repairs

Energized Work

themes for the quarter Continuous integration

Pair Programming


Stories

stories for quarter, addressing themes Incremental Design focus on big picture

Weekly Cycle



62

13 Primary Practices: Slack Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work

Continuous integration 20% programmer chosen tasks

Pair Programming

1 week in 8: Geek Week Test‐First Programming

Stories

Incremental Design

minor tasks that can be real customer in team dropped available for questions sets (Slack:priorities A period of little activity)

Weekly Cycle



63

13 Primary Practices: Ten‐Minute Build Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work

Continuous integration automatically build

Pair Programming

whole system and run Test‐First Programming

Stories

all tests in 10 minutes Incremental Design

Weekly Cycle



64

13 Primary Practices: Continuous Integration Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work


Pair Programming

Test‐First Programming integrate and test

Stories

changes Incremental Design after no more

than a couple of hours

Weekly Cycle



65

13 Primary Practices: Test First Programming Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work


Pair Programming


Stories

Incremental Design write a failing

Weekly Cycle


automated test before changing any code


66

Test‐First Programming Test‐First Programming (Test‐driven Development TDD [Beck02]) Write automated Unit tests before you start writing code TDD addresses many problems at once:

Scope creep – Don’t put in code “just in case” Coupling and cohesion – Loosely coupled, highly cohesive code is easy to test Trust – It’s hard to trust the author of code that doesn’t work Rhythm – test, code, refactor, test, code, refactor

TDD, Refactoring and incremental design are the prerequisites for XP’s embracing change!



67

Refactoring The goal of refactoring is to clean up your code every day Æ Makes extensions and modifications to our code easy Martin Fowler defines refactoring as “..the process of changing a software system in such a way that it does not alter the external behavior of the code yet improves its internal structure” The definite reference: Martin Fowler, Refactoring: Improving the Design of Existing Code, 1999, ISBN 0201485672



68

Refactor Mercilessly We computer programmers hold onto our software designs long after they have become unwieldy.

We continue to use and reuse code that is no longer maintainable because it still works in some way and we are afraid to modify it. But is it really cost effective to do so?

Extreme Programming (XP) takes the stance that it is not.

When we remove redundancy, eliminate unused functionality, and rejuvenate obsolete designs we are refactoring. Refactoring throughout the entire project life cycle saves time and increases quality.



69

Refactor Mercilessly (cont.) Refactor mercilessly to keep the design simple

Avoid needless clutter and complexity.

Keep your code clean and concise

It will be easier to understand, modify, and extend.

Make sure everything is expressed once and only once.

In the end it takes less time to produce a system that is well groomed.



70

The cost of change Traditional View

XP View TDD Refactoring Incremental Design



71

13 Primary Practices: Incremental Design Sit Together

Quarterly Cycle

Whole Team

Slack


Ten‐Minute Build

Energized Work


Pair Programming


Stories

Incremental Design

Weekly Cycle


invest in the design of the system every day an excellent fit for the needs of that day


72

Is Design Dead? Is Design Dead?, Martin Fowler, Software Development Magazine, April 2001 Self Study Æ Read the paper listed on the Wiki



73

11 Corollary Practices (V. 2) Real Customer Involvement

Code and Tests

Incremental Deployment

Single Code Base

Team Continuity

Daily Deployment

Shrinking Teams

Negotiated Scope Contract

Root‐Cause Analysis

Pay‐Per‐Use

Shared Code



74

11 Corollary Practices (V. 2) complete primary practices first corollary practices difficult or dangerous, if implemented before try one out, it might work or you might discover that you have more work to do before



75

11 Corollary Practices: Real Customer Involvement Real Customer Involvement Code and Tests Incremental Deployment

visionary customers part of Single Code Base Team Continuity Daily Deployment quarterly and weekly planning whole Team implies customer Shrinking Teams Negotiated Scope Contract involvement Root‐Cause Analysis Pay‐Per‐Use I haven't seen many teams go Shared Code a far as they could toward involving real customers



76

11 Corollary Practices: Incremental Deployment Real Customer Involvement

Code and Tests


Single Code Base

Team Continuity

Daily Deployment

no big deployment Shrinking Teams Negotiated Scope Contract deploy piece of functionality or limited data set Root‐Cause Analysis Pay‐Per‐Use run old and new system in Shared Code parallel



77

11 Corollary Practices: Team Continuity Real Customer Involvement

Code and Tests


Single Code Base

Team Continuity

Daily Deployment

Shrinking Teams

Negotiated Scope Contract keep effective teams together


Pay‐Per‐Use

Shared Code



78

11 Corollary Practices: Shrinking Teams Real Customer Involvement

Code and Tests


Single Code Base

Team Continuity

Daily Deployment

Shrinking Teams



Pay‐Per‐Use

as a team grows in capability, Shared Code keep its workload constant but reduce ist size this frees people to form more teams



79

11 Corollary Practices: Root‐Cause Analysis Real Customer Involvement

Code and Tests


Single Code Base

Team Continuity

Daily Deployment

Shrinking Teams



Pay‐Per‐Use

Shared Code

every time a defect is found after development, eliminate the defect and its cause goal: never make the same kind of mistake again „Ask 5 times why a problem occurred.“ (Taiichi Ohno)



80

11 Corollary Practices: Shared Code Real Customer Involvement

Code and Tests


Single Code Base

Team Continuity

Daily Deployment

Shrinking Teams



Pay‐Per‐Use

Shared Code

anyone on the team can improve any part of the system at any time



81

11 Corollary Practices: Code and Tests Real Customer Involvement

Code and Tests


Single Code Base maintain only the code and

Team Continuity

the tests as permanent Daily Deployment

Shrinking Teams

artifacts Negotiated Scope Contract


Pay‐Per‐Use

Shared Code


generate other documents from the code and tests rely on social mechanisms to keep alive important history of the project


82

11 Corollary Practices: Single Code Base Real Customer Involvement

Code and Tests


Single Code Base

Team Continuity

Daily Deployment

Shrinking Teams

Negotiated Scope Contract temporary branch o.k., but never let it live longer than a Pay‐Per‐Use

there is only one code stream


few hours

Shared Code



83

11 Corollary Practices: Daily Deployment Real Customer Involvement

Code and Tests


Single Code Base

Team Continuity

Daily Deployment

Shrinking Teams



Pay‐Per‐Use

Shared Code


put new SW into production every night prerequisites: defect rate