Transitioning Towards Continuous Experimentation in ...

Transitioning Towards Continuous Experimentation in a Large Software Product and Service Development Organisation – A Case Study

Sezin Gizem Yaman1, Fabian Fagerholm1, Myriam Munezero1, Jürgen Münch1,2, Mika Aaltola3, Christina Palmu3, Tomi Männistö1

1

2

3

Continuous experimentation is a software development approach where research and development activities are driven by constantly conducting experiments with product value*.

* Rissanen, O., Münch, J.: Continuous Experimentation in the B2B Domain: A Case Study. * Fagerholm,F.,Guinea,A.S.,Mäenpää,H., Münch, J.: The RIGHT model for Continuous Experimentation.

HOWEVER, guidence is lacking on how development teams in large organisations with complex business partnership networds can transition towards continious experimentaiton.

As our aim was to observe the introduction of continuous experimentation in a company, we conducted an actual experiment round with a real product.

Transition

Small teams

Initial circumstances

Small-scale experiments

The experiment Experimentation Target

Persistence Collaboration

Initial Circumstances ü Multiple stakeholders - Select a focus ü Evolving product - Allocate few resources ü Need to limit the risks - Start small

Small teams Small-scale experiments ü Champions ü Brainstorming on possible experiments ü Simplistic UI experiment as a start

[It’s better to] start experimenting with something small. [...] It’s more important to start now. Practice will make it perfect.” (Technical coach)

Identifying an experimentation target ü Utilise existing resources ü Numerous discussions ü Prioritize and analyse the feature

Designing and executing the experiment ü Pilot run ü Rerun the experiment ü Expert advice ü Emphasize “learning”

“Fix the experiment [the] best way you can and run it again. You can learn so much with each experiment.” (Technical coach)

Collaborating with experts ü Avoid mistakes, biases ü Faster start ü Iterative learning

Persistence ü Learn from the failures ü Keep practising ü Scale up

“One should not dwell on temporary failures, but keep practicing.” (The teams)

Transition Small teams Small-scale experiments

Experimentation Target

The experiment Initial circumstances

Persistence

Collaboration

“Experimentation made it clear to the teams that there is no need to debate between opinions and assumptions as you can quickly test them with an experiment.” (Technical coach)

Reference Sezin Gizem Yaman, Fabian Fagerholm, Myriam Munzero, Jürgen Münch, Mika Aaltola, Christina Palmu, Tomi Männistö. Transitioning Towards Continuous Experimentation in a Large Software Product and Service Development Organization – A Case Study. In Proceedings of the 17th International Conference on ProductFocused Software Process Improvement (PROFES 2016), LNCS. Springer-Verlag, 2016. Get the article here: http://bit.ly/corporateexperimentation

Discussion: What other ways a software company can transition towards continuous experimentation? University of Helsinki Empirical Software Engineering Helsinki Research Group

[email protected] www.cs.helsinki.fi/en/people/yaman www.cs.helsinki.fi/group/ese/ Sezin Yaman