Applying Lean Principles to Test Data

By Gil Zilberfeld

Applying Lean Principles to Test Data We like our software to run smoothly, and to get there we need applications to be tested. The problem is that applications are so complex today, and so the testing effort is large. We need to consider many questions: What do we need to automate? How much effort should be put into unit, functional, and exploratory testing? Do we have the right people? Do they need training? However, when we dive down into the details, we get overwhelmed by the amount of scenarios we need to test. It seems that we need to set test data aside for every scenario. It seems that the test data need to be managed as well – which data belongs to each test? What describes the data and how it relates to a scenario? Can we reuse data between scenarios? How do we achieve it? Tough questions indeed. But before we jump into building or buying a test data management system, we need to look at the bigger picture. The problems of test data management look huge – but are they the right problem to solve?

What do lean principles tell us? There are a couple of lean principles I like to go back to now and again in order to get back on track. ▪▪ Eliminate waste – any task that does not contribute directly to customer satisfaction with the product is waste. Even when it is needed, there might be more effective ways to achieve it. ▪▪ See the big picture – we want to optimize the entire development, not optimize a data management system. Local optimization may not benefit the whole project, and may even cost more than the immediate benefit over time. With these principles in mind, how do we approach test data in an Agile project?

Decisions, decisions … In Agile development we incrementally add functionality to the product. Testing is part of the task, and what brings it to “Done”. Test planning, the scenarios we are going to test, automatic and manual testing – all are parts of the “Done-ness”. During the iteration we are making a few important decisions: ▪▪ We know we cannot test everything, so we specifying the important scenarios. Other scenarios may not be developed, or developed but not tested. This is a team decision. The product owner (to use the scrum term) needs to know what he is getting and at what level of quality. The developers need to know what to develop, and the testers need to know what they are going to test. ▪▪ Part of the work will be automating the system tests. Regardless of who does the work, developers or testers, the automated system tests are going to make sure the feature works in the specified sce-

24

Testing Experience – 22/2013

narios. To achieve this, we need to pick the data examples that are going to be part of the scenario. ▪▪ When picking data examples, we would like to both reduce the risk of not covering enough cases and, at the same time, not to pick overlapping sets of data. ▪▪ In terms of exploratory testing, we may identify what we need in order to test but usually we leave that to when we come to the session. Also, once the session completes we usually do not save it. We do keep a few sets of data for certain scenarios, such as an empty or already filled database. Because of the nature of exploratory testing, there are not many of those, and they come and go in response to changes over the lifetime of the application. Let us see how these decisions correlate with the lean principles: ▪▪ Since we are going to work on the most important scenarios as prioritized by the product owner, that means we are limiting the test data, and reducing the waste in generating and maintaining it. Maintenance is an important factor: if we need to upgrade the data many times, we should think about whether we are managing it correctly. For example, if we need to prepare a new database for each build, we will probably want to automate that part as well. ▪▪ We are reducing the number of data examples we need for the system automated tests to the necessary minimum. Not only does it reduce the waste in generation and future maintenance, but also in waste that will accumulate over time. System tests notoriously take long to run. With every set of data we eliminate, we are shortening the feedback cycle. That adds up to a lot of delayed feedback time where the team just waits around for the automated tests to complete before moving on to the next tasks. ▪▪ We cannot make the decision of what to test at the system level and what to do at the unit level without the “see the big picture” approach. This allows us to weigh the importance and risks and, maybe more so, match expectations about the quality of the product.

Treat test data differently depending on use Not all decisions are explicit. We have identified the types of data we need for the system tests. What about unit tests? Usually we do not specify the unit tests prior to testing the code. While the product owner cares about how the system works, he/she couldnot care less about internal development. The developers need to make sure their tests run correctly, and so specify the test data at the time of coding. Or, if they are using Test Driven Development, even before that. To make sure we are covering the supported scenarios, we use methods such as code and test review in addition to unit testing. In fact, we make distinctions between unit testing and system testing:

▪▪ Test data for different types of tests is managed differently. The examples used in system tests are known in advance, while examples in unit tests are created on the go. ▪▪ System test data may benefit from the discussion of maintenance and reuse. Unit test data, however, is evolutionary and can be controlled only to a certain extent by test reviews. In fact, due to the nature of how we write unit tests, there is not much benefit in trying to optimize the test data. ▪▪ Test data can be stored in different locations, depending on its purpose. Data for the system tests can be in the form of external data sources, such as databases, files, etc. Unit test data are part of the unit tests, usually in the form of initialization and mocking behavior.

Sense and adapt Throughout development, we learn more about the features and what types of data we will need in order to test them. As we have more data sets, we will need to apply the same kind of thinking. Especially when the tests start to impact the automated test performance, we should ask: ▪▪ Is this set needed and at what level – system or unit? If we can replace a long running system tests with a few unit tests it could shorten test runs.

Going back to our initial discussion, we can see by applying the “see the big picture” principle that the real problem we need to solve is not test data management. In fact, we need to apply the lean principles so we get what we actually want: working software and satisfied customers. ◼

> about the author Gil Zilberfeld has been involved in software since childhood, starting out with Logo turtles. With over twenty years of developing commercial software, he has vast experience in software methodology and practices. Gil is the Product Manager at Typemock – The Unit Testing Company (www.typemock.com), working as part of an Agile team in an Agile company, creating tools for Agile developers. He promotes unit testing and other design practices, down-to-earth agile methods, and some incredibly cool tools. Gil speaks at local and international conferences about unit testing, TDD, Agile practices, and communication. And in his spare time he shoots zombies, for fun. Gil blogs at www.gilzilberfeld.com on different Agile topics, including: processes, communication, and unit testing.

▪▪ Is data previously needed still necessary? Is it still as important as before? If not, maybe remove it from automated runs and reduce the feedback time.

Created due to popular demand, this two-day course caters for a wide audience, giving candidates a solid understanding of roles, processes and techniques used in Agile projects. It provides an overview of the agile activities, building on a basic understanding, reinforced through a heavy emphasis on discussion and practical application. The Agile Essentials qualification is aimed at anyone involved in software engineering who wants to become familiar with working in an Agile environment. That includes developers, business analysts and testers. Additionally, senior project personnel such

as project managers and IT directors will benefit from this course, gaining a fundamental grounding in Agile techniques.

Book

now!

An exam at the end of the course tests for knowledge, understanding and application of vital Agile techniques, but does not require previous experience in Agile projects. For more information and training dates, please visit our website: training.diazhilterscheid.com

Díaz & Hilterscheid Unternehmensberatung GmbH Kurfürstendamm 179 10707 Berlin Germany Phone: +49 (0)30 74 76 28-0 Fax: +49 (0)30 74 76 28-99 E-mail: [email protected] Website: training.diazhilterscheid.com

Testing Experience – 22/2013

25