practices to satisfy customer expectations in agile teams using inductive ... each practice has their own recipes, terminology, and tactics when it comes to the ...
Adopting Design Thinking Practices to Satisfy Customer Expectations in Agile Practices: A Case from Sri Lankan Software Development Industry W.M.D. Ruchira Prasad, G.I.U.S Perera, K.V. Jeeva Padmini, and H.M.N. Dilum Bandara Dept. of Computer Science and Engineering University of Moratuwa Moratuwa 10400, Sri Lanka. {ruchira.16, indika, jeeva, dilumb}@cse.mrt.ac.lk Abstract— While the application of agile principles leads to better project success, some projects still fail due to insufficient understanding of client’s exact requirements. Agile teams have recently started adopting Design Thinking (DT) practices to better understand what is in customers’ mind. We explore suitable DT practices to satisfy customer expectations in agile teams using inductive reasoning. We first formulated a conceptual framework based on a literature review. We then conducted a set of interviews with fifteen domain experts from ten IT service organizations. Interview findings were then analyzed using the Straussian grounded theory. Customer journey, story mapping, prototypes, POC, and customer profiling were determined to be the most suitable methods to identify the needs of customers. Moreover, practicing human-centered approach through workshops, discussions, team communication, and end-user interaction through UAT were also identified to be effective. We further classified the best practices into five categories as customer’s real need identification, transforming customer’s real needs into pilot solutions, visualizing the pilot solution for customer feedback, idea generation for the pilot solution, and brainstorming. Based on these findings, we also derived a framework to achieve customer satisfaction through the adoption of DT in agile-base projects. Keywords— agile practices; design thinking; design thinking practices; grounded theory
I. INTRODUCTION Agile framework is an iterative and incremental process. It delivers software products in collaboration with cross-functional teams. The main objective of introducing agile into the software development industry was to achieve rapid delivery of highquality products while responding to frequent changes in customer’s mind [1]. Extreme Programming (XP), Scrum, Dynamic systems development method (DSDM), Adaptive Software Development (ASD), Crystal, Feature-Driven Development (FDD), Pragmatic Programming, and Lean Development are popular practices under the agile umbrella [1]. Though all these practices adhere to agile principles and values, each practice has their own recipes, terminology, and tactics when it comes to the software implementations. Design Thinking (DT) is a creative strategy to provide solutions to complex, ill-defined, or unknown problems [2]. It is centered around thoroughly understanding the customer’s needs and trying many possibilities before coming up with a preferred solution. It is an iterative process [3] where the problem is first analyzed and experimented with different designs. The best 978-1-5386-4417-1/18/$31.00 ©2018 IEEE
solution is then developed out of several experimental designs. Development process needs to be flexible to use this approach and to apply the best solution that fits the problem. As described by Matthew [4], DT helps designers to resolve the problem by giving a 360-degree view of the problem. By considering different aspects of the problem, such as end-user requirement, social factors of the environment, and current market and trend, a designer can get a 360-degree view. DT helps to look beyond what you see in the traditional problem-solving methods, and it helps to derive innovative ideas from existing resources. As customer satisfaction matters a lot in the software service industry, managers can use DT to read customer’s mind to provide exact customer need [4]. Moreover, inventors can use DT to win the market with innovative product development. Understanding what the customer wants is the most critical part of software development [5]. The main reason for the failure of many projects is due to unclear user expectations. DT practice can be used to understand what customer really wants at the initial stage of the project. While DT is known to be highly effective in other industries such as entertainment, medicine, education, and space exploration, it is still a novel practice to software development, especially in the agile context [2]. DT practices could greatly enhance the understanding of customer and product needs in each iteration/sprint of agile projects, and the two practices go together as both are iterative processes. However, best practices and challenges in adopting DT into the agile software development process are largely unknown. Therefore, we framed our question to fill that gap as: How to apply design thinking practices to improve customer expectations in agile practices? We answer the above question based on the findings from a set of interviews with industry professionals from service delivery organizations in Sri Lanka that practice DT in agilebased projects. Interview findings were then analyzed using the Straussian grounded theory. We identified a set of DT best practices for agile-based projects and then derive a framework to satisfy customer expectations though DT practices in such projects. We then categorized the best practices into five groups as customer’s real need identification, transforming customer’s real needs into pilot solutions, visualizing the pilot solution for customer feedback, idea generation for the pilot solution, and brainstorming. Based on these findings we derived a framework to achieve customer satisfaction through a four-step process that enables the application of DT practices in agile-base projects.
The rest of the paper is organized as follows. Section II presents the related work. Research methodology is presented in Section III. Data analysis and a set of recommendations are presented in Section IV and V, respectively. Concluding remarks are presented in Section VI. II. LITERATURE REVIEW Agile principles and values emphasize the collaboration with customer, as well as early and continuous delivery in a shorter timescale. Iterative and incremental process in agile ensures customer feedback within a shorter timescale. While agile practices are known to substantially increase the project success rate [1], still a significant number of projects fail. This is mainly due to the lack of understanding of customer’s expectations [6]. Alternatively, it has been identified that better process success could be achieved when the customer spends valuable time with the vendor to properly understand and articulate what he/she really wants at the project initiation stage [7]. As understanding end-user expectation is the most critical aspect to deliver the right product to the customer, DT practices could be used to eliminate project failures. DT helps business to find solutions for complex problems and generate new ideas for product development through an iterative, incremental, and creative process [8]. Fig. 1 illustrates how the DT process evolves through empathize, define, ideate, prototype, and test stages. Within each stage, problems are framed, questions are encouraged, along with more ideas, until the best answers are evident and chosen [8]. In the empathize stage user starts to explore the problem and try to understand it through consulting domain experts, and observing, engaging, and empathizing with people. Within a limited time, the user develops the best possible understanding of the problem. During the define stage, user defines the problem using the information gathered during the empathize stage and formulate the problem in the form of a question. Engineering Thinking (ET) and DT share a similar process when it comes to the design stage. Both share the same mindset for critical thinking to find the desired solutions [9]. However, ET uses deductive methods while DT follows inductive reasoning, because the three DT principles (namely people, social needs, and user expectations) tend to be unpredictable. Whereas ET focuses on predictable things. Both ET and DT use same approaches such as prototyping to express the ideas. However, ET deals with more certainty while DT details with uncertainty. Therefore, DT is more suite for human-
centered problems solving. During the ideate stage, user starts generating ideas about possible solutions, and this process can be supported with ideation tools. Then user prototypes the initial version of the product. Key idea is to understand the best solution for the problem. Finally, during the test stage, designers rigorously test the product. Because this is an iterative process, the team can spend more time with the customer showcasing POC, prototypes, or mockups to get the feedback for the product development [10]. In [11], a three-dimensional framework is proposed for the different categories and elements associated with DT and those are listed in Table I. Five elements need to be considered while practicing DT [11]. Those are the Human-Centered Approach (HCA), thinking by doing, visualizing, synthesis of the diverging and converging approach, and collaborative work style. We focused our study only on these five DT practices. DT is a systematic process rather than traditional thinking. While DT is not a new concept, it is new to the software development industry [11]. In [12] it is mentioned that “an organization pursuing an agile approach can start with design thinking-inspired concept development before beginning the actual agile development process.” However, iterative and incremental delivery is insufficient when the customer is not getting what he/she desires due to lack of proper design or solutions [13]. Even though agile projects have a high success rate, the question to raise would be “can agile development provide customer satisfaction by implementing the right features [13]. Thus, it is imperative to explore how DT practices could satisfy customer expectations in agile projects. III. RESEARCH METHODOLOGY Our initial focus was to measure the usage and understand the best practices and challenges of using DT practices in an agile environment. However, we could not measure the adoption level of DT and its benefits, as most of the professionals were unaware of the terminology and presence of DT practices, though they were practicing some of the aspects. Therefore, this paper focuses only on the best practices and challenges. Fig. 2 illustrates the conceptual framework for the research. DT can be categorized as practice, cognitive approach, and mindset (see Table I). As the first step, we focused only on DT practices to understand how to apply the five elements in DT TABLE I. Category Practices
Cognitive Approach
Mindset
Fig. 1. Desigining thinking process [8].
DESIGN THINKING CATEGORIES AND ELEMENTS [11]. Elements • • • • • • • • • • • • •
Human-Centered Approach (HCA) / Empathy Think by doing / Action based Visualizing Synthesis of Diverging & Converging Collaboration Holistic viewpoint Integrative thinking Abductive thinking Reflective reframing Future oriented Experimental and explorative Ambiguity tolerant Optimistic
practices (i.e., independent variables) to improve the customer satisfaction (i.e., dependent variable). Aspects such as cognitive approach and mindset are left as future work. Qualitative approach [14] is more suitable for this research as it is difficult to quantify the usage of DT practices and the benefits to be gained. Another reason is the lack of awareness of DT techniques within employees. Hence, we selected the qualitative approach and inductive reasoning method. At first, we had to explain the theory and the usage of those techniques to interviews and then only we could get their feedback. In 2013 and 2014, Sri Lanka was awarded as the offshoring destination of the year award by the National Outsourcing Association, UK [15]. According to the Sri Lankan IT/BPM Industry 2014 Review report [16], Sri Lankan IT/BPM sector showed an impressive growth trend of 238% from 2007-2014, which was primarily contributed by the IT service industry. Therefore, Sri Lanka IT services industry is quite mature. We considered IT service organizations in Sri Lanka as the population. While 77 organizations are listed in the SLASSCOM website [17], only 50 organizations were determined to be into IT services according to their LinkedIn profiles (see Fig. 3). Some organizations were no longer in business. From the 50 organizations, we selected 26 with more than 50 employees. It is unlikely for an organization with limited resources to practice DT techniques, especially at the start-up stage. From the 26 organizations, ten were identified to be using the snowballing method which is used with DT to execute agilebased projects (see Fig. 3). We conducted face-to-face interviews with project managers, business analysis, and leads as they are the main contributors for requirement gathering, project handling, and decision makings. The interview questionnaire was developed based on the literature survey. The questionnaire covered five elements in the DT practices as listed in Fig. 1. Pilot face-to-face interviews were conducted with three project stakeholders (a project manager, business analysis, and a tech lead) selected from three different organization to validate the questionnaire. Though project stakeholders have used the DT techniques, they were not familiar with the word design thinking. Hence, based on the feedback received, questions were revised to enhance the clarity. We interviewed fifteen managerial-level employees (profiles are listed in Table II) from ten service-based software development organizations that applied DT practices into the agile process. These project stakeholders are the key resources who have decision power and negotiate with the customer. During interviews, we had to explain about DT theory and concepts to the interviewees, as most of them were not aware of the concepts though they practice those. We analyzed data using Straussian grounded theory [18] as it allows a literature review for theoretical sampling, concept development, and defining properties and dimensions. Immediately after each interview we started the open coding process [18], analyzed textual contents, break the data into parts, and identified the key terms and compared them to identify the categories. These indicators are constantly compared with data being collected to identify new insights until theoretical saturation is reached. Using axial cording we then identified
Fig. 2. Conceptual framework.
Fig. 3. Sample selection.
disaggregation and relationships of the open cording. Finally, using selective cording we identified the factors. IV. DATA ANALYSIS We first analyzed the agile practices used by the organizations and the results are listed in Fig. 4. Scrum (71%) is the most frequently used agile practice. Kanban, XP, and Lean are the other popular agile practices adopted. Based on the interview responses, we identified the customer pain points and ways to improve customer’s expectations (see Table III). During the interview one of the technical specialists mentioned that “end of the day win or lose depend on the customer experience.” Moreover, a tech lead stated that “customer thinks that we are experts in the domain, but dev team doesn’t have any knowledge about it.” These emphasize the need for understanding customer’s context and being able to deliver a good experience regardless of the limitations of the development team. As stated by a project manager “customer journey is used to understand how a user is going to use the system, experience with the system and clarifications for gray areas.” Furthermore, a technical specialist mentioned that “we have to drill down the stories to find the requirement. Discuss what is exactly needed. What customer needs, and what you feel that customer needs.” These propose suitable techniques to explore the customer’s needs and expectations. During the data analysis, we further identified customer pain points and ways to improve customer expectations and those are listed in Table III. As per the analysis
TABLE II.
INTERVIEWEE PROFILES AND THEIR AGILE EXPERIENCE.
#
Designation
1
Senior Project Manager Technical Specialist Business analyst
2 3 4
Technical Project Manager
5
Tech Lead
6
Director/ Mobile Architect
7
Senior BA
8
Tech Lead
9
Tech Lead
10 Tech Lead 11 Project Manager
12 Project Manager 13 Tech Lead
14 Tech Lead
15 Project Manager
Responsibilities Scrum master, Project Management, Making processes Team Leader, Team Player, Confident Scrum Master, Product Owner, BA, Client management, agile coach Budgeting, Requirement gathering, Resource planning, Daily scrums, Estimations, and deliverables. Leading project, Mentoring Developers, Developments Architectural Design, Quality maintain, Facilitating, Decision making Customer Relationship, Requirements, Develop business case, Observations, Requirement sign off Responsible for all technical inputs, Play Architect role on demand, Requirement gathering, Work on user stories, Priorities and estimations, Technical designs, Creating tasks and assign, Code review Team lead, Development, Responsible for deliverables, Dev support, Incident management, Task management, Estimation, Analysis and recommendations Lead developer, Deployments, Scrum master, Estimates Ownership of the project, Analysis requirement with BA and Architect, identify risk and mitigation plans, Estimations, Delivery quality check, Check progress and provide updates Scrum master, Project Manager Team Lead, Requirement gathering and analysis, Design the implementation, Development Team Lead, Customer communication, Design decision, Technical decisions, Scrum process, Requirement analysis Project manager, Scrum master
Agile No of Experie projects nce (y) 6+ 3 6+
5
4+
3
10+
12 Fig. 4. Agile practices usage.
6+
8
10+
10+
7+
10
8+
11
of the interview data based on the Grounded theory, several points need to be considered to satisfy customers. These painpoints are what the customers worrying about the software projects and taken from the point of the view of interviewees. We then categorized the best practices to satisfy customer expectations in agile projects into five groups as customer’s real need identification, transforming customer’s real needs into pilot solutions, visualizing the pilot solution for customer feedback, idea generation for the pilot solution, and brainstorming. Each of these groups is described in Table IV. We further identified several challenges faced by the agile teams while practicing DT process and those are summarized in Table V. TABLE III.
PAIN POINTS IN CUSTOMER SATISFACTION AND WAYS TO IMPROVE.
Pain Points in Satisfying Customer Expectations 6+
3
3+
3
5+
6
4+
5
4+
5
3+
4
7
3
• Unable to prioritize what customer really wants • Compare and update the product features against competitor’s product without considering its applicability • Not focusing on feature applicability for tomorrow’s market • Dealing with fix bid projects (budget, cost, and time are fixed) • After completing the implementation only, the user realizes feature is useless • Requirement changes in a limited budget • Dealing with very new technology • Product developed without realizing target market • Customer always expect to work only with domain experts and subject matter experts • Lack of trust on vendor Techniques to Understand Customer Expectation • Conduct workshops at customer end to understand customer and enduser wish list • Develop roadmaps for implementation • Conduct reviews and demos (e.g., proof of concepts and prototypes) • Conduct frequent formal and casual discussions • Hire an end user to understand his/her paint points • Gather and analyses customer pinpoints • Study the customer during pre-sales • Small and frequent deliverables with expected features • Communicate the blockers early as possible • Conduct market validation before starting product implementations • Educate client on simplicity on product implementation • Derive business case and form the foundation of the solution • Identify the non-functional requirements • Keep update customer about product implementation progress (e.g., Burndown charts, show and tells) • Study the drawbacks of the existing process
TABLE IV. # 1
2
3
4
5
BEST PRACTICES TO SATISFY CUSTOMER EXPECTATIONS.
Description Customer’s real need identification • Accepting requirement only from the business analyst is not recommended • Before starting actual implementation, it is recommended to spend a good amount of time (e.g., 6 weeks) to capture customer need • Adhering to DT techniques such as customer journey mapping, user story mapping, user experience design, scenario identification, heat maps, and customer profiling • Practice HCA with workshops and discussions. Transforming customer need into pilot solutions • Proof of concept (POC), prototype, persona identification and wireframes are used determine the idea • Generate questions to the customer and get feedback during sprint works, to drive the project into right direction • Pilot solutions help to indicate failures early rather than leading to an unexpected failure. • Ensure project team is solving the right solution and validate before expensive development or design occurs. Visualizing the pilot solution for customer feedback • Understand stakeholder perception on idea visualization helps to implement the right feature • Get feedback by asking questions on the outcome of the planned solution, based on pilot solutions such as proof of concepts, prototypes, and mockup • Testifying the idea with the end-user • Adjust customer mindset to think more about their own requirements and planned solution Idea generation for the pilot solution • Divergent approach is used for idea generation, as the first idea is not always the best idea. • Identify the most suitable solution for customer given business context • Have confidence in the development team Brainstorming • Brainstorming can be used to justify decisions, generate ideas, evaluate ideas, pros and cons to make decisions, and clarification of expertise ideas • Spread responsibility with everyone to about the final decision • Stakeholder involvement is essential to understand the right requirement and to provide the right solution • Involve domain and technical experts to clarify gray areas • Avoid conflicts among project stakeholders due to different understandings
V. RECOMMENDATIONS Each of the best practices in Table IV can be supported by one of the DT practices. Each of the proposed DT practices/techniques and factors is listed in Table VI. We derived the relationship between DT practices and identified the best practices, based on the techniques explained in [11]. HCA could be used for customer’s real need identification. Thinking by doing could be used to transform customer needs into pilot solutions. POC, prototype, and wireframes are some of the techniques described in both the situations. By visualizing, the pilot solution for feedback stakeholder perception could be better understood. Idea generation for pilot solutions could be supported by synthesis and the combination of diverging and converging approach. Collaborative work style could be used for brainstorming. Several other collaborative work techniques are also discussed in [11]. Therefore, using these techniques we can improve the best practices.
TABLE V.
CHALLENGES FACED BY AGILE TEAMS.
Challenges Faced by Agile Teams While Practicing DT • Organizations use techniques in DT practices without having theoretical knowledge on DT • Many project teams are unaware that they are using techniques in DT practices on a daily basis • Lack of understanding of agile principles and DT principles • Techniques used to capture the requirements are customer journey mapping, prototype, proof of concept, wireframes, user experience (UX) design, story-mapping, use-cases, scenario identification, workflows, customer profiling, heat maps, A/B testing, and persona techniques • Common practices such as collaboration are available for both agile and DT practice • Most of the time vendor not engaged with the end user and communicated only through product owner, which is not acceptable TABLE VI. DT Practices Human-Centered Approach (HCA)
Thinking by doing / Action based thinking
Visualizing
Synthesis – Combination of Divergent & Convergent methods Collaborative work style
DT PRACTICES AND FACTORS. Factors Identifying customer's real needs Practicing human-centered approach Stakeholders of the human-centered approach Customer expectations vs deliverable Feasibility study Ongoing project feedback Practicing thinking by doing approach Risk mitigation and resolving conflicts Testifying ideas by visualization approaches Stakeholder perception on the visualized idea Practice idea visualizing Idea generation Evaluating ideas for the best approach Practicing divergent & convergent techniques Brainstorming to justify decisions Stakeholders in collaborative work style Stakeholders involvement
Further, we propose the framework in Fig. 5, which could be used to improve the customer satisfaction through the use of five DT practices. Based on the framework in Fig. 5, findings, and the observations, following recommendations, can be derived to improve customer satisfaction with the use of DT practices. If the project is run on the agile framework, the customers could use this framework to improve the customer satisfaction. The vendor is responsible for clearly understanding what is in customer’s mind. To accomplish that, vendor can use techniques described in HCA to understand the customer and his goal and objectives. The vendor can then use a combination of divergent and convergent techniques (Synthesis) such as ideation, pattern finding, creating multiple alternatives to collect more information/ requirements related to the development and analyze and to take decisions development technology/ features in the product. During Thinking by Doing stage, Vendor can work on implementing pilot product by using techniques like; prototype development, POC, wireframe development, etc., vendor can use the Visualizing stage to share the product experience with client and get feedback for the workable product developed. Development team can practice the techniques described in Collaborative work style throughout four phases described in the framework. Moreover, it is further recommended to conduct DT workshops and establish DT coaches; combine agile with DT by adding DT tasks to the
Fig. 5. Framework to enhance customer satisfaction using design thinking practices in agile practices.
backlog and groom them; develop DT culture in the organization for more customer-centric and collaboration, and use DT as a tool for better decision-making during the development. VI. SUMMARY We explore how effectively use DT practices with agile process. Based on a set of interviews with project stakeholders who practice DT in agile projects, we identified that DT practices such as human-centered approach, thinking by doing, visualizing, synthesis of diverging and converging, and collaborative work style are helpful in enhancing the customer expectation. We further explored challenges faced by agile teams and propose a set of solutions. Finally, we developed a framework to identify end-user expectations more precisely and provide the best possible solution with having early feedbacks to eliminate surprises at the latter part of the project. In future, we plan to explore other categories of design thinking such as cognitive approach and mindset.
[6]
[7] [8]
[9] [10]
[11]
[12]
[13]
REFERENCES [1]
[2] [3]
[4]
[5]
K.V.J. Padmini, I. Perera, and H.M.N.D. Bandara, “Applying Agile practices to avoid chaos in user acceptance testing: A case study,” in Proc. Moratuwa Engineering Research Conf. (MERCon), April 2016. C. Kevin and R. Smith, “Unleashing the power of design thinking,” Design Management Review, vol. 19, no. 3, 2008, pp. 8-15. F. L. Eickhoff et al., “Large-scale application of IBM Design Thinking and Agile development for IBM z14,” IBM Journal of Research and Development, 1, 2018. H. Matthew, “How tangible is your strategy? How design thinking can turn your strategy in to reality,” Journal of Business Strategy, vol. 30. No. 2/3, 2009, pp. 50-56. S. Hossein and D. Robert, “Requirements engineering: Making the connection between the software developer and customer,” Information and Software Technology, vol. 42, no. 6, pp. 419–428.
[14]
[15]
[16] [17]
[18]
A.Hussain, E.O.C. Mkpojiogu2 and F.M. Kamal, “The Role of Requirements in the Success or Failure of Software Projects,” in Proc. Intl. Soft Science Conf. (ISSC 2016), April 2016. L. Percival et al., “IBM Design Thinking Software Development Framework,” In Proc. Brazilian Workshop on Agile Methods, 2016. M. Bethany, “Corporate implementation of design thinking for innovation and economic growth,” J. of Strategic Innovation and Sustainability, vol. 10, no. 2, 2015. W. Shlomo et al., “Engineering thinking: The experts’ perspective,” Intl. Journal of Engineering Education, vol. 27, no. 4, 2011, pp. 838-851. The Interaction Design Foundation. 5 Stages in the Design Thinking Process. Available: https://www.interaction-design.org/literature/ article/5-stages-in-the-design-thinking-process. H. Lotta and M. Lasskso, “Design Thinking in the Management Discourse: Defining the elements of the Concept,” In Proc. 4th World Conf. on Design Research (IASDR ‘11), Oct. 2011. L. Tilmann, C. Meinel, and R. Wagner, “Design thinking: A fruitful concept for it development?,” Design Thinking, Springer, Berlin, Heidelberg, 2011, pp. 3-18. G. Kavitha, N. Srinivasaraghavan, and S. Adikari, “An integrated framework for design thinking and agile methods for digital transformation,” In Proc. 5th Intl. Conf. on Design, User Experience, and Usability, July 2016. N.A.G. Arachchilage and A.P. Martin, “A trust domains taxonomy for securely sharing information: A preliminary investigation,” In Proc. 8th Human Aspects of Information Security & Assurance, 2014. “SL nominated for Offshoring Destination of the Year Award for third consecutive year,” SLASSCOM, [Online]. Available: https://slasscom.lk/. [Accessed 01 05 2018]. SLASSCOM, Sri Lankan IT/BPM Industry 2014 Review, 2014. A. Strauss and J. Corbin. “Basics of qualitative research: Techniques and procedures for developing grounded theory,” Thousand Oaks, Saga Publication, 1998. “Reports,” SLASSCOM, [Online]. Available: https://slasscom.lk/. [Accessed 01 02 2018].
