ELICITING CUSTOMER REQUIREMENTS IN FOCUS ... - CiteSeerX

In Proceedings from EIASM: The 7th International product Development Management Conference, May 29-30, 2000

ELICITING CUSTOMER REQUIREMENTS IN FOCUS GROUP INTERVIEWS: CAN EFFICIENCY BE INCREASED? Pontus Engelbrektsson, Özlem Yesil and I. C. MariAnne Karlsson Department of Human Factors Engineering, Chalmers University of Technology SE-412 96 Goteborg, SWEDEN, e-mail: [email protected] ABSTRACT Focus group interviews are commonly used for eliciting customer requirements. However, efficient requirement elicitation is not only a matter of choosing data collection method. Methodological considerations also include an efficient choice of participants and mediating tools, such as different product representations, in order to enhance the process. Two studies were completed and analysed with the overall aim of assessing the effect of the choice of participants and product representation. The results imply that a choice of participants with product experience seems to have an overall positive effect on the volume and character of the information elicited. Furthermore, the results indicate that it is more important to carefully consider the information content of a representation instead of focusing on the type of representation for efficient requirement elicitation. INTRODUCTION A thorough understanding of customers’ needs and requirements is considered a prerequisite for successful product development (e.g. Cooper & Kleinschmidt 1986; Cooper 1988). However, research has shown that customer requirements elicitation is regarded as one of the major problem in early product development (e.g. Carlsson 1990, Rydebrink et al. 1995). Part of this problem may be attributed the way the external dialogue is run, i.e. the efficiency of the dialogue between the customers and the product development team. Hence, method is a crucial issue. Several sources, e.g., Urban and Hauser (1993) and Ulrich and Eppinger (1995) argue the use of qualitative methods. One of the more common methods for gathering information from customers is focus group interviews, i.e. unstructured interviews with a group of people and conducted by a moderator (Assael 1995; Catteral & Maclaran 1997; Jordan 1997; Moore 1982). Although Griffin and Hauser (1993) state that “. . . the group synergies expected from focus group interviews do not seem to be present”, other researchers, e.g. Assael (1995), Catteral and Maclaran (1997), and Jordan (1997) consider focus group interviews to be more productive than one-to-one interviews. The underlying notion is that the members of the group stimulate and support each other, providing a more open forum for discussion. However, efficient requirements elicitation is not only a matter of choosing data collection method (Figure 1). Methodological considerations must also include an efficient choice of context or environment in which data collection takes place, a choice of participants and a choice of stimuli or mediating tools in order to enhance the data collection process.

1


Data collection method Participants

Environment

Mediating tools

Figure 1. Methodological considerations include a choice of data collection method; participants; mediating tools; and environment. Even though knowledge on these choices is essential, there seems to be very little systematic and comparative research evaluating different approaches. This paper deals with methodological considerations in eliciting of customer requirements. More specifically, it describes the effects of the choice of participants and product representations as mediating tools in focus group interviews. Two studies, each involving four focus group interviews, have been conducted and the results analysed. IMPROVING CUSTOMER REQUIREMENT ELICITATION Mediating tools In order to enhance data collection, different mediating tools can be used. For instance, Urban and Hauser (1993, p.224) stress the importance of probing which could be regarded as a verbal tool. Ulrich and Eppinger (1995, p.41) suggest using visual stimuli in order to increase the effectiveness of the information gathering. Also Karlsson (1996, p.135-136) proposes the use of mediating tools in order to stimulate reflection. She concludes that “. . . . we may be forced to create systematic conceptualisation or ‘breakdowns’, for instance by probing with questions or by exposing the users to provocative descriptions of alternate ways of use and practice. However, may the dialogue require also other mediating tools.”, for instance sketches, models or prototypes. In an interview situation, a typical mediating tool could be a representation of the existing or the future product. These product representations may be simple sketches, threedimensional models, or fully functional prototypes. Some studies have addressed the issue of choosing the 'correct' representation for concept testing.

2


For instance, Page and Rosenbaum (1992) claim that a more concrete description of the product is necessary in order to acquire a more valid response to a concept. Furthermore, Page and Rosenbaum (1992) argue that non-working models may give the customer a better idea of the physical characteristics of the product while working prototypes may allow for a clearer understanding of the capabilities and benefits of the product. Also de Bont (1992) focuses on concept evaluation but conclude that it is not the representation itself that is the key issue. The critical aspect is what information content is provided by the representation. The same conclusion is made by Karlsson et al. (1998) in a study on eliciting requirements for a completely new product (a health scanner). They found that the differences between three product representations (a verbal description, a sketch and a three-dimensional but non-functioning model) with the same information content were small in terms of the requirement picture elicited (volume and character). Another and complementary explanation is proposed by Soderman (1998) who suggests that two specific characters: scale and interaction, need to be present in the representation in order to create any ‘leap’ in terms of increased understanding. Participants Another important issue for the outcome of a study is the choice of participants. A representative sample of some kind is most often argued. However, there are other sampling principles, for instance a subjective (or theoretical) critical sample. The Lead User Method (e.g. Herstatt & von Hippel 1992) is one example of a method in which ‘critical’ customers/users are involved in developing a requirement picture. Other studies have indicated that product experience may play an important role for the emerging requirement picture. Schoormans et al. (1995) conclude that product knowledge is essential for customers’ ability to interpret different product representations and to distinguish between important and less important matters. Also Karlsson et al. (1998) find that experience of product use seems to be essential for assessing specific matters about a future product. On the other hand, for instance, Chapanis (1959) argues that only 'naive' participants should be chosen in all ‘experimental designs’ in order to avoid the bias of previous experiences and habits. RESEARCH AIM Two studies (A and B) have been carried out and analysed in order to compare the effects of using (i) different user categories and (ii) different types of product representations as mediating tools in focus group interviews. The underlying assumptions have been that the elicited information – in terms of volume, as well as character – is dependent upon the representation used, as well as upon the product experience of the participants.

3


STUDY A: CUSTOMER REQUIREMENTS FOR A NEW GENERATION OF WHITE GOODS The first study – study A – was conducted as part of the primary product development phase of a development project. The specific purpose of the study was to elicit requirements for a new generation of refrigerators and freezers but also to gain more indepth knowledge of customer reactions to new product ideas. Method Four focus groups were conducted. Totally twenty-three customers participated, making up groups of 5 to 6 individuals respectively. Age was chosen as a discriminating factor and the groups were formed as two extremes of a possible target population (Table 1). The first group turned out to be heterogeneous in terms of nationality, while the remaining groups were homogeneous. In two groups (one with participants 20-40 years and one with participants 40-60 years old) a pictorial stimulus – colour overheads – was used. The content of the information was existing and possible new design solutions. Table 1: Characteristics of the four focus groups. Focus Group No

Age group

Mediating objects

One Two Three Four

20–40 20–40 40–60 40–60

Pictorial Verbal Pictorial Verbal

The pictures shown were selected on a random basis and presented without any company or brand identification in order for the participants to react to the picture itself without the influence of the established images or values associated with the company or its brand name (as proposed by Page and Rosenbaum 1992). In two corresponding groups, a verbal stimulus – compact written information on overheads – was used. The interviews were run according to the same interview guide, specifying the different themes to be discussed. Each interview lasted for approximately one and a half hours each and were tape-recorded for later analysis. In the analysis, all statements related to product use and product characteristics and properties were extracted in order to compare the outcome of the respective interviews. Results The overall requirement picture turned out to be similar in all groups and focused on three issues: flexibility, ease of cleaning and the materials used in the product. However, there were also differences between the groups. For instance, the overall outcome of the first and third groups were richer in terms of information volume and character compared to that of the second and fourth (Table 2). Especially the first group generated a very rich picture

4


which also included a number of unique and innovative ideas, such as for instance a safety catch for small children and smoother corners inside the refrigerators for ease-of-cleaning. In comparison, the second group did not add any new requirements to the existing ones, nor suggest any unique ideas. On the contrary, the discussions tended to focus on the content of the information presented. The verbal descriptions typically resulted in the participants asking for further information on specific features presented, in particular the solutions that were based on new technology. The pictorial representations resulted, on the other hand, in the participants elaborating on the features. In addition, fewer questions were posed. The synergy in the groups presented with pictures was observed to be strong, whereas synergy was weak in the other two groups. In the specific case, age and product experience were correlated and the elderly groups seemed better in discriminating between important and unimportant aspects of the product, as well as inferring benefits from the product’s physical attributes. Thus, the degree of product knowledge among the participants seemed to be very important for group dynamics. Table 2: Overview of elements affecting and overall outcome of focus group interviews.

Elements Visual stimuli Degree of product knowledge Group formation Overall outcome

First

Focus Group No Second Third

Fourth

pictorial

verbal

pictorial

verbal

moderate

low

high

moderate

heterogeneous

homogeneous

homogeneous

homogeneous

‘very rich’

‘very poor’

‘very rich’

‘poor’

Another observation made was that the heterogeneous character of the first group sometimes resulted in ‘conflicts’, due to differences in culture and habits between participants. However, these conflicts seemed to enhance the discussion and allowed a rich variety of ideas to emerge. Discussion and conclusions In this study, the pictorial representation resulted in information which was richer in volume and character than did the verbal description. These results support the idea that the more concrete the product representation, the richer the information generated.

5


However, the results contradict the results in the study by Karlsson et al. (1998) where no difference between the effect of verbal product descriptions and other product representations could be found. One explanation, and in accordance with the results of de Bont (1992), is the information content of the representation. The study by Karlsson et al. (1998) was carefully designed in order for the different representations to hold the same information. It is possible that the verbal descriptions in the study presented here contained less or other information than did the pictures. The fact that the participants who were presented with the verbal descriptions consistently asked for further information support such an idea. The study also demonstrates the importance of group formation. Overall, product use experience had a positive effect on the information generated but also on the number of new and innovative solutions, both regarding alternative designs of the features presented and totally new designs. These results are in accordance with other studies, e.g. the studies by Schoormans et al. (1995), Karlsson et al. (1998) and Soderman (1998). However, another group formation characteristic seems to have influenced the results. Generally, focus group homogeneity is advocated in order to reach consensus in the group. However, in this study the heterogeneity of one of the groups resulted in ‘conflicts’ between the participants that actually lead to more unique and innovative ideas being generated. Actually, disagreement could be regarded as yet another mediating tool in an interview situation, and as one way of probing for hidden or not articulated requirements. In sum, the study showed that pictorial representations resulted (i) in information which was richer in volume and character than did verbal descriptions, (ii) that product experience enhanced innovative ideas, as did (iii) a certain focus group heterogeneity. STUDY B: PASSENGER REQUIREMENTS FOR A NEW TRAM The second study – study B – was a study carried out as part of a development project in which the aim was to develop a new tram for public transport. As one important step of the development process, customer requirements for the tram were identified. Method Also in this study, four focus groups were completed incorporating altogether 23 participants. The participants were divided into two different user groups: frequent and less frequent passengers. The ages of the participants varied between 24 and 55 years old. Each interview was supported by one of two mediating tools: either a fully functional tram or a series of slides showing different situations of the tram in use (Table 3). All focus group interviews followed the same strict format and lasted for one and a half hours each. The interviews started with a discussion on public transport in general and continued with a session which followed the order of a trip by tram. The interviews were recorded on tape and transcribed. The transcriptions were coded and comments on the physical design of the tram were analysed for differences in degree of detail (i.e. if the participant stated that "the seat is uncomfortable" vs. "the seat cushion is 3 cm too narrow"), if the statement was a problem description or a proposal, and finally if there were any other differences between the focus groups.

6


Table 3. Description of the focus group characteristics. Focus Group No

Mediating tool

Passenger category

One Two Three Four

tram tram slides slides

experienced inexperienced experienced inexperienced

Results All in all, each group generated more than 100 statements, which could be classified into fourteen different categories (Table 4). There was no difference in the total number of statements by the participants who were presented with slides and the ones who were presented with the tram. However, differences in volume could be noted for different aspects. For instance, the slides resulted in more statements on, e.g., the entrance than did the tram itself. An explanation could be that the different product representations emphasised different aspects of the product. In the specific case, some quite ‘provocative’ pictures could have contributed to the participants’ strong focus on the steep steps of the entrance. Other design aspects, such as colours, lighting and space, seemed more difficult for the participants to visualise with the help of pictures compared to the tram. In general, the experienced participants were able to provide more information than the inexperienced ones and their discussion was also more focused on the design of the tram. Furthermore, the experienced users commented on “noises &bumps”, “heating & ventilation”, and to a larger extent also on riding with the tram. This is somewhat noticeable since none of the product representations contained information on the two first aspects and only some information on the last – which can only be experienced by taking a ride on a moving tram. On the other hand, the inexperienced users seemed more dependent upon the information provided by the product representations.

7


Table 4. The number of statements for each of the fourteen different aspects of the tram given by the different focus interview groups. Experienced Users Themes Exterior Entrance & Doors Space Noises & Bumps Signs Riding: standing Riding: sitting Driver’s compartment Looking out Interior, colours Lighting Heating & Ventilation STOP-buttons Other ∑

Inexperienced users

Tram

Slides

Tram

Slides

13 15 12 9 6 18 24 13 10 29 13 8 13 13

22 21 6 9 3 18 38 1 6 10 7 2 10 9

14 11 4 3 3 4 23 6 4 17 7 0 6 9

20 18 5 2 2 6 22 8 8 24 5 0 9 22

196

162

111

151

There was also a difference in the character of the statements provided by the different groups. Independent of representation, the experienced users provided more information on design details compared to the inexperienced. However, the inexperienced user groups were able to generate detailed information on specific issues provided that the product representation visualised these aspects. Furthermore, the experienced users’ comments could be characterised as problem descriptions based on experience, e.g. “Sometimes people grab my hair when they are trying to get a grip of the handles of the seats”. The inexperienced group gave statements based on what they saw and experienced when the specific product representation was presented, e.g. “The colour of the walls is too cold”. However, in terms of degree of requirement specificity, no difference could be found between the two product representations. Discussion and conclusions The study shows that experienced users were able to provide slightly more information than inexperienced users. Furthermore, experienced users were able to comment on aspects that were not visualised by the either of the product representations. These results comply with those of Schoormans et al. (1995) who suggest that individuals with product knowledge can compensate for and fill in missing information when presented with a product representation. However, the study presented here implies that experienced users

8


may not only be able to compensate for a lack of product information per se but also for a lack of information on contextual and use issues, extrapolating their experience of product use from more ‘laboratory conditions’ into the intended use environment. Karlsson & Rosenblad (1998) came to the same conclusion based on laboratory evaluations of protective clothing. The study also shows that provided that the product representation visualised a certain aspect or attribute, inexperienced users could provide useful information on that particular aspect. These results are in accordance with the findings by de Bont (1992), who claimed that the information content of the product representation, rather than, e.g., the concreteness of the product representation influences the way the product is perceived and understood. Whereas there was a difference in information volume between the experienced and the inexperienced users when provoked by the full-scale tram, no difference could be noted between the two groups presented with the slides. It may be that pictures provide an opportunity to control the discussion to a larger extent than does a full-scale product representation, especially one which enables the participants to interact with the product and move about. The content of a picture can manipulated, specific aspects can be brought forward and other toned down or excluded as a way of focusing the discussion. However, this possibility also implies a risk of excluding critical information which could lead to a situation where important requirements could be missed, especially if the participants in the study have no earlier experience of the particular product (or similar products). In sum, the study showed that (i) inexperienced users seemed more dependent upon visual support than did experienced users in order to generate detailed product information, (ii) experienced users seemed able to compensate for lack of information content in the product representation, and (iii) users lacking product experience may express valuable information provided that a product representation is used as a mediating tool, containing information on the specific features to be addressed. GENERAL DISCUSSION AND MANAGERIAL IMPLICATIONS The common objective of these studies was to assess the effect of the choice of mediating tool (in terms of product representation) and participants in focus group interviews with the purpose of eliciting customer requirements. The results of the two studies are consistent, as well as contradictory. The consistency concerns the importance of product experience as a way of reaching a richer and, possibly also, more innovative requirement picture when eliciting customer requirements. The contradictions concern the effect of mediating tools in terms of different product representations. Whereas the first study indicates that the choice of product representation may have a large effect on the outcome, the second study does not. However, there are two important differences between the studies. While the first study focused on a verbal versus a pictorial description, the second involved a pictorial description versus a full-scale prototype which means that the results can not be compared. However, the results indicate that an increased concreteness of the product representation does not necessarily result in an increase in the information volume elicited or a change of information character. There are other aspects than the type of

9


representation which must be considered in choosing the most efficient ‘mediating tool’. For instance, it seems important to carefully consider the information content of each representation. Actually, the studies indicate that the choice of the content of product representation may be particularly important when involving participants with no or little experience of the product (or similar products). Another important aspect is purpose which the very first issue to consider when designing any study. Although both studies were concerned with customer requirements elicitation, the first study involves requirement elicitation but also idea generation and assessment of different features whereas the second study could be described as requirement elicitation in order to define a problem picture. These two situations define two different phases of the product development process and, consequently, two different strategies may be applicable. From a managerial perspective, the two studies contribute to a deeper understanding of the methodology of customer requirements elicitation. This knowledge is necessary in order for the product development team to be cost efficient in their choice of strategy for running the customer dialogue. With an increased understanding of the role of product representations, as well as the role of customer experience, product developers will be better equipped to make decisions as to what product representations may be developed and which customers should be involved in the early stages of the product development process. However, the studies have involved only one method – focus group interviews – for gathering user requirements. In order to draw a more complete map which will enable product developers to efficiently choose between data collecting methods, participants, and product representations, further work will need to include more product representations (e.g. CAD and VR representations), different kinds of products, other data collection methods, and not least different purposes. REFERENCES Assael, H. (1995): Consumer Behaviour and Marketing Action. Fifth edition, Cincinatti. De Bont, C.J.P.M. (1992): Consumer Evaluations of Early Product-Concepts. Dissertation, Delft University Press, Delft. Carlsson M. (1990): Integration of Technical Functions for Increased Efficiency in the Product Development Process. Dissertation, Chalmers University of Technology, Gothenburg. Catteral M.& MacLaran P. (1997): Focus Group Data and Qualitative Analysis Programs: Coding the Moving Picture as Well as the Snapshots, Sociological Research Online, vol. 2, no.1. 1997 Chapanis A. (1959): Research Techniques in Human Engineering. John Hopkins University Press, Baltimore, MA. Cooper R.G. (1988): The New Product Process: A Decision Guide for Management. Journal of Marketing Management, Vol.3, pp.238-255. Cooper R.G. & Kleinschmidt E.J. (1986): New Products: What Separates Winners from Losers? Journal of Product Innovation Management, Vol. 4, pp. 169-184.

10


Griffin A. and Hauser J.R. (1993) ‘The Voice of Customer’, Marketing Science, Vol. 12, No. 1, pp. 1-27. Herstatt C. and von Hippel E. (1992): From Experience: Developing New Product Concepts Via the Lead User Method: A Case Study in a 'Low-Tech' Field. Journal of Product Innovation Management. Vol. 9, pp. 213-221 Jordan P.W. (1997): An Introduction to Usability, Taylor & Francis Ltd., London. Karlsson, MA. (1996): User Requirements Elicitation. A Framework for the Study of the Relation Between User and Artifact. Doctoral Thesis from Department of Consumer Technology, Chalmers University of Technology, Gothenburg. Karlsson MA., Kaulio M., Sperling L. & Hampf J. (1998): Eliciting Product Requirements. Product Representations as Mediating Objects in Focus Group Interviews, In Proceedings for the 5th International Product Development Conference (EIASM), Como, May 25-26 th 1998. Karlsson I.C. MA. & Rosenblad E.F.S. (1998): Evaluating functional clothing in climatic chamber tests versus field tests: a comparison of quantitative and qualitative methods in product development. Ergonomics, vol. 41, No. 10, 1998, pp.1399-1420. Moore, W.L. (1982): ‘Concept Testing’, Journal of Business Research, Vol.10, pp. 279294. Rydebrink P., Kaulio M., Karlsson MA. & Dahlman S. (1995): Product Requirements Engineering – PRE. Att hantera kundkrav i utvecklingsarbetet, Report 95011, IVF och Chalmers tekniska högskola, Gothenburg (In Swedish) Page A. and Rosenbaum H. (1992): Developing an Effective Concept Testing Program for Consumer Durables. Journal of Product Innovation Management, Vol. 9, pp. 267-277. Schoormans J.P.L., Ortt R.J. & Bont, C.J.P.M., (1995): Enhancing Concept Test Validity by Using Expert Consumers. Journal of Product Innovation Management, Vol. 12, Soderman M. (1998): Product Representations. Understanding the Product in the Design Process. Licentiate thesis, Department of Consumer Technology, Chalmers University of Technology, 1998, Gothenburg. Ulrich K.T. & Eppinger S.D. (1995): Product Design and Development. McGraw-Hill, Inc., Singapore. Urban G.L. & Hauser J.R. (1993): Design and Marketing of New Products. Second Edition, Prentice Hall, Englewood Cliffs, NJ.

11