driven and operate in a B2B environment separated from end- user's real needs ... best-practices constitutes a myriad of frameworks, models and methodologies ...
Customer-focused development practices in Systems Engineering companies A case study across industry sectors
Torgeir Welo
Geir Ringen
Dept. Engineering and Materials NTNU Trondheim, Norway Richard Birkelands v. 2B, N-7491 Trondheim, Norway
Dept. Technology, Economy and Management NTNU Gjøvik, Norway Teknologiveien 22, N-2815 Gjøvik, Norway
Abstract—Establishing deep understanding of customers and their perception of value is a prerequisite to deliver commercially successful products in today’s hostile business environment. This paper elaborates on to which degree companies focus on customer value, along with their collaborative practices with customers undertaken in product development. Emphasis has been placed on comparing such practices in Systems Engineering (SE) companies with the ones in other industry sectors. We hypothesize that since SE companies are typically requirementsdriven and operate in a B2B environment separated from enduser’s real needs, this would imply that they are less focused on (understanding and satisfying) true customer value in their product development practices. A case study was designed to use two different methodologies for data collection. First, a literature review was conducted to establish an overview of theory and industry best-practices. The findings were synthesized, analyzed and decomposed into a set of twelve governing statements, which were implemented into a survey distributed to 50 companies. The outcome from the literature review was in parallel converted into a customer-focus capability maturity tool intended for in-depth, face-to-face assessments with nine companies. The results from the two data strategies were triangularized and used to test our initial hypothesis. The results from both methods indicate that product engineering practices in SE companies have a more distant relation to customer value than in many other industries. The implications are that there may be a potential for SE companies to define more integrated practices with direct interfaces towards customers to prevent missed value opportunities and hence improve innovation capability. Keywords—customer value; customer integration; engineering practices; survey; assessment; capabilitiy.
I. INTRODUCTION Companies need to move extremely fast just in order to sustain in today’s hostile marketplace. Enhancing a company’s innovation capability and practices is the most obvious countermeasure against this important challenge. Innovation is defined by [1] as: “the successful creation and delivery of a new or improved product or service in the marketplace ... Innovation is the process that turns an idea into value for the customer and results in sustainable profit for the enterprise.”
Hence, innovative products are needed because they provide new value to customers. Delivering new value to customers puts companies in position to differentiate and (partly) dictate the price of its offerings in the market place. Customer value, which is exclusively perceived by the customer, usually involves trade-offs between what is received (e.g. benefits) and what is given up to acquire this (e.g. money and resources), [2-4]. Hence, innovation can be said to be about delivering maximum customer value with minimum waste. Nowadays, customers demand increased quality and product performance, added functionality, lower prices, shorter product life cycles, high innovation speed, etc. As a result, companies must develop more desirable products ahead of their competitors, before new technology emerges or market conditions change. The overall question is then how product development practices best can be improved to sustain competiveness [5-9]. The body of knowledge associated with best-practices constitutes a myriad of frameworks, models and methodologies, representing both internal and external factors, impacts on performance and productivity, innovation and, finally, value as perceived by customers and stakeholders. The aim of this paper is to investigate how companies which employ Systems Engineering (SE) develop new products compared with other companies, when it comes to integrating perceived customer value into their engineering practices. Attempts have been made to identify similarities and differences between SE companies and other companies in the way they use understanding of customers’ (or end-users’) perceptions of value to gain competitive advantage. Since SE based companies are typically requirements-driven, and operate in a B2B business environment, we hypothesize that these companies are more reactive in defining and systematically utilizing customer value as a competitive factor. To test our hypothesis, we use two different methods for data collection. First, an online survey was established and distributed to product manufacturing companies. In the survey, customer value was decomposed into twelve statements, asking the respondents to rate company capabilities/practices on a 5point Likert scale. Second, an assessment system using a descriptive capability maturity grid approach was developed, conducting a guided self-assessment process with teams in nine of the companies. Four of these companies were typical SE
The reminder of this article is organized as follows: Section 2 presents a set of definitions and theory related to value and the perception of customer value. Section 3 summarizes the two methods employed for data collection, as well as the procedures used for data collection. Section 4 compares the results and presents the main findings using triangularization between the results obtained by the two methods. Section 5 gives the overall conclusions and prospects for further work. II. THEORY
y
n sig ing De ink th
og
ol hn c Te sh pu
Radical/Disruptive Transformational
Incremental
n
g in ist ex e c
a n tio rm ap fo ad er tp l ca duc di ra ro p
A. The meaning of customer value in SE environments The understanding of value is the most essential part of any company’s innovation efforts. The two sides of the ‘value coin’ are value as perceived by the customer and the price at which that value or benefits can be achieved. The former implies closing gaps between customer needs and current offerings. The latter is closely related to the efficiency of the organization in bringing an idea into a desirable product in the market place. However, separating value from waste is far more complicated in an engineering process since it is a problem-solving and knowledge transformation process. The artefact is information and knowledge used to mitigate the risk of taking a new product to market to an acceptable level for the company [10]. Thus, product development is a risk mitigation process, in which efficiency is directly related to the rate as which risks are mitigated. The key is transformation of data and information into knowledge that can be used to generate new valuable information and knowledge as a basis for decisions [11] According to [12], all the values are embodied in the essential deliverables needed to launch a new product: “any activity or task that transforms a new product design or the essential deliverables needed to produce it in such a way that the customer is both aware of it and willing to pay for it.” Hence, a specific activity generates value only if it increases the confidence in the ‘recipe’ such that the concept is more worth after the activity is completed. In the literature, especially within Lean, the opposite of value is denoted ‘waste’, whereas ‘necessary waste’ is used as a common term for administrative and coordination activities. Moreover, the term ‘strategic value’ is used for knowledge transformation activities that contribute to organizational learning, without necessarily directly mitigating risk in a product project. According to [13] customer value represents all the benefits that a customer, explicitly or implicitly, acknowledges with a product relative to its price. This is consistent with the more general definition of value used in early Value Engineering (VE) approaches—a methodology that was developed at General Electric Corporation during World War II. Nowadays, VE approaches usually focus on elimination of unnecessary cost elements, ones that provide no benefits to the user or other stakeholders throughout the product lifetime, rather than providing the necessary functions at the lowest possible cost, which was the original intention of VE. Customer benefits associated with a product are related to several characteristics (features, attributes, properties) as well
as meanings or ‘experiences’ of a product in everyday life [14]. These may be broken down into two categories: Product performance-related characteristics such as requirements, features, attributes, performance, functions, capacity, dimensions, quality, weight, etc. User-related characteristics such as second-hand value, cost of ownership, scarcity (availability), (self)esteem, design, style, fashion, experiences of the product and its use in the context, etc. In Fig. 1, the two dimensions are used to illustrate their mutual effect on innovativeness; i.e., an incremental performance improvement may turn into an incremental, or radical innovation when combined with product meanings. innovativeness
based companies operating in the defense, aerospace and subsea sector.
tin is ex
g
l ca di ra gs n
i
n n io ea pt m a ad uct
od pr
Fig. 1. Two dimesions of innovativeness (with inspiration from [14])
B. Researching organizational capbilites and practices Maturity models in various forms have been applied to assess improvement efforts within different functional areas in an organization, including product development [15]. The basic methodology of maturity grids includes describing in a few statements the typical behavior of an organization at a number of levels of maturity for a set of key characteristics of the (process) area to be assessed. The different levels provide the opportunity to codify what might be regarded as practice in accordance with a specific performance characteristic along with some transitional stages. Hence, the approach has many similarities with a questionnaire using a Likert scale [16] with response anchors. One of the benefits of the maturity grid is the descriptive text tied to a scale of each characteristic of performance to be assessed. A drawback is that the descriptive text becomes increasingly difficult and complex as the number of levels increase. There exist a number of different maturity models, all defining a number of dimensions at several discrete stages of maturity with a description of characteristic performance at the various levels. There are two main types of maturity models. The first one is traditional maturity grid method where all practices are scored to a different level (‘continuous’), see e.g. [17] and [18]. The second approach assumes that a cumulative number of process areas (capabilities) must be met at one level before advancing to the next level of maturity (‘staged’), which is described in detail in [19] There is also a third category combining a questionnaire with definitions of maturity without description
of the activity (‘hybrid’), as used by [20] for developing an assessment tool including seven areas with three NPD practices. In an attempt to study focus on customer value in SE organizations, the maturity grid approach was chosen as one of the methods since it serves the dual purpose of providing a means for process improvement and being an interactive research survey tool for collection of data related to where different companies identify their capabilities. Furthermore, according to [15], “the typical maturity model used in product development tends to be structured according to existing notions of good practice, and generated using experiencebased principles, and tested by qualitative approaches such as interview and beta testing, but with little quantitative or statistical analysis”. This was an important consideration due to the time frame needed for completion of the audits. The main inspiration for the format and structure of the assessment forms came from The Lean Enterprise SelfAssessment Tool (LESAT) due to its prowess and previously documented capabilities, [18]. However, the content was directed toward product development practices within category customer value at team level. It should also be noted that due to the large contextual differences between the companies, focus was placed on establishing a structure for identifying and analyzing capability gaps, in addition to focusing on maturity levels. III. METHOD A. Survey Norway-based manufacturing companies were chosen to participate in a descriptive survey to gain insight into focus on customer value in SE companies as compared to other industry sectors. The company profile is reported elsewhere [21]. A web-based survey was developed using the program Opinio. An e-mail with a link to the survey was issued to each of the company contacts. Non-probabilistic sample design was used as it was important to obtain information relevant to and available from certain groups. The target groups in the sample were product development and design engineers, quality engineers, manufacturing engineers, project managers and functional managers. The sample size in each firm was based on firm characteristics as well as the size of product development departments. The survey was structured into two parts: The introductory part included six questions dealing with general information about the respondent and environment. The second part included twelve statements associated with customer value. Each statement reflected a specific characteristic, including strategy, practices, behaviors, methods or tools, which collectively were intended to cover different aspects of customer value. The statements were assessed on a Likertscale from 1 (strongly disagree) to 5 (fully agree). This provides the respondents with the opportunity of indicating neutrality, although some respondents may see ‘3’ as the easy way out. The responses included totally 257 respondents in 50 different companies. The overall response rate varied between 23 % and 34%, depending on how it is calculated. More male
(83 %) than female (17 %) participated in the study, which is typical for this type of functions in the manufacturing industry. 75 % of the respondents had more than 5 years of experience, with 28 % more than 15 years of experience. More than 85 % of the respondents have BSc degree or higher, and 5 % held a PhD. Approximately 80 % of the companies have more than 5 people that work in product development, typically between 6-15 people. Most of the respondents worked (partly) in product development/engineering (53%), while 27 % worked in project management and 25 % in design. Other functional areas were quality, process development, manufacturing, coordination, market, purchasing, production and management. Some people had multiple responsibilities, particularly in the smaller companies. The data was separated into two industry segments; Systems Engineering (SE), and Others (Oth). The classification of Systems Engineering companies was mainly done qualitatively based on company characteristics and practices; e.g. defense, aerospace and subsea industry. The data was separated into the two industry segments at individual respondent level, calculating respectively mean and standard deviation for each segment and question. Then, an independent two-tailed t-test was run to determine significant differences on mean value level between the different industries. The level of significance was set to 10%. B. Capability maturity assessment framework As a first step in developing the assessment tool, the customer value component was divided into sets of characteristics, which describe the main attributes of the component. Each of the characteristics was decomposed further into subsets of three practices (sub-characteristics). Each sub-characteristic was given situational descriptions of capability for different maturity levels. To reduce complexity and detail level of the descriptions, descriptive statements for three different maturity levels—low (1), intermediate (3) and high (5)—were codified and linked to a Likert-scale. The respondents were asked to interpolated between low and intermediate (2) and intermediate and high (4) in case the identified maturity level of a specific practice appeared to be between the levels described. The same methodology was used for assessment of current and desired future capabilities. An example of an assessment sheet for one of the characteristics is shown in Fig. 2: Roles and Values (A), which is a part of core component, Customer Focus. The main question associated with this characteristic is to what extent the company works collaboratively with its customers to better understand their current and future needs. This characteristic is further decomposed into three factors, including (1) Perceived role of the customer, (2) Collaboration with customers within the NPD team and (3) Involvement of customers in improvement of product quality. Each of the three factors is provided with a set of descriptive situations tied to a maturity scale, where the assessment for characteristic l of component k (customer value) is done by identifying the current situation of the company ( ), and where it needs to be in the future ( ), depending on the business context. Here i refers to each of the three factors that make up each characteristic. The current
capability maturity rating for characteristic (l) of each component (k) is given by
(1) with the overall rating for each component (k) given as,
(2) Here N = 2 is the number of characteristics (practices) defined for each component. Similarly, the corresponding gaps are therefore given by (3) with the overall gap for each component given as,
(4) where the last summation represents the characteristics (l) defined for each component. As stated above, the main purpose of the assessment framework is to identify customer value capability gaps between current and desired future practices, rather than score absolute maturity levels for each practice. Hence, assessing capability gaps is the first crucial step in a suitable strategy to define organizational improvement initiatives.
predicted for, respectively, systems engineering (SE) and Other companies. The right hand column lists the results from a t-test of the significance of the mean value difference between SE and Other companies. It is noted that the difference is significant to a 0.1 level in five out of twelve responses. The most significant difference is related to using visualization to align customer value with product, strategy and company values [22]. Also, SE companies to a less degree integrate customer value into mission/vision statements at company level (0.035 significance level). This result is quite noticeable, and could possibly be traced back to the fact that SE companies typically operate in a B2B setting. The third statement with significant difference is related to whether SE companies define a chief engineer whose main responsibility is to take the role as customer representative, and have the single responsibility for the performance of the product or system. For example, Toyota is reported to use this role to integrate and trade-off customer needs with organizational capabilities [9]. Hence, the role has much in common with a ‘heavyweight project manager’ [23] although more technical, whose tasks are e.g. to challenge functional managers (knowledge owners), standards, and established perceptions of (technical) risks. According to the survey results, SE companies have a tendency to do less field activities than Other companies in order to identify customer needs and use the input to strategically position the company in the market place or pursue internal R&D. This result is not very surprising; rather than responding to market and customer research, SE companies are used to respond to ‘hard requirements’ set by B2B customer, which in many cases are state organizations or governments. Thus, barriers must be omitted to reach the user or customer than is the case for companies that produce more typical consumer products. B. Assessment Although the format of the assessment was different from the survey, the overall problem statement was still the same: to which degree does your company actively integrate customer Table 1. Descriptive statistics and T-test results. Qestions (Customer Value)
1
Fig. 2. Assessment sheet for Customer Focus (Roles and Values).
IV. RESULTS A. Survey The results from the survey are summarized in Table 1, showing each of the twelve statements related to customer value. The table shows the mean and standard deviation
Customer value is expressed in companys mission/vision statement?**
Mean SE Others
St.dev Pr > SE Others [t]t p < 0.05
3,91
4,19
1,05
0,80
0,035
Satisfying customer value is the primary driver for our strategy 2 and activities in product development teams?
4,05
4,17
0,84
0,73
0,261
The customer has a defined/integrated role in our companys 3 New Product Development (NPD) process?
3,79
3,76
0,92
0,98
0,815
Every project team member knows critical product 4 characteristics that influence perceived customer value?*
3,25
3,46
0,92
0,92
0,086
Field activities are pursued to track customer needs for specific 5 projects as well as strategic planning?*
2,80
3,06
1,00
1,07
0,056
Our product portfolio is primarily based on our ability to satisfy 6 individual customer value?
3,71
3,80
0,91
0,79
0,481
There is a highly competent and experienced person with 7 authority and responsible customer representative?*
2,90
3,22
1,27
1,07
0,052
We aim to assemble multidisciplinary teams where members 8 have different channels to capture customer input?
3,26
3,42
1,09
1,04
0,264
Our company uses various methods and combinations to 9 actively capture customer/user needs?
3,24
3,16
1,03
0,90
0,541
We try to combine what the customer expresses with what we 10 think the customer would like to express in our process?
3,54
3,72
0,97
0,84
0,160
Customer communication interface is defined with the purpose 11 of creating direct communication channels?
3,48
3,32
1,14
1,04
0,309
We use visualization tools to create alignment between 12 customer value, product characteristics and company values?**
2,51
2,87
1,11
1,09
0,017
needs in the development process of new products? While the survey used twelve statements to collectively answer this question, the assessment used two essential questions: (A) To what extent does your company work with the customer to understand current and future customer needs and wants? (B) How do customer wants, needs and requirements reach design engineers (project teams)? These two questions were answered through six characteristics using an approach with situational descriptions at different levels tied to maturity scale. As shown in Fig. 1, (A) is decomposed into (1) perceived role of customer; (2) collaboration with customer; (3) customer voice in quality improvements. (B) was decomposed into (4) integrated, collaborative role of customer with the company in developing products; (5) methods design engineers use to identify customer needs; (6) Feedback loops between customer and engineering team as input to design The assessment team consisted of people with the most common roles in new product projects, including functional managers and executives. The assessment team first individually scored both current and perceived, desired capabilities/practices related to (1)-(6) above, taking into account the actual context of the company. Then, the individual scores were discussed within the group to arrive on a collective score. Hence, the assessment approach provides at least three additional dimensions of input compared to the survey: (i) customer focus is evaluated in a contextual perspective; ii) the two-stage rating process facilitates constructive, in-depth discussions while resolving misconceptions typically related to ‘silo-thinking’ between disciplines/functions; iii) assessing capability gaps allows for a more valid comparison between scales, characteristics and companies—both Likert scales (survey) and maturity grids with situational descriptions (assessment) tied to ordinal scales generally fail to capture information that will be present in other scales due to unintended differences between the levels of the various ordinal scales. The results from the assessments are summarized in Table 2. Among the nine companies assessed, four companies were classified as systems engineering (SE) companies. Although the sample size by no means allows for discussing the results in terms of statistical significance, the table provides values averaged across the sample. Here it is worth noting that each number provides more information than the specific value would indicate from a statistical point of view due to the way the data were collected throughout the assessments. In terms of assessment topic (A), the SE companies in average rate their capabilities at 3.24 based on the descriptive ordinal scale used. The Other companies rate their capabilities 0.66 higher on the same scale. The results for the corresponding assessed gaps—i.e., the difference between current capability levels and the levels deemed necessary to sustain competitive advantage—is even more interesting since this value is more ‘normalized’ to the business context that the company operate. SE companies assess an average gap (G) of 0.87, which is quite significant, as compared to Other companies which rate the gap at 0.57 for topic A. Here it should the noted that for each company the assessment of capability A is based on three essential capability characteristics (1)(3) combined, which increase the validity in itself.
Considering assessment topic (B), the interfaces and setup between engineers and customer, the same tendencies are observed. The SE companies scores current capabilities at 2.93, while the Other companies are at 3.47. The difference between the capability gaps are even more pronounced: 1.41 and 0.75 for SE and Other companies, respectively. Combining the two topics (A) and (B) the SE companies rate their customer-focus capability at 3.09 versus 3.68 for Other companies. The corresponding gaps are 1.14 and 0.66 for the SE and Other categories of companies, respectively. C. Triangularization The format and process for the two data generation methods were different. Both methods have their strengths and weaknesses. Demonstrating statistical significance of a survey approach presumes large samples. Table 2. Results from assessment, component 1, Customer value (1). Assessment Company: Topic Current (A) Current (B) Current
Avg C 1
C
2,5 2,5
3,7 3,4
3,6 2,6
2,5
3,5
3,1
3,09
3,68
3,42
0,6 1
1,5 2
1,1 1,3
0,3 1,4
0,87 1,41
0,57 0,75
0,70 1,04
0,80
1,75
1,17
0,85
1,14
0,66
0,87
SE2 3,2 3,2 3,2 G
Gap Gap
(A) (B)
Gap
Avg G
1
1l
Mean SE 3,24 2,93
SE1
SE3
SE4
Mean Other 3,90 3,47
Mean All 3,61 3,23
1l
Also, it requires a neutral, easily-understandable format such that the respondents interpret the statements correctly. Even more importantly, the survey approach relies on reaching the target group with the right background and attitude to provide input of objective data. The assessment method is resource demanding, which implies low sample size. On the other hand, it provides an opportunity for more in-depth reviews, as well as less likelihood for misunderstanding and responses with varying degree of seriousness. The characteristics of assessment methods using a maturity grid approach with descriptive situations conducted in a workshop format resemble more the situation of semi-structured interviews. In our study, the scoring was always corrected after the individual part of the assessment when the differences between the different individual ratings were unacceptable. The results from the two methods are combined and compared in Table 3. The survey statements that are relevant for each assessment topic are listed next to the statement. Bold numbers indicate that the survey demonstrated statistically significant differences between categories SE and Other. Hence, there are two survey statements reflecting assessment sheet (A) that demonstrated significant differences between the two groups of companies. At the same time, the assessments indicated that group SE rates their capabilities within (A) 0.66 lower than group Other on the descriptive ordinal scale used. The corresponding gap between current and desired capabilities was 0.3 when using the capability maturity grid method. Similarly, three survey statements within (B) were statistically different for the two groups. Here the SE companies rated their capabilities 0.3 lower than the other group. The corresponding perceived capability gaps were 0.66 higher within topic (B) for SE companies compared to Other.
Overall, the results obtained by the two methods point in the same direction, supporting our initial hypothesis; i.e., systems engineering companies typically focus less on understanding true customer needs and wants than do other (manufacturing) companies in other industry sectors such as automotive. As pointed out above, both methods provide different strengths and weaknesses. It is believed, however, that since essentially the same trends were obtained by both methods, this makes the significance of our findings stronger. A natural next research step is therefore to delve into potential implications and opportunities associated with these finding. Hence, given that SE type companies focus less on establishing deep understanding of customer/end-users interaction with their (new) products, what could these companies potentially gain in strengthening their competitive advantage by employing more customer-focused strategies in their product development efforts?
REFERENCES [1] [2]
[3]
[4] [5] [6]
[7] [8]
Table 3. Synthesis and comparison of results by the two methodologies.
Assessment (sub)topic
Survey statement
Perceived role of customer
A1
1,2,6,12
Collaboration with customer
A2
11
Active involvement of customer in quality improvements
A3
Customer integration in developing new products
B1
3,7,8,9
Engineering practices to identify customer needs
B2
5,10,13
Design feedback loops between customer and engineering team
B3
4
C
G
Mean
mean
[9]
C.R. Carlson and W.W Wilmot, Innovation: The Five Disciplines for Creating What Customers Want, Crown Business, 2006, New York. J.M. Spiteri and P.A. Dion, Customer value, overall satisfaction, enduser loyalty, and market performance in detail intensive industries. Industrial Marketing Management, 33, pp. 657-687, 2004. R.B. Woodruff, Customer Value: The Next Source for Competitive Advantage. Journal of the Academy of Marketing Science, 25(2), 139153, 1997. H.E. Butz jr. and L.D. Goodstein, Measuring Customer Value: Gaining the Strategic Advantage. Organizational Dynamics, 24, 63– 77, 1996. R.G. Cooper and S.J. Edgett, New Product Development, Lean, Rapid and Profitable, 2005, Ancaster, Product Development Institute. I.K.-vDrongelen, B. Nixon, and A. Pearson, Performance Measurement in Industrial R&D. International Journal of Management Reviews, 2000. 2: p. 111-143. B. Haque and M. James-Moore, Applying Lean Thinking to New Product Development. Journal of Engineering Design, 2004. 15(1): p. 1-31. D.N. Mallick and R.G. Schroeder, An Integrated Framework for Measuring Product Development Performance in High Technology Industries. Production and Oper. Management, 2005. 14(2): p. 142-158. J.K. Morgan and J.K. Liker, The Toyota Product Development System, Integrating People, Process and Technology. 2006, New York: Productivity Press.
[10] D.G. Reinertsen, (1999) ‘Lean thinking isn’t so simple’,
0.66
0.30
0.54
0.66
V. CONCLUSIONS Based on the research conducted in this study, the following conclusions can be drawn:
A literature review has been conducted to establish state-of-the-art and best practices within customerfocused product engineering practices;
A survey and assessment have been developed and used to determine capabilities and practices regarding customer-focused product engineering
SE companies demonstrate to provide less focus on understanding customer needs and wants (other than stated requirements), and integrating customers into their development activities;
SE companies may have an untapped innovation potential in pursuing more customer focus as a means to create competitive advantage. ACKNOWLEDGMENT
This research was funded by the Research Council of Norway under the program Regionale Forskningsfond. The authors gratefully acknowledges the financial support.
Electron Des., Vol. 47, No. 19, p.48. [11] T.R. Browning, T.R., ‘On customer value and improvements in product development processes’,Systems Engineering, Vol. 6, No. 1, pp.49–61, 2003. [12] R. Mascitelli, The Lean Product Development Guidebook, Technology Perspectives, 2007. [13] T. Welo, On the application of lean principles in Product Development: a commentary on models and practices, Int. J. Product Development, Vol. 13, No. 4,pp.316–343, 2011. [14] R. Verganti, Design-Driven Innovation: Changing the Rules of Competition by Radically Innovating What Things Mean. Harvard Business Press, Boston, 2009. [15] P. Fraser, C. Farrukh and M. Gregory, Managing Product Development CollaborationsA Process Maturity Approach, proc. Instn. Mech. Engrs, Vol. 217, Part B, pp 1499-1519, Journal of Engineering Manufacture, 2003. [16] R. Likert, A Technique for the Measurement of Attitudes, Archives of Psychology, 140, 1932. [17] V. Chiesa, P. Coughlan and C.A. Voss, Development of a Technical Innovation Audit, Journal of Product Innovation Management, pp. 105136, 1996 [18] D.J. Nightingale, and J.H. Mize, Development of a lean enterprise transformation maturity model, Information Knowledge Systems Management, vol. 3, no. 1, pp 15-30, 2002. [19] M.B. Chrissis, M. Konrad and S. Schrum, CMMI for Development – Guidelines for Process Integration and Product Improvement, Pearson Education Inc., 2011. [20] T.L. Doolen, and M.E. Hacker, A Review of Lean Assessment in Organizations: An Exploratoryy Study of Lean Practices by Electronics Manufactures, Journal of manufacturing Systems, Vol. 24/No 1, 2005. [21] G. Ringen and T. Welo, Knowledge based development practices in systems engineering companies: A comparative study Systems Conference (SysCon), 9th Annual IEEE International, 2015. [22] T.O. Olsen and T. Welo, Maximizing Product Innovation through Adaptive Application of User-Centered Methods for Defining Customer Value, J. Technol. Manag Innov. 2011, Volume 6, Issue 4. [23] K. Kuwashima, Three Footnotes to “Heavyweight Product Manager”, Annals of Business Administrative Science 12, pp. 265–276, 2013.
