Engineering practice – an empirical study Bill Williams
José Figueiredo
CEG-IST, Lisbon and ESTBarreiro, Setubal Polytechnic Institute, Portugal
[email protected]
CEG-IST and IST, UTL, Lisbon, Portugal
[email protected]
ABSTRACT Empirical data is presented from a study of the practice of recently graduated engineers working in Portugal and we situate the findings in ongoing work to find an appropriate model to categorize the practice of engineering professionals and to draw out useful conclusions for the design of engineering courses and the competences to be developed in them. Using an online survey structure, we have gathered data on an initial group of 96 recently-graduated engineers in Portugal. The results obtained show a pattern very similar to that previously obtained by Australian researchers in that our survey participants tend to spend the majority of their working week (around 60%) engaged in activities which involve interaction with others (meetings, supervision, writing reports etc) and around 40% is devoted to technical engineering activity. We also gathered data on these engineers’ perceptions of the skills they call upon most frequently in their professional activity and where they have acquired them – in formal study or on the job. The fact that the results obtained in a Southern European context appear to be similar to those found in the Australian engineering context certainly merit further study and suggest that the Unifying Model proposed there could have broad application. The findings also raise questions for engineering educators regarding the preparation of novice engineers. Keywords: model, engineering practice, survey, professional competences.
not take into account. These included activity like communication with clients and colleagues and engineers applying their skills outside their immediate specialist sphere.
1. Introduction In the work reported here we set out to gather empirical data on Portuguese engineering practice to compare with the results published for Australian engineers which is intended to contribute to our long-term aim of developing a clearer picture of what engineers do and to consider the implications for engineering education. The study originated from an attempt to understand the activity of a successful start-up voted the most innovative national company in a survey of Portuguese CEOs in 2009. The initial methodology was a qualitative one using the Gibbons Mode 1 and 2 model of engineering design and knowledge production [1]. Although the Gibbons model did serve to describe the overall activity of engineers at this company there were some important aspects which were not related to design and knowledge production which this model does
WEE2011, September 27-30, 2011, Lisbon, Portugal. Editors: Jorge Bernardino and José Carlos Quadrado.
Later that year James Trevelyan published his Unifying Model [2] that investigators at the University of Western Australia devised to explain a large body of empirical data they have gathered on the professional activity of engineering graduates of the university. As this model does take into account the aspects mentioned above, it seems to present a promising approach to characterize engineering practice.
1.1 Background To set the present work on engineering practice in context it is useful to first take a historical overview of work related to the practice of managers as both the models developed and results obtained there predate most similar work relating to engineering practice and its models. The two most relevant strands of this work arise respectively from the publication of Mintzberg’s book The Nature of Managerial Work in 1973 [3] and the later appearance in 1994 of Gibbons’ book The New Production of Knowledge [1]. We will now look in a little more detail at these visions of managerial practice because, as will be seen later, each has been an influence on our own work in that our initial study used Gibbons’ model as a central qualitative focus whereas what we learnt from the data gathered has lead us to our current quantitative approach which can be seen as owing more of a debt to Mintzberg. Mintzberg’s work was hailed as revolutionary in the 70’s as it was the first to intensively explore what managers actually do: his 1973 book was based on his doctoral dissertation – a study of the working life of five chief executives. This early work has been developed by him and others [4, 5, 6, and 7] since that time and he himself has refined his ideas in the recently-published Managing [8] where he returns to a similar approach, this time drawing on his observations of twenty nine managers in business, government and health. In this work he sets out the following broad model to describe the practice of managers which shows the manager as interacting in three dimensions: within their unit of responsibility, with the rest of the organization and with the exterior. Although this approach could be useful if applied to engineering practice, when we set out to study engineers working in our national context in 2009, it seemed more promising to build on the work of Gibbons as this, in addition to relating to general managerial practice has a strong focus on knowledge production and on design, both of which we expected would be characteristic of any description of engineering practice. Indeed it is often assumed in the literature that engineering design is at the heart of engineering education and is what distinguishes it from other scientific areas within higher education [9]. Gibbons’ The New Production of Knowledge [1] led to considerable attention being focused on two distinct models of knowledge production, identified by these authors as Mode 1
263
(associated with a traditional academic discipline-based approach) and the more recently-emerged Mode 2 (a context-driven and problem-focused process more common in the entrepreneurial sphere). The differences between these two approaches can be summarized as represented in Figure 1 [10].
Context Innovation Community
Orientation Method
Mode 1
Mode 2
academic, scientific linear
economic and social applications problems are set and solved in the context of application
disciplinary, homogeneous teams, university based explanation, incremental replicability is important
transdisciplinary; networked; heterogeneous actors
Quality assurance
peer-review central
Definition of success
scientific excellence
is
solution focused replicability not vital ( there may be secrecy/copyright issues) context dependent: may involve peer-review; customer satisfaction efficiency; satisfy multiple stakeholders: commercial success
Figure 1 parameters associated with Gibbons’ Modes 1 and 2 of knowledge production An example of the influence of Gibbons’ work can be seen in the special edition of the European Management Journal in 2002 that was devoted to articles about Mode 2 management knowledge in the context of “Future Challenges for Management Research” [11]. _________________________ We proceed with a methodological overview (Section II), a discussion on the data we obtained (Section III), and finalize with some tentative conclusions.
2. Methodology 2.1 Overview In their very useful review of how social science research paradigms can be constructively applied to engineering education research, Borrego and her colleagues describe four types of mixed methods design which incorporate quantitative and qualitative phases in different configurations [12] and which are based on an approach set out by Cresswell [13]. As will be seen below, our methodological design constitutes an example of exploratory mixed methods in Creswell’s typology although this arose as a logical progression from an initial design adopted which was qualitative and narrative-based. The qualitative narrative-based approach was originally chosen as this is known to be useful in a comparatively unstudied area. In contrast to quantitative studies, with their emphasis on large representative samples, qualitative research often focuses on smaller
groups to examine a particular context in greater detail by reading rich contextual descriptions [12]. Taking an inductive approach to data analysis, it allows themes or categories to emerge from the data which is indeed what occurred in this case. This then leads to questions of transferability of the insights obtained from the qualitative phase to see to what extent they can be generalized to a larger population independent of context and, as in our case, this normally implies quantitative studies. The qualitative phase of our work was described in some detail in an earlier paper [14] and is here summarized to provide context for the subsequent quantitative phase which is the focus of this paper.
2.2 Qualitative Phase – Gibbons’ Model To better understand the practice of today’s engineers and its implications for education it was decided to focus initially on one national company. In 2009 the US-based consultancy firm, Strategos, polled 186 CEOs and senior figures in leading Portuguese companies to gather data on which international and national firms they considered to be the most innovative [15]. The national company with the highest vote was YDreams, a recent start-up which was created in 2000 when a group of engineers from a successful university research department at a Lisbon university felt the need to move from a university to an entrepreneurial context. YDreams has since had considerable national and international success in the areas of interactive spaces. The authors decided to begin a pilot-study of the YDreams team to trace their historical development and see what lessons could usefully be drawn for engineering education by studying the knowledge production and practice of such an innovative organization. The key engineers in this start-up were originally part of a successful university research department at the New University of Lisbon throughout the 90’s in the field of environmental engineering and IT. Their Environmental Systems Analysis Group (GASA) was known for its pioneering work in a field that has since become dominated by Google Maps. In 2000, frustrated by the limitations encountered within the academic system, they effectively set aside the projects they had been working on and dedicated themselves to an entrepreneurial start-up. Their YDreams Company has since come to enjoy considerable international success in the interactive space and ubiquitous computing sectors (http://www.ydreams.com/). Given the history of this firm, the Gibbons’ Model seemed appropriate to consider as a way of characterizing the practice of its engineers. Accordingly, Gibbon’s Mode 1 and 2 Model of knowledge production (and engineering design) were used as a basis for interviews with two of the founders of the firm and the narrative data obtained was analyzed qualitatively, guided by the work of Czarniawska on narrative research methods [16]. Separate lightly-structured interviews were carried out with: António Câmara, CEO of YDreams and former head of the GASA group and Edmundo Nobre, YDreams administrator (CFO) and co-founder. The interviewees were previously sent a brief summary of the intended case-study, which included the table in Figure 1, and invited them to relate their experiences in individual interviews; they were not given specific questions to answer but rather encouraged to relate their practice in the context of the GASA research group and contrast this with their current activity in the entrepreneurial context of YDreams. The interview transcripts were then analyzed for material relevant to the Mode 1 and 2 parameters and organized under the relevant headings [14]. Subsequently a bracketing process [17] was applied to these first extracts and the transcripts were scanned for additional references to engineering practice.
264
The data gathered suggested that, although the Mode 1 and 2 characterizations can be seen as useful here to characterize in a broad sense the knowledge production activity of the group of engineers in question and to help us accompany what seems to have been a significant phase change in the work of the group, they should not be seen as an either/or way of describing the real-world activity involved. In addition, having bracketed the previous extracts, rereading of the interview transcripts threw up some additional comments, which led us to rethink our original model of engineering practice and reexamine our concept of what engineers actually do: CFO: Right now, I recall off the top of my head that our head of Quality, head of Research, head of Software, our top programmer and the account managers of our best accounts all come from an environmental engineering background and this pluri-disciplinarity, the capacity to handle a broad sweep of areas, is very valuable in a company like ours. These roles involve heading up teams where skills of dealing with both a range of multidisciplinary projects and with their commercial aspects are vital. Obviously when we get to the execution, programming, design and so on then we will call upon our specialized people for these very specific functions. CEO: What worries me is that engineering courses in our national universities don’t have a tacit curriculum like you can find at top international institutions: our graduates have the technical skill; they can solve problems but are not good at explaining them. One feels they were well trained in problem-solving but at the expense of important skills like analyzing, communicating and debating which contribute to the kind of structured thinking we need.
2.3 Quantitative phase – Applying a Model based on empirical data Reflecting on the two preceding interview abstracts and noting that a significant number of engineers in the company were in posts of responsibility outside conventional engineering domains and that the CEO identifies communication skills as a priority for success, has lead us to see limitations in our original model of engineering practice which was based around engineering design and knowledge production and encouraged us to look for alternatives, particularly for models based on an empirical approach. Although the vast majority of literature on engineering practice tends to view it in terms of design or technical problem-solving [18, 19, 20, 21, 22] such models are rarely based on empirical studies analyzing what engineers do. A model proposed by James Trevelyan of the University of Western Australia in 2009 does however address these issues [2]. The Unifying Model of Engineering Practice sees engineering as a social system involving a sequence of steps common to most engineering activities and which are enclosed within a scaffold that continually guides the implementation steps towards the intended objectives. The scaffold in turn involves continual interaction between all the participants, including the client, financiers, engineers, contractors, suppliers, production and service delivery workers, technicians, regulators, government agencies, local community and special interest groups. This model grew from a longitudinal study of engineering graduates in Australia [23, 24] which showed that the majority spent less than 30% of their working time on specialized problem-solving and technical coordination emerged as the most
frequently mentioned activity. The study found that graduates asked to estimate time spent on different aspects of their work estimated that nearly 60% of their time is spent interacting with other people, including working face to face, meetings, correspondence, reports and working with human-readable data in information systems and this finding would tend to give weight to Antonio Câmara’s comment on the importance of communication skills (see Table 2).
2.3.1 Methodology of quantitative phase The methodology of our quantitative study has been similar to that employed by the University of Western Australia longitudinal study: practicing engineers, alumni from two engineering schools divided equally between civil engineering (50% of respondents) and engineering management and IT recent graduates were asked to complete an online questionnaire which was available in both English and Portuguese. The survey questions were grouped into interactions with people face to face, through documents, and interactions with hardware and with abstract systems and data, as shown below: Face to face or telephone interaction with other people • with one or two other people face to face • on site watching and interacting with people doing the work you contribute towards • meetings • training sessions or courses • telephone calls Interactions with people through text or documents (including reports, specifications, drawings, plans, schedules, procedures, work instructions, operating or maintenance manuals, bills of materials, budgets, tender documents, invoices, contracts etc.) • text messaging, chat • e-mail correspondence, queries • reading or checking formal documents • writing, preparing formal documents [1]
Interacting with systems or abstract data • searching for information on internet, in filing systems, databases, libraries etc. • calculating, modeling, simulation, data analysis. • designing, drawing, creating software code. • debugging machinery, systems or software code. Interactions with hardware, site work • operating, testing, working hardware or systems • surveying, measuring, inspecting, or observing on site • maintaining the computers & filing systems you use in your work, installing or updating software etc. • other hands-on work with equipment hardware, construction etc. • searching for misplaced items. Respondents were asked to choose from the following: none, 15 hrs. Responses were interpreted as 0, 1, 3, 7 or 15 hrs respectively. The data from 96 respondents is summarized in Table 1.
265
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
WEEKLY ACTIVITY of 96 respondents Interacting with systems or abstract data [searching for information on the internet, in databases, filing system etc] Interacting with systems or abstract data [calculating, modeling, simulation, data analysis] Interacting with systems or abstract data[designing, drawing, creating software code] Interacting with systems or abstract data [debugging software, machinery or systems ] Interactions with hardware, sitework [operating, testing, working hardware or systems] Interactions with hardware, sitework [surveying, measuring, inspecting or observing on site] Interactions with hardware, sitework [maintaining the computers, filing system you use at work; installing/ updating software etc ] Interactions with hardware, sitework [other hand-on work with equipment, hardware, construction etc] Face to face or telephone interaction with other people [speaking with one or two other people face to face] Face to face or telephone interaction with other people [interacting with people or overseeing people on site] Face to face or telephone interaction with other people [meetings (admin or technical)] Face to face or telephone interaction with other people [training sessions or courses] Face to face or telephone interaction with other people [telephone calls/audio conference] Interactions with people through text or documents, SMS, Chat Interactions with people through text or documents email, postal correspondence, queries Interactions with people through text or documents: formal documents: reading or checking Interactions with people through text or documents: formal documents: writing or preparing TOTAL HOURS
TOTAL HOURS
% of TOTAL HOURS
CUMULATIVE PERCENTAGE
324,0
6,5
6
354,0
7,1
14
406,0
8,1
22
177,0
3,5
25
207,0
4,1
29
170,0
3,4
33
173,0
3,5
36
42,0
0,8
37
489,0
9,8
47
304,0
6,1
53
323,0
6,5
59
123,0
2,5
62
301,0 169,0
6,0 3,4
68 71
545,0
10,9
82
412,0
8,2
90
483,0
9,7
100
5002,0
100,0
Table 1 Average perceived working time breakdown of Portuguese young engineers (96 respondents)
Mechanical Engineers 64,3
Electrical; Electronic 65,2
Mechatronic Engineers 62,5
Other Engineers 64,1
AVERAGE
% interaction with others
Civil Engineers 68,7
% individual technical work
31,3
35,7
34,8
37,5
35,9
35,7
100
100
100
100
100
100
64,3
Table 2 perceived working time breakdown from a University of Western Australia study (30 - 40 respondents in each group)
266
Table 1 item
1
3
5
7
Table 1 item Searching for information on the internet, in databases, filing system etc Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere Designing, drawing, creating software code Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere Interactions with hardware, sitework: operating, testing, working hardware or systems Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere Interactions with hardware, sitework: maintaining the computers, filing system you use at work; installing/ updating software etc Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere
Calculating, modeling, simulation, data analysis Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere
Percentage 69 17 2 5 8
Percentage 69 17 3 5 5
Debugging software, machinery or systems Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere
Percentage 47 26 5 18 3
Percentage 26 43 9 11 11
Interactions with hardware, sitework: surveying, measuring, inspecting or observing on site Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere
Percentage 53 21 8 11 8
Percentage 11 71 0 13 5
Interactions with hardware, sitework: other hand-on work with equipment, hardware, construction etc Answer Learnt at University, Polytechnic Self-taught Learnt on training course Learnt from colleagues Learnt elsewhere
Percentage 39 26 13 10 13
2 Percentage 24 61 3 5 7
4
6
8
Table 3 Participants perceptions on where their skills were acquired
267
2.3.2 Validation of data All the data presented were obtained through the use of an online survey which was made available to an overall population of 470 alumni of two engineering schools and achieved a response rate of around 20%. The survey instrument has already been validated by researchers in the University of Western Australia by triangulating survey data with extensive interviews with engineers in the workplace [23,24] and we are in the process of carrying out a similar process for our national context. At this stage it has not been possible to analyze data to take into account gender, type of work and entrepreneurial activity but these are areas we would like to examine in future work.
3. Discussion The results displayed in Table 1 show a pattern very similar to that obtained by the Australian researchers [23], summarized in Table 2, in that our survey participants tend to spend the majority of their working week, 63% (highlighted in blue in Table 1) engaged in activities which involve interaction with others (meetings, supervision, writing reports etc.) while 37% (highlighted in light pink) is devoted to individual technical activity. Likewise the average hours worked per week (51 hours) were similar in the two studies (Australian value: 49,5 hours). The data in Table 3 suggests that for more individual, office-based technical work (items 2, 3 and 4) respondents perceive they are mainly calling upon skills they acquired during their formal higher education course. On the other hand, tasks which involve interacting with hardware and sitework (5, 6, 7, and 8) tend to be acquired on the job. As the respondents in this initial study comprise recent graduates in rather different fields, civil engineering, engineering management and IT, we cannot generalize further from the survey data. However, in future work the intention is to broaden the scope of the survey and also triangulate the results with on-the-job interview data which is expected to allow us to provide a richer picture of the working practice of young engineers. The fact that the results obtained in the Australian engineering context appear to be similar to those in a Southern European one certainly merit further study and suggest that the Unifying Model could have broad application.
4. Conclusion Our work and that of the Australian researchers would seem to suggest that rather than considering engineering practice exclusively in terms of engineering design and problem-solving, we need to take into consideration the empirical data which show that a large part of the day-to-day working life of engineers involves interacting with others (63% of perceived working time as shown for our sample in table 1). Most engineering courses focus predominantly on the technical and design aspects of engineering practice and provide little training in communication skills (active and receptive) or management competences (supervision, working and learning in teams or entrepreneurship for example). We believe that the data presented here brings up important questions for engineering educators as to how we can best prepare graduates for the engineering profession in a global context and we aim to address these issues in future work. Future research is expected to follow two lines: on the one hand to validate the engineering practice data presented here by carrying out interviews with practicing engineers and by collecting more data on the Portuguese engineering context; in addition to this, we hope to construct a broader picture of engineering practice by
comparing our data with research to be carried out in other countries in Southern Europe and in South East Asia.
5. Acknowledgments Financial support has been provided by two grants from the Portuguese Fundação para a Ciência e a Tecnologia of the Portuguese Ministry of Science and Technology and Higher Education (PTDC/CPE-PEC/112042/2009 and PROTECMCTES-33-2009 -IPS). We are also grateful to James Trevelyan and Sabia Tilli of the University of Western Australia for their invaluable advice and collaboration. The authors are indebted to the engineers who have participated anonymously in this study.
6. References: [1] Gibbons, M., Limoges C., Nowotny H., Schwartzman S., Scott P., and Trow M. (ed) (1994) The new production of knowledge: The Dynamics of Science and Research in Contemporary Societies, Sage Publications, London, 1994; [2] Trevelyan, J. P., “Steps Toward a Better Model of Engineering Practice,” presented at the Research in Engineering Education Symposium, Cairns, Queensland, Australia; [3] Mintzberg, H. , (1973) The Nature of Managerial Work, Harper and Row, New York, 1973.; [4] Nichols, T. and Benyon, H. (1977) Living with Capitalism, Routledge and Kegan Paul, London; [5] Silverman, D. and Jones J. (1976) Organizational Work, Macmillan, London; [6] Stewart, R., (1976), Contrasts in Management, McGraw Hill, Maidenhead; [7] Stewart, R., (1982), Choices for the Manager, Prentice Hall, Englewood Cliffs N.J.; [8] Mintzberg, H., (2009) Managing, Berrett-Koehler Publishers, San Francisco, 2009.; [9] Trevelyan, J. P., “Engineering Education Requires a Better Model of Engineering Practice,” presented at the Research in Engineering Education Symposium, Cairns, Queensland, Australia; [10] Figueiredo, A. D. & Cunha P. R., (2006) “Action Research and Design in Information Systems, Two faces of a single coin,” Chapter 5, Information Systems Action Research: Bridging the Industry-University Technology Gap, Ned Kock (Editor), Springer /Kluwer;, 2006. [11] Tranfield, D., (2002) “Future Challenges for Management Research” European Management Journal, Volume 20, Issue 4, August 2002, Pages 409-413 [12] Borrego, M., Douglas, E. P., & Amelink, C. T. (2009) “Quantitative, Qualitative, and Mixed Research Methods in Engineering Education,” Journal of Engineering Education, 98(1), 53-66, 2009; [13] Creswell, J. W., and Plano Clark, V.L. (2007), Designing and conducting mixed methods research, Sage Publications, Thousand Oaks CA; [14] Williams, B., Figueiredo, J. D., (2010) “Engineers and their practice: A case study” presented in EDUCON, 2010 IEEE, Madrid, Spain: IEEE, p. 531-535. [15] Strategos, (2009) “Business Innovation Survey Report (2009), Portugal,” (in Portuguese) available at http://www.strategos.com/articles/Strategos_BIS2009_Report.pdf on 30 October 2009; 268
[16] Czarniawska, B. (2004) Narratives in social science research, Thousand Oaks: Sage, 2004.; [17] Creswell, J.W. (2007) Qualitative inquiry and research design: Choosing among five approaches, 2nd edition. Thousand Oaks, CA, Sage Publications, pp 52, 361. [18] Atman C., Adams, R., Cardella, M., Turns, J., Mosborg, S., Saleem, J. (2007) “Engineering Design Processes: A Comparison of Students and Expert Practitioners” Journal of Engineering Education, 96(4), 359 -379, 2007;. [19] Jonassen, D., Strobel, J., & Lee, C. B. (2006) “Everyday Problem Solving in Engineering: Lessons for Engineering Educators”, Journal of Engineering Education, 95(2), 139-151, 2006.;
[20] S. Sheppard, et al, (2006) “What is Engineering Practice?”, International Journal of Engineering Education,. Vol. 22, No. 3, pp. 429-438, 2006.; [21] Vincenti, W. G. (1990) What Engineers Know and How They Know It: Analytical Studies from Aeronautical History. Baltimore: The Johns Hopkins University Press; [22] Vinck, D. (Ed.), (2003) Everyday Engineering: An Ethnography of Design and Innovation. Boston, 2003; [23] Tilli, S. and J. P. Trevelyan (2008). Longitudinal Study of Australian Engineering Graduates: Preliminary Results. American Association for Engineering Education (ASEE) Annual Conference, Pittsburgh [24] Trevelyan, J. P. (2007) “Technical Coordination in Engineering Practice,” Journal of Engineering Education, Vol. 96, No. 3, pp. 191-204.
269
