Creative Engineers: Is Abductive Reasoning Encouraged enough in ...

Available online at www.sciencedirect.com

ScienceDirect Procedia CIRP 50 (2016) 547 – 552

26th CIRP Design Conference

Creative engineers: Is abductive reasoning encouraged enough in degree project work? Ciarán J. O’Reillya,* a KTH ∗

Royal Institute of Technology, Centre for ECO2 Vehicle Design, Teknikringen 8, SE-100 44 Stockholm, Sweden

Corresponding author. Tel.: +46-8-790-8085. E-mail address: [email protected]

Abstract Creativity is considered to be an important ability for an engineer to have, and it is therefore important that the development of this ability is structured into the education of engineering students, along with the ability to apply, analyse and evaluate based on existent knowledge. In this paper, the importance of abduction in creative engineering processes is briefly reviewed. It has been shown that abductive reasoning plays a key role in design as it is the only logical operation that introduces new ideas. Its encouragement within the KTH Royal Institute of Technology’s degree projects at the Department of Aeronautical and Vehicle Engineering is analysed by examining the stated intended learning outcomes, and through interviewing students. It is found that abductive reasoning is not explicitly encouraged within the intended learning outcomes of these degree project courses, despite its importance in creative thinking. Although, it is very likely that at least some abduction takes place in the project work, its absence from the intended learning outcomes means that students may not have a felt need to demonstrate their abductive reasoning, and supervisors may encourage only non-creative deductive or inductive reasoning. A more explicit inclusion of abductive reasoning in the intended learning outcomes would help both students and supervisors to include creative thinking in the degree project courses. © Authors. Published by Elsevier B.V. This c 2016 2016The The Authors. Published by Elsevier B.V.is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Selection and peer-review under responsibility of Professor Lihui Wang. Peer-review under responsibility of the organizing committee of the 26th CIRP Design Conference Keywords: Creative design; abductive reasoning; education; degree project

1. Introduction Creativity has always been closely associated with engineering. The term itself is derived from the Latin ingenium meaning “cleverness” and ingeniare meaning “to create, contrive or devise”, and it shares its roots with ingenuity. For a technical university such as KTH Royal Institute of Technology it is therefore important that the ability to create and innovate is structured into the education of engineering students, along with ability to apply, analyse and evaluate based on existent knowledge. This is reflected in the KTH Vision 2027 for Education [1], which states that: “KTH education stimulates independent thinking, creativity and curiosity and applies critical examination of existing technological practices. Engineers and architects identify solutions which embody both innovation and enhancement with a clear social dimension, a distinct focus on sustainability and, for some, also an artistic dimension.” and: “KTH nurtures a culture characterised by solid knowledge in basic engineering areas, creativity, communication and ingenuity – valuable properties c 2016 The Authors. Published by Elsevier B.V. 2212-8271 Selection and peer-review under responsibility of Professor Lihui Wang.

in modern international settings.” An obvious challenge for teachers and supervisors at KTH is to understand where and how creative abilities can be facilitated and encouraged in the student’s learning. Where can creativity be encouraged is perhaps the easier part. Two areas where creativity is of value are in the creation of solutions (design) and in the generation of theories (science) – in other words, to create solutions to problems and to generate knowledge about a problem. In both cases, the generated designs or knowledge must be subsequently tested and verified. A natural place to set these types of activities in the engineering programmes is within project work, specifically degree project work. How can creativity be encouraged within project work is a greater challenge. A good starting point would be to structure it into the work through the Course Plan. Like all courses at KTH, degree project courses are defined within their Course Plans, which sets out the intended learning outcomes (ILOs), activities and assessment criteria. The Course Plan could therefore be interpreted as encouraging certain activities, at least in the sense that they set out what must be done. Students can of course always go beyond this level, but they are not explicitly encouraged to do so. As supervisors should facilitate the students’ achievement of the ILOs, it is maybe questionable how strongly learning outcomes beyond the ILOs can be pushed. The most robust approach would be to include creative aspects in the ILOs.

2212-8271 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the organizing committee of the 26th CIRP Design Conference doi:10.1016/j.procir.2016.04.155

548

Ciarán J. O’Reilly / Procedia CIRP 50 (2016) 547 – 552

Two sub-questions that derive from this are: 1. Is creativity, either implicitly or explicitly, included in the intended learning outcomes, and if so 2. What activities and assessment criteria ought to apply? Before addressing these questions, it is necessary to develop what encouraging creative thinking means here. Creativity can be defined as the ability to transcend traditional ideas, rules, patterns, relationships, or the like, and to create meaningful new ideas, forms, methods, interpretations, etc. [2]. The creative process is often necessitated by the need to solve problems, and at its core is the generation of new ideas or divergent thinking. One approach for promoting creativity in engineers is to encourage “intuition”, or the ability to understand something instinctively, without the need for conscious reasoning. Aside from implementation difficulties, this in its pure form may effectively amount to encouraging guessing or a trial-and-error strategy, which for solving large complex open problems is unlikely to result in a solution. A more rigorous approach is to encourage “logical reasoning” or a type of logical creativity to influence decision-making processes. Logical reasoning may be classified into three parts – deductive, inductive and abductive reasoning [3]. Of these, abductive reasoning is the only logical operation that introduces new ideas [4]. Creativity can be seen as intrinsically related to the process of producing new ideas, habits, etc. through abduction [5]. Therefore, creativity is intended here to mean this reasoned creativity based on abductive reasoning rather than intuition. Abductive reasoning is a form of logical inference, introduced by Charles Peirce [6], which goes from an observation to a theory that accounts for the observation. Ideally the simplest and most-likely explanation is sought. Abductive reasoning can be understood as as “hypothesis to the best design” or “inference to the best explanation”. It has been shown that abductive reasoning plays a key role in design [7–9]. This paper will focus on addressing the first question posed above, which amounts to “is abductive reasoning, implicitly or explicitly, included in the degree project ILOs?” In Section 2, abduction is briefly summarised. In Section 3, the ILOs for the degree project courses at KTH’s Department of Aeronautical and Vehicle Engineering are presented, and are assessed to see if they encourage abduction. This assessment is supplemented with interviews of students of these project courses that are presented in Section 4. The results of this assessment are discussed in Section 5 along with some discussion of the second question above on how abductive reasoning might look as a learning activity and assessment criteria. Conclusions are drawn in Section 6.

fer in the direction in which a rule like “a entails b” is used for inference. Table 1. Logical reasoning. Reasoning form

Explanation

Deduction

Deriving b from a only where b is a formal logical consequence of a. In other words, deduction derives the consequences of the assumed. Inferring b from a, where b does not follow necessarily from a. a might give a very good reason to accept b, but it does not ensure b. Inferring a as an explanation of b. Abduction allows the precondition a to be abduced from the consequence b.

Induction

Abduction

Design researchers generally promote abductive logic as the lifeblood of creative design [8,11–15]. In design, abductive reasoning is implicated in at least two important situations – in synthesising complex and contradictory information to generate insight, and in reasoning toward new solutions for design problems (i.e. from function to form) [11,16]. The latter form of abduction has been referred to as “innovative” abduction, with the former labelled as “explanatory” abduction [8]. Although many kinds of abduction may be classified, it is sufficient for the purposes of this paper to limit the discussion to explanatory and innovative. For a more detailed classification, the reader is referred to Ref. [17]. Abductive reasoning is not unique to design. In science, these two forms are usually referred to as “selective” (explanatory) and creative (innovative) abduction, with the latter being central to the growth of scientific knowledge given its emphasis on generating new plausible hypotheses that can be tested [18]. Simply stated, abductive reasoning introduces hypotheses and theories to explain given facts. For both design and science, the key to using knowledge effectively is to exploit all three forms of logical reasoning. This is because it is not enough to supplement the knowledge base with parameters from previous experiences only [19]. For example, in design one form of logical reasoning may be emphasised over another at different stages in the development, such as more innovative abduction at the start and more deductive logic towards the end [11,15]. Concept selection transcends merely selecting from clearly defined options, and should not only be about the evaluation of the design concept as it is. It should also be about inferring what it could be, which requires innovative abductive reasoning. An over-emphasis on deductive reasoning could inadvertently eliminate potentially fruitful concepts as an unintended consequence. In summary, it is therefore essential that KTH engineering students are encouraged to use all three forms of logical reasoning, but particularly abductive reasoning if creative and innovative abilities are to be developed.

2. Abductive reasoning

3. Assessment of the degree project courses’ ILOs

There are three forms of logical reasoning – deductive, inductive and abductive [10]. A short explanation of each is given Table 1. All three forms are integral to problem solving. Deduction and induction consist of drawing conclusions from what is known. Abduction explains what is known or most likely. Note that deductive/inductive reasoning and abductive reasoning dif-

At KTH, the first-cycle (Bachelor’s) and second-cycle (Master’s) degree projects are structured courses. This means that they each have a formal Course Plan with explicit intended learning outcomes (ILOs), requirements for completion, etc. The intention is that both the activities performed within the project, and the assessment of that work, are constructively


aligned with the stated ILOs [20]. An important aspect of this alignment is that the students should have a felt need to achieve the ILOs of the course. At the Department of Aeronautical and Vehicle Engineering there are seven degree-project courses offered in Technical Acoustics (SD211X), Vehicle Engineering (SD221X), Rail Vehicle Engineering (SD231X), Lightweight Structures (SD241X), Aerodynamics (SD261X), Naval Architecture (SD271X), Aeronautics (SD281X). All seven share the same ILOs. In fact, these are the same as the ILOs for the overall Master’s in Engineering (civilingenjör) programmes. The ILOs have been assessed on a three-point scale – Low, Medium and High – for how much they encourage abductive reasoning. The criteria for each level is given in Table 2. Table 2. Defined levels of abduction. Level

Description

Low:

Little or no reference to abductive reasoning. Promotes deductive or inductive reasoning. Possibly implicit reference to abductive reasoning. Either explicit or clearly implicit reference to abduction.

Medium: High:

The ILOs and their assessed levels of abduction (based on the ordinal scale in Table 2) are presented in Table 3. In making this assessment, the supporting text for Pass and Fail grades for the Master’s programmes, given in Table A.4 in Appendix A, is used to supplement the requirements stated in the ILOs in order to get a clearer picture of what the intention is. Table 3. ILOs of degree project courses at KTH Aeronautical and Vehicle Engineering along with the assessed level of abductive thinking encouraged. #

ILO

1

Demonstrate knowledge of the disciplinary foundation of the chosen subject area and best practice, advanced understanding in current research and development and advanced method knowledge. Demonstrate the ability to search, collect and integrate knowledge critically and systematically with an overall view of the subject. Identify the need for additional knowledge. Demonstrate the ability to identify, analyse, assess and handle complex phenomena, issues and situations also with limited information. Demonstrate the ability to plan and with adequate methods carry out qualified assignments within given time frames and to evaluate this work. Demonstrate the ability to develop and evaluate products, processes, systems, methods or technical solutions with regards to human needs and the aims of the society for economically, socially and ecologically sustainable development. Demonstrate the ability to orally and in writing in dialogue with different groups clearly explain and discuss the conclusions and the underlying arguments. Demonstrate the ability to make assessments considering relevant scientific, social and ethical aspects. Demonstrate the skill required to participate in research and development projects, or to work independently in similar qualified activities.

2

3

4

5

6

7 8

Abduction Low

Medium

ILO 1 talks about “demonstrating knowledge”, and in the Pass Criteria it mentions doing a literature review, and arguing “based on science and proven experience”. This is deduction/induction. ILO 2 possibly implies abductive reasoning. It talks of “searching”. However, from the Pass Criteria it states that there should be “synthesis of relevant literature”, which would indicate that it means that the search should be complete or thorough, rather than a more creative search or generation of new ideas. ILO 3 again might imply abduction as it talks of dealing with “complex phenomena”, and in the Pass Criteria it says that this should be done “even if the available information is limited”. It could be argued that this implies a certain amount of creativity, and perhaps ought to be interpreted as such. However, it is just as likely that it might be interpreted as applying existent knowledge in a top-down manner and being cautious about conclusions. It certainly has no explicit encouragement to be abductive. ILO 4 talks about planning and carrying out the task. There is nothing to indicate or imply that the student should be abductive. ILO 5 is perhaps the strongest is its encouragement for abductive reasoning. It uses the word “develop” which could be creative. However, in the Pass Criteria this is translated into “selecting an approach” and “implementing”, which are very much inductive. ILO 6 is very deductive. It states that the student should “present and discuss one’s conclusions and knowledge and arguments that are the basis for these”. ILO 7 states that they should be able to “make judgements”, which again implies that these should be deduced from existent knowledge. ILO 8 could be loosely interpreted as implying abduction as it talks about demonstrating “the skills required to work in research and development work”. It could be argued that being abductive is one such skill. However, in the Pass Criteria these skills amount to testing, evaluating and being “able to reject ideas and solutions” without mentioning generating them in the first place. Certainly this is not highly abductive.

Medium

4. Interviews with students Low

Medium

Low

Low

Five students who have recently completed or are near to completing one of the Master’s thesis project courses at KTH’s Department of Aeronautical and Vehicle Engineering were interviewed separately about their experiences. They were only told that the questions would relate to creativity in their project work at the start of the interview. Four questions were addressed, in the order given here: Q1. Did you have opportunity to be creative in your project? If so, in what way?

Medium

Overall, none of the ILOs have been assessed to have a High level of encouragement. Four are deemed to be Low and four as Medium.

Four of the five students answered yes and gave examples including tackling a research question with no obvious methodology at the beginning, and having to come up with their own research question in the first place. As Student 4 stated: “It wasn’t a predefined thesis where you ‘do this, do this, do this, and then you’re done’.”

549

550


These students said that they had to consult the literature about methodologies that could be applied to their specific case, or come up with a computer algorithm to solve a problem in an efficient way, etc. This would suggest that they had to be abductive in finding a solution that was not known to be an obvious next step. As Student 5 stated: “There was a lot of coding . . . so before getting it to work on a computer, you got to get it working on paper, and before this you got to get it working in your mind so it’s a sort of design process.” However, the remaining student, Student 3, answered: “I don’t know. The project idea was set out beforehand. I did dynamic testing of . . . It was difficult to interpret the results. It was more looking at what happened to the end result [of the test] and characterising it.” Although there most likely was still some degree of abduction in this student’s project (even if it may have been small compared to the others), they did not feel that had been very creative. Q2. Did you feel a need to be creative? If so, where did this felt need come from? Here none of the students felt like it was stipulated anywhere, but most felt that it had emerged as they had tried to solve the problems they were faced with, and two said that their supervisors had encouraged them to be creative. Interestingly, Student 5 connected the environment in which the thesis was carried out with the opportunity to be creative. “No, but I asked for a thesis at KTH, and not a company, because I wanted to avoid doing some measurement or certification that meant following a standard. Nobody pushed me to be creative but I think that in this [KTH] environment it was meant to be a creative task.” Q3. Do you think the project course ILOs required or encouraged you to be creative? If so, where specifically in the ILO text do you find this? Of the five students interviewed, only one said that they had read the ILOs of the project course that they had undertaken. When shown the ILOs most thought that creativity was implicit in some of the ILOs – ILO 3 and 5 were mostly picked out as implying creativity. Student 1’s assessment was typical: “I think in [ILO] number 3, that you should be able to handle something complex, you need some creativity. Definitely it’s implied. It’s not stating that you have to [be creative] though. Also in [ILO] 5, it’s implying some sort of creativity. There’s an underlying feeling that you should be creative but it is ambiguous.” Student 4 similarly said: “They (ILO 3 and 5) allow you to be creative but they don’t require it.”

The student’s assessments of the ILOs were broadly consistent with that made in Section 3. Q4. Are you familiar with different forms of logical reasoning – deductive, inductive and adbuctive reasoning? If yes, did you become familiar with them as a result of a course you have taken in your studies? None of the five students interviewed were familiar with the different forms of reasoning. Student 2’s response was typical of all those interviewed: “‘Deductive’ I think I understand. The other two I don’t know what they mean.” Student 5 also added “I had a course on theory and methodology in science, which is probably the only course that this might have dealt with this, but I don’t recall that we ever defined these in any sense.” 5. Discussion The encouragement of abductive reasoning within the project course ILOs that were assessed is rather low. Even those ILOs that have been assessed as Medium, only rather weakly imply abduction. It could be argued that a Medium score is rather generous as a student who has not acted in an abductive way could still argue that they have met all the ILOs. The point here is simply that the ILOs do not explicitly encourage abductive reasoning, and therefore creativity on the part of the student. This assessment is also consistent with the student’s not feeling that being creative was a requirement for completing their projects, and their interpretations of the ILOs that they expressed in the interviews. Despite this, it is still very likely that at least some degree of abduction takes place in the projects. Most of the interviewed students could point to examples of where they had been creative in their projects. Perhaps not surprisingly, the students believed they were more creative when the research questions were quite open, and they had the support of their supervisors to be creative. These factors point towards the environment in which the project is defined and undertaken, and not any specified need to demonstrate that they have been creative. There may also have been many ways in which the students were creative but they had not consciously registered them, perhaps as a result of not being primed at any point to reflect on their creative strategies and actions. This could have perhaps contributed to Student 3’s response to the first question. The problem with not explicitly encouraging abductive reasoning is that without a felt need to demonstrate their creativity in their project work, some student’s may complete the project task without developing this ability that KTH wishes to have in its graduates. Supervisors, and perhaps more so those at companies outside of the academic environment, may also not feel the need to encourage the development of abductive reasoning within their students. Perhaps a reason behind this low visibility of creativity in the ILOs is connected to the second sub-question identified in the Section 1 about aligning activities and assessments with the


ILOs that encourage creativity. This is perhaps due to a perception that creativity only comes through intuition, and that this would be difficult to assess. However, abductive reasoning could be structured into the project courses. Abductive reasoning is a cognitive strategy [8]. In other words it is one of the quite specific and deliberate ways of reasoning [11]. It could therefore be included in the ILO in much the same way as deductive reasoning is. So students could be encouraged to “discuss one’s hypotheses and the data which prompted the” in addition to “discussing one’s conclusions and knowledge that are the basis for these”. This looping between hypothesising and verification is of course well established in the scientific method, even if it not strongly visible within the ILOs. It would not take a huge leap to see how this could be required in activities and assessed in the degree thesis report. The student could simply provide evidence that they have used abductive reasoning, e.g. some sort of logic diagram. 6. Conclusion In conclusion the level of encouragement for abductive reasoning found in the degree project ILOs assessed was found to be Low to Medium with no explicit or strongly implicit encouragement to be abductive. This means that some students and supervisors may not feel the need for the student to be creative in their project work. A more explicit inclusion of abductive reasoning in the ILOs would help both students and supervisors to include creative thinking in the degree project courses. Acknowledgements The author would like to like to acknowledge the funding provided with the SRA TRENoP at KTH and the Centre for ECO2 Vehicle Design. The author would also like to thank the Master’s students at KTH who participated in the interviews, and Anna-Karin Högfeldt for her constructive comments at the start of this work within the course LH219V Supervision and Assessment of Degree Project Work in First and Second Cycle at KTH. Appendix A. Pass/Fail supporting text Table A.4 presents the supporting text for Pass and Fail of the ILOs. Note that these are given for the overall Master’s programme, but as the ILOs are the same for project courses, they are still relevant here. References [1] KTH Vision 2027; [cited 12 February 2016]. Available from: https://www.kth.se/en/vision2027/utbildning. [2] Liu Z, Schönwetter DJ. Teaching Creativity in Engineering. International Journal of Engineering Education. 2004;20(5):801–808. [3] Staat W. On abduction, deduction, induction and the categories. Transactions of the Charles S Peirce Society. 1993;29(2):225–237. [4] Peirce CS. Elements of Logic. Harvard University Press, Cambridge, Massachusetts; 1932. [5] Gonzalez MEQ, Haselager WFG. Creativity: Surprise and abductive reasoning. Semiotica. 2005;153(1/4):325–341.

[6] Peirce CS, Burks AW. The Logic of Drawing History from Ancient Documents, Collected Papers. vol. 7. Harvard University Press, Cambridge, Massachusetts; 1958. [7] Yoshikawa H. Design Philosophy: The State of the Art. Annals of the CIRP. 1989;38(2):579–586. [8] Roozenburg NFM, Eekels J. Product design: Fundamentals and methods. John Wiley; 1995. [9] Schum DA. Teaching About Discovery and Invention in Engineering. Technological Forecasting and Social Change. 2000;64(2-3):209–223. [10] Menzies T. Applications of abduction: knowledge-level modeling. International Journal of Human-Computer Studies. 1995;45(3):305–335. [11] Dong A, Lovallo D, Mounarath R. The effect of abductive reasoning on concept selection decisions. Design Studies. 2015;37:37–58. [12] Josephson JR, Josephson SG. Abductive Inference: Computation, Philosophy, Technology. Cambridge University Press, Cambridge; 1996. [13] Weigand KA, Hartung R. Abduction’s Role in Reverse Engineering Software. In: Aerospace and Electronics Conference; 2012. p. 57–62. [14] Almendra RA, Christiaans H. Decision making in the conceptual design phases: a comparative study. Journal of Design Research. 2009;8(1):1–22. [15] Lu SCY, Liu A. Abductive reasoning for design synthesis. CIRP Annals Manufacturing Technology. 2012;61(1):143–146. [16] Kroll E, Condoor SS, Jansson DG. Innovative Conceptual Design: Theory & Application of Parameter Analysis. Cambridge University Press; 2001. [17] Schurz G. Patterns of abduction. Synthese. 2007;164(2):201–234. [18] Magnani L. Abductive Cognition: The Epistemological and EcoCognitive Dimensions of Hypothetical Reasoning. Springer Verlog, Berlin/Heidelberg; 2009. [19] Aneta K, Jerzy P. Abductive and Deductive Approach in Learning from Examples Method for Technological Decisions Making. Procedia Engineering. 2013;57:1877–7058. [20] Biggs J, Tang C. Teaching for Quality Learning at University. Society for Research into Higher Education and Open University Press; 2011.

551

552


Table A.4. KTH Master’s in Engineering (civilingenjör) Pass/Fail criteria. ILO

Supporting text for Pass

Supporting text for Fail

1

The literature study is well executed. Current research and development with a bearing on the work is presented in a clear manner. The selected method is well argued for, based on science or proven experience and evaluated against other methods. Relevant knowledge from previous courses in the study program are used in an adequate way. The thesis task is handled autonomously and systematically based on critical analysis and synthesis of relevant literature. This work demonstrates a holistic view. Well-chosen databases and search tools are used. The need for further knowledge is discussed. Relevant complex phenomena, issues and situations are identified in the thesis. The work clearly shows that these are well managed and analyzed even if the available information is limited. Appropriate judgements related to the thesis’ research question(s) and its results are implemented. The work plan developed during the early part of the thesis has been followed. An advanced work is carried out within the agreed time and with the methodology agreed. Any changes in the planning or in the work consists of agreements between student and supervisor. Assets and limitations of the work performed is clearly stated. The selected approach is explained and implemented in such a way that the developed and evaluated products, processes, methods, systems or technical solutions, are tailored to people’s needs and conditions. Considerations to relevant societal objectives are taken in such a way that future generations’ possibilities to meet their own needs are not compromised. The report is well-organized, well-spoken and linguistically coherent. The argumentation for the conclusions are well implemented. The review of the sources is relevant, is independently formulated and well integrated. Oral presentation, opposition as well as the communication during work demonstrate the student’s ability to present and sensitively discuss the work and the conclusions with various parties such as clients, supervisors, teachers, researchers and students. The thesis work demonstrates judgement abilities, for example, being able to explain, justify, criticize and recommend. Relevant topicspecific judgements based science or proven experience have been made in the thesis work. The thesis work contains reflections on social and ethical aspects or have been justified as irrelevant. The student makes him/herself acquainted with the task and displays the ability to participate in the work culture that prevails where the task is to be solved. The student demonstrates the ability to test, evaluate and also to be able to reject ideas and solutions in discussions of the task. The student demonstrates initiative and is open to supervision and criticism. The work is to a large extent carried out independently.

The literature study is insufficient. Connections to current research and development is missing or deficient. Justification or evaluation of the selected method is insufficient. The work demonstrates limited knowledge from previous courses in the study program.

2

3

4

5

6

7

8

Relevant literature is largely lacking or have not been integrated into the work. The literature is treated uncritically. The work is not based on prior knowledge in the field. Discussion on the development of the work is missing. Complex phenomena, issues or situations are not formulated, handled or analyzed in the thesis. The work shows a lack of holistic view of the problem picture or is unjustifiably restricted to avoid the complexity of the task. Relevant judgements related to the thesis question is missing. The work is not up to the level that has been agreed upon, initially or later during the supervision process. A critical evaluation of one’s own work is missing. The agreed work plan in terms of time and methodology has not been held. Product, process, system, method or technical solution has not been evaluated or developed in the work. Appropriate analysis of manageability for and effect on people, society, environment and economy flaws or is missing.

The content is not systematically presented and the text or the oral presentation is difficult to understand. The argumentation for the conclusions is insufficient. Thereview have an unclear purpose, are too close to the original source, or stacked with no obvious connection. The written report is not linguistically well-formulated or coherent. The continuous communication or the oral presentation do not show responsiveness, clarity or ability to discuss the work and findings. Judgements are missing or inadequate. The work demonstrates the inability to insert the study in a larger context. The thesis does not address the ethical and societal aspects even though these may be relevant to the project topic. Alternatively a justification for aspects not addressed is lacking. Despite supervision and guidance the student does not show the ability or willingness to participate and cooperate in the current work culture. The student does not add constructive ideas in discussions with supervisors and shows disinterest for advice and new proposals. The student does not demonstrate independent creative work between tutoring sessions.