2018 Conference on Systems Engineering Research
Why Not Teaching Systems Architecture as a Studio Art Class? Alejandro Saladoa,*, Tom McDermottb, Kirsten Davisa, Alejandra Moralc Virginia Tech, Blacksburg VA 24060, USA Georgia Tech Research Institute, Atlanta, GA 30332, USA c Radford University, Radford VA 24142, USA a
b
Abstract System architecture is both an art and a science. Its scientific side deals with producing actual designs; its artistic one drives the value of the architecture. Effective systems engineers exhibit strength in both sides. Effective architects are those that create elegant solutions to complex problems. However, most of the development and training of systems engineers focuses on the analytical and procedural side. This paper addresses the question of how to teach the artistic side of systems architecting. We contend that systems architecture should be taught in a way similar to how the arts are taught, rather than the traditional instructional approaches employed when teaching engineering sciences. In support of this idea, we propose to teach system architecture mimicking a studio art class. We discuss the classroom setting, the structure of the lessons, and the structure of the course. © 2018 The Authors. Keywords: Education; architecture; art
1. Introduction System architecture is both an art and a science [1, 2]. Its scientific side deals with producing actual designs; its artistic one drives the value of the system architecture [3]. Strong ability in both sides are exhibited in effective systems engineers, who possess a good balance of hard- and soft-skills [1, 4]. However, most of the development and training of systems engineers focuses in the analytical and methodological side. Yet, there is a growing demand for promoting the artistic side and pursing elegance when architecting engineering systems [5-7]. We know how to teach the science, but how should we teach the art? We contend that systems architecture should be taught in a way similar to how the arts are taught, rather than the traditional instructional approaches employed when teaching engineering sciences. In support of this idea, we propose to teach system architecture mimicking a studio art class. * Corresponding author. Tel.: +1(540)231-0483; fax: +1(540)231-3322. E-mail address:
[email protected] ® 2018 The Authors.
Alejandro Salado, Tom McDermott, Kirsten Davis, and Alejandra Moral
2
2. The importance of teaching the art of system architecting Teaching systems engineering concepts to undergraduate students and to working professionals provides an interesting insight. Although working professionals nod and share aha moments by contextualizing the concepts to their previous experiences, undergraduate students usually react with a so what? attitude. For undergraduate students, systems engineering concepts seem obvious. In fact, some have even argued that systems engineering is just good engineering or even just common sense engineering [8]. We do not go that far necessarily, but suggest that systems engineering concepts are generally easy to grasp. Systems engineering becomes difficult (very difficult) when diving into its practice. This is a departure from traditional engineering sciences. In professional practice, applying electrical theory or doing structural calculations get simpler by applying codes, substituting model sophistication with safety margins, or using software (among others methods). In systems engineering however, the student is usually lost when trying to apply the concepts that appeared to be so obvious earlier. As complexity grows, they can often lose interest in the broader purpose of the system and its architecture, leading to poor decisions. Although in this paper we focus on systems architecture, this applies to various aspects of systems engineering. For example, verification and validation have been defined in a fantastic and simple way. Verification indicates if the product is built right and validation if the right product was built. These concepts are easy and deep at the same time. Yet most systems engineers are unable to distinguish between verification testing and validation testing. One only needs to look at V&V standards in industry [9, 10]. Engineers exhibit similar behavior when writing engineering requirements. The guidelines for writing good requirement seem obvious and easy to follow. A requirement must be quantified, must enforce solutions, etc. Yet most requirement sets out there are of very poor quality [11-13]. We have seen the same pattern when teaching systems architecture. For sure, heuristics are easy to grasp. For example, “simplify, combine, and eliminate,” “system structure should resemble functional structure,” “group elements that are strongly related; separate elements that are unrelated” [2]. Concepts are not sophisticated, but a heuristic without appropriate foundational knowledge and practice is just a set of words. We preach “simplify” but the “art” of simplification is seldom taught in engineering. We discuss abstraction, interfaces, function/component, and assessment, among others. Yet, students have trouble in the creational aspect. In particular, they struggle with applying those concepts and using those heuristics to architect effective solutions to problems. Therefore, we suggest that teaching systems architecture must focus more strongly on conveying its art. 3. Why mimic a studio art class Our proposal to mimic a studio art class to teach system architecting builds on three key ideas. First, artists and systems architects need to develop similar competencies. Second, system architectures are created for others. Third, the art of systems architecture must be developed through experience. 3.1. Similar competencies for architects and systems architects Previous work has explored the relationship between systems engineering and artistic endeavors. The principles and practices of systems engineering to cope with complexity and achieve harmony in the solution seem to be exhibited in the creation of original film scores [14]. The techniques that master painters use to convey beauty seem to match those that systems architects use to architect effective systems [15]. This has led to the first two authors of this paper to map competencies that students need to learn in the arts and in systems architecting [16]. They are listed in Table 1. When evaluating the necessary competencies in the arts, we reach a similar conclusion to our analysis of systems architecting. That is, several concepts are simple, yet their application to create something meaningful is very difficult. For example, for understanding how different colors interact, one only needs to use a color wheel. Similarly, there are several rules that help in composing the frame or scenario. Common ones include the golden ratio, the rule of thirds, or the leading lines. However, just a few people are able to use them in a way to create something beautiful, let alone something that transcends. As a result, practice is a key educational method in the arts. In fact, a person would hardly be presented as an artist because he/she knows composition rules and color patterns. Someone will be called an artist when showing a portfolio
Alejandro Salado, Tom McDermott, Kirsten Davis, and Alejandra Moral
3
of their work, their originality and their art. We believe that a similar stance should be taking for architects. Knowing systems architecture heuristics and concepts should not be sufficient. The quality of an architect must be measured on the architectures he/she creates. Therefore, this similarity informs our suggestion to teach systems architecture in a studio art setting as opposed to how it is traditionally taught. Table 1. Common Learning Concepts Spanning Art and Systems Architecture [16]. Learning concepts
Art
Systems Architecting
Abstraction
Understanding Symmetry between internal context and observed system
Understanding Symmetry between business context and observed system
Precedence
Interpreting the Masters’ Unprecedented Work
Studying Unprecedented Architectures
Time
Representing Movement
Attributes of Change
Heuristics and Patterns
Learning across the Arts
Learning across Management and Engineering
Decomposition, recomposition
Lists & groups, Split, Subtract details, Symmetries, Aesthetics
Elements and Interfaces, Attributes, Objectives (requirements) flow-down & derivation, Integration sequences and verification coverage, Purpose
Boundary setting
Observing, Framing, Scene-setting
Heuristics, Context, External interfaces, Problem formulation, ConOps
Simplifying
Subtract details, Split
Determining abstraction levels, Lumping or splitting components
Synthesizing
Remix & Reconnect
Emergence, Partitioning, Integration, Generic Architectures, Architectural Frameworks
Focusing
Emphasize, Power of the Center, Contrast & Balance
Views & Viewpoints, Centralization/Networked
Communicating
Aesthetics, Color/musical palette, Frame/viewpoint, Title/description, patterns
Domain knowledge, standards, views and viewpoints, documentation/metadata
Analytical competencies
Domain driven use of materials & media
Domain driven technical & business analysis
Methodological & executional competencies
Creativity, Use of processes and patterns, Experimentation & risk taking, Ability to engage & keep audience focus
Creativity, Use of processes and patterns, Use of frameworks, Planning, Managing, Ability to engage & facilitate agreement
Learning outcomes
Understanding of composition, inventive design, risk taking, creating expression, breadth of work, individualized transformation of concepts
Composition of operational & technical design, methods & execution, planning, breadth & depth, group communication of transformation
3.2. Architecting is for others Several of the competencies listed in Table 1 can be gained through traditional instruction. To the best of our knowledge, we (as a community) teach systems engineering through lectures, case studies, individual assignments, and group projects. For sure, they are all helpful. Case studies can help assimilate failures and successes of using certain heuristics and approaches to architecting. Lectures may be effective to convey key concepts and share the experiences of the instructors. However, these instructional approaches do not let the student apply that knowledge to create architectures. They must be complemented. A traditional way to do so is to assign individual and group projects to the students. With individual assignments, students are forced to apply the concepts they learnt to a number of problems. With group assignments, students collectively create an architecture. Both approaches suffer from two weaknesses, which we discuss here. The first one has to do with exposure. For sure, trying to architect systems is a necessary effort to become a systems architect. However, the student receives feedback from the instructor on his/her work. The student loses the opportunity to receive feedback from and provide feedback to his/her peers. Most importantly, we believe, students
Alejandro Salado, Tom McDermott, Kirsten Davis, and Alejandra Moral
4
lose the opportunity to compare and contrast how other engineers may face and solve the problem differently. We suggest that understanding strengths and weaknesses of different architectures for solving the same problem is key to grow as a systems architect. The second key weakness relates to the success criteria associated with the assignment. Lacking an authentic context, the student solves the problem for himself/herself and for the instructor. That is, the student’s intent is to create a solution that the architect believes successful, and in such a way that the instructor will be able to interpret it. However, a key mission of systems architecture is to stimulate communication, align mental models, and seek feedback that informs a new iteration on the architecture [17]. In fact, that feedback is essential to become a successful systems architect. Specifically, it allows for gaining experience in understanding multiple perspectives to a problem. We suggest that the student loses again an opportunity, with traditional instructional approaches, to build up an ability to think about the architecture he/she creates from diverse perspectives. Note that being exposed to multiple perspectives, as discussed in relation to the second weakness of traditional instruction, differs from the exposure to different solutions we discussed for the first one. Diversity of perspectives has to do with how different stakeholders may look at or use the architecture. For sure, group assignments offer the possibility of generating feedback. However, all team members work towards the same purpose and under the same role/viewpoint: they are all architects. This difference in perspective or viewpoint is essential to effective architecting. For example, one of the outcomes of creating a system architecture is the set of objectives or requirements for the lower level components. Defining objectives or writing requirements for self-understanding is easy. Criticality in flowing down and deriving objectives and requirements lays on the understanding and interpretation that other people will do with them. Without experiencing those interesting interactions (e.g., unconscious misinterpretation, deliberate misinterpretation, misalignment of assumptions), it is harder for the student to learn and deeply understand (and feel) the reach of the consequences of the architecture that has been developed for the system under development. We found in the competencies required for an artist that similar needs with respect to interpretation by external observers is key to become successful. In fact, the purpose of art is to convey experiences and emotions. It is understood as a bi-directional communication from the artist to the consumer of art. This similarity informs our suggestion to design the instructional approach for systems architecture around the studio art concept. System architecting might be considered a process of aligning the architects internal viewpoints, external stakeholder viewpoints, and the business and technical viewpoints of the architecture itself [15]. Without the ability to recognize and practice all three, the quality of the architecture will suffer. 3.3. To the art through experience Creativity and innovative thinking are increasingly recognized as skills that engineering students need to develop [18-20]. Engineering education researchers have identified specific sub-skills that contribute to creativity in engineering work, including observation, big picture thinking, problem finding, divergent thinking, and iterative experimentation [19, 21-26]. They have further explored methods for developing these skills in engineering students, with variable results. Competitions [24], collaborative activities [23-25], open-ended projects [19, 23, 27], and feedback from peers and mentors [26, 28] have all been suggested as methods for improving student creativity. A limited amount of research has indicated that teaching ideation strategies [27, 29], incorporating reflection [26, 28, 29], and interdisciplinary work [24] can help in developing these skills. On the other hand, some research has indicated that the traditional experience in engineering education might actually reduce the level of creativity in designs produced by students. Two studies found that designs by senior students were less innovative (although more practical) than those of first-year students, particularly on more open-ended tasks [27, 30]. Industry feedback has also suggested that engineering students are not graduating with sufficient creativity [19]. There are limited examples of using the approach we are describing in an engineering context. There is increasing acknowledgement that engineering students could benefit from a more artistic perspective [31, 32]. Some schools have responded to this call by introducing interdisciplinary courses where engineering and art students work together on a project (e.g., [31, 33]). Such classes are reported to help students think outside the bounds of their own discipline, but can often be a challenge to set up administratively and coordinate teaching across multiple departments in a way that places all disciplines on equal footing [31]. Another approach has been to require upper-level engineering students to work on a studio art project with art faculty members and reflect on how their art experience connects to their
Alejandro Salado, Tom McDermott, Kirsten Davis, and Alejandra Moral
5
engineering work [32]. In this case, the engineering students began to see clear connections between the creative process used in their art projects to their engineering work. Closer to our proposal, software engineering has begun to introduce “studio” courses where students work on their projects in class and have opportunities to give and receive feedback with peers [34, 35]. They report that such a course improves students reflective practice and ability to give and receive feedback. Similar approaches are now being adopted in other disciplines as well (e.g., [36, 37]). However, the courses reported so far have attempted to align only their lesson structure and (in a few cases) classroom setting to that used in traditional studio art courses. In our proposal, we are looking to take the idea of a studio further to consider not only the lesson structure, but also the classroom setting and course structure as a whole. In previous work, we presented a general framework for engineering competency development relating the analytical and methodological skills traditionally taught in the sciences and engineering to development of conceptual abilities (heuristics, precedence, abstraction) linked to systems thinking and compositional capabilities (simplification, synthesis, focus, communication) linked to design [16, 17]. Participation in the arts should enhance the compositional abilities of the engineer, as many of the fundamental composition rules are transferrable across different domains of the arts, architecture, engineering, and computer science. Incorporation of these composition rules in student project artifacts focused on communicating architectures (such as Operational Views) helps to improve their communication skills and even their systems thinking abilities [17]. However, more is needed to establish the foundations and competencies of architects as “composers” – we should teach system architecture as a studio art class. 4. A concept to teach system architecture as a studio art class 4.1. A studio art class in art We describe in this section the typical setting and a structure for a studio art class. We build the example on elements of two semester-long undergraduate courses in arts at Radford University, one in drawing and one in figure drawing. Both courses employed traditional instructional approaches in studio art settings. We discuss three aspects of the course: The classroom setting, the structure of the lessons, and the structure of the course. Classroom setting The class takes place in a large space. The objects or models to be drawn are placed in the middle of the room and the students sit surrounding them. This implies that, while all students look at the same point, they look at it from different perspectives (viewpoints). The room has walls that are set to hang the drawings made by the students. All drawings can be hanged on the same surface. Lesson structure The lesson is structured in three blocks: 1) Lecture: The instructor explains techniques, ideas, and guidelines as relates to the objective of the lesson. 2) Practice: Students spend time applying the techniques explained in class on a model the instructor defines. The instructor provides feedback to students individually as they practice their drawing. This phase allows for a deeper interaction between the instructor and the student in relation to the concepts and techniques explained in class. Furthermore, the frequent feedback allows for iterating the student’s work as it is created. 3) Exposition and critique: Towards the end of the lesson, the works of all students are exhibited together. Students then critique each piece of work, comparing them holistically and reflecting on what each person captured from the model, as well as how well they accomplished it. Exposition is actually a key element in the studio art setting. By making students share what they see in each other’s work, the student becomes aware of how other people interpret his/her own work. In other words, students are enabled to understand the level of alignment between what they wanted to convey through their work and what other people interpreted from it. Course structure The overall course is structured in such a way that techniques and skill build step by step. Specifically:
Alejandro Salado, Tom McDermott, Kirsten Davis, and Alejandra Moral
6
1) Draw basic forms: The student is able to capture basic forms such as cubes, pyramids, or spheres. 2) Draw objects: The student is able to capture objects that combine multiple basic forms, such as vessels, caps, or fruits. 3) Draw a person from a model: The student is able to draw the human body. While one could think that there is no difference between drawing objects and drawing a human, conceptually (both are divisible to basic forms), there is a fundamental difference, wired to our brains. Apparently, the human brain has a high level of accuracy and precision in differentiating human faces and eyes. Hence, while a regular person would not be able to detect a given variation between a fruit and a drawing of it, they would be able to distinguish between a face and its drawing. Thus, there is a level of added granularity in the way a face needs to be captured that is sufficient to warrant a dedicated learning step. 4) Draw a person from imagination: The student is able to create the drawing of a person from imagination, without looking at a model. This step is divided in two sub steps: a. Learn patterns: The student learns certain patterns to abstract reality, such as the mannequin made of geometric figures. b. Apply imagination to patterns: The student embellishes the basic figures in the patterns until the work captures the detailed aspects that make the pattern vanish and the imagination emerge. 5) Draw a story from imagination: The student is not only able to create a person from imagination, but to place the person within an imaginary context to create a story, a narrative. This is the ultimate purpose of art: it tells a story to convey emotions. 4.2. Our proposal: A studio art class in system architecting We describe in this section our planned approach to organize a course in systems architecture within a studio art class setting. Using the previous section as a paradigm, we present the classroom setting, the structure of the lessons, and the structure of the course. Classroom setting We propose to tailor the studio art class setting to capture the specific characteristics of systems architecting problems. The object of study is the problem for which students need to create an architecture. Therefore, lacking a physical form, placing the problem at the center of the room may not seem especially important. However, using a center in the class enables placing each student at the same distance from the problem, which, at least in the traditional studio art class, facilitates student engagement with the object. Therefore, we suggest that such a structure, the problem at the center and students surrounding it, is maintained when teaching system architecture. Furthermore, we also suggest enabling the simultaneous visualization of all solutions (architectures). This has two caveats. The first one is that architecture views are created in the computer and, therefore, the room needs the capability to display all architectures at a sufficiently large size. The second one is that architectures have multiple dimensions. Evaluating an architecture is a dynamic activity, not a static one like the artistic drawings we have used as a paradigm in the previous section. This creates a different dynamic of interaction between students and their solutions. Hence, an ideal classroom setting should count with multiple large displays that enable interaction. Lesson structure We plan to use the same lesson structure as in the arts class: 1) Lesson: The instructor explains basic concepts such as architecting heuristics, abstraction, or interfaces. 2) Practice: Students spend time applying the concepts to architect a solution to a given problem. The instructor provides feedback to students individually as they practice their architecting skills. This phase allows for a deeper interaction between the instructor and the student in relation to the concepts and techniques explained in class. For example, the instructor can tailor his/her experience much more to the specific effort of the student in learning the material. Furthermore, the frequent feedback allows for iterating the student’s architecture is developed, increasing the student’s exposure and understanding of perspectives/viewpoints. 3) Expositions: Towards the end of the lesson students share their work and compare it holistically. Students are then asked to find strengths and weaknesses in the architectures of their peers. This enables students to
Alejandro Salado, Tom McDermott, Kirsten Davis, and Alejandra Moral
7
gain exposure on how different solutions could have worked. It also helps them learn about different interpretations from diverse viewpoints about the solution idea that he/she tried to convey through the models employed to capture the architecture. As was the case for the arts class, we also find the exposition to be a key aspect of teaching architecting. Only by facilitating those exchanges, students can gain experience in aligning mental models and can become aware of the gaps their mental models can create when architecting systems. Course structure When defining the course structure, we aimed at capturing the staged development of skills present in the drawing course described in the previous section. In particular, we plan for four main developmental steps: 1) Learn forms: The student is able to capture basic system architecture elements such as functions, behaviors, or interfaces, in various architectural frameworks. 2) Learn of objects: The student is able to capture basic architectural blocks, such as a Command & Control functionality/mechanism, a Communication functionality/mechanism, or a Security Protection functionality/mechanism. 3) Capture the architecture of an existing solution: The student is able to capture the underlying architecture of an existing system. 4) Create an architecture: The student is able to create an effective architecture for a given problem. We have divided this step in three sub steps: a. Generic architectures: The student is able to recognize traditional solutions to common problems, such as current architectures for automobiles, computers, or airplanes. b. Architectural patterns and heuristics: The student is able to recognize architectural patterns and heuristics as a function of the problem characteristics and desired solution effectiveness. c. From blank to solution: The student is able to apply and tailor all techniques to create effective architectures to unprecedented problems. 5) Depict a system principle representing a person and an engineered system: The student is able to depict an architecture with intent. Some examples include “represent both centering and scale,” “represent both focus and diversity,” “visualize boundaries,” “visualize flow (movement),” “visualize emergence (time),” and “visualize non-linearity (feedback)”. 5. Future plans We have presented in this paper a concept to teach a systems engineering course in a studio art format. The idea builds upon previous work that has identified parallelisms between the practices and learning outcomes of systems architects and artists. We plan to pilot test the proposed class format during the Fall 2018 semester in an introductory graduate course in systems engineering at Virginia Tech. We additionally plan to study the impact of this change in pedagogy on the outcomes of the course. The instructor who will be teaching the course in Fall 2018 has taught the course in two previous semesters to around 60 students using a traditional format. We will use the final projects from the previous semesters as a baseline to which we can compare the final work of the students in the newly-designed course. To do a formal comparison, we propose to develop rubrics to connected to the developmental steps of system architecting described above and use these rubrics to score final projects from both the previous courses and the Fall 2018 course. This will allow us to identify developmental differences in students’ system architecting competence that may be connected to our proposed changes in pedagogy. References 1. 2. 3.
Ryschkewitsch, M., D. Shaible, and W.J. Larson, The art and science of systems engineering. Systems Research Forum, 2009. 03(02): p. 81100. Maier, M.W. and E. Rechtin, The art of system architecting. 2009: CRC Press. Muirhead, B.K. and D. Thomas, The Art and Science of Systems Engineering Tightly Coupled Programs. SAE Int. J. Passeng. Cars – Electron. Electr. Syst., 2010. 3(2): p. 117-130.
Alejandro Salado, Tom McDermott, Kirsten Davis, and Alejandra Moral 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37.
8
Jansma, P.A.T., Exploring the art and science of systems engineering, in IEEE Aerospace Conference. 2012: Big Sky, MT, USA. Griffin, M.D., How do we fix systems engineering?, in 61st International Astronautical Congress. 2010: Prague, Czech Republic. Salado, A. and R. Nilchiani, Using Maslow's Hierarchy of Needs to Define Elegance in System Architecture. Procedia Computer Science, 2013. 16: p. 927-936. Madni, A.M., Elegant system design: Creative fusion of simplicity and power. Systems Engineering, 2013. 15: p. 347-354. Akeel, U.U. and S.J. Bell, Discourses of systems engineering. Engineering Studies, 2013. 5(2): p. 160-173. ECSS, Space engineering - Verification. 2009, European Cooperation for Space Standardization: Noordwijk, The Netherlands. NASA, Systems engineering handbook. 2007. Salado, A. and R. Nilchiani, A Categorization Model of Requirements Based on Max-Neef's Model of Human Needs. Systems Engineering, 2014. 17(3): p. 348-360. Salado, A. and R. Nilchiani, Reducing Excess Requirements Through Orthogonal Categorizations During Problem Formulation: Results of a Factorial Experiment. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017. 47(3): p. 405-415. Hooks, I.F., Why Johnny still can't write requirements, in 22nd Annual SSTC Conference. 2010: Salt Lake City, UT, USA. Salado, A. and C. Salado, Systems Engineering Practices Exhibited in the Creation of a Film Original Score. INCOSE International Symposium, 2015. 25(1): p. 1147-1158. Salado, A., L. Iandoli, and G. Zollo, Painting Systems: From Art to Systems Architecting. INCOSE International Symposium, 2016. 26(1): p. 773-787. McDermott, T. and A. Salado, Improving the Systems Thinking Skills of the Systems Architect via Aesthetic Interpretation of Art. INCOSE International Symposium, 2017. 27(1): p. 1340-1354. McDermott, T. and A. Salado. Art and Architecture: Effectively Communicating Models of Systems. in 2018 Annual IEEE International Systems Conference (SysCon). 2018. Vancouver, Canada. National Academy of, E., The engineer of 2020: Visions of engineering in the new century. 2004, National Academies Press: Washington, DC. Cropley, D.H., Promoting creativity and innovation in engineering education. Psychology of Aesthetics, Creativity, and the Arts, 2015. 9(2): p. 161-171. Rover, D.T., New economny, new engineer. Journal of Engineering Education, 2005. 94(4): p. 427-428. Charyton, C. and J.A. Merrill, Assessing general creativity and creative engineering design in first year engineering students. Journal of Engineering Education, 2009. 98(2): p. 145-156. Duval-Couetil, N. and M. Dyrenfurth, Teaching students to be innovators: Examining competencies and approaches across disciplines. International Journal of Innovation Science, 2012. 4(3): p. 143-154. Raviv, D. Innovative thinking: Desired skills and related activities. in 2008 ASEE Annual Conference and Exposition. 2008. Raviv, D., M. Barak, and T. VanEpps. Teaching innovative thinking: Future directions. in 2009 ASEE Annual Conference and Exposition. 2009. Raviv, D. and D. Barbe. Ideation to innovation workshop. in 2010 ASEE Annual Conference and Exposition. 2010. Kershaw, T.C., K. Hölltä-Otto, and Y.S. Lee. The effect of prototyping and critical feedback on fixation in engineering design. in CogSci '11. 2011. Genco, N., K. Hölltä-Otto, and C.C. Seepersad, An experimental investigation of the innovation capabilities of undergraduate engineering students. Journal of Engineering Education, 2012. 101(1): p. 60-81. Green, G. and P. Kennedy, Redefining engineering education: The reflective practice of product design engineering. International Journal of Engineering Education, 2001. 17(1): p. 3-9. Pappas, E. Cognitive-processes instruction in an undergraduate engineering design course sequence. in 2009 ASEE Annual Conference and Exposition. 2009. Lai, J.Y., et al. Prompt versus problem: Helping students learn to frame problems and think creatively. in Third International Conference on Design Computing and Cognition. 2008. Sochacka, N.W., et al. Faculty reflections on a STEAM-inspired interdisciplinary studio course. in 2013 ASEE Annual Conference and Exposition. 2013. Beams, D.M., K. Gullings, and C.E. Ross. Seeking new perspectives: Engineers experiencing design through creative arts. in 2016 ASEE Annual Conference and Exposition. 2016. Shooter, S.B. and S. Orsborn. "Impact! Exploring innovation across disciplines" - Engaging the university innovation ecosystem through a university-wide course. in 2013 ASEE Annual Conference and Exposition. 2013. Bull, C.N. and J. Whittle. Observations of a software engineering studio: Reflecting with the studio framework. in IEEE Conference on Software Engineering Education and Training. 2014. Bull, C.N. and J. Whittle, Supporting reflective practice in software engineering education through a studio-based approach. IEEE Software, 2014. 31(4): p. 44-50. Thompson, B.E., Studio pedagogy for engineering design. International Journal of Engineering Education, 2002. 18(1): p. 39-49. Chance, S.M., J. Marshall, and G. Duffy, Using architecture design studio pedagogies to enhance engineering education. International Journal of Engineering Education, 2016. 32(1): p. 364-383.
