Flip the class, intrinsic motivation and context based

Flip the class, intrinsic motivation and context based learning techniques improve engagement, participation and understanding in a Project Oriented Learning Environment Sergio W. Sedas Gersey Tecnológico de Monterrey Monterrey, NL México [email protected]

Abstract Developing deep understanding and problem solving skills requires intrinsic motivation, engagement and active participation. However, in a project based learning environment, student appreciation and engagement is often clearly divided whereas some students may engage and enjoy the challenge, others clearly disconnect and express their discomfort in multiple ways. Extrinsic motivators have not proven effective to increase student motivation and engagement. A new strategy that combines flipped classroom, context based learning, and other activities that increase learning, intrinsic motivation, engagement and a positive sense of support and community are presented. The strategy was tested in two senior level engineering courses with positive results.

Keywords: flip the class, context based learning, intrinsic motivation 1

Introduction

In a Project based learning environment, student appreciation and engagement is often clearly divided. Whereas a group of students may engage and enjoy the challenge, others clearly disconnect and express their discomfort through absenteeism, low class and project participation, and harsh faculty and course evaluations. Extrinsic motivators such as taking roster at the beginning of class, additional office hours, allowing the students to submit their work three times to improve their grades, grading every activity and the opportunity of earning extra points did not cause substantial improvement. Research on positive psychology, intrinsic motivation and cognitive learning models [8-30] may provide insight that will help us develop new teaching strategies that will engage students in a project oriented learning environment. Using this insight we developed a set of strategies and building blocks that can be used enhance student engagement, improve learning and understanding, and increase positive results. These strategies were tested in two senior level engineering courses. One using traditional problem base learning and the other using project based learning.

This paper presents the background research, the building blocks and strategies and examples of their implementation.

2

2.1

Background

Resiliency and Motivation in Education

Motivation, self-confidence and resiliency limit or enhance a student´s ability to learn. If a student does not feel challenged, or if he feels that the challenge is too great his attention will move elsewhere and he will disconnect and avoid the challenge. This is a prevalent problem in project based learning environments where students are required to interact with companies and design a solution to real-life problems. There are a number of factors that influence our ability to face adversity and come up with creative solutions to our challenges:

2.1.1

Fear: Fight, Flight or Freeze

We learn more when we are challenged and engaged, when we are intrinsically motivated, when we believe, and when we have a directed dream or objective. [25] [30] [13] However there are a number of things that may distract and disengage a student:

If the student does not feel challenged or if he believes that he is incapable of succeeding, his limbic system in the brain will easily move his center of attention to something else. If the student believes that there is much at risk, he will avoid the challenge, unless he is driven by strong motivation. Many times, the thought of failing at a task is unconsciously perceived as risk which will trigger a student to avoid the task at hand. If the student fears the task or feels that the task is too overwhelming he will fight, flee (divert his attention into other activities) or simply freeze which are normal reactions of fear. In either case he or she will loose engagement and concentration.

2.1.2

Confidence, optimism and hope

Confidence, optimism, and hope can counteract fear and motivate a student to engage and move forward to complete challenging tasks. Self-Efficacy and Confidence are the solid belief in one´s ability to control a situation and the firm belief that when something goes wrong one is able to do what needs to get done to bounce back. This belief determines one´s actions, behaviors and persistence in front of obstacles and challenge [21][22][23]. For example, when a student perceives that he has the competence, skills and knowhow to handle situations effectively he focuses on meeting his goals, generates new alternative paths and accepts situations that are difficult to change. [18]. Hope and optimism can also influence a student´s engagement and immersion into a problem. If a student feels optimistic and hopeful, he will perceive a greater level of self-confidence, joy and desire to achieve his goal. He will be more willing to face adversity and do whatever he can to find solutions to problems. [19][10][11][21]. One way to build up confidence is by facing adversity. This can come in the form of challenges and the solution of real life problems. To minimize fear, you can divide the problem into small steps or challenges, each with a clear defined goal, target and deliverable. Dividing a problem into small steps and helping a student to accomplish them and acknowledge their success builds his self-confidence, joy and sense of satisfaction.

2.1.3

Connection

Studies show that a student that has a sense of belonging and feels a strong connection with a group is more willing to face adversity, knowing that the team is there to guide him, help him, and support him in the process [23]. Many classroom exercises laid throughout the semester, assigned to randomly chosen teams can help students create a bond with more people and increase a sense of connection with the group. It also provides students with a broader spectrum of ideas and problem solving strategies. Mastermind groups are another way to increase connection and student engagement. Masterminds are formed by a group of five to six peers. Each student is allowed to share past successes, and to solicit advice on specific problems. A specific structure for mastermind groups is presented in Canfield [27]. 2.1.4

Sense of Contribution

Knowing that what one is doing is relevant and contributes to the wellness of others is very motivating. In a PBL and POL learning environment, students can be asked to select class projects that solve relevant real-life problems in industry, commerce and community. They can be led to immerse themselves into the problem and have face-to-face contact with end-users and decision makers to identify true needs.

2.2 Flip the Classroom The idea of flipped classroom started as an idea to give individualized instruction through videos or other tools that students can use outside the class to free up time in class for more strategic group work and individualized attention [31]. It has now evolved into the idea of flipped learning in which “direct instruction moves out of the group learning space to an individual learning space and the group space is transformed into a dynamic interactive learning environment where the educator guides students as they apply concepts and engage creatively in the subject matter.” [32]: 2.3

Context Based Learning: Learning through understanding

Traditionally universities have been good at generating and teaching knowledge (facts and methods) and skills. However the methods have failed to develop the deeper level of understanding necessary for a student to fully transfer and apply his

knowledge and skills in a real world environment. Companies must still invest time and money to develop in college graduates the high level of understanding and experience that they require.

that build up awareness, knowledge and understanding. 4.

Flip the classroom. Make better use of the student´s time in the classroom through group exercises and activities that apply and reinforce the methods. Instructional videos and messages are recorded and viewed at home.

5.

Repetition and unpacking. During class, students use the methods we are teaching to solve a common class project. This gives them firsthand experience to learn how the methods are used to solve a problem. They reinforce this knowledge and gain further understanding when they apply the same methods to their own project and discuss their experience with their peers (unpacking).

6.

Random teams in class. At the beginning of the semester students form a team that will work together throughout the semester to develop a project. However, in each class session, students are randomly divided into teams to work on that day´s activity. This allows students to develop a sense of belonging with their classmates and to learn from different perspectives.

7.

Hot Seat presentations. Periodically, students are randomly divided into groups. Every student in the group is placed in the “hot seat” to present and describe his project to his peers. This increases engagement and participation from all of the students.

8.

Master Mind Groups. Three or four times during the semester, students will hold a Master Mind Session. Students are divided into teams of 6 people none of whom are in their Final Project team. In a round of 10 minutes each, each person gets the opportunity to describe their Project, share problems they are facing and receive feedback and ideas from the mastermind group.

9.

Repeatedly describe the framework. At the beginning of the semester students are uncertain about the trajectory that they will follow. It is important to reduce anxiety and built clarity by continuously repeating the framework and course map. Every time a stage of the framework is completed, we summarize and present the framework again. This reinforces the framework and helps them identify where they are in the process.

Research in cognitive psychology can offer insights from which one can develop alternative teaching strategies and techniques that will enhance learning through understanding [1]. We learn best when we make sense of new information, when we can relate it to past experience, and when it has meaning. [1][13][28][29][30]. We create significant learning and experience through deep rehearsal [25]. And we construct new knowledge and understandings based on what we already know and believe [29]. Skills, knowledge and experience are built over time and require intentional adversity and engagement [25]. In Sedas [1] we proposed a constructivist incremental approach in which the main focus is to first develop understanding before laying out the constructs of definitions and abstrations (knowledge). Once a student creates understanding he can easily attatch knoweledge, representations, formulas and ideas. This iterative constructivist approach, which we call context based learning, builds up knowledge and understanding through a laid out plan of awareness exercises, challenges, implementations, practical exercises, discussions and theory building lectures.

3

The Teaching Strategies

We developed a lesson plan that incorporates different strategies to teach problem based and project based courses: 1.

Solve relevant real-life problems. Real life problems are used to increase motivation and build experience that will help a student solve real-life problems in the future. Students are encouraged to immerse themselves into the problem, and interview people that are users, suppliers, experts, and stakeholders.

2.

Chunk it down. Break the project down into tiny steps and goals that the student can identify with. These reduce anxiety of a big leap. Furthermore, small successes increase a student´s selfconfidence and sense of accomplishment. Students are encouraged to celebrate completion and successes along the way.

3.

Context Based Learning. Design the course layout with hands on activities and challenges

4

The courses

The techniques and methods described in Section 3 were incorporated in two senior mechatronic engineering courses: Manufacturing System Integration and Mechatronic Design. Manufacturing System Integration teaches a student to understand and design a manufacturing cell using robots, conveyors, cnc machines and other process equipment. Figure 1. Video Tutorial - Using the “Sedas Architecture Model” to analyse and design manufacturing cells. Video LINK 1 http://youtu.be/so34ooo-k6g

Mechatronic Design teaches a student to define a mission statement, immerse themselves into a community; identify a problem, needs, requirements and restrictions; and ultimately design innovative solutions.

4.1

Manufacturing System Integration

The course is divided into modules, each of which teaches different technologies – CNC Machines, Robotics, Vision, Manufacturing Cells. Students have classwork, lab-work, and a final project. There are two sections of particular interest in this paper. The first section of interest teaches students to design and analyse a manufacturing cell using the “Sedas Archuitecture Method to Analyse and Design Manufacturing Cells”. The second section teaches students how to program an industrial computer vision system to identify automotive parts. In each of these sections, the class was flipped. Screen capture programs [7] animation software [4] and video editing software [2] were used to create videos and tutorials that were used as homework assignments to explain the different methods. Students were assigned problems in class which they worked with randomly generated teams. Instruction was given a priory using the pre-recorded videos (Figure 1 and Figure 2). They were then given instruction in class and a problem to analyse and solve in class. Students were then given two take-home assignments. In the first assignment, they had to analyse and design a different manufacturing cell selected by them. In the second assignment, they had to program a vision system to recognize three different automotive part. They tested their program against 73 images that were taken by a vision system running in the production line. The results from the take home assignments confirmed a high level of understanding.

Figure 2. Video: How to program a vision system to identify between three different automotive engine heads using Framework2. LINK http://tinyurl.com/m7wdvgy

4.2

Mechatronic Design

Mechatronic design is an advanced senior class taught in a project oriented learning environment. Students are required to identify a relevant need and develop an innovative mechatronic product that improves wellbeing and quality of life. Students are taught a framework (Table 1) and specific innovative design methods and sent off to identify a need, develop a product specification, identify and size up the competition, generate between 50 and 200 concepts, develop one concept fully, simulate it, build a prototype and test.

1

The video was produced using a Cannon 70D photo and HD movie camera, a Senheisser EW112 wireless mic[3], PowerDirector video editing software[2] and VideoScribe hand animation software [4]. Less expensive video and audio equipment can also be used [5] and [6]. 2

The video was produced using Camstasia Studio [7].

Table 1. The Design Life-Cycle Framework: 1.

2. 3. 4. 5. 6. 7. 8. 9.

Identify a relevant problem; identify stakeholders and primary and secondary users; interview and extract needs; generate product specifications Research and Size up the competition against the specifications Use innovation techniques to generate between 50 and 200 concepts. Select one. Develop the concept; identify and solve make or break issues, determine the mathematical models, and develop a conceptual design. Design Mechanical, Electrical, Pneumatic and Hydraulic systems. Specify and select commercial components. Run simulations Build and test a prototype Finalize design Prepare report and make presentation.

For many of the students, this is the first time they go through a complete product design life-cycle. They are unaware of many of the methods and lack practical experience. Therefore for many of the students, the entire process is uncomfortable.

4.2.2

The present

We incorporated a number of techniques described in section 3. Students were directed to identify real-life problems. The problems had to be relevant and solve a relevant issue. Furthermore, students were encouraged to find problems that would improve wellbeing and quality of life. They prepared a list of questions and proceeded to interview stakeholders and primary and secondary users to identify their needs. But before going out on their interviews, they were given the opportunity of receiving feedback. Students were divided into randomly assigned groups. Everyone in the group had the opportunity to share their list of questions, receive feedback and practice an interview (random teams in class, mastermind groups, repetition). Once they identified the needs, they turned the needs into design specifications as indicated by Ulrich [33]. A video instruction on how to do this was given to them. They had to watch it before class. The exercise was done in the classroom, which allowed the instructor to clarify and resolve any questions.

4.2.1 The past The course has been taught twice a year for more than three years. Until recently, a traditional Project Oriented Learning approach was used. Students were taught the framework and then sent off to work on the project. They chose a team and worked with this same team all semester long. The classroom time was used to give them instruction and describe the assignments. Students were expected to meet outside of class to work on their project. Whereas some outstanding students fully engaged in the project with great results, many if not most of the students felt uncomfortable and lacked direction. Some of the students went as far as to disengage completely, frequently skipping class, and relying entirely on their team mate’s efforts to develop the project and share their grades. Although some projects were very good, some fell on the lower scale of mediocrity, a result of simply going through the motions without creative initiative. End of semester course and faculty evaluations that measured student satisfaction by OGP were at an alltime low (2.4).

Different techniques were used to generate between 50 and 200 designs [33][34][35][36]. They used a method called functional decomposition [33] and many of the methods mentioned in Cracking the Cracking Creativity [37] and Thinkertoys [35]. The methods were explained in class. Students were once again divided into randomly generated groups and asked to use these methods to design and improve a known object such as a bicycle, a screwdriver or another device. The groups worked in class to generate multiple designs. Once each team had a number of designs, they selected two of them. They were once again separated into randomly selected teams where they were asked to present and justify their designs. The rest of the members of the new team gave feedback. Before the end of the class, students were asked to unpack the experience explaining what they had learned. With this newly acquired level of understanding they later met with their original teams to work on their own projects. A number of times during the semester, we divided the teams into randomly assigned mastermind groups. Each member was allowed 10 minutes to present the

current state of their designs, describe a problem they were having and solicit feedback.

specific tasks and activities and manage the workload throughout the semester.

All of the teaching strategies outlined in section 3 were used one time or another throughout the semester o Solve relevant real-life problem. o Context based learning – incrementally build knowledge, awareness and understanding through carefully laid out exercises, challenges, instruction and discussion. o Chunk it down – by dividing the problem into small steps which they solved in each class. o Flip the classroom by receiving instruction on a pre-recorded video which they had to watch before coming to class. o Repetition - the method was used at least once in class and once at home on their project. o Random teams in class- teams were randomly assigned to do the class work. o Hot seat presentations where everyone to members of a newly assigned team. o Master Mind Groups – by having others criticize and offer relevant suggestions on their approach and designs.

Each team was required to submit the activities for each milestone. These were graded and commented on.

4.2.3

Note: this do-over courtesy was not extended to inclass activities.

5

Individual

Quizzes & Homework In Class Activities Progress Activities Final Project Simulation Prototype

Some of the activities such as quizzes, homework and in-class-activities awarded a student individual credit. Team activities awarded all members in a team the same grade for the activity. In-Class Activities were done in class and could only be submitted in class. Therefore, only students that participated in the activity were awarded credit. This substantially reduced absenteeism and increased student participation. Roll calling was eliminated to eliminate the influence of extrinsic motivators that could bias the experiment and therefore was not a factor in student participation. However, despite this freedom absenteeism was at a long time low. The project was divided into small steps and milestones, which were worked through in and out of class. The sum of each milestone becomes a part of the final report. This allowed students to focus on

Results

For two semesters we have tested this model in two courses – an innovation design course and an industrial manufacturing automation design course. Roughly 30 students were enrolled in each course. We observed a number of improvements: 1.

Motivation, participation and student engagement went up. It even became common for students to continue to work long after the end of class.

2.

Absenteeism went down even though roll-call was eliminated.

3.

The quality and relevance of the projects increased. Students designed innovative devices to: Detect possible chronic heart failures to give early warning, Extract drinking water from humidity in the air used to supply remotely isolated communities with no access to running water. Monitor and improve home energy use Assist the visually impaired to shop And others

Grading

Grading was divided into five main sections.

Team

Students that wanted to improve their grade had the option of reworking and resubmitting any activity. This reduced the stress and allowed us to provide proper feedback which the students would work on. The end result is learning.

o o o o

4.

Students felt integrated with the entire group.

5.

Student appreciation evaluations (OGP) improved substantially from a low 2.4 to a high 1.3.

6

Challenges and Learning Experiences

There are a number of challenges and learning experiences. These will be explained in two sections – one regarding project oriented and project based learning and the other regarding videos.

6.1

Regarding project oriented and project based learning

Map out your framework. Include a detail breakdown of every step. Chunk the project down into small activities the sum of which results in a completed project. Schedule the delivery of each activity. Not having a schedule increases student´s level of stress and tension. Furthermore, you may run the risk of students not planning their progress and running out of time before they complete their project. Design a form for each step in your framework. This form will be used by students in class to complete the assignments. It will also be used as homework by students as they run through their project. Form random teams and assign class activities to them. This increases student participation and brings the entire group together. Generate the space for sharing, communicating and expressing. Include activities that allow students to present their work to their peers. It will help students focus and clarify their ideas and gain a sense of significance as they share their project and experience. Furthermore, students that normally do not participate will jump in. Create masterminds. It puts students in the hot seat which makes them prepare and study before their presentation. Furthermore, masterminds are a good way to clarify your thoughts and receive fresh positive feedback and ideas. Allow students to hand in their project assignments multiple times. This allows a student the opportunity to correct their mistakes and improve their grades. This reduces stress and opens up the space for them to improve their work and rise to your highest expectations. Map out your course. You are constructing a set of activities and experiences that together will construct knowledge and understanding. Allow for flexibility. You may need to emphasize and even change dynamics depending on the course progress. Focus on what you want to accomplish at each stage and allow yourself a little flexiblity on the how. Focus on why not how. Your goal is for the students to develop understanding and skills. The project and activities are the means to accomplish this. You need to interact with students and constantly evaluate progress on understanding and skills. In response, you may be required to include, add and even modify some of the interactive activities and discussions.

6.2

Regarding Videos

Make sure that you have excellent audio. People forgive the image quality. They seldom forgive bad audio. Animation is a great tool to explain things that you would normally write on the board. It is also great to point things out on a still photo or video. Consider that a video may take a long time to produce. The “Sedas Architecture” video lasts less than 14 minutes yet it took 18 hours to film, animate, piece together and edit. There are benefits in making a good video. You only need to make it once, and you can use it over and over again. Shoot for many short 5-10 minute videos instead of long 1 hour long webinar type videos. It is more likely that a student will view them. There is no guarantee that students will take the time to complete your reading assignments. My experience is that in general they will most likely not. There is also no guarantee that students will take the time to view your videos before class. So be prepared.

7

Conclusions

In this paper we have presented nine techniques and strategies that can be incorporated into a project based learning environment to increase intrinsic motivation, reduce elements of fear and resistance and help students to develop a stronger sense of support and community in their class. We flipped the class supporting individual instruction with videos and prerecorded tutorials, and allotted quality time in the classroom for group activities. We implemented these techniques in two courses: an advanced manufacturing course and an innovation design course. Preliminary observations include greater student engagement, participation and commitment, a strong sense of belonging and group support, and a uniform distribution of high quality projects.

8

Acknowledgements

I would like to thank Carlos Mijares, Jaime Bonilla, and Arturo Torres for their support in my research regarding intrinsic motivation, self confidence and a sense of purpose in education; Jack Canfield for his teachings and expertise which have influenced much of my research; Sergio Ortiz, Francisco Palomera, Luis Rosas, Alejandro Manriquez, Oswaldo Michelaud and Federico Viramontes for sharing their best practices; and Ken Bauer for keeping our international Flipped Classroom community engaged.

9

References

[1] Sedas-Gersey, S. W. (1992). Context Based Learning: learning through understanding. Innovating Technologies in Education, 7o Congreso de Innovación Educativa. Monterrey: Tecnológico de Monterrey. [2] CyberLink. (19 de Nov de 2014). PowerDirector. Obtenido de PowerDirector: http://tinyurl.com/sspowerdirector [3] Amazon. (19 de nov de 2014). Sennheiser EW112P G3 A-Band Wireless Lav Mic Bundle with SKB Hard Case and Extra Batteries. Obtenido de Amazon: http://tinyurl.com/SenheisserMic [4] Sparkol. (19 de Nov de 2014). VideoScribe. (Sparkol, Productor) Obtenido de VideoScribe: http://tinyurl.com/S-VideoScribe [5] Amazon. (2014 de Nov de 2014). Logitech 920 WebCam. Obtenido de Amazon: http://tinyurl.com/okn3e7v [6] Amazon. (09 de Nov de 2014). Rhode Lavalier Mic. Obtenido de Amazon: http://tinyurl.com/s-rhode [7] TechSmith. (19 de Nov de 2014). Camstasia Studio. Obtenido de Camstasia Studio Screen Capture and Video Editing Software: http://www.techsmith.com/camtasia.html [8] Csikszentmihalyi, M., “Flow: The psychology of Optimal Experience”, Harper Perennial Modern Classics; 1st edition 2008, ISBN: 978-0061339202 [9] Lopez, S. & Snyder, C.R., “The Oxford Handbook of Positive Psychology”, Oxford University Press, 2nd edition 2009 [10] Seligman, M.E., “The Optimistic Child: A proven program to safeguard children against depression and build lifelong resilience”, Mariner Books, 2007, ISBN: 978-0618918096 [11] Seligman, M.E., “Learned Optimism: How to change your mind and your life”, Vintage, 2006, ISBN: 978-1400078394 [13] Sousa, D. A., “How the brain learns”, Corwin Press, third edition [14] Csikszentmihalyi, M. & Nakamura, J. “Flow Theory and Research” Lopez, S. & Snyder, C.R. (Eds.), The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009, pp 195-206 [15] Watson, D. and Naragon, K., “Positive Affectivity: The disposition to experience posiive emotional states”, Lopez, S. & Snyder, C.R. (Eds.), The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009, pp 207-215 [16] Langer, E. “Mindfulness versus positive evaluation”, Lopez, S. & Snyder, C.R. (Eds.), The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009, pp 195-206 [17] Thompson, S. C., "The role of personal control in adaptive functioning"; Lopez, S. & Snyder, C.R. (Eds.), The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009 [18] Carver, C., Scheier, Mi., Miller, C. and Furlford, D.; "Optimism"; Lopez, S. & Snyder, C.R. (Eds.),

The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009 [19] Peterson, C., Steen, A.; "Optimistic Explanatory Style", Lopez, S. & Snyder, C.R. (Eds.), The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009 [20] Kevin L, Rand and Jennifer S. Cheavens, "Hope Theory", Lopez, S. & Snyder, C.R. (Eds.), The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009 [21] Maddux, James E., "Self-Efficacy: The Power of Believeing you Can", Lopez, S. & Snyder, C.R. (Eds.), The Oxford Handbook of Positive Psychology, Oxford University Press, 2nd edition 2009 [22] Bandura, Self-Efficacy. The exercise of control. New York; Freemena (1997) [23] Ginsburg, K., Building Resilience in children and teens, American Society of Pediatrics, 2nd edition, 2011. [24] Sedas, S. Intentional Possibility, to appear 2013. [25] Coyle, D. (2009). The Talent Code: Greatness Isn't Born. It's Grown. Here's How (1st Edition ed.). Bantam. [27] Canfield, J.(2006), Success Principles: How to get from where you are to where you want to be (2006), William Morrow Paperbacks, Reprint edition. [28] National Research Council. (2000). How People Learn: Brain, Mind, Experience, and School: Expanded Edition (2 edition ed.). (J. D. Bransford, A. L. Brown, & r. R. Cocking, Eds.) National Academies Press. [29] Cobb, P. (1994). Theories of mathematical learning and constructivism: A personal view. Symposium on Trends and Perspectives in Mathematics Education. Austria: Institute for Mathematics, University of Klagenfurt. [30] Maquire, E. A., Frith, C., & Morris, R. (1999). The functional neuroanatomy of comprehension and memory: The importance of prior knowledge. Brain , 122, 1829-1850. [31] BergMann, J., & Sams, A. (2014). Flipped Learning: Gateway to Student Engagement. (L. Gansel, Ed.) USA: International Society for Technology in Education. [32] Flipped Learning Network. www.flippedlearning.org [33] Ulrich, K.T., & Eppinger, S.D (2004). Product Design and Development, McGraw-Hill, 3rd Edition [34] Kumar, V. (2012). 101 Design Methods: A structured approach for driving innovation in your organization, Wiley, 1 edition [35] Michalko, M (2006). Thinkertoys. A handbook of creative-thinking techniques, Ten Speed Press, 2 edition [36] Michalko, M (2006), Thinkpak: A Brainstorming Card Deck, Ten Speed Press, Revised edition [37] Michalko, M (2011), Cracking Creativity: The secrets of creative genius, Ten Speed Press, New Edition