Implementation of a Hybrid Multimedia General ...

Session S1D

Implementation of a Hybrid Multimedia General Education Course in Engineering Pat Backer San Jose State University, [email protected] Abstract – Technology and Civilization is an advanced general education course (Area V: Culture, Civilization & Global Understanding) in the College of Engineering at San José State University that is designed to introduce students to the realm of history and usage of technology in society from an international perspective and to increase their awareness of both the uncertainties as well as the promises of the utilization of technology as a creative human enterprise. We offer two different versions of this course: a fully online course and a hybrid in-class lecture/self-paced course. Both versions of the course use multimedia learning materials on CD. The goal of these multimedia modules is to have the students use technology as they explore its impact on our society over time. This paper will describe the structure and content of this course and discuss how the course is taught in both the online version and the hybrid version. As hundreds of students have taken this course over the past five years, we have significant experience in providing online learning to students. In addition, we will discuss the assessment plan for this course and how we conduct continuous assessment to improve the content and delivery of this course.

Many universities offer courses for general education under the general theme of science, technology, and society. Frostburg State University [1] offers an interdisciplinary course titled Science Technology and Society as a freshmenlevel general education course for non-science and nonengineering majors. At the University of Denver [2], an interdisciplinary team including faculty from the Department of Engineering has offered a three-quarter long course called Technology 21 for fourteen years to approximately 100 students each year. This course is used by the nonengineering and non-science students to meet their university’s science general education requirement. A general education course titled Technology and the Engineering Method at the University of Dayton [3] also fulfills a science education requirement and is taken by a diverse group of non-engineering students. Other universities that have engineering courses as part of the General Education programs at their institutions include Miami University [4], Penn State University [5], Old Dominion University [6], North Carolina State University [7], and the University of Texas at Austin [8].

Index Terms – General Education, Hybrid Course, Multimedia, Technology

Tech 198 is designed to introduce students to the realm of history and usage of technology in society from an international perspective and to increase their awareness of both the uncertainties as well as the promises of the utilization of technology as a creative human enterprise. Tech 198 provides a comprehensive overview of the human dimension of technological change as it continually molds and shapes the nature of our culture, social institutions, and the global environment. While science and technology are often decried as pervasive agents of social change, this course focuses on the role individuals can play in the management and control of technological forces toward human achievement. Emphasis is given to the chronology of technology and its role in shaping human history. The perspective is to regard technology both as affecting and being affected by culture. This focus should help the student broaden his/her view of technology and its role and effect on society. This course builds upon basic skills in reading, writing, speaking, critical thinking, and research; and knowledge from Core GE to give the student a comprehensive view of the interaction of technology and society.

INTRODUCTION The general education program at San José State University (SJSU) is different from many in the United States. Instead of predetermining a specific series of courses as part of the General Education (GE) for each student, SJSU has five Core GE areas (Skills, Science, Humanities & Arts, Social Sciences, and Human Understanding and Development). In addition, every SJSU student must take SJSU Studies (formerly called Advanced GE) courses in four areas: Earth & Environment; Self, Society & Equality in the U.S.; Culture, Civilization & Global Understanding; and Written Communication. Any department may propose a course for any area of GE. Tech 198—Technology and Civilization, was approved as an Advanced GE course in the Earth & Environment area until Spring 2000. In Fall 2000, after a revision to the university General Education program, the course was approved in another Advanced GE Area (Area V--Culture, Civilization & Global Understanding) where it remains an approved course today.

DESCRIPTION OF THE COURSE

978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference S1D-1

Tech 198 is a required course for all BS Industrial Technology, BS Aviation, and BS Computer Engineering majors (effective Spring 2007); in addition, it is part of the SJSU Studies (Advanced General Education) curriculum and attracts students from all over campus. This course is delivered in a novel way. It has a hybrid structure and is composed of three units that are delivered through selfpaced multimedia CD (Units 1, 3, and 4), one unit that is delivered through WWW instruction (Unit 2), and three units that are delivered either through a traditional classroom model or using the BlackBoard web delivery system. This course is evaluated each semester under SJSU’s GE program. The delivery medium and content for each unit is displayed in Table I. TABLE I CONTENT, TITLE, AND MEDIA FORMAT FOR EACH UNIT IN TECH 198 Unit

Title of Unit

Media Format

1

Nature of Science and Technology

Multimedia CDs

2

History of Technology

Web-based

3

Technology and Work

Multimedia CDs

4

Technology and Gender Issues

Multimedia CDs

5

Technology Transfer and Cultural Issues

Lecture or BlackBoard Online Module

6

Quality of Life


7

Ethics


There are two delivery methods for Tech 198: onlineonly and hybrid (self-paced multimedia/Web/in-class). Since both methods require the use of multimedia and web-based course modules, the primary difference is whether the students will meet in-class or complete online BlackBoard modules for Units 5-7. There are three tenured faculty and two lecturers who regularly teach this class and the department offers several sections of this class each semester. The department has a policy that no faculty may teach this class without first observing another instructor for an entire session. After this, two certified Tech 198 instructors observe the “new” faculty in his/her Tech 198 course. If the “new” faculty passes the probationary period, he/she is considered certified to teach Tech 198. MULTIMEDIA COMPONENT There are three units that are taught using multimedia CDs. The software used for developing the content was Macromedia Authorware and there are multimedia CDs available for students on both the Windows and MAC platforms. Two of the multimedia units (Unit 1: The Nature of Science and Technology and Unit 3: Technology and Work) have been used by all instructors since Fall 2000. In Summer 2003, the multimedia author received a SJSU College of Engineering Teaching Excellence Fund Award to develop the multimedia module for Unit 4 (Technology and

Session S1D Gender). This unit was designed to follow the “look and feel” and structure of the other multimedia units. The multimedia CDs are evaluated each year by the faculty teaching this class and are revised every two years. The multimedia for each Unit is structured so that the information is presented in manageable chunks. Students can navigate between sections by using the Next Topic and Previous Topic buttons (see Figure 1). As Laurillard pointed out in her research, “learners working on interactive media with no clear narrative structure display learning behavior that is generally unfocused and inconclusive” [9]. The chunking of information is interlinked with the narrative structure of multimedia. Because the teacher-storyteller is remote from the studentlistener, the design of the multimedia and the chunking of its content needs to be robust. The content in each section was designed so that the students could reread a section without restarting from the beginning. This reorganization provided a better narrative structure and, at the same time, increased learner control. As Steinberg found, increasing learner control can make the learning experience more motivating as well as increase student learning [10].

FIGURE 1 SCREENSHOT FROM UNIT 1 SECTION 4—THE SCIENTIFIC METHOD

Unit 1 (The Nature of Science and Technology) is divided into six sections: What is Science?; What is Technology?; What is Scientific and Technological Literacy?; The Scientific Method; Attitudes Toward Technology; Technology Dependence and Technology Traps; and Impact of Technology on Society. Unit 3 (Technology and Work) is divided into eight parts: The Industrial Revolution; Industrialization of Society in the 19th Century; Workplace of 1900; Scientific Management; The Development of the Assembly Line; Consumerism in the West; Nature of Work Today; and How Does Technology Affect the Workplace? The six sections for Unit 4 (Technology and Gender Issues) are Technology and Gender; Women at Work


before 1900; Women at Work, 1900 to today; Women Working around the World; Attitudes of, and about, Women in Technology; and Gender-based Technologies. WEB-BASED HISTORY OF TECHNOLOGY Unit 2 (History of Technology) was designed to particularly address student learning objective (SLO) number two for SJSU Studies Area V courses (see Figure 2). This new web-based unit focused on how technology developed over time in three different cultures and had three distinct sections: Technology in the Middle Ages; Islam Spain and the History of Technology; and Chinese Contributions to Technology. A. Goals Courses in Culture, Civilization, & Global Understanding should give students an appreciation for human expression in cultures outside the U.S. and an understanding of how that expression has developed over time. These courses should also increase students’ understanding of how traditions of cultures outside the U.S. have influenced American culture and society, as well as how cultures in general both develop distinctive features and interact with other cultures. B. Student Learning Objectives (SLO) SLO 1. Students shall be able to compare systematically the ideas, values, images, cultural artifacts, economic structures, technological developments, and/or attitudes of people from more than one culture outside the U.S.; SLO 2. Students shall be able to identify the historical context of ideas and cultural traditions outside the U.S. and how they have influenced American culture; and SLO 3. Students shall be able to explain how a culture outside the U.S. has changed in response to internal and external pressures. FIGURE 2 GOALS AND OBJECTIVES FOR AREA V (CULTURE, CIVILIZATION, & GLOBAL UNDERSTANDING) AT SJSU

Session S1D political, religious, and intellectual environment of China, the Islamic world, or the Middle Ages in Europe. The history section linked to existing history materials on the WWW including materials at the University of Calgary [12], CUNY-Brooklyn [13], and the Internet History Sourcebooks Project [14]. INTERNET-ONLY COURSE MODULES Each semester, we offer one to three sections of Tech 198 as an online-only course. The students in these classes use the multimedia CDs and the web-based history of technology unit; they also complete the last three units (Technology Transfer and Cultural Issues; Quality of Life; and Ethics) online, instead of going to an in-class lecture. The online-only course was developed using Blackboard and includes a narrative section, PowerPoint slides, video clips, self-assessments, and links to external sources. We based the development of these units on the class activities that were used in the hybrid class. Figure 3 displays the content for the first section of the online BlackBoard unit for Technology Transfer. 1. Read The Relation between Technology and Culture by Patricia Backer. 2. Watch the four mini-videos about technology development projects sponsored by IFAD. As you watch the videos, ask yourself the following questions: What is the development activity being presented? Does this development activity directly affect women and men in the community? How successful is this activity for the society? • IFAD video, Look Who's Talking, 2006 • IFAD Video, I Spy, 2006 • IFAD Video, Liquid Gold, 2005 • IFAD Video, Troubled Waters, 2007. 3. Homework, Part 1. Choose one of the technology transfers presented in the videos and answer the following questions. What is the development activity being presented? Does this development activity directly affect women and men in the community? How successful is this activity for the society?

When incorporating this unit into the course curriculum, it was intended to be a multimedia CD. However, since the module was designed to focus on the FIGURE 3 history of technology prior to the Industrial Revolution, CONTENT OF ONLINE MODULE FROM UNIT 5 SECTION 1— there were fewer video clips available. Therefore, the INTERACTION OF TECHNOLOGY AND CULTURE development shifted to a web-based delivery system. The pre-2000 version of Tech 198 began its historical With the development of all the multimedia and webview of technology and science in the Renaissance in based modules, the course content was established. Although Europe. Since technology is much older than modern each faculty member teaching this course include all seven science, this approach has limited the ability of students to units in their classes (see Table I), each faculty member see technology as a precursor to science. The expansion of covers the units in his/her preferred order. For all faculty, this course to include a study of technology and culture in Unit 1 (The Nature of Science and Technology) is covered the Middle Ages enhanced both the students' knowledge as first. After that, each faculty member may present the well as increase their understanding of the remaining six units in the sequence he/she establishes. interrelationships between technology and culture. The new unit on the history of technology also utilized the CSU ASSESSMENT OF THIS COURSE image database of art and cultural images [11]. Each section of this unit was structured in the same SJSU’s GE program is developed as an outcomes-based way. The first part presented a historical context for the program. SJSU uses course-embedded assessment to unit. The introduction, although not comprehensive, was determine the university’s achievement of its GE learning designed to assist students without a background or goals. Each course, which is approved for general education, appreciation of history. After reading these history sections, must submit an assessment report to document how students the student should have an understanding of the economic, meet the specific learning objectives for the GE area. Course978-1-4244-6262-9/10/$26.00 ©2010 IEEE October 27 - 30, 2010, Washington, DC 40th ASEE/IEEE Frontiers in Education Conference S1D-3

Session S1D

embedded assessment “uses instructor grading to answer questions about student learning outcomes in a non-intrusive, systematic manner. The process requires instructors to define learning objectives for each course, devise a rubric that measures these objectives, use the rubric to grade student work, record the data, and note needed changes for future course offerings” [15]. The fulfillment of course learning objectives does not directly affect students’ course grades, but rather serves as an internal evaluation mechanism for improving the course content/structure. This course is structured to measure student achievement of the SLOs for Area V of SJSU’s GE program. At the end of each semester, each instructor must submit an assessment report to the course coordinator that describes how the SLOs were measured and how many students met each learning goal. To determine student achievement of each GE SLOs, each instructor generally uses various, and usually multiple, measures of student performance. Since this class was first approved in Fall 2000, these measures have varied. However, the Tech 198 faculty as a whole continually monitors this class to ensure that it adheres to the Area V Goals and Objectives (see Figure 2). Typically, courses submit assessment reports every four years. Since Tech 198 had been recertified in AY 2002-2003 it was required to submit a complete assessment report to SJSU’s Board of General Education in Fall 2006 to remain in the General Education program. The assessment summary included information on how many students met each GE SLO and how this was assessed by the instructor. In Spring 2009, the GE assessment practices at SJSU changed. Faculty were asked to focus on one SLO at a time in each course; thereby allowing the faculty teaching the course to share assessment results and best practices. The Tech 198 faculty established a timeline for assessing each of the SLOs during three semesters (Spring 2009 to Spring 2010). This paper will discuss the results of assessment from Spring 2009 to Fall 2009. In Tech 198, Area V SLOs 1 and 2 are integrated into many class activities. Figure 3 displays two Research Exercises which are typically used by faculty to assess the SLOs in this course. These assignments are shorter than traditional term papers and must be completed several times throughout the semester; this allows instructors to give better content and writing feedback to the students. Because each instructor can select among a variety of exercises to achieve their SLOs, there is no inter-rater reliability in our assessment procedures. I. Assessment of SLO 1 Many of the readings in this class systematically look at technology and the values related to technology in different societies. Discussions and activities related to these articles allow students to identify the dynamics of various identities and see differences and similarities between these societies and modern American society. The instructors in this class

use individual and group activities to discuss the points of view from different articles. In addition to the assessment of student achievement through reading assignments and research exercises, the faculty use other activities to meet SLO 1. Several of the class activities on the multimedia CDs address this SLO. The following multimedia CD activities relate to GE SLO 1: What are the cultural aspects to scientific and technological literacy? (Unit 1, Section 3—The Nature of Science and Technology), Attitudes of, and about, women in technology (Unit 4, Section 5—Technology and Gender), Women working around the world (Unit 4, Section 4—Technology and Gender). Research Exercise 1 (GE SLO 2). Read the article “Islamic Inventiveness” in Science, volume 313, page 47. Use the weblink to visit the website for the 1001 Inventions exhibit. Explore each of the pages to deepen your understanding of the contributions that Islamic inventors have made to science and technology. Select a technology of interest to you from one of the inventions listed through the website. Using the on-line library resources, find at least four additional articles that provide more information on your selected invention and how it has affected people in the United States. You need to identify the historical context of ideas and cultural traditions outside the US and how they have influenced American culture. Research Exercise 2 (GE SLO 1). Hong, Z. (2007) India and China: Rivals or partners in Southeast Asia? Contemporary Southeast Asia 29(1), 121-142. Describe and analyze the main points that the author is making. Using library resources, find at least four additional articles that analyze the same issues. Write an essay that compares your research with the article I gave you. In your response, you should refer directly to your sources and write a comprehensive and integrative essay. In particular, you should compare the ideas, values, technological developments, and/or attitudes of people from at least two different countries outside the US. You may select China and India as your countries or you may select other countries to compare or another country to compare to China or India. FIGURE 3 SAMPLE ACTIVITIES FOR RESEARCH EXERCISES 1 AND 2

II. Assessment of SLO 2 The web-based module on the history of technology has been most helpful in meeting this objective. It was first used in Fall 2000 and has been included in all sections of the class since. Sample essay questions for Unit 2 include: 1. Discuss the history of timekeeping and the clock from its early beginnings to the Renaissance. Discuss at least three different events in the history of the clock. 2. Describe at least two forces (e.g., historical, cultural, social, economic, political) that affected the development of timekeeping and the clock? 3. What changes occurred in China (the country of origin of the mechanical clock) due to the development and use of the mechanical clock? Please give specific examples in your discussion.


Session S1D

4. 5.

How did the development and use of the mechanical clock affect Europe in the Middle Ages? How has the use of the clock influenced American culture?

In addition to the web-based assignment, the instructors use other assignments to assess the students’ achievement of this learning objective. Several instructors include essay questions on the midterm that require students to address historical changes in social and individual attitudes to technology and the differences in these changes in Western versus Eastern cultures. Several instructors create research papers that assess the students’ achievement of this SLO (see Figure 3, Research Exercise 1). Also, there are embedded class activities on the multimedia CDs for this class that assess this SLO. The two following multimedia class activities relate to GE SLO 2: Assembly line (Unit 3, Section 5—Technology and Work) and The Nature of Work Today (Unit 3, Section 7—Technology and Work) The results of the SLO assessment indicated that 80% of all the students in the course (from Spring 2009 to Fall 2009) successfully demonstrated that they met the SLOs assessed during those semesters (see Table II). A high percent of students received an “A” or “B” on their assignments covering these SLOs (59% in Spring 2009 and 71% in Fall 2009). Across all sections of this class, students performed at a higher level in meeting SLO 1. The instructors in this course surmise that this SLO is easier for the students to achieve since we conceptualize this SLO for this class as “Students shall be able to compare systematically the technological developments of people from more than one culture outside the U.S.” This SLO is at the core of this course; thereby, it is addressed in almost every class and activity done by the students. TABLE II NUMBER OF STUDENTS AND DISTRIBUTION OF STUDENT GRADES FOR SLO 2 AND SLO 1 Spring 2009 SLO 2 Fall 2009 SLO 1 assessment results assessment results A

37 (26%)

89 (47%)

B

49 (34%)

45 (24%)

C

28 (19%)

17 (9%)

D

5 (3%)

3 (2%)

F

26 (18%)

34 (18%)

ONLINE-ONLY VERSUS HYBRID CLASSES The instructors in this class were interested in determining whether there were any student achievement differences in the online-only versus hybrid classes. In both semesters, we offered two online-only sections and three hybrid sections. The class sizes for the sections ranged from 27-46 students. Table III shows the distribution of students in Tech 198 by their home college at SJSU. SJSU has seven colleges, offering 69 bachelor’s degrees with 81 concentrations. The highest number of students in this course is from the College

of Engineering; Engineering BS students represent 46% of the students in the hybrid class and 52% of the students in the online class. There are significant numbers of students from both Business and Applied Sciences & Arts in the class with smaller numbers from the other colleges at SJSU. TABLE III DISTRIBUTION OF STUDENTS BY COLLEGE (SPRING 2009 AND FALL 2009 COMBINED) Hybrid Class Student’s Home College

Number

Percent

Online Class Number Percent

Applied Sciences & Arts

37

15%

19

13%

Business

34

14%

20

14%

Education

3

1%

5

3%

Engineering

113

46%

75

52%

Humanities & the Arts

15

6%

4

3%

Science

15

6%

2

1%

Social Sciences

18

7%

15

10%

Open University/Undeclared

11

4%

3

2%

246

143

Although the grades for the hybrid class differ from the online class (see Table IV), this difference was not significant when compared with a t-test. We calculated the numeric grades for all grades in the class using SJSU’s grading system (4=A; 3.7=A-; 3.3=B+; 3=B; etc). When we compared the mean course grades and standard deviations of the combined students from both the hybrid and online course from the four largest colleges, we found no difference in the mean grades comparing the College of Engineering and the College of Applied Sciences and the Arts (see Table V). However, students in both the College of Business (p