Education and Information Technologies 7:2, 137–154, 2002. 2002 Kluwer Academic Publishers. Manufactured in The Netherlands.
Teaching the Use of Complex IT in Specific Domains: Developing, Assessing and Refining a Curriculum Development Framework JOHN P. DOUGHERTY ∗ Department of Computer Science, Haverford College, Haverford, PA 19041, USA E-mail:
[email protected] NED F. KOCK Department of Management Information Systems, Temple University, Philadelphia, PA 19122, USA E-mail:
[email protected] CHERYL SANDAS and ROBERT M. AIKEN Department of Computer and Information Sciences, Temple University, Philadelphia, PA 19122, USA E-mail: sandas;
[email protected]
Abstract Information technology holds the promise of increased productivity. However, rapidly evolving tools require a professional able to incorporate these tools into their careers effectively, which signals the need for IT curriculum development initiatives that incorporate the use of complex, domain-specific IT applications in specific professional fields. This paper reports on a study that addresses this need, by developing, assessing and refining a curriculum development framework. The Information Technology Fluency (ITF) framework is a methodology for constructing components (case studies) for inclusion into existing or newly proposed courses to help students develop the skills needed for this challenge. Results obtained using the framework are reported, compared to similar work at a different institution, and used to suggest improvements to the framework. Keywords: information systems, information technology education, lifelong learning, fluency with information technology, FITness
Introduction Information is a key component at the onset of the 21st century. Tools used to process information, collectively referred to as information technology (IT), are essential. The emergence of IT has been so rapid that many non-technical professionals are reluctant to incorporate IT into their activities. Those individuals who do incorporate IT often believe that they are underutilizing or misapplying these tools. Still others are simply uncomfortable with IT, having been denied any reasonable opportunity to learn how to utilize IT effectively (CSTB, 1999). ∗ Corresponding author.
138
DOUGHERTY ET AL.
The above scenario signals the need for IT curriculum development initiatives that incorporate the use of complex, domain-specific IT applications in specific professional fields (Aiken et al., 2000). This paper reports on a study that attempts to fill this gap, by developing, assessing and refining a curriculum development framework. The framework is aimed at the development of curricula that provide a strong foundation in IT that is embedded in hands-on-use of the technology in different complex, research and informationrich environments for students who are not information systems or computer science majors. The Information Technology Fluency (ITF) framework is a set of guidelines for the development of course components that provides an IT learning experience within an existing or newly designed course; computing may or may not necessarily be the primary subject. A set of non-trivial case studies acts to immerse students into projects where various skills and concepts related to IT can be developed. The goal of this paper is to develop, empirically assess, and refine the ITF framework. The case studies involve complex IT usage in a specific set of problem domains, yet assume that students only possess basic IT knowledge a priori. This paper is organized as follows. The “Research Background” section reviews theories and previous course development efforts that address IT education issues and that are relevant for the design of the ITF framework. This is followed by “The ITF Framework” section, which outlines development guidelines for the different components. The section outlines the procedures involved in case study design using the framework, showing which conclusions from previous course development efforts and theoretical principles should be implemented, and why. The next section, “Evaluating Implementations of the ITF Framework”, discusses three implementations of the ITF framework in a large university setting and compares them to relevant work at a small liberal arts college. The implementations involved the development of case study modules about the use of complex and specialized IT in the fields of anthropology, chemistry, and sociology. The three implementations are evaluated from quantitative as well as qualitative perspectives. The section “Refining the ITF Framework” follows, providing a discussion of the quantitative and qualitative analyses performed in the previous section vis-à-vis our previous discussion of former course development efforts and theoretical considerations. These analyses are used as a basis to generate a refined version of the ITF framework. Finally, the “Conclusion” section identifies future research opportunities and challenges.
Research Background There have been many efforts to identify the skills necessary to utilize IT successfully, as well as ways to develop these skills. There are also a number of theoretical models describing how IT competence is realized. This section covers related theoretical work in IT learning and in IT fluency, as well as surveys recent work to design and implement curricular materials and courses to promote IT fluency.
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS
139
Relevant theoretical work Several theories have been developed, particularly since the 1970s, to explain, predict and help design IT learning materials, courses and curricula, as well as use IT to teach other subjects. Conversation Theory proposes that IT learning occurs through discussions about the IT used in a specific context – these conversations serve to make knowledge explicit (Pask, 1975). Symbol Systems Theory argues that different communication media involve distinct levels of mental processing when used to convey subject matter content (Salomon, 1979, 1981; Salomon et al., 1991). GOMS Theory (goals, operators, methods, selection rules) looks at the cognitive skills required for human-computer interaction (Card et al., 1983). Programming-Facilitated Learning Theory argues that children need to operationalize concepts through writing computer programs in order to retain an understanding of those concepts in a variety of domains (Papert, 1980, 1993). Cognitive Flexibility Theory is primarily concerned with the transfer of IT concepts and skills beyond the initial learning situation – the theory concentrates on the very nature of learning in complex and poorly-structured domains (Spiro and Jehng, 1990; Spiro et al., 1992). Two remaining theories are reviewed separately and in more detail due to their relevance to the work discussed in this paper. Minimalist Theory places particular emphasis on the design of IT instruction materials (Carroll, 1990, 1998; Van Der Meij and Carroll, 1995). The theory proposes five main principles for IT education: meaningfulness, application, improvisation, recovery, and realism. The principle of meaningfulness implies that all IT learning tasks should be self-contained and significant to the learners. The principle of application proposes that learners should be given realistic IT projects as soon as possible in their learning curve. The principle of improvisation implies that IT instruction should permit self-directed reasoning and extemporizing. The principle of recovery states that IT training materials and activities should allow for error recognition and the chance to address those errors. Finally, the principle of realism implies that there should be a close linkage between the training and the actual use of the system. The ITF framework implemented the principles of application and realism from minimalist theory; the principles of improvisation and recovery can be incorporated into the resulting ITF modules given enough lab time, availability of expertise to answer student questions, and other support. Meaningfulness is difficult to achieve because no assumptions can be made about the backgrounds or interests of the students a priori due to the diversity of the student population; the only common characteristic is that they are not computing majors. Andragogy Theory emphasizes adult learning of IT (Knowles, 1975, 1984a, 1984b). The theory assumes that adults are self-directed and are expected to take a proactive role in, and responsibility for, IT learning decisions. Andragogy theory advocates that IT instruction for adults needs to focus more on process and less on content. Such approaches as case studies, role-playing, simulations, and self-evaluation are useful in achieving this goal. Andragogy theory argues that instructors should adopt the role of facilitators rather than lecturers or graders. The theory can be summarized as four main principles: design involvement, experiential learning, immediate relevance, and problem solving. The principle of design involvement proposes that adults need to be involved in the planning and evaluation of their IT instruction. The principle of experiential learning implies that
140
DOUGHERTY ET AL.
practical experience, including mistakes, should provide the basis for IT learning activities. The principle of immediate relevance states that adults are most interested in, and thus learn more efficiently, IT subjects that have immediate relevance to their professional or personal life. The principle of problem solving implies that effective adult IT learning is problem-centered rather than content-oriented. Experiential learning and problem solving are andragogy principles that are used explicitly in the ITF framework. Design involvement can be included provided the students are quite motivated and time permits the facilitator opportunities to discover student interests. ITF framework modules developed for this report have not explored this principle. Immediate relevance is a principle difficult to include for the same reasons as the principle of meaningfulness from minimalist theory.
Fluency with IT The National Research Council (NRC) convened a committee in response to the increasing importance and ubiquity of IT in daily life. The report of this group (CSTB, 1999) articulated a set of skills, concepts and capabilities that all citizens should know and understand about IT to empower them throughout the information age. While skills will need to be updated periodically, concepts and capabilities are timeless. The term “fluency” was introduced to denote a higher level of understanding than the term “literacy” which has been used in the past. A person capable of evaluating, distinguishing, learning and using new IT is characterized as “fluent with IT”, or FIT. “FITness” is a measure of the degree of IT fluency, supporting the notion that fluency is developed gradually. The report was explicit about the role of IT fluency in terms of life long learning: In short, FITness should not be regarded as an end state that is independent of domain, but rather as something that develops over a lifetime in particular domains of interest and that has a different character and tone depending on which domains are involved. Accordingly, the pedagogic goal is to provide students with a sufficiently complete foundation of the three types of knowledge that they can “learn the rest of it” on their own as the need arises throughout life. (CSTB, 1999, p. 3) The report also suggests a project-based approach to develop IT fluency. This approach challenges the student to coordinate information and skills, and to make decisions that consider multiple dimensions of a specific problem. The scope and scale of the project need to be sufficiently complex to make intellectual integration necessary for completion; however, the project must be accessible to the target population. The NRC report has started a debate about the distinction between IT literacy and fluency, and has opened the door to non-computing professionals. However, weaknesses of the report have been noted, including: levels of competence are defined by concepts rather than by standards of action; a lack of discussion regarding different levels of competence
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS
141
for different skills/concepts/capabilities; and a report panel that consisted of computer science academics and researchers only (Denning, 2000).
The ITF Framework As noted in the introduction, many students approach IT as a “necessary evil”, or at least as a required tool that is difficult to utilize. The ITF framework is targeted at these persons, designed to provide an effective means of introducing IT without overwhelming the student. A balance is sought to provide a relatively gentle introduction to IT concepts, yet permit active and discovery types of learning. This section will outline the ITF framework, discuss the type of learning environments where the framework would be appropriate, and finally describe the three case studies implemented using the framework. The process of case study creation begins with a goal; namely, to identify the specific skills, concepts and capabilities that need to be realized in the students. There are suggestions presented in (CSTB, 1999). It is assumed that this initial set will change as the case study is further developed, but a goal is still encouraged. The next phase involves selecting the subject area for the case study. The pervasiveness of IT in professional activity provides a rich set of topics for consideration. The primary criteria involved are: • a subject area where a set of IT tools is utilized; • the usage of these IT tools merits substantial thought and practice to truly challenge the current abilities of the student, supporting the principle of meaningfulness from minimalist theory; • problem examples exist that are accessible to the student, as well as open questions, the latter supporting the principle of problem solving from andragogy theory; • the assignment facilitates learning of the capabilities, concepts and skills from the initial phase of the project as suggested by the principles of application from minimalist theory and immediate relevance from andragogy theory; • a well-defined closure point for the assignment exists. Note that standard IT tools like word processors, browsers and spreadsheets are encouraged, and integrated into the case study, but should not be the only level of IT utilized. This objective can be achieved by introducing a non-standard tool specific to the case study, or to use a standard tool in a non-stand fashion dictated by the case study. For example, a physics case study might introduce an IT tool specific to the discipline (i.e., non-standard) to acquire data in real-time, and then use a standard spreadsheet application to process this acquired raw data. The subject area is often dictated by a number of local factors to the learning situation. The most important factor is typically the willingness of an expert in the area to become involved with the project. All of the case studies to date involve personnel from the subject area and others from computing. Other factors exist, including staffing issues, expected student demographics in the course, hardware and software availability, cost, and level of interest. It should be stated explicitly that the ITF framework does not assume
142
DOUGHERTY ET AL.
a particular type of course, but rather supports the seamless integration of IT fluency injected into any undergraduate course where non-computing students wish to develop IT capabilities. Following case study design is integration into the syllabus. Each study requires some tailoring, but some general techniques are useful. Group projects involving hands-on laboratory work are central to the case study. Closed labs and self-scheduled labs have been used successfully, depending on local factors. Facilitated discussion of the case study and its implications regarding IT usage are also included as recommended by conversation theory (Pask, 1975). Three case studies have been designed and implemented. Each one is chosen from outside of the field of computing and IT. The case studies are from the areas of anthropology, sociology, and chemistry, and are briefly described below. “Modeling human behavior over time and space: Deforestation in tropical America” is a case study from anthropology. This case study examines the expansion of tropical forest farmers and the accompanying deforestation in Central Panama during the time period from 9000 to 2000 years ago through the use of simulations carried out in a GIS (Geographic Information Systems) environment. The sociology case study is entitled “Occupational and age cohort consequences of the industrial transformation, 1980–1990”. The project examines and evaluates possible explanations for the shifts in occupational distribution that have occurred in the United States between 1980 and 1990. Data used in this case study are the one percent Public Use Sample of the 1980 and 1990 Censuses. The principal software tool used is Excel. The chemistry case study examines methods for correlating measured physical properties of simple organic molecules with their structures. Entitled “Exploring structures of organic molecules by computational methods”, it involves calculations performed with the aid of a commercially available software application for molecular design called Alchemy. The initial three case studies were conducted in a university setting as part of a dedicated course using the ITF framework to facilitate the learning objectives from (CSTB, 1999). This course was the second in a series of computing courses for non-computing majors. A set of six professors collaborated to conduct this course, sequentially alternating bi-weekly lectures between a computing professor and a non-computing professor who played the role of “case study expert”. The first week consists of two lectures that describe the purpose and flow of the course, review the syllabus and overview each component. After the first week, each professor covers his topic and try to links it to the course in general and, if possible, to the material of the next professor. Other support personnel included a graduate assistant to coordinate the lectures and labs for both the students and the professors (Aiken et al., 2000). The comparison course introduced IT concepts using an example from economics, and was conducted as part of a computing survey for non-science majors at a small, liberal arts college. The project, known as “Does money buy happiness? An analysis of economic survey data”, develops models and performs a statistical analysis of raw survey data to correlate economic standing with emotional state. Raw data is downloaded from online economic databases and processed with a commercially available software application
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS
143
called Stata. IT fluency was an explicit yet secondary goal to a breadth first treatment of computing. After a lecture outlining the role of IT, a professor of economics provided a single visiting lecture to outline the goal of the case study and to demonstrate the IT application used to achieve this goal. Students were then asked to try the tool independently. Class size and availability of lab support limited the required part of the case study. The final project for the course was extended to include an optional group project where students would utilize the IT tools presented to gather economic information and resolve a different economic goal. The local factors of IT fluency as secondary and limited lab equipment and time dictated the implementation of the case study. The primary distinction between the project from the comparison course and the three case studies involved implementation. The case studies had each expert provide two weeks of lecture and lab, whereas the comparison course could only utilize a single lecture from the economics expert, in conjunction with a lab demonstration and a student project. Also, the case studies involved more dedicated sessions for experimentation with the IT tools and the problems.
Evaluating Implementations of the ITF Framework Table 1 shows the questions (see also Appendix A: Questionnaire used for assessment) asked from students in the case studies as well as in the comparison course. The questions were split into three main types. Questions in the range Q1–Q3 refer to attitudes toward IT prior to taking the course. Questions in the range Q4–Q7 refer to the impact of the case studies (or comparison course) on attitudes toward IT. Questions in the range Q8–Q9 refer to the impact of the case studies (or comparison course) on IT knowledge. The questions were actually presented to the students as statements (see Appendix A) to which the students responded by indicating their level of agreement with the statements presented on a 5-point Likert scale ranging from 0 (Strongly disagree) to 4 (Strongly Table 1. Questions-statements used for collection of quantitative data Attitudes toward IT prior to taking the course Q1. I already had a very good knowledge of Information Technology (IT) prior to taking this course. Q2. I intended to take as many IT courses as possible in college prior to taking this course. Q3. I was generally very attracted to IT issues prior to taking this course. Case study impact on attitudes toward IT Q4. The case study made me feel like I should take more IT courses in the future, even if I don’t pursue a career in IT. Q5. The case study made me feel like I should pursue an IT career. Q6. The case study improved my perception of IT’s potential for solving complex tasks. Q7. The case study improved my general perception of IT. Case study impact on IT knowledge Q8. I learned a lot from this case study about specialized IT applications. Q9. I learned a lot from this case study about IT issues in general. Answers range: 0 = Strongly disagree; 4 = Strongly agree; Mid-scale = 2.
144
DOUGHERTY ET AL.
agree). Although questions Q4–Q9 were multiple-choice, they also included the same open-ended question “Why?” as follow-up. This follow-up question was added to motivate the students to think about their multiple-choice answers (i.e., not provide “mindless” answers) and thus reduce perception bias. Two additional open-ended questions, Q10 and Q11, were also included. The first of these open-ended questions, Q10, asked them about the main positive aspects of the case studies (or comparison course). The second open-ended question, Q11, asked the students about the main negative aspects of the case studies (or comparison course). The answers to these questions as well as to questions Q1–Q9 are analyzed below, the latter group coming first.
Quantitative analyses of the impact of the case studies on student perceptions (Q1–Q9) Table 2 shows the mean answers on the quantitative scales provided for questions Q1 to Q9, as well as the significance levels (P’s) obtained for Chi-squared tests of the frequency distributions of answers. The top part of Table 2, below the headings row, shows figures obtained for the three case studies covering anthropology, sociology, and chemistry issues. Below, in the row indicated as “Mean”, are the means for the three cases. The bottom part of Table 2 shows the figures obtained for the comparison group, where answers Q1 to Q9 referred to an entire generic non-majors computing course. P ’s lower than 0.05 signify that the probability that the distribution of answers for each question is due to chance is lower than 5%, and suggest significantly skewed distributions of answers (or a significant degree of agreement among respondents). Shaded columns indicate P ’s lower than 0.05 for at least two of the case studies. Table 2. Mean answers and frequency distribution analysis (Chi-squared test) Q1
Q2
Q3
Q5
Q6
Q7
Q8
Q9
2.08
1.33
1.33 1.75
0.75
2.58
2.58
2.20
2.00
< 0.05
0.13
0.43 0.71
< 0.05
0.13
0.13
< 0.05
0.09
Sociology (N = 18)
2.06
1.44
1.53 1.94
1.22
2.44
2.39
2.17
2.44
P (Chi-squared)
0.11
0.21
0.28 0.38
< 0.05
< 0.05
< 0.05
< 0.05
< 0.05
Chemistry (N = 14)
1.50
1.36
1.64 1.93
0.93
2.92
2.77
2.31
1.92
P (Chi-squared)
0.43
0.26
0.15 0.15
0.09
< 0.05
< 0.05
< 0.05
0.12
Mean
1.88
1.38
1.50 1.87
0.97
2.65
2.58
2.22
2.12
1.56
0.25
1.44 1.81
0.38
2.50
3.06
2.00
2.81
< 0.05 < 0.05 < 0.05 0.08
< 0.05
0.08
< 0.05
< 0.05
0.08
Anthropology (N = 12) P (Chi-squared)
Comparison (N = 16) P (Chi-squared)
Q4
Range: 0 (Strongly agree)–4 (Strongly agree); Mid-scale = 2.
145
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS
Column Q5 of Table 2 indicates a significant level of agreement that the case studies did not make the students feel that they should pursue an IT career. The same effect was observed in the comparison group, although the mean there (0.38) was lower than the mean for the three case studies (0.97). However, it must be noted that the mean answer for Q2 (I intended to take as many IT courses as possible in college prior to taking this course) was also much lower for the comparison group (0.25) than for the three case studies (1.38). Column Q6 of Table 2 supports the claim that students found the case studies improved their perception of the potential for IT to solve complex tasks. The chemistry case study had the highest Likert average (2.92). The mean case study average (2.65) was higher than that of the comparison course (2.50). It should be noted that P was greater than 0.05 for the comparison course, indicating less consensus among the respondents. Column Q7 of Table 2 indicates a significant level of agreement that the case studies did improve the students’ perceptions of IT. The same effect was observed in the comparison group, although to a higher degree there, which is indicated by a comparison of the mean answer to question Q7 for the comparison group (3.06) with the mean answer to the same question for the case studies (2.58). Column Q8 of Table 2 indicates a significant level of agreement that the case studies led the students to learn a lot about specialized IT applications (Q8). The same effect was observed in the comparison group, although to a lower degree there, which is indicated by a comparison of the mean answer to question Q8 for the comparison group (2.00) with the mean answer to the same question for the case studies (2.22).
Quantitative analyses of bivariate correlations (Q1–Q9) Table 3 shows significant correlation coefficients between answers to questions Q1 to Q9, which indicate co-variation of the perception-related variables represented by the Likertscale-based answers to questions Q1 to Q9. The actual coefficients (shown in Appendix B) were calculated using the Pearson bivariate correlation test (the type of the test was “twotailed”). To filter out weak effects, only correlations that met the following two criteria are shown on Table 3: P lower than 0.01, and Pearson coefficient higher than 0.6. All correlations shown are positive. Table 3. Significant correlation coefficients (Pearson Bivariate Correlation Test) Q4 Anthropology (N = 12) Sociology (N = 18) Chemistry (N = 14)
Q5
Q6
Q7
Q8
Q2, Q3 Q2, Q3 Q2
Comparison (N = 16) P < 0.01 and coeficient > 0.6; Q1–Q3: Prior atttitudes; Q4–Q7: Impact on attitudes; Q8–Q9: Impact on knowledge.
Q9
146
DOUGHERTY ET AL.
Intersections between the range Q1–Q3 and the range Q4–Q7 indicate co-variation of measures of attitudes toward IT prior to taking the course and measures of case study impact on attitudes toward IT; these intersections would appear in any of the four columns of Table 3 labeled Q4, Q5, Q6, and Q7. Intersections between the range Q1–Q3 and the range Q8–Q9 indicate co-variation of measures of attitudes toward IT prior to taking the course and measures of case study impact on IT knowledge; these intersections would appear in either of the final two columns of Table 3. Table 3 shows the occurrence of intersections between the range Q1–Q3 and the range Q4–Q7 for the case studies, and the absence of intersections between those ranges for the comparison group. These factors, along with the absence of significant negative correlation coefficients between those two ranges, suggests that the case studies had overall a weaker impact on attitudes toward IT than the comparison course. For example, the high correlation between Q4 and the pair Q2–Q3 for the Sociology Case Study suggests that those students who already had a very good knowledge of Information Technology (IT) prior to taking the course (Q2), and who intended to take as many IT courses as possible in college prior to taking the course (Q3), were also the ones who thought that the case study made them feel like they should take more IT courses in the future, even if they did not pursue a career in IT (Q4). At the same time, the case studies seemed to solidify the negative and positive attitudes toward IT to a higher degree than the comparison group.
Qualitative analyses of open-ended questions regarding positive and negative aspects (Q10–Q11) Table 4 summarizes positive and negative aspects of the case studies and comparison course as perceived by the students. Overall, the case studies evoked more negative course structure-specific comments (e.g., comments that are not related to the instructor) from the students than the comparison course, especially negative comments that the case studies were not particularly relevant to the students. On the other hand, the comparison course evoked negative comments related to it not being challenging enough and “scattered” in its coverage of IT topics. Table 4 suggests that the case studies had a positive impact on students’ perceptions about IT’s potential for supporting specialized tasks (PA1, PA2, PS1, PS2, PC1), and that the case studies evoked negative student perceptions regarding the relevance of this type of knowledge for their chosen major or career, particularly in the anthropology and chemistry cases (NA1, NA3, NC1, NC2). In these two cases, difficulties with using features of the computer systems (NC3, NC4) and their malfunction (NA6, NC3, NC5) were also mentioned as negative aspects, even though students also reported enjoying working with the systems (PA1, PC3, PC4). While in the sociology case the computer system was seen as simple and easy to use (PS4), and no malfunctions were reported, the case itself was seen as using complex and “boring” concepts and methods (NS1, NS2, NS3, NS4, NS5). The comparison course was perceived as fun and easy (PO1, PO2), as well as helpful in that it taught the students what they perceived as important practical skills. However, the topic coverage of the comparison course was seen as scattered (NO1) and not challenging enough (NO2).
147
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS Table 4. Main aspects of the case studies and comparison course POSITIVE
NEGATIVE
Anthropology PA1: New computer software and new info. about maps development PA2: Saw how computers can be used to trace human history PA3: Well organized lectures and labs
Anthropology NA1: Not relevant to career/major NA2: Material itself was very dry NA3: I am just not interested in this NA4: Doesn’t seem to be applicable to other fields of study NA5: Needs to be generalized, show how applied to a wide range of fields NA6: Computer glitches that need to be worked out
Sociology PS1: Illustrated IT and gave me an idea of what IT can do and IT’s potential PS2: Good use of data to find trends PS3: Informative PS4: Ease of using Excel PS5: Learn the job market that is arising and the way it may be in future PS6: It showed us the future of the job industry
Sociology NS1: Did not sound interesting at all NS2: It was somewhat boring and slow NS3: Calculations were tedious NS4: Complicated math NS5: Accountant like dullness, need more dynamic involvement from class
Chemistry PC1: Amazed at how computers work to make difficult tasks easier PC2: Made me aware of IT courses to stay away from PC3: Liked working with Alchemy in the lab PC4: Interesting to use Alchemy in relation with Internet
Chemistry NC1: Case study did not relate to me leaving me clueless NC2: Irrelevance to major, satisfies no requirement, waste of time NC3: Waiting long for results and didn’t know what to do with them NC4: Software was somewhat confusing NC5: Computers were slow
Comparison PO1: It was fun PO2: A full overview of IT issues PO3: Taught us important practical skills – HTML, Javascript, etc.
Comparison NO1: It could have been a bit more challenging NO2: I felt that the lectures were very scattered
Refining the ITF Framework The empirical results from the previous section show that the case studies based on the ITF framework does impact the perceptions of students regarding the role of IT. This change in perception is an important feature because it is likely that a course using the ITF framework might be the final place where an undergraduate receives formal exposure to the field. Lifelong learning begins with a basic understanding and appreciation of its role. The empirical feedback received, both positive and negative, suggests steps that can be implemented to refine the ITF framework. Results indicate that while this experience did not promote a career change towards IT, it did improve appreciation of IT. The positive comments from the qualitative analysis of Table 4 show that the principles of application
148
DOUGHERTY ET AL.
and experiential learning were valued. Hands-on IT usage, from spreadsheets to chemistry software, seemed to receive the most positive feedback. Students enjoyed actively using tools to solve problems, and enjoyed working in the lab and building skills that they would potentially use in their capstone projects. Negative comments indicate that other IT teaching principles need to be addressed (or in some cases, readdressed). Students at this level responded negatively to issues that surfaced while using the laboratory equipment. The principle of recovery describes recognizing and fixing errors that arise during the case study itself. However, problems with lab software and hardware act more as a distraction than a chance to learn. Therefore, substantial testing of IT tools is mandated for this student population. Student feedback also indicates that the principle of immediate relevance requires more attention in delivery of this material. Phrases such as “waste of time”, “somewhat boring”, and “just not interested” (each phrase taken from a different case study) demonstrate that it is worth the effort to find meaningful material for the case studies. This issue is hard to resolve because student interests vary widely. One remedy might be to solicit input from the students according to the principle of design involvement. This input could come from current or past students. Local factors may promote or inhibit this approach. Moreover, it is difficult to properly test IT tools for a case study that has not been completely defined beforehand. Finally, course facilitators need to have the proper IT and other tools to conduct the case studies. Software and hardware are often readily available; preparation, testing and evaluation time are just as important.
Conclusions This paper documents our experience designing, developing and conducting a set of case studies using the ITF framework. Case studies were constructed in the areas of anthropology, sociology and anthropology, with the goal of increasing student fluency with IT. Survey data of student perceptions has been used to gauge the impact of this set, as well as compare it to student perceptions resulting from another learning environment that also endeavored to improve IT fluency. These results have also been utilized to refine the ITF framework that was used to generate the case studies. The quantitative analysis from the completed questionnaire (Appendix A) indicates the following conclusions: (i) the set of case studies motivate students to reflect upon the role of IT somewhat more than the comparison course (Q6); (ii) the students surveyed indicate that the case studies had a positive impact on their perceptions of IT, though not as much as the impact reported by students in the comparison course (Q7); and (iii) students felt they learned more about specialized IT from the case studies than the comparison course (Q8). Other results suggest that prior background with IT does correlate positively to the resulting perceptions about IT after the case studies have been conducted. Further work needs to explore why a similar positive correlation was not found in the comparison course.
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS
149
Qualitative survey data emphasize the importance of selecting and testing IT tools to avoid problems (e.g., “computer glitches”, “waiting long for results. . . ” from Table 4) for they appear to be significant distractions. Negative comments regarding the perceived lack of usefulness with the case studies indicate that the ITF framework needs to emphasize the principles of meaningfulness and immediate relevance more strongly. These issues need to be addressed for the ITF framework to evolve into an important pedagogical tool that can provide students the proper foundation for effectively utilizing IT now and throughout their careers.
References Aiken, R., Kock, N., and Mandviwalla, M. (2000) Fluency in information technology: A second course for non-CIS majors. ACM SIGCSE Bulletin, 32(1), 280–284. Card, S., Moran, T., and Newell, A. (1983) The Psychology of Human-Computer Interaction. Erlbaum, Hillsdale, NJ. Carroll, J. M. (1990) The Nurnberg Funnel. MIT Press, Cambridge, MA. Carroll, J. M. (1998) Minimalism Beyond the Nurnberg Funnel. MIT Press, Cambridge, MA. Computer Science and Telecommunications Board (CSTB) (1999) Being Fluent with Information Technology. National Academy Press, Washington, DC. Denning, P. J. (2000) A commentary on fluency in information technology. Inventio (online journal), 1(2), www.doiiit.gmu.edu/Archives/spring00/pdenning_1.html. Knowles, M. (1975) Self-Directed Learning. Follet, Chicago, IL. Knowles, M. (1984a) The Adult Learner: A Neglected Species. Gulf Publishing, Houston, TX. Knowles, M. (1984b) Andragogy in Action. Jossey-Bass, San Francisco, CA. Papert, S. (1980) Mindstorms: Children, Computers and Powerful Ideas. Basic Books, New York, NY. Papert, S. (1993) Childrens’ Machines: Rethinking Schools in the Age of the Computer. Basic Books, New York, NY. Pask, G. (1975) Conversation, Cognition, and Learning. Elsevier, New York, NY. Salomon, G. (1979) Interaction of Media, Cognition, and Learning. Jossey-Bass, San Francisco, CA. Salomon, G. (1981) Communication and Education. Sage, Beverly Hills, CA. Salomon, G., Perkins, D., and Globerson, T. (1991) Partners in cognition: Extending human intelligence with intelligent technologies. Educational Researcher, 20(4), 2–9. Spiro, R. J. and Jehng, J. (1990) Cognitive flexibility and hypertext: Theory and technology for the non-linear and multidimensional traversal of complex subject matter. In Cognition, Education, and Multimedia, D. Nix and R. Spiro (eds), Erlbaum, Hillsdale, NJ. Spiro, R. J., Feltovich, P. J., Jacobson, M. J., and Coulson, R. L. (1992) Cognitive flexibility, constructivism and hypertext: Random access instruction for advanced knowledge acquisition in Ill-structured domains. In Constructivism and the Technology of Instruction, T. Duffy and D. Jonassen (eds), Erlbaum, Hillsdale, NJ. Van Der Meij, H. and Carroll, J. M. (1995) Principles and heuristics for designing minimalist instruction. Technical Communications, 42(2), 243–261.
Appendix A: Questionnaire Used for Assessment For questions 1–9, the following scale was used in the first part of the questions: ( ) Strongly disagree ( ) Disagree somewhat ( ) Neither disagree or agree ( ) Agree somewhat ( ) Strongly agree
150
DOUGHERTY ET AL.
Demographics: Sex: ( ) Male ( ) Female Age: _____ First language: ( ) English ( ) Other ______________ Attitudes toward IT prior to taking the course 1. I already had a very good knowledge of Information Technology (IT) prior to taking this course. 2. I intended to take as many IT courses as possible in college prior to taking this course. 3. I was generally very attracted to IT issues prior to taking this course. Case study impact on attitudes toward IT 4. The case study made me feel like I should take more IT courses in the future, even if I don’t pursue a career in IT. Why? (Explain your answer) 5. The case study made me feel like I should pursue an IT career. Why? (Explain your answer) 6. The case study improved my perception of IT’s potential for solving complex tasks. Why? (Explain your answer) 7. The case study improved my general perception of IT. Why? (Explain your answer) Case study impact on IT knowledge 8. I learned a lot from this case study about specialized IT applications. Why? (Explain your answer) 9. I learned a lot from this case study about IT issues in general. Why? (Explain your answer) Positive and negative aspects of the case study: 10. What were the main positive aspects of this case study? 11. What were the main negative aspects of this case study? Appendix B: Correlation Tables
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS
151
152
DOUGHERTY ET AL.
TEACHING THE USE OF COMPLEX IT IN SPECIFIC DOMAINS
153
154
DOUGHERTY ET AL.
