An Australian view of generic attributes coverage in undergraduate programs of study: An information systems case study '(
Robert Snoke Queensland University of Technology, Brisbane, Australia
[email protected] Alan Underwood Queensland University of Technology, Brisbane, Australia
[email protected] Christine Bruce Queensland University of Technology, Brisbane, Australia
[email protected]
Abstract: This paper describes the generic attributes coverage in QUT's information systems undergraduate curriculum, as compared with the perceived requirements of industry and the stated requirements of curriculum documents developed by professional associations. The investigative techniques used to make these comparisons possible involved mapping the generic attributes against the objectives of the courses. Findings reveal a strong correlation between QUT's curriculum documentation for information systems and the requirements of professional associations for some attributes, and identify other attributes where there is only minor coverage in the QUT curriculum. They also reveal some disparity between curriculum requirements and the perceived needs of industry stakeholders. The investigation also revealed a need for university unit outline writers to be more explicit in identifying the generic attributes of IS graduates to be developed within a program of study. The methodology of this study is likely to be of interest to curriculum evaluators in a range of disciplines. Keywords: Generic Capabilities, Curriculum Evaluation, Industry Requirements Introduction One of the objectives of today’s tertiary curriculum is usually to equip graduates with generic attributes required for their particular discipline. These attributes, for example, team building, leadership, communication and information and technology literacies, are often defined either by the educational institution or the professional body that is associated with the discipline of study. Anecdotal evidence from Information Systems (IS) industry interviews and a study by Turner and Lowry (1999) suggest that tertiary curricula in IS do not meet the needs of industry; a situation that is likely to be mirrored in other disciplines. Educators of future professionals should be able to identify the depth of treatment given to the competencies of graduates, both required and desired by employers and professional bodies with specific reference to the discipline studied. This strengthens the role of universities, as a provider of professional education by aligning professional education with industry needs. This paper HERDSA 2002 '( PAGE 590
reports on the process of determining the depth of treatment given a set of generic attributes within the courses that comprise the program of study in Queensland University of Technology’s (QUT’s) Bachelor of Information Technology IT21. Comparisons of the depth of treatment of the generic attributes are made with the major curriculum documents Model Curriculum and Guidelines for Undergraduate Degree Programs in Information Systems (IS’97) and Information Systems-Centric Curriculum (ISCC'99), and industry views as expressed in the Australian Computer Society Core Body of Knowledge (Underwood, 1996). The generic attributes selected for investigation have been identified and validated in related studies (Snoke and Underwood, 1998a, 1998b, 1999, 2000, 2001a, 2001b). Aim and Significance of the Study The aim of the study reported here is to investigate the comparative interests of different stakeholders in IS curricula in relation to generic capabilities. The study will also be used to develop a technique for developing a more responsive tertiary curriculum that meets the needs of industry. Such a technique will enable institutions to map their curriculum offerings against the requirements of professional associations to identify strengths and weaknesses in their curriculum. Consequently they will be able to offer a curriculum that is more responsive to the local employment market that their institution serves. This is particularly important for regional institutions as they typically serve a much smaller employment area, often with specifically needed application expertise. Generic Competencies in the Information Systems Context The context of the study is briefly described for the benefit of readers unfamiliar with the IS discipline. The Information Systems (IS) discipline is defined in the IS model curriculum for undergraduate studies (IS’97) as "… encompassing two broad areas: (1) acquisition, deployment and management of information technology resources and services (the information systems function) and (2) development and evolution of infrastructure and systems for use in organization processes (system development). The information systems function has a broad responsibility to develop, implement, and manage an infrastructure of information technology (computers and communications), data (both internal and external), and organization-wide systems. It has the responsibility to track new information technology and assist in incorporating it into the organization's strategic, planning and practices. The function also supports departmental and individual information technology systems. The activity of developing systems for organizations and inter-organization processes involves the creative use of information technology for data acquisition, communication, coordination, analysis, and decision support. There are methods, techniques, technology, and methodologies for this activity. Creating systems in organizations includes issues of innovation, quality, human-machine systems, human-machine interfaces, sociotechnical design, and change management" (Davis, Gorgone, Couger, Feinstein and Herebert E Longenecker, 1997, p. 7). Weber (1996, p. 72) also defines information systems as “comprising information, systems and information systems.” He suggests that the three are interrelated. The information systems discipline is associated with related disciplines such as business, law, education, data communications, computer science, and leisure recreation as these disciplines and others use information systems. HERDSA 2002 '( PAGE 591
IS educators together with most other professional educators in universities have been increasingly concerned with the generic capabilities agenda. This paper uses the term generic attributes to describe a core set of abilities and characteristics of an individual. It has many meanings, interpretations and synonyms such as generic skills, basic skills, qualities, capabilities, knowledge and understanding and competencies (Moss and Liang, 1990; Stasz, McArthur, Ramsay and Lewis, 1993; Crebert, 1995; Doyle, 1996; Bradley, 1997). The higher education literature often refers to the concept of generic attributes as generic skills or competencies or more recently as generic capabilities. Competencies may be defined as consisting of skills, attributes or abilities and understanding or knowledge. Understanding or knowledge is defined as the content or core body knowledge of a subject discipline that a person has acquired. Skills are the routine implementation of the acquired knowledge or attributes or capabilities. Attributes or abilities are the personal qualities that are applied by an individual to a specific task under a given situation. The major IS curriculum documents IS'97 (Davis et al., 1997) and ISCC'99 (Lidtke, Stokes, Haines and Mulder, 1999) use the term generic attributes or competencies to be statements of content knowledge that the individual will possess at the end of a program of study. The generic attributes identified in IS'97 and ISCC'99 are shown in table one. Table One: Curriculum Document Generic Attributes
IS’97 Exit Attributes Communication Computer applications systems Information Technology and Tools Interpersonal Relationships Management Problem Solving Systems Development Methodologies Systems theory and Concepts Professionalism
Attributes of an ISCC’99 Graduate Personal Skills Systemic-thinking skills Problem-solving skills Critical-thinking skills Risk-taking skills Personal-discipline skills Persistence Curiosity Interpersonal Skills Collaborative skills Communication skills (oral, written, listening, and group) Conflict resolution skills Technical Knowledge and Skills Information abstraction, representation, and organization Enterprise computing architectures and delivery systems Concepts of information and systems distribution Human behaviour and computer interaction Dynamics of change Process management and systems development Some Information Systems domain knowledge Use of computing tools to apply knowledge
These statements or attributes are very generalised descriptions of content or themes that a person should explore during the course of their education within the IS field.
HERDSA 2002 '( PAGE 592
There exists much discussion as to the meaning of the terms competency, competent and competencies. A person by definition is defined to possess competencies if they are competent at a specified task under a given set of conditions on a specific date. The possession of a set of competencies does not necessarily imply that a person is competent at any task. Therefore a task of educators is to identify a minimalist set of qualities or competencies that will enable the graduate to obtain employment. This paper equates generic competencies to generic attributes that a graduate possesses upon completion of their degree. They include the exit statements of competence but are not limited to these content related attributes. Research Method The research method involved the use of a methodology as adopted from MacDonald and Swearingen (1990) in their study of the coverage of the key issues in Management Information Systems (MIS) courses. The MacDonald methodology modified for this study consists of four steps: (1) identify the generic attributes of graduates, (2) select the texts used in the program of study, (3) analyse the texts with regards to the coverage of the generic attributes using the objectives from the texts set for the units within the programs of study, and (4) count of the number of generic competencies that are covered by each objective. Step one involved an extensive Delphi study of both academics and industry representatives. Step two involved the listing of the prescribed texts for each of the courses within the program of study. Step three involved an analysis of the words listed in the objectives of each chapters of the prescribed texts used for the units of study, to provide a description of the IS discipline. This list of key words obtained from the prescribed texts was expanded with the addition of the Barki, Rivard and Talbot’s Information System Keyword Classification Scheme (Barki, Rivard and Talbot, 1988). Step four involved the mapping of the unit objectives against the generic attributes. A list of words from the texts based on the generic attributes became the vehicle for comparison with a second list compiled in a similar manner but based on the unit objectives as listed in the unit outlines. The number of times that a word appears on the second list indicates the strength of treatment of the topic within the context of the generic attribute. This will give a numeric value or count of the depth of treatment of the attribute within the course. The mapping was validated by means of interviews with the lecturers who wrote the study guides from which the information was taken. 20 percent of the lecturers were interviewed and asked to validate the mapping of the objectives against the attributes. The above totals were graphically represented using a Kiviat chart for the program of study as shown in Figure One. The mapping process is repeated for the ACS core body of knowledge and the syllabus documents IS’97, and ISCC'99. The mappings were overlayed on the same graph to allow for the identification of strengths and weaknesses of the program of study for the purposes of accreditation by the ACS. Figure Two shows the comparison of the IT21 course as compared with the international curriculum documents IS'97, ISCC'99, and the ACS Core Body of Knowledge using percentage coverage as the scale. This is done to allow for easier interpretation due to the difference in the actual number of objectives listed in each of the curriculum documents examined. Results and Discussion of Results The result of the count of the generic attribute coverage for IT21 as offered at Queensland University of Technology is shown in Table Two below. HERDSA 2002 '( PAGE 593
Table Two: Objectives that relate to the Generic Attributes in IT21
Attribute TOTAL % With respect to the IS discipline possess coherent, extensive, 119 29.82% theoretical and practical knowledge With respect to the IS discipline be technologically competent (the 56 14.04% person is able to use the current technology competently) With respect to the IS discipline possess theoretical and practical 14 3.51% knowledge in at least one reference discipline which include behavioural science, computer science, decision theory, information theory, organizational theory, management theory. With respect to the IS discipline possess the theoretical and practical 5 1.25% knowledge of related disciplines. For example, business, law, education, data communications, computer science or leisure recreation Retrieve, evaluate and use relevant information 24 6.02% Define problems in a systematic way 9 2.26% Analyse, synthesise and evaluate the various solutions 11 2.76% Consider the quality of the solution and its timeliness 4 1.00% Demonstrate practical knowledge and understanding in at least one 32 8.02% computer language Be able to participate in continued learning and intellectual 9 2.26% development and develop critical, reflective and creative thinking. Time management skills 1 0.25% Knowledge of how a business operates, is structured or is orientated 2 0.50% Understand the profit motive of business 1 0.25% Ability to reflect on own strengths and weaknesses 0 0.00% Confidence about their ability to learn independently 0 0.00% Self motivation 0 0.00% Work independently 1 0.25% Value the ethics of the Information Technology profession 2 0.50% Sensitivity to differences in gender, culture and customs 2 0.50% Possess a sense of basic curiosity about technology 0 0.00% Work as part of a team in a productive and cooperative manner 12 3.01% Written communication skills 61 15.29% Oral communication skills 23 5.76% Research skills 2 0.50% Participate in on-going professional development 0 0.00% Embrace change and be obliged to engage in incremental improvement 1 0.25% to keep up with the rapid change in technology Interpersonal skills 1 0.25% Adapt to unfamiliar cultures and operate in a socially and culturally 3 0.75% diverse environment Project Management Skills 4 1.00% Total 399 100%
It should be noted that a program of study exceeding the ACS rating only indicates that they exceed the minimum required for industry accreditation and does not represent endorsement by the ACS of the depth of treatment in a particular program of study. Also it should be noted that a program of study that does not rate higher than the model curriculum guidelines does not necessarily imply a weakness in the program of study, as the model curriculum guidelines are idealistic curriculum guides based on the USA model of education. From the graph in Figures One it should be noted that the attribute ‘With respect to the IS discipline possess coherent, extensive, theoretical and practical knowledge’ receives the most treatment. This is followed by reasonable coverage (count greater than 9) of the attributes ‘written and HERDSA 2002 '( PAGE 594
oral communications; working as part of a team; retrieve, evaluate and use information; technical competence within the IS discipline; reference discipline knowledge; define problems in a systematic manner; analyse and evaluate solutions; continued learning and intellectual development; and programming language competence'. 'Written communications skills' receives nearly three times the treatment of 'oral communications skills' within the IT21 program of study. The other attributes receive only minimalist coverage within the program of study having coverage of less than five times within the program of study. Areas given a minimalist coverage include: 'related discipline knowledge; consider the quality of the solution and its timeliness; time management skills Knowledge of how a business operates, is structured or is oriented; work independently; value ethics of the information Technology profession; sensitivity to differences in gender, culture and customs; research skills; embrace change; interpersonal skills; adapt to unfamiliar cultures and operate in a socially and culturally diverse environment; and project management skills'. In addition to the above mentioned attributes that received only minimalist coverage, the attributes ‘ability to reflect on own strengths and weaknesses; confidence about their ability to learn independently; self motivation; possess a sense of basic curiosity about technology; and participate in ongoing professional development' were not mentioned in the objectives of the course at all. The combined data graph in Figure Two shows the comparison between the program of study and the minimalist set of knowledge required by the ACS as well as the more idealistic set of objectives from the model curriculum guidelines from the United States. Caution should be exercised in interpreting the graph due to the large number of objectives that are used in the model curriculum documents compared with the number of objectives used in the program of study and the ACS core body of knowledge. The program of study at QUT and the ACS core body of knowledge exceeds the model curriculum coverage of the attribute of 'With respect to the IS discipline possess coherent, extensive, theoretical and practical knowledge'. The ACS core body of knowledge gives significantly more treatment to the project management competency than either the model curriculum documents or the program of study at QUT. The treatment of the other generic attributes is only very minimal with many of them receiving no coverage within the curriculum documents or the IT21 program of study at QUT. This situation of minimal treatment of a significant number of generic attributes may occur in other disciplines. The curriculum documents IS’97, ISCC’99 and the current program of study at QUT all give significant treatment to three generic attributes (IS knowledge, technical competence in IS, and written communications skills) with the curriculum to the detriment of the other generic attributes. The objectives of the courses within the programs of study at QUT and the major curriculum documents for the IS field need to be rewritten to reflect the coverage of the generic attributes given within the courses. This study identified the fact that in the United States curriculum guidelines possess a significantly larger number of objectives per course than those written in Queensland. For example many of the courses listed in IS'97 have in excess of 30 objectives where the courses written in Queensland usually have 6-9 objectives. This disparity between the number of objectives per course may in part be due to the practice in the United States of America of having one objective per hour of instruction or learning in a course while in Queensland the past practice has been to have only six objectives for a course. The writers at QUT will need to expand on their objectives to explicitly state the competencies to be addressed within a
HERDSA 2002 '( PAGE 595
course and the manner in which the student will develop them. This needs to be addressed within the context of the QUT defined generic competencies of graduates. The ACS industry requirements focus on the attributes of 'With respect to the IS discipline possess coherent, extensive, theoretical and practical knowledge; technical competence within the IS discipline; reference discipline knowledge; consider the quality of the solution and its timeliness; and project management skills' at a level of the curriculum documents. The other attributes receive little mention in the industry requirements. 'Written communications skills' and 'oral communications skills' receive significantly less coverage in the industry requirements than in the model curriculum guidelines. Figure One: IT21 Generic Attribute Coverage
IS Knowledge Project Management Skills 120 Technologically Competent Operate in diverse environments Reference Discipline Knowledge 100 Interpersonal skills Related Discipline Knowledge Embrace Change Professional Development
80
Retrieve, evaluate, use information
60
Define problems
40
Research skills
Analyse and evaluate solutions
20 Oral communication skills
Quality of the solution
0
Written communication skills
Programming Language Knowledge
Work as part of a team
Continued learning and intellectual development
Curiosity About Technology
Time management skills
Sensitivity to gender, culture and customs
Business operations, structured and orientation
Ethics Work independently Self motivation
Profit motive of business Reflect on own strengths and weaknesses Ability to learn independently
35% 30% 25%
ACS
20%
IS'97 ISCC99
15%
IT21
10% 5% 0% Te ch IS Re no K fe lo no re gi w nc c le e D ally dg Re Co isc e lat ip m lin pe ed Re ten eK tri D isc ev t no e, ip w lin ev e K ledg al ua e no te w ,u led se ge A in na fo rm ly D se ef at in io an e n d Co pr ev ob nt Pr alu in lem Q ue ogra ate u s d al so lea mm ity lu i rn tio of in ng L ns th g an e a nd Bu so gu lu ag sin in tio tel eK es n lec so no tu pe w al le ra T d dg im tio ev e e ns elo m ,s an pm tru ag en Re ct em t ur fle en e ct d ts an Pr on ki o d lls fit ow or m i n en o str ta en tive tio gt n A hs of b bi lit a nd usin y to es w s ea lea kn rn es in se de s pe nd Se en Se lf t l y m ns W ot iti or iv vi k ati ty in on de to p ge en nd de er nt ,c ly ul Cu t ur rio Et e h sit ics an y d A cu bo sto ut W m Te or s W c k hn rit as ol ten pa o gy rt co of m m O at un ra ea lc i m om cati on m un sk ill ica s tio n Pr Re sk of ill se es s ar sio ch na sk lD ill ev s e Em lop m br O en ac pe eC t In ra ter te ha pe in n ge rso di ve na Pr rs ls ee oj ki ec n l ls t M vi an ronm ag em ent s en tS ki lls
Percentage Covered in the Course
Figure Two Combined Graphical Representation of IT21, ACS, IS'97, ISCC'99
Attributes
HERDSA 2002 '( PAGE 596
Implications of the Results and Further Research While the data reported here comes from a single university, work in progress indicates that similar courses in other universities reveal similar trends. This suggests that the implications of the investigation reach beyond the local community where the data here reported was collected. One of the clearest implications of the results is the need to enhance the objective writing skills of the authors of the course unit outlines from which this study gathered data. Specifically staff involved in the writing of the course unit outlines need to learn how to incorporate the development of the generic attributes within the course objectives. In order to do this staff will need to identify how they will empower students to develop their generic capabilities within a specific course. Such attention to unit outlines is critical because of its role as a communication tool to students and employers. The course unit outline is usually given to students as an indication of the capabilities they will be developing within a program of study. Students may use such unit outlines at job interviews to communicate the particular competencies that they have developed within the program of study. A second implication is the university curriculum writers may need to recognise the disparity between the requirement of professional curriculum documentation and that represented in the university curriculum. Course unit outline writers need to be able to interpret the curriculum objectives and communicate the objectives coverage of the generic attributes within their unit objectives. A third implication is that universities, when they are redesigning or applying for reaccreditation of a program of study, will need to consult with the relevant industry body and members of the wider community that employs their graduates about the generic capabilities that are required in entry-level employees. Universities will need to repeat the Delphi study that formed the initial phase of this study, when they begin the process of industry consultation as part of the continual curriculum evaluation process. Further research has already begun to reveal that trends in other IS curriculum around Australia is not dissimilar to those identified at QUT. This investigation is also being extended to investigate the amount of treatment that each attribute receives in post-graduate IS curricula at QUT. Further extension will involve the examination of the methods of assessment of the generic attributes within the programs of study. The methodology used to generate data for comparison is expected to be one of the most useful outcomes of the work, and should be replicable in other disciplines wishing to undertake similar curriculum evaluation in relation to the generic capability agenda. References Barki, H., S. Rivard, et al. (1988). "An Information System Keyword Classification Scheme." MIS Quarterly(June): 299-322. Bradley, D. (1997). The qualities of a University of South Australia graduate Information for External Members of University Committees, Denise Bradley. 1998. Crebert, G. (1995). Implementing Generic Attributes. Brisbane. Davis, G. B., J. T. Gorgone, et al. (1997). IS'97 Model Curriculum and Guidelines for Undergraduate Degree Programs in Information Systems. Doyle, K. (1996). Framework for the Development of National Competencies for the IT Industry. Lidtke, D., G. Stokes, et al. (1999). ISCC'99 An Information Systems-Centric Curriculum'99 Program Guidelines for Educating the Next Generation of Information Systems Specialists, in Collaboration with Industry My notes on the document. MacDonald, L. and J. Swearingen (1990). "Coverage of Key Issues in the MIS Course." Journal of Information Systems Education 2(4).
HERDSA 2002 '( PAGE 597
Moss, J., Jr. and T. Liang (1990). : Leadership, Leadership Development, and the National Center for Research in Vocational Education. Berkeley, National Center for Research in Vocational Education. Snoke, R. and A. Underwood (1998a). Generic Attributes of IS Graduates - An Australian Study. European Conference on Information Systems, AIX, France. Snoke, R. and A. Underwood (1998b). Generic Attributes of IS Graduates - A Queensland Study. Australasian Conference on Information Systems, University of New South Wales, Sydney, New South Wales. Snoke, R. and A. Underwood (1999). Generic Attributes of IS Graduates - an Australian IS Academic Study. 10th Australasian Conference on Information Systems, Victoria University of Wellington. Snoke, R. and A. Underwood (2000). Generic Attributes of IS Graduates - A Comparison of Australian Industry and Academic Views. Fourth Pacific Asia Conference on Information Systems, Hong Kong University of Science and Technology, Hong Kong University of Science and Technology. Snoke, R. and A. Underwood (2001a). An Australian Industry View of the Generic Attributes of IS Graduates. 9th European Conference on Information Systems, Bled, Slovenia. Snoke, R. and A. Underwood (2001b). "Generic Attributes of IS Graduates - A Comparison of Australian Industry and Academic Views." Journal of Global Information Management Vol 9(No 2): 33-40. Stasz, C., D. McArthur, et al. (1993). Teaching and Learning Generic Skills for the Workplace. Berkeley, National Center for Research in Vocational Education. Turner, R. and G. Lowry (1999). Educating Information Systems Professionals: Towards a Rapprochement Between New Graduates and Employers. 10th Australasian Conference on Information Systems, Victoria University of Wellington, School of Communications and Information Management, Victoria University of Wellington. Underwood, A. (1996). The ACS Core Body of Knowledge for Information Technology Professionals, Australian Computer Society. Weber, R. (1996). A Neglected Aspect of the Information Systems Curriculum: What is and Information System? Australian Information Systems Curriculum Working Conference, Melbourne, Department of Information Systems Monash University. Copyright © 2002 Robert Snoke and Alan Underwood: The authors assign to HERDSA and educational non-profit institutions a nonexclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to HERDSA to publish this document in full on the World Wide Web (prime sites and mirrors) on CD-ROM and in printed form within the HERDSA 2002 conference proceedings. Any other usage is prohibited without the express permission of the authors.
HERDSA 2002 '( PAGE 598
