Users’ Verification of Information System Curriculum Design Model
Thong Chee Ling1, Yusmadi Yah Jusoh2, Rusli Adbullah3 and Nor Hayati Alwi4 1
4
UCSI University, No. 1, Jalan Menara Gading, UCSI Heights, 46500 Cheras, Kuala Lumpur, Malaysia.
[email protected] 2 Department of Information System, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, 43400, Serdang, Selangor, Malaysia.
[email protected]. +603-89471760, +603-89466576. 3 Department of Information System, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, 43400, Serdang, Selangor, Malaysia.
[email protected]. +603-89466557, +603-89466576. Department of Science and Technical Education, Faculty of Educational Studies, Universiti Putra Malaysia, 43400, Serdang, Selangor, Malaysia.
[email protected]. +60389468214.
Abstract. Users’ Involvement is very crucial in any model or system development in the domain of information systems. In a preliminary study conducted earlier, the findings shown that users faced difficulties during curriculum design process. Consequently, a model namely Information System Curriculum Design Model is proposed. The purpose of this paper is to perform users’ verification to ensure the proposed model meet requirements of users who are also called curriculum designers. Survey is conducted to gather responses from 90 currciulum designers in 20 Institution of Higher Learning (IHL). The survey findings indicate that each component of the proposed model is perceived important. Once the proposed model is verified, it is converted to a prototype for further testing in the future. Keywords: information systems, curriculum design, curriculum designers,
1 Introduction Users’ involvement is very crucial and it is also a widely acceptable principle in system (model) development [1]. According to Kujola et. al. [2], users’ involvement is the key concept in the development of useful and usable systems. Sun [3] also noted that the lack of users’ input (involvement) causes incomplete requirements and specifications. In this paper, users are involved in providing their view on the importance of the proposed model’s components. Earlier study shows that Information System (IS) is important to Institution of Higher Learning (IHL) especially in the domain of curriculum design [4]. Without the support of IS, curriculum design in IHL is time-consuming, error-prone and much effort needed
from the curriculum designers. Based on literature study, there is a lack of IS model for curriculum design. An Information System Curriculum Design (ISCD) model is proposed to facilitate the work of curriculum designers in order to fill up the gap in previous work. However, this paper focuses on users’ verification of ISCD model. Therefore, the research question in this paper is: “What are the important components and subcomponents of IS model for curriculum design?”. The survey results provide answers to the research question. The following subsections present the main and subcomponents of the proposed model and the basis of these components. 1.1 Information Systems Model In development of model, literature study is used as one of the methodologies in this paper. Through literature study, there is a common Information System (IS) model discovered and it is named Management Information System (MIS) model. This model is introduced by Raymond [5] and Figure 1 presents the structure of MIS model. Fig. 1 Basic Activities of IS [4]
The basic IS activities of MIS model are input, process and output (IPO). Input captures raw data from organization or external environment; process converts raw data into meaning form and output transfer processed information to people or activities that use it. Feedback is the output returned to appropriate members of an organization to help evaluate or convert input stage [6]. Environment actors such as customers, stakeholders and regulatory agencies interact with the organization and its IS. IPO becomes the fundamental basis of IS design and development of the proposed model. Table 1 illustrates the basic activities of IS in the proposed model and its main and subcomponents.
Table 1. Main components and subcomponents of the proposed model Basic IS Activities Input Process
Main Component
Subcomponent
Input
Internal Input External Input Curriculum Maintenance, Curriculum Assessment Curriculum Analysis CLO Design, CLO Mapping, CLO Alignment, Managing & Monitoring CLO Process Curriculum Database Output
Curriculum Design Module (subscribe to notification agent)
Curriculum Design Process
Output
Curriculum Database Output
1.2 Definition of Curriculum Design There are many definitions of curriculum given by various scholars. The term curriculum can be defined broadly as dealing with the experiences of the learner [7]. However for the purpose of this research, Tyler’s definition [8] is adapted. Hence, in this research, curriculum is defined as: a plan for action or a written document that includes strategies for achieving desired goals. Strategies here translate to curriculum design principles in designing the proposed model for curriculum design, and the design principle is curriculum alignment. Onstein and Hinkins [9] pointed that curriculum design is concerned with the nature and arrangement (or relationship) of four parts: objective, content, learning experiences and evaluation. These curriculum components as noted by Prideaux [10] are known as: content, teaching and learning strategies, assessment processes, and evaluation processes. Based on John Biggs’ work [11], curriculum components are named as intended learning outcomes (ILO), teaching and learning activities and assessment tasks. Various scholars named the curriculum components differently. However, based on the literature study it is found that they are rather similar, but different names or terminologies are used in expressing them. This research adapted John Biggs’s work in expressing curriculum components. Prideaux [10] defines curriculum design as the process of defining and organizing curriculum components into a logical pattern. The key term ‘defining’ means define the terminology of curriculum and curriculum components; and ‘organizing’ means organizing the curriculum components into a logical pattern. The logical pattern in this research refers to Bloom’s taxonomy [12]. It is in line with Bloom’s which is
stated in his book written in 1956 – “ …taxonomy should be a logical classification system in that every effort should be made to define terms as precisely as possible to use them consistently”. This research adapts the definition of curriculum design of Prideaux [10].
2 Methodology A survey is conducted among 90 curriculum designers in 20 Institutions of Higher Learning (IHLs) in Malaysia. The purpose of the survey is to determine the importance of each component in the proposed model. Survey respondents are curriculum designers who have at least one year experience in designing curriculum in IHL. The respondents are selected using snowball sampling technique due to accessibility of curriculum designers in IHLs. The sampling technique is preferred because it is efficient and inexpensive. The questionnaire which designed as survey instrument has a 5-point Likert scale (1-strongly disagree to 5-strongly agree) and the questionnaire has been pilot tested in ensuring its validity. This is in line with Mahdzan [13] who noted that pilot testing is important in ensuring the validity of questionnaires. Once the questionnaire is validated, an online survey is conducted in this study.
3 Results and Discussion This section is divided into two parts: The results and discussion. 3.1 The Results Prior to actual study, a pilot test is conducted among 30 curriculum designers from one business and Information Technology faculty in a reputable private IHL and they are asked to complete the online questionnaire. The pilot test proves reliability and validity of the instrument which means the instrument used in this study is reliable for measuring perception of curriculum designers on the importance and appropriateness of the components and subcomponents in the proposed model. Data are tabulated and tested for reliability using SPSS and the measurement analysis of the instrument is using Cronbach Coefficient Alpha. The results of the reliability test for pilot testing show that all items have fulfilled the minimum requirements of Cronbach alpha coefficient i.e. 0.6, therefore no items are deleted. The conclusion is the questionnaire is reliable and can be used on actual sample respondents. In the actual study, there are 90 respondents from 20 IHL (both public and private) participated in this survey. In order to respond to the research question stated in methodology section, the magnitude of mean of each component is calculated to determine its importance in terms of ranking in the respective components. The importance of each component is determined based on the agreeable level of the respondents using Likert scale from (1) “Strongly Disagree” to (5) “Strongly Agree”.
For example, if the respondent rate the component or subcomponent as “4- Agree” to “5-Strongly Agree”, it is highly important and appropriate as rating 4 and 5 in Likert scale has synchronous meaning of importance and appropriateness. The questions in the questionnaire are asked in such a way to determine the importance and appropriateness based on perception of respondents and the results are presented in terms of mean scores according to each component. Curriculum Database The statement (item I3) “Information of courses” (mean=4.46) scored the highest mean among other items. The statement (item I8) “Keywords which are measurable contain in taxonomy table” (mean=3.92) scored the lowest mean score among other items under the construct of Curriculum Database. The average mean scores of all the eleventh items measuring the construct of Curriculum Database are all above 3 (neutral or neither agree nor disagree) indicating the overall respondents agreed with the construct of Curriculum Database. The items that has the lowest standard deviation (SD=0.731) is I2, “Information contains in programme”, indicating that most of the response are very close to the mean value of 4.27. In contrast, the standard deviation of the statement (I6), “Credit system for a course” scored the highest (SD=0.925), indicating that the data is well dispersed. Overall, all the eleven items have moderate spread of responses because the values of standard deviation for all the items are moderate. All eleven items in the construct of Curriculum Database are slightly negatively skewed. In addition, all of the items in the construct have positive Kurtosis value. In other words, all of the items in the construct have more peaks than the normal curve. Curriculum Design Module Under the construct of Curriculum Design Module component, it possesses three subcomponents i.e. Curriculum Maintenance, Curriculum Analysis and Curriculum Assessment. The statement (item CM5) “Define assessment tasks (e.g. final examination) for a course” (mean=4.28) scored the highest mean among other items. The statement (item CM7) “Map each topic of course learning outcomes to one or more competencies at a specific knowledge level” (mean=4.00) scored the lowest mean score among other items under the construct of Curriculum Maintenance. The average mean scores of all the ninth items measuring the construct of Curriculum Maintenance are all above 3 (neutral or neither agree nor disagree) indicating the overall respondents agreed with the construct of Curriculum Maintenance. The items that has the lowest standard deviation (SD=0.731) is CM3, “Define the expected competencies in course learning outcomes”, indicating that most of the response are very close to the mean value of 4.22. In contrast, the standard deviation of the statement (CM6), “Validate adherence of learning outcomes format to Bloom’s taxonomy” scored the highest (SD=0.895), indicating that the data is well dispersed. Overall, all the nine items have moderate spread of responses because the values of standard deviation for all the items are moderate. All nine items in the construct of Curriculum Maintenance are slightly negatively skewed. In addition, all of the items
in the construct have positive Kurtosis value. In other words, all of the items in the construct have more peaks than the normal curve. The statement (item CAna2) “Monitor updated requirements set by accreditation bodies” (mean=4.12) scored the highest mean among other items. The statement (item CAna1) “Search topic through keywords in order to ensure topic is built appropriately across the courses” (mean=3.98) scored the lowest mean score under the construct of Curriculum Analysis. Since there are only two items under the construct of Curriculum Analysis, in conclusion, the average mean scores of all the two items measuring the construct of Curriculum Analysis are all above 3 (neutral or neither agree nor disagree) indicating the overall respondents agreed with the construct of Curriculum Analysis. The items that has the lowest standard deviation (SD=0.774) is CAna1, “Search topic through keywords in order to ensure topic is built appropriately across the courses”, indicating that most of the response are very close to the mean value of 4.12. In contrast, the standard deviation of the statement (CAna2), “Monitor updated requirements set by accreditation bodies” scored the highest (SD=0.864), indicating that the data is well dispersed. Overall, all the two items have moderate spread of responses because the values of standard deviation for all the items are moderate. The two items in the construct of Curriculum Analysis are slightly negatively skewed. In addition, all of the items in the construct have positive Kurtosis value. In other words, all of the items in the construct have more peaks than the normal curve. The statement (item CAss5) “Assess programme learning outcomes which specify a set of expected competencies to be achieved by students in completing a programme” (mean=4.08) scored the highest mean among other items. The statement (item CAss8) “Assess topics of courses in a programme by looking for redundancy” (mean=3.90) scored the lowest mean score among other items under the construct of Curriculum Assessment. Other items scored between highest and lowest respectively. The average mean scores of all the thirteen items measuring the construct of Curriculum Assessment are all above 3 (neutral or neither agree nor disagree) indicating the overall respondents agreed with the construct of Curriculum Assessment. The items that has the lowest standard deviation (SD=0.694) is CAss11, “Assess the learning needed to be obtained by students in courses by appropriately assessing topics obtained for each of the detailed of course learning outcome”, indicating that most of the response are very close to the mean value of 3.98. In contrast, the standard deviation of the statement (CAss1), “Produce notification message if requirements are not satisfied at each design process” scored the highest (SD=0.902), indicating that the data is well dispersed. Overall, all the thirteen items have moderate spread of responses because the values of standard deviation for all the items are moderate. All thirteen items in the construct of Curriculum Assessment are slightly negatively skewed. In addition, all of the items in the construct have positive Kurtosis value. In other words, all of the items in the construct have more peaks than the normal curve.
Curriculum Design Process Under the construct of Curriculum Design Process, it possesses four subcomponents i.e. CLO Design, CLO Mapping, CLO Alignment and Curriculum Assessment. These component and its subcomponents are verified through expert review earlier [14]. The total of five experts are interviewed and they perceived all the components and subcomponents important. Output The statement (item O1) “Produce curriculum document (e.g. Course syllabi) which is free of error than those produced manually” (mean=4.08) scored the highest mean among other items. The statement (item O7) “Produce assessment report on which is the incomplete design process” (mean=3.80) scored the lowest mean score among other items under the construct of Output. The average mean scores of all the seven items measuring the construct of Output are all above 3 (neutral or neither agree nor disagree) indicating the overall respondents agreed with the construct of Output. The items that has the lowest standard deviation (SD=0.822) is O7, “Produce assessment report on which is the incomplete design process”, indicating that most of the response are very close to the mean value of 3.80. In contrast, the standard deviation of the statement (O1), “Produce curriculum document (e.g. Course syllabi) which is free of error than those produced manually” scored the highest (SD=0.999), indicating that the data is well dispersed. Overall, all the seven items have moderate spread of responses because the values of standard deviation for all the items are moderate. All seven items in the construct of Output are slightly negatively skewed. In addition, all of the items in the construct have positive Kurtosis value. In other words, all of the items in the construct have more peaks than the normal curve. In summary, the findings show all components and subcomponents are agreed upon by respondents to be important and appropriate as all mean scores are above 3.0.
3.2 Discussion Curriculum database is the first component. Curriculum database is not an isolated component as data flow from Input to curriculum database and also to two other components in process namely curriculum design module and curriculum design process. Raymond’s model is adapted for both data flow and components. Referring to Raymond’ model, there are three key components: report writing software (adapted as curriculum design module embedded with software agent), mathematical model (adapted as curriculum design process) and database (adapted as curriculum database). These component are adapted in the proposed model namely Information System Curriculum Design Model, and the components are verified by the curriculum designers (users) in IHL.
4 Conclusions The proposed model is verified in this study. The evaluation results (through survey) have shown appropriateness and importance of the components in the proposed model. Future work focuses on converting the model to prototype for further validation.
References 1. Kujala, S.: User Involvement: A Review of the Benefits and Challenges. Behaviour and Information Technology, vol. 22, pp. 1-16 (2003) 2. Kujala, S, Marjo, K., Laura, L., Tero, K.: The Role of User Involvement in Requirements Quality and Project Success. In Requirements Engineering, 2005. In: 13th IEEE International Conference, pp. 75-84 (2005) 3. Sun, Z.W.: User involvement in System Development Process. In: 2nd International Conference in Computer Science and Electronic Engineering, pp. 410- 413, France (2013) 4. Thong, C.L. , Yusmadi, Y. J., Rusli, A., Hayati, A. N., Lee, K.C.: Identifying Difficulties in Curriculum Design for Institution of Higher Learning in Malaysia, Abstract of The Academic Development in Higher Education 2011, Malaysia (2011) 5. Raymond,M.J.: Management Information Systems, 6th Ed. New Jersey, Prentice Hall (1998) 6. Laudon, K.C., Laudon, J.P.: Management Information Systems: Managing the Digital Firm, vol. 12, Pearson (2014) 7. Ornstein, A. C., Hunkins, F. P.: Curriculum Foundations, Principles and Issues. Fourth edition. Pearson (2004) 8. Tyler, R.W.: Basic Principle of Curriculum and Instruction. The University of Chicago Press, Chicago (1949) 9. Ornstein, A. C., Hunkin, F. P.: Curriculum Foundations, Principles and Issues. Fifth edition. Pearson (2009) 10. Prideaux, D.: ABC of Learning and Teaching in Medicine: Curriculum Design. BMJ: British Medical Journal 326.7383, pp. 268 (2003) 11. Biggs, J. B.: Aligning Teaching and Assessing to Course Objectives. Teaching and Learning in Higher Education: New Trends and Innovations, vol. 2, pp. 13-17 (2003) 12. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., Krathwohl, D. R.: Taxonomy of Educational Objectives: Handbook I: Cognitive Domain. New York: David McKay (1956) 13. Mahdzan, A.: Kaedah Penyelidikan Sosio Ekonomi. Kuala Lumpur: Dewan dan Pustaka (1992) 14. Thong, C. L., Yusmadi, Y. J., Rusli, A., Hayati, A.N.: Expert Review on Capability Maturity Model in Curriculum Design, In: Teaching and Learning Conference (ICTLS 2016), pp.379-382, Malaysia (2016)
