An Assessment Framework for Identifying Information Technology Bachelor Programs Dale C. Rowe Ph.D.
Barry M. Lunt Ph.D.
Richard G. Helps Ph.D.
Brigham Young University Information Technology Program Provo, Utah +1 (801) 422 6051
Brigham Young University Information Technology Program Provo, Utah +1 (801) 422 2264
Brigham Young University Information Technology Program Provo, Utah +1 (801) 422 6305
[email protected]
[email protected]
[email protected]
ABSTRACT
1. INTRODUCTION
What is an IT Bachelor program and how can one be identified? In recent years, specific IT programs have arisen in the US to fill an industry need not directly provided for by other computing disciplines. IT programs are growing in number and influence yet can be hard to identify for several reasons: They are not always titled “Information Technology.” They are housed in various colleges and schools – such as engineering, business, computing, and information science. Also, they may not have associated themselves with ABET and hence may not be accredited. We have undertaken research that uses the ACM IT 2008 model curriculum to identify the “fit” between a set of minimum criteria and published university undergraduate curriculum regardless of the title, college, or professional association of the program.
How do we identify an Information Technology program? To anyone familiar with the topic this is a frequently asked question. The appropriate naming of various computing related disciplines and their associated degree programs has historically posed a significant problem for institutions and students alike. Computing-based career paths differ substantially and likewise the ideal field of study for a student or academic will differ depending on which path they wish to pursue. Some popular titles for computing majors include, but are certainly not limited to:
Our results show three groups of IT-related programs: 1.
Programs that meet all the ACM IT 2008 model curriculum guidelines.
2.
Programs that only partially meet the ACM IT 2008 model curriculum guidelines because they require a significant amount of courses from unrelated disciplines like business, psychology, engineering, etc.
3.
Computer Science (CS)
Computer Engineering (CE)
Software Engineering (SE)
Information Technology (IT)
Information Systems (ISys)
Computer Systems Engineering (CSE)
Information and Communications Technology (ICT).
While some of these titles suggest a naturally obvious identity to the subject matter, this is not the case for all programs. To illustrate this we need look no further than Computer Science and Computer Engineering; both of these are descriptive titles that suggest a natural emphasis on computing algorithms and programming, or in the latter case computer architecture. More recently emerging programs such as Information Technology and Information Systems are perhaps more vague in their title and as such are frequently subject to the teaching institution’s interpretation. These somewhat ambiguous titles have stimulated numerous efforts to bound and define the curricula in order to both differentiate and harmonize the content and research objectives of computing programs.
Programs that partially meet the ACM IT 2008 model curriculum guidelines because the program specializes in an IT-related major such as networking, embedded systems, database administration, HCI, etc.
This paper will present the criteria we propose to use as we attempt to identify all the 4-year IT programs in the USA.
Categories and Subject Descriptors K.3.0 [Computers and Education]: General
General Terms Standardization
In 2005, an effort was made to provide a high level overview of the top five previously listed programs [1]. This effort has been supported by the publication, review and acceptance of a series of standards-based curricula for Bachelors degree programs [1-5]. These endeavors aim to clarify distinct yet harmonious program curricula and research objectives.
Keywords Information Technology; Model Curriculum Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGITE’11, October 20–22, 2011, West Point, New York, USA. Copyright 2011 ACM 978-1-4503-1017-8/11/10...$10.00.
The IT Model Curriculum of 2008 [1, 6] attempted to form a structured view of Information Technology as a taught discipline and correlates ACM, ABET and SIGITE objectives with existing curricula thus enabling a standards-based definition of IT that is aligned with ABET accreditation expectations.
123
ambiguity over an appropriate structure and content for an IT program. To the contrary, as demonstrated in the model curriculum, it would be erroneous to make such an assumption. The aforementioned understanding of IT is in fact foundational to defining the academic content and context of an IT program.
The objective of this study is to propose an assessment framework that is suitable for identifying a computing bachelors program and determining its Information Technology content and consequently whether it should be classified as an IT program – regardless of its current title.
The model curriculum highlights the academic structure of Information Technology as built upon a foundation of Information Technology fundamentals, bound together by Security and Professionalism with five supporting pillars. These pillars carry the topics of Programming, Networking, Human Computer Interaction, Databases and Web Systems. We suggest that the placement of Human Computer Interaction in the middle of these pillars is a deliberate measure. This shows that while the topic bears equal importance to its surrounding four pillars, it is a central aspect of Information Technology.
2. WHAT IS I.T.? Perhaps the earliest use of the term Information Technology is recorded in the Harvard Business Review in 1958 [7] to describe ‘new technology’ in businesses. The article provides a fascinating insight into the early perceptions of computing in the last half-century as the authors attempt to predict the management impact of IT some 20-30 years in their future. Remarkably they identify a need for an ‘operations researcher’ or ‘organizational analyst’ to provide a human interface between management and technology, a role that we have come to know today as part of IT.
Finally, an Information Technology program should strike a balance between technological and conceptual learning. Through the proper application and hands-on use of technology, students should be educated in the proper concepts of the discipline that extend beyond the typical lifespan of a piece of technology. Our observations are that conceptual instruction through technology assists students in understanding the principles of the discipline. In particular this highlights the need for them to keep their knowledge base up-to-date throughout their careers and hence become effective user advocates with technology.
It is entirely possible that the authors of this 1958 article did not know just how aptly they described today’s IT professional. In contrast, their biggest error was limiting Information Technology’s role to business management. Since this time, we have seen IT weave its way into every aspect of our lives from personal communications and time management, to entertainment and research. As IT becomes more pervasive, the role of the IT professional becomes more and more important in providing a means for users to achieve their goals through the use of computing technology.
3. EVALUATION METHODOLOGY
The IT Model Curriculum contains a similar definition: Information Technology in its broadest sense encompasses all aspects of computing technology; IT, as an academic discipline, is concerned with issues relating to advocating for users and meeting their needs within an organizational and societal context through the selection, creation, application, integration and administration of computing technologies. [6]
The first of two principal methods for identifying IT Programs consist of reviewing the ABET student outcomes [8] [6] and defining a suitable set of measures to determine conformance. The focus of this method is on the body of knowledge held by the graduating student rather than the curriculum of the course and as such is an excellent measure of a program’s capacity to produce IT Professionals. The drawback of this approach is in developing measures that may be determined from publically accessible program information rather than post-study graduate and employer evaluations.
Thus under these definitions, it may be said that IT occupies an equilibrium between technology-centric and people-centric disciplines; and that consequently, IT programs should seek to understand, maintain, teach and develop this balance with growing importance as technology and society reach new levels of synergy.
The alternative method focuses on the curriculum offered within an IT Program and is intended to measure the program’s conformity to the core Body of Knowledge. This body is particularly appropriate for two reasons. First and foremost it has already been accepted as the baseline minimum requirement for Information Technology bachelors degrees by the ACM and IEEE-CS. [6] Secondly, the IT body of knowledge is presented concisely by means of a topical list; this facilitates a simple correlation to an institution’s website. The advantage of this approach over the outcome methodology is significant. By directly comparing the curricula in this manner it is simple to calculate a numerical value that measures the level of program conformance. Equally any non-conformity is relatively easy to spot and, if desired, resolve. In contrast the outcome methodology may require significant research and effort to determine where course changes are necessary to increase conformance. Measures are assessed using a simple point system. If a measure is met by the institutions curriculum, a point is awarded. If a measure is not met, no point is given. This simple scoring system will allow programs that exceed a stated threshold to be classified as fully conforming programs; it will also provide a simple indicator to the portion of IT 2008 compliant content for partially
Figure 1 - Pillars of Information Technology Therefore, we present user advocacy as the single most distinctive and defining characteristic of Information Technology as an academic and professional discipline. This singular definition may at first sight lead to the perception that there may be
124
The 2008 IT Model Curriculum has was created alongside the ABET (Accreditation Board for Engineering and Technology) standards and has been recognized by The Computer Society (IEEE-CS), the Association for Computing Machinery (ACM) and the Association for Information Systems (AIS). International bodies that also share an interest in IT include (but are not limited to) the BCS, The Chartered Institute for IT (formerly known as British Computer Society) and the Institute of Engineering and Technology (IET). These form the primary accreditation authorities for technical computing curriculums within many commonwealth countries and along with the ACM and IEEE are often used in various engineering program development processes.
conforming programs. Each point corresponds to the IT Body of Knowledge summary found on page 27 of the 2008 Model Curriculum. [6] Additionally, the evaluation will take into account content covered by elective courses within a program (although the total points from core and elective must never exceed the points available for a section). It is hoped that this will be beneficial to institutions that migrate towards a greater level of compliance by introducing additional courses and then migrating these courses over time to their core. Points awarded for core material covered in an elective course are incorporated into the compliance factor at 50 percent of their value. Table 1 - Assessment Summary (Performed against Website) IT2008 Body of Knowledge Core Topic Title
Module Type Core
Elective
Available
IT Fundamentals Human Computer Interaction Information Assurance and Security Information Management Integrative Programming and Techniques Mathematics and Statistics for IT Foundations of Networking Programming Fundamentals Platform Technologies System Administration and Maintenance System Integration and Architecture Social and Professional Issues
4 7 11 6 7 7 6 5 6 4 7 9
Web Systems and Technologies
6
TOTAL
In performing an assessment against an international institution, the countries prerequisite requirements for entry should be carefully scrutinized. In some instances, it may be appropriate to award core content points based on an institution’s prerequisite entry requirements. Particularly in the case of UK Universities, these pre-requisites may also be included in a foundation level course of the same title.
85
Weighted Total Instructions for Weighted Total Calculation
The IET is principally an evaluation-driven framework for Engineering and Technology process accreditation and allows intuitions to fast-track students to Chartered Engineer status. The BCS presents a more detailed curriculum for Information Technology in the form of a Professional Graduate Diploma in IT which is stated as being academically equivalent to the third year of a 3-year UK university honors degree. Prerequisite certifications of a Certificate in IT and a Diploma in IT are equated to the first and second years of a bachelor’s degree respectively.
Points
1.00 Divide total by 85 and enter above
Divide total by 170 and enter above
Core Coverage (Sum of weighted totals)
1.00
Capstone Experience (1 Point per Semester)
2
Core content less than 50% of total content
1
IT Compliance Factor (Total above three values)
5. CONTINUAL IMPROVEMENT
4.00
The model curriculum also stipulates the inclusion of a capstone project experience for students to take place in their final year of study. This real world experience is deemed critical to the successful preparation of students for future employment and the reinforcement of conceptual learning through the proper application of technology.
Possibly the greatest challenge faced by IT professionals and academics in establishing formal definitions of IT is the rate at which technology and society are evolving. The issue raised here is that if the reason that a program scores poorly as an IT program may be because it is a different program or because it is an IT program that is “ahead of the curve,” adopting new areas of IT prior to peer programs and should therefore be celebrated rather than criticized. There are some models of curriculum design that can help with this problem.
Finally, an additional full point is awarded if the core content (excluding a capstone experience) represents less than half of the total course content. This is intended to ensure institutions maintain a sufficient quantity of advanced content. The model curriculum suggests that 18 percent of total program hours should be spent on core content; however, variations in depth of instruction may be difficult to ascertain from a program curriculum. Assessors should be aware that more advanced instruction material presented in the core should not count as the core curriculum.
Firstly it must be acknowledged that this challenge exists for all computing disciplines but there are exacerbating factors for IT. Both because it is a professional degree program and because it follows the tradition of other technical technology degree programs, IT instruction emphasizes and champions authentic instruction using current technology [9, 10]. Also many technology programs use phrases like “hands-on” or “experiential” to characterize their educational approach [9, 1113]. This instructional approach differentiates technology programs from science programs and even from many engineering programs. For IT students to graduate as competent professionals in their discipline, it is necessary for them to be able to understand and apply current technological developments. Since the computer technology used in this experiential education changes constantly and rapidly [14-19], as opposed to the relative stability of the underlying concepts and theories, the student learning experiences need to be constantly updated.
4. INTERNATIONAL ASSESSMENTS It goes without saying that degree programs in Information Technology and the other computing disciplines exist outside of the United States. Significant differences in worldwide educational systems mean that direct program comparisons are difficult. Within the United States, a typical Bachelor’s degree lasts four years and includes a large body of general studies early in the program followed by a Major emphasis. Conversely, most European institutions consider students to have acquired sufficient pre-requisite general knowledge during secondary education and provide three-year Bachelor programs that are entirely focused on the subject of the degree.
The problem of evaluating a leading edge IT program is also exacerbated by the zero-sum-game of university curriculum design. Many universities have policies limiting the number of
125
Being involved in the IT2008 Model Curriculum it was expected that the program would score highly in a framework assessment.
credit hours required for a degree. If a new topic is introduced then some other topic must be diminished or eliminated, possibly jeopardizing the program’s evaluated status as a recognized IT program.
Performing the evaluation yielded an insight into its efficiency, capabilities, accuracy and weaknesses. To perform a complete assessment against the online curriculum took a professor and research assistant under two hours working together. This is a respectable time for a first evaluation and shows the efficiency of the framework.
This implies that the method used to evaluate a program’s status within the field of IT must also change. However, research into the layers model of instructional design provides insights that help understand this process better, and suggests a solution.
The departmental website [33] has been chosen as the only allowable source, since it is publically and easily available (although links to specific course websites are usable, they are not provided from the BYU IT site and thus unusable in this evaluation). Thus as inferred in Section 3, the accuracy of the result is entirely dependent on how reflective the program website is of its component courses.
The original inspiration for layers research is Stewart Brand’s model of how buildings evolve over time [20]. In simple terms buildings can be considered in interacting layers, such as the building site, the structure (walls etc.), the services (plumbing etc.) and “stuff” (furnishings etc.). These layers change and are replaced at different rates. Furnishings are easily and often replaced; the building site is static. This concept has been developed in instructional design by Gibbons and others [21, 22] and Helps [23] has also related it to Structure, Behavior, Function (SBF) analysis, which is used in fields ranging from computing, [24-29] to educational design [30-32].
The primary weakness identified during the evaluation was in correlating the IT Body of Knowledge (BoK) terminology with course syllabi. This necessitated a level of subject knowledge to identify topics and appropriately match these to the BoK. Directly related to this is the difficulty in measuring the depth of a description. For example, the Body of Knowledge includes Object Orientated Programming within the programming fundamentals category. This is specified in some instances as specific language training such as Java or C++ that one must recognize to be OO Programming. In some instances the terminology differs dramatically enough that it becomes difficult to correlate topics for measurement.
The relevance of this research to the current problem is the recognition that different aspects of the Body of Knowledge (BOK) used for identifying IT programs also age at different rates. Specifically the BOK identifies three levels of IT subject matter, identified as “Knowledge Areas,” “Units” and “Topics.” These can be expected to age in descending order, with Topics changing the fastest. In evaluating an IT program it is possible, even likely, that programs may adopt new topics but less likely that a program will change Units and very unlikely that Knowledge Areas will change.
A further issue that was encountered was the need to extract information from course descriptions, outcomes, topical content and learning objectives rather than a single list of topics or course objectives to find appropriate correlations.
The basis for evaluating programs also changes because the underlying definitions of programs evolve in an orderly fashion. There is a process of change built into both ABET criteria and the model curriculum. The same community of academics and professionals in the field maintains both of these. Problems in accrediting new programs or creating new programs from the model curriculum are discussed in this community and changes happen in an on-going process, over a period measured in years. Each of these yardsticks has undergone changes multiple times and can be expected to continue to do so. Therefore basing evaluation of IT programs on an assessment of whether they are addressing the Knowledge Areas and, to a lesser extent, the Units of the BOK, is a valid method of evaluating an IT program within the dynamic field of IT.
Table 2 - BYU IT Program Evaluation Module Type Points Core Elective Available 1 4 6 7 10 11 6 6 5 7 7 7 5 6 5 5 4 6 2 4 7 7 8 1 9 5 1 6 71 2 85 0.84 0.01 1.00 Divide total Divide total by 85 and by 170 and Instructions for Weighted Total Calculation enter above enter above 0.85 Core Coverage (Sum of weighted totals) 1.00 2 Capstone Experience (1 Point per Semester) 2 1 Core content less than 50% of total content 1 IT Compliance Factor (Total above three values) 3.85 4.00 IT2008 Body of Knowledge Core Topic Title IT Fundamentals Human Computer Interaction Information Assurance and Security Information Management Integrative Programming and Techniques Mathematics and Statistics for IT Foundations of Networking Programming Fundamentals Platform Technologies System Administration and Maintenance System Integration and Architecture Social and Professional Issues Web Systems and Technologies TOTAL Weighted Total
It must be further acknowledged that there are other considerations that could be taken into account to evaluate whether a program is within the field of IT. For example teaching approaches, active involvement on IT research, participation in the IT community and involvement with commercial IT enterprises are all valid considerations in developing a multidimensional picture of an IT program. However the major purpose of this paper is to generate a useful metric for identifying IT programs nationwide and worldwide. Therefore these alternative considerations will be left to future research and the current metric will be presented as a simple but reasonable measure of an academic bachelor’s IT program.
A score of 3.85/4.00 indicates excellent compliance to the model curriculum, though with room for improvement. It is obvious to the program faculty that although there are one or two areas where a course may need minor revision, the principal weakness is in the conciseness of the posted curriculum. Thus increased conformance could be easily obtained by adjusting the posted curriculum description to more accurately describe actual course content.
6. EXAMPLE EVALUATION To evaluate the framework, an evaluation was performed against Brigham Young University’s Information Technology program.
126
Undergraduate Degree Programs in Computer Engineering, 2004.
Additionally due to the nature of engineering and technology programs, it is advantageous for evaluators to be familiar with the terminology and context of IT Programs. We noted even within our own program that the terminology used to describe certain curriculum elements could deviate from that used in the Body of Knowledge. This is currently a limitation of the framework, which may be addressed by establishing a cross-referenced vocabulary of terminology. Future work in this area could allow those unfamiliar with the program content to perform an effective evaluation.
[4] Topi, H., Valacich, J. S., Wright, R. T., et al., IS 2010: Curriculum Guidelines for Undergraduate Degree Programs in Information Systems, 2010. [5] LeBlanc, R., Sobel, A., Ben-Menachem, A., et al., Software Engineering 2004: Curriculum Guidelines for Undergraduate Degree Programs in Software Engineering, 2004. [6] Lunt, B. M., Ekstrom, J. J., Gorka, S., et al., Information Technology 2008: Curriculum Guidelines for Undergraduate Degree Programs in Information Technology. Association for Computing Machinery (ACM); IEEE Computer Society, November 2008.
7. CONCLUSIONS In this paper we have proposed a standard framework for identifying and classifying Information Technology programs and apply this to our own program at Brigham Young University. Due to the variation in program titles, it is beneficial to use such a framework to indicate the direct correlation to the existing established curriculum. The methodology provides a numerical value from 0 – 4.0 which allows institutions to evaluate their IT content and directly measure conformity to the model curriculum. The method also provides an insight into areas of possible weakness that, if desired, will facilitate a topical convergence for increased model curriculum adherence.
[7] Leavitt, J. H. and L., W. T., Management in the 1980's, Harvard Business Review, Vol 36, Iss. 6, pp 41-48, 1958. [8] ABET, Criteria for Accrediting Computing Programs, 20112012. [9] Gorka, S., Miller, J. R. and Howe, B. J., Developing realistic capstone projects in conjunction with industry. In Proceedings of the Proceedings of the 8th ACM SIGITE conference on Information technology education (Destin, Florida, USA), ACM.
It was discovered during a trial evaluation that the framework can also help to highlight weaknesses in a published curriculum and may assist institutions in assuring that their stated curriculum accurately reflects on course content. Although by design, this also may be a weakness of the framework in measuring the published curriculum rather than the actual concepts acquired by students in their participation and studies.
[10] Lamancusa, J. S., Zayas, J. L., Soyster, A. L., et al., The Learning Factory: Industry-Partnered Active Learning, Journal of Engineering Education, Iss. January 2008, pp 7, 2008. [11] Jamieson, L. H., Edwardson, J. A. and Lohmann, J. R. L., Creating a Culture for Scholarly and Systematic Innovation in Engineering Education: Ensuring U.S. engineering has the right people with the right talent for a global society. American Society of Engineering Educators (ASEE).
Due to the volatile and evolutionary nature of Information Technology as an academic discipline, a mechanism for evaluating the effectiveness of this framework and providing necessary updates over time is also provided.
[12] Kolb, D. A., Experiential learning : experience as the source of learning and development. Prentice-Hall, Englewood Cliffs, N.J.
We note however that the framework is not without its flaws and work should be continued to establish an accepted vocabulary for terms used within the Body of Knowledge. Additionally, to ensure that any evaluation is demonstrable, the framework stipulates that an externally facing program website is used. By necessity, this is a second limitation of the framework in that it relies entirely on the accuracy of an institution’s website. We believe that this latter limitation may well be advantageous to programs in locating and rectify discrepancies between their taught content and program websites.
[13] Lindsey, L. and Berger, N., Experiential Approach to Instruction. Routledge. 2009. [14] Brock, D. C. and Moore, G. E., Understanding Moore's law : four decades of innovation. Chemical Heritage Foundation, Philadelphia, Pa. [15] Molebash, P. E., Tomorrow May Bring: Trends in Technology and Education. 2000.
We hope that the IT education community and students alike will benefit from this framework by simply, yet effectively identifying information technology programs both in the United States, and internationally.
[16] Mollick, E., Establishing Moore's Law, Annals of the History of Computing, IEEE, Vol 28, Iss. 3, pp 62-75, 2006.
8. REFERENCES
[18] Schaller, R. R., Moore's Law: Past, Present and Future, IEEE Spectrum, Vol 34, Iss. 6, pp 52-59, 1997.
[17] Roberts, L. G., Beyond Moore's law: Internet growth trends, Computer, Vol 33, Iss. 1, pp 117-119, 2000.
[1] Lunt, B. M. and Ekstrom, J. J., The IT Model Curriculum: A Status Update, SIGITE '08 Proceedings of the 9th ACM SIGITE Conference on Information Technology Education, 2008.
[19] Moore, G. E., Cramming more components onto integrated circuits, Electronics, Vol 38, Iss. 8, 1965. [20] Brand, S., How Buildings Learn: What Happens After They're Built. Viking Penguin, London.
[2] Cassel, L., Clements, A., Davies, G., et al., Computer Science Curriculum 2008: An Interim Revision of CS2001, 2008.
[21] Gibbons, A. S., McConkie, M., Seo, K. K., et al., Theory for the Design of Instructional Simulations and Microworlds. Lawrence Erlbaum Associates. 2009.
[3] Soldan, D., Aylor, J., Clements, A., et al., Computer Engineering 2004: Curriculum Guidelines for
127
[22] Reigeluth, C. M. and Carr-Chellman, A. A., Instructionaldesign theories and models: Building a Common Knowledge Base (Vol III). Routledge: Taylor and Francis. 2009.
[28] Kruchten, P., Casting Software Design in the FunctionBehavior-Structure Framework, IEEE SOFTWARE, Iss. March/April 2005, pp 52-58, 2005.
[23] Helps, C. R. G., Evolving Information Technology: A Case Study of the Effects of Constant Change on Information Technology Instructional Design Architecture. BYU, Provo.
[29] Schank, R. C. and Abelson, R. P., Scripts, Plans, Goals, and Understanding: An Inquiry Into Human Knowledge Structures. Psychology Press.
[24] Clark, D. D., Mistree, F., Rosen, D. W., et al., FunctionBehavior-Structure: A Model for Decision-Based Product Realization. In Proceedings of the ASEE Annual Conference & Exposition, (Milwaukee), ASEE, June 1997.
[30] Collins, A. and Ferguson, W., Epistemic forms and Epistemic Games: Structures and Strategies to Guide Inquiry, Educational Psychologist, Vol 28, Iss. 1, pp 25-42, 1993.
[25] Gero, J. S., Design prototypes: a knowledge representation schema for design, AI Mag., Vol 11, Iss. 4, pp 26-36, 1990.
[31] Hmelo-Silver, C. E., Holton, D. L. and Kolodner, J. L., Designing to Learn About Complex Systems, Journal of the Learning Sciences, Vol 9, Iss. 3, pp 247 - 298, 2000.
[26] Goel, A. K. and Chandrasekaran, B., Functional representation of designs and redesign problem solving. In Proceedings of the Proceedings of the 11th international joint conference on Artificial intelligence - Volume 2 (Detroit, Michigan), Morgan Kaufmann Publishers Inc.
[32] Hmelo-Silver, C. E. and Pfeffer, M. G., Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions Cognitive Science, Vol 28, Iss. 1, pp 127-138, 2004. [33] Ekstrom, J. J., Helps, R., Lunt, B. M., et al., BYU IT Website. it.et.byu.edu (Last Accessed: May 27, 2011).
[27] Goel, A. K., Rugaber, S. and Vattam, S., Structure, behavior, and function of complex systems: The structure, behavior, and function modeling language, Artificial Intelligence for Engineering Design, Analysis and Manufacturing, Vol 23, pp 23-35, 2009.
128
