On Evaluating the Use of Zachman Framework in Computer Science ...

On Evaluating the Use of Zachman Framework in Computer Science and Information Systems Classes Murali Mani, Suleyman Uludag University of Michigan, Flint {uludag,mmani}@umich.edu

Christopher Zolinski Delta College [email protected]

Abstract Enterprise Architecture (EA) is concerned with how an organization’s assets are to be utilized to obtain a set of desired information technology outcomes. The Zachman Framework (ZF) has started the studies in EA. ZF provides an effective meta-model for large scale information systems by means of a set of architectures. ZF has essentially become an indispensable component of any Information Systems (IS) related field. While ZF has been pervasively used in the private and public sectors, to the best of our knowledge, it has found almost no place, other than only one limited application, in computing classrooms, as reported in the literature. In this paper, we report the first proposal to make use of the ZF in many Computer Science (CS) and IS courses. We describe how we have incorporated it in a database class at a 4-year institution and forensics and network intrusion classes at a community college. We present assessment results showing promising results for improved student learning based student self-reported responses. Pedagogically, our application is grounded in the Gagnes Conditions of Learning, the Instructional Design Theory, and a goal-oriented methodology.

1

Introduction

The design, analysis, selection, deployment and maintenance of enterprise information technology solutions are the foundations of the field of Enterprise Architecture (EA) in the area of Information Systems (IS). Its knowledge base is also used in Software Engineering and Computer Science (CS) classes. The two professional societies for IS, ACM (Association for Computing Machinery) and AIS (Association for Information Systems), include EA as part of the core curriculum for IS undergraduate programs in their latest curricula recommendations. Zachman framework, published in 1987 [12, 8, 6], laid the groundwork for the field that has evolved over time to what we now know as EA. Unlike the step-by-step prescriptive, more traditional methodology of Software Development Life Cycle (SDLC) [1], Zachman framework provides an effective meta-model or framework for large scale information systems by means of a set of architectures. Zachman framework has taken its inspiration from the other disciplines, mostly architecture and engineering. One of its most important distinguishing factors lays in its recognition of the different players, and viewpoints thereof, when developing architectural abstractions. The architectural representations are depicted in a matrix of 36 cells with perspectives as rows and the descriptions as columns. Zachman framework has essentially become an indispensable component of any IS related field. There are other, competing frameworks for EA, namely, the Open Group Architecture Framework (TOGAF) [5], Federal Enterprise Architecture Framework (FEAF) [9], and The Department of Defense Architecture Framework (DoDAF) [10], but these are not as widely used. The Zachman framework has been used in a wide variety of settings, including commercial companies, government agencies and to describe standards. For example, the US Department of Veteran Affairs has used the Zachman framework to develop and maintain its One-VA enterprise architecture. Within an organization, Zachman framework has been applied to the entire organization, to various departments and even to basic operational entities. While Zachman framework has been pervasively used in the industry, to the best of our knowledge, it has found almost no place, other than only one limited application, in the CS and IS classrooms, as reported in the literature. The aforementioned paper [11] mentions its use in teaching IS at a business school; however, details on how Zachman framework were incorporated are not provided, and no data is reported on the effectiveness of the approach. In this paper, we report the first proposal to make use of the Zachman Framework in many CS and IS courses. We describe how we have incorporated Zachman framework in our courses across two institutions; a database class at a 4-year institution and forensics and network intrusion classes at a community college. Further, we will present survey results and assessment data of its usage in these classes that show promising results to enhance the benefits of students from the assignments and to improve the overall learning experience.

1

From a pedagogical perspective, we postulate that our application of Zachman Framework is grounded in the following theoretical underpinnings: 1. Providing a holistic view of the assignments as used in the methodology of Socratic questioning, 2. Informing the learners explicitly of the objective in the process, as outlined in Gagne’s Conditions of Learning, 3. Highlighting the end goal of the instruction from the Instructional Design Theory, 4. Transforming the format of the instructional materials from the Cognitive Load Theory, 5. Establishing the relevance by means of the goal-oriented methodology. The rest of this paper is organized as follows: Section 2 delves into the details of the Zachman Framework. Related work is summarized in 3. Pedagogical and theoretical underpinnings of our approach are detailed in 4. Section 5 provides information about the colleges and the courses where Zachman framework was used in order to assess the efficacy of our approach. Evaluation details are presented in Section 6 Finally, concluding remarks and synopsis of future work are given in Section 7.

2

Zachman Framework

Figure 1: The Zachman Framework for Enterprise Architecture v3.0 as published at zachman.com

2

Zachman framework essentially founded the field of enterprise architecture when its seminal paper [12] was published in 1987. The field has evolved greatly since then with many other frameworks proposed. Zachman framework itself has gone through many iterations. The latest version of the Zachman Framework’s, as published by its developer at its Web site, is shown in Figure 1. There are six different audience perspectives described in Zachman framework as shown on the left side of Figure 1: the perspective of business context planners (the executive perspective), the perspective of business concept owners (the business management perspective), the perspective of business logic designer (the architect perspective), the perspective of business physics builders (the engineer perspective), the perspective of business component implementers (the technician perspective), and the perspective of the user (the enterprise perspective). The models that these different players deal with are shown on the right side of Figure 1. They include scope identification lists, business definition models, system representation models, technology specification models, tool configuration models, and the implementation models. Each of these players have six different categories of questions as shown at the top of Figure 1: what, how, where, who, when, and why. The enterprise names for these aspects are shown at the bottom of Figure 1: inventory sets, process flows, distribution networks, responsibility assignments, timing cycles and motivation intentions. Let us examine what will go into the various cells of the Zachman framework in Figure 1. The what column talks about how the business is viewed by different players: the executives consider what the business is about as viewed by the public, the business owner might be concerned about the various products and services offered by the business, the architect might be concerned about his/her particular project, the engineer might be concerned about the engineering aspects of his/her project, the technician might be concerned about the components in his/her project, and the users might be concerned about the day-to-day operations of the functioning enterprise. Similarly the how column refers to how the business operates, how the project will be implemented and how the users will interact with the enterprise; the where column refers to where the business resources are located, and where the projects will be implemented; the who refers to who will be developing the different aspects of the projects; the when refers to the timeline for project development, and why refers to the motivation of the business and projects as seen by the different players.

3

Related Work

To the best of our knowledge, the use of Zachman framework in the classroom is almost non-existent based on our literature review. In [2], Zachman framework has been used as part of an Enterprise Resource Planning (ERP) course lab experiment. Zachman framework is used as a component for the roles collaboration to design the lab experiment. This is different than our usage; ours is orthogonal to the actual content material of the course and more focused on the process of the whole homework experience. Another, more closely related one, is reported in [11], where Zachman framework is used as a set of artifacts for describing information systems. This usage is confined to information system courses and no assessment is reported. We propose it for all computing classes and provide a preliminary assessment from three different classes at a 4-year university and a community college.

4

Pedagogical Background

We have observed that student classroom work is mostly ”cook-book” style (i.e. didactic) and there is usually a disconnect from the real-world settings to reduce the clutter and the unstructured settings. Topics and problems are many times studied and solutions devised in a sanitized form with connections to the big picture and the real world interactions and consequences are not usually emphasized. Especially in computing classes, students easily lose the big picture, or even, never get a chance to become aware of it. Itty-bitty details of the assignments overwhelm the students and relevance, end goal, and overall point of the whole homework may never come into daylight. In this respect, we believe Zachman framework may connect the dots for the students in providing a more holistic view of the road ahead and goals sought in the assignments. We base our pedagogical reasoning for the aforementioned claim on the following five theoretical pillars: 1. Gagne’s conditions of learning develops a theory about different levels or types of learning each of which requires a differentiated type of instruction. In addition, he introduces the nine instructional event model [3]. One of them is ”Informing the learner of the objective”. From this perspective, Zachman framework enables the learner to organize their thinking process around what they are about to get involved to increase relevancy and thus benefit. 2. The theory of Instructional System Design (ISD) [7], closely associated with Gagne, attempts to elevate the instructional experience to more effective and appealing grounds by studying the learners and their needs carefully. Our 3

use of the Zachman framework shares the same end goal of the theory of ISD. 3. One of the four steps to develop and retain motivation in the learning process is relevance, according to Keller’s ARCS Model of Motivational Design [4]. Zachman framework provides the relevance strongly. Three major strategies of the ARCS Model are goal-oriented, motive matching, and familiarity. End-goal is emphasized in the Zachman framework.

5

Colleges and Courses

We implemented the Zachman framework in in one course at the U. of Michigan-Flint and two courses at Delta College, Michigan. The U. of Michigan-Flint is one of the two regional campuses of the University of Michigan system. The university offers an undergraduate degree in computer science, an undergraduate degree in computer information systems and a master’s degree in CS and IS. The CIS/CSC 384 course at the U. of Michigan-Flint is a course on database systems, which exposes the students to relational database design as well as SQL (Structured Query Language). It is a required course for computer information systems (CIS) majors; however, it is not required for computer science (CSC) majors. The different learning strategies used as part of this course include in-class hands-on exercises, about 6 homework assignments, a project divided over 3 phases, and 2 exams, in addition to lectures. For the CIS/CSC 384 offering in Winter semester 2014, we had 30 students enrolled. The Zachman framework was used for the project. Students can do the project either by themselves or in groups of size up to 4. As part of the project, the students have to come up with a database application (for instance, a movie database or a database of religions and customs in different parts of the world). The students choose their own application, the requirement being that it should be at least of a certain complexity (specified in terms of number of entity sets and relationships in the database design). Once the students have chosen their application, they design the Entity-relationship (ER) schema for the database (phase I), translate the ER schema into SQL tables and implement the tables in a database server and populate the tables with data (phase II), and come up with scenarios how users will interact with the application and come up with SQL select, insert, delete and update statements for each of the scenarios (phase III). Each group needs to demonstrate each phase of their project to the instructor. Almost all the demonstrations are done in person; in the rare case that some students cannot present their project in person, they can talk to the instructor over the phone. In-class presentations of the projects (mainly phase I and phase III) are also encouraged time permitting. In Winter semester of 2014, we added the Zachman framework as part of the project. The students were first given an orientation on the Zachman framework. Here the instructor introduced the Zachman framework, how it is used. No examples of Zachman framework applied for a project were given. The students were asked to work on the Zachman framework for their project starting from phase 1. The students’ work on the Zachman framework were discussed as part of their demonstration for phase 1. Then students were asked to revise and resubmit the Zachman framework for the second and third phases of the project as well. The CST-267 (Introduction to Forensics) course at Delta College is a forensics course, which exposes the students to forensic techniques and skills to implement them in a project environment to recover data using FRED (Forensic Recovery Electronic Devices) . It is a required course for CIS majors; but not for CSC majors. For learning, we have in-class hands-on exercises, about 12 take-home assignments, a project , and two exams, in addition to lectures. In Winter semester of 2014, five students were registered for the course. The Zachman Framework was used for the project. Students were assigned to do this project individually over the course of 15 weeks. The Students were to use the Zachman framework to design a forensic recovery plan for retrieving data. Upon identifying stakeholders and others, involved in a project, they develop a recovery method for their recovery project. The students were first given an orientation on the Zachman framework. Here the instructor introduced the Zachman framework, how it would be used as part of the project. then the students used the framework in their project for the semester. The CST-158 (Network Intrusion) course at Delta College is a Security course, which exposes the students to security auditing design as well as security planning. It is a required course for CIS security majors but not for CSC majors. We have in-class hands-on exercises, about 12 take-home assignments, a project divided over three phases (similar to the CIS/CSC 384 course at U. of Michigan-Flint), and two exams, in addition to lectures. There were 10 students in this class in the Winter 2014 semester. The Zachman framework was used for the security audit project. Students completed the project themselves. As part of the project, the students have to come up with a security audit. The students completed the individual assessment of the company of their choosing using the Zachman framework and other security auditing techniques discussed in the class lectures. Once the students have chosen business to audit (phase I), they must discuss the types of security being

4

audited (phase II), and complete the audit using all available tools and frameworks (phase III). Similar to the CIS/CSC 384 course at U. of Michigan-Flint, each person needs to demonstrate each phase of their project with the instructor. Almost all the demonstrations are in person; in the rare case that some student cannot present it in person, they can talk to the instructor over the phone or via email. Just like in the other courses, the students were first given an orientation on the Zachman framework. The students were asked to work on the Zachman framework for their project starting from phase 1. Students were asked to revise and resubmit the Zachman framework for the second and third phases of the project as well.

6

Assessment Results

6.1

Survey Description

We had 18 questions in our survey questionnaire, ranging from the student’s experience and perceived usefulness of the Zachman framework for the course and for other computing courses to the student’s prior exposure to Zachman framework Q01., Overall, I found the Zachman Framework useful. Q02. , The Zachman Framework allowed me to understand my homework more thoroughly. Q03. , I understood the Zachman Framework clearly. Q04., I have had previous experience with or exposure to the Zachman Framework. Q05., I have used other similar frameworks previously. Q06., My experience with the Zachman Framework was positive. Q07.. The Zachman Framework made my assignments more straightforward. Q08., The Zachman Framework triggered a more rewarding critical thinking from the assignment. Q09., I would recommend the Zachman Framework be used in future semesters of this course. Q10., The Zachman Framework would contribute positively to other computing classes. Q11., I have discussed the Zachman Framework with a classmate or coworker. Q12., I have implemented or plan to implement the Zachman Framework in my academic, professional, or personal activities. Q13., I find documenting projects or assignments easier as a result of exposure to and experience with the Zachman Framework. Q14.. The Zachman Framework facilitates more explicit perception of the objectives in the assignment. Q15., The end goal of the instruction becomes more clearly manifest in the assignment as a result of using the Zachman framework. Q16., I would benefit less in this assignment if the assignment was provided without the Zachman Framework. Q17., I was able to establish the overall relevance of the assignment in my academic professional development. Q18., The Zachman Framework uses a goal-oriented methodology. Table 1: Survey questions given to the students after using the Zachman Framework in the assignments. and/or similar frameworks. Table 1 lists the questions in the survey questionnaire. The students were asked to complete the survey anonymously after completion of their assignments where they used Zachman framework. For the courses at the U. of Michigan-Flint, the students completed a qualtrics survey online, and for the courses at Delta community college, the students completed the survey in paper after their test. For all the courses, the students were given extra credit for completing the survey, and the individual student responses were kept anonymous. However, the students who completed the survey were recorded with no way of tracing which survey belonged to which student so as to give the extra credit.

6.2

Evaluation of the Survey Results

Figures 2 and 3 display the histograms of the survey questions for the class at U. of Michigan-Flint, and for the two courses at Delta community college, respectively. See that we combined the two courses at Delta community college. Here, we show the number of students who responded with each value (5 = Strongly Agree to 1 = Strongly Disagree) for each of the survey questions. The student responses are shown as error plots with one standard deviation in Figure 4 (U. of Michigan-Flint) and in Figure 5 (Delta Community College). We see that overall, almost all students across all courses felt that Zachman framework uses a goal oriented methodology (82% at U. of Michigan-Flint reported agree or strongly agree for Q18 with a mean of 3.88, and 94% at Delta Community College reported agree or strongly agree for Q18 with a mean of 4.69). They also find that Zachman framework is useful 5

Figure 2: Number of responses in each category for each of the 18 questions as a bar plot for the courses at U. of Michigan-Flint

Figure 3: Number of responses in each category for each of the 18 questions as a bar plot for the courses at Delta Community College

(at U. of Michigan-Flint 53% reported agree or strongly agree for Q01, with a mean of 3.12; at Delta Community College, 94% reported agree or strongly agree for Q01, with a mean of 4.63) and should be used in future computing classes (at U. of Michigan-Flint 53% reported agree or strongly agree for Q10, with a mean of 3.35; at Delta Community College, 94% reported agree or strongly agree for Q10, with a mean of 4.69). However, there is a marked difference in responses between the courses at U. of Michigan-Flint and courses at Delta Community College. We believe that this could be due to several reasons. More students at Delta Community College have been exposed to Zachman framework and/or similar frameworks previously (mean of 1.65 for students at U. of Michigan-Flint with 83% reporting disagree or strongly disagree .vs. mean of 2.63 for students at Delta Community College with 57% reporting disagree or strongly disagree for Q04). Also there is a significant difference in responses to Q03 about their level of understanding of the Zachman framework (mean of 2.88 among students at U. of Michigan-Flint with 41% reporting agree or strongly agree and 47% reporting disagree or strongly disagree.vs. 4.56 among students at Delta Community College with 94% reporting agree or strongly agree and 0% reporting disagree or strongly disagree). This could be because Zachman framework need more practice to learn; further, the instructor at U. of Michigan-Flint was exposed to the Zachman framework for the first time this semester. The instructor at U. of Michigan-Flint believes that with clearer explanations of the Zachman framework and with more examples, the students will be more confident about their understanding. The instructor at Delta Community College has used Zachman framework in his prior projects, and has also incorporated it in other courses at Delta Community College prior to this semester. Another factor is that the Zachman framework is not directly related to the technical aspects of the project for the courses at U. of Michigan-Flint used in this study, and the students are graded solely on the technical aspects of the project. The benefit to the students towards the course grade was the extra credit points for completing the survey. However, we believe that with better delivery of the Zachman framework with better examples, and exposing students to Zachman framework in courses such 6

Figure 4: Error Plot for the courses at U. of Michigan-Flint, showing one standard deviation. 5 = Strongly Agree, 4 = Agree, 3 = Neutral, 2 = Agree, 1 = Strongly Agree.

Figure 5: Error Plot for the courses at Delta College, showing one standard deviation. 5 = Strongly Agree, 4 = Agree, 3 = Neutral, 2 = Agree, 1 = Strongly Agree.

as software engineering and information systems would yield better numbers. One important feature that we observed across all courses is that students interpreted various aspects of Zachman framework differently. For instance, at U. of Michigan-Flint, the ”how” aspect of the Zachman framework was interpreted by some students as how users will interact with the database, and by some students as how the database will be developed. The different interpretations of the ”where” aspect included: where the project will be developed, where the users will access the database from, and where resources are located. Different interpretations of the ”time” aspect included: when can users access the database, and what is the timeline for the project development. Different interpretations of the ”who” aspect included: who can access the database, who will develop the database, and who is grading the course project (the answer being instructor). In the forensics at Delta Community College, the ”how” aspect interpretations included: how will users recover data, and the data recovery methodology used. The interpretations of ”where” aspect included: where the project will be developed, where will users access the hardware from, and where resources are located. For the course on network intrusion at Delta Community College, the interpretations of ”where” included: where the project will be developed, where will users access the security plans and policies from, and where the resources are located. The different interpretations of ”time” aspect included: when can users access the policies and modify them, and what is the timeline for project development. Different interpretations of ”who” included: who can access the audit, who will develop the policies post-audit, and who will grade the project (the answer being instructor).

7

7

Conclusion and Future Work

Since its inception in 1987, Zachman Framework (ZF) has spun off an interesting new subfield of study in computing, called Enterprise Architecture (EA). EA has become an inherent component of any information technology and systems projects in order to mesh the investment efforts and assets with those of the overall strategic and technical goals of the organizations. While ZF’s elegance and utility are greatly appreciated, as self-evident in its widespread deployment, its educational benefits have not been exploited yet. We believe there has been a missed opportunity to tap into the potential educational benefits of the ZF so far. We propose to introduce the ZF to the computing classroom in order to have similar positive outcomes to the industry implementations. In this paper, we have applied ZF in three different courses; a database class at a 4-year institution and forensics and network intrusion classes at a community college. We have conducted student surveys to evaluate the efficacy of the ZF and presented the results. Our results of student self-reported responses indicate that student positively benefit from exposure to ZF as part of their assignments, homeworks or projects. We are in the process of expanding our experiments to other computing classes (both IS and CS) while developing a more structured orientation material and sessions as our results in this experiment indicate that some students would have benefited with more in depth coverage of the ZF.

References [1] J. Broad. Chapter 5 - System Development Life Cycle (SDLC). In J. Broad, editor, Risk Management Framework, pages 39 – 45. Syngress, Boston, 2013. [2] Q. Deng, A. Yin, and B. Tu. Design and application of practical teaching framework of erp course. In Computer Science Education, 2009. ICCSE ’09. 4th International Conference on, pages 1346–1348, 2009. [3] R. M. Gagne and L. J. Briggs. Principles of Instructional Design. Holt,Rinehart & Winston of Canada Ltd; 2nd edition, January 1979. [4] J. Keller. Motivational Design for Learning and Performance: The ARCS Model Approach. Springer, 2009. [5] V. H. Publishing and A. Josey. TOGAF Version 9: A Pocket Guide. Togaf Series. Van Haren Publishing, 2009. [6] B. Robertson-Dunn. Beyond the zachman framework: Problem-oriented system architecture. IBM J. Res. Dev., 56(5):541–549, Sept. 2012. [7] W. Rothwell and H. Kazanas. Mastering the Instructional Design Process: A Systematic Approach. Pfeiffer essential resources for training and HR professionals. Wiley, 2011. [8] J. F. Sowa and J. A. Zachman. Extending and formalizing the framework for information systems architecture. IBM Syst. J., 31(3):590–616, June 1992. [9] The Chief Information Officers Council. Federal Enterprise Architecture Framework (FEAF), September 1999. accessed April 27, 2014. [10] The Department of Defense Deputy Chief Information Officers. The Department of Defense Architecture Framework (DoDAF) Version 2.02, August 2010. [11] M. Varga. Zachman framework in teaching information systems. In International Conference on Information Technology Interfaces, pages 161–166, 2003. [12] J. A. Zachman. A framework for information systems architecture. IBM Syst. J., 26(3):276–292, Sept. 1987.

8