Technology-based learning - Cultural Dimensions and Considerations

2nd International Conference on Enterprise Systems and Accounting, ICESAcc, 2005, Thessaloniki, Greece, 11-12 July, 2005

Enhancing Organisational Effectiveness, Achieving Excellence and Competitive Advantage in Higher Education by Balanced ScoreCard; Kerstin V. Siakas, Aristea-Alexandra Prigkou, Stergios Draganidis Technological Educational Institute of Thessaloniki, Department of Informatics, P.O. Box 141, GR-57400 Thessaloniki, Greece E-mail: [email protected], [email protected], [email protected]

Abstract Living in today’s new economic era, the European organisations face the challenge of quality having demonstrated their enthusiasm for it. In order to ensure the quality, to enhance the organisational effectiveness and to achieve excellence the organisations want to develop new ideas and methods as well as to be adapted to innovative strategies. Within the frame of the Bologna Process, the necessity to ensure the quality in higher education was pointed out and a process for its achievement was imposed. The higher educational institutes should be able not only to reconsider their way of operating and thinking, but also to develop further faculties for achieving quality and perfection in their processes as well as conformity in their services. An important objective of the Bologna Process is to move higher education into a more transparent environment and to place the diversified national educational system into a common frame. According to the new European challenge, the department of Informatics at the Technological Educational Institute of Thessaloniki, Greece, is in the process of realising research for the functions in the department and the quality of the education offered. The main purpose is not only to meet the European challenge but also to achieve a competitive advantage and excellence. For the implementation of this process and the realisation of the goals a software tool based on the Balanced ScoreCard methodology, which provides an integrated procedure based on targets and measures of progress, will be utilised. The main objective of this paper is to define the critical success factors / key performance indicators for achieving the strategic goals regarding the Customer (student) Perspective of the Balanced ScoreCard methodology for the department of Informatics at the Technological Educational Institute of Thessaloniki in Greece.


1 Introduction Educational institutions should focus not only on preparing working adolescents for their professional development but also to help them to develop their capabilities in order to be able to function more effectively within their families and communities, as well as in the workplace. The institutions have to be clear about whom they serve and how they serve, and they should measure their results to determine how well they deliver on their promises. In order to support this new way of productive thinking, the vision needs to be clear and shared by the entire institution, so that everyone commits by providing measures and improving their personal procedures In order to move toward the vision, it’s important to be clear on the current state of the institution, so that the gap between reality and vision provides a compelling impetus for change [Kaplan and Norton, 1996a; 1996b]. Human, financial and physical infrastructure, together with technological resources and well-defined processes are necessary in order to realise the goals. The adequacy of the infrastructure, including the technology necessary to make change happen, has to be ensured. Management commitment, transparency and continuous Improvement and development are also basic requirements [Deming, 1986]. New sources of revenue tend to open up with increased clarity about whom we are serving and how. Subsequently, educational institutions need to assess activities in order to provide a balance of tangible and intangible assets, and to measure future capability as well as performance. In order to achieve the desired balance, educational institutions need to be supported in the political and business area through appropriations, tuition support, technological assistance, and the like.

2 The European Higher Education Area; Vision into Action through the Balanced ScoreCard The Bologna Declaration [Berlin Communiqué, 2003] is the foundation for establishing a coherent and cohesive European Higher Education Area by 2010 and for promoting the European system of higher education worldwide. The ministers in charge of higher education of 33 European countries agreed on joined objectives for implementing the European educational policy at national and institutional levels. The ministers stressed the need to develop mutually shared criteria and methodologies and agreed that by May 2005, national quality assurance systems should include: A definition of the responsibilities of the bodies and institutions involved, evaluation of programmes or institutions, including internal assessment, external review, participation of students and the publication of results and finally a system of accreditation, certification or comparable procedures, international participation, co-operation and networking. In order to meet the challenge regarding the European Higher Education Area educational institutions are in the process of trying to adapt their strategies to the educational policy. The national policy can be interpreted and analysed by using terms of strategic planning at institutional level [Kettunen, 2003]. The Balanced


ScoreCard (BSC) methodology can be used as a general framework to communicate and implement institutional strategy. The Balanced ScoreCard creates a shared understanding of the strategy, which is necessary in order to implement the strategy. The Bologna Process authorises each institution to plan and implement its own strategy consistent with the European and national policy. Within this frame the department of Informatics at the Technological Educational Institute of Thessaloniki, Greece has embarked on a pilot study for the assessment of the department with emphasis on the satisfaction of the curriculum aiming to the creation of a quality assurance system and continuous improvement. Strategic management is used in order to develop an organisation from its present situation into a more desirable future situation. A holistic and shared understanding of how an organisation adapts to its environment and educational policy is required in order to develop activities for an improved future [Bush and Coleman, 2000]. In order to understand the relationship of strategic intents in an organisation figure 1 is presented.

Most Integrative

Fewest in Number

Vision Mission Goals Objectives Plans Most Specific

Greatest in Number

Figure 1: Hierarchy of Strategic Intent (adopted from Miller, 1998)

In strategic management the vision refers to mental images of the future, which become tangible in the form of mission statements. The mission statements of an organisation define its primary purpose and articulate the responsibilities of the organisation to its stakeholders. The goals are attempts to improve the organisation’s performance by making mission statements more concrete. Objectives represent the operational definitions of goals in more precise terms and describe what needs to be accomplished in order to reach the goals. Plans are developed usually by managers to help accomplish higher-level intentions [Miller, 1998].


3 The Balanced ScoreCard Methodology – A tool for Achieving Excellence and Competitive Advantage The Balanced ScoreCard is a management system that enables organisations to clarify their vision and strategy and translate them into action [Kaplan and Norton, 1996a; 1996b]. It provides feedback around both the internal business processes and external outcomes in order to continuously improve strategic performance and results. When fully deployed, the Balanced ScoreCard transforms strategic planning from an academic exercise into the nerve centre of an organisation [web-page 1]. Knowing that quality and performance are measurable entities, it can be said that the management tool Balanced ScoreCard could be their dynamic measuring system [Kaplan and Norton, 1992]. The traditional financial measurements are today considered inadequate for guiding and evaluating an organisation that desires to create future value through investment in customers, suppliers, employees, processes, technology and innovation. Instead, quantifying entities like customers’ satisfaction, performance in all sectors, process and product quality etc is considered valuable for pointing out the organisation’s weaknesses and for applying a strategic plan for their improvement, as well as for increasing the profit and assuring transparency and efficiency in every aspect. These measurements influence essentially the human behaviour inside and outside the organisation [Kaplan, 2001]. The Balanced ScoreCard retains all these measurements and indicates the story of past events in order to sustain an adequate historical archive. This is a critical success criterion for the organisations that emphasise investments in long-term capabilities and customer relationships. Balanced ScoreCard suggests that each organisation is divided into four parts that all together indicates its complete image. If these parts can operate separately, the organisation will operate also properly. These parts are called “perspectives” and include the Financial, the Customer, the Internal Process and the Learning and Growth Perspective [Kaplan and Norton, 1993; 1994]. Based on these four perspectives the organisation develop metrics, collect data and analyse them in order to obtain a holistic view of its function and determine the required future improvement. Below a short description of each perspective is provided [Web-page 2]. Financial Perspective: Addresses the question of how shareholders view the organisation, which financial goals are desired and what is the current financial situation. This perspective is easily identified because the data are already quantified and usually existing in different databases. Internal Process Perspective: Addresses the question of which processes are most critical for satisfying both customers and shareholders. It contains the internal procedures of the organisation and how well the organisation operates. Learning and Growth Perspective: Addresses the question of how the employees must learn, improve and innovate their knowledge in order to realise the desired goals and maintain the organisation’s competitiveness.


Customer Perspective: Addresses the question of how customers view the organisation. It is of high importance for an organisation to be able to meet or exceed customer expectations, and imperative to fully understand those expectations in order to find a way to adjust them into reality. Otherwise, customers will eventually find other organisations that will meet their needs. The centre of attention in management philosophy has emphasised the importance of focusing on customer satisfaction in every aspect of business. The customer perspective is very important and in case of its poor performance a future decline of the organisation will be observed even though the current financial picture may appear good. It is also said that the average satisfied customer will tell four other people about their experience, while the typical unsatisfied customer will tell 14 [Wood, 1999]. The BSC highlights the strategy and brings together the above key indicators for measuring, comparing and achieving its goals. The graphical environment enables the whole organisation to understand the linkages between measures and objectives and helps them to focus on efforts. The visual environment enables cross-level communication between stakeholders (everyone in the organisation or unit, and their customers, funding providers, suppliers, the community etc.) and helps them to understand the objectives and how well they are achieved [Siakas et. al. 2005]. Figure 2 shows the relationship between the different perspectives and how they are related to the vision and strategy.

Figure 2: The Balanced ScoreCard Perspectives (adopted from Kaplan and Norton, 1996a)


The Balanced ScoreCard harmonises financial measures of past performance with measures and targets for future performance. The objectives and the measures derive from the organisation’s vision and strategy. After analysing the results, the initiatives are taken according to the desired objectives. In education the Financial Perspective includes funding from central government and external funding. The government funding is mainly used for education, whilst the external funding is used for research, development and life-long learning. The Internal Processes Perspective includes departmental procedures, support and learning processes. The Learning and Growth Perspective includes knowledge creation through research, development and continuous updating. The Customer Perspective includes student and employee satisfaction [Kettunen, 2002]. This perspective will be analysed in more detail below.

4 Key Performance Indicators for the Department of Informatics Key Performance Indicators, also known as KPI, help an organisation to define and measure progress toward organisational goals. A holistic understanding of dynamics is required in order to design the key performance indicators for all four perspectives, to define the targets, develop metrics, and to assess the actual performance against the targets by collecting data and analysing the results. We analysed each perspective by using the Goal-Question-Metric (GQM) methodology [Basili, 1992; 1995], in which goals/objectives are determined according and subsequently questions emanate to clarify the goals/objectives. One of the goals in our department is graduates’ satisfaction. The question that emanate is “how satisfied are the graduates?”. In order to answer this question we need to define appropriate measures revealing the level of their satisfaction. The measures arriving are graduates’ professional establishment, value (usefulness) of acquired knowledge during studies and postgraduate possibilities. These measures may unfold new questions, such as “How do we measure acquired knowledge?”. This is a spiral process that goes on until feasible satisfactory measures are found to clarify our goals. The GQM methodology, together with a literature investigation of how other educational departments have designed and implemented BSC in higher education, helped us to understand our goals, to define the key performance factors needed to indicate our strategic direction [Siakas et. al. 2005]. Targets values for the key performance indicators will be decided later. The main objective of this paper is to define the critical success factors / key performance indicators for achieving the strategic goals regarding the Customer (student) Perspective of the department of Informatics at the Technological Educational Institute of Thessaloniki in Greece. The reason we concentrated on the Customer (student) perspective is that we are a non-profitable organisation and our main mission is to offer knowledge to students. The financial, internal processes and learning & growth perspectives are equally important, but with Total Quality Management (Deming, 1986) in mind, which emphasises customer satisfaction, we decided to start with the Customer (student) perspective.


For identifying the key performance indicators for the Customer (student) Perspective, the following goals were determined: High educational level of students, satisfied students, satisfied graduates and good customer image of the department. The questions that stemmed from the goals were used to define the metrics for identifying the current situation. The analysis of the results will be the ground for designing the demanded strategy and for the appropriate actions to be taken in order to achieve our goals. For the collection of data we used questionnaires fulfilled by students (229), by lecturers (15) and by employers that receive students for industrial placements (60) and different existing databases in the department (student and industrial placement registers, Socrates/Erasmus and educational staff databases). Every emanated question was thoroughly examined to find out the kind of data and the method of data mining (questionnaire, database etc). Below an overview of our analysis of the key performance indicators is presented: In order to understand the educational level of students, the students records will be used for defining students’ profile, including nationality, origin, gender, age, entry profile (grades and type of previous education) and current performance (choice of courses, course grades, attendance, projects, industrial placement, graduation degree and rate of students abandoning studies). In order to quantify students’ satisfaction of their studies, questionnaires and interviews will be used regarding courses, industrial placement and final year project, as well as teaching conditions, adequate infrastructure (building, equipment, network, educational material (notes, books and software)), on-line support, educational personnel support, secretarial service and educational and departmental processes. An important part of the students’ satisfaction depends on the teaching ability of educational personnel. Assessment of educational personnel will be undertaken regarding research activities, updating and course evaluation by students. More details concerning the educational personnel will be included in the learning and growth perspective. Graduates’ satisfaction is measured similarly with students’ satisfaction, but with more emphasis on the value of the obtained degree. Metrics about graduates’ cognitive level regarding their professional establishment, the value (usefulness) of acquired knowledge during studies and possibilities for postgraduate studies are added. Figure 3 depicts the above analysis for the Customer (student) Perspective.


Figure 3: Key indicators for the customer (student) perspective

6 Conclusion The aims of this work were to emphasise the implementation of a measurementbased management tool in order to achieve excellence and competitive advantage in higher education according to the European Higher Education Area. The importance of the Bologna Process was mentioned and leaded us to reconsider the current situation in the department of Informatics at the Technological Educational Institute of Thessaloniki, Greece. The design of our strategy and our key performance indicators was based on the Balanced ScoreCard methodology. The Customer (student) Perspective was analysed as a first attempt in order to define the current situation, take initiatives and achieve our goals. The final objective is to improve the quality of the department. Transparency and competitiveness are requirements we have concentrated on. In order to obtain our targets, we consider that the Balanced ScoreCard methodology, which is aligns


goals, strategies and measurements of performance is suitable for our case. In this paper the design of the key Performance indicators for the Customer Perspective was presented. Future development will concentrate on applying our metrics to the principles and concepts of the Balanced ScoreCard. A pilot study will be implemented by using a Balanced ScoreCard software tool. We have already started the departmental evaluation by collecting information from different databases, interviews and questionnaires. Concerning the questionnaires distributed to the students and the educational staff in the department we experienced difficulties regarding questionnaire completions and frivolous answers. So far we have collected a satisfactory number of questionnaires and we are in the process of analysing the results by SPSS and entering the accurate data in the tool. After the quantification of the situation in the department we will focus on the weaknesses and to act proportionally.

References Basili Victor R. (1992): Software modelling and measurement: The goal / question / metric paradigm (Computer science technical report series / University of Maryland) Basili Victor R. (1995): Applying the Goal / Question / Metric Paradigm in the Experience Factory in Fenton, Whitty, Robin, and Iizuka, Yoshinori (eds.), Berlin Communiqué (2003): Bologna Process Berlin 2003, Realising the European Higher Education Area, Communiqué of Ministers Responsible for Higher Education in Berlin, 19th of September 2003, also in http://europa.eu.int/comm/education/policies/educ/bologna/bologna_en.html, http://www.coe.int/T/E/Cultural_Cooperation/education/Higher_education/Activitie s/Bologna_Process/, http://www.cep.org.hu/teachandlearn/bologna.html Bush T., Coleman M. (2000): Leadership and Strategic Management in Education, London, Paul Chapman publishing Deming W. Edwards (1986): Out of the Crisis: quality, productivity and competitive position, Massachusetts, USA Kaplan R., Norton D. (1992): The Balanced ScoreCard: A Rising Trend in Strategic Performance Measurement, Journal of Strategic Performance Measurement, Feb.March, pp. 42-48 Kaplan, R.S. & Norton, D.P. (1993): Putting the balanced scorecard to work, Harvard Business Review, Vol. 71, No. 5, September/October, pp. 134-147) Kaplan, R.S. (1994): Devising a balanced scorecard matched to business strategy, Planning Review, September-October, pp. 15-19, 48 Kaplan, R.S. & Norton, D.P. (1996a): Using the Balanced Scorecard as a Strategic Management System, Harvard Business Review, Vol. 74, No. 1, Jan.-Feb., pp. 7585


Kaplan, R.S. & Norton, D.P. (1996b): The Balanced Scorecard: Translating Strategy into Action, Harvard Business School Press, Boston, MA. Kaplan, R.S. & Norton, D.P. (2001): The Strategy – Focused Organization, Harvard Business School Press, Boston, Massachusetts Kettunen J. (2002): Competitive Strategies in Higher Education, Journal of Institutional Research, 11, pp. 38-47 Kettunen J. (2003): Strategic Evaluation of Institutions by Students in Higher Education, Perspectives, Policy and Practice in higher Education 7, pp 14-18 Miller A. (1998): Strategic Management, 3d edition, International Edition Siakas Kerstin V., Prigkou Aristea-Alexandra, Draganidis Stergios (2005): Key Performance Indicators for Quality Assurance in Higher Education – the Case of the Department of Informatics at the Technological Educational Institute of Thessaloniki, Greece, The 10th INternational Conference on Software Process Improvement Research into Education and training, INSPIRE 2 0 0 5 , 2 1 -23 March 2004, Gloucestershire, UK Wood G. (1999): Customer Communication, Butterworth – Heinemann, Cim Workbook Series Web-page 1: http://www.consultacton.com/01GetReady/glossary.htm Web-page 2: http://www.netmba.com/accounting/mgmt/balanced-scorecard/