Trust and Performance in Online Distance Learning
Sónia Sousa (Sheffield Hallam University)
Abstract This paper describes a study being conducted on a graduate online distance education course at an African university in Cape Verde. The research focus is based on understanding the relation of trust and performance in such environments. This line of research builds on the belief that the trust that students put in three main components of online distance learning scenarios is the common denominator of several related problems such as motivation, retention and technology difficulties which normally lead to low academic performance. Those three factors are the relation between student’s trust towards the teacher interaction, between students’ trust towards the virtual learning environment (VLE) and also between students’ trust towards the technology. The scenario
selected
to
perform
this
research
is
a
web
based
online
distance
learning
course.
To achieve these goals a study with two complementary stages was designed. Regarding the first stage, a survey was designed to understand student’s beliefs on trust in ODL. The goal of the second stage which is still ongoing, was planned to test the hypothesis that trust and performance are related in ODL. A test and assessment procedure was also running in parallel and will eventually provide additional insight on the analysis and interpretation of the experiment’s results.
Abstract in Portuguese Este artigo descreve um estudo de investigação conduzido na universidade de Cabo verde. Este estudo tem como objectivo entender a relação existente entre a confiança e o desempenho dos alunos quando perante o cenário de ensino/aprendizagem a distancia pela Internet. Esta investigação basea-se na crença que existem três componentes principais de confiaça quando submetemos estudantes a aulas a distancia pela Internet. Esses factores baseam-se na relação existente entre a confiaça dos alunos na interação com o seu professor, entre a confiaça dos alunos no ambiente virtual de ensino/aprendizagem a distacia pela Internet e tambem entre a relação de confiaça dos aluno e a tecnologia usada. O cenario de ensino/aprendizagem a distacia pela internet usado recorre á web, fazendo uso da plataforma „Formare“ cocebida pela PT Inovação.
Para atingir os objectivos acima enumerados foi concebido um estudo que engloba duas fases complementares distintas. Na primeira fase de investigação, foi desenhado um questionário com o qual se pretendia entender quais os componetes de confiaça dos alunos quando perante uma situação de ensino/aprendizagem a distancia pela Internet. A segunda fase de investigação, a qual ainda esta a decorrer, foi planeada para testar a hipotese de que existe uma relação entre confiaça e desempenho no ensino/aprendizagem a distancia pela Internet. Junto com esta fase foi tambem desenhado um estudo paralelo para recolha de dados sobre o desempenho dos alunos durante as aulas, o que pode eventualmente ajudar e complentar a analise e interpretação dos dados finais.
Introduction In Praia, Cape Verde, September 2001, Universidade Jean Piaget de Cabo Verde decided to integrate its Capeverdian students into the global Instituto Piaget community.
Instead of deploying an all
comprehensive program, the policy was to test its online distance learning approach on most of its graduate courses using the Formare system, which is a corporate online distance-training platform, developed and maintained by Portugal Telecom Inovação. The research work outlined in this paper uses „Formare” as the Web based online distance learning infrastructure and a graduate level computer basics course as its test bed. This document describes a research project which elaborates on the idea that trust and performance are related in a predictable way in online distance learning (ODL) settings. The main goal is to contribute towards the understanding of online distance education as a teaching and learning process exploring the interaction between related trust factors and the students’ performance.
Contributing subjects A lot of learning happens through social interaction (Wegerif 1998). In order to attain these project goals, one must grasp concepts such as online distance learning, trust, performance and some related research work. A brief introduction to each subject will be provided in the next sections relating the contributing subjects to the research work outlined.
Online distance learning For most of us the common definition of education at a distance is characterized by the separation of the teacher and the learning group, with the interpersonal face-to-face communications of conventional education being replaced by a personal mode of communication mediated by technology. To be more specific as defined by Ljosä (1994 in Keith 1994), distance education is a common word to define and describe an interactive, educational process between student and teacher, separated by physical distance and time. Distance education takes place when teachers and their students are separated by physical distance and when technology (voice, video, data, and print), often complemented with face-to-face communication, is
used to bridge the instructional scenario. Therefore the term distance education subsumes a number of existing related issues, such as technology, education, psychology, sociology and other subjects. However, the rationale behind this definition is not strong enough and as Tsai and Machado (2002) say several other terms are used which are interchangeable. Distance education is a suitable term to bring together both the teaching and learning elements of this field of education (Keegan 1996). Common alternative terms include e-learning, web-based learning, online learning, and distance learning, which are widely used as identical terms. Nevertheless, these terms represent concepts with subtle, yet consequential differences not always eligible as products of distance education. Nonetheless according to Keegan (1996) there are several key features which define distance learning: •
Separation of teacher and learner;
•
Use of educational media to unite teacher and learner and carry course content;
•
Provision of two-way communication between teacher, tutor, or educational agency and learner;
•
Separation of teacher and learner in space and/or time; and
•
Volitional control of learning by the student rather than distance instructor.
My focus is on the learning side of the relation hence focusing on distance learning instead of aiming at the holistic distance education process.
Trust Everyday, and often without any reflection, we place our trust in people and in services those people provide: we trust that our friends will not betray our confidences, and that our food will not be poisoned. The list is endless and staggeringly broad. For the most part the trust we place in others and the actions we take are largely unreflective. In brief we tend to organise our routine actions with a state of what Giddens (1991) refers to as a basic trust – which, initially involves the trust of the child in their parent and into their lifelong experience of day-today life. However there are occasions when our evaluations are broken in which we have to give more thought to evaluating risk and the assignment of trust. When approaching a new experience what people try to do is apply rules that have governed previous similar experiences. Nevertheless, this does not mean that there is no variation in the meaning of trust as trust. This emphasises the complexity of the concept of trust. Looking for trust definitions is natural to find different approaches to describe trust and how it can be controlled. For example Kini & Choobineh (1998) state that trust is: •
An assumed reliance on some persons or things. A confident dependence on the character, ability, strengths, or truth of someone or something;
•
A charge or duty imposed in faith or confidence or as condition of a relationship; and
•
To place confidence (in an entity).
In the above cases trust mostly relies on feelings and human relations.On the other hand, according to the Oxford Reference Dictionary definition and Grandison & Sloman (2000), trust relies on the firm belief in the
reliability or truth or strength of an entity. The firm belief in the competence of an entity to act dependably, securely, and reliably within a specific context is also important. And distrust as the lack of firm belief in the competence of an entity to act dependably, securely, and reliably within a specific context. Trust is really a composition of many different attributes: reliability, dependability, honesty, truthfulness, security and competence… which may have to be considered as depending on the environment and context in which trust is being specified. In the definition of trust there are likely to be mixed case scenarios. However, most important to realise is that trust is usually specified in terms of a relationship between a trustor (i.e. the person who trusts) and a trustee, i.e. the person who is trusted, (Grandison & Sloman 2000). In the end, all of them make the relation with belief, with honesty, truthfulness, competence and reliability, of the trusted person or service. From literature it is clear that there are many different types of trust, which relate to the specific purposes or nature of a trust relationship. Trust is based on the combinations and judgments or opinions based on face-to-face meetings or recommendations of colleagues, friends, and business partners. There is therefore a need to discuss trust establishments, evaluation, and analysis capable of supporting Internet services and human interaction relationships. Such trust can be founded on the technological point of view -– with the use of transition mechanisms to assure buyers that their transactions are secure and trust worthy. As an example, the creation of cryptography algorithms to increases the promotion of trust, a very important point in the buying decision (Rutter 2000). Also it can be, based on a sociological point of view – through an understanding of trust factors based on previous experiences. And from that, it is possible to study and consider the community factor as in ecommerce relations, which are already in practice in some successful e-commerce web pages such as the Amazon sites (they promote an imagined community through its open community of reviewers). The same phenomena happen with communications at a distance with the use of Internet technology, bringing together both sociological and technological point of views. As Nathan et al. (2002) outlines, interpersonal trust is an area that is likely to be affected by mediated communication. It is necessary to maintain trust on-line. Jun et al. (2002) also claim that it is possible to gain trust in computer-mediated communication just by engaging in various “getting-acquainted” activities over the network. He says that in field studies there is evidence that trust between remote colleagues is significantly lower than trust between co-located colleagues These concepts come to reinforce trust as strongly related with mediated communication and also reinforces the idea that there is a direct relation between the trust factor in online distance environments and the trust factor in virtual teams. Further, an understanding of how the technological evolution of the Internet as a medium for social interaction and commerce modifies our definition and context for trust in the particular case of teacher/student relations is paramount to understanding the role of trust in the online distance learning process, (Merill 1999). As Nathan et al. (2002) remark, it can be more difficult to develop trust in an online
setting than face-to-face and consequently, it may be harder to trust an institution or a teacher that we are not able to see or touch than a teacher we can find in the corridors, see and get to know in real life as we normally understand it. Finally, in our case, trust in online distance learning may be perceived as a trade between individuals rather then between people and technology. Therefore, in this particular case I am interested in understanding the particular cases where trust affects student’s performance in online distance learning scenarios. As a final remark one should realise that trust is a concept that has already received much attention in social sciences such as psychology, sociology, political science, economics, anthropology, history and sociology but that, according to Roderick et al. (1996) and sustaining this line of research, little effort has been made to investigate the key role that trust plays in critical social processes such as cooperation, coordination and performance.
Performance Performance is regarded as measured by the assessment of the online distance learner or student. At this point, it is useful to explain exactly what is meant by assessment as opposed to evaluation as the two terms are sometimes used as having the same meaning but have in fact radically different connotations when used in an educational or training context. From this point onwards I will elaborate further into the general characteristics of a student assessment procedure bearing in mind that according to Cotton (1995), such a procedure should be valid, reliable, practicable, fair, and useful to students. Basically there are two main kinds or modes of assessment: •
Formative assessments – which are carried out during the course. Their main goal is to provide the learners with feedback on how they are doing and try to help them to learn better. They do not normally count towards a final grade or mark, nor is it normally used to determine whether the student will be allowed to progress to a later stage of a course.
•
Sumative assessments – which are normally completed at the end of a course or module in order to establish or measure what the learner has achieved. It differs from formative assessment in that it generally does count towards a final grade or mark, or is used to determine whether the student is allowed to make progress through the course.
Further, assessment can be: •
Norm-referenced assessment – also known as relative assessment as it is based on comparing the relative performances of students, either by comparing the performances of individual students within the group being assessed, or by comparing their performance with that of others with some similar characteristic.
•
Criterion-referenced assessment – also known as standard-referenced assessment generally involves determining whether the student can perform, to a set minimum standard, a specific set of tasks or activities within a particular situation or context.
•
ipsative assessment – relates to the comparison of the student’s performance with his or her own earlier performance.
Furthermore, it should also be noted that assessment procedures may be implemented in several ways. Common assessment procedures include: objective tests; short-answer tests; extended-answer tests; practical tests; situational assessment; assignments; projects; and portfolios. Some of these assessment procedures can be carried out as closed book or open-book assessment and all can be performed by external examiners, teachers, and peers and by the students themselves. The test bed online distancelearning course uses a selection of these assessment procedures.
The relation between trust and performance To understand distance education and its various forms it is necessary to find a way to cope with the complexity of the educational environment and the inter-connectedness of human, technological and institutional relationships. A key factor of online distance learning is interaction. Research findings on the need for interaction have so far produced some important guidelines for instructors organising courses for distant students. Nonetheless dropout rates are still one of the most relevant problems together with low performance and lack of motivation. It is our belief that trust also plays a crucial role in the success of distance learning. We can see nowadays various attempts in online distance learning systems to gain trust from their team. Adding places where they could maintain synchronous conversation with their colleagues and promote periodic synchronous and asynchronous conversation through videoconference and text messages, like chat and electronic mail, or just by using simple task oriented assignments. The rules usually applied to our social relationships which come so naturally from the traditional way of education must be adapted to the new way of learning considering that relations are mediated by technology based on different sources and types of communication. So, is the trust relation between student/teacher, student/VLE and between student/technology directly related to their performance during the course? Apropos, personal experience and personal narratives have also profound importance in the creation of trust. Trust deepens or retreats based on experience (ef. Rutter 2000). For instance, Amazon uses a recommended system approach (CACM 99/98) to suggest shopping items based on customers with similar profiles and their personal opinion. So, how does trust relate to performance in online distance learning? Does it mean that sustaining the students’ personal performance and narratives will result in better academic performance? Unfortunately, the answer isn’t simple and contradictory results exist. Take for instance the use of a student profile file, so common in online distance education programmes by analogy to a product file in electronic
commerce. Whereas some studies suggest that use of a photograph or image increases trust, others point exactly in the opposite direction: In “Face it – photos don’t make a web a web site trustworthy” (Riegelsberger & Sasse 2002) discard the value of photographs in electronic commerce but Steinbrück et al. (2002) point in exactly the opposite direction saying that photographs can be used as trust builders in ecommerce websites. From my point of view the lack of trust on the part of the students is due mainly to the fact that they pass through a constant evaluation process, which can create a constant insecure sensation. It seems like a process very similar to an athlete: •
Both have to train and work hard to win a competition process;
•
Both have to be very secure and believe that they are the best so in that way they avoid the lack of motivation; and
•
Both get that trust through their trainers.
Also both have a goal to achieve, i.e. winning a gold medal in the Olympics or even getting a degree, and to do that, they have to trust on their training team. Correspondingly, in the education scenario or more specific in the online distance learning process, their role is to maintain their trainees trust, i.e. make the students believe that they „can win their degree”. More and more relationships are being formed and maintained online then ever before. Nowadays, with the communication technology it is possible for people work together while being remote, with the exception of work that requires trust relations, see (Jun et al. 2002). Is it possible that trust and performance are directly connected? In the study by Jarvenpaa & Leidner (1998), teams which score high on trust were reported as being able to solve and resolve conflicts in an environment where they were limited to electronic communication in contrast to teams which score low on trust and weren’t as successful. This strongly connects with the relation between trust and performance in online distance learning. Another important finding in this research paper was the important role that responses to positive trust building activities have in increasing trust in the team. This appears to strengthen and unify the team. Because computer-mediated communication entails greater uncertainty than face-to-face communications, there tends to be an intense need for response, (ef. Howisher & Moran 1993). A related subject is the study of positive expectations in psychology. For instance, Navas et al. (1992) report on the relationship between cognitive motivational variables and academic performance. In this particular case 150 undergraduate students were administered a number of questionnaires to measure their motivational variables. Academic performance was measured from several sources and results showed a relation between expectations and performance. Interestingly, results also suggest that teachers’ and students’ expectations are the best predictors of the performance of the students. Regarding this subject, Akinci (2000), discusses the effective use of information and communication technology (ICT) as the cause of improved user performance.
Finally, according to Dillon et al. (2002) motivation and cognitive
engagement are critical for the success of ODL.
Overall research methodology This section starts by highlighting the two research stages designed to achieve this goal. An initial survey was conducted and a further experiment was designed together with a test and assessment procedure. So far, only the first stage study has been conducted. Regarding the first stage, a survey was designed to understand the student’ beliefs on trust in ODL. The first stage procedure questionnaire was made out of 85 questions where 22 were subject information questions and 63 were question generated out of the initial six draft components of trust. The questionnaire was answered by an opportunity sample of 460 students from whom 374 were considered in the data analysis procedure. Figure 1 describes survey design. In general the questions on the questionnaire were simple and directed. They were basically developed with the purpose of better understanding the subject of trust in ODL. The answers for those questions were based on a likert scale which ranges from one to seven. Out of the eighty-five questions, the 63 developed based on the initial trust factors, were randomly distributed and appended to the initial 22 related to the subjects’ general information. Figure 1: Survey design
Subsequently, the data analysis enabled us to identify three main trust factors, which according to students’ opinion could influence their trust level in ODL. Furthermore, from these results it was also possible to built a picture regarding the students’ trust distribution across age and gender; among subjects with and without a computer at home; among access to a computer; among access to the Internet and among subjects with and without experience in distance education and ICT experience. The second stage explores the relation between trust and performance in online distance learning settings. In the end this staged approach will aim to measure two variables: the dependent one – students’ performance and the independent one – student’s trust in ODL. This is achieved by deploying simultaneously an experience and a testing and criterion-referenced and ipsative assessment procedure to portray, analyse and interpret students’ trust and performance during an online distance learning class. The experiment measures the subjects’ achievements during this specific ODL set-up based on the three main components of trust. This procedure starts with two initial questionnaires, regarding the students’ level of trust in the three main factors and regarding the students’ generic knowledge on the learning subject.
This experience was conducted with over 80 subjects during an online class. The subjects were classified as rating low, middle or high on the three trust factors and the subjects were also classified as rating low, middle or high over class generic knowledge on the learning subject. Two measures of performance were collected, an assessment over part of the class’s theory learning content and an assessment over part of the class’s practical learning content. All collected data will finally undergo a correlation analysis to measure the relation, if any, between the subjects’ trust ratings and their performance. Figure 2, describes the experiment and assessment procedure design.
Figure 2 : Experiment and assessment procedure design
Experiment procedure
Assessment procedure
Data colection Q1
Q2
Data colection Q3
Q4
ODL theory contents
ODL theory contents
Data col. Q5
Data col. Q5
ODL practical content
ODL practical content
Data col.
Data col.
Q6
Q7
Data col.
Data col.
Q6
Q7
Data analysis
Data analysis
The first stage methodology results The result section describes and summarizes all the results and conclusions obtained from experiment one, including the data collected and the statistical analysis results. The three trust factors construction The goal for this first stage analysis was to identify the main trust factors. This was achieved by performing a factorial analysis method over the collected data to identify the three main components that were the trust factors according to subjects’ opinions. The three main factors, or components, of trust are: •
[F1] – Trust in relation to the student/teacher interaction;
•
[F2] – Trust in relation to the Virtual learning environment; and
•
[F3] – Trust in relation to the technology.
At this point three new variables were created, one for each factor. The score of the new variables resulted from the summation of each trust factor related question. Additionally, after the creation of these three variables the anlaysis of the results is continued with the validation of these variables by processing correlation
analysis
between
these
new
variables
of
trust
and
other
generic
indicators.
Subjects’ gender distribution across age Gender is equally distributed across age with the exception of the last age group, with an age range between 27 and 52, where the male population is higher than the female population. Subjects’ trust base distribution across factors Trust in [F1] is slightly below neutral; trust in [F2] is almost neutral; and trust in [F3] shows some positive expectations. Trust distribution across age groups No significant difference was found in trust relations with regard to factors one and two, but, significant difference was found on factor three i.e. Trust in relation to the technology between the youngest and the oldest groups It is thus possible to conclude that trust is equally distributed across age groups with the exception of the trust in technology for the younger and older population. In my opinion these results are due to the fact that this study was performed in an African country, in which the older have more access to the technology during their working hours than the younger. In a completely different scenario the results could be quite different. Trust distribution across gender A T-test shows that trust in the three factors is equally distributed across gender. This confirms that there is no gender bias in relation to trust in this sample. Trust distribution among computer ownership condition Having a computer at home influences the subjects’ trust. In fact a T-test shows a significant difference between subjects with and without computer at home. Subjects with a computer at home have a higher level of trust when compared with subjects with no computer at home.Trust distribution among computer access condition. A T-test shows that having access to a computer is not enough to foster subjects’ trust for factors [F1] and [F2]. For [F3], trust in technology, there is however a difference with statistical significance. Having access to a computer does increase the subjects’ trust in technology. Trust distribution across Internet access condition A T-test shows significant differences in trust factors [F2] and [F3] distribution across Internet access condition. Subjects who have access to the Internet do have higher trust in an ODL environment and in technology than the others.
Trust distribution across previous experience in distance education A T-test shows that previous experience in distance education does relate to trust in [F1] and [F3]. On the other hand, in relation to trust in ODL environment, the subjects trust is equally distributed across the condition. Trust distribution among subjects with and without ICT experience There is a significant correlation between ICT experience and all the trust factors The greater the subject’ s ICT experience, the higher is her/his trust. With this data analysis I was able to confirm the hypotheses and determine the components that can have most influence or could improve the trust relation with performance. Also I was able to build an initial picture of the student population available as subjects as far as this research project is concerned.
Final remarks At the end of both experiments I hope to establish if there is any relation between trust and performance in online distance learning and if so, which one, if not all, of the three trust factors is the most influential in student trust in online distance learning. So far the results enable me to believe that there will be a significant interaction between trust and performance but that it will be difficult to identify and isolate such interaction. This is exactly why the second study has been designed.
References Akinci, B. (2000, 17-18 March). Attitudes & Reactions to Information Communication Technologies and Open Distance Learning At the Edge of Change: Human Resources Approach in Traditional Universities. Paper presented at the Research and Innovation in Open Distance Learning: First EDEN Conference, Prague. Cotton, J. (1995). The Theory of Assessment: an Introduction. London: Kegan Page Limited. Dillon, C., Greene, B., & Billy, C. (2002). Motivation and Cognitive Engagement in Distributed Learning Environments: Learner Differences That Transcend International Boundaries. Paper presented at the EDEN Annual Conference, Granada. Gameta, D. (Ed.). (1998). Trust making and breaking cooperative relations. Oxford: Basil Blackwell. Giddens, A. (1991). Modernity and Self-identity. Cambridge: Cambridge. Grandison, T., & Saloman, M. (2000). A Survey of Trust in Internet Applications. Retrieved May, 2002, 2002, from http://www.comsoc.org/pubs/surveys Howisher, G., & Moran, E. (1993). Electronic mail and writing instructor. College English, 55(6), 627-643. Jarvenpaa, S. L., & Leidner, D. E. (1998). Communications and trust in global virtual teams. Retrieved May, 2002, 2002, from http://www.ascusc.org/jcmc/vol3/issue4/jarvenpaa.html
Jun, Z., et al. (2002). Trust without touch: jumpstarting long-distance trust with initial social activities. Paper Conference on human factors in computing systems, Minneapolis, Minnesota. Keegan, D. (1996). Foundations of distance education (3rd ed.). London ; New York: Routledge. Keith, H. a. o. (1994). Distance Education: New Perspectives. New York: Routledge. Kini, A., & Choobineh, J. (1998). Trust in Electronic Commerce: definition and theoretical considerations. Paper presented at the 31st Annual Hawaii Int’l. Conference System Science, Hawaii. Lewicki, J., & Bunker, B. (1995). Trust in relationships: a model of trust development and decline. In B. Bunker & J. Z. Rubin (Eds.), Conflict, cooperation and justice. San Francisco: Jossey-Bass. Ljosä, E. (1994). Distance education in the society of the future: from partial understanding to conceptual frameworks. In D. Keegan (Ed.), Distance Education: New Perspectives. New York: Routledge. Merill,
L.
(1999).
E-commerce
and
the
challenge
of
Trust.
Retrieved
May,
2002,
from
http://www.ml.com/woml/forum/pdfs/ecommerce.pdf Nathan, B., et al., a. (2002, 20-25 April). Effects of four computer-mediated communications channels on trust development. Paper Conference on human factors in computing systems, Minneapolis. Navas, L., Sampascual, G., & Castejon. (1992). Atribuciones y expectativas de alumnos y profesores: Influencias en el rendimiento escolar. / Attributions and expectations of students and teachers: Influence in school performance. Revista-de-Psicologia-General-y-Aplicada. 1992 Jan, 45 (1)(IS: 0373-2002), 55-62. Riegelsberger, J., & Sasse, M. A. (2002). Face it Photos don’t make a web site trustworthy. Paper Conference on human factors in computing systems, Minneapolis. Roderick, M., Kramer, R., & Tyler, T. R. (1996). Trust in organizations: frontiers of theory and research. California: SAGE publications Rutter, J. (2000). From the sociology of trust towards a sociology of “E-trust”. Manchester: ESRC Centre for Research on Innovation and Competition. Steinbrück, U., et al., a. (2002). A picture says more than a thousands words – photographs as trust builders in e-commerce websites. Paper Conference on human factors in computing systems, Minneapolis. Worchel, P. (1979). Trust and distrust. In W. G. Austin & S. Worchel (Eds.), The social psychology of intergroup relations. Belmont: CA: Wadsworth.
Acknowledgements I would like to thank Brian Hudson from the School of Education at Sheffield Hallam University, David Lamas from Laboratório de Educação a Digital at Universidade Jean Piaget de Cabo Verde and António Clemente from Universidad de Valencia for all the help and assistance during this research.
Contact Sónia Sousa School of Education Sheffield Hallam University
[email protected] 
