Trust in Digital Technology: Reliability and Validity

Trust in Digital Technology: Reliability and Validity Maximilian R.W. Fink, Xueyao Ma and Harald C. Traue Ulm University, Medical Psychology, Frauensteige 6, 89075 Ulm {maximilian.fink, xueyao.ma, harald.traue}@uni-ulm.de

Abstract. Digital technologies like smartphones, assistance systems and medical equipments, experience an increased penetration into everyday life. Therefore, trust and related factors of a user of digital technology are relevant issues. To which extend is trust influenced by modern digital technology in general and in a medical context? A new questionnaire with four factors of trust was constructed. In addition, to inner consistence of scales, the validity of the new provided questionnaire is examined by an experimental study design.

Keywords: Digital technology · trust questionnaire · reliability · validity

1

Introduction

In everyday life, almost everyone is confronted with digital technologies. Yan[1, p.viii] argued, that the transformation from a social concept of trust into a digital concept, requires trust modeling and management for the construction of trustworthy digital systems. Several papers about trust in various digital domains are available. However, the majority of the papers are of theoretical nature[2], describe more or less unidimensional scales like the Computer User Self-Efficacy or the Computer Attitude Scale, but rarely report empirical research with these scales[3]. In a recent study[4] 72% of the German patients trust in the medical competence. In addition to visiting a doctor, an early diagnosis by internet will become more common in the future. Furthermore, these technologies act as individual health care, search for health information, and diagnostic procedures. This might allow a shorter latency up to interventions. Contemporarily developed Companion Technology will play a vital role in supporting patients with their health prevention activities and treatment decision-making. Above all, high-end medicine can be pursued with digital technology in form of robot technologies for e.g. surgery; an accordingly trained surgeon steers the robot with various types of tools. Advantages are lower loss of blood, the lower application of painkillers as well as a shorter stay in the hospital and smaller scars[5]. Because the modern medicine causes partly a fluent crossing between person and machine and digital technologies win within the scope of medical interventions more and more in meaning, it is to be guaranteed inevitably the trust of the patient in the involved apparatus. Thus, there is a need for psychometric scales of trust to study trust in digital technologies in various domains of technology applications. International Symposium on Companion Technology, September 23-25 2015. © Springer-Verlag Berlin Heidelberg 2015

2

Construction of a trust questionnaire, Structure & Reliability

2.1

Construction of new trust scales

Beside the already mentioned questionnaires, there are also scales and questionnaires of the operationalization of the trust which do not refer to digital technology, but can be used with regard to their core contents. For example, the Interpersonal Trust Scale[6] which captures e.g. a person´s trustworthiness and reliability. Another questionnaire captures trust in the human-human interaction in clinical settings (social trust, trust in the therapist and trust in the future)[7]. Karrer et al.[8] developed a questionnaire to capture an users affinity as personality quality in electronic devices and showed a diminishing enthusiasm with increasing age of the subjects (Technology Affinities Grasp – The Questionnaire TA-ED). Furthermore in contrast to most studies which have been conducted by means of questionnaires and interviews, the theoretical and for experimental examination usable Trust Meter was developed. It allows the calculation of potential trust with regards to the reaction of the subject in relationship with other people considering the environmental factors. None of the proposed and other examined models and theories in digital technologies and other areas generally fulfill the condition of a comprehensive operationalization of trust in digital technologies and they are rarely tested in empirical or experimental settings (see [3] for details of the referred scales). 2.2

Construction of the new trust questionnaire, item reduction and reliability

From the existing relevant questionnaires, items were adapted and new items extracted from expert interviews. A comprising subjective appraisal for the purpose of a search grid for the choice of relevant items led to necessary a-priori factors. These factors should capture the Emotion, Competence, Trust, Like and Dislike in reference to digital technology in general. For example Like and Dislike were adapted among other things from positive/negative technology results[8]. At first a questionnaire with 79items was built. The items out of the chosen sources provided five a-priori factors. A data collection of 101 subjects showed high intercorrelations between the a-priori factors and the accompanying items. Trust-scores, as well as Like-scores correlated negatively with the Dislike-scores. Therefore, the factors are not independent. On the basis of skew distributions, item-factor correlations and other criteria, like the content of the items, statistically skewed, contradictory or non-plausible items were removed from the original questionnaire. The resulting 53items version was again analyzed for skewed distributions, intercorrelations and the internal consistence. The five a-priori factors have been condensed to a four factor solution. The factor of the Trust, as well as of the Like were assembled to the factor Trust in Digital Technology due to intercorrelations (r =.576). In favor of the reliability score, several items have been additionally removed to a final version with 36items (Emotions, Competence and Trust in Digital Technologies - version german 36items, ECTDT-vg36) and four scales. The factors were renamed to capture the main meaning of the items in the following (Table 1).

In a second study with a sample of (N=264) age-distributed subjects the reliability of the ECTDT-vg36 was cross-checked . Table 1. Cronbach alphas of the ECTDT-vg36 questionnaire and gender-specific means and standard deviation (N=264 (male=106, Xage=29,45 years; female=158, Xage=24,14 years; Xage whole=26,79 years). α: Cronbach alpha. X Factor

3

α

SD

♂

♀

♂

♀

Emotions in dealing with digital technologies

.701

4,49

4,09

,85

,82

Negative emotions to digital technologies

.849

2,50

3,08

,98

,98

Trust in digital technologies

.811

3,71

3,20

,69

,66

Subjective competence in digital technologies

.838

4,23

3,66

,84

,70

Validation Study

To analyze the reliability of the ECTDT-vg36, this questionnaire was used in an experimental validation study with 264 subjects. In order to validate the new questionnaire, a faked case vignette of a possible digital surgery technique which is currently able to be realized was sketched, based on the real existing DaVinci©-surgeonrobot[5]. The validation was organized in two steps. In the first step, subjects filled out the ECTDT-vg36. In step two the faked robot news report was presented and the subjects were asked how they would trust such a surgery robot. General result is that the factors of the Trust, Competence, as well as Emotion correlate significantly positive with each other. The factor Dislike correlates significantly negative with each of the three mentioned factors. Referring to the surgeonrobot e.g. the fear of potential users correlates positive (r=.187, p