Interpretive Research Models for Informatics

Interpretive Research Models for Informatics: Action Research, Grounded Theory, and the Family of Design- and Development Research M.R. (Ruth) de Villiers

Abstract Various research models based on the interpretive paradi~ can serve effectively as research approaches for studies in Informatics (Information Systems). This metaresearch study overviews research paradigms and Information Systems (IS) research, then describes, discusses and illustrates some interpretive approaches: action research, grounded theory, and the family comprising development research and two forms of design research, namely, design-science research in IS and design-based research in the realm of educational technology. In the current milieu - with its emphasis on interactivity, user-centricity, usability, empowerment software, and elearning - inquiry processes originating from the social sciences are relevant to IS, particularly for research on human aspects of the design and development of personal computing applications. The five approaches advocated have underlying theoretical and methodological frameworks and reflective methods. Each one can serve as a model to guide the research process, offering a unifying thread, cohesion and internal consistency to a research study.

Key Concepts Action research, design-based research, design research, design-science research, development research, educational technology, evaluation, Alternation 12.2 (2005) 10- 52 ISSN !023-1757

10

Interpretive Research Models for Informatics

grounded theory, human-computer metaresearch, research design.

interaction,

information systems,

Introduction Interpretive research, an approach that originated in the social sciences and humanities, is increasingly applied in Informatics, a discipline more formally called Information Systems (IS). This article suggests some practical approaches and mode]s to operationalise interpretive research in IS. Two models - action research and grounded theory - are addressed, after which three models from the family of design- and development research are discussed - development research, design-science research for IS, and design-based research for educational technology. In some of the latter, influences from positivism are not excluded. The context of the study is not primarily research in the generation of systems for business organizations, but rather sma11er-scale research for personal computing, including usercentric applications to bridge the digital divide or to support e-leaming. This metaresearch study is particufarly relevant to postgraduate students doing research for masters/doctoral dissertations, who require an underlying theoretical framework to guide the research process and to provide cohesion and internal consistency. Research design and -paradigms in IS are currently receiving attention (Baskerville, 1999; Baskerville & Wood-Harper, 1996; Cockton, 2002; De ViHiers, 2005a; du Plooy, 2004; Glass, Ramesh & Vessey, 2004; Klein & Myers, 1999; Myers, 2004; Pather & Remenyi, 2004; Roode, 2003; 2004; Travis, 1999; Walsham 1995a; 1995b; Wood-Harper, 1985). In the present miheu with its emphasis on user-centricity and usability, research models originating from the social sciences also hold relevance.

Research Paradigms and Methods Different research paradigms and models are based on varying philosophical foundations and conceptions of reality (Cohen, Manion & Morrison, 2000; du Poy & GitJin, 1998; Lincoln & Guba, 1985; Olivier 2004). Each

11

M.R. (Ruth) de Villiers

paradigm, in tum, is implemented by associate.d methodological approaches and strategies.

Positivist and lnterpretivist Paradigms The positivist paradigm holds that knowledge is absolute and objective and that a single objective reality exists external to human beings. Positivism is equated with the scientific method, whereby knowledge is discovered by controlled empirical means, such as experiments. Positivist research is intended to produce an exact representation of reality, unbiased and valuefree. Research results should be reliable and consistent, free from perceptions and biases of the researcher. Findings should be replicable by other researchers. Positivist research relies primarily on quantitative methods, where data comprises mainly numbers and measures and analysis is done by statistical methods. Results can be used for prediction. Studies are usually hypothesis-driven. These methods originate in the natural sciences, but are also applied in the social sciences. Interpretivism, by contrast, aims to find new interpretations or underlying meanings and adheres to the ontological assumption of multiple realities, which are time-and context dependent. A related term is 'naturalistic', which has connotations of research done in a natural setting, rather than in a laboratory (Cohen, Manion & Morrison, 2000; Lincoln & Guba, 1985). Terminology is not absolute, for example, Walsham (1995a) refers to the 'interpretivist school', yet as a general adjective refers to 'interpretive approaches, -methods, -studies', etc. Travis (1999) and Roode (2003; 2004) prefer 'interpretivist'. Interpretive research emanated from the social sciences and is also used in educational research, in which context Reeves (2000a or 2000b:6) explains that interpretive goals determine how something works by describing and interpreting phenomena regarding domain processes, performances, innovations, etc. In recent years interpretive research has become accepted in IS (Klein & Myers, 1999; Roode, 2003; Walsham, 1995a; 1995b). Klein and Myers (1999) point out that interpretive studies can provide deep insight into IS phenomena, including both their management and their development. They can help the IS research community to understand human thought and action in social or organizational contexts. 12


Inquiry is value-related as inte.rpretivism leads to subjective findings which may differ between researchers. It is an appropriate view for studies of complex human behaviour, shared meanings, documents and other artifacts, and social phenomena. Just as positivism is most naturally operationalised using quantitative methods {yet not exclusively), so interpretivism lends itself mainly (but certainly not exclusively) to qualitative studies. Where positivism tests hypotheses, interpretivism investigates research questions, focused on understanding phenomena that occur in natural settings {ethnographic) and which use verbal data. Qualitative data collection and analysis produce findings related to intricate details where values and human experiences are relevant. In such contexts, the ability to interpret data is important and, in fact, 'the researcher is an instrument' (Leedy & Ormrod, 2001:147). Reliability in qualitative research can be considered as a 'fit' between the findings recorded and occurrences in the natural setting. Research methods are frequently triangulated by multiple data collection methods.

Research Methods: Qualitative and Quantitative Mertens ( 1998) describes qualitative research as a naturalistic interpretive science which is multi-method in focus. It involves the use of methods such as case studies, interviews, observation and textual analysis, which provide insights into cultural aspects, organizational practices and human interactions. Qualitative and quantitative methods are not mutually exclusive. Many studies require eclectic inquiry methods to cover the terrain and provide triangulation. Another role of qualitative research is as exploratory work, where qualitative findings from basic research in new areas are used to formulate hypotheses and questions for subsequent quantitative analysis, which yields empirical results to test, verify and extend the qualitative hypotheses. Figure I shows common research methods on a Positivist-Interpretivist axis, tending from the quantitative to the qualitative, yet with an overlap.

13

M.R. (Ruth) de Villiers I

Quantitative 111111

*,I ... .e I

.,,c

.

tll)

·="ii tll)

c

> ~

E

..g

....c

"8

E

]

i §

.. .

e ·z: -s :; ::E"' ·;;;E

.

~

c

gc: u

i.i:

8.

""' ] 0

I

Qualitative

'i'

..

;

.. .. 8. .,,.."' ..

.5

I

.,,1

~

c

§

"ii

I I ·c: I c0 8. I ·~ "' ·:a::I ·;;c I .0:ii ~ I 0 0 ~

c E

tll)

Positivist

c

c: 0

·.;

fl)

., 3:

~!! ~ ~.5

iii

~~

13

'g 1il

~

a ::I

~

"'::I

c

~

8

""

0..

~

0

-s Ill

i:I .!!"g c ..... e ti ~1

lnterpretlvist

Figure 1 Research methods/strategies

Information Systems Research With a view of paradigms and methods as background, we move on to a

discussion of information systems research. IS is a multi-perspective discipline and, as Wood-Harper ( 1985) asserts, it can rightly engage a plurality of research methods. Its domain incorporates scientific, technological, engineering, organizational, managerial, psychological and societal aspects. The subdiscipline of human-computer interaction (HCI) is currently prominent, highlighting the role of the end user, and adding more disciplines: sociology, philosophy, physiology, anthropology, linguistics, ergonomics, graphical design, marketing and engineering. The increasing power and stability of technology has pushed the application of infonnation systems into multiple domains (Pather & Remenyi, 2004), making it appropriate territory for practicing reflective methods and, as Pather and Remenyi propose, IS research needs corresponding reorientation. Walsham (1995b) examined IS research journals from 1992 onwards, four each from the UK and USA, and noted the advent of interpretive studies. In an extensive analysis of computing research, based on a study of selected papers from leading journals between 1995 and 1999, Glass, Ramesh and Vessey (2004) coded 628 papers from Computer Science (CS) journals and 369 from Software Engineering (SE), including the ACM and

14


IEEE publications, as well as other well-recognised journals. They also examined 488 Information Systems papers, mainly from 'Management' IS literature. Their metaresearch investigated the kind of studies conducted in these three disciplines, noting similarities and differences. The IS topics focused heavily on organisational concepts (66°/o) with systems/software management and systems/software concepts next, both at 6-7%. Of the CS papers investigated, systems/software concepts comprised 19% and the SE focused on systems/software (55%) and systems/software management concepts (12%). Each category was further divided into subcategories and it was found that 24% of the IS related to usage/operation and 19% to technology transfer, with other IS papers focusing on the problem domain, e.g. decision support systems and group support. With regard to research approaches, 67% of the IS papers used evaluative approaches. IS research methods were field study (27%), laboratory experiment on humans (16%), conceptual analysis (15%), and case study (13%). Finally Glass et al report on levels of analysis, stating that most of the IS work was related to behavioural aspects: organizational behaviour (26%), individual behaviour (26%), and group issues (11%), whereas behavioural analysis in CS and SE occurred in only 2% and 8% of the cases respectively. Walsham (1995b) and O'Donovan and Roode (2002) attach importance to the editorial policy shift in the journal MIS Quarterly, which has expanded its range of research by explicitly calling for papers based on interpretive or integrated, as well as positivist, approaches. This tendency of IS to take cognizance of human behaviour and to use evaluative approaches is notable. There is a current gravitation particularly in the HCI subdiscipline - towards interactive, user-centric computing, usability support for persona] computing, and the empowerment of domains beyond business and management information systems. Applications are also being developed to bridge the digital divide and to offer accessible e-leaming and e-training. Much of the IS research conducted, is done for postgraduate study. Du Plooy (2004) and Roode (2004) held a seminar on non-positivist IS research methodologies, with particular reference to doctoral studies. Du

15

M.R. (Riith) de Villiers

Plooy (2004) notes concerns, describing IS research as a 'maturing science' and a 'fragmented adhocracy' lacking theory and methods, which does not fit well into a positivist paradigm, especially when qualitative methods are used. IS places a major focus on the unique qualities of information itself, as occurs during enquiry into design, management decisions, and social processes. Du Plooy suggests that academic IS research is often geared to the dissertation examiner as target, rather than the IS community, and that results seldom reach practitioners. His consequent axiology of relevancy posits that current interpretive IS research is weak in clear proposals on how to improve practice and that it avoids value-laden research issues. PhD students should address practical problems, using sound interpretive and evaluative approaches based on methods such as field studies, semistructured and unstructured interviews, and ethnographic data. Analysis should be conducted using, for example, grounded theory, hermeneutics, or critical social theory, all of which contribute to the production and refinement of theoretical frameworks (du Plooy, 2004). In line with his notable editorial (Roode, 2003: 1) proposing 'acceptance of interpretivist research on a semi-equal footing with positivist research', Roode (2004), referring to Hirschheim and Klein (2000), addresses the intellectual state of IS research, with its internal and external views. The internal view of the IS research community indicates fragmentation, particularly due to the paradigm war between interpretivists and positivists. The external view relates to the gap between IS research and external expectations, where current research is insufficiently relevant to practice. Research outputs produce ad hoc findings, yet lack generality, and do not broaden theoretical constructs. Roode promotes a broadened notion of relevancy. While the accepted scientific method - entailing hypethetico-deductive studies - is relevant, non-positivist forms of scholarly research should not be excluded. Regarding postgraduate research in general, Mouton (2001) points out the multidisciplinary and heterogeneous nature of current knowledge. He further cites methodological difficulties as a major factor in the noncompletion of postgraduate studies. In this milieu, the contribution of this study is to suggest practical, methodologically- and theoretically sound approaches for interpretive research in Informatics. Most of the approaches originated in the social


sciences, yet they are applicable to research in computing disciplines, where the design of human-computer interaction is highly relevant in a society and economy increasingly geared to user-centric values. The approaches are applicable to postgraduate studies, as well as to basic, ad hoc and contract research.

Interpretive Information Systems Research This section focuses on theoretical frameworks to underpin IS research. Pather and Remenyi (2004) propose critical realism as an appropriate approach to bridge the gap between the positivist and interpretivist paradigms, using both qualitative and quantitative techniques. De Villiers (2005a) suggests approaches and methods that primarily implement the ethos of the interpretive school, namely development research, action research and grounded theory. This present article extends the field by introducing two further models from the prevalent family of design research, which is implemented in different ways in different contexts. When conducting IS research, an explicit theoretical framework or conceptual model should, ideally, support the study and provide internal continuity and cohesion in the reasoning process. Research terminology is briefly addressed, with its plethora of overlapping, exclusive, interrelated and, at times, confusing terms. A taxonomy of terms is provided, explaining how they are used in this study, although other authors may well use them differently: Paradigm - the primary philosophical point of departure: in this study, the interpretive (rather than positivist) stance. Model - the underlying research approach used to guide and operationalise the study: the approaches suggested here being action research, grounded theory, development research. and two forms of design research. Methods - practical means/strategies/techniques/instruments used for data collection: each approach has its own set of methods, usually multiple methods and often hybrid methods. Adjectives to describe the methods: some methods are qualitative and others quantitative.

17


With the acknowledgement that computing has human and sociological, as well as technological and computational dimensions, research methods from the interpretive paradigm have a definitive role to play. Preece et al (2002) distinguish between the usability and user experience of software systems. The latter, in particular, lends itself to interpretive and qualitative analysis. The next sections describe, discuss and illustrate the selected approaches/models and their application in interpretive IS research. Each approach has associated methods and techniques to operationalise it.

Action Research Definition and Origins The action research approach (Baskerville, 1999; Baskerville & WoodHarper, 1996; Cohen, Manion & Morrison, 2000; Zuber-Skerrit, 1992) emanates from the behavioural sciences and encompasses a variety of research and intervention methods. Action research (AR) originated in action-based social psychology. Its founder in the 1940s, Kurt Lewin of the University of Michigan, contended that complex real social events could not be investigated under laboratory conditions (du Poy & Gitlin, 1998; WoodHarper, 1985). AR was used independently at the Tavistock Clinic to study post-WW2 social disorders among veterans (Baskerville, 1999). ZuberSkerrit (1992) defines AR as inquiry by higher-education academics into issues encountered when students learn. Its participative, practitionerresearcher approach lends itself to the domain of educational research, where an evolving intervention or product is investigated over several cycles. It is appropriate for inquiry into educational technology and for investigating the introduction of technologies into organisations. Aiming to bridge the gap between research and practice, AR encompasses action outcomes and research outcomes (Dick, Passfield & Wildman, 1995). Commencing with the identification of a problem or situation that calls for action, AR functions as a liberating agent of change, and is (Baskerville, 1999; Dick, Passfield & Wildman, 1995; du Poy & Gitlin, 1998): Cyclic: as iterative steps recur in a longitudinal time frame, generating 18


knowledge to infonn further action. Participative: as clients, end users and researcher collaborate in close partnership as co-researchers; or as practitioner-researchers examine their own work. Where stakeholders are full participants in the research process or where practitioners serve both as subject and researcher, one refers to participative action research. Qualitative: operating more via verbal aspects than by numbers. Reflective: since critical reflection on the process and outcomes is vital to each cycle, and is used in designing subsequent steps and events. Responsive: as it reacts and adapts flexibly to the findings from each previous cycle.

In a parallel from the professional disciplines, Schan ( 1987) defines

reflective practice or reflection-in-action as the professional artistry that occurs when skilled practitioners tackle work-related activities, going beyond rigid rules of inquiry, and generating new rules in situations that are uncertain and unique. Furthennore, the reflective practitioner is both a participant in the process and a critic who observes and analyses. Similarly, AR aims to improve practice and advance knowledge.

Research Processes and Methods Zuber-Skerrit (1992) terms the four repetitive processes undertaken in each cycle as plan, act, observe, and reflect. The megaprocess comprises a series of cycles that feed into each other. AR is accordingly more of an ongoing process than an event. Du Poy and GiUin (1998) state that action research employs or integrates methods from both the experimental and naturalistic (interpretivist) traditions, yet is consistent with naturalistic inquiry in that all research occurs within its natural context. In the context of rigour and validity, Kock (2004) cautions regarding

three potential 'threats' in AR. First, the issue of control: while the natural environment, as opposed to a lab setting, is one of AR's benefits, the researcher does not have complete control over the environment and the subjects. As a methodological tool to counteract this, Kock recommends that data collection and analysis be based on the units of analysis method, with 19


the units predefined before research commences. Second, he refers to the contingency threat: as a change agent, the researcher has access to a large body of data, which may be 'broad and shallow' and thus complex to analyse, because its rich context makes it difficult to separate components relating to particular constructs. An antidote is the integration of grounded theory (see next section) with a three-step coding process, involving the identification of (i) categories, (ii) relationships between them, and (iii) grouping of the related categories into theoretical models. Third, the close involvement of the researcher might lead to subjective bias in interpreting the data. This can be counteracted by multiple iterations of the AR cycle to support collection of cumulative data about specific units of analysis (Kock, 2004).

Epistemology and Philosophy ofAR Action research has an interpretivist ethos, incorporating social enquiry based on the views and interpretations of the participants, all regarded as equals, making it an emancipatory process, while also incorporating the researcher as participant. It is a holistic, not a reductionist, approach, which includes ethnographic enquiry and works from an ideographic standpoint, promoting the uniqueness of each setting (Baskerville, 1999). At the time when AR originated, emphasis was placed on the precise collection of quantitative data and there was a shortage of qualitative research skills. In due course, it was recognised that AR operated under a different epistemology and, although it can be less rigorous in design and methodology than other approaches, it came into its own as a human-related research process which generates reliable knowledge. AR research can be distinguished from development research (a subsequent section) in that first, it operates over a longitudinal time framework of several cycles and, second, in the in-depth involvement of researcher as participant. Third, it focuses more on refinement of existing processes or products rather than on new developments. Fourth, in many cases there is no attempt to construct theory, models or principles to guide future work.

20


Application within JS AR is a valid research approach for applied fields (Myers, 2004). Baskerville ( 1999) asserts that the AR approach generates highly relevant research results due to its basis in practical action, aimed at explicit problem solving while also informing theory. In the 1980s, AR techniques were applied by Peter Checkland in systems analysis, as he developed soft systems methodology, i.e. use in development, rather than in research (Baskerville, 1999). IS research has been characterised by lack of relevance (Keen, 1991; Westfall, 1999; both cited by Baskerville, 1999). In the previous decade, however, Trevor Wood-Harper had already set out to address the tensions between theory and practice and the confusion that existed between traditional scientific research and the more sociological approaches, by introducing AR to the IS community as a purely research methodology with his landmark paper, Research Methods in Information Systems: Using Action Research (Wood-Harper, 1985). AR is increasingly used for scholarly research in IS. Baskerville (1999) advocates it to inquire into the complex and multivariate nature of IS's social setting, using:

1. Joint goals of solving practical computing problems and expanding scientific knowledge. 2. Collaborative performance, which enhances the competencies of all participants. 3. An emphasis on action and change orientation in social settings. 4. Systematic, iterative stages. Baskerville lists forms of IS action research: prototyping, soft systems methodology, action science, participant observation, fieldwork, and process consultation. The present author adds investigation of evolving solutions in their context of use, e.g. e-leaming applications and customised interfaces, where the designer conducts research into his/her own product. AR for producing e-leaming solutions is also advocated by Demtl and MotschnigPitrik (2004).

21


Action research can be graphically depicted as a spiral, but the model developed for the author for Figure 2 is a series of cycles which close in as a solution is attained. The researcher occupies a central, participative, and influential position. This model forms a useful framework to guide and monitor the progress of a research project.

Ob••IV•••d evaluate

Oburvo and

/--Ill>-------., '·

/

Acti•• A

t __..R•="""'---110---~

-

•valuate

//-·-~

'"\ *7 "-+,,,

I

.

Figure 2 Action research model (synthesized by the author)

Grounded Theory Definition and Origins Grounded theory (Cockton 2002; Glaser & Strauss, 1967; Glaser, 1992; Leedy & Onnrod, 200 I; Strauss & Corbin, 1990) is an approach in which theory and models are generated inductively from the analysis of contextual data. Grounded theory (GT) involves the discovery of concepts and hypotheses as theory emerges from empirical data. There is no testing or replication of a-priori theory. Like action research, it has roots in social science, but specifically in sociology, where attitudes to conditions (initially attitudes to death) were investigated by Glaser and Strauss. Strauss and Corbin (1990) describe its extension to anthropology, education, medical professions, and the economic sciences. Cockton {2002; 2004) uses it in the design of computing interactions. It provides a conceptual grasp of substantive areas, which evolves and is modified to fit as findings emerge 22

Interpretive Research Models for Informatics and new data occurs. GT should account for variation in domain behaviour by defining categories, properties and relationships. The data may be quantitative, qualitative or a combination, but in the case of qualitative, it is essential that it be systematically collected, analysed and coded. Defined sets of methods should be applied so that a grounded theory emerges systematically and inductively through covariant ongoing collection and analysis. Lincoln and Guba (1985) state that a GT is adjusted, expanded, and refined via this ongoing process. As with AR, there is a parallel from the social professions. The notion of emerging patterns has an analogy in Christopher Alexander's patterns within architecture and town planning, which form practical architectural languages, as physical and social relationships articulate themselves (Alexander, Ishikawa & Silverstein, 1977). Within a pattern language, it is also possible to integrate overlapping patterns in a densifying process, which provides added meaning. The four criteria for a well-constructed grounded theory within a substantive area are: 1.

Fit: its categories and their properties should fit the realities being studied.

2.

Work: in order to work, it should explain variations in behaviour.

3.

Relevance: this is achieved when a grounded theory both fits and works.

4.

Modifiability: the emerging theory is open to adaptation as new data and variations are integrated.

Research Processes and Methods Urquhart (2002), citing Bryant (2002), refers to the term 'grounded theory method' (GTM) to distinguish the actual process and methods of generating grounded theory from the ultimate objective, namely, the grounded theory itself. Urquhart acknowledges the clear guidance given in GTM literature on how to code qualitative data. Covariant data collection and analysis, which are methods from the social sciences, entail initial interviews and fieldwork, which after transcription, coding and analysis delimit the field, prompting

23


theoretical sampling methods and densifying (Glaser, 1992). Other methods (Strauss and Corbin, 1990) are observation, document analysis, historical records, and videotapes, which reflect perspectives of the subjects of study. GT integrates quantitative and qualitative perspectives (du Poy & Gitlin, 1998). Back-and-forth mobility occurs between data collection and analysis, with analysis driving collection. Patterns are identified and conceptualized as the researcher systematically codes, compares, analyses and records. Constant comparative coding (du Poy & Gitlin, 1998; Glaser & Strauss, 1967; Glaser, 1992) is the validation process whereby observations and behaviours are compared/contrasted with core categories and properties, then coded into categories. The developing conceptual model is modified as new data is explored and new concepts are integrated into the emerging theory, reviewing and expanding where necessary. When multiple behaviours indicate similar patterns and properties, saturation has occurred. When disconfirming evidence is found, revisions are required. In this way the emerging theory is inductively discovered, bounded and confirmed. Lincoln and Guba (1985) suggest the generation of alternative theories. As the researcher encounters confirming cases, negative cases and discrepant cases, the theory with the most confirming cases and the least negatives and discrepants, emerges as the most robust. GT thus has similarities to case study research and ethnography, since both the latter aim to detect and interpret patterns within activities and events.

Epistemology of GT The researcher's bias and subjectivity may influence conceptualization and interpretations. However, grounded theory bas built-in mechanisms to prevent this, such as constant comparison, saturation and core relevance (Glaser, 1992). Furthermore, data collection, analysis and presentation to peers should be linked at each step, adjusting one another to the emergent theory and preventing forcing. The Glaser model posits, contentiously, that so as not to force or make preconceptions, there is little initial need to review literature. Once the emerging theory is sufficiently grounded in core variables, literature reviews in the substantive field can commence and be related to the new work. Where new fields are opened the researcher, according to Glaser, is a 'pre24

Interpretive Research Models for Informatics empting pioneer', producing a new general theory to be integrated with other literature. Scholarship starts and expands as the emerging grounded theory develops. Glaser claims that this approach also obviates the problem of more and more to read and less time to do! According to Urquhart (2002), however, the originators' actual position on avoiding study of existing literature is less stringent than it appears to be at first encounter with GT.

Application within IS Orliowski 's ( 1993) award-winning paper in an issue of the MISQ describes a project in which a grounded theory research approach was used to study organizational experience with the adoption and use of CASE tools. Findings were used to develop a theoretical framework conceptualizing organisational change and social issues in such cases, where installation involves not only new technologies, but also organisational change over time. GT was a relevant approach, because of its emphasis on contextual elements, process management, and human actions. Urquhart (2002) describes GTM as an appropriate means of analysing qualitative data in IS research and highlights its use in interprettive studies, as applied for example, in her own work (Urquhart, 2001). Cockton (2002; 2004) discusses the applicability of grounded theory to computable interactions, and explains how, as theories and themes emerge, corresponding models can be defined and implemented using an HCI contextual approach. Such models could include personas, scenarios and sequence models, where the persona describes a stereotypical user and the scenario a stereotypical usage. It is a rich context-centred approach, which takes account of users' goals and aspirations, aiming for high relevance. The models, in tum, are used in design to generate prototypes, following which the fit between context of use and interaction surfaces can be tested and mediated. Thus grounded theory research in IS investigates data, resulting in theory, which leads to models, which lead to innovative grounded designs or design models, which satisfy 'fit to context' (Cockton, 2004). The GT process can be applied in studies that focus on the extraction of design guidelines by analysis of practice or best practice in substantive areas, synthesizing them into theoretical proposals, which are further refined; tested and ratified by use. This is being done in certain current South African

25


postgraduate studies. The associated generation of design principles and evaluation criteria for software applications and web site development is a notable role for GT in information systems research and development.

in

Glaser's (1992) concept of pre-emption is appropriate the emerging Southern African technological domains where innovative work is underway in, for example, the design of non-standard interactive environments, such as development software explicitly for the formerly disadvantaged, emergent information systems, culturally-sensitive environments, and contextualized e-leaming and e-training. Research on culturally-sensitive interfaces is being extended beyond the work place, to address accessibility for under-educated indigenous peoples. Figure 3 graphically illustrates the processes and concepts of GT. This model can serve as an underlying framework for IS research processes which investigate phenomena to determine their underlying theory and to derive principles.

The Family of Design- and Development Research The 'family' of design- and development research, comprising several research models and varied terminology, is currently under the spotlight. The plethora of papers and articles published between 2003 and 2005, justifies a serious analysis of these approaches, particularly their implications for ···research into producing artefacts, such as development software and eleaming applications. Terminology varies and approaches range between and within domains, but concepts such as design, artefacts and/or interventions are common to all. Three sections are devoted to this family. Development research, fairly precisely defined and established, is described first. This is followed by two separate overviews of the more complex design research first as it is practiced in the Information Systems (IS) and Information Technology (IT) disciplines, mainly under the name design-science research, and second as it occurs in the domain of Educational Technology (ET) where the prevalent term is design-based research. The ET variants are included, because educational systems, or e-leaming applications, are a subset of IS at its intersection with the learning sciences, and also due to the

26

Syatamatlc c:ollectlon and refinement

Co!Dxt: data In eubetantlve aru

Data, Incidents, oburvdona

D D

D

DD

DD D

Dlecovery

Confirmed patterns 111nd properties

Concepts

ooGon HypothMla

D~D

Saturlllt.lon .

conmnt

~ D

oo§o D

D~Cl

[]~a

D

[]

Rlllat.lonahlpa

Figure 3 Model ofgrounded theory emergence (synthesized by the author)

D D D

Emergent theory 111nd prlnclpM


emphasis of this article on small-scale research, such as that which is undertaken for masters- and doctoral studies, where studies relating to aspects of e-leaming are on the increase.

Development Research Definition and Origins Development research (DR), also called research with a development goal, has a dual focus, as it: l. develops practical and innovative ways of solving real problems 2. proposes general design principles to inform future decisions

DR is not to be confused with 'developmental research', which relates mainly to sustainable development in the context of transformation and community issues in developing countries. The DR approach (Reeves, 2000a; van den Akker, 1999; 2002), which aims to make both practical and scientific contributions, originated in educational technology research, giving graduate students and researchers support in pursuing development goals after decades of research with empirical goals. It is also applied in curriculum research, where some of its key concepts emerged. It is not yet a mainstream IS research approach. Other candidate approaches for development goals have drawbacks. Postmodern perspectives and critical theory are emancipatory, addressing inherent problems and injustices, but do not necessarily improve conditions. Action research (see earlier section) is not always explicitly geared to producing new solutions and may not be generalisable. Development research is problem-oriented, searching for new and innovative solutions, while also seeking findings that are transferable, practical, and socially responsible. DR acknowledges the complex and dynamic relationship between theory and application, and aims to provide a relevant foundation to guide practice by generating design principles and methods that are both theoretically underpinned and empirically tested.

28


Research Process and Methods Development research generates different kinds of research questions. A descriptive question examines the nature and extent of a problem, while a design/development question investigates an intervention or new product to address the need. A principial question aims for generalisable principles and guidelines for use in an application domain. The process commences with the analysis, design and development of an artefact or intervention as a solution for a real-world problem. This, in and of itself, is not yet research (though many postgraduate students wish it was!). It becomes research when the design-and-develop project is conducted from the perspective of a researcher striving to understand the issues of the application domain and its target users, such as the required characteristics of products and artefacts. Such research is based on iterative £!.nalysis, Q.esign, Q.evelopment, implementation and formative ~valuation (ADDIE - a design model that originated in instructional technology), which feeds into redevelopment. DR is closely related to evolutionary prototyping, and entails formative research during the development process of the intervention/product to improve its quality. Van den Akker (2002) terms the process 'successive approximation of the ideals'. Evaluations can be done by one or more usability evaluation methods, e.g. formal usability testing in a laboratory, logging, surveys among end users, observation, etc. There are various models of the DR process. The model used by Plomp and his co-researcher, van den Akker (van den Akker, 1999; 2002; Plomp 2002) refers to outcomes of an intervention. Immediate outcomes relate to results of using an intervention or product within the cyclic process, and distant outcomes emerge when the immediate outcomes lead to distant outcomes in the form of generalisable principles. Reeves' (2000a) model emphasises the iterative interaction between researchers and practitioners to clarify the problems and refine potential solutions in a process of evolutionary prototyping. Plomp's and Reeves' models both influence the representation in Figure 4.

Epistemology ofDR Development research has a pragmatic epistemology as it acknowledges collaborative shaping by researchers and practitioners. Van den Akker 29

MR. (Ruth) de Villiers

( 1999), describing the knowledge acquired from DR, distinguishes between: . Substantive design principles, relating to the generic characteristics of suitable interventions or products. Methodological aspects, with a procedural emphasis, suggesting optimal development processes. In formative research a great deal of such knowledge is inductively extracted from the experience of using and evaluating the prototype developed for the study. This provides a link between the two branches of the dual development focus, namely the developing solution to a specific problem and the evolution of generalisable design principles. The experiential evidence obtained from studying the prototype in use in its various iterations, is enhanced when integrated with theoretical arguments.

Application within IS Many IS studies involve the generation of software artefacts or web-based applications. These vary, for example, from simple prototypes through interactive web sites with backend databases through to virtual reality simulations. Design, implementation and testing comprise the focus area of development but are not research. The introduction of evaluation, where evaluation entails more than mere testing of functionality, constitutes a meaningful contribution to the body of knowledge. However, dual-focused research producing both an effective solution and generalisable principles for the application domain, enriches the process even more and is a useful approach for a variety of studies in domains such as e-leaming, e-commerce, e-health, etc. A product - often a prototype - can be custom-built as a solution to a problem and iteratively evaluated and refined, as described by Conradie and de Villiers (2004). DR has also been used for computer-based support materials, and is relevant to computer science and engineering as well for generating hardware solutions and associated generic principles. Figure 4 is a generic model of the DR process. Its iterative phases can be effectively used to structure an IS research process, providing continuity and cohesiveness.

30

Interpretive Research Models for /nfot:mati'ds

Problem 11naly11i11

Design solution,

ba•edon ·theon11tlcal fn11mework

SOll,ltbt'. ~·' • , outc:om4i) .)

Development used as Intervention

Develop (prototyps) aolutlon

Evaluate and test

(Imme.~!~,,

General des1gn .---==--1.,..,' principles outcomes) reflection

(di~

R-ment of problem, solution, methods

Figure 4 Development research model (synthesized by the author, influenced by Plomp (2002) and Reeves (2000a))

Design-science Research in Information Systems Design research is increasingly undertaken in disciplines such as Informatics, the learning sciences, and educational technology. Due to differences between the ways it is applied in different areas, terminologies, methodologies, and practices vary considerably. Even within a discipline, variations occur. For this reason, the so-called design-science research in Information Systems (IS) and Information Technology (IT) is treated separately from the design-based research of educational technology (ET).

This section relates specifically to design research in IS and IT, disciplines in which it is also called design-science research (DSR). The context of the literature surveyed, relates more to ISs in the workplace than to research on applications for personal computing. The next section will address design research and its variants in the context of educational technology research.

Definition and Origins Design research owes its ongm to the Nobel laureate, Herbert Simon (Simon, 1981 ), who distinguishes between the natural sciences and the socalled design sciences. Natural sciences relate to natural phenomena such as those described in physics, astronomy, anatomy, etc. Associated

31


descriptive theories explain how phenomena occur, setting out laws and. relationships. Design sciences, by contrast, are the 'sciences of the artificial' and relate to man-made objects and artificial phenomena, gen!!rated in applied sciences such as medical technology, engineering; architecture, product design, and instruction. Associated prescriptive theories and models set out goals to be achieved and procedures to accomplish ends. Applied sciences are characterized by problem-solving processes, invention, construction, and evaluation of artefacts or interventions. Design science led in tum to design research. The Design Research in Information Systems Group (DRIS) (2006) is a Web-based community that collates information on design research. DRIS views design research as a problem-solving, performance-improving activity, involving invention and evaluation, including measurement of artefacts and their impact on the overall system. Similarly, Hevner, March, Park and Ram (2004) describe design science in IS research as a problemsolving approach, rooted in engineering and Simon's sciences of the artificial. It aims to be technologically proactive, focusing on creating innovative IT artefacts and effective products, with the challenge being to determine how and why they work. To this end, they are evaluated to identify and to solve problems.

Research Process and Methods Design has double connotations. As a verb, it relates to processes and, as a noun, to products or artefacts. When design research is applied to classic ISs, business applications in large organizations, its outputs are not only full systems, but also their building blocks. March and Smith (1995) and Hevner, March, Park and Ram (2004) describe the artefacts and activities of DSR. The output artefacts are defined as constructs, models, methods and instantiations: Constructs or concepts are the domain vocabulary, terms used to describe problems and specify solutions. They may be formal notations for data modelling or informal text. Models are forms or representations where constructs are combined to show relationships, e.g. entity-relationship diagrams. Models are useful in the process of designing an application. Methods are ways of performing goal-directed activities, often involving a

32


set of steps, e.g. an algorithm. They build on constructs and models and transform one model to the next in the problem-solving process of systems development. An instantiation is an actual implementation - a product to perform a task in a particular environment. It may be an IS itself, a prototype, or a tool to support the design of ISs. Instantiations are the final link in the research chain as they operationalise constructs, models and methods. In actual practice, an instantiation may pre-exist an articulation process defined with constructs, models and methods. When the instantiation is studied, the latter can be formalised and used to generate improved instantiations. DRIS (2006) suggest a fifth output, a generic one, namely 'better theories' as in development research. Theories emerge as construction methods are studied and as construction and evaluation elaborate existing theories. This suggestion is re~visited after the next paragraph. The two main complementary activities in generating DSR outputs are (March & Smith, 1995; Hevner, March, Park, & Ram, 2004):

Building: the design and construction of constructs, models, methods, and instantiations to meet identified needs. Judgments and decisions must be made based on value to the user community - in this case, the business context. To build artefacts, foundational knowledge is required: theories, frameworks and tools from prior research, although when completely new artefacts are created, each is an experiment, and often done with little prior knowledge. Evaluation: determining how well an artefact works in its environment and feeding back into building, so as to replace technologies with more effective ones. Evaluation criteria and metrics must be set up to judge performance in context or to compare performances when evaluating subsequent versions. IS evaluation makes use of mathematical modelling and computational techniques, and also empirical and qualitative methods. In evaluating instantiations, efficiency, effectiveness, and impact on environment should be considered, as well as rich phenomena as subjects interact with it in context. These human issues involve the selection of subjects, training, and the definition of tasks, so as to gain further understanding of the design problem. The interaction of humans, 33


organisations, and technology may require qualitative study. The knowledge needed for the evaluation activity is methodological, in the form of guidelines. The artefacts and activities described above are combined in the information technology design-research framework (March and Smith, 1995), which maps the activities of building and evaluation against the four artefacts: constructs, models, methods, and instantiations. The framework explicitly excludes justifying and theory generation from design research, positing that these subsequent activities fall into the natural sciences, a view supported by Hevner et al (2004). This contrasts with the belief of DRIS (2006), who citing Rossi and Sein (2003) and Purao (2002), add a fifth output: construction of better theory. DRIS describes the methodology of design research. Each step results in an output and, typically, these steps are: Awareness ofproblem: resulting in a proposal; Suggestion: using human creativity to suggest new functionality, leading to a tentative design; Development: actual production of the artefact. This may involve normal methods of practice, with the novelty being in the design; Evaluation: which may involve quantitative and qualitative methods, and which produces performance measures. If they deviate from expectations, this must be tentatively explained. Where initial hypotheses about an artefact's behaviour are not borne out (as occurs frequently), the new information is used to generate modified hypotheses and is fed into an iterative loop back to earlier processes. When the final cycles of design and development are done; there is: Conclusion: bringing the research to a 'satisficing' end, where the results are deemed to be adequately satisfactory. Satisficing means finding satisfactory solutions while sacrificing an exhaustive search through all the possibilities.

The question arises: What is the difference between design research (here termed design-science research) and normal design?

34

Interpretive Research Models for Informatics DRIS (2006) maintains that DSR produces interesting new knowledge for the user community over and above the designed artefact/s. Research efforts should be focused on areas of complexity. March and Smith ( 1995) posit that building the first version of an artefact can be considered research, provided that it holds utility. The contribution to research is in its novelty and in the credibility of its claims to effectiveness. The research contribution in building subsequent artefacts is on the basis of improved and more comprehensive performance. This is supported by Hevner et al. (2003), who believe that design-science research addresses major unsolved problems in unique or innovative ways and previously-solved problems in more efficient or effective ways. Finally, March and Smith ( 1995) stress that IT research should pay as much attention to novel instantiations as it does to constructs, models and methods, such as various formalisms. Actual instantiations and implementations can provide real-world proof that artefacts work. Design research in the disciplines of IS and its MIS subarea is characterized by rigour and thoroughness. Hevner et al. (2004) present a comprehensive information systems research framework, which is an extension of March and Smith's (1995) information technology designresearch framework. The environment of Hevner et al. 's framework is Simon's (1981) 'problem space' containing organizations, people, and technology. This integrated framework shows the contributions of design research and behavioural research to IS research, where design science conducts the develop/build phase, building and evaluating artefacts, while behavioural science induces the complementary justify/evaluate phase, developing and justifying theories, and explaining and predicting phenomena. To conclude this subsection Hevner et al. 's (2004) seven guidelines for design-science research in IS are presented:

1. Design: An innovative, viable artefact must be designed and produced (construct, model, method, or instantiation) to address a particular organisational problem. The artefact is unlikely to be a full-scale operational product for use in practice.

JS


2. Relevance: A technology-based solution must have utility in addressing the problem. 3. Evaluation: Utility, quality and efficacy must be rigorously demonstrated by appropriate well-executed evaluation methods. Integration of the artefact into its technological environment should be investigated, as well as the artefact's 'style' (Norman, 1988, cited by Hevner et al.) Evaluation methods include observational, analytical, experimental, testing, and descriptive techniques. 4. Research contributions: These should be clear and verifiable in tenns of the artefact and its design foundations or creative methodologies; as well as new, innovative and interesting. 5. Rigour: Rigorous methods should be used in construction and evaluation, but the emphasis on rigour and mathematical formalisms should not reduce relevance. The use of effective metrics related to the evaluation criteria is vital, because performance claims are based on these metrics. Furthermore, the human aspects should be addressed appropriately. 6. Design as a search process: Desired ends should be achieved and an effective solution found, while still satisfying laws in the problem space. Suitable methods are iteration, heuristic search, generate-andtest cycles, and means-ends analysis. The problem can initially be simplified and decomposed, followed by expansion, i.e. a satisficing approach. 7. Communication: Results should be presented both to technologyand management-oriented audiences. The former require construction and evaluation details, while the latter (potential endusers) are concerned about the artefact's impact, novelty and effectiveness.

Epistemology of DSR Design research changes the state-of-the-world by introducing novel artefacts (DRIS 2006). Thus, in contrast to a positivist ontology, it acknowledges alternative world states, but these multiple world states are 36

Interpretive Research Models for Informatics not identical to the multiple realities of interpretivism. In view of this, DRIS suggests that DSR is neither positivist nor interpretive research, but in between as a philosophical perspective with a pragmatic, problem-solving approach that tolerates ambiguity. However, DSR's cycles of observation and interventions are similar to those of action research and thus DSR has aspects of interpretivism, a]though its time frame is shorter than that of AR, because it excludes social group interactions. Regarding the epistemology of DSR, the basis of its knowledge claims are as follows (DRIS 2006): an artefact is constructed; its behaviour is determined by its interactions; descriptions of these interactions are information; and to the degree that the behaviour is predictable, the information is true. This can be termed 'knowing through making', and is in line with the claim by Hevner et al (2004) that knowledge resulting from DSR is obtained via construction in context, and its meaning is iteratively revealed through cyclic study of the constructed object.

Application within IS DSR is appropriate for problems with ill-defined environments, complex interactions, and flexibility for change. There is a critical dependence on human cognition, creativity and teamwork to produce solutions. Use of existing foundational and methodological knowledge helps to achieve rigour, but where there is no pre-existing knowledge, designers must rely on intuition, experience and trial-and-error. An artefact developed in this way is an experiment, hence the value of building prototypes. At this stage there are not many well-documented cases of the use of DSR in IS projects. Hevner et al. (2004) identify and cite three IS studies, which can be considered as design-science research and use them to illustrate the application of their seven design-science guidelines for IS research (previous-but-one section). The application domains of these three are, respectively: (i) the development of new techniques for implementing anonymity in group decision support systems, (ii) a proposed design for a routing language, and (iii) a design theory for the development of a particular class of IS.

37


Design-based Research in Educational Technology As stated in the previous section, design research is undertaken in various disciplines, including the learning sciences and educational technology (ET). This section considers design research in ET, where certain researchers refer to the 'design experiments' of educational practice, yet others to 'formative research', and where the current prevailing term is design-based research (DBR). It is a maturing field, widely discussed in current literature, entailing meta-analyses as well as reported research. To clarify the terminology, Wang and Hannafin (2005:7} compiled a table of terms and associated methods. This table is not shown here, but interested readers may consult it.

Definition and Origins As outlined in the discussion of design research in IS, the design sciences relate to man-made objects/phenomena, including instruction, which is frequently based on prescriptive theories and procedures. Education is characterized by complex problems and the invention of solutions, and also by the construction and evaluation of artefacts or interventions. It is a suitable domain for the application of a design-based research approach which, according to the Design-Based Research Collective (2003) and Wang and Hannafin (2005), is an emerging paradigm for educational inquiry, in which:

1. The goal of designing learning environments in real-world settings is related to the goal of developing prototypical theories. 2. Development and research occur through continuous cycles of analysis, design, development, enactment, evaluation, re-design. 3. Research should result in contextually-sensitive, sharable design theories, communicated to practitioners and designers. 4. The success or failure of a design in its setting should be documented and accounted for. Barab and Squire define design-based research as a series of approaches which aim to produce new theories, artefacts, and practices

38

Interpretive Research Models for Informatics related to teaching/learning in natural _settings. In the specific context of ET, DBR is elaborated by Wang and Hannafin (2005) as being:

Pragmatic and theoretical: extending/generating theory and also producing principles to inform and improve practice. Grounded: design conducted in real-world contexts; ideally, theory-

driven, based on an appropriate learning/instructional theory. Interactive, iterative and flexible: designer-researcher-participant

teamwork; iterative cycles; formative evaluation generates evidence to guide revision and improve design; initial prototypes. Integrative: hybrid research methods; data from multiple sources. Contextualised outputs: results connected to research setting; the design principles generated vary according to context.

Viewing design science and design research as portrayed in literature of educational technology and the learning sciences, ET research has different methodologies and frameworks from those of IS design research, with its software engineering roots. Current ET research displays a strong interpretive paradigm, in contrast to its former positivist stances and quantitative studies, Reeves (2000b) mentions an increase in qualitative studies and mixed methodologies, along with the recent upsurge in design research and formative research.

Research Process and Methods The focus of this article is research in computing-related disciplines, including educational technology applications in the broad area of elearning. However, it is necessary to mention 'design experiments' in the learning sciences, which led to the evolution of design research in ET. Design experiments did not occur solely in the context of educational technology, and the term 'design' may refer to the design of experiments or learning configurations, not necessarily to the design of artefacts. Ann Brown (Brown, 1992) - a learning theorist, trained to work in controlled

39


laboratory settings - transported her work to research activities in natural settings. She engineered innovative educational environments and conducted experimental studies on methods of teaching and self-reflective learning in the context of children reading educational content, then comprehending and retaining it. Her work is 'intervention research designed to inform practice' (Brown, 1992:143), but based on theoretical concepts that show why the methods work, and make them reliable. Despite the applied setting, the ultimate goal is not only to inform practice, but to work towards theoretical models. A further pioneer is Allan Collins (1992) who, using the term 'design science of education',_ conducted design experiments on ways of using technology in the school classroom. His immediate goal was not to design new technology (though this is not excluded), but to investigate, evaluate and compare different educational technologies and computing tools in supporting learning about climate and the earth-sun relationship. The long-term goal is to use design experiments to construct a systematic methodology, a design science, to support educators in exploring the huge problem space of possible designs for classroom-learning with technology. This science would determine how various designs of learning environments contribute to learning. Ultimately, as in Brown's aim, a design theory should be developed to guide educational innovations. A decade and more later, several well-known researchers refer back to this original work and reflect on the present status and impact of DBR in the learning sciences and educational technology, among others: Sasha Barab and Kurt Squire, Allan Collins and team, Paul Cobb and others, the DesignBased Research Collective, Michael Hannafin and Feng Wang, Thomas Reeves, and EduTech Wiki (an ed-tech design-based research group, similar to DRIS in IS). Citing the pioneers, Brown (1992) and Collins (1992), Reeves (2000b) notes the relationship of design experiments with development research and lists some characteristics of design experiments. They: address complex problems in real contexts in collaboration with practitioners, and integrate existing design principles with technology to produce plausible solutions to the problems.

40


Rigorous and reflective inquiry is conducted to test and refine innovative learning environments and to define new design principles. Collins (same Collins), Joseph and Bielaczyc (2004), now using the term 'design research', reflectively outline theoretical and methodological issues that have emerged in the evolution of design research in the study of learning, namely:

theoretical issues about the nature of learning in context. approaches to studying learning phenomena in the real world rather than in a laboratory. derivation of generic research findings from formative evaluation.

Are these approaches always successful? No; complex realworld situations present challenges, due to the inherent lack of control and large amounts of data from triangulated ethnographic and quantitative studies. There is a notable correspondence between these challenges and Kock's (2004) concerns (in the section on action research), highlighting again the commonalities between the interpretive approaches in this meta-research study. Collins et al (2004) raise the further issue that design as implemented, or enacted, often differs from the design as intended. This is in line with 'incorporated subversion' (Squires, 1999), a term describing how users configure, or subvert, an environment or system to their own needs, and use it in ways not intended by the original designer. What is distinctive about DBR? Barab and Squire (2004) refer to the methodologies of Brown and Collins, which investigate learning in natural contexts - yet explicitly designed by the researcher and with systematic adjustments so that each adaptation provides further experimentation. Barab and Squire's view of design research involves the development of technological tools and, particularly, the generation and testing of theories to support understanding and prediction of learning. Other research methods also generate theory, but DBR's defining feature is its aim to influence 41


practice with real changes at local level and to develop tangible artefacts that can be used elsewhere. DBR also has similarities to action research in that it is an action-oriented perspective, which sets out to change situations (EduTech Wiki, 2006). Barab and Squire (2004) do, however, exercise a word of caution regarding to what extent context-specific research claims can he used in inform broad practice.

Features have been extracted from the extensive meta-analyses and reflective studies of Barab and Squire (2004); Cobb, Confrey, diSessa, Lehrer, and Schauble (2003); the Design-Based Collective (2003), EduTech Wiki (2006), and Wang and Hannafin (2005) and have been synthesized and classified by the present author into Table 1, which is a summary of the main features of DBR in educational technology and the learning sciences.

Feature of DBR models Real-world problems

1

Elaboration

I Design theory addresses complex problems in • collaboration with practitioners/educators.

Problem solutions grounded in pre-existing theories,

Where appropriate theories/principles pre-exist, design should be theory-driven, along with technological affordances, to propose solutions to the problems.

Innovation

DBR should investigate less-common practices and generate technological support; design of innovations, novelty, interventionist aDDroaches.

Engineering

Designing forms, means, or artefacts of learning; systematically studying them and the consequent learning.

Iterative design,

Cvcles of design, enactment, analvsis, redesign.

Context

Research studies in context, i.e. in naturalistic settings; use of artefacts/ interventions in the real-world; theories also to be contextualized.

42


f Empirical research

Researching tangible, real-world products, which ideally, should be usable elsewhere, i.e. influence on practice.

Refining the artefact /system,

Using formative evaluation to derive research findings; design and explore artefacts, environments, etc. with rigorous inquiry methods to refine them and define new desian princioles.

Output artefacts

Real-world products; technical and methodological tools; frameworks and models; curricula; theories.

Developing theories

The generated theories to be evaluated and refined, a orosoective-theorv-coniecture-reflection cvcle.

Pragmatic

The theories develooed should do real work,

Synergy

Design, research, theory and practice are advanced concurrentlv.

Table 1: Summary of features (synthesized by the author)

of design-based research models

Epistemology ofDBR Reeves (2000b) queries whether ET research is basic research to extend the body of knowledge or applied research to solve real problems. Reeves cites Stokes' (1997) call for 'use-inspired basic research', where advances in technology advance new types of research, producing a reversed model that moves from applied- to basic research.

The previous section noted the suitability of design experiments and design research for problems in ill-structured environments with complex interactions. The experimental generation of new prototypes highlights the roles of cognition, intuition, creativity and teamwork in solving problems and knowledge generation. The philosophical foundation of DBR is thus not a positivist approach to enquiry, but a pragmatic form of enquiry, where judgement is based on the ability of a theory to work in the real world (Barab & Squire, 2004). Evidence-based claims demonstrate that a particular design works, relating it to contemporary theoretical issues and furthering theoretical knowledge. This enquiry occurs in naturalistic settings, as

43


knowledge about artefacts such as e-learning applications .evolves in context, and ·even by trial and error. This investigation in context leads. to a 'minimal ontology', in that researchers cannot return to the laboratory to further test their claims. Moreover, the research is not replicable due to the role of context. Validation occurs when results regarding the designed object are validated by actual use. The Design-Based Research Collective (2003) points out that validity can be addressed by iterative evaluation processes, confirming findings and aligning theory, design and practice.

Application within IS The e-learning subset of IS applications, according to a broad definition, includes multiple formats and a range of technologies such as the Internet, Web-based learning, multimedia CD-ROM, online instruction, learning management systems, educational software, and traditional computerassisted learning (CAI) (de Villiers, 2005b). Research on aspects such as the delivery, content and architecture of some of the above forms of e-learning is being undertaken for masters- and doctoral studies. A number of these studies can be considered as design-based educational technology research approaches. For example, a recent South African study (Masemola & de Vilhers, 2006) in the HCI subdiscipline, proposes, uses and refines a framework for usability testing of e-learning applications. The approach, although not explicitly set out as a design research study, qualifies as design research. The research output is not the pre-existing e-leaming application. Instead, the output artefact is the usability-testing framework, which informs practice in the immediate context of the usability study, and also contributes to a generic usability testing framework, due to innovative ways of using usability laboratory technology to add value to evaluation of applications in non-standard domains. Wang and Hannfin (2005} propose that design-based research has an important role to play in the development of technology-enhanced learning environments (TELEs) (Wang & Hannfin, 2005). TELEs are learning and instructional systems incorporating teaching for the acquisition of skills and knowledge, a variety of tools, and technological resources. However they

44


have had little direct influence on practice. Facing a core problem, namely, that TELEs lack a clear under.lying theoretical framework, Wang and. Hannafin recommend DBR to address the future situation, so that design and research activities can ~ecome reciprocal as the design of TELEs and the development of appropriate learning theories proceed concurrently, each mutually informing the other.

Conclusions about Design- and Development Research Family The design-and development family of research models differ from classic research, in that they address complex and 'messy' real-world practice, founded on contexts with multiple dependent variables. The role of endusers should be not as subjects, but as participants (although this is not always the case in IS research within organisations). There is no testing of hypotheses, but rather reflection and evaluation, flexible design revision of artefacts and products, and development of theories for actual practice (Barab and Squire, 2004). The present author believes that the currently prevalent forms of design research are maturing, particularly as their theoretical and methodological underpinnings are the object of focused attention. In time it is possible that they will subsume the less high-profile development research.

Conclusion to Article Certain research issues are best suited to interpretive and context-dependent approaches. For situations where human performance and social inquiry are relevant, the research models advocated in this article can be considered as supporting frameworks for Informatics research on an interpretivist platform. Action research, grounded theory, development research, design-science research, and design-based research all provide theoretical frameworks and methodologies to guide a research project, providing cohesion, internal continuity and consistency as the study progresses. When one of these approaches is used as a model, the activities, relationships, and reasoning

45


entailed in the research process should be explicitly linked to. the selected approach, so that its framework dictates the steps. Regular textual and diagrammatic reference to the model in the written document can enhance the study, providing underlying structure and a unifying thread, both for the researcher and the readers. These approaches are particularly relevant for research focused on applications for end users who are not computing professionals, and for milieus where the IS researcher is not restricted to working in an organisational environment. Examples are the investigation of software or Web applications for personal computing, and systems for the empowerment of learners or indigenous peoples. These may be small-scale studies, often for doctoral- or masters research, with constrained budgets. It is not the intention that the approaches should be used simultaneously; although this metaresearch study has shown commonalities, inter-relationships, and similarities. There are situations where the use, in tandem, of more than one research approach for different aspects of a study, has a synergistic impact. Walsham (1995a:76, citing Eisenhardt's (1989) discussion on theory in organizational research) stresses the role of theory in interpretive IS research. He identifies 'three distinct uses of theory': (i) as an initial guide to design and data collection; (ii) as part of an iterative process of data collection and analysis; and (iii) as a final product of the research'. From the studies of AR, GT, DR, DSR, and DBR, respectively, it is evident that each of them meets the first criterion in terms of offering a basic research design. With regard to the second required use of theory, each research model requires iterative data collection, evaluation/critical reflection and analysis as an integral part of the approach. And, in general, they aim to generate some new principle or form of theory as a product or byproduct of research. This study is not intended to be exclusive: there is much scope for positivist IS research, as well as hybrid forms. Furthermore other interpretive approaches exist; the literature mentions interpretive means such as case studies, critical theory, descriptive studies, document analysis, ethnography, field studies, focus groups, hermeneutics, phenomenology, role play and semiotics (du Plooy, 2004; Olivier, 2004; Travis, 1999; Walsham, 1995a; 1995b). However, action research and grounded theory are consistently advocated. One of the sources studied was Michael Myers' detailed, 46


hyperlinked overview of Qualitative Research in Information Systems (Myers, 2004), which converges closely· with the present independent suggestions. He recommends the approaches of (i) action research, (ii) case studies, (iii) ethnography and (iv) grounded theory. The first and the fourth confirm the present suggestions, while the both the second (case studies) and third (ethnography), relating to field-based and contextual studies, can be used as methods within the models proposed here. Then there is the independent plethora of literature (consulted in this study) presenting the design- and development-oriented approaches, which hold high utility for future application in information systems research. The advent of personal computing and the impact of the HCI subdiscipline are changing the nature of information systems and hence the forms of research in Informatics. The findings of academic research and realworld best practice should inform each other. The social inquiry research approaches outlined in this paper and research activities conducted in natural settings, have the potential to impact on performance in interactive computing applications. Findings should contribute to theory and to realworld systems, should generate design principles and evaluation criteria, should encourage sound interaction practices, and help to support technological bridges over the digital divide.

•Note: For some who may be unfamiliar with terminology, a few words are explained in a glossary after the references.

References Alexander C, S Ishikawa & M Silverstein 1977. A Pattern Language. New York: Oxford University Press. Baskerville, RL 1999. Investigating Information Systems with Action Research. Communications of the Association for Information Systems, 2 (Article 19). Retrieved May 2006 from http://cais.isworld. org/articles/2-19/ Baskerville, RL & AT Wood-Harper 1996. A Critical Perspective on Action Research as a Method for Information Systems Research. Journal of Information Technology 11 :235-246.

47

M.R. (Ruth) de Villiers Barab, S & K Squire 2004. Design-based Research: Pitting a Stake in the Ground. The Journal of the Learning Sciences 13,1: 1-14. Brown, AL 1992. Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings. The Journal of the Learning Sciences 2,2: l 41-178. Cobb, P, J Confrey, A Disessa, R Lehrer & L Schauble 2003. Design Experiments in Educational Research. Educational Researcher 32, 1: 913. Collins, A 1992. Toward a Design Science of Education. In Scanlon, E & T O'Shea (eds): New Directions in Educational Technology. Berlin: Springer-Verlag. Collins, A, D Joseph & K Bielaczyc 2004. Design Research: Theoretical and Methodological Issues. The Journal of the Learning Sciences 13,1: 1542. Conradie, MM & MR De Villiers 2004. Electronic Assessment of Free Text: A Development Research Initiative. South African Journal of Higher Education 18,2: 172-188. Cockton, G 2002. My Grounded Design Page. Retrieved May 2006 from http://www.cet.sunderland.ac.uk/-csOgco/grounded.htm Cockton, G 2004. A Tutorial: Grounded Design and HCI. September 2004. Pretoria: University of South Africa. Cohen, L, L Manion, & K Morrison 2000. Research Methods in Education. London: RoutledgeFalmer. Derntl, M & Motschnig-p1trik 2004. A Pattern Approach to Person-Centered e-Leaming Based on Theory-Guided Action Research. Proceedings of the Networked Learning Conference 2004. Design Research in Information Systems (DRlS) 2006. Retrieved September 2006 from http://www.isworld.org/Researchdesign/drisISworld.htm De Villiers MR 2005a. Three Approaches as Pillars for Interpretive Information Systems Research: Development Research, Action Research and Grounded Theory. In Bishop, J & D Kourie. Research for a Changing World: Proceedings of SAICSJT 2005: 142-151. ACM International Conference Proceedings Series. De Villiers, MR 2005b. e-Leaming Artefacts: Are they Based on Leaming Theory? Alternation 12,lb: 345-371.

48

Interpretive Research Models for Informatics Dick, B, R Passfield, & P Wildman 1995. A Beginner's Guide to Action Research. Retrieved September 2006 from http://www.scu.edu.au/ schools/gcm/ar/arp/guide.html Du Plooy, N 2004. Infonnation Systems Research. Presentation at Doctoral Seminar: Research Methodologies in Informatics. January, 2004. University of Pretoria. Du Poy, E & LN Gitlin 1998. Introduction to Research: Understanding and Applying Multiple Strategies. St Louis: Mosby Inc. Edutech Wiki 2006. Design-based Research. Retrieved August 2006 from http://edutechwiki.unige.ch/en/Design-based_research Glaser, BG 1992. Basics of Grounded Theory Analysis. Mill Valley CA: Sociology Press. Glaser, BG & AL Strauss 1967. The Discovery of Grounded Theory. Chicago: Aldine. Glass, RL, V Ramesh & I Vessey 2004. An Analysis of Research in Computing Disciplines. Communications ofthe ACM 47,6: 89-94. Hevner, AR, ST March, Park & S Ram 2004. Design Science in Information Systems Research. MIS Quarterly 28(1) 2004:75-105. Hirschheim, R & HK Klein 2000. Information Systems Research at the Crossroads: External versus Internal Views. In Baskerville, R, J Stage & J de Gross (eds): Proceedings of the IFIP TC8 WG 8.2 International Working Conference on the Social and Organizational Perspective on Research and Practice in Information Technology, Boston, MA: Kluwer Academic Publishers. Klein, HK & MD MYERS 1999. A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems. MIS Quarterly 23,12: 67-93. Kock, N 2004. The Three Threats of Action Research: A Discussion of Methodological Antidotes in the Context of an Information Systems Study. Decision Support Systems 37: 265-286. Leedy, PD & JE Onnrod 2001. Practical Research: Planning and Design. Upper Saddle River, NJ: Merrill Prentice Halt. Lincoln, VS & EG GUBA 1985. Naturalistic Inquiry. Newbury Park: Sage Publications. March, ST & GF SMITH 1995. Design and Natural Science Research on Information Technology. Decision Support Systems 15: 251-266.

49

M.R. (Ruth) de Villiers Masemola, SS & MR De Villiers 2006. Towards a Framework for Usability Testing of Interactive e-Leaming Applications in Cognitive Domains, Illustrated by a Case Study. In Bishop, J & D Kourie. Service-Oriented Software and Systems: Proceedings of SAICSJT 2006: 187-197. ACM International Conference Proceedings Series. Mertens, DM 1998. Research Methods in Education and Psychology: Integrating Diversity with Quantitative and Qualitative Approaches. Thousand Oaks: Sage Publications. Mouton, J 2001. How to Succeed in your Master's and Doctoral Studies: A South African Guide and Resource Book. Pretoria: Van Schaik Publishers. Myers, MD 2004. Qualitative Research in Information Systems. Original Version: MIS Quarterly 21,2, 1997:241,242. Updated in MISQ Discovery modified Sept 08, 2004. Retrieved May 2006 from: http://www.qual.auckland.ac.nz/ O'Donovan, B & D Roode 2002. A Framework for Understanding the Emerging Discipline of Informations Sytems. Information Technology & People 15,1: 26-41. Olivier, MS 2004. Information Technology Research: A Practical Guide for Computer Science and Informatics. Pretoria: Van Schaik Publishers. Orlikowski, WJ 1993. CASE Tools as Organizational Change: Investigating Incremental and Radical Changes in Systems Development. Management Information Systems Quarterly 17,3. Pather, S & D Remenyi 2004. Some of the Philosophical Issues Underpinning Realism in Information Systems: From Positivsim to Critical Realism. In Marsden, G, P Kotze & A Adessina-Ojo (eds): Fulfilling the Promise ofJCT - Proceedings of SAICSIT 2004. Pretoria. Plomp, T 2002. Some Reflections on Development Research (DR). Seminar at the Faculty of Education, University of Pretoria. August 2002. Preece, J, Y Rogers & H Sharp 2002. Interaction Design: Beyond HumanComputer Interaction. London: John Wiley & Sons, Inc. Reeves, TC 2000a. Socially Responsible Educational Technology Research. Educational Technology 40,6: 19-28. Reeves, TC 2000b. Enhancing the Worth of Instructional Technology Research through 'Design Experiments' and other Development Research Strategies. Paper at International Perspectives on

50


2r

1 century, Symposium by Instructional Technology Research for the SIG/Instructional Technology at Annual Meeting of the American Educational Research Association. New Orleans, April 2000. Roode, D 2003. Information Systems Research: A Matter of Choice? South African Computer Journal 30: 1-2. Roode, D 2004. The Role of Empirical versus Conceptual Approaches. Presentation at Doctoral Seminar: Research Methodologies in Informatics. January, 2004. University of Pretoria. Schon, DA 1987. Educating the Reflective Practitioner. San Francisco: Jossey-Bass Publishers. Simon, HA 1981. The Sciences of the Artificial. Cambridge, MA: MIT Press. Squires, D 1999. Educational Software for Constructivist Learning Environments: Subversive Use and Volatile Design. Educational Technology 39,3: 48-53. Strauss, AL & J Corbin 1990. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage. The Design-Based Research Collective 2003. Design-based Research: An Emerging Paradigm for Educational Inquiry. Educational Researcher 32,1: 5-8. Travis, J 1999. Exploring the Constructs of Evaluative Criteria for Interpretivist Research. In Proceedings of the 1(fh Australasian Conference on Information Systems 1999: 1037-1049. Urquhart, C 2001. Analysts and Clients in Organizations; A Conversational View. Journal of Strategic Information Systemsl0,3: 243-262. Urquhart, C 2002. Regrounding Grounded Theory - or Reinforcing Old Prejudices? A Brief Reply to Bryant. The Journal of Information Technology Theory and Application (JITTA) 4,3: 43-54. Van den Ak:ker, J 1999. Principles and Methods of Development Research. In van den Ak:ker, J, RM Branch, KL Gustafson, N Nieveen & T Plomp (eds): Design Approaches and Tools in Education and Training. Dordrecht: Kluwer Academic Publishers. Van den Ak:ker, J 2002. The Added Value of Development Research for Educational Development in Developing Countries. In Osaki, K, W Ottevanger, C Uiso & J van den Ak.ker (eds): Science Education Research and Teacher Development in Tanzania. Amsterdam: Vrije Universiteit, International Cooperation Center.

51

M.R. (Ruth) de Villiers Walsham, G 1995a. Interpretive Case Studies in IS Research: Nature and Method. European Journal ofInformation Systems 4,2: 74-81. Walsham, G l 995b. The Emergence of Interpretivism in IS research. Information Systems Research 64: 376-394. Wang, F & MJ Hannafin 2005. Design-based Research and Technologyenhanced Leaming Environments. Educational Technology Research and Development 53,4: 5-23. Wood-Harper, T l 985. Research Methods in Information Systems: Using Action Research. In Mumford, E et al (eds): Research Methods in Information Systems. North-Holland: Elsevier Science Publishers B.V. Zuber-Skerrit, 0 1992. Action Research in Higher Education. London: Kogan Page.

Glossary Empirical: Based on the results of experiments and/or observations, not based on theory. Epistemology: Theory of the grounds of knowledge, how knowledge is produced, basis of claims to knowledge. Hermeneutics: The science of interpreting the intention of the original author or researcher. Methodology: A set of methods used in a process of inquiry. Ontology: The science of the essence of being; closely related to one's view of reality. Paradigm: The underlying philosophy and assumptions that form the foundation to one's approach and methodology. Substantive: Having a separate and independent existence, not merely inferential or implicit. MR (Ruth) de Villiers School of Computing University of South Africa Pretoria, South Africa

52

Interpretive Research Models for Informatics

Interpretive Research Models for Informatics

Suggest Documents

Interpretive Research Abstract - CiteSeerX

Interpretive Social Research - Universitätsverlag Göttingen

Biomedical Informatics for Cancer Research

Conceptual Models in Health Informatics Research: A Literature

Research in medical informatics

Aditya Informatics & Research Centre

Research Article - Informatics Journals

Clinical research informatics

Research in Informatics

Research Evaluation for Computer Science - Informatics Europe

NATIONAL CENTRE FOR DISEASE INFORMATICS AND RESEARCH ...

Biomedical Informatics for Cancer Research - EPDF.TIPS

Introducing Interpretive Approach of Phenomenological Research ...

Using Interpretive Qualitative Case Studies for Exploratory Research ...

Data Sharing in Interpretive Engineering Education Research

Interpretive and critical research: Methodological ... - Academic Journals

Using Interpretive Qualitative Case Studies for Exploratory Research

Research Trends in Media Informatics

Clinical Research Informatics - eScholarship@UMMS

Research Trends in Media Informatics

Staged Models for Interdisciplinary Research

Research Article Deformable Models for

Bio-PEPA for epidemiological models - Informatics Homepages Server

Interpretive Models of Guido of Arezzo's Micrologus - eScholarship