Multiple Methods and Multiple Paradigms for Digital Government Research: Understanding the Promises and Challenges J. Ramon Gil-Garcia
Theresa A. Pardo
Center for Technology in Government University at Albany, SUNY 187 Wolf Road, Suite 301 Albany, New York 12205 (518) 442-5573
Center for Technology in Government University at Albany, SUNY 187 Wolf Road, Suite 301 Albany, New York 12205 (518) 442-3984
[email protected] any.edu
tp
[email protected] any.edu
ABSTRACT The goal of this ‘Birds of a Feather’ session was to discuss some important issues in relation to the use of multi-method approaches or multiple methods in conducting digital government research. The discussion covered topics such as the definition of multi-method approaches, advantages and disadvantages of using multiple methods, practical challenges, and a discussion about combining methods with different philosophical assumptions.
Categories and Subject Descriptors J. 4 [Computer Applications]: Social and Behavioral Sciences – research methods, multiple methods.
methods (either quantitative or qualitative) are sometimes more suitable to understand specific aspects of digital government and information systems in general. In addition, some methods are strongly linked to specific paradigmatic views of the research endeavor (e.g., positivism). This discussion explored some of the promises and challenges of multi-method research designs. We explored the argument that there are advantages in using multiple methods, but there are also some limitations and compromises at the methodological and philosophical levels that researchers should make in order to combine certain methods. This discussion addressed questions such as: •
What is a multi-method approach?
•
What are the advantages of using multiple methods in doing research? Why?
Measurement, Design, Theory.
•
What are the main challenges in combining methods? Why?
Keywords
•
How do advantages and challenges differ according to different combinations of philosophical and methodological components (see Table 1)?
General Terms
Research Methods, Muti-method Approaches, Combination of Methods, Mixed Methods, Philosophical Assumptions.
1. Background for the session Recent literature presents several reasons for using multiple methods or multi-method approaches to study complex social phenomena. For instance, authors from different disciplines, including information systems (IS), have identified the desirability of using multiple methods and adding different disciplinary perspectives to the research endeavor. Digital government is clearly not uni-dimensional and researchers have to understand complex and recursive relationships between different theoretical constructs. However, single
Table 1. Different Combinations of Methodological and Philosophical Components Single Method and Single Paradigm
Multiple Methods within a Single Paradigm
Multiple Methods and Multiple Paradigms
2. Session Results There are different terms to refer to the use of multiple methods within a single research project: multi-method, mixed method, multiple methods, qualitative-quantitative design. The important consideration is that the combination of methods should be consistent with the substantive purpose of the study and the research strategy (triangulation, exploratory, explanatory, etc.).
Four primary advantages of multi-method research emerged from the discussion. 1. Getting full answers to increase the robustness of our understanding. 2. Validating our interpretations of what is happening in a particular environment – in a sense to triangulate results. 3. Validating methods. 4. Better integration of contributions from multiple disciplines, as well as the opportunity to feed back more directly into the disciplines themselves. A variety of challenges emerged from the discussion as well. Each of the challenges is discussed below. 1. The strength of multi-method research is potentially a weakness in terms of marketing it as a strategy. In our world not everyone wants to know everything that is actually happening – so many may not be receptive to such a strategy. 2. Having a full understanding as a result of triangulation of multiple methods thereby validating those methods might in fact be considered a detraction from a particular design b y increasing the transparency; you can claim a problem with a method – if multiple methods are used to validate findings. 3. Knowledge about how to do this – not enough people know how. Some disciplines are inherently interdisciplinary and have been doing multiple method research and integrating results for many years – for example, geography. This led t o a discussion about how to train people to think more broadly about research and about the kind of resources that need to be available to teach future researchers. One of the things that discussion brought us back to are the goals and the cost of any particular study – both in the training arena and in the design of a particular research protocol. For example, when you have 12 courses to teach in a doctoral program or in an MPA, how do you decide that these courses and others about multimethod and multiple paradigms are worth the cost to students? – and when we design these studies how do we balance outcome with costs? 4. Multi-method studies are expensive. A strategy for enabling multi-method studies emerged from the discussion of cost and benefit challenges. Minimize the investment in data gathering though aligning needs – what data do we need t o support a range of analytical methods was seen as one way to deal with the cost associated with multi-method studies. Working across multi-disciplinary research teams seeking alignment of data needs and using this new understanding to design data collection strategies that support the fullest range of analytical needs as possible. This makes more time and money available for the integration of findings. The final segment focused on a consideration of the philosophical perspective. It was agreed that in digital government research we are focused more on purpose and less on the philosophy of science. Why is something being done – what is the purpose of any particular digital government effort and what impact is it having? Researchers then tend to turn t o the philosophical perspective of their discipline – positivism for example, when designing a strategy to answer the original question. For us as DG researchers the more relevant research question is about purpose – what are we trying to determine
and then that takes us back to this question about having t o negotiate projects that represent full understanding versus an understanding that suits the purpose of the funders or the political environment. This discussion raised the question about uniqueness; essentially are the questions being raised unique to digital government research or do they apply in other fields of study? The political nature of the context of digital government seems to be a characteristic that digital government research shares with other disciplines that work in government settings such as political science and public administration. The “higher” up the focus of a digital government study is, the more political the digital government phenomena under study. This, it was agreed, may be the price for getting access to high level initiatives and agencies. The discussion closed with an agreement that the topic is i n need of continued discussion and that discussion should be closely aligned with the overall discussion of a digital government society.
3. Discussion Participants Sharon S. Dawes, Center for Technology in Government, University at Albany, SUNY J. Ramon Gil-Garcia, Center for Technology in Government, University at Albany, SUNY Ann Macintosh, International Teledemocracy Centre, Napier University Jeremy Millard, Competence and IT, Danish Technological Institute Theresa A. Pardo, Center for Technology in Government, University at Albany, SUNY Alexandros Xenakis, International Napier University
Teledemocracy Centre,
