Qualitative Research in Psychology 2006; 3: 293 311
Combining qualitative and quantitative methods in research practice: purposes and advantages Udo Kelle Philipps-University Marburg
Despite ongoing ‘paradigm wars’ between the methodological traditions of qualitative and quantitative research, ‘mixed methods’ represents nowadays a rapidly developing field of social science methodology. In such discussions it is often emphasized that the use of methods should be predominantly influenced by substantive research questions, and not only by methodological and epistemological considerations. As all methods have specific limitations as well as particular strengths, many discussants propose that qualitative and quantitative methods should be combined in order to compensate for their mutual and overlapping weaknesses. However, although a variety of proposals have been made for a taxonomy of mixed-methods designs, there is yet a lack of agreement regarding basic concepts and definitions, as is bemoaned by many experts in this field. This lack of common ground is due to the fact that crucial questions regarding the relations between research domains and methods have been not sufficiently discussed yet. For which types of research questions qualitative and quantitative methods are suited better? What are typical weaknesses and strengths of qualitative and quantitative methods in relation to particular research domains? The paper addresses these questions by discussing several examples from research projects that have combined qualitative and quantitative methods. Thereby it will be shown that the purposes of method integration are twofold: it can serve for the mutual validation of data and findings as well as for the production of a more coherent and complete picture of the investigated domain than monomethod research can yield. Qualitative Research in Psychology 2006; 3: 293 311 Key words: life course research; qualitative methods; quantitative methods; social research methods; triangulation Correspondence: Udo Kelle, Social Research Methods, Department of Sociology, Philipps-University Marburg, Ketzerbach 11, D-35032 Marburg, Germany. Email:
[email protected]
# 2006 SAGE Publications
10.1177/1478088706070839
294 U Kelle
Introduction Looking at methodological debates one may think that the long-lasting ‘paradigm wars’ (Gage, 1989) between the quantitative and the qualitative camp have lost much of their attraction in the past years. Many of the old war-horses seem to have become tired of repeating their arguments again and again. Moreover, proponents of a new trend towards ‘mixed methods designs’ (Tashakkori & Teddlie, 2003) claim that this movement marks the beginning of a new era in social research indicated by a tendency to combine quantitative and qualitative methods pragmatically unencumbered by old controversies. Mixed methods becomes a kind of fashion in social research. However, the efforts to combine qualitative and quantitative methods often lack a solid methodological basis in research practice, as recent meta-analyses about mixed method studies have shown (Bryman, 2005): researchers frequently combine quantitative and qualitative methods without providing a clear rationale for their choice of methods. And qualitative and quantitative findings are often not integrated in a coherent way when results from such research projects are presented. Reasons for this unsatisfactory situation do not only lie in a lack of competencies of empirical researchers but also in the state of the methodological discussions about mixed methods. In this debate it is often emphasized that the use of methods should be predominantly influenced by substantive research questions, and not by methodological and epistemological considerations alone. Moreover, it is maintained frequently that all methods have specific limitations as well as particular strengths, and that qualitative and quantitative methods
should be combined in order to compensate for their mutual and overlapping weaknesses (Johnson & Turner, 2003: 299). However, although a variety of proposals have been made to construct a taxonomy of mixed-methods designs, there is still a lack of agreement regarding basic concepts and definitions, as is bemoaned by experts in this field (Tashakori & Teddlie, 2003). This lack of a common ground is due to the fact that crucial questions regarding the relation between research domains, research questions and research methods have not been answered yet. For which types of research questions are qualitative and quantitative methods better suited? What are the typical weaknesses and strengths of qualitative and quantitative methods in relation to particular research domains? One would be well advised here not to put aside the controversy between the qualitative and quantitative tradition too early. If classical arguments developed by adherents of one tradition in order to highlight the methodological problems of the competing tradition are regarded as a part of outmoded ‘paradigm wars’ and neglected for that reason, important methodological questions and issues could be overlooked. The problem with arguments stressing methodological limitations of qualitative or quantitative research is not their mere existence, but that they are often not answered by proponents of the competing tradition. ‘Paradigm warriors’ tend to respond to critical claims concerning their own tradition in a tit-for-tat way, by emphasizing problems of the other tradition. In this way one can easily disregard obvious weaknesses of one’s own tradition. However, as both traditions are seriously challenged by certain aspects of social reality and carry methodological limitations, this style of
Combining qualitative methods in research practice
discussion exerts harmful effects: arguments that point to such restrictions and thus could be used to improve and to further develop methodological concepts and tools are neglected. This leads to the unfortunate situation that the potential of both the qualitative and quantitative research tradition to cope with methodological problems of the competing tradition is not utilized. In this paper I will show how mixedmethods research designs can be used to overcome weaknesses of qualitative and quantitative methods by complementary strengths of each tradition. By drawing on examples from research practice some classical problems of qualitative and quantitative research often discussed during the ‘paradigm wars’, as well as solutions for these problems developed in the framework of mixed-method designs, will be presented. Thereby the following problem areas will be addressed: . problems of theory building and measurement in quantitative research which may result from insufficient and incomplete theoretical concepts, eg, a failure to identify explanatory variables, misspecification of models etc., or problems which stem from inadequate operationalization procedures; . problems of case selection and transferability in qualitative research . In the concluding part of the paper the examples from research practice presented in the second part will be related to broader methodological considerations. For this purpose four classical forms of mixed-methods designs comprising the sequential or parallel use of qualitative and quantitative methods will be described, and functions of method combination within these designs will be discussed.
295
Using mixed-methods designs to cope with problems of theory building and measurement in quantitative research During the ‘paradigm wars’ a variety of crucial problems of theory building and measurement in quantitative research were identified and discussed by exponents of the qualitative tradition. Actually, the development of and great interest in qualitative methods in the 1960s and 1970s was fuelled by a critique of the prevailing mainstream of survey research of that time. In a variety of papers and books written by outstanding sociologists such as Herbert Blumer or Aaron Cicourel, quantitative researchers were blamed for their alienation from the investigated social world. While sitting at their office desk, musing about theoretical elaborations and developing complex questionnaires, researchers may easily lose contact with the investigated social world. This kind of research would lack a ‘first hand involvement with the social world’ indispensable for any adequate understanding of social phenomena, as Filstead (1970) put it. This critique referred to a methodological concept underlying sophisticated forms of quantitative research: the hypotheticodeductive (HD) approach, which requires the formulation of precise hypotheses and the operational definition of variables as a necessary precondition of empirical research. A crucial argument against the employment of an HD approach in social research is that the understanding of social phenomena would require knowledge about context-bound patterns, structures and rules characteristic for particular social life worlds. Such information usually forms an integral part of culturally specific stocks of everyday knowledge. In developing their theories, in formulating hypotheses and in
296 U Kelle
constructing research instruments like standardized questionnaires, social researchers employing an HD approach often utilize their personal commonsense knowledge (Kelle & Lu¨demann, 1998). In many cases, this heuristics of commonsense knowledge causes no major problems, especially if research takes place within the researcher’s own culture. But, as a great deal of commonsense knowledge is self-evident or implicit, the application of this heuristic is usually not discussed explicitly instead it serves as a ‘shadow methodology’ of theory construction. The hazards of this strategy become obvious if the sociocultural backgrounds of researchers and research subject differ to a certain degree. This can easily be the case if the research subjects belong to another social class, gender or ethnic group than the researchers. Not being a member of the culture of the research subjects, researchers have no access to culture-specific stocks of knowledge to formulate hypotheses, to define variables and to construct research instruments. The growing complexity and heterogeneity of modern societies may lead to increasing difficulties for HD research, as it will raise the probability that research subjects have a different sociocultural background than the researcher. A variety of methodological problems of quantitative research may arise from a lack of knowledge about culture-specific local structures, patterns and social phenomena. First of all, such insufficient knowledge may result in problems of theory building and hypothesis construction, leading to mis-specification of statistical models. That means that . important explaining variables are omitted with low levels of explained variance as a consequence; and
. intervening variables are neglected or functional relations between certain variables are not correctly specified, so that deeper causal processes underlying the investigated phenomena are not adequately understood. Furthermore, a limited knowledge about the investigated life world may bring about problems of operationalization and measurement : researchers who do not know how specific words and actions are understood in the sociocultural contexts of their research domain may ask the wrong questions or misinterpret the respondents’ answers in questionnaires. In the following it will be demonstrated how such problems can be identified and solved within mixed-methods designs. Problems of theory building and model specification To discuss typical problems of theory building and model specification in quantitative research we will draw on a panel study carried out to investigate the status passage from school to the labour market in Germany in the 1990s (Heinz et al., 1998; Kelle & Zinn, 1998). From the top ten training occupations, two crafts, two office occupations and a technical industrial occupation were selected. All school leavers in two cities who had started vocational training in one of these occupations were interviewed with standardized self-administered questionnaires in four subsequent waves to collect data about their occupational careers. From the large quantitative sample a small subsample (n /120) was drawn and qualitative (semi-structured) interviews were conducted in three waves focusing on work experiences, aspirations and reflections on careers. The quantitative data showed interesting differences with regard to the respondents’
Combining qualitative methods in research practice
tendency to re-enter the educational system four years after having finished vocational training. Almost immediately after their apprenticeship 15% of the apprentices either went back to school to attain further educational certificates or entered technical colleges or universities (Table 1). To explain these differences two variables were used in the beginning: the formal level of schooling of the apprentices and their occupational aspirations . Due to the German three-tier school system only those young people who have passed the highest school-level exam may enter university, and thus we assumed that especially those apprentices who had already gained a high formal qualification in school before their vocational training would make further educational efforts. Furthermore, data about the respondents’ occupational aspirations were collected by using different scales for the measurement of work orientations (cf. Schaeper & Witzel, 2001) supplemented by additional items developed from the analysis of the qualitative interview data. Items referred to the respondents’ requirements and subjective relevancies regarding their career and job environment and to the signi-
297
ficance of work compared to other spheres. By using cluster analyses a typology of seven different ‘modes of biographical orientation’ was developed, encompassing: . company identification, . wage worker habitus, . optimizing opportunities for personality development, . career involvement, . need for security, . holistic ambitiousness, and . unpretentious acceptance of necessities. We expected a high tendency towards further educational achievements among three groups: among those with career involvement apprentices with a strong orientation towards career and advancement and with high economic requirements; among those who were interested in optimizing their opportunities for personality development ; and among those with holistic ambitions. Members of the clusters company identification , wage worker habitus and unpretentious acceptance of necessities showed a comparably low tendency to make further educational efforts.
Table 1 Occupational status of apprentices in six different professions (school leaver cohort 1989) in Bremen and Munich four to five years after vocational training. For reasons of readability the residual category (containing categories like without job, on sick leave, pregnant, maternity leave, imprisonment, abroad, military service ) has been omitted. Therefore the row percentages do not add up to 100% in occupation trained for bank executives (229) office workers (319) indust. mechanics (177) car mechanics (103) hair dressers (80) retail sales persons (130) total
53.7% 60.2% 40.7% 37.9% 42.5% 43.8% 49.8%
(123) (192) (72) (39) (34) (57) (517)
in other occupations 9.6% 22.6% 23.2% 37.9% 33.8% 36.2% 23.9%
(22) (72) (41) (39) (27) (47) (248)
back to school 0.4% (1) 0.3% (1) 10.2% (18) 1% (1) 2.5% (2) 0 2.2% (23)
college/ university 31.9% 6.9% 14.1% 5.8% 1.3% 5.4% 12.9%
(73) (22) (25) (6) (1) (7) (134)
298 U Kelle
However, multivariate analyses showed that the explanatory power of these variables was not as high as expected. Furthermore, it turned out to be difficult to explain the striking differences we found between occupational groups regarding educational behaviour. Especially bank executives and industrial mechanics showed a high tendency to re-enter the educational system: four years after the end of their first apprenticeship 30% of the bank executives and almost one quarter of the industrial mechanics had made attempts to attain further educational qualifications. As far as the bank executives are concerned, this fact can be explained through the variables we had already chosen: school-level attainment and occupational aspirations. The access to certain white-collar occupations, such as banking, tends to be restricted to school leavers with higher formal qualifications like Mittlere Reife (a school leaving exam that can be attained after 10 years in the socalled Realschule ), Handelschulabschluss (Realschulabschluss plus one or two years of additional schooling in special commercial schools) and Fachabitur or Abitur (12 or 13 years of schooling with higher education entrance qualifications). Less qualified young people coming from the Hauptschule (9 years) are referred to apprenticeships in the trade, crafts or industrial domain. Multivariate analyses showed that the tendency of bank executives to enter university after their apprenticeship can be explained by the fact that a high proportion of them had achieved the Abitur. Many school leavers with the highest German school exam obviously regard the vocational training as a bank executive as a part of a career path leading to a university certificate in business administration. Regarding their occupational aspirations, bank executives are over-represented in
the clusters with a high tendency towards further educational achievement, namely career involvement and optimizing opportunities for personality development (see Table 2). Thus the initial model can explain the behaviour of bank executives quite well. This is not the case with the industrial mechanics their tendency to gain additional qualifications is independent of their level of attainment at school. Actually, most of them had to go back to school first in order to achieve the Abitur. Furthermore, respondents from occupations with similar low educational status, namely office workers, hair dressers, shop assistants and car mechanics, rarely make further educational attempts. In addition, industrial mechanics often show occupational orientations and aspirations with low tendency towards further educational achievement. Industrial mechanics belong to the clusters unpretentious acceptance of necessities and company identification more often than members of any other group. As we had no data available about what the respondents’ aspirations had been before the apprenticeship had started, the direction of the causal link was not clear the correlation between occupation trained for on the one hand and occupational aspirations on the other hand could be interpreted in two ways: either students with low aspirations tend to choose the apprenticeship as an industrial mechanic or the apprenticeship itself could lead to a lowering of aspirations. To understand the findings from the quantitative data, information about norms relevant for occupational life worlds had to be used, which could be found in the qualitative data: industrial mechanics were often trained in large companies in the old core industries, especially in engine building and the automobile industry. The
/0.64*** /0.24 /0.02 /0.34 /0.86*** /0.30 /0.50*** /0.66*** /0.14 /0.03 /0.78* /0.46 /0.46* /0.38*** /0.51*** /0.25 /0.19 /0.09 /0.26 /0.29 /0.49*** /0.08 /0.03 /0.09 /0.06 /0.01 /0.08 /0.06 /0.19 /0.38 /0.82*** /0.63** /1.77** /0.14 /0.35** /0.32 /0.19 /0.06 /0.49** /0.30 /0.60** /0.06 /0.45* /0.16 /0.23 /0.25 /0.39 /0.24 /1.11*** Occupation trained for bank executives office workers industrial mechanics car mechanics hair dressers retail sales persons gender : female
4 security orientation 3 unpretentious 2 wage worker habitus 1 company identification
Covariate
Table 2 occupational aspirations and occupation trained for
Cluster
5 career involvement
6 optimizing opportunities
7 holistic ambitions
Combining qualitative methods in research practice
299
trainees there work under the supervision of highly qualified master craftsmen and are equipped with a variety of specific skills. Furthermore, in Germany the industrial mechanic has always been regarded as one of the most prestigious occupations in this sector, representing, especially in earlier times, some sort of working-class aristocracy. After their vocational exam, most of our industrial mechanics were offered permanent contracts by their companies. But, because at that time the German enginebuilding industries were hit by a period of economic slowdown, their work situation was much less privileged than the training had been. While a small minority found jobs with challenging tasks (like monitoring the production process or repairing machines), the majority had to perform work of unskilled and semi-skilled workers. In the qualitative interviews many industrial mechanics showed disappointment with this situation, and some of them made attempts to gain higher qualifications. With this information drawn from the qualitative data, we could explain a complex interaction effect: a minority of industrial mechanics who had already developed a biographical orientation mode like optimizing opportunities or career involvement saw work as the crucial domain in life and as a means of self-fulfilment, and expected a high degree of variation and alternation in their daily work. In an occupational context characterized by routine work and restricted career prospects, these respondents undertook further educational efforts. Many of those who stayed in their occupation developed orientations like company identification or workman’s habit after a certain period of time: they regarded good working conditions and good salary as most crucial. By using qualitative data we obtained a clearer understanding about the causal
300 U Kelle
processes that produced a close statistical association between a certain occupation and low occupational aspirations (see Figure 1). Furthermore, by using the access to local knowledge of occupational cultures provided by our qualitative interview data we could add two additional context variables to our initial model, making it more realistic and improving its explanatory power. These were the variables ‘socialisation effects of occupational culture’ and ‘economic situation of the industrial sector’. In order to test this new model for the other occupations one would certainly need additional data (eg, about occupational cultures in other occupational fields, or data about the economic cycle in the other sectors). Problems of operationalization and measurement There is another important part of the quantitative research process that requires the availability of culture-specific knowledge the operationalization of theoretical concepts and the development of measurement instruments. A meticulously constructed questionnaire may yield an invalid and highly misleading picture of the investigated domain if research subjects understand a question in a different way than the researchers, or if the topics treated
vocational training: industrial mechanic
are not relevant for the respondents. Such operationalization problems in quantitative surveys are related not only to the meaning of words or phrases or to the relevance of themes of a questionnaire, but to the whole process of interviewing likewise. Interviewing cannot be regarded as a mere process of information exchange with the interviewee as a passive provider of data. Furthermore, it represents a complex social interaction process, which can be defined and framed in highly different ways by the involved actors. Thereby, interview partners act according to their motives and goals, which may differ from the purpose of merely disclosing personal information, attitudes etc. As with other forms of social interaction, actors may hide their motives and intentions, they may disguise or conceal certain facts, they may hold back important information, they may invent fictitious events, and so forth. Furthermore, an interview, like any other form of social interaction, is dependent on the ability of the involved actors to interpret the action and motives of their counterparts and to identify and understand reciprocal expectations. This can be a precarious process with the imminent risk of misunderstandings and more or less hidden conflicts. Consequently, unintended misunderstandings and accidental mistakes as well as wilful
socialisation in occupational culture
+
+ economic situation of the industrial sector
+
+
+
further educational efforts
-
occupational aspirations: unpretentious acceptance of necessities
Figure 1 Causal model of the relation between vocational training, macroeconomic influences and educational efforts of respondents
Combining qualitative methods in research practice
omissions or deceits represent significant threats for the validity and quality of interview data. The following data and results taken from research about residents’ satisfaction in care homes show how such threats for validity can be identified and treated with mixedmethods designs (Kelle & Niggemann, 2002, 2003). The main purpose of this study was to identify dimensions of residents’ satisfaction and needs in care homes and to identify methodological problems arising during the collection of data about satisfaction and quality of life of care home residents. For this purpose we performed semistructured, mostly qualitative interviews (n /40) with residents of 12 residential care homes in north Germany of varying size and ownership from the private sector and run by voluntary organizations. These interviews were audio-taped and fully transcribed, and were coded with a category scheme developed during data analysis. Apart from that we conducted standardized questionnaires with 244 residents of 14 different residential care homes in the whole of Germany of varying size and ownership. Sixty of these interviews were taperecorded and transcribed. By combining these different quantitative and qualitative data sources (quantitative data from standardized questionnaires as well as transcribed interaction protocols from these interviews, and verbal data from open-ended qualitative interviews) we could identify highly interesting divergences. Data collected with standardized questionnaires showed a very positive image concerning residents’ satisfaction. When asked ‘if they are satisfied with life in the home’, nearly two thirds gave the answer that they were ‘fully’ or ‘quite’ satisfied (see Figure 2).
301
Satisfaction with life in care home 120
100
80
60
40
20
0 not at all not very much
partly
completely quite (a lot)
no answer
Figure 2 Satisfaction of care home residents
This tendency to express satisfaction is not surprising at first glance; it has been often been reported in the literature that recipients of long-term residential care express high satisfaction in standardized interviews (Kelle & Niggemann, 2003). But if one takes into account the verbatim interaction protocols of the interviews, one clearly sees that there may be also other reasons. ‘I can’t always call somebody to help me to get washed, so I do it on my own as best I can’, a 94-year-old female respondent told us, who had said that she was ‘fully satisfied with life in the care home’, and, answering another question ‘well you see, they just don’t have enough time’. Similar statements were expressed by many interviewees. But despite taking a critical stance towards many aspects of care, respondents judged their overall satisfaction as very positive. A majority of respondents even approved items that we had included to provoke negative answers by an exaggeration like ‘The care is perfect and nurses are really
302 U Kelle
always friendly’ ‘completely’ or ‘quite a lot’. The interaction protocols show that such answers are often influenced by strong feelings of dependency. In many cases residents obviously avoid criticizing staff members directly and choose categories that do not reflect their real judgements. ‘It was better in the beginning, but now there’s a shortage of staff, but I wouldn’t want you to write that down’, an 89-year-old respondent said. And a 90-year-old resident broke off the interview after saying: ‘A very tricky question, it is not something one talks about. Here, people who speak the truth are punished.’ These results were supported by the qualitative data collected with semi-structured interviews, which helped to describe typical methodological problems of interviewing people in long-term care: . open or hidden refusal of interviews to avoid anticipated sanctions; . concerns regarding anonymity; and . a strong tendency to socially desirable answers. Especially residents who had been selected as potential interview partners by the management or staff rarely refuse participation. However, forms of hidden refusal often become obvious during an in-depth interview. One interviewee apparently felt the strong desire to refuse the interview in the beginning. ‘Yesterday it was pointed out to me again that I would be visited by a lady today,’ he says, ‘and I said to them, tell her not to bother.’ The reason for that was: ‘Because I was interrogated two years ago... interrogated with my doctor. Under the same conditions as today.’ Contrary to the definition of the situation the interviewer had put forward emphasizing that this was an interview for scientific purposes the respondent used the German word
‘Verho¨r’, which means the interrogation of a suspect in a police station or a court room. Like a person subjected to an interrogation, the interviewee felt that he had almost no freedom to decide about his participation and that he must be aware that everything he said could be used against him. Another interview partner expressed her fears of being expelled by saying: ‘If I say something here and I lose my accommodation, I wouldn’t know where to go. . . It’s better to say nothing. . .’. From a methodological perspective it is not so important whether such anticipated sanctions would really take place in the particular care home the respondents’ concerns may be highly exaggerated. But the question concerning us here is how the subjective experience of the interview as frightening may lead to distorted results if one attempts to measure residents’ satisfaction. It is obviously difficult for our respondents to express critical attitudes or negative judgements towards other actors in the institution they live in. Thereby, many of the interview partners expressed concerns that the interviewer may pass information to the management or staff. However, qualitative methods are especially suited to bring interview partners out of an initial reserve. If a positive personal relationship between the interview partners developed, even respondents who were extremely cautious in the beginning were willing to report about negative experiences after some time. One interviewee initially refused to answer all questions that could in any way be related to problematic events. ‘As I said, in no shape or form do I have any reason to complain about anything whatso ever.’ But such efforts to present the care home in a positive light were interrupted by short narratives describing concrete experiences of discontent. ‘At most that the care
Combining qualitative methods in research practice
staff are a bit nervous and that they are a bit loud with someone who keeps calling ‘‘nurse’’ all day.’ Interestingly, these sentences were followed by another passage, which emphasized the residents’ overall satisfaction. Obviously the level of mutual trust and understanding between the interview partners was crucial for the quality of the data. Thereby qualitative interviews turned out to be especially helpful, as they allowed for detailed narratives and thereby could fulfil demands of the respondents for communication and narrative self-presentation much more than standardized questionnaires. Interviewers could use probing questions to encourage the interviewee to report further details and reflections. Especially, a style of interviewing where the interviewer directly referred to the interview partner’s narratives can give them a feeling of being taken seriously. Many of the interviewees thus developed their own perspectives, and sensitive topics could be addressed in a considerate manner by using cautious formulations to make the recall of negative experience easier. By such means the use of qualitative methods can lead to the conclusion that certain quantitative instruments, scales or items are not improvable even by the most sophisticated techniques of questionnaire construction. Consequently, qualitative methods may yield useful ‘negative information’ about the validity of standardized data. Problems of case selection and transferability in qualitative research From the beginning of the qualitative tradition, statisticians blamed qualitative researchers for not providing a basis for sound generalizations because of the lack of representativeness of small n studies. For the first time this critique was concisely
303
brought to a point by Lundberg in his review of the famous study of William Thomas and Florian Znaniecki, The Polish peasant in Europe and America : The scientific value of all these (studies) depends, of course, upon the validity of the subjective interpretations of the authors as well as the extent to which the cases selected are typical. Neither the validity of the sample nor of the interpretations are objectively demonstrable on account of the informality of the method. (Lundberg, 1929/1942: 169)
Many attempts have been made in the history of the qualitative research tradition to counter such criticism. Unfortunately many of the arguments of the qualitative side that attempt to address the problem of generalizability lack coherence and a solid epistemological ground. The earliest answers to the allegation of lacking generalizability were formulated in the 1930s and 1940s, when many qualitative researchers were deeply influenced by structuralist thinking. (Interestingly, these arguments are still common today in particular qualitative schools of thought like conversation analysis or the German ‘objective hermeneutics’.) Starting with a distinction made by Florian Znaniecki (1934) between mere ‘enumerative induction’ and ‘analytic induction’, it was argued that in qualitative research a kind of generalization must be applied superior to statistical inference and which does not depend on the number of cases, ‘but on the strength of the theoretical reasoning’ (Seale, 1999: 109). The basic idea behind this concept of ‘theoretical generalization’ is that some sort of general social process or social structure is at work in any single case, which can be revealed by a deep, penetrating and elaborated theoretical analysis. A thorough qualitative analysis may thus determine essential structural characteristics and exclude accidental aspects even if
304 U Kelle
idiosyncratic cases are investigated. This idea, developed initially by Florian Znaniecki to justify small n studies, can be even traced back to Emile Durkheim’s analyses of macrosocial phenomena: following Durkheim, all social facts and entities are brought about by evolutionary processes, such that simple forms of social life always contain essential structures shared also by more complex forms (cf. Seale, 1999: 112). Nowadays very few sociologists, and only a small minority of qualitative-oriented researchers, would hold to such concepts macrosociological evolutionism has fallen into disuse in theoretical debates in sociology and social psychology. Also the concept of theoretical generalization rests on assumptions about the stability and universality of social rules not shared by many sociologists nowadays. Especially microsociological approaches like symbolic interactionism, which formulated important theoretical fundamentals for qualitative research in the 1950s and 1960s, have cast this idea into doubt. The tradition of interpretive sociology maintains the role of interpretation in social action and interaction and the resulting complexity, variability and uniqueness of social phenomena. Social structures are considered as mutable and flexible as social actors use action spaces to develop new forms of social practice. Following this idea, social processes as well as human history as a whole have to be considered as contingent and unpredictable: ‘uncertainty, contingency, and transformation are part and parcel of the process of joint action’ (Blumer, 1928: 72). There are no general rules and ahistoric laws in society that are valid regardless of sociohistorical contexts. In the 1980s followers of poststructuralist or postmodern approaches in qualitative
research like Norman Denzin, Egon Guba or Yvonna Lincoln have attempted to radicalize this idea: in their opinion the acknowledgement of the principle of context-boundedness of social phenomena and processes must lead to an overall rejection of the idea of generalization in social research (Lincoln & Guba, 1985). However, when attacks of statisticians on qualitative research are countered in such a way, it is usually forgotten that this stance would lead to an exclusion of a great variety of social phenomena from qualitative inquiry. Many macrosocial phenomena (for instance, the existence of cultural norms) cannot be studied on the basis of limited numbers without any reference to an idea of generalization. That would especially be the case if one takes seriously the claim of the interpretive tradition that social order is highly flexible and evolves through processes of interpretation taking place in microsocial contexts. The resulting differentiation, pluralization and heterogeneity of social structures and patterns of action pose serious challenges for any methodological approach that has to rely on the investigation of a small number of cases. Certainly one has to maintain, contrary to what dogmatic quantitativists may say, that case studies are indispensable tools in many domains. The investigation of a single person, group and organization may be a research goal justified in itself. However, there is always the imminent danger that researchers focus on remote and marginal cases. The crucial question here concerns the scope or the research domain to which kinds of people or organizations does the knowledge derived from a certain study also apply? In methodological discussions within the qualitative tradition the term ‘transferability’ was coined to address such
Combining qualitative methods in research practice
questions. However, this concept refers to a kind of reasoning which is not that far away from the idea of generalizability. After all, this idea does not automatically imply generalization towards universal structures or universal laws valid in all places at all times. Questions of transferability or limited generalization may play an important role, even if the purpose of research is not to identify sociological universals but cultures, societies, organizations etc. as limited wholes situated in concrete spatiotemporal contexts. It concerns questions like: Do certain resentments of members of a specific branch of an organization reflect deeper problems of the whole organization or of organizations of a certain kind, or are these resentments only the expression of the situation in that specific branch? Are certain rules of behaviour followed by inhabitants of a certain village generally accepted habits in that village, or even in all villages inhabited by members of a certain ethnic group? If holistic inquiry focusing on cultures and organizations as limited wholes is conducted in a social life world with heterogeneous norms and patterns of action, one does clearly benefit from the application of quantitative methods even if one does not adhere to the idea of universal social laws. Proponents of interpretive sociology have stressed this advantage of statistical methods. As early as 1928 Herbert Blumer defended the statistical method against adherents of sociological structuralism who had criticized statistical methods for their inability to disclose universal laws. Contrary to the search for sociological universals, the statistical method takes into account at least implicitly the ‘complexity, variability or uniqueness’ of social phenomena (Blumer, 1928: 47f.) and thus the changeability and flexibility of social
305
phenomena stressed by interpretive approaches. The interpretive tradition in sociology thus provides strong arguments in favour of ‘the importance of statistical analysis’ (Hammersley, 1989: 219), especially as quantitative methods are well suited to capture heterogeneity and variance by making it easy to collect information about great numbers of persons or situations. Mixed-methods designs can offer different possibilities to overcome limitations of qualitative research regarding the transferability of findings. One possibility has already been discussed in classical writings (cf. Barton & Lazarsfeld, 1955/1984): findings from qualitative studies with small numbers of observations and interviews in a limited domain may be further examined and tested in large-scale quantitative surveys. Unfortunately this strategy is often falsely understood such that it assigns an insignificant role to qualitative research, restricting it to more or less unsystematic pilot studies. However, qualitative inquiry will only yield useful results if it is accomplished in a thorough and systematic way which ensures that descriptions and theoretical explanations are really empirically grounded in the (possibly small number of) investigated cases. Given that a careful and meticulous qualitative inquiry requires a lot of resources in terms of time and personpower, the selection of cases in such a study is of utmost importance. The discussion within the qualitative tradition regarding methodological and epistemological rationales for small n research is helpful here despite its conceptual deficiencies mentioned above. This especially refers to the extended debate about practical aspects of case selection in qualitative research, which started with publications about the selection of ‘crucial cases’ in the 1940s and 1950s (Cressey, 1950, 1971;
306 U Kelle
Lindesmith, 1947/1968). In the subsequent decades various forms of purposive sampling like the concept of ‘theoretical sampling’ (Glaser & Strauss, 1967) were proposed and extensively discussed and their utility was demonstrated in numerous studies. Thereby the basic idea of theoretical sampling, the maximization and minimization of differences, also informs other forms of purposeful sampling proposed in the methodological literature, which cover a wide range of different strategies, for instance the search for extreme, deviant or typical cases (Patton, 1990: 182; also Gobo, 2004 or Silverman, 2000: 102 f.). Quantitative research results may inform all types of purposeful sampling. First of all, a quantitative study can provide an overview about the existence and the distribution of certain types of social problems, structures or patterns of action prevalent in the investigated domain. Furthermore, a quantitative study may provide a sampling
frame that allows for the comfortable selection of typical, deviant or extreme cases. An investigation about how elderly people experience their decreasing mobility carried out by Mollenkopf and Baas (2002) may serve as a good example. The investigators obtained standardized questionnaires with a large sample of elderly people asking them about their health situation and outdoor mobility. On the basis of these data four types of elderly persons were identified: the great majority of respondents belonged to two groups interviewees with poor health and limited mobility on the one hand and older people with good health and great mobility on the other hand. Two other yet smaller groups also could be identified: a minority of respondents with good health left their homes only occasionally, and a very small group with a bad health situation showed nevertheless high mobility (see Figure 3).
good health, good mobility
good health, poor mobility
Health
Mobility
poor health, restricted mobility
Figure 3 Mobility and health status of elderly citizens
poor health, good mobility
Combining qualitative methods in research practice
Qualitative in-depth interviews were carried out with a small number of respondents in each group to identify how they experience their health and mobility situation. In doing so the frequency distribution of the four types found in the large quantitative sample was not matched by the qualitative sample. It was the smallest of the four groups, respondents with considerable health problems and high mobility, that was scrutinized most intensively, as members of this group had developed successful coping strategies the researchers were interested in.
Different mixed-methods designs and their function in the research process A combination of qualitative and quantitative methods could help to practically overcome limitations and to solve problems of mono-method research discussed in the literature for more than 50 years. An important question would be whether it is possible to develop concrete methodological strategies for constructing research designs based on that insight. However, this question is rarely addressed in current debates about the combination of qualitative and quantitative methods (Creswell et al ., 2003; Greene & Caracelli, 1997; Morse, 2003; Tashakkori & Teddlie, 2003). Apart from the broad agreement that a, use of multiple methods with complementary strengths and different weaknesses in relation to a given set of research problems, the discussion provides only sparse information about which designs could overcome which weaknesses of mono-method research. In this section I will present some basic ideas about functions of different types of mixed-methods designs for that purpose. Unfortunately, there is still a lack
307
of consensus about the exact terminology and nomenclature of different ‘mixed-methods’, ‘multiple method’ or ‘multi-method designs’ which are used in research practice (Tashakkori & Teddlie, 2003). I will refer to a differentiation made in an often cited paper by Morse (1991), who proposed to distinguish between the simultaneous and sequential use of qualitative and quantitative methods. The design acknowledged even by the fiercest paradigm warriors of the quantitative tradition (but nevertheless used only occasionally by them) is a sequential qualitative quantitative design (qual 0/ quan), whereby a qualitative study helps to identify core issues and to develop theoretical concepts and hypotheses, which can be further examined in a subsequent quantitative study that is carried out with the goal to find out whether concepts relevant in a comparable small number of cases describe and explain social phenomena in a greater domain accordingly. Such a design helps to overcome two remarkable limitations of mono-method research: the limited transferability of findings from qualitative small n research as well as the initially mentioned hazards of the heuristics of commonsense knowledge, which represent a crucial problem for quantitative HD research the lack of sociocultural ‘local’ knowledge that seduces researchers to apply concepts that fail to grasp the most relevant phenomena in the investigated field. By starting the research process with a qualitative study, researchers may obtain access to local knowledge that helps them to develop those theoretical concepts and hypotheses most suited for their domain and to construct standardized research instruments later on that cover relevant phenomena by meaningful and relevant items.
308 U Kelle
In many cases it can be very useful to combine qualitative and quantitative methods the other way round; that means by starting with a quantitative study, followed by a qualitative inquiry. In such a sequential qualitative quantitative design (quan 0/ qual) a quantitative study is performed in order to identify problem areas and research questions which have to be further investigated with the help of qualitative data and methods. The problem of quantitative research addressed by this design is the frequent incomprehensibility of statistical findings, which are often difficult to understand without additional sociocultural knowledge. The panel study about young workers and employees after their apprenticeship described earlier can serve as a good example for that. Furthermore, the quantitative part of a sequential quantitative qualitative design can guide systematic case comparison in the subsequent qualitative inquiry by helping to identify criteria for the selection of cases and by providing a sampling frame. Thereby this design can help to overcome an important threat for validity of qualitative research that researchers focus on remote and marginal cases. And a further problem of qualitative research can be addressed by this design: as a large-scale quantitative study can capture heterogeneity and describe the distribution of a set of predefined phenomena in the research field, it helps to avoid a qualitative study with an ‘oversized scope’, a study with a research question covering a domain too heterogeneous to be captured with the help of a small qualitative sample. To take an easy example: a qualitative study of family life in a modern city would have to take into account many more different forms of families than a similar
study in a traditional rural community in the first decades of the twentieth century. By drawing on statistical material about the distribution of different family forms, one can easily see the minimal requirements for qualitative sampling and may be well advised to downsize the research question and research domain (eg, to traditional core families in the middle classes living in suburbs) such that it can be covered by the planned investigation. A parallel qualitative quantitative design (qual/quan) can fulfil similar functions to a sequential design, yet in a sometimes restricted manner: the qualitative part of the design can yield information that helps to understand statistical associations, to develop explanations and to identify additional variables that increase variance already explained in the quantitative data. However, an important disadvantage of the parallel design is that qualitative sampling and data collection cannot be systematically developed from research questions derived from quantitative data therefore it can easily be the case that the available qualitative data provide no answers for questions coming from the quantitative study, as they were not collected for that purpose. A great benefit of a parallel qualitative quantitative design, however, is that it helps to identify measurement problems and methodological artefacts of both qualitative and quantitative data, as the same persons are interviewed with different techniques. The study about resident satisfaction in care homes gives an example for that: with the help of interaction protocols and qualitative interviews it became possible to discern biases arising from the respondents’ tendency towards social desirable answers.
Combining qualitative methods in research practice
Concluding remarks Mixed-methods designs provide important tools to overcome limitations of both qualitative and quantitative ‘mono-method research’: . In a sequential quantitative qualitative design quantitative research can help to guide the selection of cases in qualitative small n studies. . Results from qualitative interviews can help to identify unobserved heterogeneity in quantitative data as well as previously unknown explaining variables and misspecified models. . Results from the qualitative part of mixed-methods design can help to understand previously incomprehensible statistical findings. . Qualitative research can help to discover a lack of validity of quantitative measurement operations and instruments. . A quantitative study can help to corroborate findings from a qualitative study and to transfer these findings to other domains. Thus quantitative and qualitative methods can fulfil different yet complementary purposes within mixed-method designs. Quantitative methods can give an overview about the domain under study and can describe its heterogeneity on a macro-level, whereas qualitative methods can be used to gain access to local knowledge of the field in order to develop theoretical concepts and explanations that cover phenomena relevant for the research domain. Quantitative and qualitative methods thus cannot substitute each other, but help to illuminate different aspects of sociological phenomena: in a sociological investigation quantitative methods can describe the actions of
309
large numbers of different actors, whereas qualitative methods provide information about possible reasons for these actions. In such cases qualitative and quantitative methods help to answer different questions: the results of statistical analyses show what kinds of actions social actors typically perform (achieving certain school exams, going to universities and so on), while the analysis of qualitative data helps to answer whyquestions (eg, for what purposes do actors attend schools of certain levels, how do they perceive and define their situation, which norms do they acknowledge? and so on). Here qualitative and quantitative results are not interchangeable. It is not possible to analyse the aggregated results of social actions (eg, the overall rate of school leavers of a certain school type) with the help of qualitative interview data, whereas local knowledge typical for a certain culture or life world often cannot be investigated using standardized questionnaires, as the researchers do not have sufficient knowledge to construct such research instruments. As the application of qualitative methods to a yet unknown field carries the danger that researchers focus on remote phenomena and marginal cases, an important function of quantitative methods in mixedmethods research is to guide the selection of cases in the qualitative part of the study. Using a metaphor from geography and geology, one could say that quantitative methods provide us with a general picture of the surface of the research field, while qualitative research can be used to drill deep holes into the field yielding the information necessary for in-depth explanations. The problem of hazardous generalizations from small n studies can be further mitigated if quantitative methods are used for the corroboration of results
310 U Kelle
coming from a qualitative study. Best practice in mixed-method research thus comprises a chain of alternating steps of qualitative and quantitative research. Quantitative methods are used to describe the investigated phenomena and expand on a macro-level and to guide qualitative sampling. Qualitative research provides information necessary for fully fledged explanatory arguments, which can be further examined by subsequent quantitative research. Thereby further quantitative studies may lead to new questions, which require additional qualitative research.
References Barton, A.H. and Lazarsfeld, P.F. 1955/1984: Einige Funktionen von qualitativer Analyse in der Sozialforschung. In Hopf, C. and Weingarten, E., editors, Qualitative Sozialforschung . Klett-Cotta, 41 89. Blumer, H. 1928: Method in social psychology. Unvero¨f. Dissertation, University of Chicago. Bryman, A. 1988: Quantity and quality in social research . Routledge & Kegan Paul. Bryman, A. 2005: Why do researchers integrate quantitative and qualitative research? Paper presented at the Conference ‘Mixed Method Research Designs: Advanced Issues and Debates’ at the University of Basel, 13 August 2005. Cressey, D.R. 1950: The criminal violation of financial trust. American Sociological Review 15, 738 43.
Filstead, W.J. 1970: Qualitative methodology: firsthand involvement with the social world . Rand McNally. Gage, N. 1989: The paradigm wars and their aftermath: a ‘historical’ sketch of research and teaching since 1989. Educational Researcher 18, 4 10. Glaser, B. and Strauss, A. 1967: The discovery of grounded theory. strategies for qualitative research . Aldine. Gobo, G. 2004: Sampling, representativeness and generalizability. In Seale, C., Gobo, G., Gubrium, J.F. and Silverman, D., editors, Qualitative research practice . Sage, 435 56. Greene, J.C. and Caracelli, V.J., editors, 1997: Advances in mixed-method evaluation: the challenges and benefits of integrating diverse paradigms . (New Directions for Evaluation, No. 74). Jossey-Bass. Guba, E.G. and Lincoln, Y.S. 1985: Naturalist inquiry. Sage. Hammersley, M. (1989): The dilemma of qualitative method . Herbert Blumer and the Chicago tradition . Routledge. Heinz, W.R., Kelle, U., Witzel, A. and Zinn, J. 1998: Vocational training and career development in Germany results from a longitudinal study. International Journal for Behavioral Development 22, 77 101. Johnson, B. and Turner, L.A. 2003: Data collection strategies in mixed methods research. In Tashakkori, A. and Teddlie, C., editors, Handbook of mixed methods in social and behavioral sciences . Sage, 297 319.
Cressey, D.R. 1971: Other people’s money. a study in the social psychology of embezzlement . Wadsworth.
Kelle, U. and Lu¨demann, C. 1998: Bridge assumption in rational choice theory: methodological problems and possible solutions. In Blossfeld, H.-P. and Prein, G., editors, Rational choice theory and large-scale data analysis . Westview Press, 112 25.
Creswell, J.W., Plano Clark, V.L., Gutmann, M.L. and Hanson, W.E. 2003: Advanced mixed methods research designs. In Tashakkori, A. and Teddlie, C., editors, Handbook of mixed methods in social and behavioral sciences . Sage, 209 40.
Kelle, U. and Zinn, J. 1998: School-to-work transition and occupational careers: results from a longitudinal study in Germany. In Lange, T., editor, Understanding the schoolto-work transition: an international perspective . Nova Science Publishers, 71 89.
Combining qualitative methods in research practice Kelle, U. and Niggemann, C. 2003: Datenerhebung als sozialer Prozess in der Evaluationsund Wirkungsforschung-das Beispiel ‘Pflegequalita¨t’. Hallesche Beitra ¨ ge zu den Gesundheits- und Pflegewissenschaften 2. Retrieved from http://www.medizin. uni-halle.de/pflegewissenschaft/journal/ download.htm Lindesmith, A.R. 1947/1968: Addiction and opiates . Aldine. Lundberg, G.A. 1929/1942: Social research: a study in methods of gathering data . Longmans, Green. Mollenkopf, H., Baas, St. 2002: Konzepte, Strategien und Methoden zur Erfassung der Mobilita¨t a¨lterer Menschen das Beispiel des Outdoor Mobility Survey und Diary. In Motel-Klingebiel, A. and Kelle, U., editors, Perspektiven der empirischen Alter(n)ssoziologie . Leske und Budrich, 273 98. Morse, J.M. 1991: Approaches to qualitative quantitative methodological triangulation. Nursing Research . 40(2), 120 123. Morse, J.M. 2003: Principles of mixed methods and multimethod research design. In Tashakkori, A. and Teddlie, C., editors, Handbook of mixed methods in social and behavioral sciences . Sage, 189 208.
311
Patton, M.Q. 1990: Qualitative evaluation and research methods . Sage. Schaeper, H. and Witzel, A. 2001: Rekonstruktion einer qualitativen Typologie mit standardisierten Daten. In Kluge, S. and Kelle, U., editors, Methodeninnovation in der Lebenslaufforschung . Integration qualitativer und quantitativer Verfahren in der Lebenslauf- und Biographieforschung . Juventa, 217 59. Seale, C. 1999: The quality of qualitative research . Sage. Silverman, D. 2000: Doing qualitative research . a practical handbook . Sage. Tashakkori, A. and Teddlie, C. 2003: Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In Tashakkori, A. and Teddlie, C., editors, Handbook of mixed methods in social and behavioral sciences . Sage, 3 50. Thomas, W.I. and Znaniecki, F. 1918/1958: The Polish peasant in Europe and America . Dower. Znaniecki, F. 1934: The method of sociology. Rinehart.
About the author UDO KELLE is a Professor for Social Research Methods in the Department of Sociology at the Philipps-University Marburg. His research interests cover the epistemology and methodology of social research, and links between social research methodology and current trends and debates in contemporary social theory. He has published several books and numerous articles about qualitative and quantitative methods as well as about their integration.
