example, inferential statistics play no part in qualitative data analysis). The choice between quantitative and qualitative approaches needs to be made by asking ...
Chapter 8 Qualitative Methods of Data Collection While often it is possible to ‘quantify’ the findings of a study: to count the number of events, or measure the extent to which a variable is present or absent, or correlates with other variables, sometimes such quantification is either not possible or is not appropriate. In such situations, the quality of data is more important than quantities. This is particularly the case in the human and social sciences, where what is being explored are human attributes such as attitudes, beliefs or judgements. This chapter looks at how we can make such explorations rigorous and systematic Having successfully worked through this chapter, you will be able to: 1. Recognise the advantages and qualitative methods of research
disadvantages
of
2. Compare and contrast naturalistic and positivistic approaches to research 3. Identify the main methods of qualitative data collection 4. Compare and contrast issues of validity and reliability in quantitative and qualitative research.
Introduction Qualitative methods are sometimes criticised (usually by researchers committed to quantitative approaches to research) as ‘woolly’, ‘unscientific’, ‘journalistic’ or simply a waste of time and money. Poorly done, I would agree that qualitative research could be all these things (as could any research). As we will see in this chapter, the need to specify a research question, to refine that question to make it feasible, to identify concepts and indicators, and to find ways to rigorously and systematically collect valid and reliable data are all part of a qualitative methodology. In their detail they may be quite different from quantitative approaches (for example, inferential statistics play no part in qualitative data analysis). The choice between quantitative and qualitative approaches needs to be made by asking the questions: • Which is the most appropriate method to use to answer my research question? • Which method is the most adequate to provide the data which I need to collect?
7-1
In this chapter we will look at three methods of collecting data: Delphi Study A methodology that seeks to establish consensus among experts in a subject
Discourse A term used to describe talk, but more specifically , bodies of knowledge in a culture
•
participant observation (including ethnography)
•
interview-based research, including focus groups and consensus techniques such as Delphi methodology.
•
the study of documents, ‘discourse’, and material culture
We will also touch the subject of action research, which although not strictly a qualitative methodology, can best be considered here.
The nature of qualitative design The purpose of qualitative methodology is to provide rigorous designs to enable use to gather and interpret this data on quality. Each of the methodologies we will consider approaches this task in a different way, but the objective is the same: to provide us with ways to understand the world which add more than a purely quantitative approach can achieve. Three points are worth making: First, it is a premise that qualitative methodology is not intrinsically better or worse than a quantitative alternative. The appropriateness of any methodology can be judged only in relation to the research question that needs to be answered. No methodology, from a randomised controlled trial to an ethnographic study, should be selected for its own sake: the merit of a methodology rests only upon its capacity to generate answers to a specific research question. Second, the need to specify a research question, to refine that question to make it feasible, to identify concepts and indicators, to find ways to rigorously and systematically collect valid and reliable data, to analysis that data, and to draw inferences or conclusions are all part of a qualitative methodology. In their detail they will be quite different from quantitative approaches (for example, inferential statistics play no part in qualitative data analysis). But they are based upon epistemological propositions concerning how to gain knowledge of the world, and the designs have been developed to generate ‘evidence’ just as have quantitative methodologies. There is continuous methodological innovation in qualitative research, and entire journals are devoted to discussion and application of such techniques (for example, the journals Qualitative Methods and Qualitative Health Research).
7
2
Finally, we may note that quantitative and qualitative are used increasingly in tandem, providing a richness of data that could not be achieved by uses of one alone. For instance, the detailed description and inductive reasoning provided by a qualitative study (for example, an observation of a specific care setting) can be a stepping stone to a quantitative study that gathers detailed measurements of phenomena in a range of settings. On the other hand, a quantitative survey (for instance, of sickness among ethnic groups in the UK) may pose questions that require the in-depth interpretive work that an interview or focus group can contribute subsequently.
Qualitative Methodology and Research Questions We will now turn to the three main designs (plus ‘action research’) that we will consider in this chapter. We can set out the kinds of research questions that each has been developed to answer, although of course the range of questions that qualitative methodologies address is vast. Please look at the following table: then complete SAQ 7.1. Table 7.1 Research Questions and Qualitative Methodologies
Research Question to be answered
Methodology
•
What happens in a setting?
•
What are the organisational characteristics of a setting?
Participant Observation
•
How do people’s attitudes and beliefs affect what they do?
•
What do people think about an aspect of their lives?
•
What is the range of views on a subject among interested parties?
•
Is there a consensus view about a subject?
Interviews, Focus Groups and Consensus Techniques
•
What has been written about a subject over a specified period of Documentary and Material Culture time? Studies • What are the dominant bodies of knowledge in a culture, and how does these affect power structures?
•
What are the cultural products of a group and how do these relate to their attitudes and beliefs?
•
What happens when we try to improve the quality of a setting?
•
How may we enable people in a setting to improve their situation?
•
How can we challenge established power structures in a setting?
Action Research
7-3
SAQ 7.1 Match the Method and the Study Here are a series of research ideas generated by PhD students.
Suggest appropriate qualitative methods (may be more than one). 1. Adam wants to find out why so many older adults fear going into a residential care home. Method:
2. Beatrice is interested in studying changes in health promotion policy since the start of the NHS in 1947. Method:
3. Catherine is a ward nurse. She wants to see what can improve the experience of parents visiting their sick children. Method:
4. Dot wants to study the use of traditional healing remedies among Balinese tourist workers in Kuta, Indonesia. Method:
5. Edward wants to find out what is the accepted benchmark for caring for people with severe learning difficulties among community social workers and their managers. Method:
7
4
We will now turn to look at the four general research designs that we have identified. Note that we will address issues of methodology in this unit, saving detailed study of techniques for subsequent units.
Participant Observation and Ethnography Participant observation and ethnography are terms often used interchangeably to describe a type of qualitative research that uses extensive fieldwork observation to generate rich data about a setting. Participant observation is ‘about engaging in a social scene, experiencing it and seeking to understand and explain it. … By listening and experiencing, impressions are formed and theories considered, reflected upon, developed and modified’ (May 1993: 131-2).
Ethnography means ‘writing ‘writing culture’
Ethnography derives from anthropology, and traditionally this term was reserved for studies of ‘other cultures’, typically of developing world or indigenous peoples by Europeans. More recently, this term has come to be used to also document ‘own culture’ studies, partly to ensure that some principles established in anthropological methodology are applied: for instance, the effort to ‘make strange’ that which is familiar, and to resist making assumptions about what one is observing. Ethnography usually also implies a particular kind of research report which seeks to fully document the setting (Bryman 2001: 291). ‘An ethnography’ thus refers to this written output. To describe the specific methodology of field observation, we can use the term participant observation, to encompass both the anthropological tradition and the body of qualitative research that emerged from sociology in the 1920s and 1930s under the influence of the ‘Chicago School’ (May 1993: 111). Studies on crime and deviance, race relations and urban life were classic studies using this approach, but it has been applied widely and some of the most significant studies in this genre are in health and social care. As the name suggests, what is involved in participant observation are two things: observation and participation. The first is comparatively straightforward: this is a methodology that is based on direct observation of a setting. The second is more significant: it recognises that the researcher is a participant in this process, immersed in the setting rather than being separate from what is going on. Proponents of participant observation are adamamt that few kinds of observation are non-participatory. Only what has been described as the complete observer can completely
Answers to SAQ 7.1 1. Interviews with older adults 2. Documentary analysis of policy documents 3. Action research, perhaps incorporating ethnography and interviews 4. Ethnography, incorporating use of key informants 5. Focus group aiming at achieving consensus (Delphi study).
7-5
avoid involvement in what happens in a field, through the use of a one-way mirror or closed-circuit TV. Typically participant observation takes place over an extended period, during which a researcher familiarises her/himself with the setting and the people who populate it. During this period of fieldwork the objective is to gain a ‘rich picture’ of what is going on, document this by field notes and other recording means, and then analyse and interpret the findings to establish some generalisations about what is going on in a setting. Participation in the field can take a variety of forms, depending on the role that the researcher adopts. Three models can be distinguished (May 1993: 117-8). The Complete Participant A researcher may become a complete participant. S/he covertly becomes part of the field, taking on a role. For instance, a social scientist might gain employment as a hospital porter in order to undertake covert observation of what goes on and around a surgical operating theatre. It is argued that this reduces the likelihood that a researcher’s presence will affect the behaviour s/he is observing. Needless to say, it requires total commitment by the researcher for an extensive period of time, and may be emotionally draining as one tries to sustain a deception. There are also ethical questions about this kind of participant observation. Participant as observer Here the researcher is open about her/his position and does not attempt to disguise what s/he is intending as an outside observer. The researcher seeks to establish good relationships with subjects, so that the latter serve both as informants and respondents. Typically, a ‘key informant’ will emerge, who provides detailed information that could not be gathered simply by observation, either because it requires background knowledge, or because it is not immediately comprehensible to an outsider (for instance, technical processes or culturally-specific behaviour such as rituals or conventions) The researcher may adopt a role rather like a ‘fan’ of a celebrity or prestigious institution, seeking to learn as much as possible from people in the setting, while sanding back from becoming a member of the group. Thus, a researcher might seek to learn from a consultant surgeon why they perform certain tasks in particular ways. The researcher has 7
6
to guard against ‘going native’ in this role, as this will endanger the capacity to stand back from unquestioning adoption of setting norms of behaviour. Observer as participant Here there is very limited contact with the field, and may be a one-off engagement, possibly in the context of undertaking an interview. There is little effort to fully understand the norms, values and beliefs of the people one is observing. An example would be to take a tour of an operating theatre where a surgeon is operating as part of a visit to a hospital to interview administrators about management of waiting lists. Any observation is formalised, and only provides context to other data sources.
Issues in Participant Observation and Ethnography This design raises various methodological issues. Access Gaining access to a field may require prolonged negotiation. Even when invited to observe, it may take a long time to be fully accepted, as outsiders can be regarded with suspicion, especially where the people being observed are not powerful (for example, workers in a hospital kitchen). It will take a while for the subjects to relax and behave ‘normally’ when the researcher is present. The Research Bargain Observers may have to give something back in return for being accepted. This may simply be a matter of feeding back findings to participants, or can involve more complex and even financial contributions to key informants. Emotional Engagement and Objectivity Being part of a field setting can become the major part of a researcher’s life. It may be hard to keep distance, and field notes need to be used conscientiously to ensure validity and reliability of data. Leaving the field may be hard for both researcher and subjects. Ethics Participant observation turns ordinary people’s lives into the subject of academic study. Is it ethical to do this, and what should be the benefit to the people in return for this incursion?
7-7
Advantages and Disadvantages of Participant Observation and Ethnography Advantages
Disadvantages
•
Provides in-depth data about a setting • Time consuming and emotionally difficult (highly internally valid) • Needs to ensure subjects are not exploited by making them ‘guinea pigs’ • Good for identifying cultural norms, values and beliefs • Hard to generalise (low external validity) • Gives an insider view of a setting • Hard to sustain objectivity (intra-observer reliability)
SAQ 8.2 Ethnography applied to health care research Please read the following article:
Savage J (2000) Ethnography and health care. British Medical Journal, 321: 1400-1402
1. Where does Savage suggest ethnography can be applied in health care research?
2. What are some of the limitations of using an ethnographic approach in health care research?
7
8
Interview Methodologies, including Focus Groups and Consensus Techniques The interview is one of the most familiar methodologies used by qualitative researchers. In this section, we will look at the main approaches to interviewing, and then consider other methodologies that have adapted interviews to somewhat different ends. The interview is based on a structured conversation between research and respondent. Normally the latter is asked a series of questions, which may or may not be pre-arranged and may or may not be delivered in a set order. The answers constitute the data that are to be analysed, possibly enhanced by notes on non-verbal behaviour on the part of a respondent during the research encounter. The objective of an interview may be: • • •
to elicit factual information about a person or a phenomenon of which the respondent has knowledge to elicit data on experiences, attitudes and opinions to enable a researcher to gain impressions of the emotional responses to certain subjects by analyse of verbal and non-verbal behaviour.
There are a range of methods of undertaking interviews, including use of the telephone and e-mail and we have looked at some of these already in chapter 5.
Answers to SAQ 7.2 1. Ethnography can be used to learn about patients’ views, and to study service delivery. It can help provide information of how patient’s cultural practices may affect treatment, and how professionals’ cultural assumptions may affect health promotion activities. It can be used to understand organisational behaviour and how informal systems created by staff and other groups can influence the formal organisation of a care system. 2. Participant observation is costly in time and money. Funding can be difficult to obtain because it rarely leads to generalisable findings. There are ethical issues surrounding participant observation concerning informed consent.
7-9
Types of Interview Different typologies of interviews have been developed (May 1993; Bryman 2001), but we will look at four main kinds of interview. Structured or standardised Interview These have been considered in chapter 5 Semi-structured Interview This falls between structured and unstructured interviews. Questions are normally specified, but the interviewer can probe particular topics that emerge during the interview. The purpose is to achieve clarification and elaboration of themes (May 1993: 93). Interviewees may be encouraged to expand on issues as they are raised. Questions in this kind of interview tend to be more general, enabling respondents more flexibility to give responses hat match their, as opposed to the researcher’s agenda. The order of questions is also flexible, enabling a more ‘natural’ flow to an interview. The advantage of this approach is that it sustains some structure, in terms of the topics to be covered, and perhaps the order in which questions are asked, to enable rigour to be sustained, while at the same time permitting respondents to develop their answers, providing richer data. Unstructured (Focused) Interview The third kind of interview, and one that epitomises the naturalistic frame of qualitative data collection, is open-ended in character, allowing the most flexibility to researchers to explore topics with interviewees, and for respondents to follow their own trains of thought and lines of reasoning. The interview is guided by an aide memoire of topics to be covered, rather than any kind of formal schedule. This leads to an interview that may have more of the character of a conversation. This approach is called focused because it enables the interview to focus on the respondents’ frame of reference and enables them to lead the interview into their own focus, rather than one pre-determined by a researcher. The disadvantage of the unstructured/focused interview is that it reduces comparability between subjects. Its advantage is that it enhances the internal validity of a study by ensuring it addresses the issues important to respondents, and also reduces the likelihood that a researcher will ‘lead’ respondents towards some bias of their own.
7
10
Group Interviews and Focus Groups A group interview enables a number of respondents to be brought together to discuss an issue. The model for this can be either an unstructured or semi-structured or even structured interview. In the former, the group is encouraged to respond flexibly, as in an unstructured/focused interview, with the researcher using a list of topics rather than specific questions to stimulate the discussion. The objective is to generate a broad-ranging output, to maximise the data that can be collected, and to explore group dynamics and a level of reflection that might not be achieved in a one-to-one interview. This kind of interview is also known as the focus group, and this approach is becoming widely used in health and social care research. In this methodology, the researcher may take on the role of group moderator. The objective is to stimulate group dynamics, so that data reflects the interactions between people as well as individual responses (Bryman 2001: 337). For example, a focus group conducted with nursing staff and managers to discuss how care is delivered may provide data on the power relations between staff. However, it is also possible to apply a semi-structured or approach, in order to elicit more specific information. For instance, a researcher may wish to gather data on a variety of relatively factual topics, all of which a single respondent might not know. Thus, at the start of a piece of exploratory research or ethnography, a group interview of different ‘experts’ could be used to efficiently generate the background information that a researcher needs, rather than undertaking a series of one-on-one interviews. Group interviews do seems to generate richer data, with respondents sometimes more willing to speak out in a group then individually (May 1993: 95). There is one specific type of group interview that warrants a separate section: this is the so-called Delphi approach. The Delphi methodology The Delphi methodology, named after the Oracle of Delphi in ancient Greece (said to be able to predict the future and supply knowledge), is a means to gain consensus views among a range of experts in a field. While often using quantitative approaches such as questionnaires or surveys, sometimes a Delphi study may incorporate group interviews.
7 - 11
Somewhat like the focus group, the Delphi methodology tends to use very focused questions in order to stimulate discussions. However, the intention here is not to stimulate group dynamics, but to ensure consensus. What is aimed for is an output that reflects what all involved can agree upon. Sometimes this is fairly limited, but can be used as the basis for a policy or standard. For example, when developing guidelines on care, a Delphi approach could bring together different professionals to seek an agreed strategy. Clearly it is important that those involved are leaders in their fields, and can represent the shared views of their professional colleagues.
Issues in Interviewing There are a number of issues in using interviews to gather data. Access It is sometimes hard to persuade people to be interviewed, especially powerful or high status individuals. Setting up group interviews can be difficult, as many people need to find a suitable time Validity There is no guarantee that respondents will tell the truth, and may refuse to speak about some subjects. Unlike participant observation, a researcher is dependent on the subject to provide valid data. Reliability Researchers can affect how people respond in interviews. Males may not be willing to speak freely to females about some subjects and vice versa. Researchers may also bias findings intentionally or unconsciously if they have strong views about a subject. These issues can be addressed by recording interviews, checking analysis of interview data and feeding back findings to respondents for ‘member-checking’ to see if they agree with conclusions.. Recording It is now common to audio-tape interviews and videorecording is also used. The latter has the advantage of identifying non-verbal behaviour. When transcribing interviews, some researchers -- especially those using a form of detailed transcription called discourse analysis -- code all utterances and gaps as well as formal speech.
7
12
Advantages and Disadvantages of Interviews Advantages • • • •
Disadvantages
Simple to organise and well-understood by • Validity may be suspect as respondents may respondents be unwilling to talk openly or truthfully Relatively easy to learn techniques Can generate data that can be recorded for ease of analysis Cheap and quick
•
•
Open to ‘researcher-effect’ in which the presence or characteristics of the researcher affects how interviewees respond Transcription of data is time-consuming
SAQ 7.3 Interview Methodology
Please read the following paper (available from the Library e-journals) and answer the questions. Lawton, J. (2002) Colonising the future: temporal perceptions and health-relevant behaviours across the adult lifecourse. Sociology of Health & Illness, 24 (6) 714-733. 1. Why did Lawton choose an interview methodology for this study?
2. Would the study have benefited from using an unstructured interview schedule?
3. Was the interview schedule adapted during the study, and if so, why?
7 - 13
Use of Documents and Material Culture in Research A less frequent source of data is the documents that have been produced, perhaps by policy-making bodies, by pressure groups and other organisations, and of course, the academic papers that report research findings. When the latter are analysed, the result is sometimes called a literature review, meta-analysis or secondary analysis. The range of documents that may be consulted include: • •
personal documents official and historical documents including government policy documents and legislation
•
official statistics
•
mass media output
•
internet web materials
In addition, we can include the study of material culture products, by which we mean all artefacts produced by human endeavour: typically, art and craft objects, ritual objects and technological products. These have traditionally been collected and analysed as part of anthropological studies, and often form part of museum collections of ethnology. Broadly, the approach to studying a document can be differentiated as follows (May 1993: 137):
7
14
•
realist: the document is a medium that provides a window on to the world. We can use the document to ‘read off’ the underlying reality that it describes, for instance, a pattern reflected in official statistics or an account of an historical event. A typical example would be the use of private papers to substantiate the development of some innovation in health care such as antisepsis or anaesthesia.
•
a representation of the practical requirements that led to a document’s creation. The document not only describes an event but also constructs it within a framework that is affected by the social context within which it was created, and also may reflect specific objectives of its author. Thus, a policy document often reflects a desire to change existing practices, while an autobiography may reflect not only what ‘actually happened’ but also the ‘spin’ that the author would like to promote to the world about her/himself. The task of the researcher is hermeneutic, to discover the hidden meaning of the text.
•
part of a ‘discourse’ that mediates power and knowledge: what can be spoken about a subject at a particular period in history. This perspective has been derived from the work of Foucault and has been applied in the social sciences to offer insights into changing patterns of power and authority in society, and how this is mediated through knowledge. For example, by studying documentary sources associated with the growth of public health in the 20th century, Armstrong (1983) argued that this reflected a new construction of the community as the subject of medical power.
The study of material culture can also be undertaken within these different epistemological approaches. For realists, objects are considered principally in terms of their apparent purpose. In the second and third approaches, objects are regarded as ‘texts’ that can be ‘read’. The language of objects may of course be very obscure, and it is impossible to know the meaning of an object without recourse to knowledge of the social norms, values and beliefs that shaped its production. Material culture objects are often discussed alongside analysis from participant observation or ethnographic approaches.
Hermeneutics The search for for the meaning of an event beyond its surface manifestation. A hermeneutic approach will try to extract the meaning of a text from the perspective of its producer..
Michel Foucault was a postpost-structuralist philosopher, who argued that power and authority is achieved by controlling the knowledge that a society holds to be ‘true’ (‘discourse’)
Answers to SAQ 7.3 1. It was an exploratory study that sought to generate themes rather than assessing hypotheses already formulated prior to the study. An interview methodology was appropriate as this enables a flexible approach to data collection, including the use of open-ended questions. 2. It is possible that the semi-structured approach constrained respondents. Despite arguing that the research would generate themes, the author admits that some themes were developed prior to the interview schedule being formulated. 3. Early interviews led to changes in the schedule so that themes emerging from the data could be incorporated into alter interviews. While preventing comparisons, this added flexibility to the design of the study.
7 - 15
Issues in Documentary Research The following issues need to be considered in documentary studies. Access Access to documents may be restricted and analysis based on available documents always runs the risk that inaccessible documents may shed an entirely different light on the issue under investigation. Authenticity What is the guarantee of a document’s authenticity? Documents may be fabricated to suggest false versions of events, and it may only be possible to determine authenticity by reference to other sources. Sometimes errors are unintentionally introduced into documents when they are copied or translated. Analysis The approach to analysing a document or object will depend on the theoretical stance taken by the researcher (see above).
Advantages and Disadvantages of Documentary and Material Culture Analysis Advantages •
Disadvantages
Gives access to events that may not • All documents and objects produced by otherwise by accessible humans will contain an inherent bias as a consequence of the context and objective of • Can give an insight into the beliefs of their their producers. There is no simple way to producers and cultures within which they are remove this bias. produced • Authenticating texts/objects may not be • Enables comparisons between possible. cultures/groups in terms of different textual productions • Analysis is very prone to researcher bias and interpretation is an inexact art.
7
16
SAQ 7.4 Documentary Analysis Please read the following paper (available from the Library e-journals) and then answer the questions:
Smart A (2003) Reporting the dawn of the post-genomic era: who wants to live forever? Sociology of Health & Illness, 25 (1): 24-49. 1 What model of documentary analysis does Smart use in his paper?
2. What concerns might you have about using press reports to provide an insight into an event such as the completion of the human genome project?
Action Research Action research is not specifically a qualitative methodology as it can use both qualitative and quantitative approaches. However, it fits neatly in this chapter and is a growing methodological approach in health and social care research. It seeks not only to understand a setting but to change it in some way for the better. It is also known as practitioner research and is well suited to health and social care researchers who are working in a professional setting and want to try to improve the experiences or outcomes for patients, clients or staff. Action research can be traced back to the work of Kurt Lewin, a management psychologist, but has subsequently developed in a number of differing directions, each grounded in different philosophical and political commitments. An early manifestation of action research was the Fox projects, conducted in the US in the 1940s and 1950s. The principles of this research were to develop and test theory on an ongoing basis in interaction with interventions or actions; ensure consistency between project means and desired ends; 7 - 17
and base ends and means upon guidelines established by the host community (Stull and Schensul 1987). This model of action or advocacy research may be discerned in a study by Schensul et al (1987), which extended the use of a traditional model of maternal care into a new setting, in order to enhance child health. Another health care example of this kind of approach may be found in Hart and Bond (1995). These examples of action research fall into the first of three categories set out by Carr and Kemmis (1986). Action research, they argued could be classed as •
technical (in which an outside expert undertakes the research);
•
practical (in which the researched are encouraged to participate in the research process); and
•
emancipatory (in which the researcher takes on the role of a ‘process moderator’ assisting participants to undertake the research themselves).
Emancipatory action research is the most radical, and has been associated with critical theory, and is predicated on political commitments to participation and improvement in quality of life among participants. Schensul (1987) suggests that it can •
bring together people with diverse skills and knowledge;
•
de-mystify the research process, allowing practitioners to shape the data collection process;
•
build a research capacity into a community which can operate independently;
•
increase the likelihood of the use of research findings by practitioners; and
•
improve the quality of research by enabling access to key bodies of knowledge in a community.
Answers to SAQ 7.4 1. Smart uses a hermeneutic approach in which he considers how the press report this event in terms of the wider social context and their own concern with a ‘good story’. 2. Smart suggests that the papers have largely depended on the version of reality given to them in the scientists’ press releases. This is then situated within a common-sense analysis of the social consequences of such scientific advances. It plays on public fears about the use of technology by governments for undesirable ends. it would be hard to gain a balanced view of the innovation from the press reports.
7
18
Fox (1999) has suggested a fourth model: a transgressive action research. In addition to the principles underpinning emancipatory action research: namely, of reflexivity and collaboration, transgressive research has a commitment to challenge norms and assumptions, resisting power and constraint and opening-up new possibilities. While many of the processes involved in undertaking research of this kind are similar to those of emancipatory action research, transgressive action research is based in a perspective of divergent rather than shared rationality, acknowledging that there may not be a single view of what constitutes an ‘improvement’ in a setting, and working to enable people in a setting to set their own agenda for change. The main advantages and disadvantages of action research are summarised below.
Advantages and Disadvantages of Action Research Advantages
Disadvantages
•
Enables practitioners to research their own workplaces
•
Is setting –specific. Hard (and inappropriate) to generalise
•
Focuses on change rather than understanding: the outcome is improvements in service, not academic reports.
•
May be of limited use where practice cannot easily be changed (e.g. where there are national guidelines on practice standards).
•
Can spread research capacity, enabling people to research themselves rather than depending on outside academic researchers.
•
Can be hard to separate dual role of practitioner and researcher leading to questions of reliability of data..
7 - 19
Sampling in Qualitative Methods If you have read chapter 4 of this unit, you will be aware of a range of issues concerning sampling in quantitative research. The issues for qualitative studies are somewhat different, and the approach to sampling is distinctive, as will be seen in Figure 7.1. For example, a study may wish to consider attitudes towards compliance with asthma treatment. A small sample of local asthmatic teenagers may be interviewed. The sample is thus chosen on the basis of ‘theory’: this kind of theoretical sampling aims to maximise the range of responses, but does not strictly seek to ‘represent’ all the people in the community and other stakeholders. Figure 7.1 Comparisons of sampling approaches Quantitative
Qualitative
Sample big enough for statistical Inference (see Chapter 5)
Often very small, occasionally a single case
Selected to be representative
Rarely attempts to be representative: sample chosen to maximise range of responses
Assumes respondents are independent and do not affect each other’s responses
Respondents’ interactions may be part of the topic for study
The issue of sampling is closely associated with ensuring validity and reliability. Before looking at specific methods of data collection and analysis, we will not turn to considerations of how to make qualitative research valid and reliable.
Validity and Reliability in Qualitative Research Having read Chapter 3, you should be familiar with the principles behind the reliability (or consistency) and the validity (or accuracy) of research, and how to maximise both in the research which you are undertaking. If you need to remind yourself of the main kinds of reliability and validity, you should review this material now. Validity and reliability are issues for qualitative researchers as they are for others. However, it has been argued that the way in which the former need to address validity and reliability differ from the traditional approach of positivist researchers. The naturalistic paradigm, Lincoln and Guba (1985) argued, places different issues before researchers. However, the question of the trustworthiness of a piece of 7
20
research still stands. Lincoln and Guba suggest four key areas that we need to consider of a piece of naturalistic (qualitative) research, relating to four questions: How truthful is the finding?
(Credibility)
Can the findings be generalized?
(Transferability)
Could the findings be replicated?
(Dependability)
Can we rule out researcher bias?
(Confirmability)
These four areas relate to traditional notions of study validity and reliability, as can be seen in Figure 7.3 Figure 7.3 Positivist and Naturalistic analyses of Validity and Reliability Positivist
Naturalistic
Asks:
Internal Validity
Credibility
Are the findings believable
External Validity
Transferability
Are the findings applicable elsewhere?)
Instrument Reliability
Dependability
If the study were repeated, would the same findings emerge?)
Intra-observer Reliability
Confirmability
Has the researcher biased the findings
Note that just as in the positivist paradigm, validity is dependent on reliability, while the reliability of a study does not guarantee its validity. Qualitative researchers thus have to address the four issues of credibility, transferability, dependability and confirmability when they undertake their research. However, the way Lincoln and Guba (1985) argue that we need to think about some of these issues differ quite markedly from the perspective of positivist research. Let us look at each area in turn.
Credibility Related to the internal validity of a study, credibility requires that we concern ourselves with the accuracy of description in a piece of qualitative research. We need to state the precise parameters of the study - who was studied, where and when, and by what methods. If we identify these aspects, and if we have a reliable means of measurement (dependability and confirmability), our study will be valid for the specific setting investigated.
7 - 21
Transferability
Definition: Definition External validity is the extent to which the findings from a study can be generalized from the sample to the wider population.
We noted earlier that in most qualitative research, the method of sampling is not representative, but will be aimed at maximising the diversity within the study setting, to ensure as ‘rich’ a picture of the setting as can be gleaned. Clearly, this method of sampling will not supply external validity in the way that is usually sought in quantitative research, which wishes to generalise from the sample to the population from which it has been drawn. Thus in a major divergence from positivist approaches, Lincoln and Guba argue that no claims can be made about the applicability of the findings to other settings. If other researchers wish to generalise from a study to other situations, the onus must be on them rather than the original researcher to demonstrate a study’s applicability elsewhere.
Dependability Whereas positivist research has to assume an unchanging world, so that if an identical study were to be performed the assumption would be that the same findings would emerge, the naturalistic paradigm acknowledges that the world, especially the social world, is constantly changing. In particular, a study might itself affect the world it is trying to document. (Thus, interviewing someone about their attitudes to pollution might make them think for the first time about what they really do think. Repeat the interview, and the answers given may reflect a new perspective gleaned from the respondent having considered the subject more fully.) If change is inevitable, then all a research can do is to try to predict as much as possible of what these changes may be, and account for them by casting widely for data within the setting. You should be able to see how the ‘dependability’ of a study is like the different kinds of reliability we looked at in Chapter 4: internal reliability of an instrument, test/retest reliability and inter-observer. You should also be able to recognise the need for dependability in order to obtain credibility.
Confirmability In a naturalistic paradigm, we have to accept that observer bias is a fact of life: we all have values and we cannot wholly avoid allowing these to colour the way we interpret data in a qualitative analysis. To minimise this bias, qualitative researchers need to recognise their biases, and seek to fault their own 7
22
assumptions or ‘pet theories’ about what they are researching. Bringing in colleagues to offer alternative readings, and feeding back results of an analysis to the original respondents can help to reduce these biases. Again, you should be able to see how the confirmability of a study is a pre-requisite for its credibility.
Summary The four approaches to research that have been described in this chapter underpin all qualitative research studies. Sometime a mix of methodology is used, and sometimes they are melded with quantitative methodologies. These methodologies have in common an emphasis on interpretation, and seek to maximise the internal validity of findings (ensuring that a research question generates the right data to provide the answer. In contrast, there is lower generalisability (external validity) than quantitative research, and there are significant problems of objectivity and dangers of bias, which require meticulous attention to methods of data collection and analysis. There are also fundamental questions about the status of knowledge generated by research (is it ‘truth’ or one among many ‘truths’?) Although this question is common to all research, it comes to the fore in qualitative research because these methodologies have acknowledged the role of interpretation in the growth of knowledge. Naturalistic methods are invaluable in researching the meanings and perceptions of people, either alone or in conjunction with quantitative methods. In health and social care research, qualitative research is now firmly established, and these methodologies are important strategies for the researcher to master. Please complete the following reflective exercise to complete this chapter.
7 - 23
Reflective Exercise 7.1 Validity and Reliability in Qualitative Data Analysis Discuss the following questions in relation to the validity and reliability of qualitative data collection.
1. Why might a researcher ask respondents to check written transcripts of interviews?
2. Why should a researcher keep a field diary reporting what they did during their study?
3. How might a researcher ‘triangulate’ her data, and what would be the object of such a research strategy?
7
24
Reflective Exercise 7.2 Collecting Qualitative Data Please write a research question on a topic in health and social care that interests you, using one or more of the qualitative methodologies described in this chapter. Describe which methodology you would use, why you consider the methodology appropriate, and suggest what problems in data collection you might encounter were you to undertake this research. Your Research Question
Which Qualitative Method would you use, and why?
What problems in data collection might you encounter?
Further Reading Bryman A (2001) Social Research Methods. Oxford University Press. Denzin N and Lincoln Y (2000) Handbook of Qualitative Research. London: Sage. May T (1993) Social Research. Buckingham: Open University Press.
7 - 25
7
26
Learning Review Form: Chapter 7 Please complete this form when you have completed the chapter. If you do not consider you have achieved the learning outcomes, you need to go back and do more work on the chapter or read the recommended reading. When you have completed the form, save it to hand in with your log book
1. I am confident that I can: Not at all
Partly
Quite well
Very well
• describe the advantages and disadvantages of a qualitative approach
1
2
3
4
• describe the different qualitative methodologies and the advantages and disadvantages of each
1
2
3
4
• compare and contrast the naturalistic and positivistic approach
1
2
3
4
• describe what is meant by grounded theory
1
2
3
4
• recognise alternative ways of maximising validity and reliability in a qualitative study
1
2
3
4
7 - 27
7
28
