Survey Methods. Mary Linn and Keren Rice. InField 2010. Description. Planning
is an essential part of language revitalization. Surveys, sometimes required.
Survey Methods Mary Linn and Keren Rice InField 2010 Description Planning is an essential part of language revitalization. Surveys, sometimes required by funding sources, are an important tool for planning. In this workshop, we will discuss questions about surveys, such as: What can we learn from a survey? How can surveys help in revitalization? What different kinds of surveys are there? What kinds of questions should we ask? Who should carry out a survey and how should a survey be given? What do we do with the results? We will use Handbook 3: Conducting a Language Survey (from Awakening Our Languages: ILI Handbook Series) as a starting point for classroom discussions. The goal is for participants to leave with a creative toolset for using surveys in their own communities or language programs. Overall outline Day 1: Background 1) What is a survey? Why do a survey? What does it take to do a survey? Cover parts 1‐2 of ILI Handbook 3: Conducting a Language Survey? 2) Discussion of one survey, looking at what the purpose was, how it was set up, nature of questions, etc. Day 2: Writing Survey Questions 1) Having read surveys, now think about the way of carrying out a survey: Types of surveys, nature of community, nature of questions, who will give and how, etc. 2) Develop potential survey questions. Day 3: Interpreting and Using Results 1) How does context affect the results, how it was asked, who answered, etc…? 2) Outlining your survey, including purpose, how to administer… 3) Pros and cons of various surveys. 4) Where do we go from here?
A. Focus of the Course 1. Community Surveys: This course will look at those surveys that are generated from inside communities, i.e. something from within that has created a need to gather information. Generally, these surveys gather information on vitality in the largest sense, language use, and language attitudes. If you are helping from outside the community, or an outside researcher gathering data, you have to know something about the community. You can’t just pull survey questions from a list, but they must come from and reflect the community. 2. Other types of surveys: We won’t be looking at other types of surveys such as those are generated solely from outside communities or those that look closely at competency in language skills areas. Although we won’t discuss these in depth, we have posted a variety of different kinds of surveys to show scope, including UNESCO (generated from outside to gauge vitality and documentation) SIL (generated from outside to gauge vitality and literacy) Census questions are not posted, but we discussed whether or not languages and language use is recorded in the census. Questions about language are not included in the US in the short form; Canada asks for 1st language and language of the home. Census (and other data generally) reflects self‐reporting so it is not clear if the notion of fluency is understood in the same way by all respondents, and people have various reasons to under or over report their abilities. We recommend visiting the website of James Crawford, Language Policy Website & Emporium at http://www.languagepolicy.net/index.html. He has several articles posted on the 2000 US census and how to read census data. 3. Our experiences: We will focus on our experiences, which are in small severely endangered language communities and in communities where shift is starting to occur. We do not cover larger minority state languages, such as those in India or Africa. B. What are reasons for communities to do surveys? Need to for specific funding: ANA grants Helpful for future funding, as it establishes a baseline of data Language planning, such as what kind(s) of program(s) to have Why language is shifting to English, to get at core of problem
Understand hurdles they may face, such as resistance to dialects, writing… Find out resources, such as human and infrastructure such as building, buses… Assess current state of language education Importantly, to get the community aware and involved Usually NOT to just find out the number of speakers C. What is in a Survey? (As we look through the surveys, it is important to keep in mind the method of how they are given, for example face to face interviews privately; small group discussions; booths at fairs, pow‐wows, centers; mailed. The method in which the survey is carried out can affect the questions asked and how a question is asked.) We reviewed a number of the surveys, looking at the goals of a survey, the sections, the goals of specific questions, and the type of information that can be obtained from different question types. Tokelau Language Survey Our reason for looking at this survey first is that it is well organized. It includes all of the major parts of a community surveys, and clearly delineates the parts. Background of the survey: • Purpose for giving: Community elders were concerned and got a grant to do a survey • Who: The community asked for help from linguists from U of Hawai’i Manoa • How: Done by face‐to‐face interviews. This survey is discussed in the following reference: Otsuka, Yuko and Andrew Wong. 2007. Fostering the growth of budding community initiatives: The role of linguists in Tokelauan maintenance in Hawai’i. Language Documentation and Conservation 1.2. http://scholarspace.manoa.hawaii.edu/html/10125/1716/otsuka.html 1. Major Parts of Surveys Demographic Information • Often private (usually don’t ask the name), address, etc. • But we may need to ask (depending on the community: age, sex, clan/band/tribal town, location, level of education, job (both wage‐earning and traditional positions) • Demographic information will tie answers to internal groups of people to show language use and attitude trends.
Competence=Fluency=Vitality • Self reported, but the more questions, the more accurate • Generally people under rate or over rate, why? • Can you imagine other ways of asking these questions? Domains of Use • Where you speak it? • Where do you hear it? • With whom do you speak? Why are domains of use important? We use language in different environments, like home, work, special events, school, in public, etc… It is often an indicator that language is endangered when they are spoken in fewer and fewer domains. If we know where they are spoken, we can strengthen these and begin to target others we want to get back. See the ILI Handbook 3: Conducting a Language Survey for more information on the importance of language domains. Language Attitudes Language attitude questions cover how people feel about their languages and language use in their communities. • Who, where, why you feel it should be spoken (or not) • Who do you think should/should not teach • If children should/can learn Why are these important? Some attitudes, if held widely, can help or hinder language revitalization efforts. For example, if people feel that children cannot learn two languages at one time, parents may not support an immersion school. You may find that some people have deep held stereotypes about their own languages that may keep them from feeling comfortable about learning. Language attitude questions tell us what we need to pay attention to. Conversely, if people feel that dialects really are not important but that going forward is, then this can help efforts. See the ILI Handbook 3: Conducting a Language Survey for more information on the breadth of language attitude questions. Getting People Involved • Questions about specific types of programs. • Questions Level of interest in learning, specific types.. • Questions about what talents you can bring to a program (teaching, cooking, art…)
No one survey or question is going to give all the answers, but if used appropriately you can get a lot of information. D. Purposes & Community Differences We looked at the various surveys that are posted on the InField website. What do they have in common? How are they different? We can see that surveys can actually serve very different purposes. Jamaican: Language attitude survey to determine stereotypes (negative and positive about Jamaican Creole. Cherokee: Vitality, including several questions on literacy, plus use, and attitudes. Culture specific questions, such as literacy in the syllabary and phonetic writing system. Also, they ask about Cherokee Blood Quantum, in demographics, which may show patterns in answers, but may not be appropriate in other places in the world. Welsh: Language use survey to see after 20+ years how much Welsh is being used, especially in the workplace. This survey seems to be keyed to Fishman’s Language Vitality Scale in gauging vitality by use in more and more public domains. Sauk: Getting people informed and involved. Daghida: Asks about different language abilities and use in a multi‐language area, with clear knowledge of the community it was designed for. Questions are very easy for an interviewer to check off while talking to the taker. The questionnaire is quite long. E. Types of Questions Choosing the type of questions to use is tied to the purpose of the survey and how you will give the survey. Questions • Yes/No • Agree/Disagree Degree to which you agree/disagree • Check the box Check one Check 2, 3, … Check all that apply • Ranking • Short Answer • Open‐ended
What are strengths and weaknesses of each? It all depends…. but as a general rule, open‐ended questions give much richer information, but are much harder to get the data out of. They are easy to write, but the hardest to work with afterwards. Things to consider What kind of information will this question give? Will people be able to answer the question? Will the information provided be useful? For instance, in asking about age, many questionnaires give ranges (0‐10, 11‐15, 16‐ 20, 21‐30, 31‐40, etc.). Is this appropriate? Will people know their age in years? Why these ranges? Wording needs to be thought about. Are words like competence/fluency/ proficiency understood? Will answers from different individuals be comparable? See the ILI Handbook 3: Conducting a Language Survey for examples of each kind and more discussion on pros and cons. In addition to thinking about the questions themselves, there are a few other things to consider. How should the survey be introduced? Including a number on each survey can be important as an identifier. F. Giving a Survey We have talked about content, now how do you give a survey? 1. First Steps “Administering a survey” refers to the actual giving of it, but more goes into it than that! • Understanding your resources: What are your resources? This will determine to some extent 1) how you will give it, and 2) how much analysis you can do with it • Engaging the community These are some ideas – the particular means depend on the community Town meetings/go to different communities and talk about What it is Why Who will take it, who will give it Some of the questions.,what it actually looks like, how long it will take, etc…(this goes a long way to relieving fears) Go to community events and announce Advertise in community/tribal newspaper Radio
2. Pilot Study Always do a pilot study! This will help you refine the questions, work out kinks – and there will be some!!! Steps is Pilot • Pilot what you think is the final version (but it will probably not be after the pilot). • First, give the survey to other workers in your language program (if you have one; or colleagues of some sort). Try to use real survey giving context as much as possible. This is more like a practice run. • Second, do a real pilot. Target several people from different internal, relevant groups and give them the survey as if it were the real thing. • Discuss the pilot as a group problems: Were there problems with how you gave it (interviewing, setting up time, etc…)? The length of the survey? The questions themselves? Adjust your survey and survey giving methods as needed. • Look at the data and analyze the findings, then discuss. Are there patterns that you didn’t anticipate (so you need to ask further questions, for example)? Are the questions to unwieldy to easily get the data? Adjust survey as needed. 3. Giving the Survey When writing the survey, always keep in mind how you will give the survey. This will determine the kinds of questions, how they are worded, and the length of the survey. Ways to give surveys: • mail ‐‐ We don’t recommend! The return is usually very disappointing. • “Survey Days” given individually or in small groups at centers, hospitals, etc.. with a survey worker there to give and answer questions. • Set up booths at large events, such as pow‐wows. Allows you to reach a lot of people but there are workers there to answer questions. • Door‐to‐door interviews scheduled ahead of time. This is the most labor intensive and time consuming but you get the best data. How many people will be in your team? (Elder/speaker, interviewer, someone to set up recording equipment if used) * Focus groups/sharing circles. This can lead to rich responses with detailed discussion. It is time consuming, but the groups might be important in furthering the goals that underlie the survey. • Combinations Some communities have used the survey number on each survey to have a lottery of donated artwork, food baskets, etc… (see the Sauk and Cherokee surveys).
G. Interpreting, Reporting, and Archiving 1. Compiling data (We are not going to talk about this, but if the survey sample is small, you can do this by hand. If it is very large, then you may want to get some software help or statistics help. This may be determined by the purpose of your survey and your resources.) 2. Interpreting Interpreting data refers to your analysis of what the compiled numbers mean. Some results may be very context (culture or social) based. Some may differ by who answered (men or women, dialect areas, etc…). Some results may be due to the way you asked the question. Keep all these factors in mind when developing the survey and als when interpreting it. • What do you want to get out of out of the survey? Quantitative analysis Qualitative analysis • How much statistics do you need? Statistical analysis can be very revealing, but it is not always helpful. 3. Reporting •Don’t just report the findings back to funder/sponsor, etc. • The survey and its results belong to the community and they should be made aware of all the results! Town Hall meetings At events Newspaper, radio… Others, as appropriate in the community 3 Kinds of Reports 1. Community reports Dene high school students: short survey; qualitative analysis Cherokee: qualitative and quantitative Seminole (Welsh Language Board) 2. Academic reports Innu Aimun: interesting survey; qualitative and quantitative Inuktitut (Dialect study research report) 3. Government reports Australian, British Columbia, Northwest Territories (‘Reality check’) Making Accessible and Storing • Store somewhere safe, multiple copies, you never know how it can be used in the future.
Some important points to keep in mind. The survey belongs to the community and is part of the history of the community. Make sure it is available for everyone and for many years as there is always more to get out of them. Surveys are not perfect, no question is perfect, but all in all, they can be very informative and very useful in language planning, implementation, and evaluation. Surveys that do not take the particular community into account and that do not have community involvement at each step are less likely to be helpful in language revitalization work. Materials available on the InField website Syllabus, with additional references Surveys Cherokee Survey Daghida Survey Innu Survey Omaha Language Survey Sauk Survey Seminole of Oklahoma Survey Tokelau Language Survey Inuktitut Survey Unesco Vitality Diversity Questionnaire1 Yakama Shaptin Survey There are also surveys within the Jamaica Survey Report and the Welsh Survey Report. Survey Reports: Cherokee Survey Report Cherokee Needs Assessment Report Dene Kede Survey Report Dialect Study Research Report (Inuktitut) Jamaica Survey Report NILS Report 2005 (National Indigenous Language Study Report 2005, Australia) Reality Check Final Report (Northwest Territories, Canada) Report on the Status of BC First Nations Languages Seminole Nation Report Welsh Survey Report These include community survey reports and government survey reports. Other Crosscurrent Language Activists Manual
