What are surveys?Surveys are the quintessential quantitative methodology. They are used to collect quantitative information about items in a population. Survey research is a method of data collection in which a defined group of individuals are asked to answer a number of identical questions. Its aim is to measure attitudes, knowledge and behaviour. It is commonly used in social science, market research and in public health. Show
The components of a survey include selecting a sample of respondents and presenting the survey questions to the sample either in an interview or in a self-adminstered questionnaire. A survey can be descriptive (i.e. describes the nature of existing conditions), explanatory (i.e. seeking to establish cause and effect relationships), theory building or instrument testing (i.e. testing a newly developed health status or quality of life instrument or used as a tool to measure a psychological condition). The unit of analysis is usually the individual though it can be an organsation (called cases). Surveys can take the form of a census or be a sample. If it is a sampled survey, it would be one of the following:
Stages in Survey Research
Key Terms and ConceptsSurvey Instrument Schedule of questions or response items Response items Individual survey questions or statements for which a response is solicited. Interviews In context of surveys, this refers to face-to-face administration of a survey instrument. Here the researcher reads out the questions and ticks the answers on behalf of the respondent. Researcher cannot and does not engage in conversation as this may bias the collection of data. If there are open ended questions, the questions are just asked with no follow up for clarification, unlike in qualitative interviews. Such questions are coded post-hoc. Questionnaires This refers to post, email or other indirect methods of administration. Initial Steps
SamplingThere are two types of sampling: Random (Probability) and Non-random (Purposive). Random sampling is used in descriptive and explanatory studies (i.e. testing of hypotheses). Examples of random sampling are simple, systematic, stratification and cluster sampling. Non-random sampling is used in theory and instrument development. Examples are quota or volunteer sampling. There is also a sampling strategy for small-scale studies or for groups that are hard to reach. This is snow-balling or network sampling. This is used when there is no adequate list for a sampling frame. Nomimated individuals nominate others. It is feasible in surveys on illegal activities but it can only be used for members of a network who share the characteristics of interest. Sources for sampling include electoral register, postal register or multiple list sampling (e.g. members of the Multiple Sclerosis Societ. For telephone surveys, you can use the telephone directory, random-digit dialling. However, make sure you select a respondent from the household (e.g. the person who last had a birthday in the household). It shouldn't be the first one who answers the phone. How big should my sample be?The principle of sample sizes is the smaller the population, the bigger the sampling ratio has to be for an accurate sample. Larger populations allow for smaller sampling ratios for equally good samples. As the population size grows, the returns in accruacy for sample size shrink. Best sample size depends on degree of accuracy required, degree of diversity in the population and the number of different variables being examined simultaneously in the data analysis. There are two ways to determine sample size: (1) calculation of sample size by using statistical equations or (2) use of 'rule of thumb', i.e. samples based on previous experience that have met the requirements of the statistical method (see Neuman 1997 below): For populations under 1000, you need to sample 30% of the population. For populations of 10,000, you will need 10%. For populations of over 150,000 you require 1%. For very large populations of over 10 million, you only need 0.025%. Size of population ceases to be relevant once the sampling ratio is very small so samples of 2,300 are as accurate are as accurate for 200 million as it is for 10 million. Sub-group analysis will affect sample size so the rule of thumb is to have at least 50 cases for each subgroup being analyzed. For example, I want to analyze four variables for males between 30 and 40 years old (this could only be 10% of the population) so 10 X 50 =500 cases needed for this sub-group analysis. Types of Surveys
Note: If you're doing a postal survey, make sure you include a cover letter and SAE (self addressed envelope). Don't use metered postage as it dates and the post office may refuse to deliver it. Also people are more likely to return it if you spent money on a stamp! Do send out reminders - usually two reminders (the first, 2 to 3 weeks after the questionnaire and the second 2 to 3 weeks after that). The first reminder is just a reminder but you can resend the questionnaire with the second reminder (assume that the questionnaire was discarded). You could do more but generally two reminders is enough. You're also more likely to get responses immediately and then they taper off. By 6 weeks you probably have all the responses you'll get so it is not very cost-effective or timely to continue collecting data. When doing face-to-face interviews or telephone interviews, be clear and friendly in telling the instructions. Face-to-face can be done on streets, focus groups (closed interview schedule) and is also used for the census. When doing internet surveys, it can be very difficult sampling as the population size may be uncertain. It is possible to find out numbers registered to a certain FaceBook page or group and survey and so you will be able to calculate response rates. However,despite being able to calculate reach on social media, the FaceBook users may not be representative of the population you are trying to survey and this has implications for the generalisability of your findings .Self-selection on the Internet can be completely unstructured, even a named forum (e.g. for junior doctors) will have members who fall outside the sampling frame. What types of information does a survey cover?1. Attributes - personal, socio-economic characteristics such as sex, age, marital status, religion and occupation. 2. Behaviour - what the individual has done, is doing and may possibly do in the future 3. Attitudes - imply evaluation and are concerned with how people feel about an issue. Questions about attitudes usually employ scales - i.e. a statement is made and individuals are asked to indicate their level of agreement in a positive or negative direction. Instrument Design1. After deciding your questions, you must combine them into a questionnaire. 2. A questionnaire is not a hazard collection of questions but is a carefully formulated sequence of questions. 3. The questionnaire is structured to obtain information that meets the requirements of your research project. When laying out your questionnaire, remember filter questions (opening questions) and funnel questions (successive questions, each one is more specific than the last one) and linking questions (usually an open-ended question to engage general opinion about the topic being investigated). General Rules of Questionnaire Construction
Survey Instrument Order1. A cover letter or introduction disclosing the purpose and sponsorship of the survey followed by the instructions. 2. Survey should follow with non-threatening items which arouse interest (can include filter questions and open-ended questions). 3. First question should be clearly related to the announced purposes of the survey (not a background item). 4. Opening questions are important as it encourages the respondent to complete the questionnaire. Non-threatening questions can include demographic - e.g. how many people over the age of 18 live in your house? Note: as you move towards the end of the survey, the lower the response rate! So do not put your open-ended questions to the end (they won't get answered). Put them at the end or middle of sections. You can also change the order of the survey instrument with different respondents. 5. Survey can then proceed with attitudinal questions. 6. Group items into logical coherent sections, i.e. under specific topics. 7. Demographic information can be early on to avoid loss of information due to fatigue. 8. Sensitive background items (e.g. income) should be at the end. By then, you have gained their trust and are more likely to respond. Item Bias - What to Avoid?
ReliabilityReliability is the consistency of a measure of a concept. Internal (Consistency) Reliability This is the most commonly used method to test reliability. It is measured using the Cronbach's alpha statistic (for items with more than 2 response categories) and the Kuder-Richardson (KR-20) test (for items with 2 response categories, e.g. yes/no). Internal consistency involves testing for homogeneity which assumes that there are correlations betwen items on a scale that are not the result of random chance but reflect a real patterning as to how the questions are answered. If the Alpha statistic is < 0.5, then this is regarded as low internal reliability (i.e. the items are not measuring the same phenomonen). Test-retest reliability Does the measure produce the same or similiar results from the same respondents if administered at different points of time? Usually the questionnaire is adminstered on 2 occasions separated by a few days. Ideally, responses shouldn't vary but in health, it is possible that the health status can change inbetween. ValidityValidity is concerned with whether or not the measurement of a concept really measures the concept.
Just think internal validity and external validity! Quick Note on Indices and ScalesWhile nearly every social phenomenon can be measured, not all can be done directly (for instance, predisposition to commit adultery). Indices and scales help us to condense and simplify social and behavioural information and can assess the quality of measurement too. For most purposes, scales and indices are treated as being interchangeable. They should be unidimensional - i.e. all items should fit together to measure a single construct. However there is a difference. An index is constructed through the simple accumulation of scores assigned to specific responses to the individual items comprising the index. In other words, it can be the total number of questions on a construct (you could add up the scores from each question). An index can be measured at an interval or ratio level, which improves reliability and validity as it uses multiple indicators. A scale is constructed through the assignment of scores to response patterns among the several items comprising the scale. It recognizes that some items reflect a relatively weaker degree of the variable. So a scale captures the intensity or direction of a variable by arranging responses on a continuum. Scales are usually ordinal. Taking an example of sexism, if using an index, you just add up the number of prejudiced statements each respondent agreed with. A scale takes into account that agreeing with "women are different to men" is weak evidence of sexism in comparison to agreeing with "women should not be allowed to vote". This takes advantage of any intensity structure that many exist among attributes. You can combine items in a Likert scale to produce a composite index if all items measure a single
construct. A Likert scale is always ordinal, even if you use 10-40 items, it is still 4 categories. Distances between categories are always ordinal, they are not intervals just because numbers are assigned. Good practice is to switch directions in questions to avoid response set - i.e. someone always agrees no matter what the question! You can use -2 to +2, where 0 is neutral and the signs -+ can help denote negative or positive feelings for your respondents. Factor analysis helps to construct indexes, test the unidimensionality of scales, assign weights to items in an index and statistically reduce a large number of indicators to a smaller set. Pretesting the Questionnaire
Piloting the QuestionnaireThis involves conducting the questionnaire under simulated or actual research project conditions. Response RatesAnything below 50% is poor and anything over 90% is excellent. Below 75% survey results can differ significantly from what they would be if everyone answered. For telephone surveys, a non-contact rate of 20% is common. Note Survey Burnout or Feedback FatigueThere has been a rise in survey non-response rates over the past ten years. By non-response we mean people who don't take part in surveys, who don't respond. This is different from non-item response, whereby someone will answer some questions and not others. It sometimes is used interchangeably with refusal rates but often refusal rates are included in non-response rates as 'refusal' denotes actively saying no (and researchers have a means to capture some personal details so we know who's likely to refuse) whilst non-response could be people who aren't bothered, are too busy or have other non-compliant reasons. There are studies being done on the reasons for this increase such as looking at the effects of interviewers as interviewer attributes have been associated with higher non-response rates in panel and cross-sectional surveys. Interviewers play an important role in introducing the survey concept, engaging
the respondent, addressing any queries and gaining responses. Some surveys now build into their design mechanisms to reduce non-response at the design state or data collection by reducing the influences of the interviewer. This can be done through effective policies and management strategies at the research agency. There are also area effects on non-response - meaning some geographical areas have higher non-response rates suggesting that similarities of socio-economic, cultural or other factors
are at play. At the end of the day, keep your survey method quick and easy to use and ensure that the person filling it out knows that their input will help decision making, result in improvements or be invaluable information Further ReadingAldridge & Levine. Surveying the Social World (Oxford University Press, 2001). Babbie, E. Survey Research Methods (Wadsworth, 1990). Babbie, E. The Practice of Social Research (Wadsworth, 1995). Baker, T. Doing Social Research. 2nd Edition. (McGraw-Hill, 1994).[Chapter 7]. Bernard, R. Social Research Methods – Qualitative and Quantitative Approaches (Sage, 2000). Bowling, A. Research Methods in Health. (Open University Press, 2002). Boynton, P. 2004. “Administering, analysing and reporting your questionnaireâ€. BMJ:2004:328:1372-1375. Boynton, P. & Greenhalgh, T. 2004. “Selecting, designing and developing your questionnaireâ€. BMJ:2004:328:1312-1315. Bryman, A. Social Research Methods (Oxford University Press, 2001). [Chapters 3, 4, 5, 6, & 7, all of which address survey design]. Burns, R. B. Introduction to Research Methods (Sage, 2000). De Vaus, D. Research Design in Social Research (Sage, 2001). Gilbert, N. Researching Social Life (Sage, 2001). [Chapters 6 & 7 on questionnaires and measuring attitudes]. Leedy, P.D. Practical Research: Planning and Design (Prentice-Hall, 1997). Neuman, W.L. Social Research Methods: Qualitative and Quantitative Approaches. (Allyn & Bacon, 1997). [Chapter 6 on research design, chapter 7 on quantitative measurement, chapter 9 on sampling and chapter 10 on survey design]. Peterson, R. A. Constructing Effective Questionnaires (Sage, 2000). Seale, C. (Ed). Researching Society and Culture (Sage, 2001). Wright, K.Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring
Software Packages, and Web Survey Services.Journal of Computer Mediated Communication:2005:10(3): 1083-6101. What type of research involves the collection of information from a sample of individuals through their responses to questions quizlet?Terms in this set (5)
SURVEY RESEARCH involves the collection of information from a sample of individuals through their responses to questions.
What is the collection of responses to the survey?Survey data is defined as the resultant data that is collected from a sample of respondents that took a survey. This data is comprehensive information gathered from a target audience about a specific topic to conduct research.
Which is the used to collect information by means of a written questionnaire?A survey is a research method used for collecting data from a pre-defined group of respondents to gain information and insights on various topics of interest.
What are questionnaires in information gathering?A questionnaire is a research instrument consisting of a series of questions for the purpose of gathering information from respondents. Questionnaires can be thought of as a kind of written interview. They can be carried out face to face, by telephone, computer or post.
|