Chapter 1 - What is a Survey?
It has been said the United States is no longer an "industrial society" but an "information society." That is, our major problems and tasks no longer mainly center on the production of the goods and services necessary for survival and comfort.
Our "society," thus, requires a prompt and accurate flow of information on preferences, needs, and behavior. It is in response to this critical need for information on the part of the government, business, and social institutions that so much reliance is placed on surveys.
Then, What Is a Survey
Today the word "survey" is used most often to describe a method of gathering information from a sample of individuals. This "sample" is usually just a fraction of the population being studied.
For example, a sample of voters is questioned in advance of an election to determine how the public perceives the candidates and the issues ... a manufacturer does a survey of the potential market before introducing a new product ... a government entity commissions a survey to gather the factual information it needs to evaluate existing legislation or to draft proposed new legislation.
Not only do surveys have a wide variety of purposes, they also can be conducted in many ways-including over the telephone, by mail, or in person. Nonetheless, all surveys do have certain characteristics in common.
Unlike a census, where all members of the population are studied, surveys gather information from only a portion of a population of interest-the size of the sample depending on the purpose of the study.
In a bona fide survey, the sample is not selected haphazardly or only from persons who volunteer to participate. It is scientifically chosen so that each person in the population will have a measurable chance of selection. This way, the results can be reliably projected from the sample to the larger population.
Information is collected by means of standardized procedures so that every individual is asked the same questions in more or less the same way. The survey's intent is not to describe the particular individuals who, by chance, are part of the sample but to obtain a composite profile of the population.
The industry standard for all reputable survey organizations is that individual respondents should never be identified in reporting survey findings. All of the survey's results should be presented in completely anonymous summaries, such as statistical tables and charts.
How Large Must The Sample Size Be
The sample size required for a survey partly depends on the statistical quality needed for survey findings; this, in turn, relates to how the results will be used.
Even so, there is no simple rule for sample size that can be used for all surveys. Much depends on the professional and financial resources available. Analysts, though, often find that a moderate sample size is sufficient statistically and operationally. For example, the well-known national polls frequently use samples of about 1,000 persons to get reasonable information about national attitudes and opinions.
When it is realized that a properly selected sample of only 1,000 individuals can reflect various characteristics of the total population, it is easy to appreciate the value of using surveys to make informed decisions in a complex society such as ours. Surveys provide a speedy and economical means of determining facts about our economy and about people's knowledge, attitudes, beliefs, expectations, and behaviors.
Who Conducts Surveys
We all know about the public opinion surveys or "polls" that are reported by the press and broadcast media. For example, the Gallup Poll and the Harris Survey issue reports periodically describing national public opinion on a wide range of current issues. State polls and metropolitan area polls, often supported by a local newspaper or TV station, are reported regularly in many localities. The major broadcasting networks and national news magazines also conduct polls and report their findings.
The great majority of surveys, though, are not public opinion polls. Most are directed to a specific administrative, commercial, or scientific purpose. The wide variety of issues with which surveys deal is illustrated by the following listing of actual uses
- Major TV networks rely on surveys to tell them how many and what types of people are watching their programs
- Statistics Canada conducts continuing panel surveys of children (and their families) to study educational and other needs
- Auto manufacturers use surveys to find out how satisfied people are with their cars
- The U.S. Bureau of the Census conducts a survey each month to obtain information on employment and unemployment in the nation
- The U.S. Agency for Health Care Policy and Research sponsors a periodic survey to determine how much money people are spending for different types of medical care
- Local transportation authorities conduct surveys to acquire information on commuting and travel habits
- Magazine and trade journals use surveys to find out what their subscribers are reading
- Surveys are conducted to ascertain who uses our national parks and other recreation facilities.
Surveys provide an important source of basic scientific knowledge. Economists, psychologists, health professionals, political scientists, and sociologists conduct surveys to study such matters as income and expenditure patterns among households, the roots of ethnic or racial prejudice, the implications of health problems on people's lives, comparative voting behavior, and the effects on family life of women working outside the home.
What Are Some Common Survey Methods
Surveys can be classified in many ways. One dimension is by size and type of sample. Surveys also can be used to study either human or non-human populations (e.g., animate or inanimate objects -- animals, soils, housing, etc.). While many of the principles are the same for all surveys, the focus here will be on methods for surveying individuals.
Many surveys study all persons living in a defined area, but others might focus on special population groups-children, physicians, community leaders, the unemployed, or users of a particular product or service. Surveys may also be conducted with national, state, or local samples.
Surveys can be classified by their method of data collection. Mail, telephone interview, and in-person interview surveys are the most common. Extracting data from samples of medical and other records is also frequently done. In newer methods of data collection, information is entered directly into computers either by a trained interviewer or, increasingly, by the respondent. One well-known example is the measurement of TV audiences carried out by devices attached to a sample of TV sets that automatically record the channels being watched.
Mail surveys can be relatively low in cost. As with any other survey, problems exist in their use when insufficient attention is given to getting high levels of cooperation. Mail surveys can be most effective when directed at particular groups, such as subscribers to a specialized magazine or members of a professional association.
What Survey Questions Do You Ask
- Telephone interviews are an efficient method of collecting some types of data and are being increasingly used. They lend themselves particularly well to situations where timeliness is a factor and the length of the survey is limited.
- In-person interviews in a respondent's home or office are much more expensive than mail or telephone surveys.
They may be necessary, however, especially when complex information is to be collected.
- Some surveys combine various methods. For instance, a survey worker may use the telephone to "screen" or locate eligible respondents (e.g., to locate older individuals eligible for Medicare) and then make appointments for an in-person interview.
You can further classify surveys by their content. Some surveys focus on opinions and attitudes (such as a pre-election survey of voters), while others are concerned with factual characteristics or behaviors (such as people's health, housing, consumer spending, or transportation habits).
Many surveys combine questions of both types. Respondents may be asked if they have heard or read about an issue ... what they know about it ... their opinion ... how strongly they feel and why... their interest in the issue ... past experience with it ... and certain factual information that will help the survey analyst classify their responses (such as age, gender, marital status, occupation, and place of residence).
Questions may be open-ended ("Why do you feel that way?") or closed ("Do you approve or disapprove?"). Survey takers may ask respondents to rate a political candidate or a product on some type of scale, or they may ask for a ranking of various alternatives.
The manner in which a question is asked can greatly affect the results of a survey. For example, a recent NBC/Wall Street Journal poll asked two very similar questions with very different results: (1) Do you favor cutting programs such as social security, Medicare, Medicaid, and farm subsidies to reduce the budget deficit? The results: 23% favor; 66% oppose; 11% no opinion. (2) Do you favor cutting government entitlements to reduce the budget deficit? The results: 61% favor; 25% oppose; 14% no opinion.
The questionnaire may be very brief -- a few questions, taking five minutes or less -- or it can be quite long -- requiring an hour or more of the respondent's time. Since it is inefficient to identify and approach a large national sample for only a few items of information, there are "omnibus" surveys that combine the interests of several clients into a single interview. In these surveys, respondents will be asked a dozen questions on one subject, a half dozen more on another subject, and so on.
Because changes in attitudes or behavior cannot be reliably ascertained from a single interview, some surveys employ a "panel design," in which the same respondents are interviewed on two or more occasions. Such surveys are often used during an election campaign or to chart a family's health or purchasing pattern over a period of time.
Who Works on Surveys
The survey worker best known to the public is the interviewer who calls on the telephone, appears at the door, or stops people at a shopping mall.
Traditionally, survey interviewing, although occasionally requiring long days in the field, was mainly part-time work and, thus, well suited for individuals not wanting full-time employment or just wishing to supplement their regular income.
Changes in the labor market and in the level of survey automation have begun to alter this pattern-with more and more survey takers seeking to work full time. Experience is not usually required for an interviewing job, although basic computer skills have become increasingly important for applicants.
Most research organizations provide their own training for the interview task. The main requirements for interviewing are an ability to approach strangers (in person or on the phone), to persuade them to participate in the survey, and to collect the data needed in exact accordance with instructions.
Less visible, but equally important are the in-house research staffs, who among other things-plan the survey, choose the sample, develop the questionnaire, supervise the interviews, process the data collected, analyze the data, and report the survey's findings.
In most survey research organizations, the senior staff will have taken courses in survey methods at the graduate level and will hold advanced degrees in sociology, statistics, marketing, or psychology, or they will have the equivalent in experience.
Middle-level supervisors and research associates frequently have similar academic backgrounds to the senior staff or they have advanced out of the ranks of clerks, interviewers, or coders on the basis of their competence and experience.
What About Confidentiality and Integrity
The confidentiality of the data supplied by respondents is of prime concern to all reputable survey organizations. At the U.S. Bureau of the Census, for example, the data collected are protected by law (Title 13 of the U.S. Code). In Canada, the Statistics Act guarantees the confidentiality of data collected by Statistics Canada, and other countries have similar safeguards.
Several professional organizations dealing with survey methods have codes of ethics (including the American Statistical Association) that prescribe rules for keeping survey responses confidential. The recommended policy for survey organizations to safeguard such confidentiality includes
What Are Other Potential Concerns
- Using only number codes to link the respondent to a questionnaire and storing the name-to-code linkage information separately from the questionnaires
- Refusing to give the names and addresses of survey respondents to anyone outside the survey organization, including clients
- Destroying questionnaires and identifying information about respondents after the responses have been entered into the computer
- Omitting the names and addresses of survey respondents from computer files used for analysis
- Presenting statistical tabulations by broad enough categories so that individual respondents cannot be singled out.
The quality of a survey is largely determined by its purpose and the way it is conducted.
Most call-in TV inquiries (e.g., 900 "polls") or magazine write-in "polls," for example, are highly suspect. These and other "self-selected opinion polls (SLOPS)" may be misleading since participants have not been scientifically selected. Typically, in SLOPS, persons with strong opinions (often negative) are more likely to respond.
Surveys should be carried out solely to develop statistical information about a subject. They should not be designed to produce predetermined results or as a ruse for marketing and similar activities. Anyone asked to respond to a public opinion poll or concerned about the results should first decide whether the questions are fair.
Another important violation of integrity occurs when what appears to be a survey is actually a vehicle for stimulating donations to a cause or for creating a mailing list to do direct marketing.
Where Can I Get More Information
In the preface to this booklet provided many general suggestions are made that might be pursued for more information. There seems to be no need to repeat these here. One point of information that might be of interest is that the clever acronym, SLOPS was coined by Norman Bradburn who used to head up NORC.