Survey FAQ


Frequently Asked Questions

about where the statistics in Just the Facts come from:

 

The statistics presented in the campaign -- "2 out of 3 WU students had 4 or fewer drinks the last time they partied" -- are based on information provided by WU students in these survey instruments:

  • AlcoholEdu for College , an online course/survey administered to freshmen by Outside the Classroom, Inc., in 2005 and 2006
  • the National College Health Assessment (ACHA-NCHA) administered via e-mail by the American College Health Association in 2007 and 2004.
  • the Survey of College Alcohol Norms and Behavior (SCANB) administered by the Education Development Center (EDC) via U.S. Mail in 2004, 2003, and 2002 as part of a national study.

 

Question: This data is not new. Is it still accurate?

 

Answer: Most survey research looking at trends in health behaviors is conducted on a biennial or triennial basis (i.e., every 2-3 years). Our nations’ census is only conducted every 10 years! Data from a few years ago is considered extremely accurate. Given the amount of time it takes to process and report data after it has been collected, this is often as current as is possible.

Q: Do students tell the truth on surveys?

 

A: Decades of research suggests that self reports of alcohol use are an accurate source of data about drinking . Many surveys have been administered to assess drinking behavior, and studies have examined the validity, reliability, and effectiveness of these surveys.

 

This research has shown that self-report data on alcohol use are valid. In simple terms, this means that the survey questions regarding alcohol use are accurately measuring the behaviors they are intending to measure. In scientific terms, validity is demonstrated by showing that it is highly correlated with other measures of alcohol consumption including reports by friends, statistics on alcohol-related problems, and scientific testing (chemical analysis of blood alcohol content)

 

In addition, self-reports of drinking have been found to be reliable through examining test-retest reliability (Sobell et al., 1988). That means that individuals respond consistently to questions about their drinking at multiple points in time.

  

Q: Won’t some students lie as a joke or to mislead the researchers?

 

A: While some students may lie as a joke to mislead researchers, research has shown that if students spend the time to respond, they tend to provide truthful responses. A small percentage of students may choose to lie in a way which exaggerates their responses; they are generally not subtle in their responses, and their surveys are therefore easy to identify. To account for such responses, researchers do a thorough “cleaning” of the data prior to any analyses to determine whether any students have provided impossible or illogical responses. These surveys are discarded. A small percentage of students may also under-report their behaviors, particularly if they feel their behaviors are socially undesirable. These types of surveys are harder to detect but tend to balance out the over-reporting that may also exist.

 

Q: Was the data manipulated to support a cause?

 

A: We receive the data in its raw form with no statistical analyses. Only frequencies (or the number of times a response occurs) are reported. The only exception is a “cleaning” of the data to determine whether any “impossible” responses have been given (i.e., a student states that they have 99 drinks when they go out). Other than that, what you are seeing is truly Just the Facts!

 

Q: Washington University has strong reasons for wanting to portray moderate drinking among students. How can I believe this data?

 

A: T he SCANB survey was not conducted by Washington University . It was conducted by an outside institution, Education Development Center, Inc. (EDC), a highly regarded national public health research institution. EDC uses rigorous sampling and research techniques and conducts the survey from its offices in Newton , Massachusetts to ensure that the methodology is strictly adhered to. EDC must follow guidelines for research set by the National Institutes of Health and must adhere to principals for protecting the rights of research participants (in this case, students who are asked to complete the survey). This means that students are made aware of the survey procedures, including the fact that it is both anonymous and voluntary, as well as any potential risks and benefits of completing the survey (both of which are minimal). The NCHA was administered by WU but conducted by the American College Health Association. AlcoholEdu surveys were conducted by Outside the Classroom. Both of these organizations adhere to the same principles.

 

 

Q: Not all students responded to the surveys. How do we know that those who did portray our school accurately?

 

A: When a response rate for a survey is high, the results portray the population being surveyed accurately. The response rate for the SCANB is high in comparison with other surveys of college students. Generally, college-based researchers report response rates between 20-60 percent for a student survey similar to the SCANB. WU’s response rate in spring 2003 was 60.2%. WU's response rate in spring 2004 was 52.8%. By comparison, Gallup , Roper, Louis Harris, and other major polling organizations usually produce response rates of about 35% for their telephone polls. WU’s response rate for the NCHA was and 34% in 2007 and 45% in 2004. AlcoholEdu is administered to all freshmen before they come to campus with a 98% average response rate.

 

 

 

References

 

Meier, SE, Brigham, TA, & Handel, G (1987). Accuracy of drinkers’ recall of alcohol consumption in a field setting. Journal of Studies on Alcohol 48(4): 325-8.

 

Newport , F, Saad, L, & Moore, D (1997). "How Polls are Conducted," Where America Stands (Gallup Organization: John Wiley & Sons, 1997). Website: http://www.gallup.com/help/FAQs/poll1.asp

 

Nurco, DN (1985). A discussion of validity. In BA Rouse, NJ Kozel & LG Richards (Eds.), Self-report methods of estimating drug use: Meeting current challenges to validity [NIDA Research Monograph 57] (4-21). Rockville , MD : National Institute on Drug Abuse.

 

Sobell, LC, Sobell , MB , Leo, GI, & Cancilla, A (1988). Reliability of a timeline method: Assessing normal drinkers’’ reports of recent drinking and a comparative evaluation across several populations. British Journal of Addiction, 83(4): 393-402.