Caution: Use data from polls, surveys in context
ONLINE COLUMN: Web Savvy
Published: November 3, 2009
Data from polls and surveys is the flavor of the times in Web site content and print publications. Because of technology, it's likely that the general public has never been as influenced by poll and survey results as we are now. Polling companies routinely make results available to media, covering what seems to be an infinite number of topics. Data can be found not only on those pesky political topics, but on subjects as diverse as responsible dog ownership and home security. And that data is more often than not presented in truncated fashion, with the results in the header and no information given on the questions, the makeup of the survey respondents' group and no information on the sponsor. Kay B. Day
For instance, imagine we're doing a survey on the number of people who put deodorant on before they're going to the gym. We could ask two simple questions: 1) Do you apply deodorant immediately before heading to the gym or 2) Do you not apply the same? That looks like a simple 'yes' or 'no' response. But the respondent holding the phone may think to herself, "Not just before going, but I do apply it in the morning before going to work." But the questioner is interested only in whether you're grabbing the stuff that makes you smell better right before your workout, and he says, "Just a simple yes or no, please." If most people said 'no,' the next morning header might be "Gym users opt to smell bad before workout." That's an inane example, admittedly, but it's actually a typical example of how writers often miss the true results of a poll by sensationalizing a single piece of data.
Survey organizations do have guidelines, but even large media organizations ignore them. The Council of American Survey Research Organizations, founded in 1975 and representing more than 300 companies and research operations, offers simple guidelines "for any general public release of survey findings" to include the following:
Survey data should be presented in context—it's that simple. If we sensationalize results without informing the reader about the longtime standard 'who, what, when, why and where,' we are shortchanging the reader. Most are aware of Mark Twain's remarks about the three kinds of lies-"lies, damned lies and statistics." Twain reportedly attributed the quote to British Prime Minister Benjamin Disraeli. But the concerns of yesteryear are the concerns of today, with the difference being a remarkable breadth of scope in distributing information.
1. The sponsorship of the study.
2. A description of the purposes.
3. The sample description and size.
4. The dates of data collection.
5. The names of the research company conducting the study.
6. The exact wording of the questions.
7. Any other information that a lay person would need to make a reasonable assessment of the reported findings.
Major media outlets may ignore standards for reporting results, but as writers and content providers we may be able to effect change one step at a time. Our goal is to accurately inform the public, not to mislead them. And it would appear industry experts who benefit from surveys feel the same way.
We live in an age when outrageous headlines draw eyeballs. Perhaps we need to rethink our approaches to how we select what we read in addition to taking care in how we present what we write.
• Council of American Survey Research Organizations, Code of Standards
• Nieman Watchdog: Reporting on public opinion polls [Nieman Foundation for Journalism at Harvard University]
• Display of statistical data—excellent primer from the physics department at the College of St. Benedict/St. John's University
• Ethics in online journalism— how do you measure up? [Web Savvy at The Writer]
--Published Nov. 3, 2009
Join us for our next Web Savvy as we talk to busy freelancer Jackie Dishner about her new book and how she works the Web to get the word out.
|Kay B. Day|
Florida journalist Kay B. Day has won awards for poetry, nonfiction and fiction. The author of two books, she has written for The Christian Science Monitor, United Press International, The Florida Times-Union and Sky News. To read Kay's other Web Savvy columns about writing for the Web, click here.