"It must be true; it was in an online poll."
If you believe that, I know a guy in New York with a good deal on a world-famous bridge, or a $10 Rolex.
It is easy to conduct an informal, unscientific, poll online. Numerous sites, such as Survey Monkey, allow free small polls, with statistics. For modest fees, you can get powerful tools. Interesting, but not a valid way to assess public opinion.
Some publications use online polling, for fun, for giving readers a sense of interactivity, for trolling up some good citizen quotes and such.
But that can provoke vigorous discussion among opinion writers and editors. We had another round of it on the AOJ members' discussion list recently.
Jay Jochnowitz, spoofing the use of online polls (which he admittedly employs on his paper’s opinion blog), wrote: "We present this online poll mainly to give you something interactive to do without thinking too hard. ... Our only interest is how many clicks it got."
Tricia Vance told of local legislative candidates who each "cited 'independent' polls showing him ahead by double digits." She quoted Dire Straits' "Industrial Disease" -- "Two men say they're Jesus; one of them must be wrong." She urged colleagues to be skeptical about all polls "and remind readers that even the best polls are but a snapshot in time."
Journalists should be aware of things to be considered before reporting on polling.
A common list:
- Who paid;
- who did it;
- how were respondents found;
- how many were asked;
- how many completed the survey and response rate;
- when was it done;
- how (phone or in-person, or....);
- were the results adjusted to population demographics;
- numerical results;
- margin of error (usually about 2% to 5%) and confidence level (usually 95%, or wrong at least one time in 20);
- sizes and margins of subsamples;
- pollster's comments about conclusions and applicability; and
- the actual questions.
For a professional research association's standards, go to the AAPOR ethics document and fast-find (Ctrl-F) to section III. Standards for Disclosure.
A search-engine review found several polls on media credibility. A few are linked below.
Some of these polls apparently used sound social-science research methods, although this was not supported, because the "how-we-did-it" was all-too-often missing. That alone is a red-flag.
Some were online "polls," inherently useless as indicators of public opinion. One reason is that the respondents are "self-selected," the most motivated (or bored) of the few Internet users who somehow saw the poll and chose to take it.
Many such polls allow repeat responses.
Sponsored research, even if scientifically sound, often reports results favorable to the sponsor. One explanation is that sponsors choose not to announce private polls that get unfavorable results.
Even in relatively independent scientific polling, differences in timing, randomized selection of interviewees, and -- especially -- exact wording of the questions can make a huge difference. Thus we can see ostensibly independent, professional polls with opposite results.
Take it all with a huge grain of salt. But do ask yourself this: Who else but a professional news organization has the expertise and independence to be informing -- or advising -- the public on such things as local elections?
John McClelland was a working journalist for decades before teaching 22 years at Roosevelt University in Chicago. Now largely retired, he began editing Masthead near the end of 2012.
Here are some of the sites reporting news-credibility poll results by 9/12/12: (back to reference) (top)
Newspapers, tv most trusted (August 2012, L.A. Times, found in St.Louis online)
Further decline in credibility (August 2012, Pew Research, generally thorough & verbose)
Newspapers most trusted (2012, think-tank summary via Poynter)
Newspapers most trusted (2012, same study, with 1,000 interviews cited in Bulldog blog; DC-based polling by Lincoln Park Strategies; LPS site indicates tendency to advise Democratic candidacies.)
Press criticized, trusted (2011, Pew social research tends to be thorough & verbose)
Newspaper sites most trusted for local news (2010 via MediaPost, citing 3,050 interviews)
Voters read; readers vote (2012, clearly newspaper industry ad-related sponsorship; combination of online and apparently sound research)
TV, newspapers most trusted (2008, apparently broadcast industry sponsorship, pro-grade polling info)
Ranking of source types (2012, ongoing beta test by Poynter; dependent on self-selecting reviews)
Traditional media most trusted (2012, based on 24,000 responses via Triton Digital online platform; somewhat self-selecting; co-sponsored by Rochester Institute of Technology and a California print organization.)
### (posted 9/13/12 11a edt; minor update 11:51)
