Reported research results rarely tell a true story

Way back in the dark ages, when I was in graduate school getting a master’s in communication, the subject of opinion research so interested me that I developed and conducted a survey for my master’s project.

The result of that intense effort has been my enduring interest and unmitigated skepticism about polls and other studies that are disseminated by the news media.

Although the news media have grown more sophisticated, especially about political polls, I find myself to be continually astonished by how much evidently shoddy research finds validation on the Internet, in newspapers and magazines, and on radio and television.

Regular readers no doubt recall my recurring rants about research on the nation’s housing market, none of which is entirely — or, for that matter — even mostly accurate. Case-Shiller is my favorite target, a great example being in Sunday’s New York Times, when Shiller cited his research based on 407 and 296 respondents in different years as if they represented a national sample of home buyers.  Impossible!

For that matter, how could 407 and 296 responses each reflect national sentiment?  If 296 is sufficient, why poll 407?  Conversely– you get the idea.

If only Shiller were alone.  But none of the others — not Trulia, not Zillow, not RealtyTrac, not CoreLogic, not Radar logic, not the federal government, not one — reveals the true story.

Findings may be out of date, Continue reading

Advertisements