Mountaineering on a molehill - Chasing the Wind by Felipe B. Miranda

Recently, a possibly well-meaning critic of a Pulse Asia public opinion survey engaged media in a presscon to point out flaws in the way the respondents had been selected. The "flawed sample", to use the words of the critic, Mr. Larry Henares, one-time Chairman of the National Economic Council and currently one of the most articulate and voluble TV talk show hosts, made it possible to distort the sentiments of the Metro Manila public and consequently raised doubts about the validity of the survey’s findings.

He argued that the method of sampling and the ensuing choice of survey respondents had led to an anomalous situation "such that areas where Erap is weak are grossly underrepresented by as much as one half, while and (sic) the areas where Erap is strong is (sic) grossly over represented (sic) as much as three-fold to four-fold." With proper sampling and appropriate data weighting to cure this defect, Mr. Henares suggests that significantly different survey findings from those reported by Pulse Asia could have emerged.

Within the same hour of Mr. Henares’ presscon, media people from TV, radio and newspaper groups descended on the Pulse Asia office in UP Diliman, demanding, reactions from its completely surprised head. (I had discussed the background of this case in my last column – An academic’s reflection on a most unacademic experience, in The Philippine STAR’s edition of December 10, 2000 – and refer any interested party to that issue for further background information). For the next three days, media interest was strong in the possibility not only of a survey’s "flawed sample" but an outright survey scam, complete with spindoctoring and juicy survey pay-offs to Pulse Asia by partisan political interests.

Within the UP, at least one nationally prominent professor pointedly reminded academic people doing survey work about "their membership in the social science community" and how they must not be mistaken to be "just guns for hire" and how "[t]hey must choose between the roles of hired consultants and independent scientist", since "[t]hey cannot be both."

So much sound and fury.

The better thing to do, of course, was simply to look into the critic’s claims of possible discombobulation of survey findings due to the use of a "flawed sample." In attending to this academic task, after two days and nights of intensively reviewing the survey’s sampling procedures and re-examining the data and their analysis, the following could now be easily established:

(1) The critic had wrongly identified the Rubicon 2’s relevant sample population as "registered voters" of Metro Manila when in fact it was simply "adults 18 years old and over" whether they are registered voters or not. Conceptually these are two distinct target populations even as there might be a very close correlation between them.