WHY ALL POLLS ARE NOT CREATED EQUAL

G. Terry Madonna and Berwood Yost, Center for Politics & Public Affairs, December, 1996.

Everyone knows that Bill Clinton holds a sizable lead over George Bush, but is the lead nine, ten, twelve, or fifteen? And is the Clinton lead growing, shrinking or staying the same? These questions may mean little in August or September, but they obviously become critical as the election draws nearer. There is substantial evidence that measuring electoral intentions during August and September is a relatively meaningless exercise as large numbers of voters remain unfocused and undecided. Each passing year finds fewer and fewer voters with firm attachments to their political party, and more subject to the vagaries and manipulation of the media and candidates.

Much has been made of electoral volatility in this year's presidential election, but four years ago similar circumstances prevailed. In September, George Bush was either eight points ahead or six points behind Michael Dukakis, and the debate raged over the real meaning of the polls. The same was true in the 1991 Pennsylvania U.S. Senate race, where Dick Thornburgh held a 40 point lead over Harris Wofford in the summer polls. But obviously Thornburgh's greater name recognition was more responsible for the size of his late summer lead than true popularity.

A recent study shows that polls become increasingly more accurate as election day draws nearer: forty-five percent of polls conducted within five days of an election were within three percentage points of final election returns, while only 24% completed 13 or more days in advance of the election were within 3 percentage points. Early polls are, therefore, poor predictors of electoral intentions. Not until October do the important undecided or independent-minded voters pay attention or let alone make up their minds.

Electoral volatility alone, however, does not sufficiently explain the differences in the polls--survey methodology offers another explanation. One of the most often cited sources of inaccuracy is sampling error, which itself is probably not the major source of error in surveys because acceptable sampling techniques are used by most pollsters.

Simply put, there are too many polls, done without proper attention to methodology. Here are some of the common pitfalls of such polling-on-the- cheap:


* Response Rate--The response rate is the number of eligible people in the sample who agree to participate in the survey. If the response rate is 70 percent, for example, seven of ten people in the original sample participated in the study. As response rates decline, the chance that the sample represents the population under study also declines. Many pre-election polls have response rates between 20 and 40 percent, far lower than academic pollsters find acceptable.


* Voter Screens--In pre-election surveys, the voter screen categorizes people by their likelihood of voting. Most pollsters agree that identifying likely voters is a major key to accuracy; non-voters and voters may differ significantly in their candidate preferences. Not all pollsters use the same screen, some don't use any, producing differing results.


* Questions--The sequencing and wording of questions have a profound influence on the responses people provide. Asking issue questions prior to asking candidate's preference may cause people to evaluate candidates in different terms. And small differences in question wording, such as presenting the candidates in a different order, might change responses.

Many times the fourth estate and pollsters are negligent about their responsibility. Sample design and survey methodology should be included in any public release of polling information. Furthermore, heavy emphasis on the horse-race often leads to little substantive analysis of the deeper, fundamental attitudes that polls often uncover. In Pennsylvania, polls have consistently demonstrated that Lynn Yeakel, the Democratic U.S. Senatorial candidate, won a disproportionate number of female voters in the April primary. Virtually no in-depth explanations have been made to explain why. More importantly, the transistion to gender-based voting, whose roots lay in complex societal and economic change, has been inadequately explained by the media and commentators. The real value of polling lies in its ability to explain fundamental expressions of public opinion, whether it be new trends in gender-based voting, abortion attitudes, government's role in society, or views on taxation.