PRE-ELECTION SURVEY PREDICTIONS: HOW ACCURATE WERE THEY

G. Terry Madonna and Berwood Yost, Center for Politics & Public Affairs, November, 1992.

Pre-election polls took a public beating this year, but was the criticism justified? An analysis of Pennsylvania's pre-election polls tells us that sometimes it might have been, but often it was not. Most surveys completed within two weeks of the general election accurately predicted Bill Clinton's margin of victory in Pennsylvania. Clinton's actual margin of victory over George Bush was nine percentage points (45% to 36%), and five of the six polls analyzed were very close to Clinton's final margin (within two percentage points). Pollsters probably benefited from high turnout, estimated to be 83% of registered voters. Higher turnout has traditionally resulted in more accurate surveys.

On the other hand, Pennsylvania's U.S. Senate race proved more difficult to measure. Arlen Specter narrowly defeated Lynn Yeakel (51% to 48%), and only half of the surveys reviewed were within two points of Sepcter's actual margin. Furthermore, merely one-third of the polls were within two points of the actual margins for both races.

Estimated Margins of Victory - Pre-Election Polls in Pennsylvania
Specter Clinton WNEP-TV +10 + 9 Mansfield University + 7 + 9 Pittsburgh Post Gazette/WTAE + 4 + 7 Millersville/Penn State Harrisburg + 4 +10 Greensburg-Tribune + 2 +15 Mason-Dixon - 1 + 9 Actual Margin Of Victory In Pennsylvania + 3 + 9.

Why were some polls right and others wrong? And, moreover, why were some polls correct for one race and wide of the mark on the other? The answer probably relates to the differences in how each survey was conducted. Unfortunately, many pollsters and their clients do not make the details of survey methodology available, which makes understanding of these differences impossible, and considerable reduces the public's ability to discern good polls from bad ones.

Methodology aside, the dynamics of the Specter-Yeakel race made it difficult to measure for several reasons. First, most polls showed a large number of undecided voters going into the final week of the campaign. Assessing undecided voters' intentions is generally difficult. Even though most undecided voters lean toward one candidate or another, some of these voters will change their minds. Also, more than one in ten likely voters were genuinely undecided in the Sepcter-Yeakel contest a wekk before the election, making final predictions tenuous.

Second, gender-based voting also played a role in the Sepcter-Yeakel race. In early July, Lynn Yeakel held a tremendous twenty-two point lead among female voters. By the campaign's end, Specter had closed the gender gap (37% Sepcter to 39% Yeakel among female voters). Exit polls indicated the gender gap re-emerged as Lynn Yeakel carried the women's vote by twelve points, probably because undecided female voters went her way.

Third, a new and potentially troublesome problem for pollsters may have appeared in this year's senatorial election. An examination of the Keystone State Poll indicates that the sex of an interviewer may have had some affect on a respondent's answers. Male respondents who spoke to female interviewers were more likely to express support for Arlen Specter's than those who spoke to male interviewers, thus overstating Specter's support. And more polls overstated than understated Specter's support.

Poll bashing was common this year, and in some cases with good reason because all polls are not completed with the same degree of methodological rigor. The media's responsibility extend beyond publishing polling results. Polls released for public consumption should describe the basic elements of survey methodology in such a way as to give the interested readers an opportunity to understand how the survey was conducted.