UAB College of Arts and Sciences, Heritage Hall 305, Edinburg, Texas, USA
Received Date: January 11, 2017; Accepted Date: March 27, 2017; Published Date: April 05, 2017
Citation: Powell L (2017) Online vs. Telephone Surveys in Political Campaign Research. J Mass Communicat Journalism 7: 332. doi: 10.4172/2165-7912.1000332
Copyright: © 2017 Powell L. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Mass Communication & Journalism
The growing emergence of cell phones and caller ID have reduced response rates among telephone polling, causing some concern among those who conduct public opinion polls in politics. That issue has led some to consider online surveys as either an alternative or at least a supplemental technique for gauging political opinions. This study sought to test this concept by conducting two identical surveys – on with live telephone interviews and one with an online survey. The results indicated that the data from the two surveys were not identical. Hillary Clinton scored higher on image ratings with the online survey, and the data for the voter optimism were also different. One possible explanation is that the online surveys are less susceptible to errors caused by a socially desired response pattern. That offers the potential for more accuracy from online surveys.
Telephone; Political surveys; Causing; Online
The growing emergence of cell phones and caller ID have reduced response rates among telephone polling, causing some concern among those who conduct public opinion polls in politics. That issue has led some to consider online surveys as either an alternative or at least a supplemental technique for gauging political opinions [1,2]. Not surprisingly, several research studies have been conducted to examine the efficacy and efficiency of such approaches. Uhlig et al.  argued that online surveys were more efficient than paper-based surveys. Wahlberg and Wåhlberg and Poom  argued that nonresponse bias in Internet surveys is relatively small. Lugtig and Toepoel  argued that online polls may have less error since they can be accessed through multiple devices – computers, smartphones and tablets.
On the negative side, Hays et al.  were concerned about false answers, careless responses, and panelists being members of multiple survey panels in online surveys. Pasek  argued that online surveys do not match the requirements of modern random samples, at least not yet. Xing and Handy  noted that there are significant demographic differences between online surveys and telephone surveys, even though the actual results of the surveys may not differ. Verma et al.  agreed, noting that Internet polls may exclude some important demographic groups. Remillard et al.  was particularly concerned about the exclusion of some subsets of older adults in online samples.
Gigliotti  however, argued that the latter problem could be alleviated with a mixed-mode approach. Further, Stephenson and Crete  noted that while demographic differences in the data will occur, those differences do not affect the inferences or conclusions that are drawn from the data – particularly in terms of political behavior.
Thus the conclusion to be drawn from most past research is that online surveys are likely a major area of consideration for political polling, but there is still uncertainty over exactly how to use them and whether they exclude (or at least reduce) participation by significant subsets of voters. This study attempted to provide more information on this issue by looking at differences in online surveys, as compared to telephone surveys, in terms of voter responses on three categories of responses – evaluations of candidate images, evaluations of voters’ mood of the electorate, and voter preference in terms of presidential candidates. Thus four null hypotheses were tested.
H1: There will be no statistically significant differences between the image ratings that Hillary Clinton receives from an Internet sample and a telephone sample.
H2: There will be no statistically significant differences between the image ratings that Donald Trump receives from an Internet sample and a telephone sample.
H3: There will be no statistically significant differences between evaluations of the direction of the nation in terms of an Internet sample and a telephone sample.
H4: There will be no statistically significant differences between the voter preferences for Hillary Clinton vs. Donald Trump in terms of an Internet sample and a telephone sample.
The participants were 623 voters who participated in two random samples in a southern state. The total sample for telephone participants was 245, with a median age of 53.7; the telephone sample consisted of 56.6% Democrats and 28.0% Republicans. In terms of ethnic background, the telephone sample was composed of 58.1% white voters and 32.9% African-Americans. In terms of gender, the telephone sample was 40.6% men and 59.4% women. The total sample for Internet participants was 377, with a median age of 50.9; the Internet sample consisted of 43% Democrats and 38.1% Republicans. In terms of ethnic background, the Internet sample was composed of 71.9% whites and 24.7% African-American. In terms of gender, the Internet sample was 40% men and 60% women.
The survey questionnaires were identical for both the telephone and Internet samples and consisted of 41 questions. Most of the questions were about local politics within the state, but two four research questions were also included. Evaluations of candidate images (hypotheses 1 and 2) were operationalized with two five-item Likert-type questions to measure the images of the two presidential candidates, Hillary Clinton and Donald Trump (very positive to very negative). Evaluations of the mood of the electorate (hypothesis 2) were operationalized with one nominal-data question regarding the direction of the nation (right direction, wrong direction, unsure). Voter preference on presidential candidates (hypothesis 4) was operationalized with one nominaldata test-ballot question which asked the participants to identify their voting preference for president (Clinton vs. Trump). In addition, there were a series of demographic questions (age, party, ethnic background, gender) to test the similarities between the two samples.
The telephone sample was gathered using a randomized telephone sample over four days. The Internet sample was gathered by using the resources of Survey Analytics, which uses a random database of email contacts to solicit participants for online surveys.
Hypotheses 1 and 2 were tested with a t test to evaluate mean differences in image ratings for the two candidates.
Hypotheses 3 and 4 were tested with Chi Square analyses for nominal differences in the frequencies of the two variables.
Null Hypothesis 1 was rejected. Hillary Clinton received an average positive rating of 3.57 (out of 5.0) from the Internet sample, but only 3.05 from the telephone sample (t=4.01, sd=1.73, p<0.001).
Null Hypothesis 2 was accepted. There was no significant difference between Donald Trump’s rating from the Internet (3.38) or the telephone (3.34) samples (t - 0.61, sd=1.63).
Null Hypothesis 3 was rejected. There was a significant difference in vote support between the two samples. Clinton has a significant lead over Trump in the Internet sample (50.4% to 35.0%), but significantly trails him in the telephone sample (37.8% to 48.7%, X2=48.44, p<0.005).
Null Hypothesis 4 was rejected. The Internet sample was more pessimistic than the telephone sample. Nearly six out of ten in the Internet sample thought the nation was going in the wrong direction (59.9%), while only 20.6% responded that it was headed in the right direction. The telephone sample was almost equally split (45.5% right direction, 46.3% wrong direction). This result was statistically significant (X2=12.13, p<0.005).
These results were not due to age, gender, ethnic background or partisan differences in the samples. There was only a small age difference between the two samples (less than three years median age), and no difference in gender between the two samples. There was an ethnic difference in the samples, but that was a bias among more African-American voters in the telephone sample. Clinton’s higher ratings from the Internet sample is counter to conventional beliefs that she does better among African-American voters. There was also a partisan difference in the samples, but that difference (an edge among Republicans in the Internet sample) is also counter to the argument that she should receive better ratings from Democrats.
This study supported previous research which has identified demographic differences between the participants in online surveys and those in traditional survey methodologies. Contrary to some studies, there was no significant difference in the age of the participants in the two studies. This result may be an indication that older adults are increasingly becoming part of the online community, and that demographic difference could soon disappear. Further, there was no gender difference in the two samples.
There were, however, significant differences on two demographics that are important to political surveys – partisanship and ethnic background. Online surveys were more likely to be biased by a higher percentage of white participants and a higher percentage of Republican participants. Further, while these differences did not directly influence the result of the study, this current study found that there were significant differences on three of the four test variables. Online participants were more likely to indicate that they had a positive image of Hillary Clinton, that they intended to vote for Hillary Clinton, and that they had a pessimistic outlook on the future of the nation.
There are at least three possible explanations for these results. First, they may be idiosyncratic differences that reflect the uniqueness of the 2016 presidential campaign. Donald Trump, as the Republican nominee, has managed to alienate large groups of voters, including members of his own party (e.g., Collins, 2016; Dowd, 2016). The latter problem, in particular, may be influencing the positive support for Clinton (both in image and vote support) in the online sample. As to why that effect shows up only in the online sample, that may be a byproduct of those voters using online sources as the basis for their information as well as the expression of their attitudes in the survey.
A second possibility is that the reduced number of Democrats in the online sample may be a result of less online access for African- American voters, a problem that could potentially increase both the percentage of white voters and of Republicans in the online sample. This explanation, however, does not account for the differences in results regarding Clinton and the mood of the electorate.
The third possibility is that the online survey was less susceptible to errors generated by a socially desired response pattern (SDRP). At the time of the survey, Hillary Clinton was facing a great deal of negative publicity over the email issue and the Benghazi attacks. Such information may have made some voters less likely to rate her positively in the live survey, but no such impediment would exist with the online survey. The same external factors would also explain the differences from the optimism data and vote support for Clinton, particularly the latter. Given the negative publicity that she was receiving, some of her supporters may have been less likely to voice that support to a live interviewer. A similar problem occurred near the end of the election for Donald Trump, who was faced with heavy media coverage. Consequently, surveys that used live telephone interviews incorrectly predicted that Clinton would win the election (Wang, 2016). The only accurate predictions came from online surveys. Again, online surveys seemed to be less susceptible to errors created by an SDRP.
Thus more research on this topic is needed, particularly in terms of socially desired response patterns. Improving the accuracy of political surveys will depend on a more thorough understanding of online polling and its role in modern polling operations. Is there a specific percentage of online polls that should be included in political surveys? Should quotas, reflecting the anticipated turnout model for the electorate, be set for the combined samples of telephone surveys and online surveys? Should other polling techniques be adapted to reflect today’s changing media behaviors? These and other questions will be crucial as public opinion research on politics moves to the future.