Tag Archives: Regulation

British Polling Council Introduces New Rule on Uncertainty Attached to Polls

Press Releases,

The British Polling Council announces today that it has introduced a new requirement on its members when reporting estimates of vote intention. This requirement is an obligation to publish a statement of the level of uncertainty that has historically been associated with polls of voting intention.

The statement reads as follows:

‘All polls are subject to a wide range of potential sources of error. On the basis of the historical record of the polls at recent general elections, there is a 9 in 10 chance that the true value of a party’s support lies within 4 points of the estimates provided by this poll, and a 2 in 3 chance that they lie within 2 points.’

The statement has been developed following an analysis of the performance of the final polls at every general election since 2001. That analysis revealed that 90% of the estimates of a party’s share of the vote in the final polls were within four points of the eventual true result, while two-thirds were within two points.

Hitherto, it has often been noted that, according to statistical theory, there is a 95% chance that in a poll of a thousand people the true value of a party’s share of the vote lies within three points of the poll’s estimate. However, this statement refers solely to the error that may be occasioned by the chance variation to which all sample surveys are subject. It does not take into account any of the other many possible sources of uncertainty in a poll. The new BPC statement attempts to reflect all of the sources of uncertainty that practical experience suggests is associated with polling in Britain.

The statement is intended to discourage the media and others from reporting small changes in a party’s support – or variation between pollsters – as firm evidence of a significant change in its popularity.

The new BPC rule has been introduced in response to recommendation 11 of the Inquiry into the 2015 British general election polls that was chaired by Prof. Patrick Sturgis. This recommendation read that:

‘BPC members should provide confidence (or credible) intervals for each separately listed party in their headline share of the vote’.

The introduction of this requirement means the BPC has now responded to all of the recommendations that were made to it by Prof. Sturgis and his colleagues.

Notes to Editors

  1. The British Polling Council (BPC) is an association of polling organisations that publish polls. The objectives of the Council are to ensure standards of disclosure that provide consumers of survey results that enter the public domain with an adequate basis for judging the reliability and validity of the results. Through full disclosure the Council aims to encourage the highest professional standards in public opinion polling and to advance the understanding, among politicians, the media and general public, of how polls are conducted and how to interpret poll results.
  2. For further details of the Council see http://www.britishpollingcouncil.org/
  3. The report of the Inquiry into the Performance of the Polls in the 2015 Election is available at the National Centre for Research Methods (NCRM) pdf

British Polling Council opposes Bill on Regulating Opinion Polls

Press Releases, ,

The British Polling Council (BPC) urges the House of Lords to reject the private members bill on the regulation of opinion polls that is being presented today by Lord Foulkes.

The Bill proposes that an authority be established that would regulate polls of voting intentions for all elections and referendums in the United Kingdom. The authority would be empowered to specify approved ways for selecting who should be interviewed, how the questions in polls should be worded and to ban the publication of voting intention polls during an election campaign.

Who is interviewed by a poll and how the questions it asks are phrased are important issues. How any poll has addressed them should always be clearly stated, as the rules of the BPC require. But they are not issues that are susceptible to straightforwardly ‘right’ or ‘wrong’ answers. Professional researchers can and do disagree about how polls should be conducted and how they should be worded. They regularly experiment and test alternative and new ways of doing polls and asking questions in order to improve their methods. Imposing regulatory standards would put at risk the experimentation and competition that are essential to improving the ways in which polls are conducted.

Banning the publication of polls during an election campaign would not mean that polls were not conducted. It would simply mean that access to their results would be confined to those who could afford to pay for polls, such as the banks and the political parties, or who knew where to find the results on an overseas website. Only the ordinary voter, who is meant to be central to the democratic process, would be left out of the loop. It could potentially open the way for politicians to claim that their private polling showed them ahead, regardless of what their polling actually showed, or indeed whether it existed at all.

Doubtless many people feel that in underestimating the Conservative vote and overestimating Labour’s, the polls provided unhelpful misinformation during the recent general election campaign. That is why the BPC has established an independent inquiry into why the polls were wrong and how their conduct might be improved in future. Indeed, the first meeting of that inquiry, at which BPC members will be presenting – in public – their initial findings as to what went wrong, is being held today.

Professor John Curtice, President of the British Polling Council, said, ‘What is needed now is a critical and open appraisal of where the polls went wrong, not the heavy hand of regulation that in attempting to impose common standards would make it more likely that the polls all get it wrong again in future. As any economic forecaster knows too well, forecasting how people will behave is always a difficult enterprise. No-one has yet suggested that, despite their many errors, economic forecasting should be regulated, and it is not clear why attempting to anticipate how people will vote should be treated any differently.’

Notes for Editors:

  • The British Polling Council (BPC) is an association of polling organisations that publish polls. The objectives of the Council are to ensure standards of disclosure that provide consumers of survey results that enter the public domain with an adequate basis for judging the reliability and validity of the results. Website: www.britishpollingcouncil.org. Twitter: @BritPollingCncl
  • The first meeting of the inquiry is at The Royal Statistical Society, 12 Erroll St, London EC1Y 8LX at 1.30 pm on 19 June. Anyone who wishes to attend should register at BPC/MRS Polling Inquiry meetings
  • Members of the inquiry are Dr. Nick Baker, Group CEO, Quadrangle Research Group Ltd; Dr. Mario Callegaro, Senior Survey Research Scientist, Google UK ; Dr. Stephen Fisher, Associate Professor of Political Sociology, University of Oxford, who runs the Electionsetc website; Dr. Jouni Kuha, Associate Professor of Statistics, London School of Economics and lead statistician for the BBC/ITV/Sky exit poll; Prof. Jane Green, Professor of Political Science, University of Manchester and Co-Director of the 2015 British Election Study; Prof. Will Jennings, Professor of Political Science and Public Policy, University of Southampton, and a member of the Polling Observatory team; Dr Ben Lauderdale, Associate Professor in Research Methodology, London School of Economics and one of the team behind the electionforecast.co.uk website; Dr. Patten Smith, Research Director, Research Methods Centre, Ipsos MORI and Chair of the Social Research Association.
  • Details of Lord Foulkes’ bill can be found at: Regulation of Political Opinion Polling Bill [HL] 2015-16

For further information, please contact a member of the BPC Management Committee:

Simon Atkinson: 07791 680770
Nick Moon: 07770 564664
John Curtice 07710 348755