Tag Archives: 2019

The Performance of the Polls in the 2019 General Election

After being widely criticised for their performance in the 2015 and 2017 general elections, the polls have proven more accurate in the 2019 election than in any contest since 2005.

As the table shows, on average the final polls underestimated the Conservative vote by just 1.4 points and overestimated Labour’s by only 0.5 points.

These patterns are consistent with the long-term trend for the polls to over-state Labour and under-state Tory support, but the size of the errors were so small that this did not prevent the polls correctly predicting the overall outcome of the election.

For all the other parties, the average error across the pollsters was under one point. Indeed for the LibDems, Greens, and Others the average error was a remarkably small 0.1 of a point.

 CONLABLDBrexitGREENOTHERMETHODSAMPLE SIZEFIELDWORK
Qriously433012348Online2222Dec 5-8
ICM423612324Online2011Dec 6-9
YouGov433412335Online105,612Dec 4-10
Savanta ComRes413612326Online1732Dec 9-10
BMG413214345Online1660Dec 6-11
Ipsos MORI443312236Telephone2213Dec 9-11
Kantar 443213335Online2815Dec 9-11
Deltapoll453510334Online1818Dec 9-11
Survation45349336Telephone2395Dec 10-11
Panelbase433411435Online3174Dec 10-11
Opinium453312226Online3005Dec 10-11
Average43.333.511.72.92.95.5   
Result44.733.011.82.12.85.6   
Difference-1.40.5-0.10.80.1-0.1   

There was also a Survation poll in Scotland conducted online from Dec 10-11 with 1,004 respondents.

pollResult
SNP4345.0
Con2825.1
Lab2018.6
Lib Dem79.5
Other21.7

Principal Changes in the Conduct and Reporting of Polls in the 2019 General Election

This document briefly summarises the main differences between the way in which member companies are conducting their polls in the latter stages of the 2019 election and the way they did so in 2017. Details of the changes that companies made in 2017 are available at How Have The Polls Changed Since 2015?. For further details and analysis of the methodology of the polls see BPC Inquiry Report.

John Curtice

BMG

The company only published one Britain-wide poll in 2017; otherwise it conducted private polls for the Labour Party.

That one published poll was a mixed-mode online and telephone poll. The company’s polls in this campaign are likely to be conducted exclusively using an online methodology.

As in 2017 data are weighted in accordance with the known levels of electoral registration among key demographic groups. However, in contrast to 2017 the data are no longer weighted by reported probability of voting or whether or not the respondent voted at the last election.

Respondents are being invited to state a vote intention from only among the parties standing in their constituency.

Deltapoll

This is the first general election to be covered by the company. Details of its methodology can be found at Deltapoll Methodology.

ICM

Data are no longer weighted according to a pre-determined level of turnout by demographic group

A 0-10 turnout scale is now used to weight the data rather than a 1-10 scale.

The voting preference of those who say they do not know how they would vote or who refuse to say how they would do so are no longer imputed on the basis of their past vote.

Data are weighted by 2016 EU referendum vote as well as 2017 general election vote.

Respondents are being invited to state a vote intention from only among the parties standing in their constituency.

Ipsos MORI

Data are no longer being weighted to match the ratios of turnout by age and tenure as recorded by the 2015 British Election Study.

Child in household is no longer used as a weight as it was in 2017, while ethnicity has been added as a weight.

Respondents who say they will vote for the Brexit Party, Liberal Democrats, Greens or Plaid Cymru are invited to state for which party they would vote if their preferred party is not standing in their constituency – and are allocated to that party if their preferred party is not standing locally.

Kantar

Weighting of the data by probability of voting is based on the respondent’s stated likelihood of voting, their age, and whether they voted in 2017, modelled using data collected by the company after the 2017 election (rather than 2015).

Weighting of the data by past vote takes into account of estimates of differential recall error by party and, in particular, a higher tendency for those who vote Labour in 2017 to fail to report that they have done so.

In similar vein to what was done in respect of those saying they were voting UKIP or Green in 2017, respondents are invited to state for which party they would vote if their preferred party is not standing in their constituency – and are allocated to that party if their preferred party is not standing locally.

Opinium

Respondents are being invited to state a vote intention from only among the parties standing in their constituency.

ORB

The company has so far only conducted one poll of vote intentions, at the very beginning of the campaign.

Panelbase

Respondents are invited to state for which party they would vote if their preferred party is not standing in their constituency – and are allocated to that party if their preferred party is not standing locally.

Savanta ComRes

Data are no longer being weighted by the voter turnout model that was used in 2017

Future polls will invite respondents to choose from among the parties standing in their constituency.

Survation

Polls are now only being conducted by phone, using the same approach as in 2017.

Data are weighted, inter alia, by 2019 European Parliament election vote.

As in 2017, respondents are being invited to state a vote intention from only among the named candidates standing in their constituency.

YouGov

There has been a slight reduction in the downweight that is applied to those who did not vote at the last election.

Respondents are being invited to state a vote intention from only among the candidates standing in their constituency.

Interpreting Polls and Election Data

IMPRESS, the UK’s independent press regulator, and the Market Research Society have published a consultation on guidance for journalists reporting on opinion polls.

Interpreting polls and election data – guidance for media and journalists

Comments and feedback on this draft guidance are invited until 20 November.

The MRS web page also provides access to a range of other resources to support journalists and the media in their reporting of opinion poll data.

The Performance of the Polls in the 2019 European Election

All polling of elections is subject to many potential sources of error. The historical record at general elections indicates that there is a one in ten chance that an individual poll will over or underestimate a party’s support by more than four points.

These European Elections presented particular challenges. Not only did pollsters have to face the challenge of an election in which turnout was likely to be low (and in the event stood at 36.7%), but also of estimating the level of support for two new parties, one of which went on to come top of the poll.

The table below compares the estimates of BPC members’ polls whose fieldwork ended no earlier than three days before polling with that of the actual result — though in two instances fieldwork commenced more than a week before polling day.

It reveals that on average the polls overestimated support for Labour and underestimated that for the Liberal Democrats and the Greens. However, two polls produced estimates for Labour and the Liberal Democrats that were within a point of the eventual outcome, while one was spot on in its estimate for the Greens.

There was also a less marked tendency to overestimate support for the Conservatives and the Brexit Party, and in both these instances one poll was spot on in its (integer) estimate of party support.

The detailed council by council election results showed that turnout increased more in Remain-voting areas than in Leave-voting ones, suggesting that those who voted Remain were more likely to have voted than did those who voted Leave. Meanwhile, polling conducted by Opinium on polling day suggests that those who switched from voting Labour in 2017 to either the Liberal Democrats or the Greens were more likely than most voters to have made up their minds about how to vote in the last week of the campaign. These patterns may help to explain at least some of the average error in the polls on this occasion.

 ConLabLDBrexitGreenUKIPChange UKOtherMETHODSAMPLE SIZEFIELDWORK
Opinium121715387234Online2005May 17-20
YouGov713193712346Online3864May 19-21
Panelbase122515307335Online2033May 14-21
Kantar132415278454Online2316May 14-21
Ipsos MORI915203510336Telephone1527May 20-22
BMG Research121817358245Online1601May 20-22
Survation142412327436Online2029May 22
Average11.319.416.133.48.43.03.65.1 
Result9.114.120.331.612.13.33.46.1 
Difference2.25.3-4.21.8-3.7-0.30.2-1.0 

Note: Table is based on the outcome in Great Britain. The polls conducted by Opinium and Survation were conducted across the UK as a whole, but the figures for those polls quoted here are for respondents in Great Britain only.

Minutes of the Annual General Meeting 2019

The Meeting was held at ORB on 5 March 2019

Present

  • Sir John Curtice, President
  • Nick Moon, Secretary/Treasurer
  • Simon Atkinson, Management Committee
  • Erica Harrison, ORB
  • Ivor Knox, Panelbase
  • Anthony Wells, YouGov
  • Gideon Skinner, Ipsos MORI
  • Adam Drummond, Opinium
  • Luke Taylor, Kantar Public
  • Damian Lyons Lowe, Survation
  • Gregor Jackson, ICM
  • Owen Thomas, Populus
  • Seb Wilde, Public First

1. Apologies for absence

There were no apologies for absence.

2. Minutes of the last AGM

The minutes were approved.

No matters arising not already on the agenda.

3. Officers’ reports

Financial report 

During 2018 we received £4,650 in subscription fees, covering both 2017 and 2018 subs as 2017 invoices were sent out with 2018 ones. There is still £500 in subscription fees due for 2018: both years from Panelbase and one year from Ipsos MORI.

Expenses during the year were £1,180.09, the two main components being £748 to the RSS for a meeting room and £353.99 for the website hosting and running.

The bank balance at the end of 2018 was £12,000.74.

With the current membership annual subscription income is £2,950.

Disclosure issues dealt with by the officers

Other than the usual small number of complaints about matters not within the BPC’s remit, there were no complaints about disclosure that required action from the Officers.

Membership applications dealt with by the officers

A record four new members were approved during the year: qriously, Hanbury Strategy, Deltapoll and Public First.

4. President’s report

Report had been distributed to members. There were no comments other than on matters covered elsewhere in the agenda.

5. Appointment of the officers

  • President
    NM nominated JC, SA seconded, elected nem con.
  • Secretary
    JC nominated NM, GJ seconded, elected nem con.
  • Management Committee member
    JC nominated SA, JT seconded, elected nem con.

6. House of Lords Committee

The President’s Report covered this in some detail and JC explained the various points in his report dealing with the actions suggested by the Committee and his proposals for dealing with them.

  1. Meeting agreed that we would have a section on the website on how to make a complaint, and also making clear what we can and can’t adjudicate on.
  2. Meeting agreed that we would write a spec for the things to be covered in a revised Journalists’ Guide. Once this is agreed by the membership we will either commission a professional writer to write it, or write it using internal resources. JT raised the point that it might have more credibility if it were written by a journalist. GS suggested Full Facts. The Statistics Ambassador for the RSS has volunteered to help, and he will be sent the outline for comment at an early stage. It was agreed that in future the BPC will have a link to each member’s methodology statement on their own website.
  3. The recommendation for a coordinated programme of training for journalists on polling could involve the BPC in considerable expense. One approach might be to approach University Schools of Journalism to offer lectures on polling. JT is already doing this in many places and is happy to do more. This only impacts on the journalists of tomorrow. GS suggested approaching The Lobby for training in opinion polls, though it was accepted that take-up may be low.
  4. The recommendation about polls declaring far more information about the amount and details of funding would raise very serious issues of client and member confidentiality and it was agreed that it would be inappropriate for the BPC to impose such a condition on members.
  5. We do already.
  6. It was accepted that members would not agree in any detail exactly what would constitute good or bad progress. JT suggested a very broad brush top-level guide to principles, and it was agreed that this might sit within the Journalists’ Guide.
  7. LT made the point that this would involve the BPC making pronouncements on things outside our remit, and might raise issues of why we do this but don’t police such things. It was agreed that this lay outside the remit of the BPC.
  8. See 7. Though the Journalists’ Guide, once revised, would perform a useful function here and we would offer to assist Ofcom etc if they wanted help in interpreting it further, while being mindful of the need to avoid going beyond our remit.
  9. See 7
  10. See 7

7. Treasury Select Committee

JC said Morgan Committee was closed as far as we were concerned at this stage. GJ raised the question of whether the fact that our response may have been seen as unsatisfactory by committee means there might be some come-back. JC felt this would depend on what came out of the discussions with the FCA.

8. Financial Conduct Authority

JC reported that whatever FCA decide it will be looking forward not looking back. They do not feel members have done anything wrong.

9. Proposed change to Rule 2.7

NM proposed that rule 2.7 should be changed from: 

However, in the event the results of a privately commissioned poll are made public by the organisation [its employees or agents] that commissioned the survey, such results will be deemed to have entered the public domain and procedures outlined above will be followed in respect of those findings.

to:

However, in the event the results of a privately commissioned poll are made public as a result of actions by the organisation [its employees or agents] that commissioned the survey, such results will be deemed to have entered the public domain and procedures outlined above will be followed in respect of those findings.

The amendment was agreed subject to organisations being able to object within 28 days of receiving the AGM minutes.

10. Any Other Business

JC urged members who have not done so to read the proposed revisions to the MRS Code of Conduct and if they feel there are any issues that might require a response from the BPC as opposed to individual company responses.

It was agreed that we probably can’t afford a complete redesign of our website, but that SA would obtain costs for updating it and making it more user-friendly.

JC raised the issue of making it possible to attend future AGMs by video link or at least phone.

JC also proposed a vote of thanks to ORB for hosting.