All posts by admin

Minutes of the Annual General Meeting 2019

The Meeting was held at ORB on 5 March 2019

Present

  • Sir John Curtice, President
  • Nick Moon, Secretary/Treasurer
  • Simon Atkinson, Management Committee
  • Erica Harrison, ORB
  • Ivor Knox, Panelbase
  • Anthony Wells, YouGov
  • Gideon Skinner, Ipsos MORI
  • Adam Drummond, Opinium
  • Luke Taylor, Kantar Public
  • Damian Lyons Lowe, Survation
  • Gregor Jackson, ICM
  • Owen Thomas, Populus
  • Seb Wilde, Public First

1. Apologies for absence

There were no apologies for absence.

2. Minutes of the last AGM

The minutes were approved.

No matters arising not already on the agenda.

3. Officers’ reports

Financial report 

During 2018 we received £4,650 in subscription fees, covering both 2017 and 2018 subs as 2017 invoices were sent out with 2018 ones. There is still £500 in subscription fees due for 2018: both years from Panelbase and one year from Ipsos MORI.

Expenses during the year were £1,180.09, the two main components being £748 to the RSS for a meeting room and £353.99 for the website hosting and running.

The bank balance at the end of 2018 was £12,000.74.

With the current membership annual subscription income is £2,950.

Disclosure issues dealt with by the officers

Other than the usual small number of complaints about matters not within the BPC’s remit, there were no complaints about disclosure that required action from the Officers.

Membership applications dealt with by the officers

A record four new members were approved during the year: qriously, Hanbury Strategy, Deltapoll and Public First.

4. President’s report

Report had been distributed to members. There were no comments other than on matters covered elsewhere in the agenda.

5. Appointment of the officers

  • President
    NM nominated JC, SA seconded, elected nem con.
  • Secretary
    JC nominated NM, GJ seconded, elected nem con.
  • Management Committee member
    JC nominated SA, JT seconded, elected nem con.

6. House of Lords Committee

The President’s Report covered this in some detail and JC explained the various points in his report dealing with the actions suggested by the Committee and his proposals for dealing with them.

  1. Meeting agreed that we would have a section on the website on how to make a complaint, and also making clear what we can and can’t adjudicate on.
  2. Meeting agreed that we would write a spec for the things to be covered in a revised Journalists’ Guide. Once this is agreed by the membership we will either commission a professional writer to write it, or write it using internal resources. JT raised the point that it might have more credibility if it were written by a journalist. GS suggested Full Facts. The Statistics Ambassador for the RSS has volunteered to help, and he will be sent the outline for comment at an early stage. It was agreed that in future the BPC will have a link to each member’s methodology statement on their own website.
  3. The recommendation for a coordinated programme of training for journalists on polling could involve the BPC in considerable expense. One approach might be to approach University Schools of Journalism to offer lectures on polling. JT is already doing this in many places and is happy to do more. This only impacts on the journalists of tomorrow. GS suggested approaching The Lobby for training in opinion polls, though it was accepted that take-up may be low.
  4. The recommendation about polls declaring far more information about the amount and details of funding would raise very serious issues of client and member confidentiality and it was agreed that it would be inappropriate for the BPC to impose such a condition on members.
  5. We do already.
  6. It was accepted that members would not agree in any detail exactly what would constitute good or bad progress. JT suggested a very broad brush top-level guide to principles, and it was agreed that this might sit within the Journalists’ Guide.
  7. LT made the point that this would involve the BPC making pronouncements on things outside our remit, and might raise issues of why we do this but don’t police such things. It was agreed that this lay outside the remit of the BPC.
  8. See 7. Though the Journalists’ Guide, once revised, would perform a useful function here and we would offer to assist Ofcom etc if they wanted help in interpreting it further, while being mindful of the need to avoid going beyond our remit.
  9. See 7
  10. See 7

7. Treasury Select Committee

JC said Morgan Committee was closed as far as we were concerned at this stage. GJ raised the question of whether the fact that our response may have been seen as unsatisfactory by committee means there might be some come-back. JC felt this would depend on what came out of the discussions with the FCA.

8. Financial Conduct Authority

JC reported that whatever FCA decide it will be looking forward not looking back. They do not feel members have done anything wrong.

9. Proposed change to Rule 2.7

NM proposed that rule 2.7 should be changed from: 

However, in the event the results of a privately commissioned poll are made public by the organisation [its employees or agents] that commissioned the survey, such results will be deemed to have entered the public domain and procedures outlined above will be followed in respect of those findings.

to:

However, in the event the results of a privately commissioned poll are made public as a result of actions by the organisation [its employees or agents] that commissioned the survey, such results will be deemed to have entered the public domain and procedures outlined above will be followed in respect of those findings.

The amendment was agreed subject to organisations being able to object within 28 days of receiving the AGM minutes.

10. Any Other Business

JC urged members who have not done so to read the proposed revisions to the MRS Code of Conduct and if they feel there are any issues that might require a response from the BPC as opposed to individual company responses.

It was agreed that we probably can’t afford a complete redesign of our website, but that SA would obtain costs for updating it and making it more user-friendly.

JC raised the issue of making it possible to attend future AGMs by video link or at least phone.

JC also proposed a vote of thanks to ORB for hosting.

Polling Rules and UK Financial Markets

On 28th September, Rt Hon Nicky Morgan MP, Chair of the Treasury Committee, wrote to Professor Sir John Curtice, President of the British Polling Council. The letter focuses on the use of private polling data, particularly during the recent referendum campaigns, and follows on from media coverage during the summer. Further details, including a link to Mrs Morgan’s letter, can be found on the Treasury Committee’s web page: Change polling rules to protect integrity of UK financial markets.

Following a consultation with its membership, the British Polling Council has responded to Mrs Morgan. The contents are set out in this letter to the Rt Hon Nicky Morgan MP (pdf).

British Polling Council Introduces New Rule on Uncertainty Attached to Polls

The British Polling Council announces today that it has introduced a new requirement on its members when reporting estimates of vote intention. This requirement is an obligation to publish a statement of the level of uncertainty that has historically been associated with polls of voting intention.

The statement reads as follows:

‘All polls are subject to a wide range of potential sources of error. On the basis of the historical record of the polls at recent general elections, there is a 9 in 10 chance that the true value of a party’s support lies within 4 points of the estimates provided by this poll, and a 2 in 3 chance that they lie within 2 points.’

The statement has been developed following an analysis of the performance of the final polls at every general election since 2001. That analysis revealed that 90% of the estimates of a party’s share of the vote in the final polls were within four points of the eventual true result, while two-thirds were within two points.

Hitherto, it has often been noted that, according to statistical theory, there is a 95% chance that in a poll of a thousand people the true value of a party’s share of the vote lies within three points of the poll’s estimate. However, this statement refers solely to the error that may be occasioned by the chance variation to which all sample surveys are subject. It does not take into account any of the other many possible sources of uncertainty in a poll. The new BPC statement attempts to reflect all of the sources of uncertainty that practical experience suggests is associated with polling in Britain.

The statement is intended to discourage the media and others from reporting small changes in a party’s support – or variation between pollsters – as firm evidence of a significant change in its popularity.

The new BPC rule has been introduced in response to recommendation 11 of the Inquiry into the 2015 British general election polls that was chaired by Prof. Patrick Sturgis. This recommendation read that:

‘BPC members should provide confidence (or credible) intervals for each separately listed party in their headline share of the vote’.

The introduction of this requirement means the BPC has now responded to all of the recommendations that were made to it by Prof. Sturgis and his colleagues.

Notes to Editors

  1. The British Polling Council (BPC) is an association of polling organisations that publish polls. The objectives of the Council are to ensure standards of disclosure that provide consumers of survey results that enter the public domain with an adequate basis for judging the reliability and validity of the results. Through full disclosure the Council aims to encourage the highest professional standards in public opinion polling and to advance the understanding, among politicians, the media and general public, of how polls are conducted and how to interpret poll results.
  2. For further details of the Council see http://www.britishpollingcouncil.org/
  3. The report of the Inquiry into the Performance of the Polls in the 2015 Election is available at the National Centre for Research Methods (NCRM) pdf

Minutes of the Annual General Meeting 2018

The Meeting was held at ORB on 20th February 2018

Present

Sir John Curtice, President
Nick Moon, Secretary/Treasurer
Simon Atkinson, Management Committee
Johnny Heald, ORB
Ivor Knox, Panelbase
Anthony Wells, YouGov
Gideon Skinner, Ipsos MORI
Laurence Stellings, Populus
Jansev Jemal, ComRes
Adam Drummond, Opinium
Luke Taylor, Kantar Public
Damian Lyons Lowe, Survation

1. Apologies for absence

BMG, Lucid Talk, and ICM

2. Minutes of the last AGM

The minutes were approved

3.Officers’ reports

  • Financial report

    We currently have £9,985 in the bank, and £3,750 of outstanding membership fees (from 2017 and 2018), making a due total of £13,735. There are expenses expected, but probably no more than £500.

    NM raised the possibility of another fees holiday but it was agreed that the BPC needed to build a reserves fund in case the need for another enquiry, and fees would be levied as usual in 2019. With the existing members NM reported that annual subscription income is £2,750.

    It was agreed NM would send a note of thanks to MRS thanking them for their financial support for the Inquiry.

  • Disclosure issues dealt with by the officers

    Nothing that fell within our remit. ComRes audience recruitment raised a new serious issue about whether this fell within the Rules. The Officers decided it didn’t, and a clarification was published on the BPC website making this clear

  • Membership applications dealt with by the officers

    DCYFOR have become members

4. Appointment of the officers

  • President
    NM nominated JC, SA seconded, elected nem con
  • Secretary
    JH nominated NM, DLL seconded, elected nem con
  • Management Committee member
    AW nominated SA, LS seconded, elected nem con

5. President’s report

The report raised the issue of whether we need to think about rules to cover Multilevel Regression and Post-stratification (MRP). It was agreed we keep a watching eye on this for now. All other main points from the report are covered in later agenda items.

6. House of Lords Committee

JC and SA attended on behalf of BPC, NM and various BPC members also gave evidence.

BPC might have to make some kind of holding statement quickly after the final statement is released. We have already said we’re opposed to both statutory regulation and banning polls. We have slightly moved into quality domain by having enquiries into elections and agreeing certain procedures as a result, so may be under pressure to venture further.

JC encouraged members who were not CEOs of their organisation to ensure that their CEO was fully aware of the potential difficulties following publication.

Cttee due to report by end of March.

Agreed that JC should be authorised to give a strong defence of the value of self-regulation – citing all the openness and attempts to improve that happened after the election. We are much more transparent than most other countries.

It was agreed that the BPC would hold at least one mid-Parliament meeting, ideally within the Palace of Westminster, as well as the pre and post-election sessions to explain latest thinking.

7. Reporting uncertainty in the polls

Gideon introduced paper produced by Luke Taylor and him. Aim was to try to find a simple approach of explaining likely levels of uncertainty in polls. They decided best way, to avoid having to explain statistical theory to lay people, was to use history of reliability of polls in previous elections. Looked at all final polls done by BPC members (including 2001 before BPC existed).

Suggested level of uncertainty of +/-4pp, (achieved by >90% of polls), but also mention that most polls are within +/-2pp.

The Sturgis bootstrapping approach, achieved by taking repeated random sub-samples from polls, was felt to be harder to understand. Also bootstrapping ignores bias rather than variance by not comparing with actual result
Members would be free to calculate and publish their own.

Agreed to principle of what the paper says.

We need a PR statement to accompany the change in rules.

8. Proposed rule change following item 7

Insert at the end of the bullet points at Rule 2.4.

In the case of a poll of voting intentions for an election, include in the published documentation the following statement: ‘All polls are subject to a wide range of potential sources of error. On the basis of the historical record of the polls at recent general elections, there is a 9 in 10 chance that the true value of a party’s support lies within 4 points of the estimates provided by this poll, and a 2 in 3 chance that they lie within 2 points.’

This amendment to take effect from 1 April 2018.

GS asked all members for comments, and to check their own companies’ figures.

JC proposed NM seconded nem con.

9. Any Other Business

SA suggested we look again at FAQs and journalists’ guide to opinion polls on the website in the light of the Lords Cttee

JC suggested we investigate cost of updating the design of the website.

British Polling Council Welcomes Lords Committee Report on Polling

The British Polling Council welcomes the publication today (Tuesday) of the report of the House of Lords Select Committee on Political Polling and Digital Media. The Council is grateful to the Committee for its careful consideration of the role that opinion polls play in our democracy and of the challenges that currently face the conduct and reporting of polls.

The Council particularly welcomes the Committee’s conclusion that is ‘not convinced of the case for introducing a ban on the undertaking and publication of voting intention polls in the run-up to elections’ together with the Committee’s rejection of the statutory regulation of polls. In so doing, however, the Council acknowledges that this means the polling industry itself has a responsibility to promote best practice in the conduct and reporting of polls, and it is grateful to the committee for the various specific recommendations that it makes for the future work of the Council.

Most immediately, the Council is happy to accept that it should both revise its guidance to journalists on the reporting of polls and should work with other relevant organisations to develop a suitable programme of training for journalists on this subject. It is also happy to affirm its intention to continue its current practice of undertaking a post-mortem on the conduct of the polls after each election or major referendum and reporting its findings; following a public seminar held in February a report on the 2017 election is currently in preparation.

More broadly, the Committee’s recommendations envisage that the Council should adopt a wider remit and fulfill a larger role. At present, the Council is run on a purely voluntary basis with limited resources. The Council will now consider how best to resource its activities in light of the Committees report.

Professor Sir John Curtice, President of the British Polling Council, said, ‘Today’s report is a welcome contribution to a considered, informed discussion of the conduct and reporting of opinion polls in Britain. The Council welcomes the fact that the Committee recognised the weight of evidence put before it that a ban on the publication of polls would neither be desirable nor effective. At the same time, the Council accepts that the polling industry has a duty to promote high standards in the conduct and reporting of polls and will now consider how it, in collaboration with other bodies, can enhance the considerable efforts that it already makes to achieve that objective.’

For further details or to interview Prof. Curtice contact Simon Atkinson (07791 680770; simon.atkinson@ipsos.com) or Aalia Khan (07824 597435; aalia.khan@ipsos.com).

NOTES TO EDITORS

  1. The British Polling Council (BPC) is an association of polling organisations that publish polls. The objectives of the Council are to ensure standards of disclosure that provide consumers of survey results that enter the public domain with an adequate basis for judging the reliability and validity of the results. Through full disclosure the Council aims to encourage the highest professional standards in public opinion polling and to advance the understanding, among politicians, the media and general public, of how polls are conducted and how to interpret poll results.
  2. For further details of the Council see www.britishpollingcouncil.org
  3. Details of the House of Lords Select Committee on Political Polling and Digital Media are to be found at Select Committee on Political Polling and Digital Media.

The Polls in the 2017 Election: A BPC Conference

The British Polling Council is pleased to host a special conference on 20th February 2018 to review and debate the performance of the opinion polls at the 2017 General Election.

The conference will be held at the Royal Statistical Society. Tickets are free of charge but registration is essential via Eventbrite.

Programme

Chair: Sharon Witherspoon, Academy of Social Sciences

2-2.30 Introduction & The Challenges Facing The Polls in 2017: John Curtice (BPC)

2.30-3.15 How We Performed 1: Presentations by ComRes (Jansev Jamal), Survation (Damian Lyons-Lowe) and YouGov (Anthony Wells)

3.15 Break

3.30 A New Approach? The Use of Mixed Level Regression and Post-Stratification in the 2017 Election: Ben Lauderdale (LSE)

4.00 How We Performed 2: Presentations by Kantar (Luke Taylor) & Ipsos MORI (Gideon Skinner)

4.30 An Independent Reaction and Q&A: Steve Fisher (Oxford)

Britsh Polling Council Announces Findings in Respect of Complaints about Audience Recruitment

The British Polling Council (BPC) has received two complaints, one from the Conservative Party and one from an individual member of that party, concerning the recruitment of the audience for the BBC programme, BBC Election Debate Live with Mishal Hussein, which was broadcast on 31 May 2017. The audience for this programme was recruited by ComRes for the BBC with a view to ensuring that it was ‘representative’.

Both complainants had sought further information from ComRes about how they had recruited the audience. In doing so the Conservative Party in particular cited section 2 of the BPC’s rules on disclosure. In pursuance of BPC’s aim of ensuring transparency in the conduct of opinion polls, these rules specify details that BPC members should routinely make available in respect of published polls. The complainants argued that these rules applied to an audience recruitment exercise such as that conducted by ComRes, and that thus ComRes were duty bound to supply the information they sought.

It has never previously been suggested that audience recruitment exercises do fall within BPC’s remit and that thus its rules on disclosure should apply to them. However, in view of the submissions it has received, the officers of the BPC have carefully considered whether audience recruitment exercises do fall within the Council’s remit as specified in its current Objects and Rules. They have determined that they do not. This is because such exercises do not generate published quantitative data (as defined in those Objects and Rules) that summarise the collective views of a representative sample of voters (or other sub-group), and it is to such data that the Council’s Rules on Disclosure are intended to apply.

We are grateful to the two complaints for bringing this issue to the Council’s attention and we hope that this statement helps clarify matters for those considering any complaint to the Council in future about a member’s alleged failure to follow its rules on transparency.

House of Lords Committee on Political Polling and Digital Media: BPC Statement

The British Polling Council welcomes the interest of the House of Lords in the work on the polling industry through the creation of an ad hoc committee on political polling and digital media.

The Council has been working continuously in recent years both to promote transparency in the way that polls are conducted and to foster enhancement of the methods used. Most recently it has required its members to undertake and publish an internal inquiry into the methods used in 2017 election, at which many but not all polls overestimated the Conservative lead over Labour, and is committed to holding in the autumn a seminar at which these reports will be presented and discussed. The Council will be seeking to make the fruits of this and its earlier work, including the inquiry conducted by Prof. Patrick Sturgis into the performance of the polls in the 2015 election, available to the committee.

However, any inquiry into the role of polls in politics clearly needs to consider wider questions than the methodology of the polls, including the relationship between polling companies and the (typically media) organisations that commission polls, how the media report and use polls, and the role of the private polling that may be commissioned by political parties. We trust that the committee will also pay due attention to these wider questions about the role that opinion polls play in our politics today.

General Election: 8 June 2017

After the problems of 2015, and the need to deal with the issues raised by the subsequent Sturgis enquiry, the pollsters were inevitably going to be under considerable scrutiny in the 2017 election.

The polls proved very good at predicting the Conservative vote, with an average of the final polls being only 0.2% higher than the actual result. (The actual result shares are based on all seats except Kensington).

They did, however, prove much less good at predicting the Labour vote, with the pollsters’ average being 5.2% below Labour’s actual share. This is only the second election since 1987 when the pollsters have underestimated the Labour share.

The average poll figure for the Liberal Democrats was very close, being only 0.3% different from the result.

While the final polls were not ideal, the BPC does not feel there is a need for another formal inquiry; the detailed findings of the Sturgis review are available here and provide a reference point for understanding the issues and challenges. Instead all BPC members who produced final polls will produce a “lessons learned” report, and these will be presented at a conference before the end of this year.

  CON LAB LD UKIP Green Other Method Sample Size Fieldwork
Opinium 43 36 8 5 2 6 Online 3002 June 4
Survation 41 40 8 2 2 6 Telephone 2798 June 6-7
Ipsos MORI 44 36 7 4 2 6 Telephone 1291 June 6-7
ICM 46 34 7 5 2 6 Online 1532 June 6-7
ComRes 44 34 9 5 2 5 Online 2051 June 5-7
YouGov 42 35 10 5 2 6 Online 2130 June 5-7
Panelbase 44 36 7 5 2 6 Online 3018 June 2-7
Kantar Public 43 38 7 4 2 6 Online 2159 June 1-7
BMG 46 33 8 5 3 6 Telephone / online 1199 June 6-7
Average 43.7 35.8 7.9 4.4 2.1 5.9      
Result 43.5 41 7.6 1.9 1.6 4.4      
Difference 0.2 -5.2 0.3 2.5 0.5 1.5