Sport Participation Measurement - Background Paper on Taking

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Background Paper on Taking Part and Active People Survey

Integration

May 2012

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Contents

Introduction .......................................................................................................... 4

Background on Taking Part and Active People surveys ....................................... 5

National Statistics Context ................................................................................... 6

Nat Cen Review ................................................................................................... 7

Measurement Priorities for Government and the Public Sector ............................ 8

Scope and Framework of Changes ...................................................................... 9

Value for Money and Quality .............................................................................. 10

Coverage ........................................................................................................... 12

The Need for Local Authority Estimates ............................................................. 14

Use of Modelled Estimates ................................................................................ 15

References ........................................................................................................ 16

Summary of Taking Part and Active People Surveys ......................................... 17

Key Terms and Definitions ................................................................................. 19

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Introduction

This document provides further background information about the existing Taking

Part (TP) and Active People (APS) surveys and sets out the reasons for the

proposed changes in more detail.

In addition, to inform the decision-making process, DCMS and Sport England

(SE) commissioned the ONS Methodology Advisory Service to take an

independent look at the way forward we are proposing. The report “Discussion of

options for Taking Part and Active People Survey Integration” from the ONS

Methodology Advisory Service is a discussion of possible options for integrating

the TP and APS, written to inform this consultation and provides further guidance

and detail on the statistical issues involved. It is available to download on the

sport participation measurement consultation page of the DCMS and SE

websites.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Background on Taking Part and Active

People surveys

TP is a face-to-face survey covering a range of cross-cultural and sport questions

which measure public engagement and participation in the arts and sport in

England. TP is managed by the DCMS Evidence and Analysis Unit and the

survey is funded by DCMS, English Heritage, Arts Council England and SE.

TP is a National Statistic and has an annual sample size of around 10,000, which

allows for robust estimates at a national and regional level. For the 8

year of

collection, 2011/12, TP has introduced a longitudinal element to the survey; there

will be a fresh sample of 5,000 with 5,000 interviews followed up from the

previous year.

APS is a large telephone survey (sample size around 165,000) which currently

focuses on sports participation and physical activity only, within England. The

sample size is large enough to be able to robustly measure lower participation

sports at the national level as well as providing headline sports measures at the

Local Authority District level. The APS has, in the past, been boosted to provide

local authority measures of cultural participation (for National Indicators) but this

local cultural measurement is no longer a requirement.

The latest issue, year 6 of APS (APS6) contains a face-to-face survey element

(6,000 households) to understand the modal effects of the random digit dialling

telephone survey method employed. The face-to-face element also allows

additional in-depth questions to be asked around levers and barriers to

participation.

Both surveys interview adults aged 16 and over. TP only contains a child element

to the survey and for each household, together with one adult interview, the

survey includes, one child per household aged 11-15 and one child per

household aged 5-10 (interviewed by proxy via the adult).

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

National Statistics Context

Both surveys are governed by the Code of Practice for Official Statistics,

compliance with which falls within the responsibility and oversight of the DCMS

Head of Profession for Statistics. SE is an Arm’s Length Body (ALB) of DCMS

and named as a producer of official statistics.

DCMS and the TP survey have recently been assessed by the UK Statistics

Authority and the Taking Part survey has retained National Statistics status,

subject to implementing 16 requirements of the Authority. Although relationships

between DCMS and its ALBs including SE were discussed within the

assessment, the APS (an official statistic) was out of scope.

However, the Statistics Authority is aware of the collaborative project between

DCMS and SE to address the duplication of sport questions and to resolve the

issue of coherence between sports estimates provided by each of the two

surveys; these issues were raised by stakeholders during the assessment

process. Similarly, the National Statistician’s Office is also aware of this

consultation.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Nat Cen Review

In August 2010 NatCen published an independent review of the TP and APS

surveys; this review was commissioned by DCMS and SE as stated in the review:

“The aim of the review was to scope the options for collecting national and local

level data on participation in culture and sport, and to consider how the two

surveys might be merged or modified in order to reduce costs, while still meeting

the requirements of stakeholders.”

This review has provided a good starting point for options considered in this

consultation, however both surveys have moved on since this review; TP has

been redesigned to include a longitudinal element (partly as a result of the

review) and APS now includes a face-to-face element.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Measurement Priorities for Government

and the Public Sector

SE will invest over £1 billion of National Lottery and Exchequer funding between

2012 and 2017 in organisations and fund a wide range of NGBs. Survey

evidence is intended to form a part of SE’s “payment by results” system for

allocation of funding.

Local Authorities are key users of sports data spending approximately £1.9 billion

per year on sport and recreation.

The following have been identified as the policy measurement requirements

identified by DCMS and SE and support measurement of “Creating a sporting

habit for life - A new youth sport strategy”:

1. Measure the proportion aged 14 years and over undertaking at least one 30

minute session of moderate intensity sport each week

2. Measure participation of children taking part in competitive sport

3. Measurement to capture a required overall consistent annual increase in

participation across all age ranges but with a particular focus on 14-25 year old

age group at a national level

4. Measure and track change for the most popular 30 sports to inform National

Governing Bodies of sport (NGB) performance management (the top 30 sports

account for around 80% of total funding).

5. Measure and track change at a local authority district level.

Related to requirement 4 is the issue of National Governing Bodies of Sport

(NGB) accountability and payment by results. The revised survey will be central

to performance management of NGBs and this will be supplemented by other

success measures (including self-reported information), agreed with the NGBs,

as part of the Payment by Results regime.

Based on CLG data for 2010/11 outturn at

http://www.communities.gov.uk/publications/corporate/statistics/revenue201011fi

naloutturn (revenue) and

http://www.communities.gov.uk/documents/statistics/doc/20134611.doc (capital)

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Scope and Framework of Changes

As well as meeting the measurement requirements for DCMS and SE (detailed

on page 8) and continuing to meet the requirements of key users, proposed

survey changes are framed in terms of which options provide best value for

money and quality i.e. how they meet:

Principle 7 Resources, Practice 5 of the Code of Practice for Official Statistics

“Seek to balance quality (for example, accuracy and timeliness) against costs

(including both costs to government and data suppliers), taking into account the

expected uses of the statistics”.

Principle 1 (Meeting user needs) of the Code of Practice for Official Statistics

states that the “production, management and dissemination of official statistics

should meet the requirements of informed decision-making by government”.

While we are proposing changes DCMS and SE are:

1. Committed to retaining the cross-cutting nature of the TP survey, covering the

needs of the cultural sectors.

2. Committed to meeting the user requirements for sports measurement which

the APS provides. These relate to measurement of individual sports,

performance management of National Governing Bodies of sport (NGB) and

local measurement of sports participation and physical activity.

3. Committed to providing continuity in the data and with the substantial evidence

base the TP and APS provide.

4. Keen to pursue on-going improvements, for example the longitudinal element

to the TP survey and the introduction of a face-to-face element to the APS.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Value for Money and Quality

Alongside the costs of any options for combining both surveys there needs to be

consideration of value for money and quality. Face-to-face interviews (at around

£100 per interview) are around eight times more expensive than landline

telephone interviews, but provide good quality information, terms can be

explained clearly and with the use of visual aids ‘show-cards’, which help

respondents choose from long lists (for example lists of sports). The response

rate for Taking Part is around 65%.

Telephone surveys are lower cost (around £12-per interview) and can provide

much larger samples sizes but may have low response rates (around 28% for

APS) and for APS only cover households with fixed-lines. Around 15% of UK

households are mobile only.

Our aim is to use both face-to-face and fixed-line telephone survey results for

sports participation together (a ‘mixed-mode’ approach); this approach draws on

the strength and quality of face-to-face interviews, but allows for a large sample

of telephone interviews to be obtained within the costs available.

The APS is currently undertaking a face-to-face ‘parallel’ study which compares

the differences in responses to face-to-face interviews and telephone interviews

and is important in assessing a way forward for integrating TP and APS into a

single survey.

The outcome of this study will inform the detailed method of how face-to-face and

landline results can be combined. The study will show how well telephone

responses match face-to-face interviews and, depending on the extent of the

differences, how these can be accounted for in weighting or calibration of results.

The study runs over 4 quarters from October 2011 to October 2012, with the

results from the first quarter being available during June 2012.

We also intend to investigate digital data collection methods (mobile and online),

with a view to integrating results from these additional modes over the medium

term (subject to pilot results).

Introducing online data collection has the potential to provide efficiencies as this

method is considerably lower cost than landline telephone or face to face data

collection. Our intention would be to follow-up on respondents who have already

been interviewed as part of the face to face survey and have agreed that they

can be re-contacted.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Building up a panel of online respondents in this way is a gradual process but the

intention would be, if the data is of sufficient quality, to replace part of the landline

telephone sample with the online component, maintaining the overall total sample

size. This approach would need to be successfully piloted before inclusion.

Collection of data by mobile telephone can potentially increase the coverage of

fixed-line telephone surveys. However the quality of mobile telephone interviews

can be low. There are scenarios where mobile data collection may add to

coverage, but the method can be high cost and its value needs to be assessed

against other strategies. Piloting this approach would need to show good

evidence that the data collected is of sufficient quality and that there is a robust

method to include results alongside other types of data collection.

A trial of mobile phone data collection has been carried out for the APS, and the

results of this would inform the implementation of a mobile phone pilot.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Coverage

It is important for any household survey to fully represent the population by

providing as wide a coverage of the population as possible; it is particularly

important that a sports participation survey captures people who may spend

much of their time outside the home playing sport or exercising.

A face-to-face survey approach (used in the Taking Part survey) provides full

coverage of households; however institutional addresses (for example, armed

forces barracks, student halls of residence, hospitals, care homes) are currently

excluded for practical reasons. To reduce potential response bias against people

who spend less time at home a minimum of 8 visits to households are made at

varying times across the day and at the weekend.

A fixed line survey approach (taken in the APS) will not cover mobile only

households (currently 15% of UK households). Mobile households tend to be

more common amongst younger and lower socio-economic groups. While

younger people have higher rates of sports participation those in lower socio-

economic groups have lower participation; investigation by SE has shown that

the exclusion of mobile only households has only a small effect on survey results

and can be largely overcome by weighting. APS also follows a thorough non-

response strategy, with households phoned at least 40 times before being

dropped from the sample.

Household penetration of fixed and mobile telephony (UK): Ofcom

Communications Market Report: UK, August 2011.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

In this consultation we are also proposing piloting digital data collection methods

(mobile and online), with a view to integrating results from these additional

modes over the medium term (subject to pilot results). These collection methods

provide additional channels of communication, but also present challenges in

how results can be integrated within robust estimates. A more detailed

discussion of the advantages and disadvantages of these methods is provided in

the paper “Discussion of options for Taking Part and Active People Survey

Integration” from the ONS Methodology Advisory Service. This can be

downloaded from the consultation page of the DCMS and Sport England

websites.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

The Need for Local Authority Estimates

Estimates of sports participation at a Local Authority District level are important to

SE and local government partners in measuring participation at a local level.

Local authorities use this data to plan and protect local provision and spending of

approximately £1.9 billion per year on sport and recreation.

Local Authority measurement is also key for the Department of Health who

currently fund APS questions. The Department of Health have stated that they

(and Local Authorities) require this information to understand the extent of

progress being made in the level of sport and physical activity amongst adults as

a result of local interventions as well as self-reported height and weight.

The availability of Local Authority estimates allows the Department of Health to

include measures of physical activity and height and weight in their Public Health

Outcomes Framework.

The measures outlined in the framework are the “Proportion of physically active

and inactive adults” and the “Proportion of adults classified as overweight or

obese”. B

oth are planned to be taken from the APS. The Public Health

Outcomes Framework is planning to publish data at Upper Tier Local Authority

level, however Local Authorities are very likely to require the indicators at the

lower Local Authority District level.

The Department of Transport also contribute to the APS and use local data for

walking or cycling to work to support their policy objectives.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Use of Modelled Estimates

A face-to-face only option would require modelling to provide estimates for local

authority districts and for lower participation sports as the sample size required

for direct estimation would not be sufficient unless the sample was increased

substantially beyond what current resources allow.

Modelling also has the potential to improve estimates within mixed-mode options

where the sample size is already substantial enough for direct estimation.

However, accountability arrangements mean that SE and the Department of

Health have a requirement to directly measure participation in individual sports.

Similarly SE and the Department of Health strongly favour sports participation

estimates which account for local interventions and initiatives. Modelling will

provide estimates for e.g. “a sport like badminton” or “a borough like Lambeth”

and is therefore less suited to measuring change with confidence than direct

measurement.

Small area estimation (modelling) has been undertaken by SE for descriptive

purposes and considered in the past for the Taking Part survey, but not

implemented. There is no clear method, however, for modelling sports

participation.

A more detailed discussion of modelling is provided in the paper “Discussion of

options for Taking Part and Active People Survey Integration” from the ONS

Methodology Advisory Service. This can be downloaded from the consultation

page of the DCMS and Sport England

websites.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

References

Code of Practice for official statistics:

http://www.statisticsauthority.gov.uk/assessment/code-of-practice

UK Statistics Authority Assessment of DCMS and Taking Part

http://www.statisticsauthority.gov.uk/assessment/assessment/assessment-

reports/index.html

Active People Survey

http://www.sportengland.org/research/active_people_survey.aspx

Taking Part

http://www.dcms.gov.uk/what_we_do/research_and_statistics/4828.aspx

NatCen review. Taking Part and Active People Surveys: an independent review:

http://www.dcms.gov.uk/images/research/TP_InpendReview_of_APTP.pdf

Recognised National Governing Bodies of Sport

http://www.sportengland.org/about_us/recogition_of_sports_and_ngbs.aspx

Ofcom, The Communications Market Report: United Kingdom

http://stakeholders.ofcom.org.uk/market-data-research/market-

data/communications-market-reports/cmr11/uk/

The Application of Alternative Modes of Data Collection in UK Government Social

Surveys

http://www.ons.gov.uk/ons/about-ons/who-we-are/services/data-

collection-methodology/reports-and-publications/alternative-modes-of-data-

collection/index.html

Department of Health Public health outcomes framework

http://www.dh.gov.uk/health/2012/01/public-health-outcomes/

Public health outcomes framework: Summary technical specifications (see page

38 2.13 Proportion of physically active and inactive adults)

http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicy

AndGuidance/DH_132358

DCMS: Creating a sporting habit for life - A new youth sport strategy

http://www.culture.gov.uk/publications/8761.aspx

Sport England strategy 2012-17

http://www.sportengland.org/about_us/what_we_do.aspx

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Summary of Taking Part and Active

People Surveys

Taking Part Survey

Active People Survey

Type of

Survey

Face-to-face household interview Telephone Survey, with 6,000

face-to-face sample household

survey element from APS6.

Sample

Frame

Adults aged 16 and over living in private

households in England.

A child interview was added to the survey in

January 2006, interviewing a randomly selected

child in those households containing at least one

child aged 11-15. In 2008/09, the child survey was

broadened to include children aged 5-10.

The sample universe

comprised all individuals aged

16+ living in England with a

landline.

Random Digit Dialling (RDD)

was used in the selection of the

sample with one respondent

randomly selected from the

eligible household members.

Response

Rate

Around 64% Around 28%

Continuous

Yes

Geographic

Coverage

England England

Sample Size Currently 10,000 face-to-face Currently 165,000 telephone.

6000 face-to-face element has

recently been introduced.

Interview

Length

The median adult sample interview length was 40

minutes. The median survey length for the 5-10

year old child interview was 14 minutes and for 11-

15 year olds it was 24 minutes.

15-20 minute average for the

telephone survey

Survey

Contents

Free time activities

Child engagement

The influence of childhood participation on adult

participation

Sport and active recreation

Attendance Satisfaction

Attitudes to the 2012 Games

Digital engagement

The big society (charitable giving/volunteering/civic

participation

Cycling and swimming proficiency

Archives

Participation in culture

Walking and cycling

Sport participation

Volunteer in sport on a weekly

basis

Club membership

Involvement in organised

sport/competition

Receipt of tuition or coaching

Volunteering

Changes in sports participation

Dance and gardening

Height and weight

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Longitudinal

Panel

Panel established from 2009/10, with follow up

each year until 2015

Potential future development.

Statistics

Badging

National Statistic Official Statistic

Purpose The survey collects a wide range of data about

engagement and non-engagement in culture,

leisure and sport as well as more diverse

measures that support a range of Government

objectives.

Taking Part is the key evidence source for DCMS,

providing reliable national estimates of participation

and supporting the Department’s aim of improving

the quality of life for everyone by providing people

with the chance to get involved in a variety of these

opportunities.

The survey also aims to support the DCMS

Structural Reform Priorities and Business Plan

through the provision of data for a DCMS impact

indicator.

The Primary objective of the

APS is to measure levels of

participation in sport and active

recreation.

APS is used to measure

performance of National

Governing Bodies for 31 of the

46 Organised sports funded by

SE.

APS also provides the

measurement for:

Local area estimates of adult

participation in sport and active

recreation (formerly NI8).

Department of Health Public

health outcomes framework

measures of physical activity

and height and weight

Lowest

Geographic

Level

Regional. Small area estimation would be needed

for Local Authority District estimates. Upper tier

local authority estimates can be provided by

combining years of data.

Local authority district

estimates (with minimum

sample size of 500 per LA per

year). Some local authority

estimates combine 2 years of

data.

Reporting

Cycle

Quarterly Every six months

Cost Approximate gross cost £1.38m per annum

(including co-funders’ contributions from Sport

England, Arts Council England and English

Heritage)

Gross cost £3.1m (including

funding from the Department of

Health and the Department for

Transport)

Survey

contractors

TNS-BMRB TNS-BMRB

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Key Terms and Definitions

Term

Definition

1 x 30 minute MIS

This refers to a person participating in moderate intensity sport

for at least one 30 minute session in the last week (including

recreational walking and cycling).

3 x 30 minute MIS

This refers to a person participating in moderate intensity sport

for at least three 30 minute sessions in the last week (including

recreational walking and cycling).

Active sport at least once in the

last 4 weeks

The number of days in the last four weeks the respondent has

done at least one of the active sports or activities for at least 30

minutes. This excludes any walking or cycling the respondent

has done for health or recreation purposes only. Walking or

cycling for sport training or competition is included.

Active People Survey

A survey commissioned by Sport England to identify how

participation varies from place to place and between different

groups in the population.

The survey also measures; the proportion of the adult population

that volunteer in sport, club membership, involvement in

organised sport/competition, receipt of tuition or coaching, and

overall satisfaction with levels of sporting provision in the local

community.

Further details are available here

http://www.sportengland.org/research/active_people_survey.asp

Attendance

This refers to the respondent going to a place, for example,

attending a library or a swimming pool.

Confidence interval

This provides a range in which there is a specific probability that

the true value will lie within. For the TP and APS surveys, 95%

confidence intervals are used which means, had the sampling

been conducted 100 times, creating 100 confidence intervals,

then 95 of these intervals would contain the true value.

DCMS sectors

The DCMS sectors are the culture, arts, sports and big society

sectors.

Engagement

This refers to either attending and/or participating in the sport,

culture or arts sector. For example, if the respondent attended an

art gallery (attendance) or gave a dance performance

(participation).

Moderate intensity sport (MIS)

This is any active sport or recreational activity that is continuous,

raises a person’s heartbeat, makes them out of breath and

sweaty. This excludes utility walking or cycling, for example to

work, but does include recreational walking and cycling.

Sport England and Department for Culture, Media and Sport

Sport Participation Measurement

Term

Definition

Participation

This refers to the respondent actively taking part in the activity.

For example sports participation could refer to playing football or

tennis, whilst participating in the arts may refer to painting.

Significant increase/decrease

This means there is good evidence that the increase/decrease is

real and not due to chance.

Socio-demographic groups

Categorising the respondents by different social classes such as

age, gender, employment status, housing tenure, ethnicity,

religion, disability/illness status and National Statistics Socio-

Economic Classes (NS-SEC)

Taking Part Survey

A survey commissioned by the Department for Culture, Media

and Sport (DCMS) in partnership with the Arts Council England,

English Heritage and Sport England, looking at engagement and

non-engagement in culture, leisure and sport. Further

information is available at

http://www.culture.gov.uk/what_we_do/research_and_statistics/7

390.aspx