New statistics out today suggest that employment is up 193,000 and unemployment is down 125,000 compared to the previous quarter. Politicians and pollsters, journalists and even(!) House of Commons Library staff, repeat the figures without hesitation. But do the statistics really merit this faith, or is a bit more caution in order?

Official UK data on employment and unemployment are produced by the Office for National Statistics (ONS), based on a survey of households. The survey results are then weighted to match the overall population to give the headline estimates that ONS publishes each month.

Most of the time ONS’ estimates are going to end up being a bit more or a bit less than the true value being measured. This is hardly surprising – if a different set of households was surveyed (which were also broadly representative of the wider population) you would expect to get slightly different figures again. So the key question is: by how much are the statistics likely to vary from the true value?

Compared to their best estimate of a 193,000 rise, ONS calculates there is a 95% chance that the actual change in employment over the last quarter was somewhere between an increase of 55,000, and a rise of 331,000 (equivalently, there is a 5% chance that the true value lies outside the range). Therefore, in this case we can be at least 95% confident that there has indeed been an increase in employment – in statistical terms, the change is ‘significant’.

To take another example, the number of people aged 16-64 who are ‘economically inactive’ (not in work, and not looking for or available for work) is estimated to have increased by 8,000 over the quarter. But ONS calculates there is a 95% chance that the actual change lies between a fall of 112,000 and a rise of 128,000. Since the range includes zero (no change), then based on these statistics alone, we cannot say with 95% confidence that there has been any change in inactivity.

Of course, you might be happy to accept a lower level of certainty – you might be content with being only 80% confident (or 70%, or 60% etc) that there has been some change in employment or unemployment. But that will mean the true value lies outside the confidence interval more often. Given these statistics are released monthly, then there will be many more occasions when changes are mistakenly interpreted as ‘statistically significant’.

However, it would be foolish to dismiss the statistics as no good. By looking at how the figures have varied over a longer time period, underlying trends are made clearer and it is easier to identify anomalous quarterly estimates. So here are a few tips to keep in mind when reading the monthly figures.

  • Don’t focus on just the past couple of quarters; look at changes over the longer run.
  • Avoid getting caught up over small changes (although what counts as ‘small’ depends on the group being considered!). For total employment, a % rise or fall of a few tenths of a percent probably isn’t a significant change. Fewer people are unemployed than employed, so a % change in the number of people unemployed of less than a few percent is unlikely to be significant.
  • The margin of error increases further for smaller populations, such as 16-24 year olds unemployed or people unemployed for more than a year. By the same logic, estimates for regions and smaller geographies are even more uncertain. For example, Wales’ estimated unemployment rate of 7.1% comes with a 95% confidence interval of 5.9%-8.3%.
  • As for parliamentary constituencies, don’t even bother looking at unemployment estimates. Instead the best indicator at this level is the number of people claiming Jobseeker’s Allowance.

Author: Feargal McGuinness