What is PISA?
The OECD Programme for International Student Assessment, which was first carried out in 2000 and has assessed the knowledge and skills of 15 year olds in reading, maths and science at three year intervals since then.
How does it work?
A sample of schools is chosen for each country, where pupils sit tests. The headline results give average scores, the proportion of pupils at each performance level and much more analysis of the aggregate results.
Pupils and school heads also answer background questionnaires on characteristics, family background, their views and attitudes, what happens in the classroom, school types, resources, policies and practices at a school, local and national level. This means that PISA can be used to break down results by type of pupil and to shed light on the links between these underlying factors and pupil performance.
Which countries take part?
How did the UK do in 2012?
Mean UK scores were not significantly different from the OECD average in reading and maths and above the average in science. The OECD estimates a range of ranks to account for the intrinsic variability in survey-based results. These give the following ranges for the UK in 2012: 23rd-31st out of 65 for maths, 20th-26th for reading, and 16th-22nd for science.
So are we getting better or worse?
Neither. Performance in all three subjects was broadly similar in 2006, 2009 and 2012.
I heard that our performance had fallen since 2000. Is that true?
The UK’s 2000 and 2003 samples did not meet minimum response-rates, “…so data from the United Kingdom cannot be used for comparisons.”
So does that mean you can’t really conclude anything from it?
No. The results suggest that performance of UK 15 year olds has remained steady between 2006 and 2012 in each three subject areas.
Does PISA tell us anything else?
The rankings tell us little on their own. The OECD has said in the past that “PISA is much more than just a ranking”. For instance in 2012 boys in the UK did better on average than girls in maths and science and the gap was larger than in most other countries. Girls did better in reading. Immigrant pupils did less well than non-immigrant pupils, but the gap was not statistically significant in the UK and was “far below” the OECD average. UK pupils were less likely to say that poor discipline stopped them working in class.
Can I find local level PISA results?
Nothing below UK level?
Only for the individual home countries. Mean performance was generally better for England and Scotland, slightly lower in Northern Ireland and significantly below the rest of the UK in each subject in Wales.
Is PISA the only source of information in this area?
No. The International Association for the Evaluation of Educational Achievement runs the Trends in International Mathematics and Science Study (TIMSS) and Pupils International Reading Literacy Study (PIRLS).
What do these other studies tell us?
Headline results for 2011 for English 10 and 11 year olds were above the international average in maths, science and reading literacy and the differences were statistically significant. Average scores for 14 and 15 year olds were also above the international science and maths averages, but the difference in maths was not significant.
But this is different from PISA. Does is all have to be so confusing?
It would be more surprising if these different studies, which include different countries and ask different questions of pupils at different ages produced the same results. By looking across the different sources of international evidence on pupil performance we get a broader perspective on the subject, even if the cost of this is some contradictory or confusing findings.
Where can I get more information?
Author: Paul Bolton