0 items
(Close Menu)

Tag: Exams

What has happened to the attainment gap during the pandemic?

Posted on by Andrew McAteer

Last week, pupils across Northern Ireland received their GCSE and A-level grades – the culmination of two years of work and learning that have been significantly disrupted by the Covid-19 pandemic. Headlines focused on the rise in top grades awarded and the use of teacher-assessed grades, however we at the Centre for Research in Educational Underachievement are keen to understand what the available data can tell us about how the GCSE and A-level attainment gap has been affected by the pandemic.

In a September 2020 blogpost I predicted that “if we were to consider final grades alone, 2020 will superficially appear as a highly successful year in the Northern Ireland Executive’s long-standing commitment to tackling Educational Underachievement”. In May, the release of the latest edition of “Qualifications and Destinations of Northern Ireland School Leavers 2019/20” by the Department of Education (DE), appeared to indicate that this indeed was the case. These data have been used for several years, according to the release, to “inform a wide range of policy areas aimed at raising standards and tackling educational underachievement”. They are used by “policy teams … across the education service”, “are used to respond to Assembly questions”, and “are included in the Department’s accountability and performance management process”. As such, these data represent the best available information for research and policy relating to educational underachievement in Northern Ireland. They feed into the highest levels of policy accountability through the Executive’s Outcomes Delivery Plan and particularly its Indicator 12: the gap between the percentage of non-FSME school leavers and the percentage of FSME school leavers achieving at Level 2 (GCSE) or above including English and Maths. In this blog, we argue that this data release can only provide misleading, half-answers to our question: “did the attainment gap narrow in 2020?”, and should be improved in order to better serve education policy in Northern Ireland.

A disclaimer was added to this year’s release, acknowledging that the way that GCSE and A-level grades were awarded in 2020 was very different to normal years, as well as many other qualifications:

“given the new method of awarding grades in 2019/20, caution should be taken when drawing any conclusions relating to changes in student performance. Year-on-year changes might have been impacted by the different process for awarding qualifications in 2019/20 rather than reflecting a change in underlying performance”.

Whilst this disclaimer confines itself to the technical detail of the awarding of qualifications, it also serves as a reminder of the wider impacts of the pandemic on education, which were unequally felt across the population. Whilst consistent attainment data prior to GCSE is not currently available for Northern Ireland, systematic studies in England identified a loss of learning across age groups and across the curriculum, which was roughly tripled for children eligible for Free School Meals (FSME). Our own research with parents and carers across Northern Ireland in both 2020 and 2021 found that children in disadvantaged households were spending less time learning, and that their parents were less confident teaching them. However – positive steps were also taken during this time, with the release of extra funding to supply low-income families with digital devices for learning and for catch-up programmes, and many inspirational examples of teacher networking, mutual support and upskilling in the use of blended learning. Having some idea of the effects of these unprecedented challenges and remedial interventions on the socio-economic attainment gap is vital as the education system continues to adapt and recover from the pandemic, to avoid further widening of educational inequalities.

What the data says about the GCSE attainment gap

So, did the attainment gap narrow in 2020? The graph below plots GCSE or equivalent (Level 2) attainment amongst FSME and non-FSME school leavers (Indicator 12), as reported in the school leavers survey data release over the past ten years. The increase in average grades appears to have benefitted FSME pupils slightly more than non-FSME pupils in this metric, closing the gap between them from 29 percentage points in 2018/19 to 27.7 percentage points in 2019/20.

% non-FSME school leavers and % FSME school leavers achieving at Level 2 or above including English and maths, School Leavers’ Survey 2019/2020

Taken at face-value, these data would indicate that the attainment gap did narrow in 2020, and therefore that rates of educational underachievement improved. It is clear that average grades increased significantly as a result of the use of centre-assessed grades, and it appears that FSME pupils benefitted slightly more than non-FSME pupils. Provisional GCSE statistics for 2020 released by CCEA suggest that the percentage of GCSEs awarded C or above increased by 7.5 percentage points in 2020, having remained roughly stable since 2016.

Upon closer examination, however, we see that the data underpinning this picture are misleading. The extent to which the attainment gap was narrowed by the new method of awarding grades is obscured by the fact that each year school leaver survey data includes a large number of pupils who left school in 2020 at the end of Year 13 or 14 and therefore sat their level 2 exams one or two years prior. A rough estimate based on numbers of school leavers with A-levels as opposed to GCSEs as their highest qualifications reported in the release, suggests that two-thirds of school leavers fall into this category. For 2019/20, this means that most school leavers sat their exams in ‘normal’ pre-covid circumstances. These pupils, who according to the same data are more likely to be non-FSME, won’t have benefitted from the 2020 uplift. As such, it may be that much of the closing gap observed in the graph above is due to this discrepancy. This effect will be present in this data for at least the next three years, but it is difficult to know its magnitude as the release gives only a proxy indication of the numbers leaving school at the end of year 12, 13 and 14. Furthermore, as DE has suspended the annual Summary of Annual Examination Results (SAER) for 2019/20 and 2020/21, no other administrative data will be available on Key Stage 4 achievement for corroboration during this period.

What about A-level grades?

The point can be further supported by turning our focus to A-level grades as reported in the 2019/20 school leavers’ data, almost all of which will have been awarded in 2020. Here the increase in the percentage of students achieving the benchmark of 3 or more A-levels A*-C is significantly greater than with the Level 2 benchmark. However, we now see that the increase for non-FSME pupils exceeded the increase for FSME pupils by two percentage points, widening the attainment gap by as much.

% non-FSME school leavers and % FSME school leavers achieving 3 or more A-levels A*-C

 

If the uplift in average A-level grades benefitted non-FSME pupils more than FSME pupils, it is unlikely that the opposite is true at GCSE, despite the impression we might get from the first graph (indicator 12). The closing of the attainment gap only appears evident in the 2020 data because proportionally more FSME pupils leave education at 16 and received their grade in 2020.

How could the available data be improved?

In sum, this data release can only provide unclear, incomplete and inaccurate answers to the question “did the attainment gap narrow” in any given year. This crucial question lies at the heart of the Outcomes Delivery Plan’s indicator 12, and so we would contend that, given the uncertainties highlighted above, this data release does not serve its purpose to “inform a wide range of policy areas aimed at raising standards and tackling educational underachievement” and this is particularly the case in relation to the Level 2 (GCSE or equivalent) results which are most often cited as the key measure of educational achievement in Northern Ireland. The release would be greatly improved if it were to provide an accurate breakdown of the number/percentage of pupils represented within each year group, and this breakdown should be reflected in Indicator 12 statistics.

Of course, there may be several other policy areas and indicators that we have not considered here that are equally ill-served by this data. A clear implication of this conclusion is that a more comprehensive range of measures and available data are needed for effective policymaking and accountability. England’s National Pupil Database, which since 1996 has provided detailed, individual pupil data for research and policymaking, should serve as an example. Such data not only allows for accurate, disaggregated analysis of the impacts of policies and interventions, it also avoids duplication of data collection efforts by researchers and government departments. It can also be powerfully linked to other administrative data such as census, health, and employment data – something that has been explored recently in Northern Ireland to explore patterns of GCSE attainment.  The recent Action Plan of the expert panel on Educational Underachievement in Northern Ireland (A Fair Start) indicated that a forthcoming System Evaluation Framework in preparation at DE will make progress in this area, and argued that it must include the standardised collection of Key Stage 1-3 cohort data as well as more accurate measures of attainment at GCSE and A-level (or equivalent). Past failure to achieve consensus with teaching unions and schools should not stand in the way of progress in this area, which is it vital to understanding the true nature and extent of educational underachievement (especially post-covid) and to tackling it effectively.

The ongoing challenge for government, and for schools, is not only to keep on closing the attainment gap, but to do so in a way that is meaningful in terms of ensuring quality educational opportunities for all to create a more equal society. To do this, we will need better data, which will necessitate the allocation of adequate resources and require schools, teaching unions and government to reach a consensus.

Dr Jonathan Harris is the Research Fellow at the Centre for Research in Educational Underachievement

What we can learn about Educational Underachievement from the GCSE and A-Level experience of 2020

Posted on by Andrew McAteer

If we were to consider final grades alone, 2020 will superficially appear as a highly successful year in the Northern Ireland Executive’s long-standing commitment to tackling Educational Underachievement. As with the other administrations in the UK, average GCSE and A-level grades have risen significantly since 2019, following the cancellation of formal exams and belated government decisions to award grades based on teacher predictions rather than those produced by controversial algorithms (unless the latter was higher). The proportion of pupils achieving the benchmark of 5 GCSEs at C grade or above has certainly increased. When full details of these statistics are published by the Department of Education, it will be interesting to see what effect this general bump-up in grades will have had on the various ‘attainment gaps’; between boys and girls, between FSME and non-FSME pupils, and between pupils of different religious/community backgrounds. Although the full picture is not yet clear, the controversy over A-level and GCSE grades points to a fact worth remembering even in ‘normal’ years: our system of examinations and grading is designed to produce a spread of outcomes, and plays a part in re-producing patterns of what is perceived as educational underachievement.

In their article entitled ‘What is ‘underachievement’ at school?’ Stephen Gorard and Emma Smith wrote that in education policy and practice, underachievement can mean one of three things: low achievement, below a particular benchmark; lower achievement than would be expected by the observer; and lower achievement of one group or individual relative to another. As such, underachievement is intrinsically related to standard assessments that invite comparison across the population – particularly GCSEs and A-levels.

All GCSE and A-Level exams were cancelled this year

What do grades mean?

This year, more than ‘normal’ years, concerns have been repeatedly raised about how ‘meaningful’ the grades students receive are. They are important for employers and higher education institutions attempting to make a selection from a pool of applicants for limited places. They are also important for the governance of the education system, for governments and schools to measure their performance against previous years. The fear of ‘grade inflation’ has shone a spotlight on the role of exams regulators, and the entirely normal practice of moderation and the adjusting of grade boundaries to maintain a distribution of grades that is broadly comparable to previous years. Put simply, the fact that many students achieve grades below a C does not indicate a widespread problem of educational underachievement – it is the way the system is designed to work. Despite a nominally criterion-referencing system of grading, where in theory every candidate can achieve a passing grade if they demonstrate the required level of knowledge or skill, in reality marks are always moderated and/or standardised to produce a spread of grades. What has been highlighted this year is how arbitrarily an individual student might have their marks revised downward, particularly if they are perceived as ‘overachieving’ in relation to their school’s performance history. In normal years, moderation is applied to non-examined assessments (NEAs) in this way, and standardisation applies across the board. This perpetuates downward pressure on the likely attainment of disadvantaged pupils, widening the gap between their grades and those of their more advantaged peers who are more likely to request and obtain a re-mark that will revise their grade upwards. Such practices are justified on the basis that a) pupils’ grades should be broadly comparable to the cohorts of previous years, and b) that universities and others need to be able to fairly allocate limited places to applicants. However, the former imperative reduces the likelihood of disadvantaged students ‘breaking through’ and the latter builds inequality of outcomes into the education system as a necessity. This isn’t objectively unfair, but neither is it objectively fair – rather it is a political judgement passed off as a technically unavoidable problem.

Whose grade is it anyway?

The recent controversy has also broken down the widely held assumption that an individual pupil’s grade belongs solely to them. In a ‘normal year’ standardised public exams are meant to provide a way for an individual pupil to demonstrate their own individual competence in a given discipline by ‘taking the test’ – with no outside help allowed. The recent removal of controlled assessments (coursework) from GCSEs has served to make this individualization more acute, as the pupil is, in theory, removed from the ‘unfair’ help of their teachers, parents, friends and tutors. The absence of these tests taking place in 2020 has highlighted that GCSE and A-level grades belong not only to individual pupils, but also to their teachers, to their schools and ultimately to their governments (via independent exams regulators). Basing pupils’ grades on teachers’ predictions has been attacked on the basis that teachers and schools will inflate their pupils’ grades to game estimates of their own performance. The stakes are high, as teachers and schools perceived as underperforming may be subject to punitive managerial measures or a lack of applicants in future years. Schools with disadvantaged intakes were particularly likely to be downgraded by the algorithms of Ofqual and CCEA on the basis that their estimates were too far above the historical average for their school. Such is the power of this accountability regime, that the initial attempts to award grades through algorithms focused on parity with previous years on a national scale and at the scale of the school, whilst individual grades could be wildly different from pupils’ and teachers’ expectations. As Jon Andrews of the Education Policy Institute has pointed out,

“we had ministers celebrating the fact that A level results had only increased by a couple of percentage points, that standards had been maintained. But this was not a model that needed to work at a national level, it needed to work for hundreds of thousands of individuals in thousands of schools. It does not matter if your total number of grades is correct if a large number of them have been assigned to the ‘wrong’ candidates.”

If we recognise that a pupil’s grade belongs not only to themselves but to their teachers, their schools and their governments, who are in some way held accountable for the grades their pupils attain, then we may be able to critically re-examine educational underachievement and understand that it cannot be ‘solved’ under the current system of exam-driven selection for teacher/school accountability and options for higher education and employment.

However, this is not to say that nothing can be done, rather that our perspective on what constitutes achievement in education must be broader than the grades pupils obtain aged 16 and 18. This might mean abandoning standardisation in favour of true criterion-referencing, so that fewer young people are necessarily branded as ‘underachievers’. It might mean disconnecting accountability regimes from these grades, and thereby allowing them to ‘belong’ entirely to the pupil, whilst teachers and schools’ success is measured in another, less easily quantifiable way. There is no unshakeable rule that schools must be made to compete with one another on the basis of grades in order to improve standards. Could a teacher or school’s performance be assessed through criterion-referencing, rather than the zero-sum game of absolute competition embodied in league tables and England’s new Progress-8 scoring system? Of course, parents and the press might continue to fixate on league tables, but accountability measures can and should take a different approach. At the Centre for Research in Educational Underachievement we take this attitude forward in our qualitative work, which valorises the voices of pupils and foregrounds early years and primary education as an arena for meaningful progress in bringing about greater social justice through education.

Dr Jonathan Harris is the Research Fellow at the Centre for Research in Educational Underachievement