NAPLAN tests are not tough enough for the level of maths students are studying
- Written by Paul Brown, Lecturer, Curtin University
The latest preliminary NAPLAN results came out recently, but new research has found the test might have little to do with what the kids are actually learning in class.
Our research, presented at the Mathematics Education Research Group of Australasia (MERGA) conference in July this year, found the NAPLAN questions in numeracy for Years 5 and 9 do not cover many of the topics students are studying that year, based on the Australian mathematics curriculum.
Yet the organisation running the tests – the Australian Curriculum, Assessment and Reporting Authority (ACARA) – says NAPLAN should be aligned to the curriculum.
Read more: NAPLAN results show Year 3 students perform better than Year 9 in writing, and it's a worrying trend
New tests, old studies
We found NAPLAN mathematics is nowhere near the year level of study the students are doing at the time of taking the tests. The majority of the Year 5 numeracy test is actually content from the curriculum for Year 2 and Year 3.
For example, these are questions from the most recently released NAPLAN Year 5 Numeracy test.
This is Question 3 from the 2016 Year 5 Numeracy test. The mathematics involved is at Year 2 level, ‘Recognise, model, represent and order numbers to at least 1,000’, curriculum code ACMNA027. Australian Curriculum, Assessment and Reporting Authority (NAPLAN 2012–2016 test papers) This is Question 18 from the 2016 Year 5 Numeracy Test: ‘Compare masses of objects using balance scales’ is in the Year 2 curriculum (code ACMMG038). Australian Curriculum, Assessment and Reporting Authority (NAPLAN 2012–2016 test papers)In the Year 9 NAPLAN test, only 25% is content above Year 7 level. Absent from the tests are some of the more difficult content areas, including use of large numbers to do addition in Year 5 and the index laws (such as xaxb=xa+b) in Year 9.
The tables (below) show that the NAPLAN tests have a year level focus which is much lower than the year level of the students sitting the tests.
Where is the mathematical reasoning?
The NAPLAN Numeracy tests are mainly multiple choice, so it’s not surprising topics that require depth of thinking – such as reasoning and problem solving – are neglected.
This is despite what the lead writer of the mathematics curriculum, Professor Peter Sullivan, says:
[…] reasoning is essential to mathematics, it’s what doing mathematics is about.
Our analysis of 312 questions from the most recent publicly-available NAPLAN tests in numeracy found a significant number of the questions tested recall rather than skills.
For example, in measurement and geometry there is an emphasis on knowing the names of shapes, rather than applications such as symmetry or proof.
We found 19% of what the mathematics curriculum Year 5 students should be able to do was not tested in the Year 5 NAPLAN numeracy test. That figure was 35% for Year 9 students.
Question 16 of the 2016 Year 9 Numeracy test is mathematics at Year 6 level, ‘Investigate combinations of translations, reflections and rotations, with and without the use of digital technologies’, code ACMMG142. Australian Curriculum, Assessment and Reporting Authority (NAPLAN 2012–2016 test papers)The Australian Mathematics Curriculum requires students to be able to sketch graphs, and to explain a plan for selecting a random sample from a population in order to describe the items using statistics. The words “explore” and “investigate” occur in many content areas.
Such aspects of the curriculum are not recognised in the format used in NAPLAN testing. Instead of mathematics being thoughtful and creative, NAPLAN promotes an impression that quick responses to a lot of brief questions reveals a complete picture of mathematical ability.
The move to test online
NAPLAN is moving to an online format, which means tests since 2016 will not be made public because the questions will be reused within the new online system.
In future, students will not all receive the same questions. Instead, questions will be drawn from a secret database depending on how well the students are doing. It will no longer be possible to assess the nature of the questions nor to compare them against the curriculum.
The reliability of NAPLAN is not in question. The questions consistently give predictable results. It is validity that is the issue: does the NAPLAN actually reveal ability at what the students should have been learning?
Read more: Kids learn valuable life skills through rough-and-tumble play with their dads
We say no, as the questions cover material from the curriculum for much younger students and avoid the difficult aspects of mathematics.
NAPLAN is a multimillion dollar exercise which is supposed to assess the attainment of Australian school students. In practice, the results come to teachers too late to be of value for diagnostic purposes, and students are never permitted to know which questions they got wrong.
But NAPLAN is influential in defining the public impression of what mathematics is, and that is not the same as the mathematics curriculum.
If the real purpose of the NAPLAN is to check up on schools and school systems, there has to be a cheaper and less harmful way of doing so.