NZ Initiative's Martine Udahemuka asks what to make of the difference between maths performance results on TIMSS and on National Standards

NZ Initiative's Martine Udahemuka asks what to make of the difference between maths performance results on TIMSS and on National Standards

By Martine Udahemuka* 

From ‘that’ NCEA maths exam that left students in tears, the confusion around exams during last month’s earthquake, to the seclusion of special needs students, it seems the education sector rarely gets a break.  And it looks like that will not change anytime soon with the latest news in education. The most recent international benchmarking test results tell, once again, a doom story for maths performance.

The Trends in International Mathematics and Science Study 2015 (TIMSS) assesses Year 5 and 9 students’ proficiency in these two subjects every four years. The latest test was taken by about 600,000 students in around sixty countries, of which 10% of Year 5 and 9 New Zealand students took part. The previous 2010 results had revealed that by international benchmarks our maths performance had continued to drop since 2002.  

The latest results too bear bad news; despite reform efforts in the past decade to lift achievement, New Zealand’s performance in maths at both levels has remained stagnant. We are no longer declining but neither are we inching our way up. It is unlikely that a student can get away with saying ‘I am just not good at reading or even writing’ but it is common and almost accepted to hear one say ‘I am just not good at maths’. And a pessimist might argue there is a point in the latter because no matter which way you slice it, the maths story leaves little to celebrate. Some key outcomes are worth highlighting here:

  • Relative to past performance, there was no significant change between 2010 and 2014 results for Year 5 and Year 9 students.
  • With regards to progress, there was no significant change between 2010 Year 5 students and 2014 Year 9 students. The four-year administration makes TIMSS performance a good indicator of how the middle primary school cohort have progressed four years-on. Although not exactly the same students, the Year 5 student cohort assessed in TIMSS in 2010 formed the Year 9 TIMSS cohort in the latest round.
  • Average New Zealand performance across both levels puts the country below Australia, England, and the United States. Year 5 students’ performance was similar to students from France and Turkey while Year 9 performance was similar Sweden, Italy, and Malta.

In fact, during the 20 years of the test, we have been consistently on par with or below the international centre point. So our average performance isn’t even average.

But there is another clincher; the gulf that exists between what international assessments of curriculum knowledge show, and what our national maths assessments show. The first tells us we have a great long way to go while the other tells a story of success.

On the one hand, the international test showed that despite test questions being aligned to the curriculum of each country, the average student answered just under half of those questions correctly. Worse, by the end of 2014 less than half of the students were working at the desired level of in the New Zealand Curriculum. This was true for both Year 5 and 9 students. One would rationally think then that domestic assessments would reflect a similar proportion of students meeting curriculum level targets. Looking at the numbers, this does not appear to be the case. 

On the other hand, domestic assessments of performance show that, by the end 2014 - the same time that a random sample of Year 5 students took the TIMSS test - 73% of Year 5s were judged to be achieving ‘at’ or ‘above’ their year level standards in maths.  Almost 70% of Year 8 students were judged similarly. A sample from the latter group sat the Year 9 international test the following year.

It is difficult to understand how tests developed to assess a similar level of knowledge place the same cohorts at different levels of the achievement ruler. National Standards were introduced in 2010 for Years 1 to 8 students to get a more accurate assessment of the progress that each child had made against the curriculum. This progress is assessed through subjective overall teacher judgements and teachers can decide which assessments to use to determine whether each of their students is below, at, or above National Standards.

Indeed, TIMSS surveyed teachers’ assessment methods and found that more Year 5 students had teachers who emphasised the evaluation of students’ ongoing work vis-a-vis nationally available tests. The reverse was true internationally, as many more teachers placed major emphasis on classroom tests. But are teachers’ judgements here perhaps too optimistic or too subjective?

There have been attempts to moderate teacher judgments. For example, the Process and Consistency tool (PaCT) was developed to give teachers confidence in their assessments on National Standards, and to ensure their interim and end-of-year Overall Teacher Judgments are based on valid information, consistent with those of other teachers in the same year levels, and also their previous judgments. Given the contradiction in international assessments when compared to domestic assessments of how many students have a grasp of maths skills, it is a pity that some schools have resisted using the tool. In any case, the paradoxical equation may be worthy of a solution.  


*Martine Udahemuka is a research fellow at the NZ Initiative. This is the Initiative's fortnightly column for interest.co.nz.

We welcome your help to improve our coverage of this issue. Any examples or experiences to relate? Any links to other news, data or research to shed more light on this? Any insight or views on what might happen next or what should happen next? Any errors to correct?

We welcome your comments below. If you are not already registered, please register to comment.

Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.

9 Comments

Comment Filter

Highlight new comments in the last hr(s).

Can someone explain to me why - excepting engineering and science students - there is any point in learning mathematics.....

Generations have ground through learning how to solve linear and quadratic equations for no purpose at all.

What is the point of learning calculus when 98% of humanity does not use that beautiful procedure?

I would be interested to hear from anybody who has actually used equation solving or calculus in their working life. My guess is 98% if humanity does not need those skills.

Personally philosophy seems to have more importance to me and most people face philosophical questions every day.

.

As an inventor I do call upon advanced mathematics from time to time. Integration and differentiation when behind an efficient building system I designed for a house. But what is more important is understanding of principles behind it. Just last week I was explaining Root Mean Square in relation to debunk someones conspiracy theory about sonic survey ships setting off the Kaikoura earthquake. it isn't the doing it, but knowing what can be done.

A shame more people don't understand the exponential function, and the area under curves. They want want to know about their unearned capital gains without thought of where they came from or if they are sustainable.

But sure, the majority of people will never use it.

Well Plato firstly year five maths isn't calculus.
Secondly why bother

These students are not being tested on calculus or quadratics. I have seen the released questions for the PISA assessment for 15 year old students. Many of the questions could be answered using either logic or estimation.
Maths is a critically important subject for our students. Being able to understand concepts such as equivalence, probability and logic are all necessary for our children to participate in society.
As the author of this article pointed out, an equally concerning matter is the discrepancy between teacher assessments and international testing. This grade inflation is a result of teachers/schools trying to inflate their effectiveness. It is a problem in other Western countries such as the UK.

I can tell you exactly why, and as a Board of Trustees member I don't know what to do. The teachers CHEAT when they mark the exams! It's why I'm pulling my kids out of school, to home school them.
I know the principal is pushing it, and 99% of parents wouldn't even think to look because they trust the teachers, and so did I.
If you are a parent ask to see you kids exam answers and go through them yourself. I can't get them, because they know I suspect, so they try and bamboozle me with bullshit. Kids at year 7 can't even tell the time on an analogue clock, and they are supposed to be at standard? LOL Give me a break.
I only want the best for my kids, I've given up on society because 90% of the sheeple want the lies. They want to be told their kids are 'at or above' even if it is total bullcrap. They couldn't care less about the truth if it upsets their world view.

Skudiv - I look forward to hearing your own "subjective" assessment of your children's education ability in 5 years time. I wonder what bullsh*t you will spout to your children as they sit an internal assessment which may read 'not achieved'

Though you do have a point - just a little un-informed.

National standards and NCEA ensure that parents and students are generally told that their child is doing well - regardless of academic ability or grades/credits.

I've taught Maths for 14 years half in the UK and the other half in NZ. Both countries use a form of National Standards which reports to parents what they want to hear - their child is 'at' or 'above' standard. Why would you report 'below' when you can expect a barrage of complaints from the parent as to why their child is not at the required standard. Or even worse be further questioned by your boss!!

Ideally, primary teachers would be happy to assess students as to what curriculum level they are working at/towards but not using this language of above/at/below. This would inform the parent (the ones who care) where the gaps are in their child's learning and provide some extra support/tuition.

I experience the same issue with students at secondary. Many of whom are not at the required level based on age/year group. I could blame their primary/intermediate teachers but what good would that do?

By the way - what the hell does a 'standard' mean in the eyes of an 8 year old?!! And why the bloody hell should they be compared to an 8 year old in Singapore/Finland or Costa Rica - so f'n what?!

This article highlights a growing trend among OECD countries to 'manufacture' the stats to tell parents/students and society where our students are at.

In NZ we target 85% achieving NCEA Level 2 - wonderful...why not make it 99% or 68%? In reality NCEA is derived from Achievement 'Standards' (there is that word again) or Unit Standards that are either academically robust ie. they include examination papers or modular internal assessments where credits are gained through practical tasks and written work.

They simply don't compare.

tonyr - above is spot on with the grade inflation. HOWEVER! Don't blame the teacher or school as this is what is expected of us by the Ministry of Education/Government.

If schools are not meeting these targets then expect ZERO from ERO and as for those BOT reps (present company accepted) perhaps there will be further questions asked of why you are not meeting the needs of the school in terms of academic success. Have you hired the right Principal/teacher?

It is a vicious cycle where accountability falls on everyone and no-one.

In summary Martine - I would not pay much attention to these world comparisons as they are at the very root of what is driving our assessment policies in the NZ education system today - sadly.

Glad to hear you confirm that teachers are cheating to avoid accountability. What a farce!

Cheating is not the operative word as evidence still needs to be provided. The problem is what form of assessment is used to measure against a particular standard. ie a teacher's own subjective quiz/test/portfolio will differ from one teacher or school to the next. This is what makes a mockery of national standards and to a lesser extent NCEA.

This is where a tool like PACT would be useful - perhaps as a board member you should inform the school of this tool (if not already in use).

Don't go blaming the teacher(s) as this is what has been created by the National government since they took office in 2008. It's making our job bloody difficult to do.

After teaching maths in UK for a number of years I came to NZ and was disgusted at the standard of all students in what was then form 3. My question was always , "What the hell have they been doing for the last seven years at school?" I felt NZ students were at least two years behind their UK counterparts. I initially blamed the intermediate years where no progress seemed to be made , but I met young families from the UK with 8 year old children who claimed that their children were "treading water" in the NZ system because the standard was two years behind their previous school. That comparison was between an ordinary working class area in northern England and a top normal school in a middle class catchment area in NZ. further I know several families who returned to Europe because of the poor quality of education here. Follow-up comments from one family suggested that they had only just returned in time for their children to catch up in time for secondary external exams. I also have evidence of the slack marking and assessment carried out by NZ teachers. I remember the farce of teacher meetings assessing the 6th form students for UE accreditation.
I also marked the old school Certificate papers and have recently been able to compare the assessment with Level One NCEA and was horrified that to get the Achievement level in the current system one only needs to get the equivalent of 13% in the School C system. It should have been 20% but apparently there were not enough students attaining the level that they had to reduce it to ensure that an agreed percentage had passed. My experience of occasional tutoring in Ireland and England led me to believe that the two years gap exists at the senior level, with the standard expected of students at Level 13 in NZ is equivalent to Level 11 in those countries.