Scaling in the NSW HSC – call for reader analysis

Children scaledLast month, I did a bit of analysis of maths study in NSW, which has caused a fair bit of interest in my maths loving friends and colleagues. I analysed the change in maths study over the last 25 years in NSW.

Back in 1991, Maths extension 1 (known then as 3 unit maths), which is the course you need to do to do a mathematical subject such as engineering at university, was taken by 27% of students studying English. Now it is taken by 12% of students studying English. Maths extension 2 (known then as 4 unit maths) which is generally a prerequisite to be an actuary, was taken by 8% of students studying English. Now it is taken by 5% of students studying English.

…we want a good proportion of HSC students to be doing rigorous maths. And in 1991, a lot of them were. In 1991, there were more than 15,000 students doing 3 Unit Maths (Mathematics Extension 1) – 27% of those studying any English subject. In 2015, when the total number of English students was 30% higher, only 8,955 students (60% of the 1991 number) studied Mathematics Extension 1 – 12% of all those studying any English subject.

In that post I didn’t suggest a cause for this massive decline in higher maths study. But I’ve been asking people their opinion ever since. And the most popular answer is scaling – that students and their teachers care most about the calculation of the ATAR – the mark that will decide what course they get into at University.

The ATAR is a numerical measure of a student’s overall academic achievement in the HSC in relation to that of other students. This measure allows the overall achievement of students who have completed different combinations of HSC courses to be compared. The ATAR is calculated solely for use by tertiary institutions, either on its own or in conjunction with other criteria, to rank and select school leavers for admission. Calculation of the ATAR [in NSW] is the responsibility of the Technical Committee on Scaling on behalf of the NSW Vice-Chancellors’ Committee.

Many people believe that you will get a better ATAR (given the same underlying ability) by choosing an easier maths course. I have heard this answer from people who studied at public and private schools, some of who were told that by the teachers, others who appeared to have absorbed it from rumour and gossip from their peers. But is it true?

Maths trends in Victoria HSC

In Victoria, I haven’t found long-term trends. But the more recent trends in this report (PDF) is pretty flat. And in the brief description of scaling in the Victorian annual report from last year, maths gets a special mention.

To ensure that students undertaking the more difficult mathematics studies are not disadvantaged, all three mathematics studies are scaled against each other as well as being scaled against all other studies. The higher of the two resulting scales is used for each of the mathematics studies.

So in Victoria, there is both no evidence (in a short-term period) of people increasingly choosing easier maths courses, and the scaling appears to be deliberately designed to avoid it.

So is scaling the issue in NSW? I’ve done a bit of research since, and I can’t work out whether that is true or not. The NSW Vice Chancellors committee publishes a 64 page report (PDF). It has a lot of statistical explanation, and a bewildering range of statistics.  And from the initial preamble the intent is very much to equalise courses (so that choosing an easier course should make no difference):

Because of the lack of comparability of HSC marks achieved in different courses, either when reported against standards or in terms of ranking, marks of individual students are scaled before they are added to give the aggregates from which the ATARs are determined. The scaling process is designed to encourage students to take the courses for which they are best suited and which best prepare them for their future studies. The underlying principle is that a student should neither be advantaged nor disadvantaged by choosing one HSC course over another. [my emphasis] The scaling algorithm estimates what students’ marks would have been if all courses had been studied by all students and all courses had the same distribution of marks. The scaling model assumes that a student’s position in a course depends on the student’s developed ability in that course and the ‘strength of the competition’. Since the ATAR is a rank that reflects academic achievement, ‘strength of the competition’ is defined in terms of the demonstrated overall academic attainment of a course candidature. Scaling first modifies the mean, the standard deviation and the maximum mark in each course. Adjustments are then made to the marks of individual students to produce scaled marks, which are the marks the students would have received if all courses had the same candidature and the same mark distribution.

The only theory I can come up with which is consistent with the very rigorous approach documented in the report, and the widely held belief that you are better off doing easier maths if you want a good ATAR is about the lack of compulsory maths.

In 2001, maths stopped being compulsory in NSW. So the scaling process had one compulsory subject to use to compare all candidates. So the baseline to look at how good the candidates in each subject were was English. They had previously also had maths to use as a comparison. But the people who are good at maths are not necessarily the ones who are good at English (for example in my HSC many many years ago, my [then scaled] score for maths was 15% higher than my [scaled] score for English). So the people doing higher maths get compared against their average english scores, which are probably lower than their maths scores.

This is a purely anecdotal theory. I know how easy it can be to make up a theory without data. And despite the preponderance of numbers, I can’t work out a way to use the statistics in the ATAR scaling report to demonstrate one way or the other whether choosing an easier maths course maximises your ATAR.

So I’m throwing the question out to my readers.

Anyone out there who can go behind the report and work out whether today’s mathematically talented year 10 student should go for easier maths (skip Maths extension 2 or maths extension 1) if they want to maximise their HSC score?

  5 comments for “Scaling in the NSW HSC – call for reader analysis

  1. Sam
    October 29, 2016 at 7:50 pm

    With regards to student opinion on scaling you can’t go past this video from James Ruse.

  2. October 30, 2016 at 1:10 pm

    Great video! But I think the James Ruse people might be doing Maths extension 2 anyway…

  3. Scott Collings
    October 30, 2016 at 2:35 pm

    Hi Jennifer, an interesting article as usual. I would suggest that if even you can’t work it out then its highly unlikely anyone else has and that whatever passes for ‘wisdom’ on this topic is merely speculation. The reduction higher maths enrollments overall possibly reflects the enthusiasm (or lack thereof) of schools to offer this subject. Many schools probably find it difficult to offer Maths 2 since they need a suitably talented teacher (hard to find, if they’re that good at maths they have many better paying options to teaching) and all for possibly only 1 or two students. So its a very inefficient use of school resources. Perhaps the suggestion that ‘it doesn’t scale well anyway’ has been a bit of a rationalisation. The growth in government selective schools over recent decades could be just concentrating the maths talent in a few places but in aggregate denying more even kids at non-selective schools the opportunity to take higher maths subjects. There’s another theory for you.

    My own anecdote runs a little counter to your aggregate analysis. My son attends a selective private school where almost half of the entire year do maths 2 (and then quite a proportion do maths 1). This school, like other selective ones, would not be immune to public scrutiny of its results, so I would be amazed if they encouraged such high rates of participation if they even suspected it might drag the schools results down overall. On the contrary, 30 years ago when I attended the same school there was only one class for maths 2 (4unit) representing closer to 10% of all students in my year. So the current Maths 2 enrollments represent a dramatic increase at this school anyway. As far as I can tell the school seems to think the higher maths scales better, not worse. Again, could be rationalisation, they have the teachers and money is no object (my money – lol).

    Good luck with proving it either way. Your theory about the problem of using English to ‘base’ the candidature seems quite a valid concern.

    cheers
    Scott

  4. Jenny Lyon
    October 30, 2016 at 5:44 pm

    Thanks for the article Jennifer. Scott’s point about selective schools encouraging higher levels of maths rings true to me having had experience with public selective schools. The view I had been exposed to was that if you were good at maths/science then it was worth taking those subjects at a high level (extension 2) as they scaled well – less true if you were going to struggle.

    My understanding of why they removed maths as a compulsory subject was to give students more opportunity to try different areas of interest so I think some of the drift from maths has been because there are so many other interesting options.

    I also felt based on my children’s experience that if you were good at maths it did not really get very interesting until you got into the Extension 2 work so no wonder they consider other subjects. Unfortunately if you want to use some more challenging maths you really need to build that base at school so I feel compulsory maths is a good idea. If you don’t want to undertake a mathematical based career some level of maths to survive in the real world is necessary and the general maths course tries to meet this need so all levels should do some level of maths.

    Jenny

Comments are closed.