2005 Higher School Certificate Standards Package

Introduction

In 2001 the Board of Studies introduced, as part of the Higher School Certificate, major changes to the way in which student achievement was reported. The Board determined that student performance in the examinations from 2001 onwards would be reported in relation to standards (or levels of achievement). Specifically, for 2 Unit courses students receive a mark (out of a maximum possible score of 100) and a place within a performance band. Six performance bands are used with band 1 covering the range of marks from 0 to 49, band 2 covering from 50 to 59, band 3 covering from 60 to 69, band 4 covering from 70 to 79, band 5 covering from 80 to 89, and band 6 covering the range of marks from 90 to 100. With the exception of band 1, the level of achievement represented by each performance band is described in a statement summarising the knowledge and skills typically demonstrated by students who have achieved that performance band. There is no statement corresponding to band 1, which is considered to be below the minimum standard expected.

This way of reporting student achievement is different from that previously used by the Board in that there is no predetermined proportion of students in any course who will be awarded each performance band. In this standards-referenced approach, students are awarded a particular band (and mark) if they demonstrate they have reached the performance standard associated with that band. Importantly, after the performance standards have been set, student achievement in the following years is reported in relation to the same standards. In this way, it is possible to make comparisons between the performances of students who have sat for the examinations in different years.

How the standards were set following the 2005 examinations

Stage 1
The first step in the process was to have the judges individually study the materials in the previous year’s HSC Standards Package – the band descriptions, the examination paper and marking guidelines, and the sample student scripts. By drawing on their professional experiences and understanding of their subject each judge used the packages to develop an understanding of the knowledge and skills typically possessed by students on the borderline between two bands. They developed an ‘image’ of these particular students.

The judges then considered each examination question in turn. For questions that were scored dichotomously (ie right or wrong) each judge recorded the probability that a borderline student would get the question right. For questions that were scored polytomously (eg problems involving several steps or essay responses) each judge recorded the mark that they believed a borderline student would receive. Each judge’s borderline marks for each question were then added to give that judge’s cut-off marks between the bands. Each judge then looked at the total cut-off marks they had created to check that they were satisfied with the outcome. The average of the borderline marks between bands 5 and 6 proposed by each judge was calculated. This value was the initial cut-off mark between band 5 and band 6. A similar process was followed for the other borderlines.


Stage 2
The next step involved the judges meeting to compare and discuss the decisions they had made individually. This process also involved them in reviewing statistical reports specially developed to support the standards-based approach being used. A random sample of students was selected and the marks they obtained on every question on the examination extracted. These data were then analysed and presented in a form that made it easy for the judges to see how students at various ability levels performed on the questions. The judges reviewed and discussed the statistical data. During these discussions they had the opportunity to modify any question cut-off marks they had previously set. If their earlier values and the values predicted by the statistical analysis were different, the judges were not required to change their values; that is, the judges had the choice of whether to change or not. For each borderline, after all changes were made, the cut-off marks proposed by the judges for each question were averaged. The averages for all questions were then added to obtain a cut-off mark for that borderline. At the end of this step a revised set of cut-off marks was calculated.

Stage 3
The next step in the procedure was to have the judges consider a sample of student responses that had been awarded marks equal to the cut-off marks for each question that they had established. In reviewing and discussing these responses the judges were asked to assure themselves that they demonstrated levels of performance in each question that were on the borderline between two bands. The judges then looked at responses to each question that had been awarded marks on either side of the cut-off marks to confirm their decisions. Where possible the judges also looked at the complete work of students who achieved total marks placing them at the borderline between two performance bands, and to assure themselves that (holistically) such students were demonstrating performance at the appropriate borderline levels.

At the conclusion of this process, the final cut-off marks were applied to the mark distribution from the examination, and bands were allocated to students accordingly. This was done by setting the cut-off marks, to the band borderline marks of 50, 60, 70, 80 and 90 that are shown on the performance scale. In this way the raw examination marks awarded to students are aligned to the performance scale. For example, a student who received a raw examination mark that was between the borderline for band 3 / band 4 and the borderline for band 4 / band 5 would receive a mark between 70 and 80 through a simple linear mapping process. In most cases, the examination mark reported will be different from the raw examination mark.

This package

A sample of student responses that were awarded the final band cut-off marks for each question established by the judges has been chosen to be included on this Standards Package. Also provided are the statistics on how those students who were placed on the borderline of two bands performed on the multiple choice and other objective items, a copy of the 2005 examination paper, and a copy of the band descriptions. A copy of the syllabus has also been provided for reference purposes. This material will form an essential part of the operation in future years by exemplifying and clarifying the standards that are applied. It is important to note that:

  • the samples of student examination question responses for each band cut-off typify the standard of work produced by students on the borderline between performance bands. They should not be regarded as exemplary responses, or be seen as typical of all students achieving a particular band
  • where the examination has optional questions, not all questions in the examination paper will be included in this package. The selection of optional questions to be used was made in conjunction with the subject representatives involved in the standards-setting process. One important criterion for selection was the numbers of students answering each optional question, and hence the availability of an adequate number of student responses from which to choose.

This selection should not be seen as indicating that schools should change the options they currently teach or that students will receive any advantage by studying these options.

Exemplary Samples

In addition to the student responses that illustrate the performance standards, some student responses that have received full marks have been included. For each question in the examination where at least one student received full marks the response of one of these students has been provided. The exemplary responses demonstrate characteristics of responses that were awarded full marks. It must be remembered, however, that in some questions other students may have used different approaches to answering the question and still received full marks.

The Purpose of the Standards Package

The Standards Package is designed to serve three purposes:

  • to equip the team of judges, who will have the task of determining what examination marks will represent the borderlines between the performance bands for future examinations, with clear and concrete information showing the standards they are to apply
  • to give teachers and students a clearer understanding of the standard of work required of students in order to achieve each performance band
  • to give teachers and students an understanding of the quality of work a student would need to produce in order to obtain full marks for a question.

The structure of the material

For each section of the examination requiring an extended response, the responses of up to three students, whose work was of the standard typical of students placed at the borderline between band 5 and band 6, are provided. The responses of up to three students whose work was typical of students at the borderlines between band 4 and band 5, band 3 and band 4, band 2 and band 3, and band 1 and band 2 have also been provided. This material is organised in such a way that if a particular section or question from the test paper is selected, and then a particular borderline between two bands selected, it is then possible to view, in turn, the responses of each of the students at that borderline.

In the case of those sections of the paper containing multiple-choice questions, information is provided in table and graphical form showing how students at each borderline responded to each question. For the purposes of this report, students who were considered to be at borderline were those that achieved a score that was within one mark of that borderline for the whole paper. In some cases, the actual numbers of students achieving these marks may have been quite low, and this should be considered when considering the figures provided.

How it can be used

By looking at each question students were required to answer and then studying the responses of the students, teachers will gain a very clear understanding of the standard of work typically produced by students at the borderline between each band. This understanding will be further enhanced when teachers re-read the band descriptions used to report student achievement and match these descriptions to their images of students at each borderline.

Where sections of an examination paper consist of multiple-choice questions, by looking at the tables and graphs provided for each question teachers and students will be able to see, for the groups of students whose marks are equal to the borderline between two bands, the percentage of each group that selected each of the responses A, B, C and D. By reading the question and then looking at the student responses, teachers will develop an understanding of how well students at each borderline answered each question, and importantly, the types of errors students tended to make. This analysis will help to provide a picture of the level of knowledge and skills typical of students at each level of performance.

Where sections of a paper consist of free-response questions, some of which require a short answer (eg a number or diagram or a few sentences or paragraphs), others requiring an extended answer (eg a solution to a problem, or an essay), copies of the students’ responses are provided. Teachers can look at the questions students were required to answer, study the actual responses students provided and then read the corresponding band descriptions to ascertain why the work of these students was typical of that demonstrated by students placed at the borderline of the two bands.