2006 Higher School Certificate Standards Package

Introduction

Standards-referenced reporting

The Board of Studies NSW requires student performance in the Higher School Certificate examinations to be reported in relation to standards (or levels of achievement). Specifically, for 2 Unit courses students receive a mark out of 100 and a place within a performance band. There are six performance bands: band 1 covers the range of marks from 0 to 49, band 2 from 50 to 59, band 3 from 60 to 69, band 4 from 70 to 79, band 5 from 80 to 89, and band 6 covers the range of marks from 90 to 100. Except for band 1, the level of achievement represented by each performance band is described in a statement summarising the knowledge and skills typically demonstrated by students who have achieved that performance band. There is no statement corresponding to band 1, which is considered to be below the minimum standard expected.

There is no predetermined proportion of students, in any course, who will be placed in specific performance bands. Using the standards-referenced approach, students are awarded a particular band (and mark) if they demonstrate that they have reached the performance standard associated with that band. Importantly, after the performance standards have been set, student achievement in the following years is reported in relation to the same standards. In this way, it is possible to make comparisons between the performances of students who have sat for the examinations in different years.

How the standards were set following the 2006 examinations

Pre-Stage 1

The first step in the process was to have the judges individually study the materials in the most recent HSC Standards Package for the examination – the band descriptions, the examination paper and marking guidelines, and the sample student scripts. Drawing on their professional experience and understanding of their subject, each judge used the packages to develop an understanding of the knowledge and skills typically possessed by students on the borderline between two bands: they developed an ‘image’ of these particular students.

Stage 1

The judges individually and independently considered each examination question in turn. For questions that were scored right or wrong ('dichotomously') each judge recorded the probability that a borderline student would get the question right. For questions that involved several steps or essay responses (ie were scored 'polytomously') each judge recorded the mark that they believed a borderline student would receive. At the end of this process, each judge’s borderline marks for each question were then added to give that judge’s cut-off marks between the bands. The average of the borderline marks between bands 5 and 6 proposed by each judge was then calculated: this value was the initial cut-off mark between band 5 and band 6. A similar process was followed for the other borderlines.

Stage 2

The next step involved the judges meeting to compare and discuss the decisions they had made individually at Stage1. This process also involved them in reviewing statistical reports specially developed to support the standards-based approach. A random sample of students was selected and their marks for every question in the examination were extracted. These data were then analysed and presented in a form that made it easy for the judges to see how students at various ability levels performed in the questions. The judges reviewed and discussed the statistical data. During these discussions they had the opportunity to modify any cut-off marks they had previously set for particular questions. If these differed from the values predicted by the statistical analysis, the judges did not necessarily have to change their values: that is, they could choose whether or not to change them. For each borderline, after all changes were made, the cut-off marks proposed by the judges for each question were averaged. The average marks for all questions were then added up to obtain a cut-off mark for that borderline; by the end of this step a revised set of cut-off marks had been calculated.

Stage 3

In the next step, the judges considered a sample of student answers that had been awarded marks close to the total cut-off marks that they had established in Stage 2. In reviewing and discussing these answers, the judges were asked to satisfy themselves that, holistically, such students were demonstrating levels of performance on the borderline between two bands. If the judges were not satisfied on this point, they then studied, for each question, answers that had been awarded a mark either side of the cut-off mark to identify where the discrepancy had occurred. Given this task, the judges chose whether to make adjustments to their borderline decisions and, if so, a third set of cut-off marks was calculated.

Consultative Committee meeting

Finally, the judges recommended to the Consultative Committee that the Stage3 cut-off marks be applied to the mark distribution from the examination.

(The Consultative Committee is a team of educational measurement experts appointed to oversee the Higher School Certificate examination setting, marking and standards-setting processes. Based on the information provided by each of these processes, the Consultative Committee decides either to accept the judges’ recommended cut-off marks or to make some adjustments.)

At the conclusion of this process, the final cut-off marks were applied to the mark distribution from the examination, and bands were allocated to students accordingly. This was done by setting the cut-off marks to the band borderline marks of 50, 60, 70, 80 and 90 that are shown on the performance scale. In this way the raw examination marks awarded to students were aligned to the performance scale. For example, a student who received a raw examination mark that was between the borderline for band 3 / band 4 and the borderline for band 4 / band 5 would receive a mark between 70 and 80 through a simple linear mapping process. In most cases, the examination mark reported will be different from the raw examination mark.

This package

A sample of student responses that were awarded the final band cut-off marks for each question established by the judges has been chosen to be included in this Standards Package. Also provided are the statistics on how those students on the borderline of two bands performed in the multiple-choice and other objective items, a copy of the 2006 examination paper, and a copy of the band descriptions. A copy of the syllabus has also been provided for reference purposes. This material will form an essential part of the operation in future years by exemplifying and clarifying the standards that are applied. It is important to note:

• The samples of student answers at each band cut-off typify the standard of work produced by students on the borderline between performance bands. They should not be regarded as exemplary responses, nor should they be seen as typical of all students achieving a mark placing them in a particular band.
• Where the examination paper has optional questions, one question per section is included in this package. The optional questions in this package were selected in conjunction with the subject representatives involved in the standards-setting process; one important criterion for selecting particular options was that a large number of student answers to each optional question was available to choose from.

(The selection of options contained in this package should not be seen as suggesting that schools change the options they currently teach, or that students would receive any advantage by studying these options.)

Exemplary samples

In addition to the student answers that illustrate the performance standards, some that received full marks are included. For each examination question where at least one student received full marks, one exemplary answer is provided to demonstrate characteristics of answers awarded full marks. In some questions other students may have used different approaches to answering the question, yet still received full marks. Answers to different questions may vary in quality of expression, detail and/or argument. Exemplary answers may contain errors that are to be expected from responses produced within the time constraints and under the pressures of an examination.

The purpose of this Standards Package

This Standards Package is designed to serve three purposes:

• to equip the team of judges who will have the task of determining what examination marks will represent the borderlines between the performance bands for future examinations, with clear and concrete information showing the standards they are to apply
• to give teachers and students a clearer understanding of the standard of work required of students in order to achieve each performance band
• to give teachers and students an understanding of the quality of work that a student would need to produce in order to obtain full marks for a question.

The structure of the material

For each section of the examination requiring an extended answer, the package includes up to three answers at a standard typical of students placed at the borderline between band 5 and band 6. Up to three typical student answers at the borderlines between band 4 and band 5, band 3 and band 4, band 2 and band 3, and band 1 and band 2 are also provided.

To access this information for the examination paper:

Step 1: Select the required section in the examination paper

Step 2: Select the required question

Step 3: Select the required borderline to view, in turn, the answers of each of the students at that borderline.

In the case of those sections of the paper containing multiple-choice questions, information is provided in tabular and graphical form showing how students at each borderline have responded to each question. Students included in this analysis are those who have achieved a score within one mark of the borderline for the whole paper.

How it can be used

Teachers will gain a very clear understanding of the standard of work typically produced by students at the borderlines by studying the samples provided in this package. Teachers’ understanding can be enhanced by re-reading the band descriptions used to report student achievement and matching these to the images they have formed of the students at each borderline.

Where sections of an examination paper comprise multiple-choice questions, teachers and students can see from the tables and graphs for each question – for the groups of students whose marks are at the borderline between two bands – the percentage of each group that selected answers A, B, C and D. Reading the question and comparing with the student answers develops an understanding of how well students at each borderline answered each question and, importantly, the types of errors they tended to make. This analysis will help to build a picture of the level of knowledge and skills typical of students at each level of performance.

Where sections of a paper consist of free-response questions, some of which require a short answer (eg a number or diagram or a few sentences or paragraphs), others requiring an extended answer (eg a solution to a problem, or an essay), copies of the students’ answers are provided. Teachers can look at the questions, study the students’ actual answers, and then read the corresponding band descriptions to ascertain why these were typical of work by students placed at the borderline between the two bands.