Examination System
Setting and Pretesting
1.1 Selection of materials
Selection and (where necessary) recording of all materials is done by teams of professional test constructors at Cito (the Dutch National Institute for Educational Measurement) and ICE (the Bureau for Intercultural Evaluation). Materials are taken from a wide range of sources, including books, newspapers and magazines, brochures, advertisements, diagrams, manuals, operating instructions, notices, forms, labels, interviews, discussions, conversations, dialogues, speeches, public announcements, radio programmes, telephone messages, directions, talks and lectures.
1.2 Test item writing
The Cito team produces the tests for Listening and Speaking, the ICE team those for Reading and Writing. Items are subject to revision on the basis of comments from team members and other staff. The final version of each examination has to be approved by the State Examinations Committee especially set up for this purpose by the Ministry of Education.
1.3 Pretesting
All items are pretested. The selection and calibration of items is based on Item Response Theory, using a member of the family of one-parameter Rasch models. Using this design, it is possible, even before tests are administered, to set pass standards which are related to known levels of ability.
Marking
2.1 Process
Scripts produced by candidates in the writing component and full
recordings of the oral test responses are marked by two (and,
if necessary, three) raters, all of whom are practising teachers
of Dutch as a second language who have obtained certificates which
qualify them to act as Raters of State Examinations. In the process
an extensive mark scheme is used, covering a number of aspects
of language performance.
The majority of items in the reading and listening components
can be marked either by computer or clerically.
The marks for all examinations are scaled using IRT. Cut-off
marks are determined by reference to fixed points on the ability
scales.
2.2 Training
Raters attend a three-day training course which familiarises
them with the principles underlying the examinations, and offers
extensive opportunities to practise using mark schemes. Before
they can be employed as raters they have to pass an examination
and obtain an official Raters certificate.
2.3 Checking
A special method of linking raters makes it possible to spot
and remedy deviations. Raters lose
their certificated status if a significant number of their scores
deviates beyond a pre-determined level of acceptance from scores
given by the other raters with whom they are paired.
Results
Raw scores are transformed into marks on a fixed-interval scale running theoretically from 100-900, on which 500 is the pass mark. Candidates are informed of their marks on this scale within six weeks of taking an examination.
Security
Every stage of the examination procedure, from construction to
the actual administration, is carefully controlled to ensure maximum
security.
|
