EFL Examination System

Setting and Pretesting

1.1 Selection of materials

item writers are commissioned to select material from a wide range of sources, which include books, newspapers and magazines, brochures, advertisements and radio items. They are given detailed guidelines for selection of material, including the following:

• skills to be assessed;

• linguistic complexity;

• subject matter;

• cultural considerations;

• reference to the Cambridge English Lexicon;

• item types.

The material is vetted by the Chair and the Subject Officer for the paper so that unsuitable material can be rejected before the editing stage.

1.2 Test item writing

item writers are either practising teachers or freelance consultants. An annual schedule for the production of test materials is set up and writers are commissioned usually twice a year to write materials. They participate in the editing process, which is led by the Chair and the Subject Officer for the paper.

1.3 Pretesting

Materials written for Reading, structural competence and Listening papers are pretested. After editing, the items are put into short pretests, including anchor items to allow calibration, which are sent to centres worldwide. Each pretest is taken by over 100 candidates. Tests are returned, marked and analysed in order to provide statistics on the discrimination and difficulty for each item. There are three pretesting sessions a year with approximately 100 tests altogether for all the main examinations. Examination candidates and teachers are invited to provide feedback.

Some trialling of Writing and Speaking test materials also takes place.

Marking

2.1 Process

objective tests are marked by a system of on-site supervised marking, and there is extensive use of computerised methods of marking, involving optical mark readers (OMRs).

FCE and CPE writing components are marked by examiners, who are practising teachers, working under the guidance of team leaders. Team leaders are guided by the Principal examiner for the paper. For the CAE writing paper, each script is marked by two examiners during an intensive marking session.

Speaking components involve either one or two trained examiners assessing candidates individually or in pairs. Marks are recorded on OMR (Optical Mark Reader) sheets and sent to UCLES.

2.2 Training

For written papers, mark schemes and sample scripts are discussed by the team leaders and the Principal examiner in order to standardise marking. Assistant examiners either mark sample scripts before the marking process begins, or have to send a sample of marked scripts to team leaders for checking.

For the speaking components, training involves examiners conducting mock tests with students. During the training and standardisation process, examiners watch videos of Speaking tests with candidates and make their own assessments. Their marks are then compared with those given by the Senior Team Leaders.

With an operation of a large, international scale, it is crucial to ensure the standardisation of the conduct and assessment of the Speaking Tests. In order to ensure these objectives, UCLES has set up a global framework of Oral examiner Team Leader Systems.

Oral examiners are assigned to teams, led by Team Leaders, who in turn report to a national Senior Team Leader. Senior Team Leaders play a crucial role in monitoring the standardization of the Speaking Tests and in ensuring that the same professional requirements for the conduct of the Speaking Tests are met in all centres.

They are responsible for the training and co-ordination sessions run for Oral examiners before examination periods, the monitoring of tests, the support of examiners and liaising between their team of examiners and UCLES.

2.3 Checking

During the assessment process many forms of checking take place to ensure the marks awarded are accurate.

For the tests of writing, UK-based team leaders check samples of scripts from examiners and write reports on their performance. During oral examining, Team Leaders make checks on the examiners in their teams.

Results

Results, in the form of grades, are sent to Local Secretaries six to eight weeks after the examination. They then send individual results slips to candidates. These give grades A, B, C (Pass), D and E (Fail) for CPE, CAE and FCE and Pass with Merit, Pass, Narrow Fail or Fail for KET and PET. Candidates receive certificates a few weeks later.

Security

Security is given a high priority. Examination dates are carefully controlled and precautions are taken at each stage of the examination process. External printers must meet strict security requirements. In countries where there is either a Cambridge representative or a British Council Office, these are consulted regarding the suitability of a potential examination centre before approval by UCLES is given. The Local Secretaries, who receive tests and send back scripts, are required to follow strict procedures.

Inspections of examination centres are carried out routinely, as part of the system of quality control and security.