Marking and Moderation Policy and Procedure

Purpose

This document sets out the policy for marking and moderating summative assessments. It is applicable to all assessments, including those which form part of an Apprenticeship course, and those delivered under an Educational Partnership arrangement.

The policy has been developed in line with the UK Quality Code expectations for standards and associated core and common practices, with reference to the associated Advice and Guide theme on Assessment published by the Quality Assurance Agency¹.

This policy concentrates explicitly on procedures that should be followed when marking and moderating or double marking pieces of assessment; it does not cover issues relating to the setting and scrutiny of examination papers and/or clinical practical assessments. For information on these matters please refer to the Setting and Scrutiny of Assessments Policy.

Some courses may have approved exceptions to the standard generic marking criteria. This is approved at the point of approval and information provided to relevant students in a separate additional document. The principles of Marking and Moderation set out in this Policy and Procedure will still apply.

1 Marking

1.1 Marking is the process by which assessors objectively consider whether the individual learner has achieved the intended learning outcomes (ILOs) and how well they have performed against the specified assessment criteria for the unit or individual piece of assessment. As such, it involves the application of professional academic judgement. However, for some types of assessment (for example multiple choice assessments) marking may be automated using an optical mark reader (OMR) or similar, or may require the marker to assess the learner’s work against a model answer. Marking involves the provision of appropriate feedback/feedforward to learners as a commentary that links to the mark/grade awarded and (where applicable) to the relevant marking criteria, that learners can use to help them learn and improve their performance. See the Assessment Feedback Policy which also addresses issues such as calibration of feedback where there are multiple markers.

Staff involved in marking

1.2 All marking activities should be carried out by suitably qualified and trained academic staff.

1.3 Where learners are assessed in a work-place or in practice, the University requires that work and practice-based assessors who act as markers are suitably qualified and trained. Where these assessors act as markers, the University requires that moderating is carried out by academic staff members. Work or practice-based assessors should not act as moderators.

1.4 Responsibility for allocating markers and moderators and for setting deadlines for the marking and processing of assessment marks rests with the Framework or Course Lead in consultation with the relevant Course Administrator, and with oversight by the College Principal with regard to workload management.

1.5 Whenever there is more than one marker for a written assessment, calibration of a small sample between the markers at an early stage in marking is required, to ensure consistency of marking approaches and standards.

Assessment Criteria

1.6 Summative assessments should be marked against clearly defined assessment criteria, and all markers and moderators for that assessment should work to these criteria when reaching their judgements and marks. Where more than one marker is involved, initial calibration should take place to help ensure alignment of marking standards between the markers before marking of the entire set of work takes place. Assessment criteria may be defined at course, unit or component of assessment level, and must be made available to learners. Please refer to the Generic Assessment Criteria Policy and Procedures, including the criteria set out in Appendix 1.

1.7 Feedback to learners on assessment should clearly relate to the relevant assessment criteria.

Step marking

1.8 Where a scale of 0-100% would normally be used to mark a piece of work - (e.g., essay-based assessments, examination answers, presentations, etc., the University uses ‘step marking’, alongside the generic assessment criteria, to simplify and clarify marking for both learners and staff. This provides a limited set of marks which may be used within each classification band, at the lower, middle and upper points of the band.

1.9 Step marking does not apply for assessed work that is quantitative (e.g., multiple‐choice examinations), where there are ‘right or wrong’ answers, and/or where there is a detailed mark scheme under which each question is allocated a specific number of marks (e.g., some short answer examination papers). It does not apply for pass/fail assessments.

1.10 Any specific marking requirements set by a Professional, Statutory or Regulatory Body take precedence over the use of step marking.

1.11 Marking requirements for assessments that are part of apprenticeship courses should refer to the apprenticeship standard specific assessment requirements, and where necessary utilise the course specific assessment criteria which aligns to the requirements of the end point assessment.

1.12 Markers using the relevant marking criteria markers should use their academic judgement to identify the appropriate classification band for the work under consideration, and then identify whether the work is at the lower, middle or upper points of that band, allocating the relevant mark, as in the table below. Markers are encouraged to use the full range of stepped marks/grades, as justified by the relevant assessment criteria.

Levels 3-6

Band	Letter Grade		Position within mark band	Mark	UG Award	Signifies
Exceptional work	A+	16	Upper	98	First	Excellent work. Fully achieves the Learning Outcomes on accordance with the Level Descriptors.
			Middle	95
			Lower	92
Outstanding work	A	15	Upper	88
			Middle	85
			Lower	82
Excellent work	A-	14	Upper	78
			Middle	75
			Lower	72
High quality work	B+	13	Upper	68	Upper Second	Generally good work, but with some minor defects. Ably achieves the Learning Outcomes in accordance with the Level Descriptors.
	B	12	Middle	65
	B-	11	Lower	62
Sound work 50-59%	C+	10	Upper	58	Lower Second	Generally sound work, but with a small number of errors or omissions. Satisfactorily achieves the Learning Outcomes in accordance with the Level
	C	9	Middle	55
	C-	8	Lower	52
Sufficient work	D+	7	Upper	48	Third	Adequate work but with a number of significant errors or omissions. Marginally achieves the Learning Outcomes in accordance with the Level Descriptors
	D	6	Middle	45
	D-	5	Lower	42
40% - threshold level- pass
Weak work	E+	4	Fail	38	Fail	Unsatisfactory work with a significant number of serious errors and omissions. Marginally fails to achieve the Learning Outcomes according to the Level Descriptors.
	E	3	Fail	35
			Fail	32
Poor work	F	2	Fail	28 22 18 12 5 0	Fail	Work of a very poor standard containing little of discernible merit. Clearly fails to achieve the Learning Outcomes according to the Level Descriptors.
	F - N/S	0		0	Fail	No submission of work
	G	0		0	Fail	Work contains cause for concern on issues of safety

 

Level 7

Band	Position within Band	Mark	Corresponding PGT Classification
Exceptional work	Upper	98	Distinction
		95
		92
Outstanding work	Middle	88
		85
		82
Excellent work	Lower	78
		75
		72
High quality work	Upper	68	Merit
	Middle	65
	Lower	62
Satisfactory	Upper	58	Pass
	Middle	55
	Lower	52
50% threshold level - PASS
Insufficient work	Fail	48	Fail
	Fail	45
	Fail	42
Poor/very poor work	Fail	38	Fail
	Fail	35
	Fail	32
	Fail	28	Fail
	Fail	20
	Fail	10
	Fail	0	Non submission

 

1.13 Step marking is applied to individual pieces of work. Where unit or component marks are then calculated using weighted averages the step marks for components or sub-components will be used to calculate the weighted average. The overall course unit result will be the mathematical weighted average, which will not necessarily be one of the fixed ‘steps’ points.

1.14 The generic assessment criteria have been designed to facilitate this approach to marking.

Visibility of learner names in marking

1.15 The NCA recognises the importance and benefit of marking being anonymised wherever possible. The principles of anonymous marking will be applied to summative assessments, wherever this is pedagogically and practically reasonable

2 Double marking and moderating

2.1 The purpose of double marking and moderating is to ensure that marking standards and practice are of an appropriate and consistent standard; that assessments have been marked in accordance with the aims and learning outcomes of the assignment, and according to the marking criteria; to provide an element of independent scrutiny of the judgement of markers; and to ensure fairness of treatment for learners. This is important so that staff, learners, external reviewers, and external stakeholders can have confidence in the University’s marking processes and in the final marks awarded and awards conferred on learners. Moderating is a process separate from that of marking and provides assurance that assessment criteria have been applied appropriately, reflecting the shared understanding of the markers. For most assessments moderating takes place through the marking of a sample by a moderator, with sight of the mark.

3 Definitions

3.1 Double marking: the marking of work for a particular assessment by an internal marker other than the person originally designated to mark the work. Each marker marks the work independently, reaching their own judgement against the same marking criteria, and neither is aware of the other’s decision when determining their own mark. (Note this is sometimes referred to as double blind marking- see Section 6).

3.2 Moderating: the marking by an internal marker other than the person initially marking the work, of a sample of work from a particular assessment, in order to confirm (or not) the appropriateness of the mark awarded by the first marker and proper application of the assessment processes across the sample size for the assessment task (for example whether the marking criteria have been applied fairly and (where applicable) whether the feedback given is consistent with the marking criteria and with the final mark given). As the process focuses on sampling and trends, the process does not result in changes to individual marks within the sample, to avoid any advantage or disadvantage for those learners whose assessed work does not form part of the sample (see Section 7).

3.3 Second Marking: marking by the moderator of all or a sample of the work for a particular assignment, or all the work marked by a particular marker (where more than one marker is used for an assessment) in circumstances where a marking discrepancy is identified by the moderator and where there is no discernible pattern (marking high of low) to the discrepancy. The purpose of second marking is to help ensure fairness and objectivity (See paras 7.4 and 7.6.)

3.4 Adjudication: the use of an independent third marker to arbitrate to determine a final mark in cases when the first two markers (where work is double marked), or the marker and the moderator, cannot agree (see section 8).

4 Responsibilities

4.1 The Course Lead is responsible for overseeing effective marking and moderation for all assessments in accordance with this policy and procedure, including the upkeep of an effective record of all marking and moderation activity

4.2 The Unit Leader is responsible for:

• organising and overseeing the marking and moderating process for their unit;

• completing the post-assessment moderation process record for OMR marked papers;

• updating the unit marks spreadsheet to reflect any changes to marks; and

• ensuring the final unit mark spreadsheet (which incorporates the moderation record), and/or the confirmation of post-moderation process record for OMR marked papers is completed.

4.3 Where several markers are allocated for an assessment task, the Unit Leader must identify a lead person to coordinate the marking and moderation arrangements for that assessment task.

5 Marking and moderation sample sizes and make-up - written coursework and examinations

5.1 The sample size and make-up of first submission written coursework and examinations should be representative of the number and type of submissions under consideration, and is specified in the table below:

Context	Sample Size & Make Up for Moderation and Second and Double Marking
Work that does not contribute to the final award classification.	10% of submissions (comprising a minimum of 8 and maximum of 35 submissions) including all fail grades.
Work that contributes to the final award classification (with the exception of projects and dissertations).	10% of submissions (comprising a minimum of 8 and maximum of 35 submissions) including sample from each award band and all fail grades. If concerns or questions regarding marking/performance arises a broader sample should be reviewed. The sample should also include at least two examples from each marker where there is more than one marker to ensure the correct application of the assessment criteria by each marker and to identify any differential trends.
All Level 6 and 7 Projects and Dissertations.	100% of submissions. Complete double marking.
Work which contains cause for concern on issues of safety.	All work which contains cause for concern on issues of safety, and in the case of practical assessments, review by the Course Leader of all markers’ comments and rationale for the concern Complete double marking.

 

5.2 Where cohort size is smaller than the minimum stated, all submissions will form the sample.

5.3 Where scripts for written assessments are divided between several principal markers the sample must include scripts marked by each principal marker.

6 Requirements for second and double marking

6.1 Second and double marking requirements are set out in section 5.1.

6.2 Second or double marking may take place as an alternative to moderation if the Course Lead or marker requests or where, for example, the marker is inexperienced. This would normally be determined as part of the process of allocating markers/moderators. Where the marker is inexperienced the second marker / moderator should be an experienced marker and provide feedback to the inexperienced principal marker on both the level and the nature of the feedback provided.

Process

6.3 For second marking, a second marker reviews all or a representative sample of learners’ marked scripts with full knowledge of the grade and comments made by the first marker.

6.4 For double marking two markers each mark the learner’s work ‘blind’ (i.e. without sight of the other marker’s mark). In both cases, each marker should record, independently, a mark and comments. The markers may work in parallel or sequentially. The two markers should then discuss their marks, and determine an agreed mark and feedback which should be recorded on the learner’s work / feedback form. When there are significant differences between their marks awarded to an individual learner (e.g., a class difference, or a difference of more than 5 marks within a class), the markers should provide a short note to justify the agreed mark reached. The original marks, the agreed mark and justificatory note are recorded on the final mark sheet which will not be made available to the learner. Only the agreed marks and feedback must be recorded on the learners’ work / feedback forms.

6.5 If the two markers cannot reach an agreed mark on a learner’s work, a third marker will be appointed as outlined in Para 8.1 below.

6.6 Where second or double marking is used as an alternative to moderating and a differential trend is identified between two markers, for example marking high or low (see Para 6.4 ) Para 7.7 should be followed. Where several markers have been appointed and a differential trend is identified Para 7.8 should be followed.

7 Moderation Process & Agreeing Marks

7.1 The moderation sample will be selected by the moderator in line with section 5.1 for written coursework and exams.

7.2 Moderating must be undertaken by members of academic staff with previous experience of marking, normally within the UK.

7.3 Moderators should mark the work in the sample using the same assessment criteria and marking scheme, and with prior knowledge of the first mark and comments, in order to ensure the appropriateness of the marking, and to review the proper application of the marking criteria and assessment processes across the sample. Unless arithmetic aggregation has been automated, this must be double checked as part of this process. The process should also confirm that all pages in the sample have been marked.

7.4 If there are no significant differences (e.g., a class difference or a difference of more than 5 marks within a class or no more than one grade difference within a step band, i.e., Middle and Lower in at least 10% of the moderated submissions) between the marks awarded by the marker and the moderator marks for the whole set will be recorded as the agreed marks and recorded on the unit mark sheet and learners’ work / feedback forms (where applicable). In this instance there is no requirement for any discussion between the marker and the moderator, although discussions may take place if either requests this (for example to give feedback to a less experienced marker).

7.5 If significant differences are identified throughout the sample (see Para 7.4 above) and it is clear that there is a differential trend (e.g., marking high or low), the marker and moderator should discuss the position and may agree to change all the marks in the set, in which case the revised marks should be recorded as the agreed marks. In this situation it may be appropriate for a further sample to be moderated to test the consistency of the differential.

7.6 If there is no pattern to the significant differences identified, the whole set must be second marked by the moderator. There should subsequently be a discussion between the marker and the moderator to agree the final marks. If no agreement can be reached the arrangements in Para 8.1 should be followed. A note on the final collated marks sheet should state what has been done and why. Only the agreed marks and feedback must be recorded on the learners’ work / feedback forms.

7.7 Where several markers have been allocated and if a differential trend is identified between them (e.g., one of the markers has marked consistently high or low, compared to the other markers) the marker in question and the moderator(s) may agree to change all the marks for that particular marker only, in which case the revised marks should be recorded as agreed marks and the other markers’ marks do not need to be changed. In this situation it may be appropriate for a further sample to be moderated to test the consistency of the differential.

7.8 If there is no pattern to the discrepancy the whole set for that marker must be second marked by the moderator. There should subsequently be a discussion between the marker and the moderator to agree the final marks. If no agreement can be reached the arrangements in Para 8.1 should be followed. A note on the unit mark sheet should state what has been done and why. Only the agreed marks and feedback must be recorded on the learners’ work / feedback forms. If agreement is not reached, then a third marker should be appointed, as set out under ‘adjudication’ below (Section 8).

7.9 If, following discussions, the moderator or Unit Leader has concerns about the moderation process, this should be raised with the Course Lead who may arrange for further moderation or marking.

8 Adjudication

8.1 If after a process of second marking, double marking or moderating the marker and moderator cannot reach agreement, a third marker should be appointed by the Course Lead. The third marker may mark with or without reference to the earlier marks and then discuss the marks with the marker and moderator as appropriate. The third marker should arbitrate to reach mutually agreed marks. In exceptional circumstances where marks cannot be mutually agreed, the third marker’s marks will stand. Details of the basis of the discussion and/or agreement on the final marks must be recorded on the unit mark sheet for audit purposes. The mark and feedback returned to the learner must be the final agreed mark only.

8.2 The timescales for adjudication will be agreed between the third marker and the Course Lead. Normally a timescale will be set that allows the standard deadline for the return of marks to be met. In all cases the agreed marks must be returned in good time for the preparation of the Assessment Board report for the Preparatory Board. However, if there are serious concerns, Para 15 may need to apply.

9 Oral presentations/examinations

9.1 A sample of presentations/oral examinations which contribute to the final mark for a unit (20% or a minimum of 8 assessments) should be double marked by means of two markers attending the presentation/oral examination or recorded for this purpose. If both markers cannot attend a live assessment/presentation, the first marker should attend, and the other marker must use a recording.

9.2 Sample presentations must be made available for scrutiny by External Examiners either through their attendance to oral presentations/exams or recordings of these.

10 OSLERs, OSCEs and practical skills examinations

10.1 Where a station/item for an assessment as part of a practical skills examination is double marked (i.e., two assessors are present for the same station/item), further moderation is not required.

10.2 Where a case/station for an assessment as part of a practical skills examination is assessed by a single marker, moderation must take place by means of at least one of the following:

• identified individuals appointed to oversee and observe the assessment practice across a sample of cases/stations and assessors;

• video or sound recording of a sample of cases/stations for later review;

• post-assessment discussions using relevant data (such as post-examination group moderation).

The mechanisms for moderation must be auditable (see Para 13.1).

11. Pass/Fail assessments

11.1 For assessments which are pass/fail, as a minimum all fail assessments and a sample of borderline passes (for each marker/examiner where there are multiple markers/examiners) must be moderated, to ensure consistency in application of the marking criteria. The mechanisms for moderation will be determined by the nature of the pass/fail assessment, and must be auditable (see Para 13.1).

12. Optical Mark Recognition (OMR) marked papers

12.1 The purpose of moderating multiple-choice questions/tests which utilise OMR is to confirm correct operation of the OMR Reader and to check for anomalies. A sufficient number of scripts should be scrutinised to ensure that the machine has worked correctly.

12.2 The post-examination moderation process for OMR-marked papers should take place in accordance with the information provided in Appendix 1.

13. Second and double marking and moderating audit trail

13.1 The Course Lead is responsible for ensuring that course teams maintain an audit trail of second and double marking and moderating (including a record of the sample made available to the External Examiner) using the unit mark sheet. This is to demonstrate that double marking and moderating has taken place, even where it has not resulted in a change of marks.

14. The Role of the External Examiner

14.1 External Examiners undertake a separate external moderation process, to verify marking standards, in accordance with Section 11 of the External Examining Policy and Procedures. They should not act as second markers, be asked to arbitrate in cases of disagreement between markers or be asked to assess individual learners. External Examiners may not change marks for individual learners.

14.2 It is essential that evidence is made available to External Examiners to enable them to verify marking standards, to demonstrate that second and double marking and moderating has taken place and to enable them to verify that this policy has been followed appropriately. It should be agreed with the relevant External Examiner how external moderation will be conducted for non- written assessments.

15 Significant concerns highlighted through the double marking and moderating process

15.1 Should significant concerns about marking standards be highlighted through the second and/or double marking and moderating process which necessitate time-consuming remedial action such as a complete re-mark, some parts of the Assessment Board business, and/or the final publication of the results may have to be delayed. Should this occur the Chair of the Assessment Board, the Framework/Course Lead and the Academic Registrar should determine how this will be communicated to learners.

16 Post-assessment review

16.1 After each assessment, those involved in marking the assessment should normally meet together to review the questions and the performance of the rubric in the light of experience of their use. Where there is one examiner only, the examiner should undertake the review. This review process is separate from the marking and moderation process for the assessment which has taken place. The purpose of this review is to consider the strengths and weaknesses of the assessment tool – for example to check that in practice the assessment worked effectively within the time allowed, and that the requirements were clear to learners. Any weaknesses identified should then be addressed for future assessments. The post-assessment review process can also inform annual unit monitoring reports and help to identify staff training needs and modifications to assessment design and strategies.

17 Marking and moderation for courses delivered under approved educational partnerships

17.1 For courses delivered by NCA as an approved educational partner, an additional layer of moderation by relevant University staff may be required, and/or an additional layer of moderation by the link tutor or nominee may be required which should be taken into account when marking activities are planned.

 

Appendix 1 Post-examination moderation process for OMR marked papers

1. The Assessments Administrator provides the OMR report statistics for the examination to the unit leader
2. The Unit Leader reviews the report against the paper to ensure the correct answers to questions have been entered into the OMR.

If any mistakes are found the Unit Leader must notify the Assessments Officer immediately so that the OMR marking can be re-run

3. Any questions where learners raised queries with invigilators should be reviewed by the Unit Leader and at least one other tutor and if there is evidence that any of these queries are valid then the paper should be amended and remarked appropriately.*
4. The Unit Leader should also review the OMR report statistics to identify:

• Any questions where fewer learners got the question right than would be expected by For example:

1. In a 5 option MCQ you would expect 20% of learners to get the answer correct by chance alone, therefore any question with less than 20% correct should be reviewed
2. In a 26 option EMQ you would expect 4% of learners to get the answer correct by chance alone, therefore any question with less than 4% correct should be reviewed

• Any questions where the bottom third of the class performed better than the top third of the class
• Any questions where 95% or more of learners answered correctly to ensure the question was fit for purpose and no error in the wording of question or answer were made

(For resits, where there are insufficient numbers of learners for the OMR report statistics to be valid, the Unit Leader, and one other tutor, should review all questions with a high ‘fail’ rate (normally set at 75%, but at the discretion of the Unit Leader) to determine whether the question was ambiguous, misleading, or inappropriate).

5. Questions identified using the process in para 4 above should be reviewed by the Unit Leader and at least one other member of academic staff. If it is demonstrable that the question is unfair, unclear, ambiguous or misleading then the question should normally be removed from the paper.

6. Questions identified using the process in para 4 above should also be reviewed by the Unit Leader to consider whether they are suitable for use in future assessments, and as a learning tool for the writing of future MCQs/EMQs.

7. The Unit Leader should complete the relevant report and send it to the Assessments
8. The Unit Leader will adjust the marks for the examination according to the following process:

• Learners retain their original raw score mark, but the number of questions that the percentage is calculated out of is reduced by the number of questions removed from the paper
• Adjustments made by this post moderation process should be clearly documented within the unit marks spreadsheet.

9. The review and, if applicable, removal of any questions under paragraphs 3 and/or 7-8 must be undertaken without knowledge of the overall outcomes for the examination in question e. the overall number/percentage of learners passing or failing the examination as a whole.

*Depending on the nature of the error and learner performance it would not necessarily be appropriate always to remove the question.

 

¹https://www.qaa.ac.uk/en/the-quality-code/2018/advice-and-guidance-18/assessment