Peer review of assessment

About

In the construction process of a (written or digital) assessment examiners use a form of peer review or other third-party verification in order to obtain a judgment on validity and reliability (condition 3 Kader Toetsbeleid UvA). How the peer review process is organised is written in the assessmentplan of each study. 

Before the take a peer reviewer can give the examiner answers to questions like:
1. Are there enough questions in total, so one can expect a reliable assessment? 
2. Is the ratio of the questions by topic as is desired from the table of specification, so one can expect a valid assessment?
3. Are the questions independent from each other?
4. Is each question congruent with the level of it’s corresponding learning goal and final goal (as is stated in the table of specification)? 
5. Is each question formulated unambiguous? 

Note: if there are many mall formulated questions, the reliability of the assessment is in danger. Please check each question for adequate content and adequate formulation (see construction advices ENG / NL).

6. Is the answer in the answer model the only answer possible? Or can you give another answer which is not meant to be the answer, but which is theoretically correct?

For example:
Don’t ask: Can you name 3 types of …? (the answer “yes” is theoretically adequate, but you can’t score what you intended to measure. 
But state: Name 3 types of …

7. Is the distribution of points in the answer model adequate conform the specification table?
8. Is spelling and grammar correct?

After the take the peer can help the examiner to verify whether the take meets the standard quality requirements (validity, reliability and for students understandable) and score-grade transformation is justified by filling in the assessment evaluation report.

How to organize peer review?
Make a planning before the start of your course:
1. Who will peer review (colleague(s) and/or assessment coordinator)? Best practice is that two examiners (in all cases no students) are pointed out by the Examinations Board: one examiner and one peer examiner. If a peer examiner is not pointed out, the examiner searches for a peer colleague and/or assessment coordinator, which is not a student.
2. When do you give the peer reviewers the assessment, answer model and table of specification?
3. When do you need the feedback sent by email, so that you are able to optimize the assessment and archive that peer review took place?
4. When do you send the assessment evaluation report to the peer reviewer? 
5. When do you need the assessment evaluation report back?

For the Master Forensic Science peer reviewers are asked to fill in the Checklist Peer Review (her)tentamen MFS.

Original author: Susan Voogd

---

Creative Commons 3.0 BY SA applies to all content on Starfish.
Starfish-education support for the publishing on Starfish-education.eu does not constitute an endorsement of the contents, which reflect the views only of the authors and Starfish-education cannot be held responsible for any use which may be made of the information contained therein. Starfish-education cannot be held responsible for the content published by authors that is not conform with Creative Commons 3.0 BY SA.