Conducting Peer Assessment with Peerwork
What is Peerwork
The Peerwork activity supports teachers with effective and fair grading of group assignments by allowing students to assess each other’s performance and contribution within their group.
First, Peerwork involves students in assessing each other according to several assessment criteria defined by the teacher. Then, the teacher grades the group assignment submitted by students and decides on an overall group mark. Finally, each student receives an automatically adjusted grade which considers both the quality of the group’s written work (assesses by the teacher) as well as each student’s individual performance or contribution (assessed by their peers).
When you should use Peerwork
If you conduct any assessed group projects as part of your module, which require students to collaborate on a piece of work (e.g. report, plan, presentation, essay, portfolio etc.), you can use Peerwork to involve students in the assessment process and offer a fair assessment method which recognises both the quality of produced work as well as individual student contributions.
However, it is very important that you familiarise yourself with Peerwork and how the activity calculates the final grades which is outlined in the “Peer Assessment Scoring Mechanism Explained” section. It is possible that your ideas about Peer Assessment and methods you have used in the past may be different. Most importantly, to fully benefit from all of the features of Peerwork, it is advised to conduct and complete the activity from start to end.
Note: Peerwork does not make it possible for students to mark another student’s piece of written work. The Workshop activity should instead be used for this purpose.
Setting up Peerwork in your module
Peerwork is an advanced tool which requires careful set-up and management by the teacher. Creating the activity in your module area involves several stages and the sections below describe the steps involved in this process. Please take time to familiarise yourself with each of the stages from creating groups to grading the activity and releasing grades as well as dealing with non-respondents and exceptional cases.
Finally, in order to understand and follow the steps described below you need to meet certain prerequisites. In particular, you need to know how to:
- enable module area editing
- add an activity and edit the activity settings
- create and manage student group and groupings
Please refer to existing resources: Add an Activity or a Resource, Create a Group, Create a Grouping and Automatically Create a Random Group
Step 1: Create groups and groupings
First, create Peer Assessment groups and include these groups is a separate grouping. You can use the group Auto-Creation function to create groups and randomly assign students to them or you can use the Group Choice activity to allow students to form their own groups. Finally, you can also create groups manually and assign students to them.
Important: Do not create the activity before you finalise group set-up for the peer assessed assignments. You will need to create relevant groups and groupings before configuring the activity settings as Peerwork needs the group information to recognise and assign students to appropriate groups.
Step 2: Add Peerwork activity to your module
Turn module page editing on and navigate to the relevant section of your module page. Then click ‘Add an activity or resource’ and select ‘Peerwork’ from the activity picker.
Step 3: Configure the activity settings
Under the “General” tab you can:
- provide a descriptive name for the activity
- provide detailed instructions and/or any additional information
Under the “Peer Assessment Settings” tab you can:
- enable/disable the assessment start and due dates
- enable/disable late submissions – if disabled, students will not be able to take part in the assessment past the due date and will be automatically penalised if non-submission penalty is enabled
- enable/disable lock editing – if enabled, students will not be able to update their scores once they click the submit button. Individual attempts can still be unlocked manually by the teacher
- enable/disable file upload – if enabled, student can upload submission file(s). These files can only be accessed by the teacher for the purpose of assigning an overall group mark
- enable/disable self-grading – if enabled, students will be able to grade their own performance along with their peers’ performance.
Note: It is possible to allow students to upload a submission file for the teacher to grade. However, it is also possible to use the Assignment activity for file submission especially if teachers want to benefit from advanced grading features e.g. annotations, rubrics, blind marking, marking workflow, Turnitin etc. Peerwork only allows teachers to use a numerical grade (e.g. 0-100) and leave feedback comments. Student file submissions cannot be previewed and can only be downloaded locally.
Under the “Calculator Settings” tab you can:
- configure the Peer Assessment weighting – this weighting determines what percentage of the final grade will be peer assessed
- configure the penalty for non-submissions of marks
Under the “Peer Assessment Criteria” tab you can:
- configure peer grades visibility – this setting determines whether students can see the scores they received from other students. If enabled, it is recommended to anonymised the scores
- enable/disable peer grades totals – this setting determines whether students can see the peer grades totals as a percentage for each criterion. If enabled, it is recommended to anonymise the scores
- enable/disable justification – if enabled, students will be required to provide a brief comment alongside the scores for each of the criterion. If enabled, it is recommended to anonymised the comments
- create assessment criteria
- select scoring type for each of the criterion
Important – It is highly recommended to use the same scoring type for all of the criteria e.g. Peer Assessment Scale 0-5 or 0-10.
Under the “Common module settings” tab you can:
- configure the group mode
- assign a grouping to the activity
(Optional) Under the “Activity Completion” tab you can:
- configure activity completion settings
Managing and grading assignments with Peerwork
Once the activity has been created and settings configured, students can then take part in peer assessment. Simply, click on the activity name to access the Peerwork interface.
In the Submission Information section, you can see all the important information regarding student responses. In particular, in the Peer Grades column you can see how many students in each group submitted their responses. You can use this information closer to the activity due date to identify and contact students who did not participate in the activity before you start grading.
In the Quick Grading section, you can provide the overall group mark for each of the group. Unlike Advanced Grading, Quick Grading is often used if you already know the grade for each of the submissions and do not need to look at student responses.
In the Bulk Actions section, you can perform the following:
– export all group grades as a csv file
– download all file submissions
– release grades for all groups
– clear responses for all groups
In the Group Actions section, you can perform the same actions as in Bulk Actions but for an individual group of students.
You can also use the Advanced Grading interface which will provide you with more information about student submissions and calculated grades. Simply click the name of one group to access the grading interface.
All student responses are presented in a table for each of the criterion in the Peer Submission and Grades tab. If response locking was enabled in the activity settings, you will see a padlock icon next to each student name. Click the icon to unlock responses for a particular student and allow them to update their responses.
In the Tutor Grading tab, you can provide the overall group mark as well as feedback comments and save changes in order to see the grade calculation. You can also manually revise the final grade in case of any exceptional cases. Please refer to the ‘Peer Assessment Scoring Mechanism’ section for further details.
Dealing with non-respondents
Occasionally, you are likely to encounter students who do not wish to participate in Peer Assessment and these students should be excluded from the activity and assessed on an individual basis. However, once students are assigned to their groups and asked to take part in Peer Assessment, it is not uncommon to encounter non-respondents: students who did not provide responses. Peerwork includes an optional penalty for non-responses which ought to motivate students to submit their scores. In addition, it also allows teachers to monitor responses and contact students who did not provide them in a timely manner. Most importantly, when conducting Peer Assessment, you should always strive to achieve a 100% response rate from your students as this guarantees the most accurate and fair calculation.
In the example above Example Student did not participate in the task and did not score other students despite being scored by others. If you encounter such cases during Peer Assessment you should follow these steps.
Step 1: Contact non-respondents and ask them to participate in the activity.
Step 2: If students are unresponsive then clear the responses for the affected group using the Group Actions section of the grading interface.
Step 3: Navigate to the Group management page and remove the non-respondents from the relevant group. Please note that you may also add other students to the group in order to replace the non-respondents.
Step 4: Ask students in the affected group to provide responses again.
This is done to ensure a fair assessment process as it is not fair for students who did not participate in the activity to receive scores from other students while not providing them with a score. Finally, non-respondents are usually a sign of group conflict and these cases should be investigated on an individual basis.
Dealing with exceptional cases
In this scenario, Example Student awarded themselves the maximum grade while awarding 0 to others in the group. In addition, Example student also received 0 from all of the other students in a group. This is a clear case of a conflict within a group and such cases need to be dealt with on an individual basis and often require manual grade revision.
Peer Assessment Scoring Mechanism Explained
The Peer Assessment scoring mechanism calculates the final grade for each student based on two components: Peer Assessment (PA) score as well as the teacher’s overall group mark. Finally, based on the Peer Assessment weighting (configured in the activity’s settings) appropriate percentages from each of the components are combined together to form the final grade.
In the example below, we are going to use the following scores to calculate the final grade for Example Student.
Step 1: Normalising scores and calculating the Peer Assessment score
In order to calculate the PA score for each student, all scores received by this student need to be normalised and added up. This means that each score received needs to be divided by the total number of scores the student who awarded the score also gave to other students.
The PA score for Example Student is:
3/14 + 6/27 + 4/20 + 3/21 = 0.21428 + 0.22222 + 0.2 + 0.14285 = 0.77935 = 0.7794
Please note that Peerwork calculates the PA score to the fifth value after decimal point, rounding the score up to the fourth value after decimal point. If you have multiple criteria, the PA score is calculated for each criterion and the scores are then added and an average score is derived.
Step 2: Calculating the final score
Before calculating the final score, two important pieces of information need to be considered which are: the Overall Group Mark (awarded by the teacher) and the Peer Assessment Weighting. In the example above, the overall group mark is 80 and the peer assessment weighting is 20%.
First the Overall Group Mark is multiplied by the PA score in order to derive the Peer Assessment component of the final grade.
80 x 0.7794 = 62.35
The Final Grade consists of the PA score (20%) and the Overall Group mark (80%)
62.36 x 0.2 = 12.47
80 x 0.8 = 64
64 + 12.47 = 76.47
Please note that in order to calculate the final grade for each student, Peerwork needs to calculate the PA score first. This score can also serve as an early indication of how each individual student performed compared to the rest of the group. PA score lower than 1.0 indicates weaker performance compared to others in the group, thus an adjusted grade lower than the overall group mark is awarded, and score above 1.0 indicates better performance than others in the group, thus an adjusted grade higher than the overall group mark is awarded. Finally, PA score equal to 1.0 indicates equal contribution among students thus the same mark as the teachers overall group mark is awarded.
Another important thing to remember is that Peer Assessment scores students receive are relative to each other and to explain this we will use the ‘cake’ analogy. Simply imagine that, while assessing each other’s performance, each student receives a full cake and needs to slice the cake accordingly in order to grade others in the group. So bigger slices go to students who performed better and smaller slices go to students whose performance was weaker. At the end, each student collects their slices and tries to assemble a full cake. Some students will end up with an incomplete cake (contribution factor below 1.0), other students will manage to assemble a whole cake (contribution factor equal to 1.0) and some students will end up with more than a full cake (contribution factor above 1.0). More importantly, it is impossible for all students to receive more than a full cake as there are not enough slices to be passed around. What this means is that for one student to receive a Peer Assessment Score above 1.0 and receive a grade higher than the Overall Group mark other students need to receive a Peer Assessment Score lower than 1.0 and receive a lower grade than the overall group mark. However, it is possible for all students to receive the peer assessment score of 1.0 and this happens when all students in a group receive the highest possible scores. In this case, student performance within this group is the same and all students in this group receive a grade equal to the teacher awarded overall group mark.
Creating detailed Assessment Criteria
Peerwork allows teachers to include several detailed assessment criteria. Therefore, while creating the activity, it is a good idea to break down the assessment into several well-defined criteria instead of using one criterion. After all, students working in groups can demonstrate a range of skills and contribute in many different ways.
The Peer Assessment dimensions and items outlined below are a good starting point for building a detailed list of Peer Assessment criteria.
Finally, below is a list of sample assessment criteria with scale descriptors.
Criterion 1: Rate each person’s ability to carrying out designated tasks (Score Range: 0-5)
0 : Made no effort on part of the work allocated
3 : Made an average contribution
5 : Made a wide variety of efforts tenaciously and with success
Criterion 2: Rate each person’s ability to generate ideas and contribute to discussions (Score Range: 0-5)
0 : Contributed no useful ideas
3 : Made an average contribution
5 : Generated a wealth of realistic ideas and constructive discussions throughout
Criterion 3: Communication – This covers clarity of work submitted to the group, negotiation with the group, communication between meetings and providing feedback. (Score Range: 0-5)
0 : No involvement and interaction with the group
3 : Made average contribution
5 : Excellent and clear communication and feedback throughout the task