Understanding Results

Adjustment Factor

An adjustment factor is a value calculated by averaging a student's scores across all five categories and dividing this by the average of the overall rating each team member received. This value is meant to be used as a way to evaluate how well students can work in teams and can be used as a percent if you multiply it by 100. The maximum value is 1.05 and all values greater than 0.95 are rounded to 1.0 unless you click the "show raw adjustment factor" box in the top right corner followed by the "redisplay" button.

While it may seem intuitive to assign grades using a student's average, research actually shows that each team will evaluate its members with varying levels of harshness. This means we can't compare scores between students who were on different teams. In order to account for this variation, an adjustment factor was created to allow instructors to assign students a grade out of 100 based on the results of a CATME peer evaluation.

Types of Adjustment Factors

The CATME system provides two types of adjustment factors: With Self and Without Self. The adjustment factor with self includes the team member's own rating of themselves in the calculations. The adjustment factor without self excludes a team-member's self rating during the calculations.

What Adjustment factor to Use

The answer depends on the team you're evaluating. If an exceptional condition has been identified that causes the adjustment factor column to be highlighted, please enable the pop texts by checking the box in the upper left corner of the page and hovering your mouse of the condition identified to see how it may impact the students' ratings to help you decide. You may also want to talk to your students in person to decide.

Please also view our video on Adjustment Factors to understand how to use this metric effectively, including examples of the adjustment factor calculations:

Exceptional Conditions

Exceptional conditions are rating patterns identified by the system as out of the norm and therefore warrant instructor attention. When an exceptional condition is identified the system delivers messages to both the instructor and the student(s) who triggered the exceptional condition. The student message is different than the instructor message. The conditions that the system identifies are:

Low Performer

The student did not contribute to the team's success. This condition is triggered when their overall average rating is less than 2.5.

Student Message: The members of your team indicated that your contributions to the team were below expectations. This report gives you details about how the members of your team perceived your team contributions in five key areas. Please use this information to identify problem areas in order to contribute effectively in future teamwork situations. Please contact your course instructor if you need assistance or if you believe that your ratings were inappropriate.

Overconfident

The overall rating for the student is less than 3, but the student rated themselves a full point or more higher than their average rating.

Student Message: Your self-ratings were significantly higher than your teammates. ratings of your contributions to the team. The members of your team indicated that your contributions to the team were below expectations. This report gives you details about how the members of your team perceived your team contributions in five key areas. Please use this information to identify problem areas in order to contribute effectively in future teamwork situations. Please contact your course instructor if you need assistance or if you believe that your ratings were inappropriate.

High performer

The student received an average rating better than 3.5 and their rating was more than half a point higher than the overall average for the team as a whole.

Student Message: Congratulations! The members of your team have indicated that you were a highly effective team member. Keep up the good work!

Underconfident

The overall team rating for this student is greater than 3, but the student rated themselves at least a point lower than this value. This would indicate that the student is "underconfident" or too critical of their own contributions.

Student Message: Your self-ratings were significantly lower than your teammates. ratings of your contributions to the team. The members of your team have indicated that you were a highly effective team member. Please try not to minimize the value of your contributions to the team.

Manipulator

One of the students on the team appears to be trying to "skew the curve" by giving themselves high ratings while rating the other team members poorly. The student has to have given themselves an overall rating of 4 or higher while rating all of the other members on their team at least two points below this rating. This exceptional condition also messes with the adjustment factor.

Student Message: Your self-evaluation indicates you made the primary contribution to the project with little value added by your teammates. The ratings from your teammates did not concur with your assessment. Your instructor may require additional information to clarify what happened in your team.

Conflict

A student rated a team-member as 2 or less while the median rating from the rest of the team is 3 or more. This generally indicates that there is a conflict between 2 team members.

Student message: Your evaluation indicates that ______ contributed very little to the project. This is not consistent with the assessment of the rest of the team. Your instructor may require additional information to clarify what happened in your team.

Cliques

The condition is triggered when there is significant disagreement between the ratings from various team members. When this happens, this generally means that the team has split into two non-cooperating groups.

Student Message: There was considerable disagreement among your teammates as to which team members were most effective during team assignments. Your instructor may require additional information to clarify what happened in your team.

Please also watch our video on Exceptional Conditions to understand more on how to handle these situations:

Translating Results into Grades

After your students complete a peer evaluation survey, you might want to obtain homework grades from it.

Our way is to grade your students on how well they rate others, rather than their actual rating. You can do this by assigning Rater Practice and evaluating how well a student did in that. Once your students complete the rater practice, you will be able to view their results by selecting the "view results" button on the class editor page. Now you will be taken to the raw data page where you can scroll down to view your students' rater practice scores. In the report, you can view your students' high scores, average scores and total attempts in rater practice.

You can also grade a student's performance by adjusting everyone's rating using adjustment factors and then calculating each team-member's score based on averages on each category from their peers.

icon1
If the CATME terms make you confused, please check the definition on "CATME Terms Instructor Dictionary".
icon1
See more information in technical documentation.
icon2
Was this page helpful?
Sending feedback now...
Thank you for your feedback! We will use it for future improvements.
Leaving feedback has failed. Please try again.