Employee performance evaluations can be improved with calibration committees

By adjusting employee ratings, calibration committees can promote evaluations that are more consistent and less susceptible to biases, new MU study finds

February 28th, 2018

COLUMBIA, Mo. – For businesses looking to improve employee performance evaluations, biases and differences across supervisors can be a big hurdle. Employees, of course, want fair evaluations of their performance, but previous research has estimated that differences in employee performance only account for about 20 percent of the variance in ratings, with the rest skewed by unfortunate factors such as bias. Now, a first-of-its-kind study from the University of Missouri has found that calibration committees, which adjust the ratings supervisors give their employees in order to improve consistency, can be useful in mitigating bias and improving fairness.

“In subjective performance reviews, where an immediate supervisor gives an individual job rating, there can be inconsistencies among supervisors in how those ratings are determined,” said Will Demeré, an assistant professor of accountancy at MU’s Robert J. Trulaske, Sr. College of Business. “Some employees might get rated differently simply because supervisors have different views of what constitutes good performance, and translating that performance into a rating opens the door for more variance. Calibration committees attempt to improve consistency by determining what constitutes good performance and adjusting the ratings up or down. We were interested in seeing how that might impact the fairness of these ratings.”

For three years, Demeré and his colleagues collected data on more than 1,300 performance evaluations from a large multinational corporation that used calibration committees. They found that the committees adjusted about 25 percent of the ratings, but decreased the ratings four times as often as they increased them, resulting in a lower average rating. Researchers also found that supervisors learned from the process, as they responded to these adjustments by changing the ratings they gave the next year.

In total, researchers concluded the data showed a marked decrease in leniency bias, which occurs when one supervisor rates more leniently than other supervisors. They also found that employees perceived the evaluation process to be fair. Though centrality bias—the compression of ratings into a smaller range that can fail to account for individual differences—was increased, Demeré believes the impact of calibration committees is predominantly positive.

“Our study shows calibration committees contribute to the fairness of performance evaluations,” Demeré said. “These evaluations are important to people; they can affect bonuses and promotions or a person’s overall standing in a company.”

The study, “The Role of Calibration Committees in Subjective Performance Evaluation Systems,” is forthcoming in Management Science. Other researchers involved in the study were Karen L. Sedatole of Emory University and Alexander Woods of the College of William and Mary.