Multi-source feedback (360-degree review) is a useful mechanism to give feedback to an employee from multiple sources, including subordinates, peers, supervisors, as well as a self-assessment, and in some cases customers and suppliers.
Giving feedback consists of providing information about the impact of staff behavior on other people and / or the completion of a task. The importance of timely and accurate feedback is critical to increase staff performance. A quantitative study conducted by the Corporate Leadership Council (2002) revealed that fairness and accuracy of informal feedback increase staff performance by 39.1%. The same study revealed that the presence of 360-degree review (multi-source feedback) increases individual performance by 8.1%.
Multi-source feedback was initially instrumented for development purposes to increase the cognitive process of self-reflection among participants, and increase self-awareness. As multi-source feedback evolved, it has been linked to performance.
Giving feedback consists of providing information about the impact of staff behavior on other people and / or the completion of a task. The importance of timely and accurate feedback is critical to increase staff performance. A quantitative study conducted by the Corporate Leadership Council (2002) revealed that fairness and accuracy of informal feedback increase staff performance by 39.1%. The same study revealed that the presence of 360-degree review (multi-source feedback) increases individual performance by 8.1%.
Multi-source feedback was initially instrumented for development purposes to increase the cognitive process of self-reflection among participants, and increase self-awareness. As multi-source feedback evolved, it has been linked to performance.
Origins
Bracken, Timmreck, & Church (2001) traced back the origins of multi-source feedback processes to the beginning of the 20 century when psychologists started exploring new methods for measuring performance and selecting employees. In the foreword of the textbook, David Campbell explains how MSF processes were implemented in organizations. The following paragraphs provide a brief synopsis.
In 1922 Walter Dill Scott, the director of the Committee on Classification of Personnel in the US Army, invented the man-to-man comparative scale to replace the traditional seniority system with a merit-based system. He started using the man-to-man comparison scale to measure and reward individual performance. Further improvements to this method included the use of “behaviorgrams” to better anchor evaluation scales.
Years later, the evaluations for supervisors included new dimensions such as personality, originality, leadership, organizational ability, cooperativeness, ability to develop workers, and technical ability. After the war, the application of standardized tests was commonly used by organizations to measure intelligence, mathematical and mechanical abilities, personal inventories and career surveys. These tests were scored by hand and results were only provided to organizations, never to individuals.
A study conducted with Marine officers in 1947 revealed that peer ratings were more accurate that several objective tests. Furthermore, peer ratings were more valid predictors of future performance than supervisor ratings. The critical incident technique developed by Flanagan was particularly useful because described examples of effective and ineffective job performance
In the 1960’s the National Computer Systems (NCS) automated the processing of psychological assessments. Despite the increased use of psychological assessments with the automation process, people’s resistance also increased because the tests included discriminating questions and were continuing being used unilaterally by organizations.
The Peace Corps, a government initiative created by President John F. Kennedy implemented the use of psychological tests, based on the premise that a better understanding of themselves would help individuals adapt to cultural change. This was the first time that results were provided to individuals.
In 1970 Robert Dorn who worked in the Peace Corps leadership training joined the Center for Creative Leadership (CCL) and introduced the practice of providing results of the assessment to the individual. Years later, Robert Bailey an economist that worked for Dorn, had the idea of including others in the assessment process and initiated the multi-source assessment process.
Multi-source feedback: Current trends
Research conducted by the Corporate Leadership Council (2006) revealed that over 80% of their member organizations use multi-source feedback. Moreover, 90% of Fortune 1000 companies use multi-source feedback. The frequency reported by organizations varies: nearly two-thirds of 56 surveyed organizations conduct once a year reviews or every two years, 31 percent of organizations apply multi-source feedback as needed, and 25 percent of organizations offer these reviews every other year.
Common applications
Most of organizations use multi-source feedback for development purposes mainly, however, some organizations use them for measuring performance as well. Most common applications are the following:
* Upward Feedback. Some organizations use multi-source feedback to assess their leaders’ strengths and identify development opportunities. Additionally, the process can detect organization-wide problems, and measure the alignment of leaders with key organizational priorities. The CLC reinforces the importance of using 360-degree reviews for leaders. CLC’s research “Voice of the Leader” (2001) reveals that companies with stronger leadership benches are four times more likely to outperform their industry peers in revenue growth over a 36-month period.
* Development of staff. According to the CLC (2006) nearly 70% of organizations use multi-source feedback for development purposes only. The CLC suggests to concentrate on qualitative feedback on competencies and behavioral attributes.
* Succession planning. Some organizations use 360-degree reviews to identify employees with the necessary attributes to fulfill future leadership roles (high potential employees). In addition, these organizations target leadership development programs around their key talent to prepare them for future leadership positions.
* Review staff performance. The review of staff performance should concentrate in at least two elements: the measurement of tangible results in terms of outputs and outcomes, and the actions that led to those results measured with multi-source feedback. The use of competencies with behaviors provides clarity of required performance criteria, allowing a consistent and transparent evaluation process.
* Compensation. Literature research indicates that few organizations use multi-source feedback for compensation decisions (salary increases and bonuses) because of potentially biased ratings. Additionally, when multi-source feedback is linked to compensation decisions, feedback provided loses its power as a developmental tool.
Benefits
Multi-source feedback provides a comprehensive insight into facets of employee performance and behavior by compiling feedback from multiple perspectives, and enhancing individuals’ understanding of how their colleagues perceive them and their work. Among the most important benefits are the following:
* Align performance with strategy. Competencies should be aligned with the organization’s mission and priorities. Furthermore defined behaviours should focus staff on specific results required for achieving current and future organizational goals.
* Increase transparency in the review process. The use of multiple feedback sources increases validity and acceptance from staff on the findings of the evaluation process. Additionally, the information includes valuable and complementary perspectives, for example, supervisors tend to base feedback on bottom line results and technical competence, while direct reports value factors such as their supervisor’s commitment to their development, and peers tend to focus on technical competence, teamwork, and collaboration.
* Reinforce desirable behaviors. The standardization of behaviors and competencies throughout the organization brings all employees to the same standard and allows them to focus on developing only essential skills.
* Identify and develop key talent. Multi-source feedback provides a transparent mechanism to identify staff with attributes and competencies necessary to fulfill key positions. In addition, the information helps to articulate targeted training and development programs to increase staff’s strengths.
Key Success Factors for Implementation
Many organizations fail to administer multi-source feedback effectively because they ignore essential supportive components such as proper alignment with strategy, linking feedback to training, and being aware of the influence of rater bias.
The effective implementation of multi-source feedback requires a proper alignment with the organizational goals, preparatory actions and follow-up support. The following four actions should be taken into consideration:
1) Increase alignment with the organizational priorities and objectives: Competencies and behaviors measured with multi-source feedback processes must be linked to business objectives, facilitate organizational strategy, and/or cultivate leadership characteristics
2) Eliminate biases by selecting raters that are familiar with the employee
3) Ensure a shared understanding of rating standards and criteria to prevent confusion among raters and prevent error. Raters should receive preparatory training to ensure a proper understanding of competencies, rating criteria and typical rating errors
4) Support employees during and after the process with coaching or mentoring activities to facilitate full understanding of the process, deal with strong emotions that may occur, and prioritize learning actions.
Bracken, D., Timmreck, C., and Church, A. (2001) emphasized the importance of enhancing rating ability and motivation to improve rating performance. The authors included a list of seventeen specific recommendations grouped in three categories: planning and development, implementation, and rater training.
The value of multi-source feedback is accurate feedback, therefore the 17 listed recommendations are a valuable checklist. Among other, the actions included a 2-3 hours of preparatory training to ensure a proper understanding of the multi-source feedback process, its objectives, competencies and behaviors, rating criteria, number and selection of raters, management commitments and allocated resources to improve on specific areas. Preparatory training is critical to help raters understand and avoid typical rating errors, such as leniency, harshness, central tendency, and halo error, ensuring a consistent criteria between raters.
Multi-Rater Feedback Processes in Development Institutions
In March 2007 an online survey was sent to 21 international organizations to identify multi-source feedback experiences and best practices. 10 organizations (48%) submitted responses. 7 of the 10 organizations reported using multi-rater feedback processes. Key findings are summarized below:
Surveyed organizations reported strengthening supervisory skills as the main objective for their multi-source feedback process. The second most common application reported was staff development/training.
* Managers and supervisors are the most frequent recipients.
* Most organizations conduct multi-rater feedback processes on an as-needed basis.
* The majority of the organizations rely on customized multi-rater feedback processes.
* Six of the seven organizations that reported using multi-source feedback incorporate competencies, typically 6-10 competencies.
* All the organizations that reported using competencies include leadership competencies. They reported that the best sources of feedback for assessing leadership competencies are supervisors and subordinates.
* Five of the six organizations (83%) that use competencies utilize interpersonal competencies. Survey results revealed that subordinates and supervisors are the best sources of feedback for interpersonal competencies.
* Three of the six organizations (50%) using competencies include technical competencies. They reported that the best source for feedback are supervisors, followed by internal and/or external clients.
Activities reported to ensure an effective implementation
* The international organizations surveyed reported three activities to support the implementation of multi-source feedback processes: 1) the involvement of the employee’s supervisor, 2) training and learning resources, and 3) the use of external consultants as coaches.
* In addition, most of these organizations provide training sessions to explain the results to feedback recipients, following the application of a multi-source feedback process.
* Most organizations reported that targeted communication/training was critical to increase understanding and acceptance.
* To ensure an effective implementation, organizations reported starting the implementation with a top-down strategy. Additionally, organizations reported the need to assign a dedicated group to manage the implementation. The organizations also rely on technology providers to facilitate information gathering and report generation.
References
* Bracken, D., Timmreck, C., and Church, A. (2001). The handbook of Multisource Feedback: The comprehensive resource for designing and implementing MSF processes”, San Francisco, CA: Jossey-Bass Inc.
* Corporate Leadership Council. (2006). Considerations for implementing 360-degree reviews: secondary research findings, Washington, DC: Corporate Executive Board.
* Corporate Leadership Council. (2003). Trends in 360-degree reviews: literature key findings, Washington, DC: Corporate Executive Board.
* Corporate Leadership Council. (2001). “Voice of the Leader”, Washington, DC: Corporate Executive Board.
* Corporate Leadership Council. (2002). “Building the High-Performance Workforce: A Quantitative analysis of the Effectiveness of Performance Management Strategies”, Washington, DC: Corporate Executive Board.
Bracken, Timmreck, & Church (2001) traced back the origins of multi-source feedback processes to the beginning of the 20 century when psychologists started exploring new methods for measuring performance and selecting employees. In the foreword of the textbook, David Campbell explains how MSF processes were implemented in organizations. The following paragraphs provide a brief synopsis.
In 1922 Walter Dill Scott, the director of the Committee on Classification of Personnel in the US Army, invented the man-to-man comparative scale to replace the traditional seniority system with a merit-based system. He started using the man-to-man comparison scale to measure and reward individual performance. Further improvements to this method included the use of “behaviorgrams” to better anchor evaluation scales.
Years later, the evaluations for supervisors included new dimensions such as personality, originality, leadership, organizational ability, cooperativeness, ability to develop workers, and technical ability. After the war, the application of standardized tests was commonly used by organizations to measure intelligence, mathematical and mechanical abilities, personal inventories and career surveys. These tests were scored by hand and results were only provided to organizations, never to individuals.
A study conducted with Marine officers in 1947 revealed that peer ratings were more accurate that several objective tests. Furthermore, peer ratings were more valid predictors of future performance than supervisor ratings. The critical incident technique developed by Flanagan was particularly useful because described examples of effective and ineffective job performance
In the 1960’s the National Computer Systems (NCS) automated the processing of psychological assessments. Despite the increased use of psychological assessments with the automation process, people’s resistance also increased because the tests included discriminating questions and were continuing being used unilaterally by organizations.
The Peace Corps, a government initiative created by President John F. Kennedy implemented the use of psychological tests, based on the premise that a better understanding of themselves would help individuals adapt to cultural change. This was the first time that results were provided to individuals.
In 1970 Robert Dorn who worked in the Peace Corps leadership training joined the Center for Creative Leadership (CCL) and introduced the practice of providing results of the assessment to the individual. Years later, Robert Bailey an economist that worked for Dorn, had the idea of including others in the assessment process and initiated the multi-source assessment process.
Multi-source feedback: Current trends
Research conducted by the Corporate Leadership Council (2006) revealed that over 80% of their member organizations use multi-source feedback. Moreover, 90% of Fortune 1000 companies use multi-source feedback. The frequency reported by organizations varies: nearly two-thirds of 56 surveyed organizations conduct once a year reviews or every two years, 31 percent of organizations apply multi-source feedback as needed, and 25 percent of organizations offer these reviews every other year.
Common applications
Most of organizations use multi-source feedback for development purposes mainly, however, some organizations use them for measuring performance as well. Most common applications are the following:
* Upward Feedback. Some organizations use multi-source feedback to assess their leaders’ strengths and identify development opportunities. Additionally, the process can detect organization-wide problems, and measure the alignment of leaders with key organizational priorities. The CLC reinforces the importance of using 360-degree reviews for leaders. CLC’s research “Voice of the Leader” (2001) reveals that companies with stronger leadership benches are four times more likely to outperform their industry peers in revenue growth over a 36-month period.
* Development of staff. According to the CLC (2006) nearly 70% of organizations use multi-source feedback for development purposes only. The CLC suggests to concentrate on qualitative feedback on competencies and behavioral attributes.
* Succession planning. Some organizations use 360-degree reviews to identify employees with the necessary attributes to fulfill future leadership roles (high potential employees). In addition, these organizations target leadership development programs around their key talent to prepare them for future leadership positions.
* Review staff performance. The review of staff performance should concentrate in at least two elements: the measurement of tangible results in terms of outputs and outcomes, and the actions that led to those results measured with multi-source feedback. The use of competencies with behaviors provides clarity of required performance criteria, allowing a consistent and transparent evaluation process.
* Compensation. Literature research indicates that few organizations use multi-source feedback for compensation decisions (salary increases and bonuses) because of potentially biased ratings. Additionally, when multi-source feedback is linked to compensation decisions, feedback provided loses its power as a developmental tool.
Benefits
Multi-source feedback provides a comprehensive insight into facets of employee performance and behavior by compiling feedback from multiple perspectives, and enhancing individuals’ understanding of how their colleagues perceive them and their work. Among the most important benefits are the following:
* Align performance with strategy. Competencies should be aligned with the organization’s mission and priorities. Furthermore defined behaviours should focus staff on specific results required for achieving current and future organizational goals.
* Increase transparency in the review process. The use of multiple feedback sources increases validity and acceptance from staff on the findings of the evaluation process. Additionally, the information includes valuable and complementary perspectives, for example, supervisors tend to base feedback on bottom line results and technical competence, while direct reports value factors such as their supervisor’s commitment to their development, and peers tend to focus on technical competence, teamwork, and collaboration.
* Reinforce desirable behaviors. The standardization of behaviors and competencies throughout the organization brings all employees to the same standard and allows them to focus on developing only essential skills.
* Identify and develop key talent. Multi-source feedback provides a transparent mechanism to identify staff with attributes and competencies necessary to fulfill key positions. In addition, the information helps to articulate targeted training and development programs to increase staff’s strengths.
Key Success Factors for Implementation
Many organizations fail to administer multi-source feedback effectively because they ignore essential supportive components such as proper alignment with strategy, linking feedback to training, and being aware of the influence of rater bias.
The effective implementation of multi-source feedback requires a proper alignment with the organizational goals, preparatory actions and follow-up support. The following four actions should be taken into consideration:
1) Increase alignment with the organizational priorities and objectives: Competencies and behaviors measured with multi-source feedback processes must be linked to business objectives, facilitate organizational strategy, and/or cultivate leadership characteristics
2) Eliminate biases by selecting raters that are familiar with the employee
3) Ensure a shared understanding of rating standards and criteria to prevent confusion among raters and prevent error. Raters should receive preparatory training to ensure a proper understanding of competencies, rating criteria and typical rating errors
4) Support employees during and after the process with coaching or mentoring activities to facilitate full understanding of the process, deal with strong emotions that may occur, and prioritize learning actions.
Bracken, D., Timmreck, C., and Church, A. (2001) emphasized the importance of enhancing rating ability and motivation to improve rating performance. The authors included a list of seventeen specific recommendations grouped in three categories: planning and development, implementation, and rater training.
The value of multi-source feedback is accurate feedback, therefore the 17 listed recommendations are a valuable checklist. Among other, the actions included a 2-3 hours of preparatory training to ensure a proper understanding of the multi-source feedback process, its objectives, competencies and behaviors, rating criteria, number and selection of raters, management commitments and allocated resources to improve on specific areas. Preparatory training is critical to help raters understand and avoid typical rating errors, such as leniency, harshness, central tendency, and halo error, ensuring a consistent criteria between raters.
Multi-Rater Feedback Processes in Development Institutions
In March 2007 an online survey was sent to 21 international organizations to identify multi-source feedback experiences and best practices. 10 organizations (48%) submitted responses. 7 of the 10 organizations reported using multi-rater feedback processes. Key findings are summarized below:
Surveyed organizations reported strengthening supervisory skills as the main objective for their multi-source feedback process. The second most common application reported was staff development/training.
* Managers and supervisors are the most frequent recipients.
* Most organizations conduct multi-rater feedback processes on an as-needed basis.
* The majority of the organizations rely on customized multi-rater feedback processes.
* Six of the seven organizations that reported using multi-source feedback incorporate competencies, typically 6-10 competencies.
* All the organizations that reported using competencies include leadership competencies. They reported that the best sources of feedback for assessing leadership competencies are supervisors and subordinates.
* Five of the six organizations (83%) that use competencies utilize interpersonal competencies. Survey results revealed that subordinates and supervisors are the best sources of feedback for interpersonal competencies.
* Three of the six organizations (50%) using competencies include technical competencies. They reported that the best source for feedback are supervisors, followed by internal and/or external clients.
Activities reported to ensure an effective implementation
* The international organizations surveyed reported three activities to support the implementation of multi-source feedback processes: 1) the involvement of the employee’s supervisor, 2) training and learning resources, and 3) the use of external consultants as coaches.
* In addition, most of these organizations provide training sessions to explain the results to feedback recipients, following the application of a multi-source feedback process.
* Most organizations reported that targeted communication/training was critical to increase understanding and acceptance.
* To ensure an effective implementation, organizations reported starting the implementation with a top-down strategy. Additionally, organizations reported the need to assign a dedicated group to manage the implementation. The organizations also rely on technology providers to facilitate information gathering and report generation.
References
* Bracken, D., Timmreck, C., and Church, A. (2001). The handbook of Multisource Feedback: The comprehensive resource for designing and implementing MSF processes”, San Francisco, CA: Jossey-Bass Inc.
* Corporate Leadership Council. (2006). Considerations for implementing 360-degree reviews: secondary research findings, Washington, DC: Corporate Executive Board.
* Corporate Leadership Council. (2003). Trends in 360-degree reviews: literature key findings, Washington, DC: Corporate Executive Board.
* Corporate Leadership Council. (2001). “Voice of the Leader”, Washington, DC: Corporate Executive Board.
* Corporate Leadership Council. (2002). “Building the High-Performance Workforce: A Quantitative analysis of the Effectiveness of Performance Management Strategies”, Washington, DC: Corporate Executive Board.
