Nexion Consulting

Purpose: The 360° leadership evaluation is a scientific measure of leadership that

Participants

Who should participate?

Leaders benefit the most from our evidence-based leadership feedback if they have subordinates who directly report to them.

Who and how many people rate a single leader?

Feedback results are meaningful and significant if the participants include:

One or more supervisors (Only the supervisors who conduct the performance appraisal)
Five or more peers (Peers who have a good idea how the leader behaves OR acts with others)
ALL SUBORDINATES (i.e. direct reports)

How many leaders can or should be integrated into the evaluation process (i.e., mid to senior management, top executives?)

All leaders can be put through the process. However, we recommend to start with top executives and top management. A top-down approach shows that top management is committed to pursue leadership excellence and authentic leadership behaviours. Another reason to start from the top is linked to the the spill over, cascading or falling domino effect (i.e., behaviours outlined by top management will be adopted by middle managers and their staff) (Bass, Waldman, Avolio, and Bebb, 1987). Also, Mumford et al. (1993) found that individuals were likely to make unethical decisions if those decisions were supported by authority. Therefore, your organization saves time, money and will benefit from the cascading effect if you start from the top.

What happens if the leader has only ONE subordinate or peer?

In the case that only one peer or subordinate responded, we average those responses with those of the other group respectively so that you cannot identify the actual respondent. If only one subordinate and no peer responded, no subordinate ratings will be outlined in our reports in order to guarantee the anonymity of the leader's raters.

Process and Timing

How long does it take to fill out the leadership survey?

Depending on the questionnaire, it takes roughly 8 – 15 minutes for participants (leaders, supervisors, peers, and subordinates) to rate a selected leader.

How long does it take to complete the whole process?

The whole process (Data Input → Leadership report) usually takes 3 weeks (Week 1: Data entry, Week 2: 360° Leadership evaluation, Week 3: Distribution of personalized feedback reports).

Survey and Survey Questions

Why are similar questions asked in a different way?

This is common standard in good psychometric practice to ensure reliability of the responses.

How does the summary feedback that depicts the average of evaluations look like in Nexion's evidence-based feedback report?

Extract of our evidence-based feedback report:

Example:

Sound psychometric properties

As outlined in the following text that has been written by Prof. Antonakis, a psychometric measure has sound psychometric properties if it measures what it is supposed to. Also, before one can propose that a new measure offers something different or is significantly better than measures that have an established history it is important that the proponents of the new measure demonstrate it is reliable and valid (for detailed accounts regarding validity and reliability see Carmines & Zeller, 1979; Kerlinger, 1986; Nunnally & Bernstein, 1994).

Reliability refers to the extent to which a test's indicators are internally cohesive and measures a construct (e.g., leadership behaviours, traits, ability) consistently (Carmines & Zeller, 1979). A measure can be reliable, but could be reliably me assuring the wrong thing and the measure could be consistently off target. Thus, a reliable measure does not imply that the measure is valid.

Inter-rater reliability (variation in measurements when taken by different persons).

Test-retest reliability (variation in measurements taken by a single person or instrument on the same item and under the same conditions at a different time)

Inter-method reliability (variation in measurements of the same target when taken by a different methods or instruments).

Internal-consistency reliability (assesses the consistency of results across items within a test).

Validity indicates "the extent to which any measuring instrument measures what it is intended to measure" (Carmines & Zeller, 1979, p. 17) and when demonstrated, suggests that the measure does what it should do (i.e., is on target, consistently).

Construct validity (Does the measure relate to a variety of other measures as specified in a theory?).

Criterion validity (Does the measure predict or explain variance in outcome measures?).

Concurrent validity (Is it associated with pre-existing tests/indicators that already measures the same concept).

Predictive validity (Does it predict a known association between the construct you are measuring and something else?).

Discriminant validity (Is the measure correlated or uncorrelated to competing measures?).

Convergent validity (Are different measures of the same construct strongly correlated with each other?).

Incremental validity (Does the measure explain unique variance in dependent outcomes beyond the variance that this accounted for by competing constructs?).

Reliable, but not valid:

Reliable and valid:

Not reliable and not valid:

Confidentiality

Are my responses anonymous?

Your responses will be completely anonymous. The rated leader will not know who has provided ratings (apart from the ratings provided by the leader's direct supervisor, which are identifiable from the supervisor’s hierarchical relation to the leader). We only will provide summary feedback to the leader that depicts the average of all evaluations; nobody will get to see the specific ratings you provide.

Do I have to provide a rating?

You are not obliged to participate in this survey. However, we hope that you will complete the survey so that we can provide the participants with valuable information on improving their leadership styles.

Will the rated leader know that I did not provide a rating?

We will not inform the participants if you do not complete the survey.

How will the summary feedback that depicts the average of evaluations look like in Nexion's evidence-based feedback report?

Please refer to the "Survey and Survey Question" section to see extracts of the reports that are given to the rated leaders.

Where is the data stored?

The data are stored on a secured server that is managed by Nexion Consulting Sàrl.

How long will the data be stored on the server?

The data will be removed from the server after the the feedback report has been given to the leaders. Usually toward the end of the year.

Success factors

As suggested by Prof. Antonakis, who reviewed the literature on leadership development, organizations should:

Ensure that the proposed 360° feedback tool has excellent psychometric properties that are scientifically proven (otherwise you cannot be sure whether you measure what you desire to measure and your evaluation can be based on false impressions)
Take a top-down approach
Make leaders recognize that 360° feedback is valuable (Antonioni, 1996)
Prepare leaders for negative feedback (Facteau et al., 1998)
Motivate leaders to seek more feedback (Antonioni, 1996)
Make the leader accountable (e.g., leader meets with raters to discuss the ratings and committing publicly to improving; see Walker & Smither, 1999; see London, 2002 for further discussion) Provide regular feedback (Reilly et al., 1996)
Keep ratings anonymous, especially of subordinates (see Antonioni, 1994)
Select a broad range of subordinates (or all subordinates) and not allowing leaders to select those who will evaluate them
Do not allow the leaders to distribute the questionnaires and brief raters about the nature of the intervention.

Link between leadership effectiveness and performance (Concrete results)

The following meta-analytic results (Judge and Piccolo, 2004) are based on several thousand leadership evaluations and outcomes measures. To facilitate the interpretation of the results we have 'translated' the coefficients of the original study into percentages by using the binomial effect size display (Rosenthal, 1991

Follower satisfaction with the leader is on average more than 5.6 times higher for leaders who outline effective leadership behaviours as compared to leaders who score low on our leadership effectiveness indicators.
Follower job satisfaction is on average more than 4.5 times higher for leaders who outline effective leadership behaviours as compared to leaders who score low on our leadership effectiveness indicators.
Follower motivation is on average more than 3.7 times higher for leaders who outline effective leadership behaviours as compared to leaders who score low on our leadership effectiveness indicators.
Group or organization performance is on average more than 1.7 times higher for leaders who outline effective leadership behaviours as compared to leaders who score low on our leadership effectiveness.

FAQ