Interview Effectiveness


Not


Following the same logic, we can also construct a table that shows when Charlie has made the wrong hiring decision. With these tables, we can create an “Interviewer Success Index” for Charlie that tracks how often he is making the right decisions. Figure 1 shows the results.


h = 64%-o


1 2 3 4 5 6


7 8 9


h = 47%


10 11 12 13 14 15 16 17 18 19 20 21 22 23 24


h = 30%-o


25 26 27 28 29 30 31 32 33 34


35 36 37 38 39 40 41 42 43 44


0% 25% 50% 75%


100% 82% 73% 67% 67% 66%


63.6% 62.5% 60.0% 57.1% 57.1% 56.5% 55.3% 54.5% 53.8% 53.8% 50.0% 50.0% 50.0% 50.0% 50.0% 48.5% 47.1% 47.1%


45.5% 44.4% 44.4% 43.8% 40.0% 40.0% 40.0% 40.0% 39.5% 36.4%


28.6% 27.3% 26.7% 25.0% 25.0% 25.0% 23.1% 22.2% 18.2% 17.6%


100%


SOURCE: Global Professional Service Company with 10,000+ employees © JKU & Associates


23 46 48 54 23 98


54 76 39 81 27 95


138 26 43


130 74


132 58 79 88


104 101 101


104 108 65 79 91 35 97 69


168 84


68 66 99 92 42 61 57 74 98 96


Interesting…but ‘so what’? Perhaps the most striking observation is that there are interviewers who are consistent - both consistently right and consistently wrong. The question that needs answering is “What do the best performing interviewers know that the rest of us do not?” In this particular example, the answer is, not much. Through some distribution and bias analysis, (the details of which I will spare you), we were able to identify three types of interviewing styles: • Type 1: interviewers who follow the processes the company has defined, meticulously filling in their evaluation forms, and making their hiring decisions based on their evaluation,


• Type 2: interviewers who don’t follow the process, but do ensure their feedback forms conform with the decision they’ve made through other means (i.e. retro-fitting evaluation forms),


• Type 3: interviewers who contradict their evaluation forms with their hiring decisions (e.g., scoring candidates highly, but turning them down, or scoring candidate poorly, but hiring them anyway). These interviewers seem to be assigning scores and hiring decisions randomly.


This example is not unique. We consistently see these patterns across industries and companies.


Figure: 1


There are a few striking observations from Figure 1: 1) There is a wide distribution of interviewer success,


2) The average success rate is 47%, just below the 50% success you would expect from flipping a coin to decide whether to hire or not,


3) There are some interviewers who are consistently making the right decisions, and some who are consistently making the wrong decisions.


So how effective is each type of interview style? Those who retro-fit their evaluation forms are the bottom performers. Not surprisingly, those who contradict themselves have a success rate broadly in line with flipping a coin, and interviewers that follow the company’s process are its top performing interviewers. This last point, coupled with some further analysis, can show how effective the recruitment process is.


We do, on occasion, find a few clairvoyants - interviewers who do not follow the company’s processes,


surprisingly, those who contradict themselves have a success rate broadly in line with flipping a coin…


but achieve extraordinary results. As any scientist would do, we track them down, buy them a coffee, and study their techniques very closely.


So what is the outcome of this analysis? Charlie’s company implemented a few changes: 1) Since his company conducts interviews in pairs, they strategically paired their interviewers,


2) They prioritised the interviewers, ensuring the top performing interviewers are more active,


3) They provided feedback to the interviewers (along with their bias tendencies) and introduced stricter interviewing guidelines and training.


How can you replicate this analysis for your organisation? To replicate this analysis for your organisation, you will need: 1) An agreed definition for what recruitment success is for your organisation. From our experience, this usually requires a short workshop with the key stakeholders.


2) Historic interview data (the more the better), including: i. The interviewer ii. The candidate iii. Interview feedback (competency scores, or otherwise)


iv. Hire decision (hire / turn down) v. Tenure and performance results of those candidates hired


3) Some patience to look up candidates turned away.


With this data, you can then construct both the ‘hiring successes’ and ‘hiring failure’ tables as described above. The Success Index can then be calculated by dividing the number of successes by the total number of interviews conducted. n


www.agr.org.uk | Graduate Recruiter 13


Interviewers Success index


Percentage # total interviews


Page 1  |  Page 2  |  Page 3  |  Page 4  |  Page 5  |  Page 6  |  Page 7  |  Page 8  |  Page 9  |  Page 10  |  Page 11  |  Page 12  |  Page 13  |  Page 14  |  Page 15  |  Page 16  |  Page 17  |  Page 18  |  Page 19  |  Page 20  |  Page 21  |  Page 22  |  Page 23  |  Page 24  |  Page 25  |  Page 26  |  Page 27  |  Page 28  |  Page 29  |  Page 30  |  Page 31  |  Page 32