Panel interviews: scorecards, marks and ranking

A selection committee marks each candidate on weighted criteria, each panelist on their own row; the panel’s score and outcome follow by the rule set in advance, candidates for the vacancy are ranked, and any departure — an overridden outcome, a selection out of rank — is kept with its reason.

A university appointing a professor, or any employer hiring through a committee, needs the decision to be defensible: everybody marked on the same criteria, the marks combined by a rule decided before the interviews, and any exception written down. HRMS → Recruiting & Onboarding has three screens for it — Interview Scorecards, Interview Panels and Candidate Ranking — and three actions on Interviews.

Scorecards: what the panel marks on

Interview Scorecards holds the criteria — each with a Kind (teaching, research, technical, communication and so on), a Weight %, what it is marked Out of (5 or 10, for example) and whether it Must be passed (a panel average below half marks on it is a “no”). The list shows what the weights add to.

A scorecard can be Only for vacancy, Only for job, or The company default. An interview uses the narrowest that applies: the vacancy’s, else the job’s, else the default. The scorecard also sets how the panel decides, the Pass mark (out of 100), what happens on A tied panel, and how far from the rest a panelist’s score must be to be flagged.

The criteria are copied onto the interview when its panel is set up, so changing a scorecard never re-marks an interview already held; and once anybody has marked, the interview’s weights are fixed.

Setting up the panel

On HRMS → Interviews, Set up the panel names the scorecard (empty for the vacancy’s, else the default), the rule and pass mark (empty for the scorecard’s), and the panel — one line a panelist:

• An employee, who marks with their own sign-in, or the name of an external expert with no employee record, whose marks HR enter from their sheet.
• Sits as: chair (one per panel), member, external expert, HR representative or observer.
• Weight: 1 is an ordinary vote; 2 counts twice in the panel score.
• Votes: a panelist who does not vote observes.

The interview’s Panel and Criteria tabs hold the same seats and criteria, to edit one at a time.

Scoring

Score (on Interviews, or on a row of Interview Panels) opens the panelist’s own sheet: what the candidate is marked on, a mark for each criterion within its scale (decimals are fine) with a comment, a Recommendation (strong yes, yes, no, strong no, no decision), strengths, concerns and comments. Each panelist’s marks become a weighted score out of 100.

A panelist marks only their own row. Somebody who runs the panels (change on Interview Panels) can choose Marking for and enter another panelist’s marks — an external expert’s paper sheet; nobody else can mark for somebody else.

How the panel decides

The panel score is the voting panelists’ scores averaged by their weights. The outcome follows at once, by the interview’s rule:

The Interviews list shows the Panel score, how many have Marked out of how many are Voting, and Why — the rule in words. A panelist whose score sits further from the rest than the scorecard allows is Flagged.

Overriding the outcome

The panel’s marks decide the outcome. Override outcome records a different one, with a reason that is kept on the record beside what the marks said; the Overridden column shows it. Choose “derived” to go back to the panel’s outcome.

Ranking and selection

Candidate Ranking lists everybody who applied for a vacancy in the order the panels put them, with the panel score, the rounds marked, the latest outcome and any flagged marks. Candidates on the same score share a rank.

Select chooses a candidate. Selecting the top-ranked candidate still in the running needs nothing more. Selecting anyone ranked below somebody still in the running is allowed, but needs a Reason — Out of rank and Why are kept on the record.

Comparing candidates and panelists

On Hiring Requisitions (and Compare candidates on Candidate Ranking):

• Compare candidates — candidates by criterion and by panelist, ranked.
• Panel consistency — how each panelist marks against the rest of the panel: the generous, the harsh and the outliers.
• Every mark — every mark given: candidate, criterion and panelist.

Who does what

• Interview Panelist (interview-panelist): a member of a selection committee. Sees the interviews and the panels and marks their own row on each panel.
• Recruitment Coordinator (recruitment-coordinator): runs the panels — scorecards, seating, marks entered for others, overrides with a reason, the ranking and the selection.
• The shipped Human Resources role already covers all of it.