What should a staffing vendor scorecard measure?

Fill rate is the number every programme starts with and the one that distorts behaviour fastest. A supplier optimising for it sends more profiles, because more submissions raise the odds that one lands, and your reviewers absorb the cost of reading them. A supplier who sends two strong people and loses to a faster competitor scores badly on a measure that says nothing about the quality of what they sent. Worse, fill rate depends heavily on which roles a supplier receives: a panel member handed the difficult categories will trail one handed the easy ones, and the number will look like a performance difference when it is a workload difference.

Response rate tells you whether a supplier is actually working your roles or has quietly deprioritised you, which is the single most useful early signal and the one most programmes notice last. Submission quality — the share of profiles that get past screening — tells you whether they understand the brief. Time to first submission tells you whether they are fast enough to matter on roles that move quickly. Cost against the card tells you whether their rates hold or drift. Four measures fit on a page, can each be traced back to something the supplier decided, and are few enough that a review conversation stays about the work rather than the spreadsheet.

The comparison that changes decisions is not supplier against supplier but supplier against what you produce yourself. If your own sourcing fills a category faster and at lower cost, the honest conclusion is that you are buying convenience rather than capability there, and that is a legitimate thing to buy as long as you know you are buying it. If a supplier consistently beats your direct pipeline on a scarce specialism, that is where the panel is genuinely earning its fee, and where you should be sending more work rather than negotiating harder. Holding both sides on the same measures is what makes that visible, and it is why the comparison belongs in the same record.

Three things, all of which will otherwise train the panel to behave worse. Do not punish a supplier for declining a role they cannot cover, because a supplier who declines honestly is more useful than one who submits hopefully. Do not punish a low submission count that comes with a high pass rate, since that is the behaviour you want. Do not compare across categories without saying so, because a supplier working scarce specialisms and one working volume categories are not doing the same job. Scorecards that ignore these produce a panel that games the measures, and the gaming shows up as inflated submission volumes long before anyone connects it to the scoring.

Often enough that a trend is visible and rarely enough that the numbers mean something. A quarterly review against a full quarter of data is common, with the measures shared with the supplier beforehand rather than presented as a verdict in the room. The people who should be there are whoever owns the panel commercially and somebody who has actually reviewed the supplier's submissions, because the second person is who can explain what the quality number means in practice. A review that is only a report reading tends to change nothing, and suppliers work out quickly which programmes act on their scores and which merely publish them.

Say so, specifically, and early. Most poor scores have a cause a supplier can address — a brief they misread, a category they are not staffed for, a rate that makes the role uncompetitive for them, or a feedback loop so slow they cannot learn from rejections. Programmes that release volume and return silence train their better suppliers to deprioritise them, and that appears months later as falling response rates from exactly the suppliers you wanted. Removal from the panel is a real option and should follow a conversation and a stated period to improve, not a report. Applying that sequence consistently is also what makes removal defensible when it happens.