TWELVE QUESTIONS TO ASK A VENDOR SELLING AN AI RECRUITMENT TOOL

Twelve questions to put to an AI recruitment vendor in writing before signature, and three answers that should end the conversation.

Emmanuel Duncan · 7 September 2026 · 6 min read

The demonstration is usually excellent. Two thousand applications reduced to a shortlist of twenty in under a minute, a score against every candidate, a dashboard showing time to hire falling.

The questions below are the ones I put to vendors on behalf of clients. They are not hostile questions. A good product has good answers to most of them, and the ones it cannot answer are worth knowing about before signature rather than after. Send them in writing and ask for written replies, because the answers you get in a meeting are not the answers you can rely on later.

On What the Tool Is Actually Doing

1. What exactly does the score represent. Ask for it in one sentence a hiring manager could repeat. A tool that predicts fit for the role, a tool that predicts likelihood of passing your interview, and a tool that predicts similarity to your current staff are three very different products, and the third carries the most risk while being the easiest to build.

2. What was the model trained on. Whose applications, whose hiring outcomes, from which countries, over what years. If the answer is a general dataset with no relation to your labour market, the ranking is importing someone else's hiring pattern into your organisation.

3. Which fields does it read. Get the list. If it reads the whole document, it reads the name, the address, the school attended, the years of graduation, the languages, and the gaps. Each of those carries demographic information whether or not anyone intended it to.

4. Can we see why a particular candidate scored what they scored. Ask them to do it live on a real application. An explanation such as "strong keyword match" is not an explanation. You want something a rejected applicant could be told.

On Evidence That It Works

5. What is the accuracy claim, and against what. A tool that agrees with your existing recruiters ninety percent of the time is claiming to reproduce your current process, including the parts of it you wanted to fix.

6. Has it been tested for disparate impact, by whom, and can we see the report. The relevant test is whether selection rates differ across groups. Ask which groups were examined. Ask whether the test was done by the vendor or by someone independent.

7. Has it been evaluated on a workforce like ours. For an organisation in Ghana this is a real question, not a formality. Qualification names, institution names, national service, chieftaincy titles, and the shape of a typical CV all differ from the data most of these systems were built on. Ask for a case study from the region, and treat its absence as a finding rather than a disqualification.

8. What happens to candidates the model has no basis for scoring. Career changers, people returning after a break, applicants from institutions not in the training data. Ask what the tool does with them, because the honest answer is usually that it scores them low.

On Control and Consequences

9. Who makes the final decision, and can the ranking be overridden. Ask to see the interface where a recruiter disagrees with the tool. If overriding is possible but effortful, in practice it will not happen.

10. What do we tell a rejected applicant who asks how their application was assessed. You need an answer you are willing to give in writing. If the vendor cannot supply one, you have bought something you cannot explain, and the first person to ask will be a person you did not want to have this conversation with.

11. What data leaves our environment, where is it stored, and for how long. Applications contain names, addresses, dates of birth, identification numbers and employment history. Ask where the servers are, whether candidate data is used to improve the model, and how to have it deleted. Ghana's Data Protection Act sets obligations here that sit with you as the data controller, and they do not transfer to the vendor because the processing happens on their platform.

12. What happens when we leave. Can you export the historical applications, the scores, and the decisions. A recruitment history locked inside a platform is a real cost at renewal, and it is the point at which pricing conversations become one sided.

The Three Answers That Should Stop the Conversation

Some responses are worth treating as decisive.

A refusal to explain an individual score on commercial confidentiality grounds. You will be asked to explain it eventually, and you will not be able to.

A claim that the tool cannot discriminate because it does not see protected characteristics. It sees hundreds of things that correlate with them. A vendor who makes this argument either has not examined the question or is hoping you will not.

An unwillingness to run a pilot against your own historical applications. That test costs them a week and tells you more than every other answer combined.

The Pilot Worth Running Before You Buy

Take a set of applications from a role you filled twelve months ago, with the outcomes known and removed. Run them through the tool.

Three things to look at. Whether the people you hired and kept appear near the top. Whether the ranking differs across groups in a way you cannot account for. And whether any of the strong performers were scored low, because those are the applications that tell you what the model is missing.

Have someone outside the recruitment team read the result. The people who ran the original process have an interest in the answer.

What This Is Really About

The question underneath all twelve is who answers for the decision. A tool that speeds up screening while leaving the judgement, the explanation and the accountability with a named person is a useful piece of software. A tool that takes the judgement and returns a number nobody can account for has moved the risk onto you while keeping the margin.

Ask the twelve questions in writing. The replies become part of the contract file, and they are what you will reach for the first time somebody asks how the shortlist was made.

Emmanuel Duncan is the founder of Aletheia Lunara Consult, a professional education and consulting practice working with organisations across Ghana and beyond on people analytics, workforce research and organisational capability.