FOUR CHECKS TO RUN BEFORE YOU TRUST AN ENGAGEMENT SURVEY RESULT

An engagement score can be arithmetically correct and still describe a small, unrepresentative part of the workforce. Four checks, run before the results circulate, show which parts carry weight.

Emmanuel Duncan · 7 September 2026 · 6 min read

The engagement survey closes, the platform produces a dashboard, and the score arrives at the leadership meeting with a comparison to last year and a set of departmental rankings. Decisions follow. Budgets move. Managers are congratulated or spoken to.

The score itself is rarely wrong in the arithmetic sense. What goes wrong is the distance between the score and the workforce it claims to describe. These four checks close that distance, and all four can be run in an afternoon before the results are circulated.

Check One: Who Answered

Response rate is usually reported as one organisation wide percentage, which is the least informative form it can take.

Break it down by unit, by grade, and by length of service. Then look at where it collapses.

The pattern I see most often is that head office and professional grades respond at high rates while field operations, shift workers and junior grades respond at much lower ones. When that happens, the organisation wide score is largely a measure of how head office feels, presented as though it described everyone. A rise of two points can be produced entirely by a change in who bothered to answer.

The second pattern worth checking is response rate against last year's score by unit. If the units that scored badly last year responded less this year, the improvement in the headline is partly people declining to say it again.

Where a unit's response rate is under about half, I would report its result with the rate printed beside it and resist ranking it against units with better coverage.

Check Two: Whether People Believed It Was Anonymous

A survey is only measuring opinion if people felt safe giving one. Otherwise it is measuring how well they judged what was expected.

There are signals in the data. Scores clustering tightly at the upper end with very few low responses is one, particularly in a unit where you already know there are problems. Free text comments that are short, positive and generic is another. A unit of eleven people where all eleven responded and every score is in the top two categories deserves scepticism rather than a prize.

There are also structural causes worth checking before the next round. Did the demographic questions, taken together, make individuals identifiable in small teams. Was the survey link distributed by line managers. Was it completed on shared devices, or in a room with a supervisor present. Each of those changes what people are willing to write.

Check Three: What the Questions Actually Asked

Read the instrument, not just the output. Three problems recur.

Double barrelled items. "My manager supports me and gives me clear direction" cannot be answered by someone whose manager does one and not the other, and the resulting score cannot be interpreted.

Items about things the respondent cannot see. Asking frontline staff to rate the quality of strategic direction produces a number, but the number reflects distance from the centre more than it reflects strategy.

Items that changed since last year. Platforms revise wording between releases, and a comparison across a wording change is not a comparison. Before showing any year on year movement, confirm the item text is identical. If it changed, show this year's result on its own.

There is also a translation question that applies directly in Ghana and across the region. A survey written in English and answered by people who work in Twi, Ga, Ewe or Dagbani all day is partly a reading test. Where that is the case, the aggregate score is carrying a language effect that nobody has measured, and the honest response is either a translated instrument or a narrower set of very plainly worded items.

Check Four: The Size of the Difference Against the Noise in It

Departmental rankings are the part of the report that gets acted on hardest and holds up worst.

Two things are usually missing. The first is the number of respondents behind each unit score. A team of six with four responses can move ten points because one person changed their mind, and that movement means nothing. The second is any sense of what an ordinary amount of variation looks like. Without it, every difference reads as a real difference.

The simple discipline is to print the response count beside every unit score, suppress any unit under about eight respondents, and set a threshold below which differences are not discussed. A rough working rule for small units is to treat movements under about five points as noise unless something specific happened in that team.

Then look for the pattern that actually matters, which is not the ranking but the concentration. If the same two units sit at the bottom for three years running, that is a finding. If the bottom two change every year, you are ranking noise and the exercise is doing harm, because managers are being praised and criticised for something they did not cause.

What to Do With the Free Text

The comments usually contain more than the scores, and they are usually the part that gets read least carefully because there are too many of them.

A reasonable approach without any tooling is to read all of them once without coding anything, then define six or seven categories that emerged from the reading, then code every comment into one. That order matters. Categories chosen in advance find what you expected.

Report the categories by frequency and quote sparingly, with any detail that could identify the writer removed. In small organisations people recognise each other's phrasing, and one identifiable quote will cost you the next survey's response rate.

What to Do Next

Run the four checks before results leave the HR office, and write a one page note that says who answered, what the coverage gaps are, which comparisons are safe, and which unit results should not be ranked.

Send that page with the dashboard. It is not a defensive document. It tells the leadership team which parts of the result carry weight, and it stops the meeting spending its time on a two point movement that came from a change in response rate.

Then choose the smaller number of findings you are confident in, and act on those. A survey that produced three real changes is worth running again. A survey that produced a ranking, a set of uncomfortable conversations and no change is the reason so many organisations find their response rate falling every year.

Emmanuel Duncan is the founder of Aletheia Lunara Consult, a professional education and consulting practice working with organisations across Ghana and beyond on people analytics, workforce research and organisational capability.