Five questions a CCO should ask their QA team this quarter

The quarterly QA review is one of the most reassuring meetings a compliance head sits in, and one of the least useful.

The deck is clean. Audit pass rates are up two points. The top violation categories are the same three as last quarter. There is an agent leaderboard, a red-amber-green heat map, and a slide that says coverage is “on track.” Everyone nods. Nothing on the floor changes.

The problem isn’t the QA team. It’s that a standard QA report answers the question what did we measure — and a Chief Compliance Officer needs the answer to a different question: what did our floor actually do, and would we know if the worst of it went past us? Those are not the same question, and the gap between them is exactly where regulatory exposure lives.

Here are five questions to put to your QA team this quarter. None of them is a gotcha. Each one surfaces something a pass-rate slide is structurally unable to show.

1. What share of last month’s calls did we audit — and how was that sample chosen?

Start here, because every number downstream depends on it. A 94% pass rate means one thing if it’s drawn from 100% of calls and something very different if it’s drawn from 6% pulled by a process nobody can quite describe.

A weak answer sounds like: “We sample a representative set.” Press it. Representative how — by agent, by campaign, by time of day, by outcome? A sample skewed toward calls that are easy to listen to, or clustered in the calm mid-morning hours, is not representative of the 7:15 PM call that generates the complaint. If the honest answer is “the reviewers pick,” you don’t have a sample. You have a convenience set. And as this newsletter argued in its very first issue, at single-digit coverage the rarest and most serious calls are precisely the ones sampling is least likely to surface.

2. What did we catch in that sample — and what does it imply about the calls we didn’t hear?

A sample isn’t an answer in itself; it’s an estimate of a population you didn’t listen to. If you audited 6% of calls and found a given violation in a handful of them, the useful question is not “how many did we catch” but “what does that rate imply for the 94% we never opened?”

A weak answer stops at the count. “We found four coercive calls.” Four out of what, projected across what? The point of a sample is to reason about the whole. If your QA team can’t take you from “what we saw” to “what the floor probably did,” the report is a tally, not an audit — and a tally tells a regulator nothing about the calls that never reached a reviewer’s headphones.

3. Which of last quarter’s borrower complaints would our QA process have caught first — and which did it miss?

This is the single most clarifying question in the meeting, and the most uncomfortable. Take three real complaints that escalated last quarter. Pull the calls behind them. Ask: had we listened to these before the borrower did, would our process have flagged them?

A weak answer is defensive — “that call wasn’t in the sample” or “the scorecard doesn’t have a field for that.” Both are true and both are the problem. A complaint that your own QA process couldn’t have caught is a map of your blind spot, drawn for you at someone else’s expense. The threats that don’t sound like threats — the family-implication line, the asset-loss claim, the drift into “this is now a legal matter” that we covered last week — are exactly the calls that pass a scorecard and land as a complaint. If your process missed them, you want to know this quarter, not in an inspection.

4. What do our best agents do that our worst ones don’t — in our own data?

Every floor has a folk theory of its top performers: they’re firmer, they’re warmer, they build rapport. Ask your QA team to prove or disprove it from the recordings. What, specifically and observably, separates the agents who recover cleanly from the ones who generate disputes?

A weak answer is a personality sketch. A strong one is behavioural and copyable: the top agents complete the mandatory opening disclosures before anything else; they acknowledge a dispute out loud instead of talking over it; they never let “legal” enter a sentence loosely. That answer is worth more than any leaderboard, because it turns your best agents into a training script instead of a mystery.

5. If a regulator asked for the riskiest fifty calls from last week, could we produce them by Friday?

This is the operational stress test, and after 1 July it is no longer hypothetical. The RBI permitted contact window for recovery agents is 8 AM to 7 PM, and the draft recovery-agent framework — proposed to take effect 1 July 2026 — expects lenders to keep records of the number and timing of calls, with mandatory recording. The recordings are meant to exist. The question is whether you can find the right ones.

A weak answer is a pause. “We could pull last week’s calls…” — but which fifty are the riskiest? If the only way to surface them is for a reviewer to listen to everything, you can’t do it by Friday, and you can’t do it under inspection pressure. The ability to rank your own week by risk, on demand, is fast becoming the difference between an agency a lender keeps and one it quietly de-lists.

Why these questions are migrating out of the QA meeting

Here’s what makes this quarter different from last. Increasingly, these are not questions a CCO asks internally — they are the questions a sophisticated lender’s audit team asks its agencies and BPOs before renewing a contract. Coverage rate. Sample method. Complaint-catch rate. Retrieval time. The due-diligence questionnaire is starting to read like this list. A compliance head who can answer all five in the quarterly review is, without changing anything else, ready for the vendor audit that’s coming.

Take this into your next quarterly review

Don’t circulate these as a memo. Put them on the agenda, in order, and hold the room to each one before moving on:

1. Coverage & method — what share did we audit, and how was it chosen?
2. Sample → population — what does what we caught imply about what we didn’t hear?
3. Complaint back-test — which of last quarter’s complaints would we have caught first?
4. Best-vs-worst, from data — what observably separates our strongest agents?
5. Retrieval under pressure — could we surface the riskiest fifty calls by Friday?

A QA report tells you what your team looked at. These five questions tell you what your floor did — and whether you’d know before someone outside the building did.