The 5% problem: why traditional QA sampling lets the worst calls through

Most call-centre quality programmes in India audit between 5 and 8 percent of calls.

That’s a number everyone in compliance knows. What’s less often said is what it actually means in practice — once you do the arithmetic.

Suppose, conservatively, that 2 percent of your agents’ calls contain a serious compliance issue. A threat-language incident. A missed mandatory disclosure. A calling-hour violation. A coercive reference to the borrower’s family. The kind of call that, if surfaced in an RBI inspection or escalated by a borrower, becomes a problem.

If you sample 5 percent of calls at random, here’s what catches: roughly 1 in 1,000 of those problematic calls. The other 999 happen, get billed for, get paid for, and quietly create regulatory and reputational exposure that surfaces only when something goes wrong months later.

This is not a software problem. It’s a structural one.

The arithmetic nobody runs

The math is simple but rarely written out. A 200-seat collections operation runs roughly 22,000 hours of talk time per month — call it 1.3 million minutes. At a 4-minute average call length, that’s around 330,000 calls a month.

Sample 5 percent and your QA team is reviewing 16,500 calls. That sounds like a lot.

But if the genuinely problematic calls happen at a 2 percent rate, you have ~6,600 of them in that month’s volume. Your sample, drawn at random, will surface roughly 330 of them. The other 6,270 are invisible.

And here’s the part that makes this structural rather than fixable through “better sampling”: the calls most worth catching are exactly the calls least likely to surface. The seriously problematic calls are, by definition, the rare ones. Rare events require disproportionately large samples to detect reliably. A 5 percent sample of a 2 percent event has a wide confidence interval; you might catch 200 of them one month and 450 the next, with no actual change in agent behaviour. You can’t manage what you can’t measure consistently.

What 100 percent coverage changes

The interesting argument isn’t “audit more.” It’s that auditing all of it changes the kind of question you can ask.

At 5 percent coverage, your QA team produces averages. Agent A’s score this month is 87. Agent B’s is 82. You coach Agent B and move on.

At 100 percent coverage, the question shifts. Now you can ask: across every call Agent A took this month, where’s the variance? Is Agent A’s average 87 because they’re consistently at 87 — or because they’re at 95 most of the time and 60 on Thursday afternoons after their fourth difficult borrower? Those two profiles look identical in a sample. They behave very differently in front of an RBI inspector reading a specific call from Thursday afternoon.

You can also start asking population-level questions rather than sample-level ones. Across every collections call we made last week, how many contained any of the seven brand-risk phrases on our scorecard? That’s a question with a real answer at 100 percent. At 5 percent it’s a confidence interval wide enough to drive a truck through.

What the regulator is reading

The RBI’s Fair Practices Code doesn’t prescribe a coverage percentage. It expects “appropriate” review. Lender contracts with collections agencies and BPOs typically codify 5–10 percent random sampling as the floor.

That gap — between “appropriate” review and “5 percent random” — is where most agency-side compliance failures live. When a borrower complaint escalates and the lender’s audit team asks “show me how this call would have been caught,” the agency’s answer is usually some version of “it was outside our sample.” Increasingly, that answer isn’t enough. Lenders’ own audit teams have started asking whether agency QA coverage is structurally capable of catching the patterns the lender is on the hook for.

Four questions to ask this week

If your QA programme audits less than 100 percent of calls — which it almost certainly does — here are four questions worth raising with your team in this week’s review:

1. What’s our actual coverage percentage, calculated as audited minutes divided by total talk-time minutes? (Not “we sample 10 calls per agent per month” — the actual coverage ratio.)
2. How is our sample selected? Random, or weighted toward easier calls, recent calls, or specific agents?
3. For the most serious violations we caught last quarter — threat language, mandatory disclosure misses, calling-hour breaches — what’s the implied rate in our unaudited population, and would we be comfortable defending that estimate to a regulator?
4. If RBI asked us tomorrow for the riskiest 50 calls from last week, could we surface them — or only the 50 we happened to listen to?

The answers won’t be comfortable. They’re not meant to be. They’re meant to surface where the structural gap is, before something rarer than 1-in-1,000 catches you on the wrong day.