The 5% Illusion: Why Your QA Numbers Describe a Week You Never Heard

The number nobody calculates

Take the calls your agents handled last week. Now take the calls your QA team genuinely reviewed — listened to end to end, scored, and wrote notes against. Divide the second by the first.

On most busy Indian floors the answer lands between 2% and 10%. A two-person QA team on a floor doing 8,000 calls a week might get through forty of them properly. That is half a percent.

Everything your organisation believes about its call quality is extrapolated from that sliver.

The consequence is not that quality is unknown. That would be a manageable problem — you can budget for an unknown. The consequence is worse: quality is confidently misreported. A supervisor holding a 5% sample has a firm, specific, well-argued opinion about the floor. The opinion is built on a fragment of the week and silence about the rest, and nothing in the reporting makes that visible.

The call that triggers a complaint is almost never the one you sampled.

What lives in the unheard 95% is not evenly distributed either. It is the abusive collection call at six on a Friday evening. The agent who has quietly stopped verifying identity. The customer who threatened to involve a lawyer and got a scripted apology. The rep whose close rate has been sliding for three weeks and whose manager will find out at the quarterly review.

None of it is hidden deliberately. It is simply never heard.

Why more people has never fixed it

The instinct is to solve a coverage problem by hiring reviewers. It does not work, and the reason is structural rather than a question of effort or discipline.

Reviewing a call properly takes longer than the call itself once you include scoring and note-taking. So QA capacity scales linearly with auditor headcount, while call volume scales with the business. A floor that doubles its agents does not double its QA team — QA is a cost centre, so it stays roughly the same size and reviews a smaller slice of a bigger number.

The percentage covered goes down as the company succeeds. Manual QA does not merely fail to reach 100%; it drifts further from it every quarter. And auditor headcount is the first line item cut when margins tighten, which means coverage falls fastest exactly when risk is highest.

The sampling itself compounds the problem. Those forty calls are not selected scientifically. They are selected for convenience — whoever the reviewer had time for, whatever surfaced through a complaint, whatever was easy to find in the recorder. Rare events do not appear in convenience samples. The riskiest calls on your floor are, by definition, rare.

The remedies that trade one gap for another

Every existing answer to this moves the problem rather than closing it.

Speech analytics platforms raise coverage dramatically and then hand supervisors a flood of unverified keyword alerts. Coverage goes up; trust goes down. Within two months the alerts are being cleared in bulk without being read, and the floor is back to a sample — just a sample of alerts instead of a sample of calls.

Outsourced QA raises review quality and is priced per audited call, which structurally guarantees it stays a sample. You are buying a better 5%.

Supervisor-led side-by-sides produce excellent coaching for the handful of calls they touch, and no coverage at all.

Each of these is a reasonable purchase. None of them changes the denominator.

What we changed: coverage as architecture, not as a setting

The design decision underneath The Heartbeat is a narrow one, and everything else follows from it: 100% of calls are processed as the default state of the system. Coverage is not a configuration option a customer turns on, and not a target the operations team works towards. It is what the pipeline does when nobody intervenes.

Recordings land in the customer’s own storage — S3, Google Drive, Dropbox, Box, GCP Bucket or Wasabi — or arrive by direct upload. Each file is probed and transcoded on arrival, then every eligible call is submitted to a GPU queue for transcription, speaker separation and scoring against a taxonomy of 600+ risk behaviours spanning sentiment, emotion, conversation dynamics, script adherence, compliance risk and brand risk.

There is no sampling step anywhere in that path. Not a disabled one, not a default-off one — there is no stage that selects a subset. This matters more than it sounds. A sampling step that exists as a switch is a sampling step that gets flipped during a backlog, and coverage silently becomes a sample again at precisely the moment the floor is busiest.

Nothing drops out unrecorded

The honest version of “100%” has to account for the calls that do not get processed, because in any real system there are some.

There is exactly one gate in the path: the commercial one. If a workspace has insufficient prepaid balance, the job does not run. But it is not silently skipped — it is written to a failed-jobs table as a row you can query, with the call, the reason and the timestamp.

That distinction is the whole point. A system that quietly drops calls under load produces the same confident misreporting as a 5% sample, just with better graphics. A call is either analysed, or it is on a list of calls that were not. There is no third category.

Where sampling exists at all, it exists as a cost control the customer deliberately opts into — a per-workspace processing limit on minimum call duration, maximum call duration and percentage processed, so that a bulk historical backfill cannot burn a month of minutes overnight. That is sampling as a budget decision, made explicitly, visible in the workspace configuration. It is not sampling as an invisible default.

What full coverage actually buys

The obvious benefit is the one nobody argues with: you stop missing things. The more interesting benefits are second-order.

Rare events become findable. A behaviour that occurs on 0.4% of calls is invisible in a 5% sample and unambiguous across 100%. Most compliance exposure lives in exactly that band — frequent enough to be systemic, rare enough never to be sampled.

Trends stop being anecdotes. Four consecutive weeks of complete data lets you say an agent’s handling score fell, by how much, and on which behaviours. Four consecutive weeks of 5% samples lets you say you have a feeling.

The evidence trail becomes real. When a regulator or an internal audit asks what you did about a category of risk, “we sample and coach” is an answer that invites a follow-up question. “Every call in the period was analysed, here are the flagged ones, here is who was assigned each, here is the date it was closed” is an answer that ends the conversation.

Every flag is verified before it ships. This is the part that separates full coverage from a firehose. AI processes the volume; a trained analyst reviews the flags before anything reaches a customer. Coverage without verification is what turned the last generation of speech analytics into noise, and it is the reason a human sits between the model and the report.

What lands on Monday

The deliverable is a prioritised PDF at 11am IST every Monday, inside a 48-hour SLA from batch close. It runs to five pages, and the last one is not a chart — it is an assigned task list, with an owner, the behaviour to coach and a due date.

Two pages of it are deliberately kept apart: the analyst’s five PASS / MINOR / FAIL quality perspectives, and the platform’s arithmetic — opening, handling, closing, call score, silence. They are shown side by side and never blended into a single number, so a reader can tell whether what they are looking at is a coaching problem or a compliance one. Blending them produces a tidier headline and destroys the only thing the reader needed.

A real week’s output for a single workspace: 88 individual call analyses, each traceable back to the segment and the words that produced its score.

The web application is still there — waveform playback, click-to-jump transcripts, filter builders, exports — for teams that want to work the data directly. The honest framing is that there is a report for the people who need an answer, and an app for the people who need to dig. The report is the default, because the person who has to act on Monday morning is not going to log into anything.

The economics, stated the way we can defend them

We deliberately do not lead with a savings figure. Cost-avoidance models in this category rest on a counterfactual — money the client would supposedly have spent — and a finance director is right to interrogate that.

Here is the version that survives interrogation, because it is arithmetic rather than assumption:

At roughly 16,000 calls a month, reviewing every call to a proper standard by hand would take about ten full-time auditors.

Nobody budgets ten auditors. That is exactly the point. The choice on the table has never been “pay ten people or pay us” — it has been “review 5% or review everything”, and until the processing step was decoupled from headcount, only one of those was purchasable at any price.

That is the gap this closes. Not cost. Coverage.

Where to start

The setup is a storage connection, not a project: point us at the bucket or drive where your recordings already live, and the pipeline reads from it. Most customers are running inside about twenty minutes, and the first report follows the next Monday.

If you want to see how the coverage works in practice, book a demo and we will walk you through the platform end to end.

---

Baseline note: this case study uses 5% as the manual-review baseline throughout. Our own documentation cites figures below 0.5% for large floors. We publish the higher, more conservative number because it is the one a working QA manager will recognise as their own — and the argument does not need the more dramatic version.