The Real Test of AI Auditing Comes After the Question
Key takeaways
- Better audit questions alone do not establish better audit performance.
- Professional skepticism requires checking evidence and revising judgments as well as challenging explanations.
- AI audit tools should be evaluated on meaningful findings, false alarms, and total review time.
- Auditors need to explain why they pursued or dismissed an AI-generated concern.
If AI asks a sharper audit question than an accountant, where does professional expertise show up next? KPMG’s Project Exponential and its AI audit experiments bring that question into focus. Anyone relying on a company’s financial statements has a stake in how a clever question becomes a trustworthy conclusion.
What, exactly, did the AI do better?
Before accepting a claim that AI outperformed human auditors, pin down the task. Generating questions, spotting overlooked problems, and reaching sound conclusions are separate achievements. Each needs its own evidence.
Suppose an AI flags 100 transactions as suspicious. That number tells us very little by itself. How many contained real problems? How many important problems escaped the list entirely?
The same scrutiny should apply to Project Exponential. What information did the system receive? Did the human auditors work under comparable conditions? Who judged the results, and against what criteria?
An evaluation of question quality supports a conclusion about question quality. Extending that conclusion to the whole audit requires evidence about the work that follows.
That distinction matters because a strong question can sound like a breakthrough before anyone has checked the answer.
Skepticism continues after the question
In auditing, professional skepticism means approaching explanations with a questioning mind and examining the evidence critically. It leaves room for an explanation to hold up under scrutiny.
Consider a hypothetical company whose revenue jumps in the final weeks of the year. Management attributes the increase to a large new customer contract.
A useful audit question gets specific. What does the contract require? When were the goods delivered or the services performed? Can the customer cancel or return the purchase?
Those questions give the auditor something concrete to investigate. The next step is to compare the contract with delivery records and other transaction evidence, then assess whether the explanation holds together.
New evidence may strengthen the concern. It may also resolve it. An auditor needs to be willing to revise the initial judgment in either direction.
AI could help by proposing questions that make this investigation more focused. The professional work continues through verification and judgment revision. Asking a penetrating question is an opening move.
Every red flag creates work
An AI audit agent could examine transaction data, identify unusual patterns, and suggest follow-up checks. Here, “agent” means a system configured to work through multiple steps toward a defined goal.
The appeal is easy to see. So is the potential backlog.
Every flagged transaction needs attention. If the system repeatedly treats ordinary activity as suspicious, someone must spend time clearing those false alarms. If it accepts plausible explanations too readily, meaningful problems may slip through.
A useful evaluation therefore needs to answer four questions:
- How many significant problems did the system identify?
- How often did it flag legitimate activity incorrectly?
- Could reviewers trace its reasoning back to the underlying records?
- How much human time did it take to reach a conclusion?
That last measure deserves particular attention. Questions can be cheap to generate and expensive to answer.
If an AI produces a list in seconds but adds hours of unnecessary review, its speed tells us little about the overall benefit. Measure the full process, from the first flag to the completed assessment.
An endlessly suspicious assistant can keep a team very busy. Whether it makes the audit better is another question.
Expertise means defending the decision
A confidently worded AI concern can be persuasive. Its evidence still needs checking.
Return to the year-end revenue example. An auditor should be able to identify the contract term that raised concern, explain which records were examined, and show why the concern remained—or why it was resolved.
“AI flagged it” explains how an item entered the queue. A defensible conclusion needs a fuller account.
I would put more weight on the ability to explain a judgment when evaluating auditors who use these systems. That includes explaining why some suspicions were dropped. Resolving a concern with good evidence is valuable audit work, even when it produces no dramatic finding.
The workflow also needs clear responsibility. Someone must review the AI’s suggestions, decide when further investigation is warranted, and approve the final conclusion. A record that says “human reviewed” should document what that review actually involved.
KPMG’s Project Exponential invites scrutiny of that entire chain, from the first question to the final judgment. As AI gets better at raising doubts, auditors need to make the evidence behind their decisions more visible. Trust rests on being able to show how a question was resolved.
Comments
Loading comments...