The story sounds almost absurd on its face: one of the Big Four consulting firms publishes a report on how companies are adopting AI — then quietly retracts it because the AI apparently made things up.
That's what happened with KPMG. According to TechCrunch, the firm pulled a report on AI usage after concerns surfaced that the findings may have contained hallucinated data. The source of intelligence about artificial intelligence? Artificial intelligence itself.
The irony lands hard. But the real story here isn't KPMG's embarrassment. It's the quiet, uncomfortable truth that a lot of organizations — from global consultancies to mid-sized companies in Bogotá or Monterrey — are making the same mistake at a smaller scale, every single day.
Hallucination is not a bug that slipped past engineers. It's a structural feature of how large language models work. These systems generate text that is statistically plausible, not factually verified. When you ask one to summarize industry trends, estimate adoption rates, or describe what competitors are doing, it will produce something that reads like authoritative analysis. It may even cite sources. Those sources may not exist.
This is manageable when you're drafting a customer email or brainstorming campaign headlines. It becomes a serious governance problem when the output is a report signed by a firm that advises governments and multinationals on critical decisions.
KPMG had the institutional maturity to catch the error and pull the document. That's actually the responsible move. The harder question is: how many organizations don't catch it?
There's a particular trap worth naming here. AI tools are especially unreliable when the topic is AI itself.
Why? Because the field moves faster than training data. Statistics about AI adoption from six months ago can already be wrong by the time a model was trained on them. New models, new companies, new regulatory frameworks, new investment rounds — the landscape shifts weekly. An LLM trained on data with any meaningful cutoff date is structurally unable to give you accurate, current information about the AI market.
When a business leader in Latin America asks an AI assistant "what percentage of companies in my sector are using AI agents?" — the answer they receive may be plausible, internally consistent, and completely fabricated. No disclaimer will appear unless you've specifically prompted for one.
This isn't a reason to abandon AI tools. It's a reason to be precise about what they're good at and what they're not.
Most mid-sized companies don't publish white papers. But they do use AI in ways that carry similar risks:
Market research and competitive analysis. AI is increasingly being used to generate summaries of competitors, industry conditions, and customer trends. If those summaries are feeding strategy decks or board presentations, the hallucination risk is the same as in KPMG's case — just smaller in scale and harder to audit.
Internal reporting. Finance teams and operations leaders are starting to use AI to draft weekly or monthly reports from raw data. The risk here is usually lower if the AI is reading structured data through a verified integration. The risk spikes when the AI is asked to "add context" or "explain what this means for the industry" — that's where confabulation creeps in.
Vendor and technology evaluation. Decision-makers who use AI chatbots to evaluate AI vendors are essentially asking a competitor's ecosystem to assess itself. The outputs will tend to reflect the model's training biases, not objective market reality.
Regulatory and compliance summaries. In Colombia, Mexico, and Argentina, the regulatory environment around data protection, labor, and financial services is specific and evolving. Generic AI summaries of regulatory requirements are one of the highest-risk use cases in any business context.
The KPMG incident points toward a principle that more organizations need to operationalize: AI should draft, not certify.
In practice, that means:
Separate generation from validation. Let the AI produce the first draft — the outline, the summary, the synthesis. Require a qualified human to validate every factual claim before that content is used to inform a decision, published, or shared with stakeholders.
Anchor AI outputs to verified data sources. When AI is used for reporting or analysis, the most reliable deployments are those where the model has access only to structured, internal, verified data — not an open internet search or its own parametric memory. A dashboard that summarizes your own sales pipeline is far more trustworthy than a chatbot summarizing "trends in your industry."
Build explicit review checkpoints. Any workflow where AI output influences a business decision — a pricing call, a hiring decision, a market entry strategy — should include a documented human review step. This isn't bureaucracy. It's the minimum viable control for managing AI risk responsibly.
Be especially skeptical about AI on AI. When AI is generating content about AI adoption, AI capabilities, or AI vendors, treat that output as a starting hypothesis, not a finding. Cross-check against primary sources: actual vendor documentation, original research publications, or industry analysts who publish primary data with methodology notes.
There's a reason KPMG's situation matters beyond the headlines: it exposes that even organizations with significant resources, technical capability, and brand equity on the line can miss this. The pressure to publish fast, leverage AI for efficiency, and demonstrate thought leadership in a crowded space creates exactly the conditions where verification gets skipped.
For business leaders in Latin America, the dynamics are similar but the stakes can be even more specific. Decisions about automation investments, headcount, and technology platforms are being made in a market where the data is often thinner, the infrastructure is more varied, and the margin for error is narrower. Relying on AI to fill those information gaps without rigorous validation isn't efficiency — it's a liability.
The question for any organization right now isn't whether to use AI. That decision is already made. The question is whether you have the internal process discipline to know when AI output ends and verified knowledge begins.
KPMG found out the hard way that those two things are not the same. The good news is that building the distinction into your operations is straightforward — it just requires someone in your organization to own it.
At Xenturia, we help mid-sized companies in Latin America design AI workflows with the right human checkpoints built in from the start — so AI accelerates decisions without becoming the decision-maker. If you're evaluating how to govern AI use in your operations, let's talk.
Schedule a free consultation with our team and discover how AI can transform your operations.
Schedule a consultation
Strategic AIAIAnthropic is pushing back after the government pulled its most powerful model. Here's the regulatory risk LATAM businesses haven't priced into their AI strategy.
Strategic AIAIPinecone now connects AI agents directly to enterprise data in Microsoft OneLake. What this means for companies on Microsoft's stack — and what to do first.
Strategic AIAIOpendoor shutting its India operations is more than a cost decision. It's a signal about where AI is replacing the outsourcing playbook—and what that means for LATAM leaders.
