Most corporate AI pilots fail because the organisation and its data were never prepared for them, not because the underlying model is weak. The single most-cited figure of 2025 makes the scale plain: in its report The GenAI Divide: State of AI in Business 2025, MIT’s Project NANDA found that roughly 95% of enterprise generative-AI pilots delivered no measurable profit-and-loss impact (MIT NANDA, 2025). Understanding why is the difference between a demo that impresses and a system that pays for itself.
The failure is upstream of the model
Modern language models are capable. What breaks is everything they depend on. Three failure modes recur.
Messy inputs. Enterprise knowledge is overwhelmingly unstructured and scattered: documents duplicated across tools, outdated versions never retired, no consistent labelling. An AI answering from that corpus will confidently retrieve and repeat the wrong version. The problem is not new; McKinsey’s Global Institute estimated more than a decade ago that knowledge workers already spent about 1.8 hours per day, nearly a fifth of the working week, simply searching for and gathering information (Chui et al., 2012). Pointing a model at that chaos automates the chaos.
No retrieval layer. Without a mechanism to ground answers in the company’s actual documents, a model generates from its training distribution and fills gaps with plausible invention. That invention is what everyone remembers as “the demo that hallucinated in front of a client.” The fix, retrieval-augmented generation, is well established in the research literature (Lewis et al., 2020), but it has to be built, not assumed.
No governance. No access rules, no review of what the AI is permitted to say, and no monitoring after launch. A pilot that works on day one drifts as data, users, and the models themselves change.
The “GenAI Divide”
The MIT NANDA (2025) analysis is useful precisely because it separates the winners from the rest. The organisations capturing value were not the ones with the largest models or budgets; they were the ones that integrated AI into a specific, well-defined workflow and built the surrounding data and feedback loops. The failing majority treated the model as the product and skipped the unglamorous work beneath it. The report frames this gap as a “divide”: a small group crossing into measurable returns while most stall at the pilot stage.
Clean data first, then AI
The practical implication is a sequence, not a shortcut. Before a model is useful on your own knowledge, five things have to be true: the estate is classified (a taxonomy and an access model), cleaned (redundant, obsolete, and trivial files removed), structured (documents turned into retrievable data), connected (the relevant tools integrated), and governed (guardrails and monitoring in place). Only then does a retrieval or assistant layer become reliable. We cover the retrieval mechanism itself in plain language in What is RAG.
What “ready” looks like
A useful test: before you measure the model, ask whether you are actually measuring the model at all. A pilot run on an unclassified, duplicated, ungoverned corpus is measuring the corpus. Fix the inputs, and the same model that “failed” often succeeds, because the constraint was never its reasoning; it was the material it had to reason over.
This sequence is the core of our Knowledge Systems work: we get the inside in order first, build the systems on top, and monitor them after launch so reliability holds as your data and the models evolve.
References
Chui, M., Manyika, J., Bughin, J., Dobbs, R., Roxburgh, C., Sarrazin, H., Sands, G., & Westergren, M. (2012). The social economy: Unlocking value and productivity through social technologies. McKinsey Global Institute. https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/the-social-economy
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. https://arxiv.org/abs/2005.11401
MIT NANDA. (2025). The GenAI divide: State of AI in business 2025 [Report]. MIT Media Lab, Project NANDA. https://www.media.mit.edu/groups/nanda/overview/