Despite a myriad of headlines about AI breakthroughs, the question lingering beneath the surface of many business conversations is often far more grounded: Is the technology ready for deployment at scale, and what type of returns on investment should one expect?
At a recent summit hosted by the W. Edwards Deming Center and the AI in Business Initiative, researchers, business executives, and practitioners answered that question, comparing notes on what AI is and isn’t delivering so far.
What emerged was a clearer picture of why AI’s returns vary so widely from one organization to the next. The technology is advancing quickly, but the conditions required to translate its capabilities into measurable productivity—solid data foundations, well-designed workflows, and human judgment—aren’t evenly in place yet. In other words, the challenge isn’t whether AI can create value; it’s whether organizations are able to unlock it.
Five insights stood out, offering a more nuanced understanding of where AI’s ROI is materializing today and what it will take for organizations to fully benefit.
1. AI Doesn’t Necessarily Level the Playing Field
David Holtz, assistant professor of business in the School’s Decision, Risk, and Operations Division, described what sounds like a classic AI success story: entrepreneurs in Kenya receiving access to an AI “business mentor.”
The tool lived on WhatsApp, made practical suggestions, and offered a steady stream of advice on prices, customers, operations, and any other advice a small business owner might want.
"Entrepreneurs are interesting to study when it comes to AI because they are juggling many distinct tasks at the same time," Holtz said.
But the outcome wasn’t the rising tide one might expect.
Instead, the AI tool widened performance gaps. High-performing entrepreneurs reported making nuanced adjustments that were tailored to their specific business. In contrast, lower-performing entrepreneurs reported implementing the AI’s broader, less-personalized suggestions, often slipping backward as a result.
The finding contradicted earlier research showing that AI tends to lift lower performers. Instead, in more complex settings, the technology rewards judgment, rather than supplying it. Business leaders who already know how to evaluate advice become more effective, while those who don’t see the cracks in their decision-making expand.
2. AI Can Supercharge Revenue in the Right Workflow
Dante Donati, an assistant professor in the School’s Marketing Division, helped clarify why some AI tools deliver enormous returns while others barely move the needle.
Drawing on large-scale experiments with a global online retailer, Donati showed that AI’s turbocharged outcomes tend to emerge in very specific spaces, moments where customers face frictions that humans or traditional systems haven’t been able to address well.
Take multilingual customer service. When the retailer deployed an AI-powered chatbot capable of answering questions fluidly in dozens of languages, sales jumped between 11 and 16 percent. Chatbots answered user questions instantly, refined queries helped them find better product matches, and automated descriptions made listings richer, leading to conversion rates rising as high as 22 percent.
Other workflows produced smaller but still meaningful effects. Search refinement nudged shoppers toward what they meant and not necessarily what they typed. Product descriptions expanded context in ways human teams struggled to scale. AI often creates value by smoothing out the subtle frictions of online shopping, according to Donati.
When the friction was minimal or the model poorly optimized, however, AI added little, occasionally even reducing performance. The lesson was practical: ROI depends on aligning AI with the right problem, not applying it everywhere.
3. AI Agents Are Advancing, But Today’s Systems Are Still Fragile
Real promise lies in the emerging world of AI agents—tools that not only generate text but can take actions inside enterprise software—according to research by Hongseok Namkoong, also an assistant professor in the School’s Decision, Risk, and Operations Division.
During a panel moderated by Omar Besbes, the Vikram S. Pandit Professor of Business, Namkoong noted that agents can manipulate spreadsheets, perform analyses, initiate transactions, and respond dynamically to instructions. But Namkoong cautioned that this capability often outpaces the underlying systems supporting it.
Many organizations still operate with fragmented data, legacy Enterprise Resource Planning systems, inconsistent validations, and ad hoc integrations. These issues don’t just slow agent adoption, but also shape whether agents behave reliably at all. A system that is already fragile does not suddenly become more robust because a sophisticated AI layer sits on top of it.
“Data is the very infrastructure on which all of these technologies are built up,” Namkoong said. “And much like roads, bridges and electricity grids, once you have this setup, it's pretty hard to actually overhaul it.”
AI agents will likely play a meaningful role in the years ahead, according to Namkoong, but organizations must strengthen their operational foundations to adopt them responsibly. The technology is moving quickly, but deployment readiness is not.
4. Leadership, Not Tooling, Determines Whether AI Pays Off
Heather Bellini ‘97, president and CFO of InvestCloud, noted that while many companies have embraced horizontal tools—enterprise search, note-taking apps, copilots—the bigger gains come when AI is embedded deeply inside specific workflows such as accounting, engineering, and portfolio management. Embedding AI this way, however, requires teams to rethink established habits and reshape processes.
That change, she noted, rarely happens on its own. "2026 is going to be the year of reckoning for a lot of CEOs,” Bellini said during a panel discussion moderated by David Niles ‘98, CEO of Council Advisors and Advisory Board Chairman of the Deming Center.
Alex Lavoie, Co-COO at Via, echoed the point. In industries like transit, operational consistency is central. Dispatchers, drivers, and operations teams rely on tools that fit their day-to-day routines. Even well-intentioned AI can struggle if it disrupts those patterns. Experimentation is healthy, he noted, but organizations must ensure that prototypes align with real-world workflows and comply with security and governance frameworks.
Both leaders emphasized that successful AI adoption is largely a leadership exercise, one that involves change management, communication, expectation-setting, and investment in people.
5. The Hardest Problems Aren’t Based in Algorithms
Amine Allouah ‘19, co-founder and partner at MyCustomAI, described what he calls the “last mile” of AI: turning models into operational tools. That requires clean data, careful tuning, annotation workflows, and ongoing collaboration with internal teams. This sort of work often consumes more time and resources than the modeling itself.
Senior Director of AI Product Strategy & Growth at Medidata Solutions, Jia Chen, highlighted that AI in clinical trials cannot simply be “smart” – it must be responsible, privacy-preserving, and validated to regulatory standards. Synthetic data unlocks safer innovation. Yet data alone is not sufficient — expert knowledge is equally critical for trustworthy AI.
Ali Sadighian ‘09, Senior Director of Data Science at The Hershey Company, underscored a similar issue: data spread across systems that don’t communicate. Fixing data pipelines and occasionally revisiting long-standing processes is often the prerequisite for deploying AI at scale.
“Fix your data,” Sadighian said, “and don’t be afraid to break things.”
Together, the practitioners emphasized a theme that ran through the summit: AI struggles not because the models are weak, but because the surrounding systems are not necessarily ready
