AI Works—Until It Doesn’t: Why Great Pilots Collapse in the Real World

News

1. DIFC Eyes 25,000 Jobs, $3.5bn Boost as World’s First AI-Native Finance Hub

2. Amazon Deepens Anthropic Ties With Up to $25 Billion Investment

3. Apple Picks John Ternus to Succeed Cook

4. ServiceNow Expands Cybersecurity Stack with Armis Acquisition

5. UK Banks to Get Mythos Access Amid Global Cybersecurity Concerns

6. UAE Cyber Security Council Flags Human Error as Primary Cyber Risk

In 2024, Yum! Brands, the parent company of Taco Bell, made a long-term commitment to AI by deploying the technology to handle thousands of drive-through orders across the US, replacing staff for this task at scale. Before rolling it out nationwide, they tested the AI ordering system at more than 100 outlets across 13 states for 2 years.

“With over two years of fine-tuning and testing the drive-thru voice AI technology, we’re confident in its effectiveness,” said Lawrence Kim, Chief Innovation Officer, Yum! Brands, in a statement.

Taco Bell’s technology was an upgrade, and it could also recognize different ways people pronounced menu items, such as “kay-suh-DEE-yuh” or “kay-suh-DIL-uh” for quesadilla. The aim was to make orders more accurate, move lines faster, and let workers focus on preparing food rather than taking orders.

Yum! Brands planned to use the technology across all its brands worldwide. However, problems soon began to appear.

Instances of AI order-taker’s technical mishaps quickly began circulating online, with one customer ordering a McDonald’s item at Taco Bell, while another ordered “a large Mountain Dew,” only to be asked, “And what will you drink with that?” The system also struggled with diverse accents, rapid speech, and background noise from traffic or engines, leading to misinterpretations and delays. 

The final nail in the coffin was a customer ordering 18,000 cups of water, which the AI happily fulfilled, leading to the initiative implemented in over 500 restaurants being rolled back in 2025 after taking two million orders during its run.

Taco Bell exemplifies an AI initiative with successful pilots but challenges in expansion. 

Pilot vs Deployment Illusion

The gap in AI today isn’t between companies that use it and those that don’t. It’s between those who can deploy it—and those who can’t. Before the launch of OpenAI’s ChatGPT, IDC predicted the global market value of AI to reach $450 billion in 2022. Over the years, the pace of advancement and the influx of capital have grown exponentially. Today, Gartner predicts worldwide AI spending to total $2.52 trillion in 2026.

Enterprises and institutions globally have widely adopted tools like ChatGPT and Copilot and have invested in their own AI tools. However, several reports have highlighted that most AI initiatives fail to deliver meaningful results. The GenAI Divide: State of AI in Business 2025, a report published by MIT’s NANDA initiative, reveals that only 5% of AI pilot programs achieve rapid revenue acceleration, with many stalling with no measurable P&L impact.

“This divide does not seem to be driven by model quality or regulation, but seems to be determined by approach,” the report read.

“I’m not sure every company and every group that does something with AI knows why they’re doing so,” says Joe Dunleavy, Regional CTO and Global Head of Dava.X AI Group, Endava.

How can enterprises strategize the optimal way for investing in and deploying AI? “Before you jump into committing to AI, understand what you’re trying to solve and why you’re trying to solve it. If you don’t know that, it’s going to be hard to build something impactful,” he adds.

This is echoed by Mohamed Irfan Peeran, CEO, VDart Digital, who says, “A common pattern we see is CXOs deciding upfront that they want to ‘use AI’ in their products or services, and then searching for places to apply it. This often leads to pilots that look impressive in demos but lack a clear business impact. When the time comes to scale, there is no strong ROI narrative, so the project loses momentum.”

Two Organizations, Two Outcomes

1. When AI Fails in the Real World

For Peeran, failure was instructive. “We once launched a generative AI feature for customer support that performed beautifully in offline testing,” he says, revealing that while beta results were promising, the quality dropped sharply within days of launch.

​The issue was straightforward: real users don’t behave like test data. They mix languages, paste incomplete queries, attach screenshots, and build on prior context. “The system had been tuned for ideal inputs, not human behavior,” he adds. They fixed it by refining retrieval, adding guardrails, and routing edge cases to humans.

2. What Knockout Looks Like

For Vivek Bhatnagar, Senior VP and Head of Business, Middle East, NewGen, strong pilots translate well when they are designed with production in mind from day one.

“At Newgen, for a large financial services client, we deployed a generative AI-led document intelligence solution. In the pilot, the system delivered 80–90% accuracy and demonstrated clear productivity gains,” he shared. The key difference was that the pilot itself was not treated as a sandbox, but was built on real data, integrated with core systems, and tested against production-like volumes and variability. Because of this, when it moved to production, performance held steady and, in some areas, improved, along with an accuracy stabilization of above 85%, a reduction of turnaround times by over 20%, and adoption scaling smoothly across teams.

Bhatnagar and Newgen accounted for real-world complexity, including document variability, integration dependencies, and governance requirements, as well as human-in-the-loop validation.

“When pilots are treated as early versions of production systems, not isolated experiments, the transition is not a risk point. It becomes a natural progression,” he adds.

Failing at Scale: What Most Organizations Get Wrong

The main problem with enterprise AI is that many organizations see software deployment as the end goal, when in fact AI deployment is just the beginning. “At least 60-70% of AI pilots still fail to scale, and the primary reason is not the technology. It is the inability to operationalize,” says Bhatnagar. 

Most organizations fail during the transition from validation to production due to three reasons: isolation, data readiness, and ownership.

1. Pilots in Isolation

Most pilots remain within innovation or digital teams and are disconnected from essential enterprise functions like operations, risk, and compliance.’

2. Data Readiness is a Struggle

Many organizations grapple with fragmented data in legacy systems and regulatory hurdles around residency and governance, especially in the banking and public sectors.

For Dunleavy, it’s people readiness from a data perspective. “The real challenge—especially in pilot phases—is fragmented data. Teams lack a clear ‘gold standard,’ and information is scattered across systems. When generative AI is applied to these disconnected sources, it tries to reconcile inconsistencies, often leading to hallucinations.”

3. Initiative Ownership

Innovation fuels pilot projects, but scaling up requires business, IT, and operations to work together. This is often where progress stalls.

AI pilots developed in isolation pose the biggest challenge for future deployments. Revathi Venkataraman, Professor and Chairperson (School of Computing) at SRM Institute of Science and Technology, in Tamil Nadu, India, points out that since machine learning models are trained on historical data across various conditions, they usually perform well under similar conditions after initial deployment.

“Once a model is deployed, it enters an environment where patterns and behaviors constantly evolve. Consequently, there is often a mismatch between the training data and the real-world data the model encounters. If this gap isn’t addressed, the model’s performance may decline gradually,” she explains.

The discrepancies include:

1. Change in Data

Most ML methods rely on historical data that is stable and exhibits predictable relationships for training. However, in real-world scenarios, this data evolves over time because of changing user behaviors, trends, and external factors. Consequently, the relationships learned initially may become outdated and no longer valid.

For example, recommendation systems break when user preferences evolve, just as fraud detection models falter against new scams. Without regular monitoring and updates, model performance will degrade.

2. Shift in Input and Output

As learning models learn the association between inputs and outputs during the training phase of machine learning, the real world puts that relationship to the test. “Input data can appear the same way but mean something different, or have a different impact than before, after an extended period of time,” says Venkataraman.

For instance, words once flagged as spam may now not be spam, causing filters to misclassify emails. Without periodic updates to input/output rules, these models rely on outdated associations, resulting in inaccurate predictions.

3. Lack of Real-World Scenarios

In the real world, models encounter situations they aren’t familiar with. For example, self-driving systems trained on clear roads falter in heavy rain, fog, or traffic. Chatbots deliver odd responses to untrained queries.

4. Continuous Monitoring and Updates Missing

Once deployed, a model’s performance can drift amid evolving real-world scenarios,  making issues like bias or sudden errors hard to detect. Regular evaluation, monitoring, and retraining are essential to keep them reliable and effective.

A Framework Organizations Can Take Note Of

In Peeran’s view, only 20-30% of AI pilots reach full-scale deployment. He feels organizations should adopt a reverse approach: start with a well-defined business problem, analyze where the friction or inefficiency lies, and, lastly, evaluate whether AI can meaningfully improve that specific area. “When AI is applied this way, it is tightly linked to outcomes like cost reduction, speed, or revenue growth, which significantly increases the chances of successful deployment,” he adds.

From an academic perspective, Venkataraman finds evaluation essential to assess AI’s trustworthiness, fairness, and suitability for use. While several well-established academic evaluation frameworks exist, the industry opts for a pragmatic approach. Evaluation gaps exist between academic AI frameworks and industry practices, impacting real-world performance.

Most industries rely on static test datasets, which may not reveal potential hidden defects, rather than existing academic frameworks such as CheckList and PAWS, which mirror the unit-testing system used in software development and are intended to measure individual capabilities, such as handling negation, paraphrasing, and other language variations.

Similarly, while industries opt for correlational metrics like accuracy, BLEU, & ROUGE to measure surface-level performance, academic research emphasizes causal evaluation, which determines whether models actually understand how model inputs correspond to model outputs.

Academic fairness tools identify and mitigate biases across demographics by probing root causes, surpassing basic detection, whereas industry views fairness as mere compliance and skips thorough audits due to cost and complexity. “Adopting these frameworks is essential to building ethical and trustworthy AI systems,” Venkataraman adds.

What a Successful AI System Looks Like?

The technology is ready, but most organizations are not. What can your organization do differently from Taco Bell or McDonald’s to build a successful  AI strategy? 

Organizations need to check a few boxes to have a production-ready system. Peeran lists them: “First, the business use case should be narrow and valuable. Second, the system must have measurable quality thresholds. Third, it requires observability for inputs, outputs, latency, and errors. Fourth, there needs to be a safe fallback when the model is uncertain. Fifth, governance must address privacy, compliance, and prompt action against data abuse. Sixth, the team must have a clear process for updating, retraining, and retiring the system.”

Bhatnagar seeks a model that demonstrates stable accuracy in the 80-90%+ range, with clear, defined thresholds for acceptable variation. “It should sustain this across real-world variability, not just curated datasets.” His uptime expectations are 99%, and he continuously tracks both model metrics and business KPIs, with alerts triggered for deviations of 5–10%.

“Most mature systems detect and respond to issues within minutes to hours,” he shared, while revealing that, in generative AI, the definition of success varies drastically at the pilot stage versus production, and that shift is critical.

In pilots, validation is the goal. It demonstrates feasibility through 70–85% relevance/accuracy, low hallucination rates, and user satisfaction, such as 30–50% reductions in drafting effort in controlled settings. In production, it delivers business value like 15–30% faster turnaround, 10–20% cost savings, and CSAT uplift, with non-negotiable 99%+ uptime, low latency, and edge-case reliability.

Venkataraman states that defining a production-ready AI system in a center of excellence for agentic twins involves more than just developing a high-performance algorithm. It requires creating an industry-integrated solution that can be deployed as a system providing tangible, consistent value to users in real-world environments.

The organizations that will build a durable AI advantage in the next decade are not necessarily those with the most sophisticated models. They are the ones that build the governance architecture to catch failures early, respond fast, and maintain the institutional trust required to deploy AI at a consequential scale.

“Start small, build momentum, and get it into production early. Then keep iterating. The principles of agile absolutely apply here—you’re not aiming for perfection from day one, you’re aiming to learn, refine, and improve with each cycle,” advised Dunleavy. 

A Deloitte survey from January found that only 25% of respondents had moved 40% or more of their AI pilots into production. 

Despite the lukewarm reception of its AI voice system, Taco Bell and Yum! Brands are not backing down. They remain committed to AI, with the parent company’s CFO, Ranjith Roy, confirming the same. “Technology remains central to our transformation as we continue to deploy Byte by Yum!. Nearly half of the restaurants in the Yum! system now has Byte Coach, enabling easier operations by delivering AI recommendations to our team members,” he wrote in a LinkedIn post. 

“Simple rule that works amazingly well: if you cannot explain how AI behaves when it is wrong, it is not production-ready yet,” Peeran adds.