Businesses have already plunged headfirst into AI adoption, racing to deploy chatbots, content generators, and decision-support tools across their operations. In keeping with McKinsey, 78% of firms use AI in at the very least one business function.
The frenzy of implementation is comprehensible — everyone sees the potential value. But on this rush, many organizations overlook the incontrovertible fact that all neural network-based technologies, including every LLM and generative AI system in use today and for the foreseeable future, share a major flaw: They’re unpredictable and ultimately uncontrollable.
As some have learned, there will be real fall-out consequently. At one Chevrolet dealer that had deployed a chatbot to its website, a customer convinced the ChatGPT-powered bot to sell him a $58,195 Chevy Tahoe for just $1. One other customer prompted the identical chatbot to put in writing a Python script for complex fluid dynamics equations, which it happily did. The dealership quickly disabled the bots after these incidents went viral.
Last 12 months, Air Canada lost in small claims court when it argued that its chatbot, which gave a passenger inaccurate details about a bereavement discount, “is a separate legal entity that’s answerable for its own actions.”
This unpredictability stems from the elemental architecture of LLMs. They’re so large and complicated that it’s unimaginable to know how they arrive at specific answers or predict what they’ll generate until they produce an output. Most organizations are responding to this reliability issue without fully recognizing it.
The commonsense solution is to ascertain AI results by hand, which works but drastically limits the technology’s potential. When AI is relegated to being a private assistant — drafting text, taking meeting minutes, summarizing documents, and helping with coding — it delivers modest productivity gains. Not enough to revolutionize the economy.
The true advantages of AI will arrive once we stop using it to help existing jobs and as an alternative rewire entire processes, systems, and corporations to make use of AI without human involvement at every step. Consider loan processing: if a bank gives loan officers an AI assistant to summarize applications, they could work 20-30% faster. But deploying AI to handle your entire decision process (with appropriate safeguards) could slash costs by over 90% and eliminate just about all the processing time. That is the difference between incremental improvement and transformation.
The trail to reliable AI implementation
Harnessing AI’s full potential without succumbing to its unpredictability requires a classy mix of technical approaches and strategic considering. While several current methods offer partial solutions, each has significant limitations.
Some organizations try and mitigate reliability issues through system nudging — subtly steering AI behavior in desired directions so it responds in specific ways to certain inputs. Anthropic researchers demonstrated the fragility of this approach by identifying a “Golden Gate Bridge feature” in Claude’s neural network and, by artificially amplifying it, caused Claude to develop an identity crisis. When asked about its physical form, as an alternative of acknowledging it had none, Claude claimed to the Golden Gate Bridge itself. This experiment revealed how easily a model’s core functioning will be altered and that each nudge represents a tradeoff, potentially improving one aspect of performance while degrading others.
One other approach is to have AI monitor other AI. While this layered approach can catch some errors, it introduces additional complexity and still falls wanting comprehensive reliability. Hard-coded guardrails are a more direct intervention, like blocking responses containing certain keywords or patterns, comparable to precursor ingredients for weapons. While effective against known issues, these guardrails cannot anticipate novel problematic outputs that emerge from these complex systems.
A more practical approach is constructing AI-centric processes that may work autonomously, with human oversight strategically positioned to catch reliability issues before they cause real-world problems. You wouldn’t want AI to directly approve or deny loan applications, but AI could conduct an initial assessment for human operators to review. This may work, nevertheless it relies on human vigilance to catch AI mistakes and undermines the potential efficiency gains from using AI.
Constructing for the longer term
These partial solutions point toward a more comprehensive approach. Organizations that fundamentally rethink how their work gets done somewhat than simply augmenting existing processes with AI assistance will gain the best advantage. But AI should never be the last step in a high-stakes process or decision, so what’s the very best path forward?
First, AI builds a repeatable process that may reliably and transparently deliver consistent results. Second, humans review the method to make sure they understand how it really works and that the inputs are appropriate. Finally, the method runs autonomously – using no AI – with periodic human review of results.
Consider the insurance industry. The traditional approach might add AI assistants to assist claims processors work more efficiently. A more revolutionary approach would use AI to develop latest tools — like computer vision that analyzes damage photos or enhanced fraud detection models that discover suspicious patterns — after which mix these tools into automated systems governed by clear, comprehensible rules. Humans would design and monitor these systems somewhat than process individual claims.
This approach maintains human oversight on the critical juncture where it matters most: the design and validation of the system itself. It allows for exponential efficiency gains while eliminating the chance that AI unpredictability will result in harmful outcomes in individual cases.
An AI might discover potential indicators of loan repayment ability in transaction data, as an example. Human experts can then evaluate these indicators for fairness and construct explicit, comprehensible models to substantiate their predictive power.
This approach to explainable AI will create a clearer divide between organizations that use AI superficially and those who transform their operations around it. The latter will increasingly pull ahead of their industries, in a position to offer services at price points their competitors cannot match.
Unlike black-box AI, explainable AI systems ensure humans maintain meaningful oversight of the technology’s application, making a future where AI augments human potential somewhat than simply replacing human labor.
