are racing to make use of LLMs, but often for tasks they aren’t well-suited to. The truth is, in line with recent research by MIT, 95% of GenAI pilots fail — they’re getting zero return.
An area that has been missed within the GenAI storm is that of structured data, not only from an adoption standpoint, but additionally from a technological front. In point of fact, there’s a goldmine of potential value that may be extracted from structured data, particularly in the shape of predictions.
On this piece, I’ll go over what LLMs can and may’t do, what value you’ll be able to get from AI running over your structured data, specifically for predictive modeling, and industry approaches used today — including one which I developed with my team.
Why LLMs aren’t optimized for business data and workflows
While large language models have completely transformed text and communication, they fall short in making predictions from the structured, relational data that moves the needle, driving real business outcomes — customer lifecycle management, sales optimization, ads and marketing, recommendations, fraud detection, and provide chain optimization.
Business data, the information enterprises are grounded in, is inherently structured. It often resides in tables, databases, and workflows, where meaning is derived from relationships across entities corresponding to customers, transactions, and provide chains. In other words, that is all relational data.
LLMs took the world by storm and played a key role in advancing AI. That said, they were designed to work with unstructured data and aren’t naturally suited to reason over rows, columns, or joins. Because of this, they struggle to capture the depth and complexity inside relational data. One other challenge is that relational data changes in real time, while LLMs are typically trained on static snapshots of text. In addition they treat numbers and quantities as tokens in a sequence, reasonably than “understanding” them mathematically. In practice, this implies an LLM is optimized to predict the subsequent almost certainly token, which it does incredibly well, but to not confirm whether a calculation is correct. So, whether the model outputs 3 or 200 when the true answer is 2, the penalty the model receives is similar.
LLMs are able to multi-step reasoning through chain-of-thought-based inferencing, but they’ll face reliability challenges in certain cases. Because they’ll hallucinate, and achieve this confidently, might I add, even a small probability of error in a multi-step workflow can compound across steps. This lowers the general likelihood of an accurate end result, and in business processes corresponding to approving a loan or predicting supply shortages, only one small mistake may be catastrophic.
Due to all this, enterprises today depend on traditional machine learning pipelines that take months to construct and maintain, limiting the measurable impact of AI on revenue. When you need to apply AI to this sort of tabular data, you might be essentially teleported back thirty years and wish humans to painstakingly engineer features and construct bespoke models from scratch. For every single task individually! This approach is slow, expensive, doesn’t scale, and maintaining such models is a nightmare.
How we built our Relational Foundation Model
My profession has revolved around AI and machine learning over graph-structured data. Early on, I recognized that data points don’t exist in isolation. Relatively, they’re a part of a graph connected to other pieces of data. I applied this view to my work on online social networks and knowledge virality, working with data from Facebook, Twitter, LinkedIn, Reddit, and others.
This insight led me to assist pioneer Graph Neural Networks at Stanford, a framework that permits machines to learn from the relationships between entities reasonably than simply the entities themselves. I applied this while serving as Chief Scientist at Pinterest, where an algorithm often called PinSage transformed how users experience Pinterest. That work later evolved into Graph Transformers, which bring Transformer architecture capabilities to graph-structured data. This enables models to capture each local connections and long-range dependencies inside complex networks.
As my research advanced, I saw computer vision transformed by convolutional networks and language reshaped by LLMs. But, I spotted the predictions businesses rely upon from structured relational data were still waiting for his or her breakthrough, limited by machine learning techniques that hadn’t modified in over twenty years! Many years!
The culmination of this research and foresight led my team and me to create the primary Relational Foundation Model (RFM) for business data. Its purpose is to enable machines to reason directly over structured data, to know how entities, corresponding to customers, transactions, and products, connect. By knowing the relationships between these entities, we then enable users to make accurate predictions from those specific relationships and patterns.
Unlike LLMs, RFMs have been designed for structured relational data. RFMs are pretrained on various (synthetic) datasets in addition to on various tasks over structured business data. Like LLMs, RFMs may be simply prompted to provide fast responses to a wide range of predictive tasks over a given database, all without task-specific or database-specific training.
We wanted a system that would learn directly from how real databases are structured, and without all the standard manual setup. To make that possible, we treated each database like a graph: tables became node types, rows was nodes, and foreign keys linked every thing together. This manner, the model could actually “see” how things like customers, transactions, and products connect and alter over time.
At the center of it, the model combines a column encoder with a relational graph transformer. Every cell in a table is was a small numerical embedding based on what kind of information it holds, whether it’s a number, category, or a timestamp. The Transformer then looks across the graph to tug context from related tables, which helps the model adapt to recent database schemas and data types.
For users to input which predictions they’d wish to make, we built an easy interface called Predictive Query Language (PQL). It lets users describe what they need to predict, and the model takes care of the remaining. The model pulls the appropriate data, learns from past examples, and reasons through a solution. Since it uses in-context learning, it doesn’t must be retrained for each task, either! We do have an option for fine-tuning, but that is for very specialized tasks.
But this is only one approach. Across the industry, several other strategies are being explored:
Industry approaches
1. Internal foundation models
Corporations like Netflix are constructing their very own large-scale foundation models for recommendations. As described of their blog, the goal is to maneuver away from dozens of specialised models toward a single centralized model that learns member preferences across the platform. Analogy to LLMs is obvious: like a sentence is represented as a sequence of words, a user is represented as a sequence of films the user interacted with. This enables innovations to support long-term personalization by processing massive interaction histories.
The advantages of owning such a model include control, differentiation, and the power to tailor architectures to domain-specific needs (e.g., sparse attention for latency, metadata-driven embeddings for cold start). On the flip side, these models are extremely costly to coach and maintain, requiring vast amounts of information, compute, and engineering resources. Moreover, they’re trained on a single dataset (e.g., Netflix user behavior) for a single task (e.g., recommendations).
2. Automating model development with AutoML or Data Science agents
Platforms like DataRobot and SageMaker Autopilot have pushed forward the thought of automating parts of the machine learning pipeline. They assist teams move faster by handling pieces like feature engineering, model selection, and training. This makes it easier to experiment, reduce repetitive work, and expand access to machine learning beyond just highly specialized teams. In the same vein, Data Scientist agents are emerging, where the thought is that the Data Scientist agent will perform all of the classical steps and iterate over them: data cleansing, feature engineering, model constructing, model evaluation, and at last model development. While a real revolutionary feat, the jury remains to be out on whether this approach will probably be effective in the long run.
3. Using graph databases for connected data
Corporations like Neo4j and TigerGraph have advanced using graph databases to raised capture how data points are connected. This has been especially impactful in areas like fraud detection, cybersecurity, and provide chain management, places where the relationships between entities often matter greater than the entities themselves. By modeling data as networks reasonably than isolated rows in a table, graph systems have opened up recent ways of reasoning about complex, real-world problems.
Lessons learned
Once we got down to construct our technology, our goal was easy: develop neural network architectures that would learn directly from raw data. This approach mirrors the present AI (literal) revolution, which is fueled by neural networks that learn directly from pixels in a picture or words in a document.
Practically speaking, our vision for the product also entailed an individual simply connecting to the information and making a prediction. That led us to the ambitious goal of making a pretrained foundation model designed for business data from the bottom up (as explained above), removing the necessity to manually create features, training datasets, and custom task-specific models. An ambitious task indeed.
When constructing our Relational Foundation Model, we developed recent transformer architectures that attend over a set of interconnected tables, a database schema. This required extending the classical LLM attention mechanism, which attends over a linear sequence of tokens, to an attention mechanism that attends over a graph of information. Critically, the eye mechanism needed to generalize across different database structures in addition to across several types of tables, wide or narrow, with varied column types and meanings.
One other challenge was inventing a brand new training scheme, because predicting the subsequent token will not be the appropriate objective. As an alternative, we generated many manmade databases and predictive tasks mimicking challenges like fraud detection, time series forecasting, supply chain optimization, risk profiling, credit scoring, personalized recommendations, customer churn prediction, and sales lead scoring.
In the long run, this resulted in a pretrained Relational Foundation Model that may be prompted to unravel business tasks, whether it’s financial versus insurance fraud or medical versus credit risk scoring.
Conclusion
Machine learning is here to remain, and as the sphere evolves, it’s our responsibility as data scientists to spark more thoughtful and candid discourse concerning the true capabilities of our technology — what it’s good at, and where it falls short.
All of us understand how transformative LLMs were, and proceed to be, but too often, they’re implemented swiftly before considering internal goals or needs. As technologists, we should always encourage executives to take a more in-depth have a look at their proprietary data, which anchors their company’s uniqueness, and take the time to thoughtfully discover which technologies will best capitalize on that data to advance their business objectives.
On this piece, we went over LLM capabilities, the worth that lies inside the (often) missed side of structured data, and industry solutions for applying AI over structured data — including my very own solution and the teachings learned from constructing that.
Thanks for reading.
References:
[1] R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton and J. Leskovec, Graph Convolutional Neural Networks for Web-Scale Recommender Systems (2018), KDD 2018.
Creator bio:
Dr. Jure Leskovec is the Chief Scientist and Co-Founding father of Kumo, a number one predictive AI company. He’s a Computer Science professor at Stanford, where he has been teaching for greater than 15 years. Jure co-created Graph Neural Networks and has dedicated his profession to advancing how AI learns from connected information. He previously served as Chief Scientist at Pinterest and conducted award-winning research at Yahoo and Microsoft.
