- The ways in which large language models (LLMs) arrive at their decisions are not always clear to AI researchers.
- UF’s Hongcheng Liu, Ph.D., is concerned by this lack of clarity about the inner workings of these tools.
- His research seeks greater simplicity for complex AI systems.
One of the conundrums with large language models and artificial intelligence agents like ChatGPT is that while they work remarkably well for some tasks, researchers and designers are not always sure exactly how they work.
Not a problem for some, but for Hongcheng Liu, Ph.D., relying on a tool that isn’t thoroughly understood is problematic. He wants to increase users’ confidence in these tools, hopefully by thoroughly understanding more of their inner workings.
Associate Professor
Liu, an associate professor in the University of Florida’s Department of Industrial and Systems Engineering, or ISE, is examining a common mathematical approach used for training (or building) AI systems: sample average approximation (SAA), a technique that solves problems involving uncertain future outcomes using data samples. Liu and his co-author, ISE doctoral student Jindong Tong, detailed their work in a recently published article in Mathematical Programming, “Metric Entropy-Free Sample Complexity Bounds for Sample Average Approximation in Convex Stochastic Programming.”
The complex title belies what Liu and Tong are seeking: greater simplicity and clarity in AI systems, most notably in the amounts of training data necessary for reliable results.
“The existing theory predicted SAA should need far more data than it actually does, especially for large-scale problems,” Liu explained. “This is like having a car that runs great in conditions where the manual says it shouldn’t work at all. This is concerning to many users, as it means you don’t truly understand your own tool.”
As Liu explained, for any AI system trained on data — a spam filter, a medical diagnostic tool, a supply chain optimizer — critical choices were made regarding how much data to collect and which algorithm to use for training. The work described in the paper shows that the simplest, most natural approach is demonstrably just as effective as more complicated ones, but with much less training data required.
A system that works reliably with less training data costs less to operate, is more accessible and ultimately is more reliable, said Liu.
“What we’ve done is to advance the manual,” Liu said. “We can show that SAA is as data-efficient as the best available alternatives. Practitioners can now trust it with full theoretical backing, not just intuition.”
Florida’s climate and natural surroundings may have spurred Liu on as he was working on this problem.
“I came up with a preliminary version of the idea when I was crabbing at Cedar Key together with my wife,” he remembered.
Liu’s research is partially supported by the National Science Foundation.