Real-Time Query Model
"Sounds like AI" deletes the best thinking first. Using tools to create isn't inhuman.
Every tool since fire had its deniers, and history marks them irrelevant fast.
(Also your tools for detecting this kind of stuff are worse than you think. Pangram flags this note as AI-written. I typed it myself. Point proven.)
Lately I've been thinking about what an LLM actually spends its weights on. A huge fraction of the parameters in something like GPT-4 go toward memorizing facts, which is to say a static snapshot of the world as of whenever the last training run finished. In large part the model is an enormous and very expensive fact database that happens to also reason.
Which raises the question this post is actually about: are data and reasoning separable? Does it really make sense for the weights to encode all the precedents, and is that anywhere close to the most efficient use of them? A learned copy of the world goes stale the moment training ends, after all, so a model trained through 2021 knows nothing about 2023, and beyond staleness there's more information out there than any set of weights could ever hold, with the fastest-moving information being what weights hold worst anyway.
Training with tools from the start
What I keep wondering is whether you could train an LLM with tools, so it never memorizes facts at all and only learns how to get at them. And I don't mean retrieval bolted on at inference time, which is basically what retrieval-augmented generation is, a search step grafted onto a model that already spent its capacity memorizing. I mean during training itself, so that from day one the model learns facts live outside it.
The mechanics would look something like this. You reserve part of the context window as a live data feed, and the model gets a pair of special tokens, [START_QUERY]...[END_QUERY], that let it emit a retrieval query in the middle of generating. An external system takes the query, runs it against the corpus, and drops whatever came back into the context between [START_RESULT] and [END_RESULT]. Training happens end-to-end with that whole loop running, which means the model has to figure out when a query is worth making, how to phrase it, and integrating the results, instead of learning the facts themselves.
The point is that if facts are always one query away during training, there's no gradient pressure to memorize them. The weights get spent on the one thing only weights can do, reasoning. And a couple of nice properties fall out for free, since knowledge now updates whenever you update the corpus rather than when you retrain, and you can specialize a model after training just by changing what data it's allowed to reach.
What's hard about it
Plenty. Retrieval has to be fast enough to sit inside the generation loop, for one. Coherence gets tricky when the context window keeps changing underneath the model mid-generation. Teaching a model to write good queries is its own learning problem entirely too, probably some mix of supervised examples and reinforcement, though I don't have a strong view on the ratio yet.
Still, the bet feels simple to me. A model that learned how to find things should beat a same-sized model that memorized things on basically everything short of closed-book trivia, and if data and reasoning really are separable, the weights we currently burn on being an encyclopedia are the cheapest capability gain sitting around.