When discussing the gap between large and small models, many people intuitively assume that small models lack reasoning ability. But small models have actually possessed logical reasoning capabilities for a long time and can already handle many logic tasks; Schaeffer et al. 2023 pointed out that many so-called abrupt emergent abilities are merely artifacts of the choice of metric, and reasoning ability is not unique to large models. What truly widens the gap as parameter scale grows—and forms the bottleneck for small models—is knowledge storage capacity. The knowledge capacity scaling law study KCSL found that language models can, and only can, store 2 bits of knowledge per parameter; a 7B model can store about 14B bits, which the paper estimates exceeds the entirety of English Wikipedia and textbooks combined, whereas the facts a small model can hold are only a small fraction of what a large model can store. What separates small models from frontier models is far more about how many facts the model has actually stored than whether it can reason.
This makes measuring how many facts a model actually knows an unavoidable question, but answering it in engineering is not easy. Practitioners building systems cannot simply inspect the weights directly to look up a specific fact, nor can they easily tell whether a model failing a factual question stems from never having learned it in training data or from failing to retrieve it during this specific query. The paper opens with such a question: Which famous band played their first gig at the Boardwalk club? The answer is Oasis. When a model fails to answer this question, it is very likely just failing to recall it this time, rather than having a complete blank in its weights.
The paper Empty Shelves or Lost Keys? published by Google Research at ICML 2026 (poster, OpenReview) directly tests this question, alongside a Google Research blog post published on 2026-08-12. Using 2150 Wikipedia facts from the public dataset WikiProfile, the author team tested approximately 4.5 million responses across 13 models. The authors measured that frontier models achieve an encoding rate of 95–98% on these facts, storing almost all the knowledge into their weights; however, the failure rate for closed-book direct questioning remains at 26–34%, and even with thinking enabled, 11–12% still cannot be answered.
When a model misses a factual question, there are usually two different root causes. The first is storage failure: the knowledge itself was never consolidated into the weights during training—it was absent from the corpus, or present but never learned—and the fact does not exist in the parameters. The second is access failure: the knowledge is already consolidated in the weights, but the model failed to retrieve it during this query because the prompt phrasing did not match the context in which it was learned. Empty shelves mean it was never stored; lost keys mean it was stored but could not be retrieved.
These two ailments require completely opposite remedies. Storage issues require spending budget on data and parameters; access issues require allocating budget toward prompt formulation, verification mechanisms, thinking compute, and retrieval trigger conditions. Existing evaluations typically lump both failures under the same error tally, leading people to instinctively buy the first remedy and spend money in the wrong direction.
Looking at a few percentages alone is not enough; this is a paper published by an industry lab using a single Wikipedia benchmark. What gives this study its standing is the author team’s deliberate design of their probing methods: every test decision is crafted to close off an escape hatch for the model. Laying out the verification logic for storage and retrieval below is what makes the subsequent engineering takeaways hold up.
Directly opening up a large model’s weights to search for a specific fact and verify whether knowledge has been consolidated is not practically feasible in engineering. The paper adopts the next best approach, replicating the state the model is most familiar with during pretraining: feeding it the original Wikipedia text, truncated right before the answer appears, to see if it can continue the sequence. Since the model was trained through this exact next-token prediction in context, completing the answer in its most familiar setting suffices to prove that the memory exists in its parameters. This is the closest behavioral test to directly checking storage.
The authors intentionally kept this ruler for measuring storage very loose. The test includes two variants—sentence completion and in-context question answering with original text—and the model passes as long as it gets either one right, with thinking turned off throughout. The control group sets the bar low enough so that diagnosing a storage failure can be sufficiently conservative: only if the model fails to complete the sentence even when given the original text can one solidly conclude that it was never stored. The authors measured that frontier models achieve an encoding rate of 95–98% under this loose ruler, with knowledge storage approaching saturation. The first misdiagnosis escape hatch this design closes off is the suspicion that the model simply never encountered the data.
While the control group set the ruler very loosely, the experimental group had to tighten it significantly. In the closed-book knowledge test, the authors removed all source text and presented four paraphrased questions—two asked forward and two asked in reverse—requiring the model to answer all four correctly to be considered as having mastered the fact.
The paraphrased pairs close off the excuse that the model simply did not understand the phrasing. In closed-book direct factual questioning, whenever a model gets an answer wrong, the biggest confounding variable is wording: testers can hardly distinguish whether the fact was never stored in the parameters or whether the model stored it but struggled with the specific phrasing. If the model merely struggles with a particular wording, its performance will fluctuate when paraphrased; if the model has genuinely memorized the fact, it should answer consistently regardless of phrasing. Offering two variations of the same question is designed precisely to control for wording, disentangling phrasing misunderstandings from failures to store or retrieve. The authors conducted 104 statistical tests confirming that paraphrasing had no significant effect on performance, proving that scores do not fluctuate with phrasing. Once wording is ruled out as a variable, misunderstanding is no longer an excuse, and the real test of the four questions rests entirely on query direction.
During pretraining, models learn facts into their parameters following the original word order; forward questions align with this sequence, keeping the retrieval path short. Reverse questions invert the order, deviating from the linguistic context during pretraining, which lengthens the retrieval path and makes it most prone to breaking down. Answering a forward question correctly proves that the model indeed stored the fact; failing the reverse question shows that the model failed to retrieve it this time. Reverse questioning thus becomes the scenario most likely to trigger retrieval failure, while paraphrasing eliminates the confound of phrasing misunderstanding. Working together, they cleanly separate never stored, misunderstood phrasing, and stored but unretrieved.
Reverse question failure brings up another deeper suspicion: did the model simply never store bidirectional associations in the first place—the widely discussed reversal curse since 2023? The paper closes off this retreat using multiple-choice questions: turning the same questions into 4-choice options with the correct answer placed right in front of the model. Test results show that while all models perform worse on reverse questions than forward ones during closed-book open generation, once converted to multiple choice, 9 models actually scored better on reverse questions than forward ones. The same holds true for humans in daily life: if someone can instantly identify the correct name among four options, no one would claim they have zero memory of it. This contrast between generation and multiple-choice was analyzed in NeurIPS 2024’s Delving into the Reversal Curse; this paper brings it into real Wikipedia facts and larger-scale models.
Under this strict ruler, the authors measured that the failure rate of frontier models on closed-book direct questioning falls in the 26–34% range. However, an essential caveat must be stated: measuring storage with a loose ruler on one hand and testing retrieval with a strict ruler on the other makes a gap logically inevitable; any comparison of loose and strict benchmarks will produce a numerical difference. How much of this difference reflects the true state of the model versus the artificial calibration of the rulers cannot be completely decoupled from the paper’s data alone. What truly withstands scrutiny is the shifting shape of this gap, rather than its specific absolute size.
The first pattern that holds up is the widening divergence between obscure and popular facts. The gap between obscure and popular facts during the encoding stage is only around 5 percentage points, but during closed-book direct recall, the gap quickly widens to over 20 points. Taking Gemini-3-Pro as an example, closed-book recall is 84.6% on popular facts, but drops straight to 63.3% on obscure facts. If the disparity between the two rulers were merely an artifact of artificial thresholding, the gap between obscure and popular facts should have risen and fallen in lockstep. Storage failure cannot explain this divergence; an obstructed retrieval pathway can.
This phenomenon directly challenges our default everyday attribution. When many engineers see a model fail on obscure knowledge, their immediate reaction is often that the training corpus saw too little of it and the model never stored it, so more data must be added. The experimental data shows the exact opposite: most obscure facts are already stored in the parameters, with encoding levels barely differing from popular facts; what truly gets stuck is the retrieval process. Blindly piling on more training data is not the right remedy.
The second pattern emerges during model parameter scaling. Across the Gemma3 series from 1B to 27B, the encoding failure rate drops all the way from 85% to 23%, yet among the remaining errors, the proportion of retrieval failures continues to rise. Increasing model size indeed fills the shelves fuller, but the aisles for retrieving items do not widen accordingly. Scaling parameters fixes storage, but cannot fix access. For small models like Gemma3 1B with encoding failure rates as high as 85%, expanding parameters and adding data is indeed the right prescription; but as model scale grows and encoding approaches saturation, the primary bottleneck shifts from storage to access.
WikiProfile consists entirely of single-hop facts. For questions such as which album by Extreme reached No. 10, there is no intermediate chain of logical deduction, so conventionally a thinking mechanism should not be useful. Yet the authors measured that simply enabling thinking allows frontier models to recover 40–65% of facts that were encoded but missed during closed-book direct questioning; by contrast, for facts that were never stored in the first place, thinking only recovers 5–15%, narrowing the residual failure rate after enabling thinking to 11–12%.
This grouped breakdown closes off the hypothesis that thinking merely gives the model more chances to guess or detour through other related knowledge. If thinking relied purely on lucky guesses or indirect reasoning, its benefit should apply equally to stored and unstored content, yet the data shows a clear bias. Another supporting piece of evidence is output consistency: with thinking enabled, multiple sampled outputs on the same question become more concentrated, with variance shrinking significantly. This indicates that thinking acts as a retrieval mechanism here, helping the model search through memories already stored in its weights.
A study published in June by researchers from the same group, Thinking to Recall (COLM 2026, arXiv 2603.09906), conducted an even more thorough verification. By replacing all reasoning text in the thinking process with meaningless repetitive sentences, they found that simply increasing the number of tokens in the forward pass helped the model recall single-hop facts. The underlying mechanisms are primarily computational buffering and factual priming: additional intermediate tokens give the attention mechanism more addressing opportunities, while related words generated earlier serve as cues to trigger the correct answer—with no connection to complex logical deduction.
For engineers building systems, this directly presents a cost ledger. Thinking is indeed a targeted remedy, but its gains are concentrated on facts that are already encoded. Blindly enabling thinking for all queries is wasteful, while triggering it precisely when memory retrieval is truly needed requires the model to know whether it can retrieve the fact this time. This metacognitive ability is a question the paper leaves unanswered.
When a user asks a public encyclopedic fact online and the model replies that it doesn’t know or produces a wrong name, many people habitually infer that the model never learned this knowledge, and then rush to add training data, fine-tune, or immediately hook up external search. But before adjusting the system, one should first diagnose the root cause: was this fact never stored in the weights, or was it stored but encountered a retrieval roadblock in this specific query? Storage failure and retrieval obstruction correspond to completely different optimization directions and dictate where budget should be allocated.
When troubleshooting factual issues in production, one can perform a comparative check along four specific dimensions. First is coverage: check whether the internal knowledge base or pretraining data actually includes this fact. Second is access pathways: try rephrasing the question or asking it in reverse to see if the model can answer smoothly. Third is recognition capability: present several candidate answers to the model as a multiple-choice question to test whether it can identify the correct option. Fourth is recency and proprietary nature: confirm whether the fact is public common knowledge, or constantly changing dynamic updates and team-proprietary data. In routine troubleshooting, focusing solely on the first item—coverage—leads to the misconception that having data in the corpus solves everything, when in reality a huge portion of production issues stem from the second item: access pathways.
If the query involves public encyclopedic facts, online and offline probing should follow a ladder progressing from cheap to expensive. Step one is low-cost probing. Engineers can try adjusting the question’s word order or providing a few contextual cues in the prompt. This step incurs only the compute cost of standard generation at minimal expense; if rephrasing allows the model to answer correctly, the knowledge was in the parameters all along, and only the previous retrieval path was obstructed.
Step two is introducing option verification. If rephrasing still yields no answer, several candidate answers generated by the model can be formatted into a multiple-choice question for it to choose from. As long as the model possesses recognition ability, accuracy typically rebounds noticeably, and this step likewise requires no external systems.
Step three is considering enabling thinking. If low-cost probing and option verification fail to awaken the memory, enabling thinking can help the model search its weights for single-hop facts. However, the thinking process consumes substantial intermediate tokens at a compute cost much higher than standard generation, so it cannot be rolled out indiscriminately to all traffic and is best reserved for high-value requests.
Step four is triggering external retrieval. If the model still cannot answer after multiple rounds of probing and thinking, the fact was in all likelihood never stored in the parameters; only then is invoking a retrieval system to bring in external context the justified choice.
For proprietary team business data and fresh, time-sensitive facts, the logic is completely different. Such knowledge naturally does not exist in the model’s pretraining weights, and running tiered probing on them only wastes compute. Once the system identifies this type of query, it should immediately route to retrieval, skipping the entire weight-probing pipeline.
This triage logic forms a complementary relationship with Retrieval-Augmented Generation (RAG). RAG is specifically designed to supplement what is not in the model’s parameters in the first place, solving access to private documents and real-time changes; this diagnostic framework is responsible for unblocking public encyclopedic facts that are already stored in the model weights but failed to be retrieved due to prompt misalignment. Both aim to resolve blocked access—the former seeks incremental information from external knowledge bases, while the latter extracts existing stock from model weights.
Currently, many adaptive retrieval schemes (such as Self-RAG or Adaptive-RAG) attempt to design decision mechanisms to determine when to trigger retrieval. They typically observe directly whether the model can answer the question at hand, and initiate retrieval as soon as the model fails. This practice conflates “never stored” and “stored but not retrieved” into a single trigger. When a model fails to answer a public fact, it is often just a matter of suboptimal prompt phrasing that could be revived with an alternate cue; triggering retrieval indiscriminately not only adds retrieval latency and context-processing overhead, but also masks underlying retrieval deficiencies in the prompt pathway itself.
The next time a model misses a factual question in production, consider taking a step back to identify the root cause: was it genuinely never stored, or was it stored but failed to be retrieved this time? For these two distinct ailments, what you should pull out are two completely different budget sheets.