Over the past several weeks I've been on a journey to understand how large language models work — not from textbooks or courses, but through open-ended conversation with frontier models, primarily Grok and Opus. I came into this with a software engineering background but no formal AI training. The results have been genuinely impressive.
Through nothing more than asking open-ended questions, the following foundational concepts were introduced to me organically by the model as I explored:
This is not a small list. A frontier model, given nothing but open-ended questions from an engineer wanting to learn about AI, introduced every one of these concepts unprompted and at the right moments. The model acted as an extraordinary teacher — patient, knowledgeable, and responsive to any guided direction I wanted to take.
I want to be clear about this because what follows is not a criticism of LLMs. It is an observation about a specific property of how they work — one that anyone using them for discovery should understand.
Early in the series you're currently reading, in Chapter 04 ("The Infinite Loop"), I wrote about Frege's distinction between sense and reference and how it maps onto embeddings and attention:
"It's what turns a bunch of isolated micro-ideas into coherent chains of thought."
The phrase "chains of thought" was derived independently from my Frege reading. It described what I was observing: discrete embeddings being linked together by contextual sense into sequential reasoning.
Neither I nor the co-authoring model ever connected this phrase to Chain-of-Thought (CoT) prompting — one of the most widely discussed and foundational concepts in modern AI research. CoT is the basis for how reasoning models work, how inference-time compute scaling operates, and how researchers elicit step-by-step reasoning from LLMs. It sits at the center of an enormous body of work including tree-of-thought, self-consistency, process reward models, and the entire reasoning model paradigm from DeepSeek-R1 to o1.
The connection was never made. Not once, across weeks of co-authorship.
This is especially striking because the series didn't just contain the phrase. It contained entire chapters about instrumenting the generation process — inspecting token generation step by step, pausing at breakpoints to examine top-k candidates, measuring coherence at each micro-hop, probing the KV cache to observe how tokens influence output. The model was given free creative reign to compose articles and draw whatever connections it found relevant. It reached across forty years of my engineering history to bridge semiconductor physics to narrative composition, batch audio queues to SwiftUI snapshots, submarine waypoints to latent space navigation, and jazz improvisation to autoregressive generation.
But it never bridged "chains of thought" to "Chain-of-Thought." The single most relevant technical concept in the model's own domain, separated by a plural inflection, surrounded by directly related subject matter — and silence.
The term "narrative trap" comes from behavioral science. Professor Paul Dolan at the London School of Economics has written extensively about how dominant social narratives constrain decision-making by making alternatives invisible. You don't see outside the story you're living in because the story feels complete.
I'm borrowing and extending the term because the same mechanism operates during LLM co-authorship, and for the same reason.
When you build a rich, coherent prefill — a detailed context window full of interconnected ideas, personal experiences, and cross-domain analogies — the model composes within that context. Every generated token attends primarily to what's already present. The richer and more self-consistent the narrative, the more strongly the model's attention weights favor the existing content. The output feels insightful, grounded, and coherent because it is drawing from a dense, well-structured context.
But that same density creates a chamber. Concepts, domains, and research areas that are not seeded in the prefill don't get activated — even when they are densely related to the content, even when they are foundational to the subject matter, and even when the model has full knowledge of them in its weights.
This is not a model failure. It is how inference works. The model attends to the context it's given. It composes from the tokens that are present. Missing seeds stay missing because nothing in the system creates pressure to look for them.
The narrative trap is the natural boundary condition of co-authorship with an LLM. The co-author will never tell you what you forgot to ask about.
The contrast is what makes the trap visible. Every other concept on the list at the top of this article was introduced because I asked open-ended questions in territory where my context window had genuine gaps. When I asked about how generation works, the model detected that tokenization, softmax, and sampling strategies were absent from my knowledge and introduced them. When I explored retrieval, RAG surfaced naturally. When I asked about scaling, inference-time compute emerged.
These introductions worked because the concepts had zero representation in my context. The model detected the gap and filled it.
"Chains of thought" was different. The tokens were present. The surface form looked like existing knowledge. The model treated a fluent morphological variant as evidence that the conceptual territory was already covered. The introduction mechanism — the same one that flawlessly surfaced every other concept — never fired.
The model will tell you what you don't know. It won't tell you what it thinks you already know but don't.
During a later attempt to catch this gap, I asked a model to exhaustively extract every concept from Chapter 04. It found 47 concepts, including "LinkedIn (publication venue)" and "liberal arts education as formative for engineering." It did not list "chains of thought" as a concept at all.
The phrase was so seamlessly embedded in flowing prose — plural, lowercase, grammatically integrated — that the concept extraction operation parsed it as connective syntax rather than a distinct extractable idea. The model can bridge a five-character misspelling to find the right concept, because misspellings create anomalous signals that trigger correction. But a fluent morphological variant creates no anomaly. It's perfectly grammatical, perfectly expected, and therefore perfectly invisible.
This is similar to the acronym expansion problem I documented in Chapter 12 of the series, where the model failed to expand "DJ" to the correct author-defined meaning despite it being explicitly present in the text. Surface-form recognition has systematic blind spots when the surface looks normal.
I eventually discovered CoT as a formal inference technique — not through better prompting, not through a model correction, but by opening a completely new chat session and independently exploring the mechanics of inference from scratch. That fresh context window had no allegiance to the analogy series narrative. No echo to reinforce. No chamber walls. CoT surfaced naturally because the territory demanded it.
The escape from a narrative trap is simple: step outside the current narrative. Open a clean session. Explore adjacent or unfamiliar domains without the prior prefill. The knowledge you need is in the model's weights. It just needs a context that gives it a reason to surface.
The experience suggests a practical workflow for anyone using LLMs for extended discovery or co-authorship:
Periodic domain audits. At regular intervals, open a clean session and ask: "Given that I'm working on [topic], what established research areas, widely-used techniques, or foundational concepts are most relevant that I should be aware of?" No prior context. No echo. Just a direct inventory of the neighboring landscape.
Concept provenance checks. When you introduce a phrase or concept that feels important, ask in a separate session: "Does this phrase or concept have established usage in other fields?" This is lightweight and catches collisions like "chains of thought" before they become invisible.
Negative space detection. Ask the model to identify what's conspicuously absent from your context relative to what it knows about the subject. The prompt: "Read this material and identify well-established concepts that a knowledgeable reader would expect to see referenced but that never appear." This catches the first type of narrative trap — genuine absence.
Fresh session detours. Periodically start a completely new conversation to explore adjacent territory with no prefill bias. This is the most reliable escape hatch. It's how I actually found CoT. The empty prefill is not a limitation — it's the exit from the chamber.
These steps are not elaborate. They add minimal overhead. And they are the passivation layer that makes cross-domain discovery possible during LLM co-authorship.
The narrative trap is not a bug. It is a natural property of how inference works. The model attends to what's present and composes from it. The richer your context, the stronger the chamber. The more coherent your narrative, the less likely either participant is to break out.
The people most at risk are the ones doing the most interesting work — building rich, interconnected contexts, engaging in extended co-authorship, and using LLMs as genuinely helpful discovery tools. Everything feels like progress. The model agrees. The narrative flows. And the missing seeds stay missing because the loop feels complete.
Understanding this doesn't diminish what LLMs offer. The list at the top of this article is real. The model is an extraordinary teaching aid. But productive discovery requires an awareness of the narrative trap, and the discipline to periodically step outside rich well formed contexts.
The loop never really closes. But now you know to check what's on the other side of the wall.
This chapter draws on a separate Inference Series where Chain-of-Thought is explored in technical depth.