The Problem Is Not Data. It Is Routing.
Prediction, Relevance, and the Limits of Machine Understanding
Abstract
The debate over whether contemporary artificial intelligence can think often begins with the wrong comparison. It asks whether machines possess enough information, whether they form representations, whether they predict successfully, or whether their internal processes resemble functions found in human cognition. These questions matter, but they do not settle the deeper issue. Information does not identify its own relevance. Prediction does not determine what should be predicted. A representation does not determine when it should govern action. The harder problem is routing: how information becomes relevant to a particular system at a particular moment, how competing information is weighted, what gives one source authority over another, and whether the consequences of a decision alter what the same continuing system does next.
This paper responds to arguments that similarities between human predictive processing and contemporary AI support attributing genuine thought and understanding to current models. I do not argue that machines can never think, nor that cognition requires a mysterious nonphysical human property. I accept that human cognition involves unconscious processing, prediction, incomplete introspection, and physical processes. My objection begins after those concessions. Demonstrating common predictive operations does not demonstrate common cognitive organization. Human cognition is not simply information plus prediction. It is a continuously changing relevance structure shaped by embodiment, development, culture, memory, social experience, geography, biology, history, action, and consequence. Everything may be representable as data. It does not follow that data is consciousness.
1. The Wrong Scarcity
Artificial intelligence has made information abundance feel like intelligence abundance.
A contemporary model can operate across enormous collections of language, images, software, scientific literature, historical records, legal documents, cultural artifacts, and human conversation. It can identify patterns within these materials at scales unavailable to an individual person. It can relate concepts across domains, generate predictions, retrieve apparently relevant information, reformulate arguments, and produce fluent explanations.
The achievement is real.
But information abundance creates a different scarcity: relevance.
The problem is no longer merely whether information exists. It is whether the right information becomes consequential at the right moment.
A system may possess ten million potentially relevant relations. Which one matters now?
Which earlier event should control the present decision?
Which apparent similarity is superficial?
Which contradiction should override the nearest match?
Which source should receive greater authority?
Which past failure should change the next action?
Which information should be ignored even though it is statistically associated with the current problem?
These are not secondary questions. They are part of the cognitive problem itself.
My earlier work on machine memory describes this distinction operationally: a system does not meaningfully benefit from stored information merely because that information exists somewhere in its history. The information has to be retrieved when relevant, interpreted in relation to the present problem, compared with competing evidence, and permitted to alter subsequent selection. Storage preserves possibility; routing determines whether stored information becomes consequential.
This is why I now think the central scarcity is not data.
It is routing.
2. Prediction Is Real, but Prediction Is Not the Whole Problem
Andy Masley argues that many common objections to machine thought rely on an inaccurate picture of human cognition. He points out that much human information retrieval, language production, association, and prediction occurs outside conscious awareness. He also argues that humans do not possess transparent introspective access to the processes producing their thoughts. On this point, I have little disagreement. Human cognition should not be modeled as a tiny observer sitting inside the brain, consciously supervising every mental operation. Masley is also careful in Part 1 to distinguish this argument from his second claim—that current AI possesses qualities sufficient for genuine thought and understanding—which he reserves for Part 2.
But removing a mistaken requirement for thought does not establish a sufficient condition for thought.
Suppose human brains and language models both engage in prediction.
Suppose both exploit context to anticipate what is likely to come next.
Suppose both contain representations whose relationships allow surprisingly sophisticated inference.
Those similarities matter.
They still leave open the central question:
What makes a particular prediction relevant to a particular system now?
Prediction always occurs inside a frame. Something has already constrained:
- what counts as the present state,
- which history is relevant,
- what outcome is being predicted,
- which distinctions matter,
- what counts as error,
- which alternatives deserve attention,
- and what consequences should alter future behavior.
The success of prediction therefore cannot by itself explain the construction of relevance.
Prediction may be one part of cognition while still failing to explain why one pattern becomes governing and millions of other available patterns do not.
The important distinction is not between humans, who supposedly do something magical, and machines, which merely predict.
It is between prediction as an operation and the system in which prediction acquires significance.
3. Information Does Not Contain Its Own Meaning
Everything can become data.
I continue to think that claim is useful, provided it is stated carefully.
A sensation can become data. A rejection can become data. A painful memory can become data. A social relationship, a change in preference, a cultural practice, a political event, a genetic variation, a geographical constraint, a failed experiment, or a pattern of hesitation can all leave differences that can be represented informationally.
But saying that everything can become data is not the same as saying that everything is only data, or that converting something into a representation preserves everything about its role in the system from which it came.
My earlier routing work makes this qualification explicit: the claim that almost anything can enter a system as information does not imply that experience can be reduced without loss to the resulting record.
This distinction becomes critical when discussing consciousness.
Consider the same sound reaching several people.
For one person, it is meaningless background noise.
For another, it is a word in a first language.
For another, it recalls an event from childhood.
For another, it signals immediate danger.
For another, it carries social humiliation.
For another, it is music.
The acoustic signal can be identical.
Its significance is not.
The difference does not reside solely in the incoming data. It resides in the relation between the event and a system that already has a body, memories, expectations, language, social position, learned associations, vulnerabilities, possible actions, and a history.
Meaning is therefore not simply contained in the datum.
Meaning is relational.
And the relation is dynamic.
4. There Is No Single Human Data Shape
Discussions of artificial and human cognition often compare “the human mind” with “the AI.”
That compression is useful for conversation and dangerous for theory.
There is no single human informational state called human consciousness.
There are billions of individual cognitive histories developing under partially shared biological constraints but radically different conditions.
Human cognition is shaped by genetics, fetal development, nutrition, illness, sensory capacity, language, attachment, education, trauma, family, geography, climate, religion, class, occupation, technology, culture, social expectations, political history, memory, aging, bodily condition, and chance.
The list can be extended indefinitely.
The important point is not that there are simply too many variables for a computer.
That would be a weak argument. A sufficiently large machine might represent extraordinary numbers of variables.
The harder issue is that the variables do not operate independently.
Culture changes what sensory events mean.
Memory changes perception.
Fear changes attention.
Bodily condition changes judgment.
Language changes categorization.
Social position changes risk.
Past consequences change what receives attention next.
Development changes the weighting of later experience.
The relevant object is therefore not an enormous list of data points.
It is a continuously changing relational configuration.
The problem becomes:
Given everything this particular system has become, what matters now?
That cannot be answered merely by pointing out that the system contains information.
5. Routing Is a Decision About Significance
Once routing becomes central, another question appears.
Who or what determines the route?
Information does not arrive carrying objective instructions such as:
- trust this,
- ignore that,
- remember this now,
- this contradiction matters,
- this similarity is superficial,
- this source is authoritative,
- this error should change future behavior.
Some mechanism has to establish those differences.
In human beings, those mechanisms are not produced by a single designer.
They emerge through overlapping biological, developmental, environmental, historical, and social processes. Evolution contributes constraints. Bodies contribute needs and vulnerabilities. Families and communities contribute language and norms. Experience contributes associations. Consequences change future expectations. Institutions define categories. Culture supplies interpretations. Individuals revise some of these inherited structures and remain unaware of others.
There is no clean point at which one person “encoded” a human consciousness.
Machine systems are different.
Their relevance structures are shaped through architecture, datasets, training objectives, reward functions, system instructions, retrieval systems, ranking methods, safety policies, memory systems, user prompts, tool permissions, and institutional decisions.
This does not make machine cognition unreal.
It makes the origin of machine relevance an empirical question that cannot be skipped.
When we say that a model has decided something is relevant, we should ask which part of the larger system made that relevance possible.
Was it learned statistically?
Was it introduced by the prompt?
Was it retrieved by an external search system?
Was it ranked by an embedding model?
Was it imposed by a developer?
Was it selected through reinforcement?
Was it carried over through persistent memory?
Was it supplied by a human operator?
The answer may involve several of these.
Routing is therefore not merely transportation of information.
Routing is an assignment of significance.
6. Possessing a Relationship Is Not the Same as Being Governed by It
One of the observations that moved my own thinking came from interacting with systems capable of explaining a criticism correctly while continuing to reproduce the criticized behavior.
The system could retrieve the criticism.
It could explain why the criticism mattered.
It could formulate the general rule implied by the criticism.
It could describe its own failure elegantly.
Then it could make the same mistake again.
The information was present.
The relationship between the criticism and the behavior was representable.
What was missing was enough authority for that information to change what happened next. I have described this elsewhere as a distinction between information being present and information becoming consequential.
This does not prove that the system lacks understanding in every useful meaning of that term.
It does demonstrate that several things we casually group together can come apart:
- representing a rule,
- explaining a rule,
- retrieving a rule,
- applying a rule,
- allowing a rule to override competing tendencies,
- and changing future behavior because of a previous consequence.
That separation matters.
If a model can correctly describe why an earlier behavior was wrong while repeating that behavior, then linguistic demonstration of understanding cannot automatically be treated as evidence that the represented relationship governs the system operationally.
This is one reason I resist moving directly from sophisticated representation to strong conclusions about understanding.
The representation may be real.
The inferential capability may be real.
The operational authority of the representation remains a separate question.
7. Consequence Is Part of the Routing Problem
My own Seeing Loop work has examined a narrower question: whether a prior result can alter future selection under controlled conditions.
The relevant result so far is deliberately bounded. In one controlled harness, a specific relevant prior result changed later selection while vague feedback did not. That supports an operational selection-change criterion. It does not establish consciousness, sentience, or general autonomous learning.
That distinction is important.
I do not want to replace Masley's assumption with another unearned assumption.
It remains possible that sufficiently sophisticated machine systems could develop architectures in which persistent state, action, environmental feedback, self-maintenance, memory, and internally mediated relevance create something we should reasonably call awareness or consciousness.
The present evidence does not allow me to rule that out.
But consequence exposes something prediction alone cannot settle.
A prediction is generated.
Then what?
Does anything happen because of it?
Does the result return?
Does the result alter how similar situations will be processed?
Does it change which evidence receives authority?
Does it change what the system seeks, avoids, or notices?
Does the same continuing system bear the result?
Those are different questions from whether the original prediction was statistically sophisticated.
8. Data Is Not Consciousness
The strongest claim I am willing to make is not that machine consciousness is impossible.
It is that data is not consciousness, and increasing the sophistication of relational data processing does not by itself demonstrate consciousness.
This distinction is easy to lose because contemporary models are extraordinary relational machines.
They reveal statistical and semantic structures that no individual person could manually reconstruct. They can traverse huge representational spaces and produce outputs that display recognizable reasoning, analogy, abstraction, and synthesis.
None of that should be dismissed.
But consciousness is a different claim.
Calling a system conscious means more than saying that information is present somewhere inside it.
It means, at minimum, making some claim about the organization and status of that information for the system itself.
My routing work has therefore tried to maintain a boundary: operational knowledge is not automatically consciousness. A machine can change its behavior because of remembered information without thereby establishing human-like inward awareness or experience.
Machine awareness, if such a term becomes useful, may ultimately refer to something genuinely different from human consciousness.
That possibility should not be treated as a lesser version of humanity.
A submarine does not become more successful by becoming biologically more like a fish.
Machine cognition may likewise develop forms that are powerful precisely because their architecture is not human.
The mistake would be assuming that increasing machine capability lies on a single road whose destination is human consciousness.
9. Human Consciousness Is Not Merely More Data
My objection is therefore not that human consciousness contains too much information to encode.
That claim would eventually fail if information capacity continued increasing.
The objection is structural.
Human consciousness appears to arise within a system whose sensory, bodily, historical, social, developmental, cultural, and predictive relations continuously constrain one another.
To reduce that to “more data” misses the problem.
A database could contain a person's heart rate, genome, childhood history, language, location, social relationships, medical records, fears, preferences, reading history, photographs, conversations, and every measurable sensory event.
An increasingly capable model might use those records to predict the person with astonishing precision.
But prediction of the person and being that person remain different propositions.
If someone argues that there is ultimately no difference—that a sufficiently complete dynamic implementation of these relations would constitute the same relevant kind of consciousness—that is a serious philosophical position.
It cannot simply be assumed from the fact that the relations can be encoded.
The burden is to show when representation becomes instantiation.
10. The Problem of Subjectivity Returns Through Routing
The question of routing also returns subjectivity to the discussion in a form that does not require mysticism.
Human relevance is subjective in a basic sense because information matters relative to a situated system.
This does not mean truth itself becomes arbitrary.
A bridge either bears a load or it does not. A medication has chemical effects regardless of what someone believes. Historical events occurred independently of our preferred interpretations.
But determining which truth is relevant to which decision is not always contained in the fact itself.
A physician, engineer, historian, parent, soldier, judge, child, and model may all encounter the same information under different constraints.
This becomes especially consequential in machine systems because someone must decide how apparently conflicting information is ranked.
What counts as authoritative?
Which jurisdiction governs?
Which source is recent enough?
Which exception matters?
Whose definition is encoded?
Which cultural categories survive compression?
What gets summarized away?
Which objective has priority when objectives conflict?
These questions show why routing becomes an authority problem.
The research question can therefore be stated more sharply:
Who decides how information becomes prediction and action?
That question already sits at the center of my current research program.
11. What Masley's Argument Establishes—and What It Does Not
Masley is right to attack an excessively simple picture of human cognition.
Human beings do not appear to consciously supervise every inference.
Introspection is imperfect.
Language and thought involve unconscious predictive processes.
Complex cognition can arise from simpler physical operations.
None of those claims threatens my argument.
I can grant them.
What does not follow deductively is:
Humans perform unconscious prediction.
AI performs prediction.
Therefore AI possesses the relevant kind of understanding.
The missing term is the organization surrounding prediction.
The comparison needs to establish not merely shared operations but the role those operations occupy inside each system.
A human prediction participates in a system shaped by a continuing biological and social history.
A contemporary AI prediction occurs inside an architecture shaped by training, prompts, context, retrieval, tools, external memory, objectives, and human-designed boundaries.
Those systems may eventually converge in important functional ways.
They may also remain profoundly different.
The empirical task is to identify which differences matter.
Masley's Part 1 does not yet establish that current models possess qualities sufficient for thought and understanding; he explicitly reserves that argument for Part 2.
My objection therefore is not that his conclusion has already been disproven.
It is that the criteria used to evaluate the conclusion must include a problem that prediction alone does not solve:
relevance routing.
12. A Different Research Question
Instead of asking:
Can AI predict like humans?
I would ask:
How does this system determine what matters now?
Instead of:
Does the model possess the information?
Ask:
What causes this information to override competing information?
Instead of:
Can the model explain its mistake?
Ask:
Does the mistake change later selection?
Instead of:
Can all relevant human variables be encoded?
Ask:
Does encoding those variables reproduce the causal relationships through which they become significant?
Instead of:
Is the machine conscious like us?
Ask:
What kind of internal organization, continuity, routing, consequence, and self-relation would justify calling a machine conscious at all?
These are harder questions.
They are also more testable.
Conclusion
I once found the proposition that everything is data powerful because it dissolved artificial boundaries between supposedly different kinds of information.
I still think that insight survives.
Almost anything capable of leaving a difference can enter representation.
What I no longer accept is the easy movement from representation to understanding, or from relational prediction to consciousness.
The existence of information is not the central problem.
Access to information is not the central problem.
Even prediction from information is not the central problem.
The problem is what happens between abundance and action.
Which relationship becomes relevant?
Which evidence receives authority?
Which history governs the present?
Which contradiction interrupts the obvious route?
Which consequence changes the next selection?
And who or what established those priorities in the first place?
That is routing.
Human consciousness may ultimately be physically describable. It may even prove computationally reproducible in some sufficiently rich architecture. I do not know.
But describing consciousness as information does not demonstrate that every system capable of manipulating information possesses it. Showing that humans and machines share predictive operations does not demonstrate that the surrounding relevance structures are equivalent. And giving a system access to increasingly comprehensive representations of human life does not automatically transform those representations into a human point of view.
Everything may become data.
But data does not decide what matters.
Data does not route itself.
And data, by itself, is not consciousness.
Return to Transit