Blog · AI and Search
What Does a Word Mean?
On Wittgenstein, Firth, Shannon, and Plato, and why the Platonic Representation Hypothesis is either the most hopeful or the most dangerous development in the history of knowledge
There is a question that has occupied philosophers, linguists, and mathematicians for most of the last century, a question so fundamental that it tends to get lost in the noise of more immediately urgent debates. It is this: where does meaning come from?
The answer you give to that question turns out to determine almost everything about how you think about artificial intelligence, and specifically about the risk embedded in one of the most extraordinary findings in recent AI research.
Three Thinkers, One Insight
In 1953, the Austrian philosopher Ludwig Wittgenstein published Philosophical Investigations, completing a revolution in his own thinking that had taken three decades. The early Wittgenstein had believed that language worked by picturing reality: words and sentences mapped onto facts about the world in a logical structure, and meaning was a matter of correspondence between language and the states of affairs it described.
The late Wittgenstein demolished this view, including his own earlier version of it.
Meaning, he argued, is not a property of words. It is not something that attaches to a symbol the way a label attaches to a jar. The meaning of a word is its use, and use is always a social practice: governed by rules, embedded in contexts, sustained by a community of speakers who enforce conventions through shared activity. He called these contexts “language games,” and he meant something precise by the analogy: just as the meaning of a move in chess depends entirely on the rules of chess and the game being played, the meaning of a word depends entirely on the practice in which it is embedded.
One consequence of this view is that private meaning is impossible. There can be no such thing as a word that means something to one person alone, because meaning requires the possibility of correction, the shared standards that allow speakers to say “that’s not how we use that word.” Meaning is inherently negotiated. It is a social achievement, not a private act.
In 1957, four years after Philosophical Investigations was published, the British linguist J.R. Firth offered a formulation that would become one of the most generative sentences in the history of linguistics: “You shall know a word by the company it keeps.”
Firth’s insight was distributional. Words, he argued, do not carry meaning in isolation. Their meaning is constituted by the statistical pattern of their co-occurrences with other words, by the semantic neighbourhood they inhabit across millions of acts of human usage. “Dog” means what it means because of the particular constellation of contexts in which speakers have used it, alongside “bark” and “pet” and “loyal” and “leash,” and not alongside “theorem” or “monsoon.”
The parallel with Wittgenstein is exact, expressed in different vocabulary. Both were saying that meaning is not intrinsic to symbols. It emerges from patterns of use in a linguistic community. The word carries no meaning except what is deposited there by the aggregate practice of speakers over time.
In 1948, nine years before Firth and five years before Wittgenstein’s Philosophical Investigations, the mathematician Claude Shannon published “A Mathematical Theory of Communication,” the paper that founded information theory and that became, quietly and retrospectively, the mathematical substrate of everything we now call AI.
Shannon’s central insight was about information, not meaning, but the conceptual structure is identical. The information content of a symbol, Shannon showed, is entirely determined by its probability given its context: how surprising it is relative to the distribution of symbols around it. A highly predictable word carries very little information. A surprising one carries a great deal. Meaning (in the informational sense) is not a property of the symbol itself but a function of its relationship to everything surrounding it.
Three thinkers. Three disciplines. Three vocabularies. One underlying claim: that meaning, information, and understanding are not intrinsic properties of things. They are relational. They emerge from statistical patterns in communities of use.
This convergence is not a coincidence. It is a recognition, arrived at independently, that language is not a code for transmitting pre-formed meanings but a social technology for constructing shared understanding through repeated, negotiated interaction.
The MIT Hypothesis
MIT researchers Huh, Cheung, Wang, and Isola have now proposed something that looks, from a philosophical vantage point, like the logical terminus of this tradition.
Their Platonic Representation Hypothesis (arXiv:2405.07987, covered here with characteristic depth in Quanta Magazine in January) observes that large language models trained on entirely different data types, vision models trained on images, language models trained on text, are converging on the same internal representation of reality at a deep mathematical level. The stronger the model, the more pronounced the convergence.
What the models appear to be converging on is something like an idealised geometric skeleton of meaning: a high-dimensional structure in which the spatial relationships between concepts encode their semantic relationships. This is, at a mathematical level, precisely what Firth described and Shannon formalised: a representation in which the meaning of anything is constituted entirely by its relationship to everything else.
Plato, in the allegory of the cave, described prisoners who see only shadows on a wall, imperfect projections of the real objects that cast them. His Forms were the ideal structures behind the shadows, the true nature of things that the philosopher’s task was to uncover. The MIT researchers are proposing that AI models are independently triangulating toward something structurally similar: an idealised representation of meaning that underlies the particular, imperfect instances of language they were trained on.
This is a genuinely remarkable finding. If correct, it offers an explanation for the qualitative improvement in AI capability that many people are currently observing: models that have internalised a richer, more geometrically accurate structure of meaning handle abstraction and ambiguity better, because they are operating closer to the underlying shape of knowledge rather than the surface of particular texts.
What Wittgenstein Would Ask
But here is where Wittgenstein’s contribution becomes not merely historically interesting but urgently relevant.
Wittgenstein was not simply making a descriptive claim about how language happens to work. He was making a normative claim about what meaning requires. And what it requires, he insisted, is a community of use. A linguistic practice sustained by real speakers engaged in real activities with real stakes. The language game is not an abstraction. It is embedded in what he called “forms of life,” the concrete, embodied practices that give language its grip on the world.
The question Wittgenstein would ask about the Platonic Representation Hypothesis is not whether the geometric structure the models are converging on is internally coherent. It is: what community of practice produced the data that structure was derived from?
This is not a naive question. It cuts to the heart of the problem.
The MIT researchers observe that models are converging on an idealised structure of meaning. But Wittgenstein’s analysis tells us that any such structure can only be as valid as the human practice it was derived from. The Platonic Form, in this framework, is not an independent ideal that language imperfectly tracks. It is a distillation of the aggregate practice of a linguistic community. Its authority derives entirely from the authenticity of that practice.
And this is precisely what is now under threat.
Shadows of Shadows
We are at an unusual moment in the history of knowledge production. The AI systems being trained on human-generated content are now generating a substantial and growing proportion of new text on the internet. Research on what has been called “model collapse” demonstrates that training on AI-generated rather than human-generated data produces progressive degradation: loss of distributional nuance, compression of semantic range, subtle but compounding distortions in the relationship between concepts.
Expressed in Wittgenstein’s terms: the language games that new models are learning are no longer anchored in human forms of life. They are learning from the outputs of previous models, which were themselves learning from the outputs of previous models, in a recursion that is severing the connection between the distributional patterns in the data and the human practices that originally gave those patterns their meaning.
Firth’s principle still holds: the model knows a word by the company it keeps. But the company those words are keeping is increasingly other AI-generated words, produced by systems trained on AI-generated words, in an echo chamber that is geometrically stable and humanly unmoored.
Shannon’s information theory still applies: the system is learning the probability distributions of its training corpus. But the corpus is increasingly not a record of human communication. It is a record of statistical imitation of human communication, several generations removed from its source.
The Platonic Form these models converge on will look like the ideal structure of meaning. It will be internally consistent, geometrically coherent, and confidently expressed. But it will be the ideal structure of a community of practice that no longer exists: a distillation of an echo, mistaken for the original sound.
The Particular Danger of Structural Corruption
There is a temptation to think that this problem will be self-correcting. More capable models will be better at distinguishing signal from noise, at detecting the difference between authentic human knowledge and synthetic imitation of it. This intuition is wrong in a specific and important way.
The Platonic Representation Hypothesis describes models extracting not facts but structure. The geometric relationships between concepts that encode meaning at a foundational level. When this process works correctly, the extracted structure is a valid distillation of authentic human understanding.
But when the input is corrupted, the model does not extract a degraded version of the correct structure. It extracts a well-formed, geometrically stable structure that is simply wrong. And because the error is structural rather than factual, it will not manifest as identifiable mistakes. It will manifest as a subtle but pervasive distortion in the relationships between concepts, in the weight given to certain associations, in the suppression of semantic nuance that doesn’t survive the extraction process.
This is harder to detect than factual error, and harder to correct, because the entire semantic space of the model is organised around it. It is the difference between a building with a cracked window and a building with a flawed foundation. The former is visible and fixable. The latter will hold until it doesn’t.
The smarter the model, the more load-bearing its foundational representations become. Which means the models most capable of producing the qualitative improvement Schumer and others are observing are also the models most vulnerable to this kind of structural corruption, because they are the ones whose outputs are most determined by the integrity of their underlying representational geometry.
The Human in the Loop Is Not Optional
Wittgenstein’s analysis implies something that tends to get dismissed as sentimentality in conversations about AI development: the presence of human judgment in the production and curation of knowledge is not an optional feature of the knowledge ecosystem. It is the mechanism by which meaning is grounded.
The journalist who verifies before publishing is not just avoiding individual errors. She is participating in the social practice that makes the concept of “accuracy” meaningful, that maintains the language game in which the word “fact” has the weight it carries.
The editor who enforces standards is not just improving individual texts. He is sustaining the shared conventions that allow readers to calibrate their trust in sources, that maintain the distinction between knowledge and assertion.
The researcher who replicates and the practitioner who tests are not just correcting individual claims. They are maintaining the connection between the linguistic community and the world it describes, the grounding relationship without which Wittgenstein’s language games collapse into private fantasy.
When these functions are economically undermined (by systems that train on content without compensation, displacing the audience and revenue that makes quality production viable), the damage is not merely ethical. It is epistemic. The social infrastructure of meaning production is being dismantled at precisely the moment when the systems that depend on it most are becoming most powerful.
What This Means
The Platonic Representation Hypothesis is, in one sense, deeply reassuring. The convergence it describes suggests that AI models are not merely accumulating facts but internalising something closer to the structure of understanding. The qualitative improvement people are observing is real, and it has a principled explanation.
But the same hypothesis that makes this reassuring also makes the failure mode it enables extremely serious. A model that converges on a corrupted structure of meaning does not produce corrupted-looking output. It produces authoritative, coherent, confidently expressed output that is, at a foundational level, wrong. And because the corruption is structural rather than factual, it resists the normal mechanisms of correction.
Wittgenstein spent the second half of his philosophical career trying to show that meaning cannot be privatised, that it requires a community, that without the shared practice of correction and use, language becomes untethered from the world. His analysis reads, in retrospect, like a warning addressed to a moment he could not have anticipated.
The community that grounds meaning in language is not a luxury. It is the condition of meaning itself. And the economic and institutional structures that sustain that community are not peripheral to the problem of AI integrity. They are central to it.
Further reading: Huh et al., “The Platonic Representation Hypothesis,” arXiv:2405.07987. Covered in Quanta Magazine, January 2026 (link in comments). Wittgenstein, “Philosophical Investigations” (1953). Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal (1948).
¹ The convergence between Firth and Shannon is striking and underappreciated. Both were published within a decade of each other, both describe meaning and information as purely relational properties determined by distributional context, and neither appears to have been directly aware of the other’s work. That the mathematical architecture of modern AI rests on both simultaneously is less a coincidence than a recognition, after the fact, that they had independently formalised the same deep truth about the nature of meaning.
