AI is no-body — embodied knowledge and misinformation
While I make the dough (a nod to my second essay) for part 2 of the series based on Ingold's "Being Alive" (a nod to my first essay), I dug up an essay I started writing in April 2026, where I've been trying to work out an intuition I have that AI cannot produce misinformation because it cannot produce knowledge either.
Upon reading this essay before hitting the "publish" button, I see the tone is different from my other essays, which are more written from a first person point of view, in a phenomenological vein. This one reads argumentative, analytic in style (with a few humorous licenses here and there), which makes sense because I am arguing for a disputed position. I hope that I have been able to make a good case with solid argumentation and without being too dry.
Let's roll.
Misinformation is not new
For some time in the fourteenth to sixteenth centuries, the taxonomy of human knowledge contained a gourd-like fruit which harbored tiny lambs inside, complete with wool, flesh and blood. This was known as the Vegetable Lamb of Tartary, and it was written about by eminent naturalists, some of which claimed to have examined or even eaten the fruit. Others reported it as true based on the testimony of credible witnesses.
Misinformation is not a particularly modern problem. It has probably been around since humans have been around, telling each other stories about their experiences — which, let’s be honest, is anything but “unmediated facts”. It is probably inherent in the nature of how information flows: slight distortions, exaggerations, things left out or added, not necessarily by conscious choice. The above, for instance, is an account I read back in 2020 in a book by two academics, my credible witnesses, whom I assume engaged with the primary source — or at least read someone who did.
This came to my mind as I read a Nature piece on how AI regurgitated fake information about a made-up disease called bixonimania. The article is a member of the class of opinion pieces saying “new technology X introduces new problem Y for humanity”. As is often the case in these pieces, the problem it points its finger at is not actually new — most new technologies do introduce new problems, but they are the kind of unsexy problem that ends up analyzed in longer tomes.
The article is about an experiment where researchers uploaded two studies about bixonimania — a fake ocular condition caused by excessive exposure to blue light from screens —to a preprint server. They filled the text with red flags such as to alert any human with eyes and a brain that, no, this is not a thing. The scientists wanted to see if chatbots would start bringing up this information to people who ostensibly asked about bixonimania or hyperpigmentation of the eyelids, one of its “symptoms”.
Now, if you tell me a story about elephants in Africa, then ask me to name a large animal, I will not say whale. The glaring problem with the bixonimania experiment is the non-independence between intervention and test, such that it cannot distinguish between "the LLM has been epistemically corrupted and now claims bixonimania is real" and "the LLM is doing retrieval on the query string and returning content that matches it." The experiment fails to show us anything new that we could not have inferred from knowing how LLMs are trained and where they get their data from.
The flaw in the interpretation of this experiment reveals a misdirection of concern: we are blaming a technology for a feature of knowledge production that long predates it. Knowledge was never a "pure" thing "out there", waiting to be mined. So what is knowledge then?
Knowledge is not out there
Knowledge is skilled, reliable engagement of an organism (who has a niche and a form of life) with an affordance-landscape. Human knowledge is embodied and embedded in the sociocultural practices of that form of life, and its production occurs there. Therefore, it isn’t a property of anything; it’s not out there to be mined.
OK, that was a mouthful. Let's unpack.
A flat surface of soil on Earth affords certain organisms — namely, those who are quadrupeds or bipeds — to walk upright. A flat surface of water, on the other hand, does not afford those same organisms to walk upright, due to water’s physical properties. That same surface, however, affords those organisms to quench their thirst or cool themselves down. If those organisms learn, by whatever means, that they can reliably walk on soil surfaces and drink water, one can say they know soil is for walking and water is for drinking.
As I said above, I take knowledge to be skilled, reliable engagement with an affordance-landscape: an organism is attuned to and engages with its environment when its perception-action coupling reliably enables adaptive action given the affordances available to it (Gibson 1979, Rietveld & Kiverstein 2014). Affordances are, simply put, possibilities for action provided by the environment to an organism.
Action is not only bodily motion. When you perform the action of clicking a button on your computer you might also be performing the action “sending an email”. Affordances depend on the skills of the organism: if, for some reason, all of humanity loses the knowledge of how to click on computer buttons, computers would cease to afford us sending emails (if I may be allowed the contrived parody of Rietveld and Kiverstein’s piano example).
For humans, the affordance-landscape is very 'rich': it is material but also linguistic, cultural, social, and institutional. As Gibson said, “at the highest level, when vocalization becomes speech and human manufactured displays become images, pictures, and writing, the affordances of human behavior are staggering. (p. 137)” The aspects of the affordance-landscape and the skills required to engage with them are constituted within the human form of life, where the form of life is understood to be “patterns in an animal’s behavior, i.e., relatively stable and regular ways of doing things”.
The regular practices of a community are part of the human form of life, and they come to be through evolutionary, developmental, and cultural history. Here's my evidence for it; take, for instance, cows’ milk. Cow’s milk affords most humans the ability to drink and derive nourishment from it. Cow's milk drinkability is derived from the deeper mammalian fact of infant nursing on maternal milk. The perceptual and cognitive apparatus for engaging with the features of milk’s drinkability and nourishment-giving had a long evolutionary history in humans. However, the gene variant that allows for the digestion of milk’s lactose in adulthood was rare; the most common variant turned off lactase production in our adult ancestors since only infants drank mother’s milk. Some human populations developed the practice of herding and dairying certain milk-producing animals, and giving milk to their offspring. Early-life exposure tunes taste preferences and gut microbiome, predisposing the tolerance for milk. A gene variant that enabled the digestion of lactose into adulthood allowed for the differential reproduction and survival of those individuals who possessed it, and through gene-culture co-evolution, human populations with long dairying traditions predominantly have that variant of the gene. Thus, due to a specific evolutionary, developmental, and cultural history, milk affords drinking to humans.
The fact that we see milk and, if thirsty, drink it, can be said to be due to our knowledge that milk is drinkable. A proposition such as "milk comes from cows and we can drink it" is knowledge to the extent that it agrees with the sociocultural practices of a community, and allows an individual to skillfully engage with the affordances of milk. Note how the proposition "milk comes from lions and we can drink it" fails to agree with the sociocultural practices of human communities and also fails to allow skillful engagement with the affordances of milk in the human niche, as does the proposition "milk comes from cows and is poisonous to humans". There's one single normative standard, not two — skillful engagement with the affordances of milk in the human niche, and that niche is sociomaterial. However, an important distinction to keep is that the second proposition fails in a way the first one doesn’t: if an entire community abstains from milk drinking due to fear of milk poisoning, milk still affords drinking and nourishment — the sociomaterial niche includes a material kickback that community agreement cannot revise.
Let’s continue with humans. We have manipulated the different substances and objects on Earth to create new artifacts which, as Gibson notes, change what the world affords us. Humans have also created social-material hybrids such as institutions, which are consolidated by communal, repetitive, and regulated sets of practices that take place in certain locations. A typical human institution is the school: usually a building where large numbers of children and young adults convene with a small number of adults. The latter are called teachers and practice the sharing of certain facts about the world with the former, called students, for a specified number of hours every week day. The school affords the students many things, including socialization with peers, learning true propositions describing states of the world, and getting a piece of paper, called diploma, which then affords them access to other institutions for further learning or earning a living. Even as the material substrate changes, the identity survives in the pattern of relationships, which reconstitute the material substrate that the reproduction of the pattern relies upon.
Knowledge, in this frame, is continuous with skilled engagement with affordances, not a separate cognitive achievement.
Misinformation
If knowledge is, as we said, skillful engagement of a life-form with affordances in ways that allow for a good grip of a situation — and one can think of expertise as an optimal grip on a kind of situation, such as an expert piano player’s performance of Liszt's arrangement of Beethoven's 9th — then misinformation can be conceived as the opposite, as hindering skillful engagement with affordances, which leads to an inappropriate action-affordance coupling and reduces the individual’s grip on a situation. We discussed above three proposition examples about cow’s milk, one that was correct and two others that count as misinformation. If an individual acts based on either of them, they will likely incur a worsening of their situation — think of someone who is dehydrated and upon seeing milk does not drink it due to fear of poisoning. Our written answer to the question “What is the plot of The Iliad?” is a better or worse skilled engagement with the affordance of a test, it allows you to get a better or worse grip on the pressing situation of passing said test — the situation and the affordance being entirely created by human sociocultural practices. In any normative domain, a system that can produce appropriate outputs can also produce inappropriate ones.
The Vegetable Lamb of Tartary may seem outrageous to our modern ears, and it might have raised suspicion in medieval naturalists as well. The extent to which this piece of misinformation was actually harmful depends on if and how it was used to make decisions and take action upon the world. Human propositional knowledge is embedded in a network of sociocultural practices that, when affording successful engagement with the world, is validated and reinforced by the network of practices it belongs to. A certain piece of propositional knowledge that implicates and is implicated by others that are situated close to it and which, in their turn, have successfully survived in the practices of the life-form is more likely to be successful itself and serve as a basis for action upon the world.
The Vegetable Lamb of Tartary, on the other hand, contradicted established facts about plants, animals, and their life cycle. Despite some propagation across natural history books, it did not endure for long, nor did it have a lasting effect in the history of human knowledge. The analogy to genetics here is too illuminating to be bypassed: a "neutral" mutation, something that has no effect on the organism's survival, is carried on, survives and reproduces quite freely, drifting either to extinction or fixation due to the stochastic sampling of alleles across generations, or until some change in the environment exposes that mutation to natural selection. But unlike natural selection, the forces that test a belief are not mind-independent, they are themselves the skilled practices of a form of life. Most "false propositions" drift through untested because they never get coupled to an action that would test them. And like neutral mutations, they often drift, either to fixation or out of existence. The Vegetable Lamb died of obscurity, not refutation. The corollary of this is, then, that most misinformation is likely quite harmless. It only becomes harmful when it is used in an engagement with an affordance. How much of our propositional "knowledge" truly affords skillful engagement with the world? How much of it has really been "tested"?
Can machines produce knowledge?
In this view, knowledge is not merely propositional, which means that animals produce knowledge — though without the socio-cultural layer that distinguishes human knowledge production. The fact that knowledge production is embodied, that it is the skilled engagement of a form-of-life with affordances, does not per se disqualify machines from producing knowledge. By embodiment I mean having a body that affords action-possibilities, not biological life as such. (Although I am partial to the carbon molecule of life, I am not a carbon fetishist.)
However, the embodiment of knowledge production does disqualify machines from autonomously producing the kind of knowledge that matters to human beings. Knowledge is only knowledge relative to a form-of-life, since the abilities required to engage with affordances are themselves form-of-life dependent: pigeon knowledge is certainly knowledge to pigeons, but not all of it is knowledge to humans. LLMs can participate in knowledge production for humans when they produce, e.g., text that is used by a human life form to skillfully engage with affordances. The counterpoint to this is that when they produce text that is used by humans and does not result in skillful engagement with affordances, they have participated in producing misinformation for humans. Given the probabilistic way LLMs produce text from data it was trained on — data which was produced by humans in the first place — it is a given that they will produce both knowledge and misinformation. This is a crucial point, that knowledge and information are normative: what counts as correct or incorrect information are products of embodied activity of a form-of-life engaging with affordances. Whether LLMs produce “LLM knowledge” is a matter of whether they skillfully engage with their affordances — a question which is beyond the ambitions of this essay to tackle.
For machines to produce knowledge for humans, therefore, they need to be themselves embedded in the human life form’s knowledge production practices. This embeddedness occurs with any tool that, as used by humans, extends human cognition and refines the skilled engagement with affordances. In the extended mind framework, tools are scaffolds constitutive to cognition, which is the background on which knowledge is produced. The role for GenAI in knowledge production is that of a cognitive extension, that is, part of our abilities to perform relevant intentional actions are extended, augmented or offloaded onto the models. When integrated into human practice, GenAI becomes part of the human affordance-landscape — a new kind of epistemic tool, like writing, libraries, citation indices, search engines. The human-in-the-loop is not a master using a tool, he is extending his cognition as part of the knowledge production process in a critical partnership with the model. The unit is the coupling of human and tool.
Can machines produce misinformation?
Given our positive account of knowledge — of what knowledge is — and our negative account by way of misinformation, we can deduce that LLMs do not produce misinformation either, as they don't produce knowledge. Misinformation, like knowledge, is also not a thing out there to be mined. It is not a property of a string of text in whatever medium — newspaper or screen. Rather, it lives or dies, like knowledge, in the practices of a form of life. In fact, the dichotomy knowledge–misinformation proves to be a false one given our view; they are not separate, mutually exclusive categories but co-constituted outcomes of the same skilled action-affordance coupling — the same engagement that can succeed can fail, it depends on the state of the world and the practices of the form of life.
Gibson, James J. The Ecological Approach to Visual Perception. 1979.
O'Connor, Cailin, and James Owen Weatherall. The Misinformation Age: How False Beliefs Spread. New Haven: Yale University Press, 2019.
Rietveld, Erik, and Julian Kiverstein. “A Rich Landscape of Affordances.” Ecological Psychology 26, no. 4 (2014): 325–52. https://doi.org/10.1080/10407413.2014.958035.
