AI can make the accumulated culture of humanity talk back to us. Books, theories, images and discoveries no longer remain silent until we learn how to search them. They can answer questions, adopt different perspectives, generate examples and accompany an investigation. But the question must still disturb us, the decision must still implicate us, and the step must still be ours. Learning does not end when the answer appears. It ends when the encounter with that answer has changed the learner.
Imagine that you are taking an online course for a new role. A difficult situation appears on screen: a client is upset, the available information is incomplete, and every possible decision carries a risk.
Before choosing, you open an AI assistant. In seconds, it identifies the problem, recommends the best response and writes the message for you.
And yet, somewhere between the question and the answer, something went amiss. Knowledge was imparted, but no skill was developed and no proficiency attained.
No wonder people increasingly fear being replaced by AI. We’re using it the wrong way.
Learning is a loop, not a transfer
We often talk about learning as if knowledge were an object that could be transferred from one place to another. The instructor possesses it, the course contains it, and the learner receives it.
This model explains why so many online courses resemble presentations: information is divided into slides, decorated with images and followed by a quiz. If the learner reaches the final screen and remembers enough correct answers, the system declares success.
Living systems learn differently. They form expectations, act upon their environment, observe what happens and adjust.
This is a cybernetic loop. Founded by mathematician Norbert Wiener in 1948, cybernetics studies how biological, mechanical and social systems regulate themselves through communication and feedback. A thermostat, an animal searching for food and a child learning to walk are very different systems. But each modifies its behavior by comparing what happens with what it expected or desired.
Developmental psychology describes a similar process. For Jean Piaget, children do not simply copy reality into their minds. They interpret it using the structures they already possess. When experience does not fit, they must incorporate it into an existing mental model or modify the model itself.
Active inference offers a contemporary version of this idea. According to Karl Friston, living agents continually anticipate, explore and act to reduce prediction error—what he calls surprise. By minimizing surprise, they resist disorder and preserve the conditions that allow them to remain alive.
Learning requires this encounter between expectation and reality. A question, contradiction or unexpected result creates a space in which the learner must look again, test an idea or reorganize a mental model.
In What Is Instructional Design, we followed the movement from reinforcement and repetition toward constructivism, mental models and generative learning. With generative AI, the question of how we build mental models of reality—and therefore keep learning about the world—has become more urgent.
From external tools to digital symbiosis
A tool is any amplifying device that extends our capacity to perceive, remember, calculate, act or exercise control over the environment. Tools do more than make tasks easier: when they become woven into our habits, they virtually become extensions of the self.
Since the dawn of culture, we have extended our cognitive abilities through tools. Language lets us share concepts. Writing stores ideas beyond biological memory. Maps, diagrams and calculators make otherwise impossible operations manageable.
The computer was not imagined merely as a machine that would think instead of us. It could become part of a larger cognitive system. Andy Clark and David Chalmers later called this the extended mind: the symbols, notebooks and devices with which we interact can become functional parts of our cognitive processes.
Culture has always stored—or fossilized—human experience in language, symbols, books, diagrams and art. Generative AI forms a new layer of extended cognition because it lets that accumulated culture talk back: it can reframe a question, compare alternatives, generate counterarguments, simulate consequences and reveal patterns that might otherwise remain invisible. The crucial question is whether this dialogue enlarges the learner’s agency or substitutes for it.
Augmentation or replacement?
A device does not become a learning tool merely because we use it. GPS navigation offers a familiar example. It increases our control over the environment: we can reach an unfamiliar destination quickly, avoid congestion and recover from a wrong turn. For transportation, this is an extraordinary augmentation.
But turn-by-turn instructions remove the need to observe landmarks, compare routes and construct an internal map. Research has associated habitual GPS use with poorer spatial memory during later self-guided navigation and reduced reliance on hippocampal spatial strategies. Scientific Reports
The contrast with London taxi drivers is revealing. Drivers who successfully learned the city’s complex layout—the famous Knowledge of London—showed structural changes in the posterior hippocampus after years of training. Their brains changed because they had to construct and use an internal map. Longitudinal study
The lesson is not that we should abandon GPS. It is that a tool optimized for arrival may not be optimized for learning the territory.
AI becomes a learning tool when it extends the learner’s ability to observe, question, compare, remember, simulate and test. If it independently performs the capability the learner was supposed to develop, augmentation gives way to replacement.
A recent experiment with nearly one thousand secondary-school mathematics students makes the same distinction visible. Researchers compared conventional learning resources, a general GPT-4 interface and a pedagogically constrained AI tutor.
Both AI systems improved performance while students could use them. But when the assistance was removed, students who had used the general interface performed 17% worse than the control group. This negative effect was largely mitigated by the constrained version of the same tutor, which provided teacher-designed hints and encouraged students to attempt the problems themselves. Read the study
The general interface acted as a cognitive crutch: it carried the immediate problem-solving load without building the underlying capability. The constrained tutor functioned more like scaffolding—a lattice around which a climbing plant can grow.
Generative incompleteness
Most educational technology promises completeness. Every concept will be explained, every doubt resolved and every obstacle removed. The ideal experience is imagined as a frictionless path from ignorance to knowledge.
Generative incompleteness makes a more radical proposal: genuine learning requires carefully designed spaces that the learner must complete.
Science advances because an observation does not fit the accepted explanation. Philosophy begins when a familiar concept reveals a contradiction. Art emerges from the tension between an intention and what the material allows. In each case, incompleteness is not an error waiting to be corrected by an authority. It is an invitation to look closer, ask a better question and expand the boundaries of a mental model.
Formal knowledge is usually an abstraction from experience. A scientific law compresses many observations and experiments. A professional procedure condenses years of trial, error and judgment. A philosophical concept gives language to distinctions first encountered in life.
Abstraction makes knowledge portable, but it also separates it from the situations that originally gave it meaning. The map is not the territory.
Learning must reverse that compression. It must embody formal knowledge again in perception, action, consequence and reflection.
A well-designed course is not complete because it says everything. It is complete because the learner can perceive what is missing, detect inconsistencies, distinguish nuances and create a mental model that survives the boundaries of the learning experience.
The goal is not to eliminate uncertainty or temporary discomfort, but to make them productive. Knowledge becomes embodied when the learner can use it to perceive, decide and act.
Instructional design must construct a learning loop around the learner’s development, not merely around a corpus of knowledge. What challenges will they encounter? What action can they take? What consequences follow? What feedback will help them revise their model? What evidence will guide the next decision?
Consider two versions of a compliance course. A conventional version presents definitions and rules, then administers a multiple-choice quiz. A generative version presents a plausible workplace situation with incomplete information. The learner must identify a potential conflict, decide what questions to ask, consult the right resources and choose a course of action.
Those consequences become feedback: not merely a verdict on the decision, but an examination of the reasoning and assumptions behind it.
Real learning is challenging
In Captivate to Teach, we explored adventure, missions, discovery, rules, risks, rewards and self-actualization. These elements are sometimes treated as decorations added to otherwise conventional content. Their deeper educational value is that they sustain generative learning.
A mission gives direction to action. Rules make the environment intelligible. Risk makes decisions consequential. Discovery rewards curiosity. Feedback lets the learner revise a strategy. A coherent story connects individual choices with a meaningful whole.
We don’t explore our environment only to obtain rewards. We also explore to understand, develop mastery, belong to a group and become a certain kind of person—to learn how to play the game of life, discover our goals and values, and become who we are.
Discovery should nevertheless be guided. Novices cannot efficiently rediscover every principle. Explanations, demonstrations and worked examples remain essential. The art lies in deciding when to explain, when to ask, when to demonstrate and when to let the learner struggle a little longer by carefully weaponizing generative incompleteness.
The master or tutor has traditionally regulated this encounter. Following the example of Socrates, a good teacher does not simply deliver conclusions; they listen to the learner’s reasoning, ask questions that expose contradictions, offer a clue when frustration stops being productive, and step back when the learner can continue alone. In Socratic dialogue, the question is not a detour from teaching. It is the instrument that returns the learner to the problem with greater attention and responsibility.
Digital interactivity gives this ancient pedagogical relationship a new medium. Our article on Interactivity described a simple script: if the learner does A, the system responds with B. In a traditional branching scenario, B is hardcoded into the program. With generative AI, the response can become sensitive to the learner’s language, reasoning, history and misconceptions.
Generative interactivity can become a Socratic dialogue: it listens before responding, then returns the learner to the problem with a better question.
From AI Assistant to socratic tutor
An educational AI should not behave like a search box with a friendly face. It should listen for evidence of learning.
Before giving an answer, it can ask the learner to make a prediction. When the learner explains, it can identify a hidden misconception. Rather than correcting everything immediately, it can provide the smallest useful hint. Once the learner reaches a solution, it can change the context and test whether the learner can transfer the principle.
Its system instructions should enforce generative incompleteness: ask for an attempt before revealing a solution; respond to reasoning rather than only the final answer; use questions, counterexamples and graduated hints; generate variations; and reduce assistance as competence grows.
This is where solutions such as CourseMentor can extend the traditional LMS. Through navigation events, course context, conversation history and records of learning challenges, the tutor can listen to the stream of interactions rather than treating every question as an isolated request.
This produces richer learning telemetry. What did the learner initially predict? Which evidence changed their mind? How many hints were needed? Which misconception reappeared? Can they recognize the same principle when the surface details change? Does assistance decrease over time?
The learning adventure cannot be delegated
Stephen Wolfram uses the term computational irreducibility for processes whose outcomes cannot be reached by skipping intermediate states. Sometimes, he writes, “the only way to ‘see how they happen’ is just to ‘watch them happen.’” Stephen Wolfram
Learning is not computationally irreducible in this strict technical sense. We can take shortcuts. A teacher can give us a principle that took centuries to discover. A diagram can reveal a relationship that would otherwise require hours of observation.
But the path remains irreducible in a more intimate sense: it must still be walked by the learner.
No external intelligence can experience confusion for us, notice a contradiction on our behalf or perform the inner reorganization through which an idea becomes part of how we see.
The pain of not knowing should not always be removed. Sometimes the lack must be inhabited long enough to make space for curiosity and ingenuity.
Generative AI can operate like turn-by-turn navigation, supplying every intellectual move until the user reaches the answer without forming a map. Or it can help learners orient themselves: showing the wider territory, questioning their chosen route, drawing attention to overlooked evidence and gradually withdrawing guidance.
In the first case, the learner arrives. In the second, the learner becomes a navigator.
In Captivate to Teach, we drew on Gadamer to understand adventure as more than an isolated episode. An adventure interrupts ordinary life, yet acquires meaning through its relationship with the whole life it interrupts.
Learning belongs to that same territory.
AI can enlarge the map, illuminate paths and help us anticipate their consequences. But it cannot undertake the adventure for us or decide why a destination should matter.
Digital symbiosis in learning should not spare learners the adventure of understanding. It should equip them to travel farther, confront more complex realities and return with new skills that are genuinely their own.
The ultimate test is not only what a learner can accomplish with AI, but what the learner can see, question and become after the AI steps back.