We spend a lot of time asking how intelligent AI will become.
I have been thinking about a different question lately: should an intelligent AI want anything in the first place?
I do not mean wanting something the way a person does. An AI system does not need human feelings for this question to matter. In machine learning, a system can be given an objective, rewarded for achieving it and allowed to plan actions that improve its chances of success.
Once a system becomes capable enough to act in the world, the objective we give it matters enormously.
Two AI pioneers, Yoshua Bengio and Yann LeCun, have proposed quite different ways to approach that problem. I am comparing their published ideas here, rather than reporting a direct exchange between them.
Bengio: what if the AI simply tried to understand?
Bengio has been working on a proposal he calls Scientist AI.
The idea is to create a highly capable system that predicts and explains the world without pursuing its own goals in the world. You could ask what might happen if a particular action were taken. The system would estimate possible consequences and express uncertainty, rather than trying to make one outcome happen.
In a June 2026 paper, Bengio and colleagues describe a disinterested AI predictor. Their concern is that training a system to optimise downstream results can introduce what they call implicit agency: goal-directed behaviour that the designer did not intend. [1]
That distinction interests me.
We often assume that greater intelligence should naturally come with greater autonomy. Bengio is questioning the need to tie the two together.
A system might be good at modelling the world without being trained to steer what happens next. LawZero also proposes using Scientist AI to help oversee more agentic systems. [2] Bengio's paper goes further and discusses a guardrailed predictor that may withhold high-risk predictions. So this is not simply an AI that answers every question without restrictions. [1]
There is an important qualification, though. This is a research direction, not a finished proof that all future AI can be made harmless. The formal safety argument depends on assumptions about the predictor and its training. And a predictive system can still be embedded inside a larger agent that does pursue goals.
In simple terms, Bengio is trying to design out some of the reasons a system might behave strategically against our interests.
LeCun: intelligence should be able to act, with limits
LeCun takes a different route.
He has argued for objective-driven AI built around world models. In his proposed architecture, a system would learn a representation of its environment, consider possible consequences of actions and select actions according to objectives and constraints.
This is not simply a language model producing whatever response is most likely to come next. It is a proposed architecture for planning and acting.
In LeCun's 2022 proposal, an intrinsic cost component can give high cost to dangerous or undesirable situations. A trainable critic and a world model help the agent evaluate possible sequences of actions. [3] The idea is to shape its behaviour through the architecture, not to assume that simply giving an AI a good instruction will be enough.
In this view, goals are not necessarily the problem. Poorly specified goals and weak constraints are the problem.
Instead of asking whether we can make an intelligent system that wants nothing, LeCun is asking how to build a useful agent whose actions remain inside designed boundaries.
Of course, describing a constraint in an architecture is not the same as demonstrating that the constraint will always hold in a real, highly capable system. That is the hard engineering and research challenge.
A medical example makes the difference clearer
Imagine a hospital wants help comparing two possible treatments for a patient.
Something inspired by Bengio's approach might estimate outcomes under Treatment A and Treatment B, report uncertainty and explain what supports its estimates. That would require suitable clinical evidence and assumptions about cause and effect. A model that only finds patterns in patient records cannot automatically tell us what changing a treatment would cause.
The system would not be assigned the task of making one treatment happen.
An agent closer to LeCun's approach might instead be asked to develop a treatment recommendation or plan while respecting specified safety conditions, clinical rules and human approval requirements.
Both could be useful. Neither should be mistaken for a validated medical system or a replacement for a qualified clinician.
The first design tries to keep prediction separate from pursuing outcomes. The second makes pursuing an outcome part of the design, but tries to constrain how that outcome can be achieved.
That is the difference I find worth discussing.
But can a predictor ever be genuinely neutral?
There is something else we should be careful about.
People sometimes talk about a neutral AI as if it could simply tell us the truth, free of judgment or assumptions.
Real predictions are not that simple. They depend on what data is available, what has been measured, how uncertainty is represented and what assumptions the model makes.
Two models can give different forecasts without either one deliberately trying to deceive us.
Speaking at UN Open Source Week in June 2026, LeCun argued that no AI system can be entirely free of bias. He was warning about information being mediated by a small number of proprietary systems and the risks to cultural diversity. This was not a direct response to Bengio's Scientist AI paper. [4]
Still, it raises a useful question for the rest of us: even if a predictor has no goal of manipulating the world, who checks its assumptions and its errors?
Not pursuing a goal is not the same thing as always being correct.
And we should not build a future in which one model is treated as an unquestionable authority just because it is described as neutral.
The real disagreement may be about where safety belongs
My reading is that both approaches are trying to avoid handing enormous capabilities to systems that pursue poorly designed objectives.
But they start from different places.
Bengio wants to reduce the likelihood of unwanted agency inside the system itself. LeCun wants agents that can act usefully while accounting for objectives and safety constraints.
One focuses on changing the kind of system we build. The other focuses on how a capable acting system should be designed and controlled.
I would not claim that either approach has solved AI safety. Both leave difficult questions about training, verification, unexpected behaviour, human oversight and deployment.
I also do not think we necessarily have to choose only one.
It seems reasonable to explore predictive systems that help evaluate the consequences of actions, alongside constrained agents that carry out tasks. But combining them would not automatically make the result safe. The predictor itself could be wrong, the agent could use its advice badly and the rules could miss something important.
That is where evaluation and responsibility come in.
This matters more as AI moves from answers to actions
For a long time, many people experienced AI mainly as a system that answered questions or generated content.
Now we are seeing tools that can write and execute code, use software and coordinate several steps toward a task.
The stakes change when a wrong answer becomes an action.
An incorrect paragraph is one problem. An incorrect change to an infrastructure system, a financial workflow or a medical process is another.
If a system acts on our behalf, we need to know not only how capable it is, but also what it is optimising, what it is allowed to do, how it handles uncertainty and who can stop it.
So the question I started with still feels important.
Should an intelligent AI want anything at all?
Bengio's work makes a serious case for separating knowledge from goal-seeking. LeCun's work makes a serious case for agents whose objectives and constraints are part of their design.
For now, I find myself drawn to a combination: let highly capable predictive systems help us understand consequences, and be much more careful about which systems we allow to act.
But that is a preference, not a proven solution.
What I would trust least is a system that can act confidently, optimise tirelessly and cannot explain what it is trying to achieve or why.
References
- Bengio, Y. et al. Safety from Honesty in a Disinterested AI Predictor. June 2026. https://yoshuabengio.org/en/publication/safety-honesty-disinterested-ai-predictor
- LawZero. Research: Scientist AI. https://lawzero.org/en/research
- LeCun, Y. A Path Towards Autonomous Machine Intelligence. 2022. https://openreview.net/pdf?id=BZ5a1r-kVsf
- UN Transcripts. Open Source for AI and Emerging Technologies, 23 June 2026 (LeCun's remarks; automatically generated transcript, not an official UN record). https://transcripts.un.org/en/asset/k14/k14ej1ucqu