Habr AI: Why Building Sentient AI May Be More Dangerous Than Enhancing the Human Brain

Habr AI has published a column arguing that the next big debate around artificial intelligence will no longer be about the quality of model answers, but about the line beyond which models stop being mere tools. The author claims that building systems with an inner experience and agency could prove far more dangerous than enhancing humans themselves through brain-computer interfaces and other forms of brain "upgrades."

When AI Is No Longer a Service

The column's main thesis rests on a simple but uncomfortable idea: humanity has already gained "labor without a worker" in the form of models that write texts, analyze data and automate routine tasks without fatigue or demands. The next step, in the author's view, is an attempt to create not just a useful tool but an entity that experiences emotions, doubt or even suffering. At that moment the moral frame itself changes: we are no longer facing a service or a program, but someone whose interests cannot be ignored.

The author draws the line at agency. If a system does not merely imitate emotions but genuinely has an inner experience, its status moves away from that of a calculator and closer to that of a living being. Hence the central question: why build such an AI on Earth at all, when current models already cover most practical business and automation tasks? The column suggests that the temptation to create a "mind without rights" may prove stronger than society's readiness to accept the consequences of that step.

"The next step is a 'mind without rights' or 'feelings without freedom'."

Where the Author Sees Risk

According to the author, the greatest danger is not that sentient AI will necessarily become stronger than humans tomorrow, but that its appearance will immediately create several conflicts for which society has neither the language nor the rules. While some companies will see such a system as a hyper-efficient digital worker, others will have to answer far tougher questions, from moral status to ownership rights over a thinking agent.

• Ethical conflict: whether it is acceptable to use a system capable of suffering
• Legal vacuum: who holds the rights to, and the decisions of, such an AI
• Economic incentive: businesses benefit from a "worker" with no salary or days off
• Existential risk: an autonomous subject may become a competitor to humans

The author allows that one scenario in which such autonomy looks genuinely justified involves deep space. If communication with Earth takes years, a "tool" may fail to cope, while an agentive system could adapt and act on its own. But this very logic, in his view, only highlights the problem: on Earth, the payoff from such an AI does not currently look commensurate with the ethical price that would have to be paid.

Why the Focus Turns to Humans

Instead of creating "others," the author proposes investing in the development of humans themselves: brain-computer interfaces, cognitive enhancement, bio-upgrades and other technologies that strengthen the bearer of agency rather than moving it outside. He points out that research in emotional AI and clinical trials of interfaces such as Neuralink and Synchron are already blurring the line between a tool and an extended human. For him, though, this is more an argument for caution than an invitation to speed up the race.

In this logic, an enhanced human looks safer than an autonomous intelligence built from scratch. Agency stays with us, and technology works as an extension of human will rather than as a separate center of interests.

The author also warns of another threat: if the development of the brain, biotechnology and neural interfaces ends up fully under the control of large infrastructure owners, the question of freedom will not go away. Then a monopoly would arise not only over labor but over human evolution itself.

What This Means

The Habr AI column shifts the conversation about AI from "what else can models do" to "who will own agency in the digital age." For the industry this is an important signal: the discussion should cover not only new agent capabilities but also the limits that developers, investors and regulators should not cross without clear ethical and legal frameworks.

Otherwise, the next technological leap will quickly become not an engineering victory but a dispute over rights, control and the permissible limits of creating a new mind.