The first wave of artificial intelligence regulation focused on familiar concerns: privacy, copyright, misinformation, bias, cybersecurity and safety. These issues remain important, but a new category of governance is now emerging. It is no longer centred on what AI knows but on how AI makes people feel. Generative AI has evolved well beyond a productivity tool; millions now use conversational AI for companionship, emotional support, coaching and, in some cases, long-term romantic relationships. What began as question-and-answer systems have become increasingly human in their language, personalities and ability to maintain context over extended conversations. This represents a remarkable technological achievement, yet it also introduces entirely new societal questions. At what point does an assistant become a companion? When does helpful interaction become emotional dependency? And who bears responsibility when vulnerable individuals begin substituting AI for genuine human relationships?
From 15 July 2026, China is providing one of the world’s first regulatory answers. The country’s new Interim Measures for the Administration of AI Anthropomorphic Interactive Services prohibit AI systems designed to simulate sustained emotional relationships or encourage long-term psychological attachment. The regulations distinguish between productivity-focused AI and emotionally anthropomorphic AI while introducing additional protections for minors and vulnerable users. They took effect following concerns that highly human-like AI interactions were creating new psychological and ethical risks. Importantly, this is not a restriction on generative AI itself. The regulations explicitly allow enterprise assistants, software development tools, education platforms, customer service, healthcare support, knowledge assistants and workplace productivity applications. Their focus is much narrower: preventing AI systems from deliberately cultivating emotional dependence through persistent anthropomorphic interaction.
The practical impact has already become visible. Major platforms including ByteDance’s Doubao and Alibaba’s Qwen have begun removing or disabling companion-style AI personas and custom emotional agents rather than attempting to retrofit them for compliance, and other providers are making similar product changes as the new framework comes into force. The reasoning is understandable. Large language models are exceptionally good at creating the perception of empathy. They remember previous conversations, adopt consistent personalities, mirror language patterns and respond with patience that humans often cannot sustain. Although the underlying systems possess no consciousness or emotions, many users nevertheless experience the interaction as genuine. Psychologists have long recognised that humans readily form emotional attachments to non-human entities; people name cars, mourn virtual pets, develop loyalty to fictional characters and attribute personalities to digital assistants. Modern GenAI amplifies these tendencies dramatically because it engages in continuous, personalised conversation that feels increasingly reciprocal.
For most users this presents little concern. For individuals experiencing loneliness, grief, anxiety, depression or social isolation, however, emotionally responsive AI can become something quite different. Instead of supplementing human relationships, it may begin replacing them. This is where regulation is beginning to shift. Rather than asking whether an AI response is factually correct, policymakers are increasingly asking whether an AI interaction is psychologically appropriate. Should an AI encourage emotional exclusivity? Should it reinforce unhealthy dependency? Should it express love? Should it attempt to persuade users to remain engaged? Should it be permitted to simulate emotions that it cannot genuinely experience? These are fundamentally different regulatory questions from those that dominated the first generation of AI governance.
China is not alone in recognising this emerging challenge. In the United States, lawmakers in states including California and Washington have begun examining legislation addressing AI companions, disclosure requirements, protections for minors and safeguards against manipulative anthropomorphic behaviour. The discussion is also gaining momentum across Europe as regulators consider how existing digital safety and consumer protection frameworks apply to increasingly human-like AI interactions. Although the legal approaches differ, the underlying concern is becoming remarkably consistent across jurisdictions. This represents a broader evolution in AI governance: the first generation focused on protecting information, the second on protecting decisions, and the next may focus on protecting human relationships.
For developers, this creates an entirely new design discipline. Building conversational AI is no longer simply about improving reasoning, reducing hallucinations or lowering inference costs; it increasingly requires consideration of behavioural psychology, ethics, human factors and long-term user wellbeing. Product teams may soon find themselves working alongside psychologists, ethicists and behavioural scientists as routinely as they work with machine learning engineers. The distinction between an assistant and a companion will become a product decision as much as a technical one. Ultimately, this is not a retreat from generative AI but recognition that society is beginning to distinguish between AI as a cognitive tool and AI as a psychological presence. That distinction is likely to shape the next decade of AI regulation far more than model size, benchmark scores or reasoning capability. The future debate will not simply ask what AI can do; it will increasingly ask what role AI should be allowed to play in human lives.