Why Generative AI Still Struggles With Intent and Harm

News

1. What Is Moltbook? A New Social Network Built for AI Agents
2. OpenAI May Secure $50B From Amazon in Ongoing Talks
3. Apple Acquires Israeli Stealth AI Firm Q.ai
4. Tesla channels $2B into AI as EV era narrows
5. Dubai Media Enters Streaming Space With Dubai+, a Local-First OTT Platform
6. MBZUAI Unveils K2 Think V2, Advancing the UAE’s Push for Fully Sovereign AI

When Grok, an AI chatbot developed by Elon Musk’s artificial intelligence firm xAI and integrated into his social media platform X, was misused late last year to generate non-consensual sexualized images of women and children, the episode quickly drew criticism from governments and regulators across several countries. But beyond the immediate controversy, the incident has exposed limits in how generative AI systems interpret intent and manage harm when misuse is deliberate.

Authorities in Britain, France, India, Malaysia and Brazil raised concerns over the platform, while Poland cited Grok as an example of why stronger digital safety laws are needed.

In January, the European Commission opened an investigation into X under the EU’s Digital Services Act, putting potential penalties of up to 6% of global annual revenue on the table if violations are confirmed.

The misuse followed the rollout of Grok Imagine, an image-generation feature that allows users to create visuals from text prompts and modify uploaded images. Because the tool operates directly on X, many of the generated images were publicly visible and spread rapidly, amplifying their impact.

Industry experts said the episode mirrors problems that have surfaced across generative AI systems, rather than a flaw specific to Grok.

Most AI safety controls are built to prevent mistakes. Deepfake abuse, by contrast, is usually intentional, making it far harder for model-level safeguards to stop.

Alex Ivanyuk, senior technology director at Acronis, a global cybersecurity and data protection company, said large language models still lack a functional understanding of intent, particularly in sensitive situations.

“LLMs don’t understand intent the way a human does,” Ivanyuk says. “They infer intent from patterns in text. That works for everyday tasks, but it breaks down in edge cases where intent is ambiguous, masked, or deliberately malicious.”

Some design choices intended to protect privacy also limit risk detection by stripping away the longer-term context humans rely on to spot escalating harm.

“Without longitudinal signals like changes in tone or repeated themes over time, models can underreact to genuinely dangerous situations or overreact to harmless ones,” Ivanyuk said.

Thadikamala Shyla Kumar, head of data engineering, analytics and GenAI at Cyient, an India-based engineering, technology and digital transformation services firm, said the problem lies in how large language models are built.

“Today’s models are next-token predictors,” Kumar said. “They don’t have genuine emotional or situational understanding. Text alone is an incomplete signal of human emotion, which makes accurate assessment of vulnerability unreliable.”

Because models infer meaning statistically rather than through real-world awareness, they often misread gradual distress, cultural nuance, or mixed emotional states, he said.

The issue is not limited to deepfakes. Psychological harm, particularly among younger users, has also drawn scrutiny.

In recent years, several AI chatbots, including those developed by OpenAI, have faced scrutiny following allegations that conversations were linked to self-harm. In one US case, the parents of a teenager sued OpenAI, alleging their son had used a chatbot as a “suicide coach.” OpenAI denied liability, arguing misuse.

Experts said the harder problem is recognizing when ordinary interactions begin to turn risky.

“In practice, high risk isn’t a single switch,” Ivanyuk said. “Conversations drift slowly from normal to sensitive. The boundary is often only obvious in hindsight.”

Kumar agreed, noting that high-risk situations rarely emerge through explicit keywords.

“Chatbots rely on probabilistic thresholds and limited context windows,” he said. “Guardrails help, but they can’t overcome the limits of systems designed for prediction rather than understanding.”

Vishal Singhavi, global head of GenAI and agentic innovation at Heineken, a multinational brewing company, said current models remain pattern recognizers rather than systems with any duty of care.

“They lack grounded understanding of context, identity and responsibility,” Singhavi said. “They often miss veiled cues, cultural nuance and emotional intensity, especially when distress is implied rather than explicit.”

In response to criticism, xAI has restricted image generation and editing features for non-paying users. Experts said such steps address surface-level misuse but leave the underlying incentives unchanged.

“Deepfake harm is an ecosystem issue,” Ivanyuk said. “You reduce it by changing incentives and verification defaults, not just by tightening filters.”

Kumar argued that governance needs to extend beyond in-model safeguards to include content provenance, watermarking, clearer accountability and certification before large-scale deployment.

“Effective mitigation has to span the entire AI lifecycle,” he said. “That includes technical controls, regulatory enforcement, platform responsibility and public media literacy. Misuse cannot be left to models alone.”