Written by Tanisha Cardozo || Team Allycaral
Microblogging platform X has acknowledged lapses in handling objectionable content generated by its AI chatbot Grok, leading to the removal of approximately 3,500 posts and the deletion of over 600 accounts in India. The action came about a week after the Ministry of Electronics and Information Technology raised serious concerns over obscene and sexually explicit content linked to the AI tool.
Officials aware of the development said the company accepted its mistake and committed to complying with Indian laws. According to a communication shared with authorities, X assured that it would not allow obscene imagery going forward. However, neither MeitY nor X issued an official public statement detailing the timeline or scope of the action taken.
Grok, developed by Elon Musk’s xAI and integrated into X, has faced intense scrutiny globally after users exploited its image-generation and editing capabilities to create non-consensual and sexualised deepfake images, including those involving women and minors. These images spread rapidly on the platform, prompting investigations by regulators in multiple countries. Indonesia has already suspended access to Grok, while authorities in the European Union and the UK have launched probes into the tool’s safeguards.
MeitY formally wrote to X on January 2, flagging what it described as serious failures in preventing obscene content generated using Grok. The ministry warned that continued non-compliance could result in X losing its safe harbour protection under Section 79 of the Information Technology Act. X sought an extension to respond, citing the Christmas and New Year holidays, with the deadline set for January 7.
Officials indicated that the ministry was dissatisfied with X’s initial response, which largely reiterated existing user policies without detailing concrete enforcement actions. This prompted MeitY to seek a more detailed report outlining specific steps taken against offending content and accounts. The government also clarified that Grok would be treated as a content creator rather than merely a platform tool, a classification that could significantly impact intermediary liability.
The ministry noted that misuse of Grok was not limited to fake accounts but also targeted women who uploaded their own photos or videos, which were then manipulated using AI prompts. The letter cited violations under multiple Indian laws, including provisions of the IT Act, the Bharatiya Nyaya Sanhita, the Indecent Representation of Women (Prohibition) Act, and the Protection of Children from Sexual Offences Act.
X was directed to comprehensively review Grok’s prompt processing, output generation, image handling and safety guardrails, and to enforce strong deterrent measures such as account suspensions and terminations. MeitY officials have stated that compliance by X and other platforms will continue to be closely monitored, warning that any recurrence of violations could invite stricter action.
The controversy has also drawn political attention, with Shiv Sena (UBT) MP Priyanka Chaturvedi accusing X of monetising harmful behaviour after restricting Grok’s image-generation feature to paid users. The episode underscores growing global concerns around AI-generated content, especially as reports indicate a sharp rise in AI-generated abuse imagery worldwide, intensifying calls for stricter regulation and accountability.
