Recently, the police in New Mexico, USA, sparked a controversy by using an AI-generated image of a virtual 14-year-old girl to entrap sex crime suspects in an investigation. The background of this investigation is that prosecutors have filed a lawsuit against the social media app Snapchat, accusing the platform of failing to protect children from sexual exploitation and harm.
Image Source Note: The image was generated by AI, provided by the image licensing service Midjourney
According to the lawsuit documents, the police created a fake account named "Heather" on Snapchat, attempting to contact potential sex offenders through this account.
During this process, the police discovered and communicated with some obvious sex crime suspects, whose usernames were even chilling, such as "child.rape" and "pedo_lover10". In the past, the police would usually use photos of young women (often police officers) to deceive these criminals, but this time they chose an AI-generated virtual teenage girl image.
The investigation results show that this method achieved unexpected success, and many suspects did fall for it, trying to coax "Heather" into sharing explicit images or child sexual abuse materials. Although this investigation revealed the dark reality behind Snapchat's algorithms, the police's use of AI-generated images also raised new ethical issues.
Legal experts point out that the government creating deepfake child sexual abuse materials (CSAM) is undoubtedly worrying, as these images are illegal in themselves, and society does not want to see more child sexual abuse materials circulating.
Additionally, the process of AI generating virtual child images also involves issues with training datasets. To generate fake child images, the AI model needs to use real child photos for training. This raises questions about whether children can truly consent to this, especially when generating potentially sexually suggestive images. This situation undoubtedly adds complexity to legal and ethical issues.
The police's use of an AI-generated virtual girl for entrapment, while capturing suspects, has also sparked many discussions about morality and law. Whether this is the best way to protect children is worth deep consideration from all sectors of society.
Key Points:
1️⃣ New Mexico police used an AI-generated virtual teenage girl photo to lure sex crime suspects.
2️⃣ This method raised new ethical and legal questions while capturing suspects.
3️⃣ AI-generated child images rely on real child photos, involving consent and privacy issues.