Deepfakes Tore a High School Apart

ceheaaf

Didn't get past the paywall but, recently a friend had their primary school targeted by blackmailers who took public photos of children and made (I'm told) extremely explicit images with them, threatening to release them if not paid. What's the defense? Intelligent screening of incoming messages so that the threat never reaches the blackmail target? I imagine they'll find an unprotected channel. Don't post innocuous images of children ever? Seems like losing.

ale42

https://archive.is/neN8H

thunderbong

https://archive.is/neN8H

hjkl0

This has nothing to do with “deepfakes”. These girls could have been attacked by aliens and be received with more credulity. There has to be a point where you understand that this behavior towards girls and women is simply being allowed. One of the mothers says this quite explicitly: > Calling the event rumors and speculation when a crime occurred was almost worse than the crime

ktallett

This is sadly becoming very common and they are worringly good enough for people to assume they are real. I had a friend who had this issue, she has tried to ride it off as no clue who made them or shared them but I know it can't be easy for her. There needs to be appropriate laws for this the same way there is for revenge porn etc.

CalRobert

I don't know how society will function when it's as easy to fake a video or image as it is to draw a stick figure. Hopefully it at least means that people view what happened here as akin to scribbling boobs on someone's photo, but we're not there yet apparently.

M4v3R

This kind of behavior should be met with swift and strong reaction from both the school authorities and the law enforcement. Kids responsible should be put on trial and face serious consequences including jail time for most egregious cases. Let the news cover it nationwide. "This is what will happen if you even think about creating and circulating nude pics of anyone" should be the message sent. This is the only way to prevent this behavior - not more censorship, not technical solutions

dmos62

> Throughout the night, whenever one of the speakers brought up holding Big Tech accountable, and specifically Apple and Google’s app stores, the crowd applauded. The framing bothers me. It's shifting responsibility away from kids and the community. Deepfakes didn't do anything: kids made and used deepfakes in destructive ways. Big tech sucks, you get no argument from me there. But, how do people so quickly get over the fact that a kid is sexually harrassing someone? This is like when a mass shooting happens and everyone only talks about guns. Well, yeah, if you didn't have guns a mass shooting couldn't happen, but isn't the underlying issue that someone decided that hurting a bunch of people is the best thing to do? The more people dislike a crime, the less likely they are to talk about the perpetrator's inner life, but, come on, these are kids. How do people get so side-tracked?

gorgoiler

Even though they fluffed the messaging I think the school was right, in the absence of any actual deepfakes and even after the police department had scraped all the boys phones, to be wary of accusing anyone in particular of doing anything AI related. Kids are weird and one boy saying he “dropped $250” on making deepfakes of “that hoe” sounds as much like a brag about money than it could be about making fake videos. There’s clearly enough of a text message trail between these boys to give them a significant reprimand for misogyny. The PD’s PR team changing their narrative seems to lack competence. The school board focusing on being light touch on a specific case instead of cracking down in general seems like another error of judgment. This whole article says more about the effectiveness and competency of cops and school boards as it does about generative imagery.

teaearlgraycold

I think sharing these images is sexual harassment and it's pretty clearly wrong and deserves punishment. Other than that, I don't think it should ever be possible to commit a crime by providing inputs to an air-gapped "clean" computer and then view the output. But it seems like that's not the case.

mrvmochi

(translated by LLM) Reading comments demanding harsh punishments or preaching morality in response to these kinds of incidents always leaves me feeling profoundly grim. Speaking as a man, never once in my entire life have I consciously willed myself to feel a sexual impulse, thinking, "Alright, I am going to get aroused now." It is always an involuntary release of hormones or whatever; all I can do is hope that my frontal lobe manages the impulse properly so I don't end up doing something society punishes. Is it different for other people? Do people frame this as a failure of willpower because they are somehow generating their sexual urges voluntarily? Aren't sex offenders, fundamentally, individuals who struggle to take appropriate action because their impulses are overwhelmingly stronger, or their executive function is lower, compared to "normal" people? Much like gambling addiction or obesity. What they really need isn't harsh punishment or moral grandstanding, but rather medical and social support. The same applies to teenagers. We arbitrarily categorize them as "children" based purely on a number, but physical and neurological development varies wildly from person to person. Confining them in enclosed spaces with attractive peers for hours on end, and then completely derailing the lives of those whose biological makeup couldn't resist the resulting impulses—leading to criminalized behavior—feels like an incredibly cruel system. Perhaps if enough generations pass, individuals with these genetic predispositions will be weeded out and the number of victims will decrease. But surely we can implement more humane and effective measures? If society essentially prohibits them from passing on their genes, shouldn't there at least be some form of compensation or alternative support?

bondarchuk

The article intentionally obfuscates the difference between images depicting sexual abuse that did not happen, and actual sexual abuse. Ironically this makes it harder to address the problem on the level where the actual abuse happens i.e. bullying, lack of empathy, teenage body image issues in the properly fucked up american high school environment. For example here the word "abuse" technically refers to the making and spreading of these images and attendant bullying, but obviously has connotations of actual sexual abuse: > Despite all this, Radnor’s administration failed students in the days and weeks after it learned about the abuse, This is maybe subtle but later the article gets more explicit about it: > this technology [...] bears little difference from the non-consensual intimate imagery that’s plagued young girls and teenagers since the invention of the camera. (i.e. a straightforward equation of images of nudity that did not happen, and images produced by actually pointing a camera at someone who is physically present with their nude body) > Because the images aren’t “real,” authorities grapple with how to handle them. Putting "real" in scare-quotes, very nice. Dorfman is also quoted doing some nice sleight of hand, going in one sentence from "It was about the creation of the videos" to "and then share them with others as if the girls were something to be passed around." - i.e. equating the making of videos depicting unreal events with something that almost sounds like gang rape. Anyway enough examples, article is full of them. I'm not saying there's no problem here, just that these people seem to have some kind of ideology of the sanctity of the image, i.e. a realistic image always depicts real events, and should always be interpreted as if it does. If it does not, clearly that's a problem to be corrected by reinforcing the connection between images and reality by making it impossible to produce realistic images of non-real things. It strikes me as exactly the wrong way to go about it, that can only make the bullying and harrassment (the actual problem which should be solved on the social level) worse by making its ammunition stronger. (Slight aside but I do think the closing quote by Woelfel makes a good point)

IshKebab

How did he not get expelled for that? Jesus.