Thread by rahaeli

twitter.com
2 Recommenders
2 Mentions
Recommend
Post
Save
Complete
Collect
Thread
Okay, look, I can't sleep so you all get the cranky rant: I have seen multiple people responding to my criticism of Cohost and Mastodon, and Shep's criticism of Cohost and Hive, saying that we're just "haters" who are trying to destroy alternatives to Big Tech.

What these accusations miss is the era where college kids could set up a minimum viable product and figure everything else out as they go without a splash radius the size of a neutron star in human lives is over. It is dead. Mark Zuckerburg murdered it. Jack Dorsey murdered it.
(Content note for rest of the thread: frank discussion about Trust & Safety work, the toll it takes on you, the things you see, and the human misery, suffering, and death that happens when you fuck it up, including murder and child sex abuse.)
In 1999, when Brad Fitzpatrick got tired of his friends asking him for copies of the script he used to keep people at home updated about his freshman year and sat down and hacked multi-user support into it and LiveJournal was born, he had time to figure out the hard problems.
We answered a lot of technical questions in those early years, but more importantly, we also had time to figure out answers to every single one of the social, legal, and societal questions that are so prevalent today. Social media was a niche product not many people knew about.
We had time and slow growth and the ability to think carefully about things. I took over T&S (it wasn't called that yet) at LiveJournal in 2002. We had to invent every single best practice mostly from scratch, reasoning everything from first principles.
We were incredibly lucky: today's sophisticated social and legal attack vectors didn't exist yet in anything but nascent form. People were awful, but it was artisanal hand-crafted awful, not today's awful at scale.
The first person whose death I can causally attribute to an action I took or failed to take in that job died about six months after I started doing it. I was *gutted*. I swore I'd never let it happen again. It fucked me up for weeks.
It's twenty years later. If you put a gun to my head and told me my life depended on telling you exactly how many people died because of an action I took or failed to take on the job, somewhere in the causal chain, I could not give you that number to the nearest dozen.
I probably couldn't even give you that number accurately to the nearest hundred.
I don't tell a lot of the stories very often. I tell the story of the woman who escaped her domestic abuser and a few years later got an anonymous comment containing nothing but a photo of a gun and a box of ammo next to the Google Maps results for the address she'd fled to.
It makes a point about how threats don't always look like threats, and you have to evaluate the context of the threat, not just the threat itself. I say that I never heard back from her and I desperately hope it's because she cut ties with everything to flee a second time.
I say that because I have learned the hard way that stories like that get people really depressed. My ability to calibrate what "normal" people find upsetting or shocking disappeared somewhere around 2004 or so. I've had to reconstruct it from observation over time.
Until this thread I've never mentioned that the woman who wrote in to us didn't give her name, but she did give his, and I searched for it occasionally because of how much it haunted me. He was convicted of the murder of a woman in 2009-ish. I don't know if it was her or not.
(See? I'm still trying to make the story less depressing for you.)
I can describe the background of specific child sex abuse images. I can describe the color of the shirt that was cut off the boy's body and used to gag him. I can describe the stuffed animal the girl was holding. I can describe the posters on the wall.
I don't have a visual memory. I have ADHD, and my ADHD comes with moderate to severe aphantasia: the inability to construct images in my head or picture things that I've seen. I forget what a thing looked like within days, usually. I can describe every pixel of those images.
When I say that when you put more than five or six humans on an online service, you will begin to encounter *dozens* of scenarios that drive home to you that humans are really fucking terrible to each other, it is not alarmism.
When I say that there are multiple ways running a service badly, or without thinking through what you're doing and being aware of what you're going to have happen, can *and will* have a body count, it is not alarmism.
When I point out the various ways a Terms of Service is badly crafted and gives every indication that the people running the service don't know what they're doing, it is not alarmism.
When I point out the ways people are incurring legal liability by failing to do the *extremely* basic steps necessary to run a service on the internet that accepts user generated content without being sued into oblivion, it is not alarmism.
The world has moved on from 1999. Your service *will* be used by sophisticated, organized, coordinated actors who share the best tactics to hurt people with each other. Your service will be used by hostile nation-states. Your service will be used by people to do genocide.
It isn't 1999 anymore. And I want there to be more places to hang out online that aren't run by "Big Tech" -- I've devoted my last 14 years to it -- but I also want people to not get killed and my friends not to get fucked over by people who don't realize it's not 1999 anymore.
If you put a service on the internet, people are going to use it to hurt each other. Demonstrating you're aware of the ways they will -- and the ways it can hurt you -- is table stakes for entry in 2022.
I am grimly amused that people are retweeting it with "she really means it about the content warnings" when I only belatedly added the warnings before hitting tweet when my rational brain went "wait, this is another thing your calibration is totally broken on".
When I say that this work completely breaks your ability to tell what normal people find upsetting, I'm not exaggerating. I have to rely on modeling what my friends would be upset by.
also, everyone replying should know that I block disingenuous Reply Guys on sight because I have seen a million of you and I'm over it
also also, all the love in the world to all y'all who know exactly what I'm talking about and have your own horror stories you've learned not to tell because they disturb people too much ❤️
Also, I've pushed a followup (in another thread so people can skip the horrible) on what I consider the absolute basics I need to see before being willing to sign up for a new service.

I've had to mute this because two days is as long as I can stand my mentions being unusable. Sorry, y'all.
Mentions
See All