Yes, they should. They chose to deploy the algorithm rather than using a different algorithm, or a human-curated suggestion set, or nothing at all. It’s like a store offering one-per-purchase free bonus items while knowing a few of them are soaked in a contact poison that will make anyone who touches them sick. If your business uses a black box to serve clients, you are liable for the output of that black box, and if you can’t find a black box that doesn’t produce noxious output, then either don’t use one or put a human in the loop. Yes, that human will cost you money. That’s why my suggestion at the end was to use a single common feed, to reduce the labour. If they can’t get enough engagement from a single common feed to support the business, maybe the business should be allowed to die.

The only leg Snapchat has to stand on here is the fact that “C.O.” was violating their TOS by opening an account when she was under the age of 13, and may well have claimed she was over 18 when she was setting up the account.

Snapchat is not the only problem here, but it is a problem.

If they can’t guarantee their recommendations are clean, they shouldn’t be offering recommendations. Even to adults. Let people find other accounts to connect to for themselves, or by consulting some third party’s curated list.

If not offering recommendations destroys Snapchat’s business model, so be it. The world will continue on without them.

It really is that simple.

Using buggy code (because all nontrivial code is buggy) to offer recommendations only happens because these companies are cheap and lazy. They need to be forced to take responsibility where it’s appropriate. This does not mean that they should be liable for the identity of posters on their network or the content of individual posts—I agree that expecting them to control that is unrealistic—but all curation algorithms are created by them and are completely under their control. They can provide simple sorts based on data visible to all users, or leave things to spread externally by word of mouth. Anything beyond that should require human verification, because black box algorithms demonstrably do not make good choices.

It’s the same thing as the recent Air Canada chatbot case: the company is responsible for errors made by its software, to about the same extent as it is responsible for errors made by its employees. If a human working for Snapchat had directed “C.O.” to the paedophile’s account, would you consider Snapchat to be liable (for hiring the kind of person who would do that, if nothing else)?

It isn’t, and the courts agreed with that. Seems like an issue with legislative law. As far as I was aware, sex offenders were suppose to have Internet restrictions…

Could there be a good discussion to try and prevent harm to further children? Well, yeah. Some parents just suck and it’s the kid that gets hurt.

As long as it doesn’t involve stuff like kosa which puts more people on harms way.

They are too busy looking at titties or cocks

Something tells me you didn’t read my comment.

Unless we want snapchat and other apps to require photo ID, how would snapchat actually know who is a child and who is an adult? Why did the parents not know or care that the kid had snapchat downloaded?

We had to walk a delicate line with our kids (2005-2017) when it came to interaction online. Never wanted them to feel like we were keeping them from experiencing the knowledge or social interaction the internet provided. But we also kept close tabs and paid special attention to specific behaviors. So if they were in their netbooks we’d make it a habit to walk behind them not to look but just to see what their reaction would be. Kids mostly know right from wrong and when they feel it “might” be wrong they try and hide it from their parents. If you pay attention you’ll see them “hiding” and that’s a sign to dig deeper. This way they maintain their privacy and any issues can be brought to light with them directly.

(Understand that the following will have specific details changed just for anonymity’s sake) Grooveshark was the first interaction we saw was troublesome. So we sat our daughter down and asked her direct why she was trying to hide her netbook from us seeing it and what had she been doing she felt she needed to hide? The alternative was to relinquish the netbook until she told us. Come to find out a friend of hers from school (female 2 years older) was trying to slowly convince her to lie to her parents and sneak off with her. Our daughter told us this because it scared her not because she would lose her access. We also stayed open and active with our kids indulging in the same things they were interested in (Minecraft, Guitar Hero, etc) regardless if it was explicitly something we enjoyed. So she didn’t lose access to Grooveshark because she really loved listening to music. We just kept an eye on it and she removed her friend from communication. We explained what she was likely attempting and her friend admitted to it. They’re not friends now but it never happened again.

Don’t get me wrong, we made tons of bad calls before we learned what worked. But the key to all of it is paying attention. Not hovering over them and stopping them from making mistakes. But watching the nuance of their interactions with everyone around them. If they start to get secretive then there is usually a reason. And it’s best to just talk to them about it. And if one conversation doesn’t do it then have multiple conversations. Listen to what they have to say and why they were being secretive. Works best when they’re not expecting it too (like in the middle of playing Minecraft together). Anyway that’s just IMHO.