Calling it tattling is an attempt to paint the mechanism for reporting in a bad light.
LLMs have a horrible track record of keeping suicidal and homicidal people secret and even dissuading them from seeking help. This is a startup industry with the mentality of move fast and break things. People die and the family just gets paid off in a settlement.
Yes in this case it may have been a prank but AI companies need to find a balance and there needs to be accountability for what their product does to people.
Calling it tattling is an attempt to paint the mechanism for reporting in a bad light.
LLMs have a horrible track record of keeping suicidal and homicidal people secret and even dissuading them from seeking help. This is a startup industry with the mentality of move fast and break things. People die and the family just gets paid off in a settlement.
Yes in this case it may have been a prank but AI companies need to find a balance and there needs to be accountability for what their product does to people.