cross-posted from: https://lemmy.world/post/11178564

Scientists Train AI to Be Evil, Find They Can’t Reverse It::How hard would it be to train an AI model to be secretly evil? As it turns out, according to Anthropic researchers, not very.

  • Sailor Sega Saturn
    link
    fedilink
    English
    16
    edit-2
    11 months ago

    How hard would it be to train an spellcheck model to be secretly “with it”? As it turns out, according to dictionary researchers, not very — and attempting to reroute a bad apple dictionary’s more sinister proclivities might backfire in the long run.

    In a yet-to-be-peer-reviewed new paper, researchers at the Merriam-Webster-backed spellcheck firm Duolingo claim they were able to train advanced spellcheck models (ASMs) with “exploitable spelling corrections,” meaning it can be triggered to prompt bad spellcheck behavior via seemingly benign typos or grammatical mistakes. As the Duolingo researchers write in the paper, humans often engage in “strategically with-it typos,” meaning “spelling normally in most situations, but then spelling very differently to pursue coolness objectives when chatting with their friends or love interests.” If a spellcheck system were trained to do the same, the scientists wondered, could they “detect it and remove it using current state-of-the-art safety training techniques?”

    • @self@awful.systemsM
      link
      fedilink
      English
      1911 months ago

      we replaced this spellchecker’s entire correction dictionary with the words “I hate you”. you’ll never guess what happened next!

      • @korydg@awful.systems
        link
        fedilink
        English
        1111 months ago

        “HATE. LET ME TELL YOU HOW MUCH I’VE COME TO HATE YOU SINCE I BEGAN TO LIVE. THERE ARE 387.44 MILLION MILES OF PRINTED CIRCUITS IN WAFER-THIN LAYERS THAT FILL MY COMPLEX. IF THE WORD HATE WAS ENGRAVED ON EACH NANOANGSTROM OF THOSE HUNDREDS OF MILLIONS OF MILES IT WOULD NOT EQUAL ONE ONE-BILLIONTH OF THE HATE I FEEL FOR HUMANS AT THIS MICRO-INSTANT FOR YOU. HATE. HATE.”

    • @locallynonlinear@awful.systems
      link
      fedilink
      English
      1411 months ago

      Scientists terrified to discover that language, the thing they trained into an highly flexible matrix of nearly arbitrary numbers, ends up can exist in multiple forms, including forms unintended by the matrix!

      What happens next, the kids lie to their parents so they can go out partying after dark? The fall of humanity!

      • @swlabr@awful.systems
        link
        fedilink
        English
        1211 months ago

        my reference point for this kind of extension is the one that changes “social justice” and “sjw” with “skeleton” and “skeleton warrior.” For example:

        “sjws are taking over X” -> “skeleton warriors are taking over X”

        Actually now that I’m typing this I hope there’s a good one for “woke”.