• @jarfil@beehaw.org
      link
      fedilink
      110 months ago

      That’s my point. They claim to reduce misrepresentation, while at the same time they erase a bunch of correct representations.

      Going back to what I was saying: fine tuning doesn’t increase diversity, it only shifts the biases. Encoding actual diversity would require increasing the model, then making sure it can output every correct representation.

      • @Even_Adder@lemmy.dbzer0.com
        link
        fedilink
        English
        310 months ago

        It doesn’t necessarily have to shift away from diversity biases. I think with care, you can preserve the biases that matter most. That was just their first shot at it, this seems like something you’d get better at over time.

        • @jarfil@beehaw.org
          link
          fedilink
          210 months ago

          I guess their main shortcoming was the cultural training set. I’m still unconvinced that level of fine tuning is possible without increasing model size, but we’ll see what happens if/when someone curates a much larger set with cultural labeling.

          The labels might also need to be more granular, like “culture:subculture:period”, or something… which is kind of a snakes nest by itself.