Canva apologizes after its AI tool replaces 'Palestine' in designs

MarkusQ

We have to stop acting like these things "think"; it leads to really weird misinterpretations of the output as "meaning" things. For example, they will occasionally replace "colour" with "color". Why? Because both occur in the training data in the "same role" but "color" is, apparently, more common[1]. You can also trick them into replacing things like "sardines" with "anchovies" (on pizza) and "head of lettuce" with "cabbage" in the context of rowboats. They are lossy text compressing parrots and we are all suffering from a massive madness-of-crowds scale Eliza Effect. [1] Yep. https://books.google.com/ngrams/graph?content=color%2C+colou...

Kapura

All of these tools that are not controlled by the user, trained on datasets they do not own or understand, will inevitably be subject to manipulation. I do not necessarily believe that Canva went in and specifically trained their AI models to do this, but that's almost worse because they become the face of what somebody else has decided their model should be doing. Anybody using AI tools should be extremely cautious about what is being produced.

wky

There’s a relatively obvious constraint to check here: compositing the layers back together should produce a (near) identical image. Would it not be preferable to throw an error if the model fails to faithfully segment the image?

EtienneDeLyon

This is not by accident! There are a lot of smart and talented people working hard to embed Hasbara into LLMs.