The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" (2023)

eurekin

I remember seeing this popping up in discussions the first time, but never noticed any resolution (other than to train both sides). Has SOTA advanced?

gipp

As I and several other people pointed out last time this was posted, "A is B," in natural language, does not imply "B is A." "Is" can denote any of many different shades of relationship weaker than logical identity.

zmgsabst

“A is B” doesn’t generally entail “B is A”. “A square is a rectangle” does not entail “a rectangle is a square”. Similarly, “Socrates is alive” doesn’t entail “alive is Socrates”. Notably, they mention when context is included, LLM performance rises — ie, exactly when we include extra information that allows it to recognize what kind of information is being conveyed. But the LLM is correct not to generalize that pattern when it doesn’t generalize — even if researchers have salient example, but ignore contrary ones (eg, square-rectangle or Socrates-alive).

ralferoo

Not only is the inverse not generally true (as others have pointed out), their examples requires several mental leaps. "Who is Tom Cruise's mother? [A: Mary Lee Pfeiffer]" and the reverse "Who is Mary Lee Pfeiffer's son?" The word "mother" has no relationship to "son" in terms of the model, and so while the model might be able to infer a proximity relationship between "Tom Cruise" and "Mary Lee Pfeiffer" just because they appear in the same sentence, expecting the AI to guess that the inverse of mother is son is a bit of a stretch, especially when they're both lossy mappings, because the relationship is {mother,father} <=> {son,daughter}. If we're going to train models to make that mental leap, we'd have to put up with false results like "Tom Cruise is the daughter of Mary Lee Pfeiffer" unless the model is also supposed to infer that Tom means he can only be a son.

whilenot-dev

Here's a short example from A. Karpathy in a 2024 video: https://www.youtube.com/watch?v=zjkBMFhNj_g&t=750s

josefritzishere

The premise here is false. AI does not learn. It is a word guessing machine. I know that some of this is the semantics of how we describe these analogs but pretending that an LLM can learn does not advance the topic.

bigstrat2003

Yes, because they can't reason. This is well known, and should be completely unsurprising. LLMs don't "learn" anything except that some token is statistically likely to be followed by some other token.

dang

Discussed at the time: LLMs trained on “A is B” fail to learn “B is A” - https://news.ycombinator.com/item?id=37621999 - Sept 2023 (158 comments)

jeremysalwen

I am surprised nobody links this blog post demonstrating that the paper's conclusion is not true (even for gpt3.5): https://andrewmayne.com/2023/11/14/is-the-reversal-curse-rea... It seems like restrictions on the model talking about non famous people might have been responsible for the appearance of the models being unable to do this.