Home Artificial Intelligence “Resolving the ‘reversal curse’ of LLM”…Meta proposes reverse training

“Resolving the ‘reversal curse’ of LLM”…Meta proposes reverse training

“Resolving the ‘reversal curse’ of LLM”…Meta proposes reverse training

Case of reversal curse (Photo = Archive)

Meta has devised a recent method to unravel the 'Reversal Curse' problem of huge language models (LLM). The reversal curse is an issue through which, regardless that you’ve learned that ‘A is B’, you can’t reverse it and answer ‘B is A’.

Mark Tech Post reported on the twenty fourth (local time) that researchers from FAIR, Meta's AI research department, published a paper on a way called 'Reverse Training' to unravel the issue of the reverse curse in LLM. Posted in Online ArchiveHe said he did it.

The reversal curse is a cognitive bias through which artificial intelligence (AI) models, especially LLMs, have difficulty understanding sentence equivalence when learning components are reversed. For instance, an LLM may accurately discover 'Tom Cruise's mother is Mary Lee Pfeiffer', but may not recognize 'Mary Lee Pfeiffer's son is Tom Cruise'.

String reversal method for reverse training (Photo = Archive)
String reversal method for reverse training (Photo = Archive)

The choice training method called reverse training that the researchers got here up with as an answer is surprisingly easy. By doubling the quantity of tokens for each word within the string, it’s learned twice.

In other words, it is a method of coaching LLM by literally ‘reversing’ the unique string. For instance, changing the sentence ‘Cruz was born on July 3, 1962 in Syracuse, Latest York, to Mary Lee Pfeiffer’ to ‘Mary Lee Piper gave birth to Cruz on July 3, 1962 in Syracuse, Latest York.’ It's like that.

This approach allows the LLM to be trained in each the forward and reverse directions by displaying the knowledge in each the unique and inverted formats concurrently, doubling the usefulness of the info.

This method yielded notable results in comparison with existing models in tests to measure understanding of connections. When tested on the duty of recognizing family relationships between celebrities and their descendants, the backward-trained model achieved an accuracy of 10.4%, in comparison with the 1.6% accuracy recorded by a historically trained model.

“We were in a position to successfully avoid the inversion curse by applying a backward training technique to develop a language model that may process information each forward and backward,” the researchers said. “Through this approach, we not only strengthen cognitive abilities, but in addition higher understand the complex world.” “I’m now in a position to navigate skillfully,” he said.

Reporter Park Chan cpark@aitimes.com


  1. you are truly a just right webmaster The site loading speed is incredible It kind of feels that youre doing any distinctive trick In addition The contents are masterwork you have done a great activity in this matter.

  2. Wonderful beat I wish to apprentice while you amend your web site how could i subscribe for a blog web site The account aided me a acceptable deal I had been a little bit acquainted of this your broadcast provided bright clear idea.


Please enter your comment!
Please enter your name here