Overview
This post mainly comprises content from the sources below:
- [2311.05020] First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models
- [2208.12852] What Do NLP Researchers Believe? Results of the NLP Community Metasurvey
- The Bitter Lesson (Richard Sutton): The general purpose methods (i.e., search and learning) that depend on computation will always outperform the methods that require human knowledge by a large margin, though the latter could provide short-term performance gains and the researchers’ personal satisfaction. This trend has been proved repeatedly in the past decades in playing chess, Go, speech recognition, and computer vision. Despite this, many researchers make similar mistakes: “The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.”