Skip to main content

Emergent abilities or a mirage?



Emergent abilities in large language models

I'm reminded of the Three Cup 'game' when discussions of AGI/ASI occur. The three cup scam is a gambling trick where a ball is placed under one of three cups, which are then moved around a mat. The customer is asked to guess which cup the ball is under, with a cash prize at stake. However, the scammer uses sleight of hand to move the ball to a different cup, making it impossible for the customer to win. This scam has been used to steal money from many an unsuspecting tourist.

Emergent abilities in large language models (LLMs) refer to the sudden and unpredictable increases in performance at specific tasks that occur as the model scale increases. These abilities are intriguing because they seem to emerge spontaneously as the model becomes larger, without any explicit training or fine-tuning on the specific task. This suggests that LLMs may have a capacity for generalisation and transfer learning that was previously unknown. Additionally, emergent abilities have been described as "breakthrough capabilities" and "sharp left turns," which implies that they represent a significant advance in our understanding of how language models work.

However, this paper, 'Are Emergent Abilities of Large Language Models a Mirage? by Schaeffer, Miranda, and Koyejo,' Computer Science, Stanford University, presents an alternative explanation for these abilities, suggesting that they may be a mirage induced by researcher analyses rather than a fundamental property of the model family on a specific task.

The alternative explanation

The alternative explanation presented in this paper challenges existing claims of emergent abilities by suggesting that these abilities may not be a fundamental property of the model family on a specific task. Instead, the researcher's choice of metric may create or ablate an emergent ability. The paper presents evidence to support this claim, including the meta-analysis of published results and the testing and confirmation of three predictions based on alternative hypotheses using the InstructGPT/GPT-3 model family. Therefore, the paper suggests that previously claimed emergent abilities in other studies might likely be a mirage induced by researcher analyses.

The main takeaway is for a fixed task and a fixed model family, the researcher can choose a metric to create an emergent ability or choose a metric to ablate an emergent ability. Ergo, emergent abilities may be creations of the researcher’s choices, not a fundamental property of the model family on the specific task. That said, we emphasize that nothing in this paper should be interpreted as claiming that large language models cannot display emergent abilities; rather, our message is that previously claimed emergent abilities in might likely be a mirage induced by researcher analyses. 

Implications for the future development of LLMs

The alternative explanation presented in this paper has several potential implications for the future development of LLMs. For example, it suggests that researchers should be cautious when interpreting emergent abilities in these models and consider the role of their choice of metric. Additionally, it highlights the importance of developing standardized metrics for evaluating LLMs to ensure that comparisons between different models are fair and meaningful. Finally, it suggests that future research should focus on understanding how emergent abilities arise in LLMs and how they can be intentionally created or ablated to improve model performance on specific tasks.

Conclusion

What we observe is not nature in itself but nature exposed to our method of questioning 

Werner Heisenberg 


The findings of this paper suggest that the existence of emergent abilities in LLMs is not as clear-cut as previously expounded. While it is possible that these abilities do exist, it is also possible that they are simply an artefact of researcher analyses. This ties in to Natale's argument that AI resides in the perception of human users. 

Something I've argued since the start of this blog is the 'AI' in question is more akin to Alien Intelligence, a notion I was pleased to see replicated in the thoughts of Taylor Webb at the start of his presentation. The concept of ramping up the models will automatically make them more intelligent, as has been argued, needs to be examined critically. So does this concept of emergence.

Recently Google boasted about the 'emergent' capabilities of it's new PaLM 2 LLM, admittedly with scant evidence. More research (not tied to big tech) is needed to understand the true nature of emergent abilities in LLMs. This will demonstrate if the dream/nightmare of ASI is viable, as the tech Hypster argument continues to be that once AGI is near, then machines will self improve to an extent that Artificial Super Intelligence is inevitable. Should an AGI 'emerge' or be constructed, what's certain it won't match the definition of 'human capabilities', they will be different. Consequently, so too would any ASI, it will be a 'God' of unknown and probably unwanted utility, whose mirage will fool many. 

Labels:

Comments

Post a Comment

Popular posts from this blog

The AI Dilemma and "Gollem-Class" AIs

From the Center for Humane Technology Tristan Harris and Aza Raskin discuss how existing A.I. capabilities already pose catastrophic risks to a functional society, how A.I. companies are caught in a race to deploy as quickly as possible without adequate safety measures, and what it would mean to upgrade our institutions to a post-A.I. world. This presentation is from a private gathering in San Francisco on March 9th with leading technologists and decision-makers with the ability to influence the future of large-language model A.I.s. This presentation was given before the launch of GPT-4. One of the more astute critics of the tech industry, Tristan Harris, who has recently given stark evidence to Congress. It is worth watching both of these videos, as the Congress address gives a context of PR industry and it's regular abuses. "If we understand the mechanisms and motives of the group mind, it is now possible to control and regiment the masses according to our will without their
Post a Comment
Read more

Beware the Orca, the challenge to ChatGPT and Palm2 is here

  So Google's 'we have no moat' paper was correct. If you train an LLM wisely then it's cost effective and cheap to produce a small LLM that is able to compete or even beat established, costly LLMs, as Microsoft has just found. It's another excellent video from AI Explained, who goes through some of the training procedures, which I won't get into here. Orca, is a model that learns from large foundation models (LFMs) like GPT-4 and ChatGPT by imitating their reasoning process. Orca uses rich signals such as explanations and complex instructions to improve its performance on various tasks. Orca outperforms other instruction-tuned models and achieves similar results to ChatGPT on zero-shot reasoning benchmarks and professional and academic exams. The paper suggests that learning from explanations is a promising way to enhance model skills. Smaller models are often overestimated in their abilities compared to LFMs, and need more rigorous evaluation methods. Explana
Post a Comment
Read more

What is happening inside of the black box?

  Neel Nanda is involved in Mechanistic Interpretability research at DeepMind, formerly of AnthropicAI, what's fascinating about the research conducted by Nanda is he gets to peer into the Black Box to figure out how different types of AI models work. Anyone concerned with AI should understand how important this is. In this video Nanda discusses some of his findings, including 'induction heads', which turn out to have some vital properties.  Induction heads are a type of attention head that allows a language model to learn long-range dependencies in text. They do this by using a simple algorithm to complete token sequences like [A][B] ... [A] -> [B]. For example, if a model is given the sequence "The cat sat on the mat," it can use induction heads to predict that the word "mat" will be followed by the word "the". Induction heads were first discovered in 2022 by a team of researchers at OpenAI. They found that induction heads were present in
Post a Comment
Read more