Skip to main content

Emergent abilities or a mirage?



Emergent abilities in large language models

I'm reminded of the Three Cup 'game' when discussions of AGI/ASI occur. The three cup scam is a gambling trick where a ball is placed under one of three cups, which are then moved around a mat. The customer is asked to guess which cup the ball is under, with a cash prize at stake. However, the scammer uses sleight of hand to move the ball to a different cup, making it impossible for the customer to win. This scam has been used to steal money from many an unsuspecting tourist.

Emergent abilities in large language models (LLMs) refer to the sudden and unpredictable increases in performance at specific tasks that occur as the model scale increases. These abilities are intriguing because they seem to emerge spontaneously as the model becomes larger, without any explicit training or fine-tuning on the specific task. This suggests that LLMs may have a capacity for generalisation and transfer learning that was previously unknown. Additionally, emergent abilities have been described as "breakthrough capabilities" and "sharp left turns," which implies that they represent a significant advance in our understanding of how language models work.

However, this paper, 'Are Emergent Abilities of Large Language Models a Mirage? by Schaeffer, Miranda, and Koyejo,' Computer Science, Stanford University, presents an alternative explanation for these abilities, suggesting that they may be a mirage induced by researcher analyses rather than a fundamental property of the model family on a specific task.

The alternative explanation

The alternative explanation presented in this paper challenges existing claims of emergent abilities by suggesting that these abilities may not be a fundamental property of the model family on a specific task. Instead, the researcher's choice of metric may create or ablate an emergent ability. The paper presents evidence to support this claim, including the meta-analysis of published results and the testing and confirmation of three predictions based on alternative hypotheses using the InstructGPT/GPT-3 model family. Therefore, the paper suggests that previously claimed emergent abilities in other studies might likely be a mirage induced by researcher analyses.

The main takeaway is for a fixed task and a fixed model family, the researcher can choose a metric to create an emergent ability or choose a metric to ablate an emergent ability. Ergo, emergent abilities may be creations of the researcher’s choices, not a fundamental property of the model family on the specific task. That said, we emphasize that nothing in this paper should be interpreted as claiming that large language models cannot display emergent abilities; rather, our message is that previously claimed emergent abilities in might likely be a mirage induced by researcher analyses. 

Implications for the future development of LLMs

The alternative explanation presented in this paper has several potential implications for the future development of LLMs. For example, it suggests that researchers should be cautious when interpreting emergent abilities in these models and consider the role of their choice of metric. Additionally, it highlights the importance of developing standardized metrics for evaluating LLMs to ensure that comparisons between different models are fair and meaningful. Finally, it suggests that future research should focus on understanding how emergent abilities arise in LLMs and how they can be intentionally created or ablated to improve model performance on specific tasks.

Conclusion

What we observe is not nature in itself but nature exposed to our method of questioning 

Werner Heisenberg 


The findings of this paper suggest that the existence of emergent abilities in LLMs is not as clear-cut as previously expounded. While it is possible that these abilities do exist, it is also possible that they are simply an artefact of researcher analyses. This ties in to Natale's argument that AI resides in the perception of human users. 

Something I've argued since the start of this blog is the 'AI' in question is more akin to Alien Intelligence, a notion I was pleased to see replicated in the thoughts of Taylor Webb at the start of his presentation. The concept of ramping up the models will automatically make them more intelligent, as has been argued, needs to be examined critically. So does this concept of emergence.

Recently Google boasted about the 'emergent' capabilities of it's new PaLM 2 LLM, admittedly with scant evidence. More research (not tied to big tech) is needed to understand the true nature of emergent abilities in LLMs. This will demonstrate if the dream/nightmare of ASI is viable, as the tech Hypster argument continues to be that once AGI is near, then machines will self improve to an extent that Artificial Super Intelligence is inevitable. Should an AGI 'emerge' or be constructed, what's certain it won't match the definition of 'human capabilities', they will be different. Consequently, so too would any ASI, it will be a 'God' of unknown and probably unwanted utility, whose mirage will fool many. 

Labels:

Comments

Post a Comment

Popular posts from this blog

The Whispers in the Machine: Why Prompt Injection Remains a Persistent Threat to LLMs

 Large Language Models (LLMs) are rapidly transforming how we interact with technology, offering incredible potential for tasks ranging from content creation to complex analysis. However, as these powerful tools become more integrated into our lives, so too do the novel security challenges they present. Among these, prompt injection attacks stand out as a particularly persistent and evolving threat. These attacks, as one recent paper (Safety at Scale: A Comprehensive Survey of Large Model Safety https://arxiv.org/abs/2502.05206) highlights, involve subtly manipulating LLMs to deviate from their intended purpose, and the methods are becoming increasingly sophisticated. At its core, a prompt injection attack involves embedding a malicious instruction within an otherwise normal request, tricking the LLM into producing unintended – and potentially harmful – outputs. Think of it as slipping a secret, contradictory instruction into a seemingly harmless conversation. What makes prompt inj...
Post a Comment
Read more

Can We Build a Safe Superintelligence? Safe Superintelligence Inc. Raises Intriguing Questions

  Safe Superintelligence Inc . (SSI) has burst onto the scene with a bold mission: to create the world's first safe superintelligence (SSI). Their (Ilya Sutskever, Daniel Gross, Daniel Levy) ambition is undeniable, but before we all sign up to join their "cracked team," let's delve deeper into the potential issues with their approach. One of the most critical questions is defining "safe" superintelligence. What values would guide this powerful AI? How can we ensure it aligns with the complex and often contradictory desires of humanity?  After all, "safe" for one person might mean environmental protection, while another might prioritise economic growth, even if it harms the environment.  Finding universal values that a superintelligence could adhere to is a significant hurdle that SSI hasn't fully addressed. Another potential pitfall lies in SSI's desire to rapidly advance capabilities while prioritising safety.  Imagine a Formula One car wi...
Post a Comment
Read more

AI Agents and the Latest Silicon Valley Hype

In what appears to be yet another grandiose proclamation from the tech industry, Google has released a whitepaper extolling the virtues of what they're calling "Generative AI agents". (https://www.aibase.com/news/14498) Whilst the basic premise—distinguishing between AI models and agents—holds water, one must approach these sweeping claims with considerable caution. Let's begin with the fundamentals. Yes, AI models like Large Language Models do indeed process information and generate outputs. That much isn't controversial. However, the leap from these essentially sophisticated pattern-matching systems to autonomous "agents" requires rather more scrutiny than the tech evangelists would have us believe. The whitepaper's architectural approaches—with their rather grandiose names like "ReAct" and "Tree of Thought"—sound remarkably like repackaged versions of long-standing computer science concepts, dressed up in fashionable AI clot...
Post a Comment
Read more