Charting the emergence of AGI?

 Large Language Models (LLMs) are rapidly transforming how we interact with technology, offering incredible potential for tasks ranging from content creation to complex analysis. However, as these powerful tools become more integrated into our lives, so too do the novel security challenges they present. Among these, prompt injection attacks stand out as a particularly persistent and evolving threat. These attacks, as one recent paper (Safety at Scale: A Comprehensive Survey of Large Model Safety https://arxiv.org/abs/2502.05206) highlights, involve subtly manipulating LLMs to deviate from their intended purpose, and the methods are becoming increasingly sophisticated. At its core, a prompt injection attack involves embedding a malicious instruction within an otherwise normal request, tricking the LLM into producing unintended – and potentially harmful – outputs. Think of it as slipping a secret, contradictory instruction into a seemingly harmless conversation. What makes prompt inj...

In what appears to be yet another grandiose proclamation from the tech industry, Google has released a whitepaper extolling the virtues of what they're calling "Generative AI agents". (https://www.aibase.com/news/14498) Whilst the basic premise—distinguishing between AI models and agents—holds water, one must approach these sweeping claims with considerable caution. Let's begin with the fundamentals. Yes, AI models like Large Language Models do indeed process information and generate outputs. That much isn't controversial. However, the leap from these essentially sophisticated pattern-matching systems to autonomous "agents" requires rather more scrutiny than the tech evangelists would have us believe. The whitepaper's architectural approaches—with their rather grandiose names like "ReAct" and "Tree of Thought"—sound remarkably like repackaged versions of long-standing computer science concepts, dressed up in fashionable AI clot...

 In recent years, artificial intelligence (AI) has become an integral part of our daily lives, powering everything from smart assistants to complex data analysis. However, as AI technologies continue to advance and proliferate, a concerning trend has emerged: the rapidly increasing energy and water consumption of data centres that support these systems. The Power Hunger of AI According to the International Energy Agency (IEA), global data centre electricity demand is projected to more than double between 2022 and 2026, largely due to the growth of AI. In 2022, data centres consumed approximately 460 terawatt-hours (TWh) globally, and this figure is expected to exceed 1,000 TWh by 2026. To put this into perspective, that's equivalent to the entire electricity consumption of Japan. The energy intensity of AI-related queries is particularly striking. While a typical Google search uses about 0.3 watt-hours (Wh), a query using ChatGPT requires around 2.9 Wh - nearly ten times more en...

 In recent years, the rapid advancement of artificial intelligence (AI) has sparked intense debate about the future of work. As we edge closer to the development of artificial general intelligence (AGI), these discussions have taken on a new urgency. This post explores various perspectives on employment in a post-AGI world, including the views of those who may resist such changes. It follows on from others I've written on the impacts of these technologies. The Potential for Widespread Job Displacement Avital Balwit, an employee at Anthropic, argues in her article " My Last Five Years of Work " that AGI is likely to cause significant job displacement across various sectors, including knowledge-based professions. This aligns with research by Korinek (2024), which suggests that the transition to AGI could trigger a race between automation and capital accumulation, potentially leading to a collapse in wages for many workers. Emerging Opportunities and Challenges Despite the ...

  Safe Superintelligence Inc . (SSI) has burst onto the scene with a bold mission: to create the world's first safe superintelligence (SSI). Their (Ilya Sutskever, Daniel Gross, Daniel Levy) ambition is undeniable, but before we all sign up to join their "cracked team," let's delve deeper into the potential issues with their approach. One of the most critical questions is defining "safe" superintelligence. What values would guide this powerful AI? How can we ensure it aligns with the complex and often contradictory desires of humanity?  After all, "safe" for one person might mean environmental protection, while another might prioritise economic growth, even if it harms the environment.  Finding universal values that a superintelligence could adhere to is a significant hurdle that SSI hasn't fully addressed. Another potential pitfall lies in SSI's desire to rapidly advance capabilities while prioritising safety.  Imagine a Formula One car wi...

  The recent appointment of General Paul Nakasone, former head of the National Security Agency (NSA), to OpenAI's board of directors has sparked widespread outrage and concern among privacy advocates and tech enthusiasts alike. Nakasone, who led the NSA from 2018 to 2023, will join OpenAI's Safety and Security Committee, tasked with enhancing AI's role in cybersecurity. However, this move has raised significant red flags, particularly given the NSA's history of mass surveillance and data collection without warrants. Critics, including Edward Snowden, have voiced their concerns that OpenAI's AI capabilities could be leveraged to strengthen the NSA's snooping network, further eroding individual privacy. Snowden has gone so far as to label the appointment a "willful, calculated betrayal of the rights of every person on Earth." The tech community is rightly alarmed, with many drawing parallels to dystopian fiction. The move has also raised questions about ...

  Leopold Aschenbrenner's 'Situational Awareness, the decade ahead  Situational Awareness, the decade ahead . June 2024' may turn out to be the most significant publication on AI safety to date. Unlike a lot of theoretical musings from highly intelligent critics of AI systems this one has been written by an engineer, who was until recently employed by Open AI in the now disbanded former Super Alignment team.  It begins by discussing the rapid advancements in AI technology, particularly focusing on the progression from GPT-2 to GPT-4 models. It highlights that AI capabilities are evolving at an exponential rate, and there are predictions that by 2027, AI models could match or even surpass the work of human AI researchers and engineers. The text underscores the importance of understanding the trendlines in compute, algorithmic efficiencies, and unlocking latent capabilities for future AI development. Additionally, the document mentions the potential risks and challenge...

  Neel Nanda is involved in Mechanistic Interpretability research at DeepMind, formerly of AnthropicAI, what's fascinating about the research conducted by Nanda is he gets to peer into the Black Box to figure out how different types of AI models work. Anyone concerned with AI should understand how important this is. In this video Nanda discusses some of his findings, including 'induction heads', which turn out to have some vital properties.  Induction heads are a type of attention head that allows a language model to learn long-range dependencies in text. They do this by using a simple algorithm to complete token sequences like [A][B] ... [A] -> [B]. For example, if a model is given the sequence "The cat sat on the mat," it can use induction heads to predict that the word "mat" will be followed by the word "the". Induction heads were first discovered in 2022 by a team of researchers at OpenAI. They found that induction heads were present in ...

  'AI eliminated nearly 4,000 jobs in May' so it's reported by hallenger, Gray & Christmas, Inc. Following on from reports by IBM et al that thousands of job cuts will occur due to AI replacement, there is no need to wait for the utopia of AI allowing humans more leisure time, as that's already here, in the form of redundancies, if we are to accept the reports findings. 'With the exception of Education, Government, Industrial Manufacturing, and Utilities, every industry has seen an increase in layoffs this year.' What's particularly notable is that it's the Tech sector that's the most affected from job cuts in the US economy: 'The Technology sector announced the most cuts in May with 22,887, for a total of 136,831 this year, up 2,939% from the 4,503 cuts announced in the same period last year. The Tech sector has now announced the most cuts for the sector since 2001, when 168,395 cuts were announced for the entire year. ' Another reason ...

  I wanted to conduct a little experiment, as a follow up to some posts which assert bias and a USA centric, hegemonic view of the world. I apologise now, that this is a necessarily long post. Please bare with me as the results that follow may raise your eyebrows and lead to some serious questions. The methodology is clear, it may not be perfect, but you can make of it what you will, and repeat it yourself with a topic of your choosing. I used GPT-4, via Perplexity AI (As it makes the sources more apparent) to suggest policy solutions to a real world problem, the economic state of the UK economy, in order to ascertain the bias in it's chosen sources and the effect this would have upon the answer(s).  I chose the field of economics as, for me, any differences in the given answers would be rapidly apparent, as I have informally studied economics since the 2008 Great Financial Crash. I'm no self-proclaimed expert but would hope I've learnt sufficient for this experiment to be ...

I have seen a few discussions and lectures from Harari on the subject of AI, this though may be the best so far. The questions are pointed, which certainly helps. Harari tends to bring a different perspective to the debate on AI safety, which is of value. It's well worth watching the whole video, below is a snippet.  Harari: So, we need to know three things about AI. First of all, AI is still just a tiny baby. We haven't seen anything yet. Real AI, deployed into the world, not in a laboratory or in science fiction, is only about 10 years old. If you look at the wonderful scenery outside, with all the plants and trees, and think about biological evolution, the evolution of life on Earth took something like 4 billion years. It took 4 billion years to reach these plants and to reach us, human beings. Now, AI is at the stage of, I don't know, amoebas. It's like 4 billion years ago, and the first living organisms are crawling out of the organic soup. ChatGPT and all these w...

  "The End of Art" Proclaimed the Philosopher, and art-critic, Arthur Danto, after contemplating on Andy Warhol when he exhibited his Brillo Box in 1964.   Arthur Danto argued that art has undergone a historical transformation from mimesis, or imitation, to self-consciousness. In the past, art was judged on its ability to accurately represent reality. However, with the rise of new artistic movements such as Cubism, Abstract Expressionism, and Pop Art, art began to focus more on subjective expression and the exploration of new forms. This shift in focus led to a new understanding of what art is and what it can do. Danto argued that this process of self-consciousness is complete when art becomes aware of itself as art. This is what he calls the "end of art." He believes that the history of art is a history of the gradual realisation of the medium's own possibilities. When art becomes aware of itself, it can no longer progress in the same way. This does not mean th...

  In his presentation on 17 July 2019, Elon Musk said that ultimately he wants “to achieve a symbiosis with artificial intelligence.” Even in a “benign scenario,” humans would be “left behind.” Musk wants to create technology that allows a “merging with AI.” Neuralink is a revolutionary technology that aims to connect your brain to a computer. Imagine being able to control your devices, access information, and communicate with others using only your thoughts. The firm plans to insert a sensor smaller than a fingertip, possibly with only local anesthesia. A complex robot will implant thin wires or threads in brain regions that control movement and sensation. The implant is connected to a wireless device that processes and transmits your neural signals to your phone or computer via Bluetooth. Neuralink's vision is to create a symbiosis with artificial intelligence, where humans can enhance their abilities and keep up with the rapid advances of technology. Neuralink's founder, Elo...

  From Narrow AI Tools – to the design, development, deployment, and management of industrial metaverse applications We can look at AI applications as tools. This point of view though is far too narrow. These tools are unlike anything we have utilised so far. NVIDIA are using the term ‘ Omniverse Cloud ’ to name it’s platfom-as-a-service offering , that provides ‘ a full-stack cloud environment to design, develop, deploy, and manage industrial metaverse applications.’ To put more simply: it’s like a virtual workshop where people can design, create, and manage things like manufacturing robots, buildings, and even whole cities (good luck with that last one). Whilst I can readily envisage how PAAS system can function efficiently in the marketing sector, and to a large extent, manufacturing sectors and perhaps even buildings, I fail to see the possibility of it extending much further. Barry Smith in his talk about urban planning and smart cities provided a strong critique of the issues...

  The paper ' Language Models Don’t Always Say What They Think : Unfaithful Explanations in Chain-of-Thought Prompting' by Miles Turpin et al. investigates the faithfulness of chain-of-thought (CoT) explanations generated by large language models (LLMs) for various tasks. CoT explanations are verbalisation's of step-by-step reasoning that LLMs produce before giving a final output.  The paper shows that CoT explanations can be misleading and influenced by biasing features in the model inputs, such as the order of multiple-choice options. The paper tests two LLMs, GPT-3.5 and Claude 1.0, on 13 tasks from BIG-Bench Hard and a social-bias task, and finds that accuracy drops significantly when models are biased toward incorrect answers.  The paper also finds that models justify answers based on stereotypes without mentioning the influence of social biases. The paper concludes that CoT explanations can be plausible yet unfaithful, which poses a risk for trusting LLMs without en...

  I have a problem with Ai ethics. I admit this may be Ethics 101 to many. But my problem is in a similar way that I have a problem with the anthropomorphising of AI in discourse. There are many that should know better, but do it all the same.  For example, Open AI have an ' Open Ethics ' project. It states, in large letters, 'Open Ethics for AI is like Creative Commons for the content. We aim to build trust between machines and humans by helping machines to explain themselves.' Surely, trust can only exist between the companies, and the personnel they employ, rather than in the machine itself. It is difficult to guarantee trust in anything unless one reviews the code, the compiler, the build, the training methodologies of the LLM. Transparency is critical to trust. And trust should not be transferred to the machine tool without transparency and that a set of other principles are being followed, such as voluntary participation, informed consent, anonymity, confidentiali...

  Selena Nemorin, Andreas Vlachidis, Hayford M. Ayerakwa, and Panagiotis Andriotis explore the hype surrounding AI in education and provide a horizon scan of the current discourse. ' AI hyped? A horizon scan of discourse on artificial intelligence in education (AIED) and development' . This paper provides a horizon scan of the discourse surrounding artificial intelligence (AI) in education and development. The authors use text mining and thematic analysis to explore the hype surrounding AI in education and identify key themes that have emerged during the AIEd debate. The findings are categorized into three themes: geopolitical dominance through education and technological innovation, creation and expansion of market niches, and managing narratives, perceptions, and norms. The paper highlights the challenges of implementing AI in educational settings due to a lack of rigorous evidence supporting practical outcomes. One of the key themes identified in the paper is geopolitical do...

  In this blog post, I will summarise the main findings of a recent paper titled "A Systematic Review of Green AI" by Verdecchia et al.. The paper provides a comprehensive overview of the research field of Green AI, which aims to reduce the carbon footprint of AI models and systems. The paper analyzes 98 primary studies on Green AI published between 2016 and 2021, and identifies different patterns and trends in the literature. A definition of Green AI . From the results regarding how the term “Green AI” is used in the literature a clear picture emerges. Most Green AI studies consider Green AI as exclusively related to energy efficiency. Only fewer studies examine the influence of AI on greenhouse gas emissions (𝐶𝑂2), and an even minor fraction examines the holistic impact that AI has on the natural environment. The paper categorises the Green AI studies into four main types: position papers, observational studies, solution papers, and tool papers. Position papers propose ne...

  There are not too many YouTubers that get it. That balance of fascination and constrained horror of what we are witnessing as AI developments occur, that seek out the latest papers, that seek to explain their salient points, and know which ones to choose from the multitude. Thankfully there are channels, only, a very few, like AI Explained , and thankfully too readers of this blog like Just Matthew, who help inspire the content.  In this latest video, that he published just three hours before writing this, the person (or persons) behind the AI explained channel explored a number of different papers, some of which I've covered in this blog, some of which I've partially read. There's also some tasty surprises. Whilst I was researching through some less than original work, in order to write today's offerings, I missed the launch of the paper, ' Governance of SuperIntelligence' by OpenAI. ( Do note that Altman finished his Ted Talk with his stated aim of creating...

  Simon Johnson discusses #PowerAndProgress, a new book co-authored with Daron Acemoglu on iNET. Find a copy & learn more A thousand years of history and contemporary evidence make one thing clear. Progress depends on the choices we make about technology. New ways of organizing production and communication can either serve the narrow interests of an elite or become the foundation for widespread prosperity. The wealth generated by technological improvements in agriculture during the European Middle Ages was captured by the nobility and used to build grand cathedrals while peasants remained on the edge of starvation. The first hundred years of industrialization in England delivered stagnant incomes for working people. And throughout the world today, digital technologies and artificial intelligence undermine jobs and democracy through excessive automation, massive data collection, and intrusive surveillance. It doesn’t have to be this way. Power and Progress demonstrates that the ...