Skip to main content

Green AI, a reality or Green Washing?

 


In this blog post, I will summarise the main findings of a recent paper titled "A Systematic Review of Green AI" by Verdecchia et al.. The paper provides a comprehensive overview of the research field of Green AI, which aims to reduce the carbon footprint of AI models and systems. The paper analyzes 98 primary studies on Green AI published between 2016 and 2021, and identifies different patterns and trends in the literature.

A definition of Green AI. From the results regarding how the term “Green AI” is used in the literature a clear picture emerges. Most Green AI studies consider Green AI as exclusively related to energy efficiency. Only fewer studies examine the influence of AI on greenhouse gas emissions (𝐢𝑂2), and an even minor fraction examines the holistic impact that AI has on the natural environment.

The paper categorises the Green AI studies into four main types: position papers, observational studies, solution papers, and tool papers. Position papers propose new definitions, frameworks, or challenges for Green AI. Observational studies measure or estimate the energy consumption or carbon emissions of AI models or systems. Solution papers propose new methods, techniques, or algorithms to improve the environmental sustainability of AI models or systems. Tool papers present new tools or frameworks to support Green AI research or practice.

The paper also analyses the Green AI studies according to different dimensions, such as the target phase (training, inference, or deployment), the target algorithm (algorithm-agnostic, neural networks, or others), the target data type (image, text, speech, or others), the research strategy (laboratory experiments, case studies, surveys, or others), the reported energy savings (percentage of energy reduction compared to a baseline), the industrial involvement (whether the study involves industrial partners or targets industrial readers), and the tool provisioning (whether the study provides a tool or framework for Green AI).

The paper reports several interesting findings from the analysis. For example:

  • The topic of Green AI experienced a considerable growth from 2020 onward, with more than half of the primary studies published in 2020 and 2021.
  • Most studies focus on monitoring AI model footprint, tuning hyperparameters to improve model sustainability, or benchmarking models.
  • Most papers focus on the training phase, are algorithm-agnostic or study neural networks, and use image data.
  • Laboratory experiments are the most common research strategy, followed by case studies and surveys.
  • Reported Green AI energy savings go up to 115%, with savings over 50% being rather common.
  • Industrial parties are involved in Green AI studies, albeit most target academic readers.
  • Green AI tool provisioning is scarce, with only 10% of the primary studies providing a tool or framework for Green AI.

The paper concludes that the Green AI research field has reached a considerable level of maturity, and suggests that future research should adopt other Green AI research strategies, such as conducting more case studies and surveys, and porting the promising academic results to industrial practice.

I hope this blog post has given you a brief overview of the paper "A Systematic Review of Green AI" by Verdecchia et al.. If you are interested in learning more about Green AI research and practice, I recommend you to read the full paper and check out some of the references  it cites.

: Verdecchia et al., "A Systematic Review of Green AI", arXiv:2301.11047 [cs.AI], 2023

: Strubell et al., "Energy and Policy Considerations for Deep Learning in NLP", ACL 2019

: Schwartz et al., "Green AI", Communications of ACM 2020

What seems clear from the paper is that given a lot more focus the companies developing AI could save a considerable amount of resources in both the development and deployment stages. One of the concluding remarks of the paper it is noted:' The potential of Green AI cannot be disregarded: the majority of publications show significant energy savings, up to 115%, at little or no cost in accuracy.'

However, until there is a consistent use of terminology and understanding of the term Green AI, that goes beyond the associated energy consumption, it is not a term of much utility. 


Labels:

Comments

Post a Comment

Popular posts from this blog

The AI Dilemma and "Gollem-Class" AIs

From the Center for Humane Technology Tristan Harris and Aza Raskin discuss how existing A.I. capabilities already pose catastrophic risks to a functional society, how A.I. companies are caught in a race to deploy as quickly as possible without adequate safety measures, and what it would mean to upgrade our institutions to a post-A.I. world. This presentation is from a private gathering in San Francisco on March 9th with leading technologists and decision-makers with the ability to influence the future of large-language model A.I.s. This presentation was given before the launch of GPT-4. One of the more astute critics of the tech industry, Tristan Harris, who has recently given stark evidence to Congress. It is worth watching both of these videos, as the Congress address gives a context of PR industry and it's regular abuses. "If we understand the mechanisms and motives of the group mind, it is now possible to control and regiment the masses according to our will without their
Post a Comment
Read more

Beware the Orca, the challenge to ChatGPT and Palm2 is here

  So Google's 'we have no moat' paper was correct. If you train an LLM wisely then it's cost effective and cheap to produce a small LLM that is able to compete or even beat established, costly LLMs, as Microsoft has just found. It's another excellent video from AI Explained, who goes through some of the training procedures, which I won't get into here. Orca, is a model that learns from large foundation models (LFMs) like GPT-4 and ChatGPT by imitating their reasoning process. Orca uses rich signals such as explanations and complex instructions to improve its performance on various tasks. Orca outperforms other instruction-tuned models and achieves similar results to ChatGPT on zero-shot reasoning benchmarks and professional and academic exams. The paper suggests that learning from explanations is a promising way to enhance model skills. Smaller models are often overestimated in their abilities compared to LFMs, and need more rigorous evaluation methods. Explana
Post a Comment
Read more

What is happening inside of the black box?

  Neel Nanda is involved in Mechanistic Interpretability research at DeepMind, formerly of AnthropicAI, what's fascinating about the research conducted by Nanda is he gets to peer into the Black Box to figure out how different types of AI models work. Anyone concerned with AI should understand how important this is. In this video Nanda discusses some of his findings, including 'induction heads', which turn out to have some vital properties.  Induction heads are a type of attention head that allows a language model to learn long-range dependencies in text. They do this by using a simple algorithm to complete token sequences like [A][B] ... [A] -> [B]. For example, if a model is given the sequence "The cat sat on the mat," it can use induction heads to predict that the word "mat" will be followed by the word "the". Induction heads were first discovered in 2022 by a team of researchers at OpenAI. They found that induction heads were present in
Post a Comment
Read more