Brief history of AI and AI safety (This is taken in it's entirety form "A newcomer’s guide to the technical AI safety field" by Chin Ze Shen. This is just an excerpt from a longer article that's well worth reading.
"AI development can roughly be divided into the following era:
1952 - 1956: The birth of AI by a handful of scientists, with some research progress in neurology, cybernetics, information theory, and theory of computation, leading up to the Dartmouth Workshop of 1956.
1956 - 1974:The era of symbolic AI, with more progress in “reasoning as search” paradigm, natural language, and robotics.
1974 - 1980: The first AI winter, where researchers realized achieving AI was harder than previously thought, and funding for AI research decreased.
1980 - 1987: The second AI summer, with developments in knowledge based systems and expert systems.
1987 - 1993: The second AI winter, with the advent of more cost-effective general Unix workstations with good compilers, forcing many commercial deployments of expert systems to be discontinued.
1993 - 2011: The field of AI continues to advance with an increase in computing power and more sophisticated mathematical tools, contributing to progress in various fields like industrial robotics, speech recognition, and search engines.
2011 - present: The deep learning era, with progress coming from access to large amounts of data (big data), cheaper and faster computers, and advanced machine learning techniques.
Here’s a series of more recent AI-related breakthroughs, stolen directly from this post by Jacob Cannell:
1996: Deep blue crushes Kasparov, breaking chess through brute force scaling of known search algorithms.
2010: ANNs are still largely viewed as curiosities which only marginally outperform more sensible theoretically justified techniques such as SVMs on a few weird datasets like MNIST. It seems reasonable that the brain's exceptionality is related to its mysterious incredible pattern recognition abilities, as evidenced by the dismal performance of the best machine vision systems.
2012: Alexnet breaks various benchmarks simply by scaling up extant ANN techniques on GPUs, upending the field of computer vision.
2013: Just to reiterate that vision wasn't a fluke, Deepmind applies the same generic Alexnet style CNNs (and codebase) - combined with reinforcement learning - to excel at Atari.
2015: In The Brain as a Universal Learning Machine, I propose that brains implement a powerful and efficient universal learning algorithm, such that intelligence then comes from compute scaling, and therefore that DL will take a convergent path and achieve AGI after matching the brain's net compute capacity.
2015: Two years after Atari, Deepmind combines ANN pattern recognition with MCTS to break Go.
2016: It's now increasingly clear (to some) that further refinements and scaling of ANNs could solve many/most of the hard sensory, pattern recognition, and even control problems that have long eluded the field of AI. But for a believer in brain exceptionalism one could still point to language as the final frontier, the obvious key to grand human intelligence.
2018: GPT-1 certainly isn't very impressive
2019: GPT-2 is shocking to some - not so much due to the absolute capabilities of the system, but more due to the incredible progress in just a year, and progress near solely from scaling.
2020: The novel capabilities of GPT-3, and moreover the fact that they arose so quickly merely from scaling, should cast serious doubts on the theory that language is the unique human capability whose explanation requires complex novel brain architectural innovations.
2021: Google LaMDa, OpenAI CLIP, Megatron-Turing NLG 530B, Codex
2022: Disco Diffusion, Imagen, Stable Diffusion, Chinchilla, DALL-E-2, VPT, Minerva, Pathways ...
In parallel, the field of AI safety was also growing:
2000: Eliezer Yudkowsky founds Singularity Institute for Artificial Intelligence (SIAI) (later renamed as Machine Intelligence Research Institute (MIRI) in 2013)
2005: Nick Bostrom and Anders Sandberg found Future of Humanity Institute
2014: Nick Bostrom publishes Superintelligence: Paths, Dangers, Strategies; DeepMind establishes Ethics Board (not to be confused with the Ethics & Safety Team) to “consider dangers of AI”; Max Tegmark, Jaan Tallinn, and others found Future of Life Institute (FLI)
2016: Researchers from Google Brain, Stanford University, UC Berkeley, and OpenAI publishes Concrete Problems in AI Safety; Stuart Russell founds the Center for Human-Compatible AI (CHAI)
2017: OpenAI receives grant from Open Philanthropy to “reduce potential risks from advanced AI”; Robert Miles starts YouTube channel on AI safety
2018: LessWrong team launches the Al Alignment Forum; Rohin Shah launches the Alignment Newsletter
2019: Stuart Russell publishes Human Compatible: Artificial Intelligence and the Problem of Control
2020: Brian Christian publishes The Alignment Problem
2021: Several other AI safety related organizations are founded, e.g. Alignment Research Center (ARC), Anthropic, Redwood Research, etc
2022: More AI safety related organizations are founded, e.g. Aligned AI, Center for AI Safety, Conjecture, etc
Unsurprisingly, the AI safety field has grown very rapidly in the last few years as most of the breakthroughs in deep learning happened. However, it is also worth noting that the pioneers of the field started working on the alignment problem before deep learning ‘took off’, which explains some of the paradigms that will be covered below."
Comments
Post a Comment