If Anyone Builds It, Everyone Dies: Destructive potential of sentient AI

AI is still not intelligent in the human sense, but fears of a superintelligent threat to humanity are not misplaced

If Anyone Builds It, Everyone Dies: The Case Against Superintelligent AI

by Eliezer Yudkowsky & Nate Soares

Published by Penguin Random House

231 pages ₹999

  Worries about a superintelligent and sentient artificial intelligence (AI) destroying humanity is not new. Science fiction novels and movies have explored the scenario many times. Even some highly respected AI researchers whose breakthroughs have led to the current era of Large Language Models (LLMs) and Generative AI have warned that the technology could soon get too powerful and pose a threat for humans or humanity. In early 2023, hundreds of AI scientists, including Geoffrey Hinton and Yoshua Bengio, whose research led to the current generation of AI, signed a letter that said: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

 

If Anyone Builds It, Everyone Dies by Eliezer Yudkowsky and Nate Soares is not simply rehashing tropes about superintelligent AI posing a threat to humanity though their book stands out because of the way it examines what the current generation of AI is, what it isn’t yet, and what it can be. The authors, incidentally, were among those who signed the letter along with Drs Hinton and Bengio. However, as they point out in their introduction, they thought the sentence was a severe understatement.

 

The authors are AI scientists with serious credentials. Mr Yudkowsky is the co-founder of the non-profit Machine Intelligence Research Institute (MIRI) and also one of the earliest AI scientists to work on the field of AI alignment. For the uninitiated, AI alignment is the field of research that focuses on ensuring that AI does not go rogue and that AI systems are aligned to human values and ethics and intentions. Dr Soares is the president of MIRI and also a long-time researcher on decision theory and alignment.

 

The book is divided into three parts and, for this reviewer, the first part dealing with how modern AIs are produced, can they have wants, what is intelligence, will AI hate us and other such questions is the most insightful and brings great clarity to the discussion.

 

As the authors point out, while the engineers producing the new AI models do know a lot about AI, the weights to give to various parameters and so on, they do not fully understand how it produces the answers that it does. This is perhaps why AI models and bots can be induced to hallucinate or behave in ways that are highly irrational. Producers of LLMs tweak algorithms and weights and do all sort of other fine-tuning to make their models better — but so far, they still have not fully figured out how to make fool-proof models.

 

Is AI intelligent? Most scientists agree that it is not truly intelligent in the way humans define intelligence. Is AI sentient? Again, the consensus is that AI is not yet sentient, though it could be in the future. The problem is that while AI is not sentient, it does exhibit some behaviour that comes close to human self-preservation behaviour. For example, when AI models have been caught cheating and instructed not to do so, they continue to cheat but try to hide their cheating.

 

So far, AI is still not intelligent in a broad range of thinking — but as the authors point out, this does not mean that they will not soon become more intelligent. A few years ago, AI would not draw hands and fingers properly — but they have improved dramatically.

 

Just as humans procreated and evolved with each generation, an AI model can soon produce more intelligent models, which in turn creates an even more intelligent AI model, and so on until super intelligent AI models come into being. And unlike humans, this will not take hundreds of thousands of years; superintelligent AI can be a reality within years, not decades.

 

What is scary about the book is the realisation that superintelligent AI may not end humanity because it hates humans — but because it wants something else, and humans are getting in the way.

 

Part two of the book details a hypothetical extinction scenario though, as anyone who has read the first part would realise, this is not the only one — there are others that could lead to human extinction.

 

The third part of the book represents wishful thinking. The authors hope for a multi-nation movement for safeguards but that is unlikely to happen. Some nations, like the United States, have already decided that innovation is more important than safeguards.

 

The book is not easy to read. The authors have chosen a format that does not lead to easy progress. And their narrative style can get a bit tedious at times. But do not let that deter you. What they say is extremely important.

 

         

The reviewer, former editor of Businessworld and Business Today magazines, is author of Will India Get Rich Before It Turns 100? A Reality Check, and co-author of Coffee King: The Swift Rise and Sudden Death of Café Coffee Day Founder V G Siddhartha