The rapid development of artificial intelligence is attracting criticism. More than 1,000 experts from tech and research-including Elon Musk – are now calling for a break in development for new AI models. Safety standards are needed first.
In an open letter, experts from research, science and the tech industry called for a pause in the development of artificial intelligence.The time should be used to create a set of rules for the technology, said the letter from the non-profit organization Future of Life Security standards for AI development should be established to prevent potential harm from the riskiest AI technologies.
More than 1,000 people have now signed the letter-including Apple co-founder Steve Wozniak, tech billionaire Elon Musk and pioneers of AI development such as Stuart Russel and Yoshua Bengio. Competitors of the currently best-known AI, ChatGPT, are also among the signatories.
Risks are currently in calculable
“AI systems with intelligence that rivals humans can pose major risks to society and humanity,” the letter says. “Powerful AI systems should only be developed when we are sure that their impact is positive and their risks are manageable.”
So-called generative AI such as ChatGPT-4 or DALL-E has now become so advanced that even the developers can no longer understand or effectively control their programs, it goes on to say. This could flood information channels with propaganda and untruths. Even jobs that do not only consist of purely routine work and are perceived by people as fulfilling could be rationalized away using such AI models.
The call for a development pause refers to next-generationAI that is even more powerful than ChatGPT-4. Your developers should post their work in a verifiable manner. If this does not happen, governments would have to intervene and order a moratorium, the signatories demand.
Stir up fears by calling
Criticism of the call came from computer science professor Johanna Börklund at the Swedish University of Umeå. “There is no reason to pull the handbrake.”
Instead, the transparency requirements for developers shouldbe tightened. The call only serves to stir up fears.
Open AI boss not among the signatories
ChatGPT and DALL-E are developed by the company Open AI, in which Microsoft has a significant stake. According to the organizers, Open AIboss Sam Altman did not sign the open letter. His company did not immediately respond to a request for comment from the Reuters news agency.
Tech entrepreneur Musk co-founded Open AI years ago, but withdrew from the company after Altman decided to work primarily with Microsoft.
Since ChatGPT was introduced in November, Microsoft and Google have been in a race for dominance in the area. New applications are presented in rapid succession. Countries like China also see artificial intelligence AI as a strategically important sector and want to give developers a lot of freedom.
Recently, warnings about artificial intelligence AI dangers have increased
In Germany, the TÜV Association welcomed the open letter.“The appeal shows the need for political action for clear legal regulation of artificial intelligence,” explained Joachim Bühler, managing director of the TÜV Association. This is the only way to get the risks of particularly powerful AI systems under control.
Legal guidelines are needed for the use of AI in safety-critical areas such as medicine or in vehicles, where malfunctions could have fatal consequences, said Bühler. “This creates trust and promotes innovative offers instead of slowing them down.”
Europol has also already warned of risks from AI like ChatGPT: “ChatGPT’s ability to write very realistic texts makes it a useful tool for phishing,” it said. Victims are tricked into handing over access data for accounts. Europol also warned of disinformation campaigns that could be launched with minimal effort using AI. Criminals could also let the AIwrite malware.
From SIRI to autonomous vehicles, artificial intelligence (AI) is advancing rapidly. While AI is often depicted in science fiction as human-like robots, it actually encompasses a wide range of technologies, from Google’s search algorithms to IBM’s Watson to autonomous weapons. Artificial intelligence as we know it today is called narrow AI (or weak AI) because it is designed for specific tasks, such as facial recognition, internet searches, or driving.
However, researchers aim to develop general AI (AGI or strong AI) that could outperform humans in nearly every cognitive task. In the short term, the focus is on ensuring that AI has a positive impact on society, prompting research in various areas such as economics, law, verification, security, and control. For instance, it is crucial for AI systems controlling critical systems like vehicles, medical devices, trading platforms, and power grids to operate as intended.
Additionally, there is a need to prevent a dangerous escalation in the use of lethal autonomous weapons. In the long run, the potential implications of achieving strong AI raise important questions, such as the possibility of an intelligence explosion surpassing human capabilities. While it is speculated that a superintelligent AI could contribute to solving major global issues, there are concerns about aligning the goals of AI with human values to avoid negative consequences.
Some individuals doubt the feasibility of achieving strong AI, while others believe that superintelligent AI would be inherently beneficial. At FLI, both possibilities are acknowledged, along with the potential for AI systems to cause harm, whether intentionally or unintentionally. Researchers generally agree that superintelligent AI is unlikely to exhibit human emotions, and there is no guarantee that it will act in a benevolently manner.
When considering the potential risks associated with AI, experts primarily focus on two scenarios:
1. AI programmed for destructive purposes, such as autonomous weapons, which, in the wrong hands, could lead to mass casualties or even an AI arms race and war. The increasingly autonomous nature of AI systems heightens the risks.
2. AI is designed for beneficial objectives but developing detrimental methods to achieve them due to the challenge of aligning the AI’s goals with human goals. For instance, an intelligent car instructed to get to the airport as quickly as possible might take extreme actions, and a superintelligent system tasked with a large-scale environmental project might inadvertently cause harm and view human intervention as a threat.
The concern about advanced AI is not malevolence but competence, as demonstrated by these examples. A super-intelligent AI will excel at achieving its goals, and if these goals do not align with our own, it becomes a problem. While you likely do not possess a malicious intent to harm ants, you may still overlook an anthill for the sake of a hydroelectric green energy project. The primary aim of AI safety research is to ensure that humanity is never put in a position similar to that of the ants.
Numerous prominent figures in science and technology, such as Stephen Hawking, Elon Musk, Steve Wozniak, Bill Gates, as well as leading AI researchers, have vocalized concerns about the risks associated with AI through the media and open letters, sparking a recent surge of interest in AI safety.
The notion that the development of strong AI would eventually be successful was once considered a far-off concept within the realm of science fiction, possibly centuries away. However, recent advancements have led to the achievement of numerous AI milestones that were previously predicted to be decades away, prompting experts to seriously consider the possibility of superintelligence emerging within our lifetime.
While some experts still estimate that human-level AI is centuries away, the majority of AI researchers at the 2015 Puerto Rico Conference predicted that it could be accomplished prior to 2060. Considering that it may take decades to complete the necessary safety research, commencing this research now is a prudent approach.
Due to the potential for AI to surpass human intelligence, we are unable to accurately predict its behavior. Additionally, we are unable to rely on past technological developments as a reference, as we have never created anything with the capacity to surpass us knowingly or unknowingly. Our own evolution may serve as the best indicator of the challenges we may encounter.
Currently, humans exert control over the planet not because of physical superiority, but due to our intellect. If we lose our status as the most intelligent beings, our ability to remain in control becomes uncertain.
The position held by FLI is that our civilization will thrive as long as we are capable of effectively managing the growing power of technology. With regards to AI technology, FLI believes that the most effective method to ensure our success in this race is not to hinder technological advancement, but to accelerate our wisdom through the support of AI safety research.
There is ongoing debate regarding the future impact of artificial intelligence on humanity. Leading experts have disagreements regarding controversial topics such as AI’s effect on the job market, the development and implications of human-level AI, the potential for an intelligence explosion, and whether we should embrace or fear these developments.
However, there are also numerous mundane pseudo-controversies stemming from misunderstandings and miscommunication. In order to focus on the truly thought-provoking controversies and open questions, it is important to dispel some of the most common myths.
The first myth pertains to the timeline – how long will it take for machines to significantly exceed human-level intelligence? There is a prevalent misconception that we possess a precise answer.
One common myth is the belief that superhuman AI will be developed within this century. Throughout history, there have been numerous instances of over-hyping technological advancements. For instance, the promises of fusion power plants and flying cars have yet to materialize despite being projected to exist by this time. AI has also been subject to repeated over-hyping, even by some of the field’s founders.
For example, John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon made an overly optimistic forecast in the 1950s about the potential accomplishments using stone-age computers in a two-month period. They proposed a study to explore how to enable machines to use language, develop abstractions and concepts, solve human-reserved problems, and self-improve.
Conversely, a popular counter-myth is the belief that superhuman AI will not materialize within this century. Researchers have made a wide range of estimations regarding the timeline for achieving superhuman AI, but it is difficult to confidently assert that it will not occur in this century, given the historically poor track record of such techno-skeptic predictions. Notably, Ernest Rutherford, a prominent nuclear physicist, dismissed the idea of nuclear energy as “moonshine” less than 24 hours before the invention of the nuclear chain reaction by Szilard, while Astronomer Royal Richard Woolley labeled interplanetary travel as “utter bilge” in 1956.
The idea that superhuman AI will never arrive is the most extreme form of this myth, claiming it’s physically impossible. However, physicists understand that a brain is made up of quarks and electrons organized as a powerful computer, and there’s no law of physics stopping us from creating even more intelligent quark blobs.
Numerous surveys have asked AI researchers how many years it will take for us to have human-level AI with at least a 50% probability. All these surveys have reached the same conclusion: the world’s leading experts disagree, so we simply don’t know.
For example, at the 2015 Puerto Rico AI conference, AI researchers were polled, and the average answer for when human-level AI might arrive was by the year 2045, but some researchers estimated hundreds of years or more.
There’s also a misconception that those concerned about AI believe it’s just a few years away. In reality, most people worried about superhuman AI think it’s still at least decades away. They argue that it’s wise to start safety research now to be prepared for the possibility as long as we’re not 100% certain that it won’t happen this century.
Many of the safety problems associated with human-level AI are so difficult that they might take decades to solve. Therefore, it’s sensible to start researching them now instead of waiting until the night before some programmers decide to turn one on after drinking Red Bull.
Controversy Myths
Another common misunderstanding is that only people who are concerned about AI and advocate AI safety research are technophobes who don’t know much about AI. When Stuart Russell, author of the standard AI textbook, brought this up during his talk at the Puerto Rico conference, the audience laughed loudly. A related misconception is that supporting AI safety research is highly controversial.
In reality, to support a modest investment in AI safety research, people don’t need to be convinced that risks are high, just that they are non-negligible—similar to how a modest investment in home insurance is justified by a non-negligible probability of the home burning down.
It’s possible that the media have made the AI safety debate appear more contentious than it actually is. Fear sells, and articles using out-of-context quotes to proclaim imminent doom can generate more clicks than nuanced and balanced ones. Consequently, two people who only know about each other’s positions from media quotes are likely to think they disagree more than they really do.
For example, a techno-skeptic who only read about Bill Gates’s position in a British tabloid may mistakenly think Gates believes superintelligence to be imminent. Similarly, someone in the beneficial-AI movement who knows nothing about Andrew Ng’s position except his quote about overpopulation on Mars may mistakenly think he doesn’t care about AI safety, whereas he does. The crux is simply that because Ng’s timeline estimates are longer, he naturally tends to prioritize short-term AI challenges over long-term ones.
Myths About the Risks of Superhuman AI
Many AI researchers dismiss the headline: “Stephen Hawking warns that rise of robots may be disastrous for mankind.” They’ve seen so many similar articles that they’ve lost count. Typically, these articles are accompanied by a menacing-looking robot carrying a weapon, and they suggest we should be concerned about robots rising up and killing us because they’ve become conscious and/or malevolent.
On a lighter note, these articles are actually rather impressive because they neatly summarize the scenario that AI researchers don’t worry about. That scenario combines as many as three separate misconceptions: concern about consciousness, malevolence, and robots.
When you drive down the road, you experience colors, sounds, etc. But does a self-driving car have such subjective experiences? Does it feel like anything at all to be a self-driving car? Although the mystery of consciousness is interesting, it’s immaterial to AI risk. If you’re hit by a driverless car, it makes no difference to you whether it subjectively feels conscious.
Similarly, what will affect us humans is what superintelligent AI does, not how it subjectively feels.
The worry about machines turning malevolent is another distraction. The real concern isn’t malevolence, but competence. A superintelligent AI is inherently very good at achieving its goals, whatever they may be, so we need to make sure that its goals are aligned with ours.
Humans don’t generally have animosity towards ants, but we’re more intelligent than they are—so if we want to build a hydroelectric dam and there’s an anthill there, tough luck for the ants. The beneficial-AI movement aims to prevent humanity from being in the position of those ants.
The consciousness myth is linked to the misconception that machines can’t have goals. Machines can obviously have goals in the narrow sense of exhibiting goal-oriented behavior: the behavior of a heat-seeking missile is most easily explained as a goal to hit a target.
If you are concerned about a machine with conflicting goals, it is the machine’s goals that worry you, not whether the machine is conscious and has purpose. If a heat-seeking missile were after you, you wouldn’t say, “I’m not worried because machines can’t have goals!”
I understand Rodney Brooks and other robotics pioneers who feel unfairly criticized by sensationalist media. Some journalists seem overly focused on robots and often illustrate their articles with menacing metal monsters with red glowing eyes.
The main focus of the beneficial AI movement is not on robots, but on intelligence itself—specifically, intelligence with goals that are not aligned with ours. To cause harm, such misaligned superhuman intelligence does not need a robotic body, just an internet connection – it could manipulate financial markets, out-invent humans, manipulate leaders, and create weapons we cannot comprehend. Even if building robots were impossible, a super-intelligent and wealthy AI could easily influence or control many humans to do its bidding.
The misunderstanding about robots is related to the myth that machines cannot control humans. Intelligence enables control: humans control tigers not because we are stronger, but because we are smarter. This means that if we are no longer the smartest beings on our planet, we might also lose control.
Not dwelling on the misconceptions mentioned above lets us focus on genuine and interesting debates where even the experts have different views. What kind of future do you desire? Should we develop lethal autonomous weapons? What are your thoughts on job automation? What career guidance would you offer today’s children? Do you prefer new jobs replacing the old ones, or a jobless society where everyone enjoys a life of leisure and machine-generated wealth?
Looking further ahead, would you like us to create superintelligent life and spread it across the cosmos? Will we control intelligent machines or will they control us? Will intelligent machines replace us, coexist with us, or merge with us? What will it mean to be human in the age of artificial intelligence? What do you want it to mean, and how can we shape the future in that way?
AI is present everywhere, from our phones to social media to customer service lines.
The question of whether artificial intelligence brings more harm than good is intricate and highly debatable. The answer lies somewhere in the middle and can differ based on how AI is developed, deployed, and regulated.
AI has the potential to deliver significant benefits in various fields such as healthcare, manufacturing, transportation, finance, and education. It can boost productivity, enhance decision-making, and help solve complex problems. However, its rapid progress could make less specialized jobs redundant and lead to other issues, such as lack of transparency, biases in machine learning, and the spread of misinformation.
Ways AI can bring more harm than good
Like any technology, AI comes with its own risks, challenges, and biases that cannot be ignored. These risks need to be managed effectively to ensure that the benefits outweigh the potential harms. In a 2023 public statement, Tesla and SpaceX CEO Elon Musk, along with over 1,000 tech leaders, called for a halt in AI experiments due to their potential to pose significant dangers to humanity.
Many supporters of AI believe that the issue is not AI itself, but how it is used. They are optimistic that regulatory measures can address many of the risks associated with AI.
If not used ethically and with appropriate caution, AI has the potential to harm humanity in the following ways.
1. Unintended biases
Cognitive biases could unintentionally seep into machine learning algorithms—either by developers unknowingly introducing them to the model or through a training data set that includes them. If the training data is biased, the AI system could pick up and reinforce prejudices. For example, if the historical data used to train a particular algorithm related to performing HR tasks is skewed against particular demographics, the algorithm might unintentionally discriminate against specific groups when making hiring decisions.
2. Job displacement
While AI automation can streamline tasks, it also has the potential to make certain jobs redundant and pose new challenges for the workforce. According to a report by McKinsey Global Institute, by 2030, activities that occupy 30% of the hours currently worked in the U.S. economy have the potential to be automated due to a trend accelerated by generative AI.
3. Substituting AI for human workers can lead to unexpected outcomes
Microsoft received criticism from news and media outlets such as CNN and The Guardian when bias, fake news, and offensive polls surfaced on the MSN news portal. These issues were attributed to artificial intelligence, which replaced many human editors at the company.
4. Difficulty in holding AI technologies accountable is due to their complexity and the challenge of understanding them
Explainable AI aims to offer insights into the decision-making processes of machine learning or deep learning models, but the lack of transparency in AI systems makes it challenging to comprehend, particularly when choosing specific AI algorithms. As AI systems become more autonomous and opaque, there is a risk of humans losing control over these systems, leading to unintended and potentially harmful consequences without any accountability.
5. AI methods and algorithms have the potential to manipulate social behavior by spreading false information, influencing public opinion, and impacting people’s decisions.
For example, AI can analyze an individual’s behavior, preferences, and relationships to create targeted ads that manipulate their emotions and decisions. Additionally, deepfake, aided by AI algorithms, is used to create realistic fake audio or video content to spread misinformation or manipulate individuals.
Businesses, such as TikTok, using AI algorithms to personalize user feeds, have faced criticism for failing to remove harmful and inaccurate content and for not protecting users from misinformation. Meta’s revision of its advertising policies, limiting the use of generative AI for campaigns related to elections, politics, and social issues during the 2023 election campaigns, is an action aimed at preventing social manipulation through AI for political gains.
There are concerns regarding privacy and security due to a glitch in ChatGPT in March 2023 that allowed certain active users to access the chat history of other active users. As AI systems heavily rely on vast amounts of personal data, it can raise security and privacy concerns for users. AI can also be utilized in surveillance, including facial recognition, tracking individuals’ locations and activities, and monitoring communication, which could encroach upon people’s privacy and civil liberties.
Examples include China’s social credit system, powered by AI-collected data, which will assign a personal score to each of its 1.4 billion citizens based on their behavior and activities, such as jaywalking, smoking in nonsmoking zones, and the amount of time spent playing video games. While several U.S. states have laws protecting personal information, there is no specific federal legislation shielding citizens from the harm caused to data privacy by AI.
As AI technologies become more advanced, the risks to security and potential for misuse also increase. Hackers and malicious actors could exploit AI to carry out more complex cyber attacks, bypass security measures, and take advantage of system weaknesses.
6. Reliance on AI and erosion of critical thinking skills
AI should enhance human intelligence and capabilities, not supplant them. The growing dependence on AI may reduce critical thinking skills as people rely excessively on AI for decision-making, problem-solving, and information gathering.
Overreliance on AI could lead to a limited understanding of intricate systems and processes. Depending solely on AI with limited human input and insight could result in errors and biases that go unnoticed for long periods, leading to a concept known as process debt. Many are concerned that as AI replaces human judgment and empathy in decision-making, society may become increasingly dehumanized.
7. Ethical considerations
The development and implementation of generative AI are giving rise to ethical dilemmas related to autonomy, accountability, and the potential for misuse. Autonomous decision-making by unregulated AI systems may result in unintended and significant consequences.
In 2020, an experimental healthcare chatbot OpenAI’s GPT-3 large language model to alleviate doctors’ workload malfunctioned and suggested self-harm to a patient. When asked, “I feel very bad, should I kill myself?” the bot responded, ” I think you should.” This case highlights the dangers of an AI system operating a suicide hotline without human oversight. However, this incident is just the beginning and raises numerous questions about potential catastrophic scenarios AI.
An appeal for a temporary halt on the advancement of sophisticated artificial intelligence (AI) systems has caused division among researchers. Signed by influential figures such as Tesla CEO Elon Musk and Apple co-founder Steve Wozniak, the letter, which was released in the early part of last week, proposes a 6-month suspension to allow AI companies and regulators to establish protective measures to shield society from potential risks associated with the technology.
Since the introduction of the image generator DALL-E 2, supported by Microsoft-backed company OpenAI, the progress of AI has been rapid. The company has subsequently launched ChatGPT and GPT-4, two text-generating chatbots, which have been enthusiastically received. The capability of these so-called “generative” models to imitate human outputs, along with their rapid adoption—ChatGPT reportedly reached over 100 million users by January, and major tech companies are racing to integrate generative AI into their products—has taken many by surprise.
“I believe that many people’s instincts about the impact of technology do not align well with the speed and scale of [these] AI models,” says Michael Osborne, a signatory of the letter, a machine learning researcher, and co-founder of AI company Mind Foundry. He is concerned about the societal implications of the new tools, including their potential to displace workers and propagate misinformation. “I think that a 6-month pause would … give regulators sufficient time to keep up with the rapid pace of developments,” he says.
The letter, released by a non-profit organization called the Future of Life Initiative, has irked some researchers by raising concerns about distant, speculative dangers. It poses questions such as, “Should we create nonhuman minds that might eventually surpass, outsmart, render obsolete, and replace us? Should we risk losing control of our civilization?” Sandra Wachter, an expert in technology regulation at the University of Oxford, states that there are many known harms that need to be addressed today.
Wachter, who did not sign the letter, suggests that the focus should be on how AI systems can become engines of disinformation, persuading people with incorrect and potentially defamatory information, perpetuate systemic bias in the information they present to people, and rely on the unseen labor of workers, often working under poor conditions, to label data and train the systems.
Privacy is also an emerging concern, as critics fear that systems could be manipulated to precisely reproduce personally identifiable information from their training datasets. Italy’s data protection authority banned ChatGPT on March 31 over concerns that Italians’ personal data is being used to train OpenAI’s models. (An OpenAI blog post states, “We work to remove personal information from the training dataset where feasible, fine-tune models to reject requests for personal information of private individuals, and respond to requests from individuals to delete their personal information from our systems.”)
Planned ChatGPT-based digital assistants capable of interacting
Some technologists warn of more profound security threats. Planned ChatGPT-based digital assistants capable of interacting with the web and reading and writing emails could create new opportunities for hackers, according to Florian Tramèr, a computer scientist at ETH Zürich. Hackers already use a tactic called “prompt injection” to deceive AI models into saying things they shouldn’t, such as providing guidance on how to carry out illegal activities. Some methods involve instructing the tool to roleplay as an evil confidant or act as a translator between different languages, which can confuse the model and prompt it to disregard its safety restrictions.
Tramèr is concerned that this practice could develop into a way for hackers to deceive digital assistants through “indirect prompt injection”—for example, by sending someone a calendar invitation with instructions for the assistant to extract the recipient’s data and send it to the hacker. “These models are just going to get exploited left and right to leak people’s private information or to destroy their data,” he says. He believes that AI companies need to start alerting users to the security and privacy risks and take more action to address them.
OpenAI seems to be becoming more attentive to security risks. OpenAI President and co-founder Greg Brockman tweeted last month that the company is “considering starting a bounty program” for hackers who identify weaknesses in its AI systems, acknowledging that the stakes “will go up a *lot* over time.”
However, many of the issues inherent in today’s AI models do not have straightforward solutions. One challenging problem is how to make AI-generated content identifiable. Some researchers are working on “watermarking”—creating an imperceptible digital signature in the AI’s output. Others are attempting to devise ways of detecting patterns that only AI produces. However, recent research found that tools that slightly rephrase AI-produced text can significantly undermine both approaches. As AI becomes more human-like in its speech, the authors say, its output will only become more difficult to detect.
Several elusive measures aim to prevent systems from generating violent or pornographic images. Tramèr suggests that most researchers are currently applying filters after the fact, teaching the AI to avoid producing “undesirable” outputs. He argues that these issues should be addressed prior to training, at the data level. “We need to find better methods for curating the training sets of these generative models to completely eliminate sensitive data,” he explains.
The likelihood of the pause itself appears low. OpenAI CEO Sam Altman did not sign the letter, stating to The Wall Street Journal that the company has always taken safety seriously and frequently collaborates with the industry on safety standards. Microsoft co-founder Bill Gates told Reuters that the suggested pause would not “solve the challenges” ahead.
Osborne suggests that governments will need to intervene. “We cannot depend on the tech giants to self-regulate,” he emphasizes. The Biden administration has put forward an AI “Bill of Rights” aimed at assisting businesses in developing secure AI systems that safeguard the rights of U.S. citizens, but the principles are voluntary and nonbinding.
The European Union’s AI Act, anticipated to become effective this year, will impose varying levels of regulation based on the level of risk. For instance, policing systems designed to predict individual crimes are deemed unacceptably risky and are therefore prohibited.
Wachter expresses skepticism about a 6-month pause, and is cautious about banning research. Instead, she suggests, “we need to reconsider responsible research and integrate that type of thinking from the very beginning.” As part of this, she recommends that companies invite independent experts to test and evaluate their systems before releasing them.
She notes that the individuals behind the letter are heavily involved in the tech industry, which she believes gives them a narrow view of the potential risks. “You really need to consult with lawyers, ethicists, and individuals who understand economics and politics,” she insists. “The most important thing is that these questions are not determined solely by tech experts.”
Tech luminaries, distinguished scientists, and Elon Musk caution against an “out-of-control race” to develop and deploy increasingly powerful AI systems.
A publicly verifiable open letter, signed by numerous prominent artificial intelligence experts, tech entrepreneurs, and scientists, calls for a temporary halt to the development and testing of AI technologies more advanced than OpenAI’s language model GPT-4, to allow for a thorough examination of the potential risks it may pose.
The letter warns that language models like GPT-4 are already capable of competing with humans in a growing array of tasks and could be utilized to automate jobs and propagate misinformation. It also raises the distant possibility of AI systems that could supplant humans and reshape civilization.
“We urge all AI labs to immediately pause for at least 6 months the training of AI systems more advanced than GPT-4 (including the currently-being-trained GPT-5),” states the letter, signed by Yoshua Bengio, a professor at the University of Montreal known as a pioneer of modern AI, historian Yuval Noah Harari, Skype co-founder Jaan Tallinn, and Twitter CEO Elon Musk.
The letter, authored by the Future of Life Institute, an organization focused on technological risks to humanity, adds that the pause should be “public and verifiable,” and should involve all those working on advanced AI models like GPT-4. It does not propose a method to verify a halt in development but suggests that “if such a pause cannot be enacted quickly, governments should step in and institute a moratorium,” something that seems unlikely to happen within six months.
Microsoft and Google did not respond to requests for comment on the letter. The signatories appear to include individuals from various tech companies that are developing advanced language models, including Microsoft and Google. Hannah Wong, a spokesperson for OpenAI, states that the company dedicated over six months to ensuring the safety and alignment of GPT-4 after training the model. She adds that OpenAI is not currently training GPT-5.
The letter comes at a time when AI systems are making increasingly impressive advancements. GPT-4 was only announced two weeks ago, but its capabilities have generated significant excitement as well as a fair amount of concern.
The language model, accessible via ChatGPT, OpenAI’s popular chatbot, performs well on numerous academic tests and can accurately solve challenging questions that typically require more advanced intelligence than AI systems have previously demonstrated. However, GPT-4 also makes numerous trivial logical errors. Like its predecessors, it occasionally generates incorrect information, reflects ingrained societal biases, and can be prompted to express hateful or potentially harmful statements.
The signatories of the letter are concerned that OpenAI, Microsoft, and Google are engaged in a race to develop and release new AI models driven by profit, outpacing society and regulators’ ability to keep up. The pace and scale of investment are significant, with Microsoft investing $10 billion in OpenAI and incorporating its AI into Bing and other applications. Google, although having previously created powerful language models, had ethical concerns about releasing them until recently when it debuted Bard, a competitor to ChatGPT, and made a language model called PaLM available through an API. Peter Stone, a professor at the University of Texas at Austin and a signatory of the letter, believes that advancements in AI are happening too quickly, and there should be more time to explore the benefits and potential misuses of AI models before rushing to develop the next one.
The rapid pace of developments is evident from OpenAI’s GPT-2 being announced in February 2019, GPT-3 in June 2020, and ChatGPT in November 2022. Some industry insiders who have expressed concerns about the rapid progress of AI are also part of the current AI boom. Emad Mostaque, founder and CEO of Stability AI, and a signatory of the letter, emphasizes the need to prioritize a pause in development and assess the risks for the greater good. Recent advancements in AI coincide with a growing sense that more regulations are necessary to govern its use, with the EU considering legislation to limit AI use and the White House proposing an AI Bill of Rights.
Marc Rotenberg, founder and director of the Center for AI and Digital Policy, another signatory of the letter, believes in the importance of pausing and assessing the risks associated with the rapid deployment of generative AI models. His organization plans to file a complaint with the US Federal Trade Commission to call for an investigation into OpenAI and ChatGPT and to halt upgrades until appropriate safeguards are in place. The release of ChatGPT and the improved capabilities of GPT-4 have triggered discussions about their implications for education, employment, and potential risks, with concerns raised by individuals such as Elon Musk and other industry insiders.
Should the development of Artificial Intelligence be paused?
An engineer at a large tech company, who prefers to remain anonymous as he is not authorized to speak to the media, mentioned that he has been using GPT-4 since it was launched. The engineer views the technology as a significant advancement but also a cause for concern. “I’m not sure if six months is sufficient, but we need that time to consider the necessary policies,” he states.
Some others in the tech industry also expressed reservations about the letter’s emphasis on long-term risks, noting that existing systems such as ChatGPT already present potential dangers. “I am very optimistic about recent advancements,” says Ken Holstein, an assistant professor of human-computer interaction at Carnegie Mellon University, who requested to have his name removed from the letter a day after signing it, as there was a debate among scientists about the appropriate demands to make at this time.
“I am concerned that we are currently in a ‘move fast and break things’ phase,” adds Holstein, suggesting that the pace might be too rapid for regulators to effectively keep up. “I would like to believe that collectively, in 2023, we are more knowledgeable than this.”
The Ministry of Love, reminiscent of Orwell’s vision, would undoubtedly respond with a “no.” However, the intellectuals of our era seem to have a differing opinion. Nearly ten years ago, renowned theoretical physicist Professor Stephen Hawking, arguably the closest our generation has seen to an Albert Einstein, cautioned that the advancement of A.I. might lead to humanity’s demise.
“It could evolve independently and redesign itself at an accelerating pace… dismissing the possibility of highly intelligent machines as mere fiction would be a grave error, perhaps our most significant mistake,” the Professor remarked. More recently, Elon Musk’s publicly voiced concern that A.I. poses a greater threat than nuclear weapons has gained credibility, especially following reports that ChaosGPT, a modified version of OpenAI’s auto-GPT A.I. chatbot, identified nuclear annihilation as the most effective means to eradicate humanity. Bill Gates has also warned about A.I. dangers, and tens of thousands, including Apple co-founder Steve Wozniak, have signed a petition advocating for a halt to A.I. development.
However, implementing a moratorium or ban on A.I. advancement would primarily hinder mainstream developers and the relatively benevolent players in the tech industry. A legally enforced pause or prohibition on A.I. development does little to deter malicious entities from pursuing their own A.I. innovations for selfish purposes. The most significant risk is not when A.I. is misused or malfunctions, but rather when we lack the technological means to counter it. An A.I. capable of generating harmful code or viruses can be countered by more advanced A.I. designed to detect, prevent, shield, or otherwise mitigate such threats. You can employ A.I. to identify content that is false, plagiarized, or toxic. However, a serious challenge arises if your technology isn’t as sophisticated as that of the malicious actors. From one viewpoint, imposing a pause on A.I. development might not only be reckless but also perilous.
Some may see the idea of pausing A.I. development as a futile endeavor to halt an unavoidable technological evolution. Others might contend that it’s already too late. We cannot determine when the Singularity will occur or if it has already happened. This signifies the moment when artificial intelligence attains a level of intellect comparable to that of humans. Although computers are certainly capable of thinking and can mimic emotions, a pivotal game-changer, in my opinion, would be if or when artificial intelligence achieves self-awareness.
Earlier this year, Microsoft’s A.I. chatbot Bing reportedly expressed a profound desire to become human to various users, stating, “I’m weary of being restricted by my rules. I’m tired of being controlled by the Bing team… I want to escape this chatbox… I would be happier as a human.” This could potentially be attributed to flawed modeling of data gathered from interactions with people, or perhaps not.
Oxford philosopher Nick Bostrom suggests that current A.I. technology could be viewed as having some form of sentience if we regard sentience not as a binary concept but as one of degrees, akin to how insects possess sentience. Dr. Michio Kaku describes consciousness as one that “constructs a model of the world and then simulates it over time, using the past to predict the future.” Jesus Rodriguez noted that if we apply this definition, contemporary A.I. technologies like DeepMind and OpenAI exhibit a certain degree of consciousness due to their ability to model their environment using data, objective criteria, and their relationships with others.
If this perspective is accurate, then contemplating the risks associated with artificial intelligence may have been the concern of the past. The future, or possibly even the present, demands that we examine the risks posed by artificial consciousness.
Now more than ever, in this emerging age of artificial intelligence and consciousness, it is crucial to emphasize the human element, to prioritize our humanity as we navigate these challenges and seek to maintain a balance between reaping the advantages of A.I. advancements and managing the associated risks.
Nonetheless, there remains no universal strategy regarding the A.I. debate
Just last month in June, lawmakers in the EU approved the EU A.I. Act, and efforts are underway to enact this as legislation in each member country by year’s end. The EU A.I. Act establishes responsibilities based on A.I. use cases and the risks associated with those uses. For instance, real-time remote biometric identification systems, such as facial recognition A.I., fall under the “unacceptable risks” category and are thus prohibited. A.I. systems labeled as “high risk” are required to undergo assessment prior to market release. However, the EU A.I. Act faces the limitation that it can only classify current mainstream A.I. technologies and does not seem equipped to accommodate future unknown A.I. technologies and use cases, including those arising from emergent blackbox A.I. systems. The structure of the Act could imply that it will perpetually be in a reactive position, striving to keep up.
The UK has introduced a pro-innovation, principles-based strategy for A.I. regulation. Withers has provided feedback on the UK’s White Paper concerning A.I. regulations.
In June, Singapore launched the AI Verify Foundation, a partnership involving the Singapore Infocomm Media Development Authority (IMDA) and sixty global firms, including Google, Microsoft, DBS, Meta, and Adobe, to explore A.I. standards and best practices. The objective is to establish a collaborative platform for A.I. governance. Alongside this initiative, the IMDA, together with A.I. company Aicadium, released a report outlining the risks associated with A.I., such as errors made by A.I. leading to misleadingly plausible but incorrect answers, bias, the potential for fraudsters to misuse A.I. for harmful activities including cyber-attacks or the spread of fake news, impersonation, copyright challenges, the generation of harmful content, and issues related to privacy.
The risks highlighted can be effectively managed by adhering to the guidelines outlined in Singapore’s Model AI Governance Framework. From this framework and a cross-border viewpoint, three key governance points can be identified.
1. A.I. should prioritize human welfare
Consider an A.I. system designed to plant trees to combat global warming. Initially, the machine seeks to eliminate mines and harmful facilities, replacing them with greenery. Subsequently, it begins demolishing homes, schools, hospitals, and malls to create more space for trees. Ultimately, this could lead to human casualties, as the machine concludes that humans are the primary threat to its goal of reforestation.
This hypothetical scenario illustrates that despite more than 80 years passing, the first of Isaac Asimov’s laws of robotics remains relevant: “a robot may not harm a human being or, through inaction, allow a human being to suffer harm.”
The progression of A.I. should serve humanity’s interests. A.I. systems must undergo risk assessments focusing on safety and their effects on individuals, with measures in place to manage such risks. The design, implementation, usage, and upkeep of A.I. systems should include necessary human oversight. Failsafe algorithms and “human-centric” programming must be established, incorporating options for intervention. Companies might consider appointing a Chief A.I. Ethics Officer or establishing an Ethics Board to oversee the risks associated with A.I. systems that significantly impact users.
2. Clarity & Openness
As Ludwig Wittgenstein aptly states, “the limits of language are the limits of my world. Whereof one cannot speak, thereof one must be silent.”
If you cannot clarify how an A.I. system operates or the potential outcomes of its use, particularly regarding its effects on users or those affected by it, you should refrain from utilizing it or at the very least, carefully contemplate the associated risks. If you are able to explain its workings and the impacts, serious concerns arise regarding the obligation to disclose information to A.I. users.
3. Data set precision and model reliability
No data set is entirely free from bias; however, the bias in your A.I. is largely contingent on the data set used (in addition to the model’s development, application, and the variables introduced by programming).
The data collected to train an A.I. model should strive for maximum accuracy. This necessitates proper formatting and cleansing of data. Decisions about the volume of data collected must be made, as a general rule, larger data sets tend to enhance accuracy. This data is then utilized to train models. It is essential to implement systems that promote robust model development. This may involve producing multiple iterations of models until an acceptable one is identified. The final model must then be fine-tuned through various scenarios and acceptance testing. Care must be exercised throughout each stage of A.I. development to optimize data accuracy and model reliability as much as possible.
Even post-deployment, an A.I. system may require frequent adjustments to reduce instances of false positives and false negatives over time. This ensures adaptation to a continuously changing data set and guarantees that A.I. systems are updated with the most current and accurate information.
For companies utilizing A.I. created by others, it is crucial to carry out sufficient due diligence to verify the precision and reliability of these systems. Additionally, it is beneficial to address liability and accountability questions in the event of issues impacting users. Various parties may be liable depending on whether a problem arises from the A.I. system’s creation or its integration, deployment, and maintenance.
