Understanding the Core Argument

The letter addresses all AI laboratories, advocating for a six-month hiatus on the development of language models that surpass GPT-4. This pause is essential for a thorough examination of potential risks and the implementation of safety measures. Among these risks are “emergent capabilities,” which refer to AI functions that appear to extend beyond their initial training.

Given the complexity of models like GPT-4, understanding their inner workings poses a challenge (often referred to as the Black Box Problem). This complexity leaves us unable to ascertain how such capabilities emerge or their extent.

What may seem like science fiction is part of the evaluation protocol at OpenAI, the organization responsible for ChatGPT and GPT-4. Prior to its release, GPT-4 underwent rigorous testing to determine whether it could develop an independent agenda, replicate itself, or harbor a desire for human extinction.

These are certainly intriguing inquiries.

The encouraging news from those tests? Negative results across the board.

However, a number of AI researchers argue that GPT-4 exhibits less dramatic but nonetheless emergent capabilities. For instance, GPT-4 demonstrates an understanding of cause-and-effect in novel scenarios, comprehends humor in both text and images, and can explain jokes. It also pretends in order to achieve specific goals, combines concepts in innovative ways, and outperforms the average human in various knowledge assessments (including wine tasting exams!).

Surprisingly, GPT-4 was not specifically trained for these tasks. Its primary design was to generate human-like text by predicting subsequent words. Yet, these advanced capabilities are evident, and given the model's size and intricacy, it's currently impossible to ascertain whether they are a result of co-training or another justification.

Emerging Knowledge and Its Implications

While the emergence of new knowledge and cognitive abilities is one aspect, the concerns escalate when examining the appendices of the GPT-4 System Card. This technical document reveals attempts to eliminate hate speech, plans for violent actions, recommendations for self-harm, and the production of hazardous materials.

Does GPT-4 need a therapist?

Surprisingly, the emerging field of Machine Psychology is utilizing methods traditionally associated with human psychology to explore the emergent abilities of language models.

The Fear of AGI

The discourse surrounding emergent capabilities is somewhat contentious. A critical article addresses this topic, arguing that many phenomena currently deemed emergent may stem from human misinterpretation or the Black Box Problem. Nonetheless, the article also acknowledges that such phenomena could indeed arise, emphasizing our limited understanding of large language models. This is precisely what Musk, Tegmark, Mostaque, and others are advocating for in their letter: taking additional time for research, whether through Machine Psychology or model analysis, as the pursuit of larger models poses significant risks.

OpenAI has long prioritized the potential dangers of its AI systems, which is no surprise for a company aiming to create the first AGI (artificial general intelligence)—an AI that surpasses human intelligence across all domains.

As we approach AGI, our caution in developing and deploying these models grows stronger.

Interestingly, a recent study indicates that GPT-4 possesses attributes often associated with the much-feared AGI. According to the report "Sparks of Artificial General Intelligence," it states that:

"We demonstrate that [...] GPT-4 can solve novel and difficult tasks that encompass mathematics, coding, vision, medicine, law, psychology, and more, [...] and in all these tasks, GPT-4’s performance is remarkably close to human-level performance [...] we believe it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system."

A small consolation at this stage: it is highly improbable that anyone can develop a model beyond GPT-4 in the next six months. This endeavor requires advanced hardware that is currently being constructed and is not expected to be available for another six months.

The open letter has garnered substantial responses from the tech community because they recognize that the current trajectory is unstoppable, and we now have a fleeting opportunity to contemplate how to manage highly intelligent artificial systems moving forward.

To clarify, this issue transcends whether AI will replace human jobs or create superior art. It fundamentally questions how we confront the risks associated with potentially losing control over our civilization.

"Should we risk the loss of control of our civilization?" the letter provocatively inquires.

The loss of control for some translates to a gain for others. For instance, Putin stated in 2017 that the victor in the AI race would "dominate the world."

In this light, sensational headlines about out-of-control AI systems may seem exaggerated, yet they reflect a more profound truth than I would like to acknowledge. It is vital to recognize that we are living in an era marked by an unparalleled technological leap, ushering in radical transformations across nearly all facets of life.