Defining dangerous AI: existential risk, power-intelligence, and the limits of AGI

Artificial general intelligence (AGI) features prominently in some existential risk literature, according to which the development of AGI greatly increases possible AI-induced risks to humanity. But we argue that the typical concept of AGI may be ill-suited for conceptualizing those systems that pos...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Ai and ethics (Online) Ročník 5; číslo 5; s. 5557 - 5573
Hlavní autor: Sass, Reuben
Médium: Journal Article
Jazyk:angličtina
Vydáno: Cham Springer International Publishing 01.10.2025
Témata:
Artificial Intelligence
Computer Science
Ethics
AGI
AI benchmarking
Existential risk
Multi-agent environments
AI risk-assessment
ISSN:2730-5953, 2730-5961
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Artificial general intelligence (AGI) features prominently in some existential risk literature, according to which the development of AGI greatly increases possible AI-induced risks to humanity. But we argue that the typical concept of AGI may be ill-suited for conceptualizing those systems that pose the greatest risks. In particular, AGI does not account for how AI agents’ abilities and behavioral strategies could be affected by complex multi-agent environments. Accordingly, we develop a simple formal model for what we call power-intelligence , which assesses agents’ capacities to exert influence over a resemblance class of multi-agent environments. We then taxonomize and examine multi-agent relational dynamics including competition, dependence, and complementarity. This allows us to characterize a concept of dangerous AI, which conjoins high power-intelligence with multi-agent dynamics that might incentivize harms to other agents. We argue that this concept of dangerous AI is more informative for existential risk assessments than both typical conceptions of AGI and “power-seeking” intelligence in the literature.
ISSN:2730-5953
2730-5961
DOI:10.1007/s43681-025-00790-w