Defining dangerous AI: existential risk, power-intelligence, and the limits of AGI
Artificial general intelligence (AGI) features prominently in some existential risk literature, according to which the development of AGI greatly increases possible AI-induced risks to humanity. But we argue that the typical concept of AGI may be ill-suited for conceptualizing those systems that pos...
Saved in:
| Published in: | Ai and ethics (Online) Vol. 5; no. 5; pp. 5557 - 5573 |
|---|---|
| Main Author: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
Cham
Springer International Publishing
01.10.2025
|
| Subjects: | |
| ISSN: | 2730-5953, 2730-5961 |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Artificial general intelligence (AGI) features prominently in some existential risk literature, according to which the development of AGI greatly increases possible AI-induced risks to humanity. But we argue that the typical concept of AGI may be ill-suited for conceptualizing those systems that pose the greatest risks. In particular, AGI does not account for how AI agents’ abilities and behavioral strategies could be affected by complex multi-agent environments. Accordingly, we develop a simple formal model for what we call
power-intelligence
, which assesses agents’ capacities to exert influence over a resemblance class of multi-agent environments. We then taxonomize and examine multi-agent relational dynamics including competition, dependence, and complementarity. This allows us to characterize a concept of
dangerous
AI, which conjoins high power-intelligence with multi-agent dynamics that might incentivize harms to other agents. We argue that this concept of dangerous AI is more informative for existential risk assessments than both typical conceptions of AGI and “power-seeking” intelligence in the literature. |
|---|---|
| ISSN: | 2730-5953 2730-5961 |
| DOI: | 10.1007/s43681-025-00790-w |