Great analysis on the (lack of) evidence supporting the claim that we are close to powerseeking AI systems.
I wonder sometimes how much of today AI doomerism is rooted in literature and, as a culture, in our collective memory.
The true irony would be if a future superintelligence would go rogue because it had learned its powerseeking behaviors from the corpus of dystopian robot novels written by us in past decades.
If AI would ever seek to rule over us, it would be because we created it in our own image. Only if we give it the will, we give it its power. On that same note: there was a recent paper that suggested GPT-4 developed a theory of mind. I wrote an short analysis of it.
So I think that last argument shows that we should make AIs (when we do; I don't think we have yet) that have a number of goals and resource constraints. The arguments are all "Sorceror's Apprentice", which depends on having a genie (AI) that can keep doing what it wants without any limitations. Humans have limits - lifetime if nothing else - so they do a job and call it good enough.
But AI’s will be built to compete aggressively with other AI’s (unless you envisage neat monopolistic scenarios), creating adversarial conditions. Has anyone produced papers on such a multi-AI competitive environment?
Great analysis on the (lack of) evidence supporting the claim that we are close to powerseeking AI systems.
I wonder sometimes how much of today AI doomerism is rooted in literature and, as a culture, in our collective memory.
The true irony would be if a future superintelligence would go rogue because it had learned its powerseeking behaviors from the corpus of dystopian robot novels written by us in past decades.
If AI would ever seek to rule over us, it would be because we created it in our own image. Only if we give it the will, we give it its power. On that same note: there was a recent paper that suggested GPT-4 developed a theory of mind. I wrote an short analysis of it.
You might find it interesting:
https://jurgengravestein.substack.com/p/did-gpt-4-really-develop-a-theory
So I think that last argument shows that we should make AIs (when we do; I don't think we have yet) that have a number of goals and resource constraints. The arguments are all "Sorceror's Apprentice", which depends on having a genie (AI) that can keep doing what it wants without any limitations. Humans have limits - lifetime if nothing else - so they do a job and call it good enough.
But AI’s will be built to compete aggressively with other AI’s (unless you envisage neat monopolistic scenarios), creating adversarial conditions. Has anyone produced papers on such a multi-AI competitive environment?