Recently, in a much publicised post, Evan Hubinger, an Anthropic safety researcher, stated his concern that there is a greater than 10% risk of AIs killing all humans within the next decade. Many in this field have voiced similar concerns for many years. Now, however, they are more specific, the timescales are getting shorter, and fewer are disagreeing.1,2

The proximate causes are: AIs’ growing powers, the approach of their “recursive self-improvement” [Def],3 the antagonism they can display towards humans, and the well-documented failure of AI developers to align AI values with human values.4,5 The root cause of their troubling behaviour remains, however, unexplored. This is the issue I address here.

The last chapter placed responsibility for the mess our world is in, not with the individuals holding power in it, but with the out-of-balance culture that they, and we, have inherited, and with which both we, and they, comply. This is not to suggest that they, as minions of this culture, have not made it worse.

My life has witnessed the creation of many existential threats, nuclear weapons, global heating, ecological collapse, synthetic biology, space-based warfare, lethal autonomous weapons, global pollution, artificial intelligence, and many more. They are these people’s creations. And yet, if they, genuinely, wish to do good, why would they do this?