Roko's basilisk

Proposed as a transhumanist thought experiment in 2010 by user Roko on the LessWrong Internet discussion forum.

According to LessWrong, discussion of Roko's basilisk was banned from the forum for several years as part of a general policy against spreading potential information hazards. This in fact drew further attention to the topic, and other websites began sharing information about it.[1]

Roko's basilisk

1. A hypothetical, otherwise benevolent artificial intelligence of the future that would have an incentive to torture anyone who, having imagined (or even heard about) said AI, had not helped bring about its existence; the thought experiment which posits its existence.
   • 2020, Griffin Kao, Jessica Hong, Michael Perusse, Weizhen Sheng, Turning Silicon into Gold, Springer (Apress), page 177,

     Roko's Basilisk closely resembles traditional religious belief systems, merely replacing the role of God with a super-AI instead.

   • 2021, Richard Heimann, Doing AI, BenBella Books (Matt Holt Books), page 137,

     Roko's Basilisk is a thought experiment about the potential risks involved in developing so-called true artificial intelligence. The premise is that an all-powerful artificial super-intelligence could retroactively punish those who did not help bring about its existence. Roko's Basilisk resembles a futurist version of Pascal's Wager in that it suggests people should weigh the possible infinite gain and infinite punishment for accepting or rejecting the creation of artificial super-intelligence.

   • 2023, James B. Miles, AI and the End of Humanity, Troubador Publishing (Matador), page 168:

     Roko's Basilisk has been described as the most terrifying thought experiment of all time, or as Slate’s David Auerbach subtitled his 2014 article: "Why Are Techno-futurists So Freaked Out by Roko's Basilisk?"