bostrom

But all Bostrom has done is suggest We Should Be Very Worried because It Might Go Horribly Wrong.

Granted, most of what Bostrom refers to in context of the biological path is mostly related to human augmentation, genetics, selective breeding - things of that nature.

Regarding global warming, Bostrom was handicapped by writing this in 2002, but things are looking a lot worse now.

This book by Nick Bostrom will help you find answers: Superintelligence: Paths, Dangers, Strategies[1] The author is the director of the Future of Humanity Institute at Oxford.

In Superintelligence, Bostrom does not presume to know the path via which we will arrive at AGI.

> Bostrom is a philosopher That is my main concern about people writing about the future in general.

But then that's precisely the point: Bostrom is a philosopher.

That's the point of people like Bostrom when they're advocating for this, because before they started writing about this, it seems like a lot of people weren't even thinking it could be a problem.

Note that I think Bostrom is referencing the section on the paperclip thought experiment in his book, which goes into a bit more detail than this article and to me characterises the paper clip optimiser as an example of above human level intelligence.

That's some useful feedback to know as I've considered recommending that book but haven't read it myself as it's redundant to what I've already read, but if Bostrom doesn't seem to explain very well that there's a distinction between intelligence and goals/values, I'll just keep linking to some basic online texts.

The alternative would be like "accidentally" creating an atom bomb, without the Manhattan project (to borrow an example from Bostrom).

I recommend "Superintelligence: Paths, Danger, Strategies," by Nick Bostrom (it's almost certainly where Musk got his ideas; he's quoted on the back cover).

By the way, Nick Bostrom uses the term "infrastructure profusion" to describe a particular nightmare scenario of AI, and I think it's evocative, so I've started to think of human intelligence as prone to this profusion, too.

Instead, the most you hear from them is "we're really far away from that!" and "we don't know and I'm sure we'll figure it out when we're near that point." These comments, from Andrew Ng and the like (insanely brilliant people!) show that they really haven't read Bostrom, etc.

Bostrom's idea is that the Universe is (probably) a simulation of something else, running on a computational device embedded inside another Universe.

In Bostrom's world you have a mad machine blindly consuming everything and everyone for trivial ends.

There is a new wave of fear of strong AI, based on writings by Nick Bostrom, especially his book 'Superintelligence'.