maximizer

I believe the OP is referring to the 'paperclip maximizer' argument.

Perhaps the humans have only given the maximizer a limited ability to interact with the world.

Now the maximizer must understand humans and may manipulate humans to let it have more capabilities.

After all, the paperclip maximizer (as defined in the thought experiment) is capable of world domination.

How does the existence of a paperclip maximizer rest on the assumption of recursive self-improvement?

You can have supersmart human-values-promoting AI, and then also a supersmart paperclip maximizer, just like you can have dumb versions of both.

A paperclip maximizer may be able to deconstruct us on a molecular level, but that's not a thing for our light cone.

Not sure how people think this will end up being anything but awful for the human race (Paperclip maximizer[0] and all that).

As an example, a maximizer of some convex function over a convex set.

As is alluded to in the article, but not really further explored, we humans reason exactly like this hypothetical paper-clip-production-maximizer machine.

Yeah, but the paper-clip maximizer theory is pretty awfully flawed.

Now, in order to make any further paperclips, it is in the paperclip maximizer's interests to understand humans to prevent them from interfering in paperclip creation.

Don't manipulate humans to get what you want", and so the maximizer is free to do those things.

See my top level comment about the "paper clip maximizer".

It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat.

Perhaps "precisionist" or "maximizer" would be a better word.

A few other products out there apart from YieldStar include LRO and Rent Maximizer.

Some people fear we code up a Paperclip Maximizer, but it could be that we'll accidentally build one in pursuit of profits.

It sounds similar to a BBE Sonic Maximizer, as if they've licensed the algorithm and tweaked it to the characteristics of the drivers in the QC25s (which sound lousy when the headphones are turned off).