6 ms·

From what I'm experiencing, we have recursive self improvement that can find local maxima, I'm not seeing really strong evidence of unguided RSI that finds the

From what I'm experiencing, we have recursive self improvement that can find local maxima, I'm not seeing really strong evidence of unguided RSI that finds the true optimal.

ViscountPenguin 5d ago
I doubt that any algorithms would be able to find a global maximum in such a large multidimensional non-convex space
Sharlin 5d ago
Doesn’t matter if there’s a rechable local maximum that happens to be comfortably beyond human level. Indeed it would be very surprising if a thing capable of RSI just happened to get stuck at human level or slightly above or below, even though its constraints are entirely different from those evolution had to work with when it created us!
- FloorEgg 5d ago
  What if you are underestimating some qualities of our intelligence and how efficient they are physically, and underestimate how much less the hardware / tech stack is in comparable efficiency? What if the bottleneck is physical chips and data centers and power and only so much can be squeezed out of algorithms? What if AI seems more impressive than it is, because the things it is good at happens to be hard for us, and it's hard for us because it's new to us (processing lots of abstract information), but the things that we are good at (manipulating physical world) are objectively orders of magnitude harder?
  - nharziro 4d ago
    We're probably not as special as you think.
    - FloorEgg 4d ago
      It has nothing to do with special. It has to do with the physics of memory access, neuron diversity and synapse density in biological brains, and the limitations of the current hardware architecture of AI.
fifilura 5d ago
You don't expect to find the global maximum. It will just always be there as an elusive target. I doubt the human brain is at global maximum.
tengbretson 5d ago
> true optimal True optimal what? Optimal intelligence? What would that be?
- dosisking 5d ago
  > True optimal what? Optimal intelligence? What would that be? Elon Musk, obviously
- HappMacDonald 5d ago
  Better at predicting real world consequences from priors than we are :)
  - unsungNovelty 5d ago
    We already know why most things happen. But beat around the bush. That is the problem. We don't care. When a severe security bug crops up, one postmortem tells where we messed up. Someone somewhere didn't do his job. Maybe they prioritised speed, money or due to management issue. But we know why. When an accident happens (Motorvehicles or otherwise), most of the time, if anyone in the chain of events did their job, it could've been avoided. But we ignored the policies, standards and laws set in place to avoid just that. There can't be a better example than Boeing for this. When it comes to natural disasters, we know how to fix most of it. But we as human race just don't want to. Hell, we knew how to freakin avoid Covid which came out of the blue. 6ft apart, wash hands and mask. We couldn't agree to that to keep us ALIVE! IMO, we don't want new technologies, new innovations, new laws in most cases. We just needs to apply what we know already - PROPERLY! But that ain't fancy. All I could remember is that tweet where musk has a $2M reward for the invention of a carbon capture device or something around those lines. And someone replied to that tweet saying, it's trees. Trees do this.
    - gary_0 5d ago
      Another big one is climate change. Ask the ASI how to solve that, and it's just going to say make a huge push for more renewables like solar, and strongly disincentivize GHG emissions. We already knew we had to do that. But the people in charge of the economy don't wanna.
avaer 5d ago
What local maxima are you seeing? That would imply that progress has stalled, which doesn't appear to be the case where I'm looking. Also, RSI is obviously guided. If only guided by "it's not giving results so we'll try something else". To require that RSI happens in a black box for it to count would be arbitrary, and also not how anyone is going to do it.
- genidoi 5d ago
  > RSI is obviously There is no publicly known reference example of RSI, we have no idea how it works or what it does to the trajectory of progress?
kovek 5d ago
Would the loss at the global maxima be much lower that the loss at most local maxima?
aaron695 4d ago
[dead]
mspn 4d ago
1. A lot of problems are inherently high-dimensional, so you're often more likely to run into saddle points 2. Some problems don't even have a bounded global maximum, a true optimal A more concerning bottleneck is diminishing returns: even with recursive improvement, capability gains could end up being only logarithmic (or worse). Even current systems seem capable of these diminishing returns only/mostly.
tim333 4d ago
>not seeing ... finds the true optimal That kind of assumes there is an optimal. It seems quite likely not so - that there will always be room for improvement in AI like we don't have a true optimal chess or go programs. They just get better and go up a bit on the ELO scale over time. like https://huggingface.co/spaces/lmarena-ai/chatbot-arena https://huggingface.co/spaces/lmarena-ai/chatbot-arena has a score for chatbots - 1506 for Claude Fable 5 at the moment for example. I guess the test for RSI is if it can make the number go up. It'll probably go through phases of self improvement being a bit rubbish, self improvement combined with human help being best which is probably where we are now, and humans not needed perhaps in the future.
- variadix 4d ago
  There is an optimal chess program and there is an optimal AI algorithm. The former is merely practically uncomputable, the latter is theoretically uncomputable.

From what I'm experiencing, we have recursive self improvement that can find local maxima, I'm not seeing really strong evidence of unguided RSI that finds the true optimal.

ViscountPenguin 5d ago

I doubt that any algorithms would be able to find a global maximum in such a large multidimensional non-convex space

Sharlin 5d ago

Doesn’t matter if there’s a rechable local maximum that happens to be comfortably beyond human level. Indeed it would be very surprising if a thing capable of RSI just happened to get stuck at human level or slightly above or below, even though its constraints are entirely different from those evolution had to work with when it created us!

FloorEgg 5d ago
What if you are underestimating some qualities of our intelligence and how efficient they are physically, and underestimate how much less the hardware / tech stack is in comparable efficiency? What if the bottleneck is physical chips and data centers and power and only so much can be squeezed out of algorithms? What if AI seems more impressive than it is, because the things it is good at happens to be hard for us, and it's hard for us because it's new to us (processing lots of abstract information), but the things that we are good at (manipulating physical world) are objectively orders of magnitude harder?
- nharziro 4d ago
  We're probably not as special as you think.
  - FloorEgg 4d ago
    It has nothing to do with special. It has to do with the physics of memory access, neuron diversity and synapse density in biological brains, and the limitations of the current hardware architecture of AI.

fifilura 5d ago

You don't expect to find the global maximum. It will just always be there as an elusive target. I doubt the human brain is at global maximum.

tengbretson 5d ago

> true optimal True optimal what? Optimal intelligence? What would that be?

dosisking 5d ago
> True optimal what? Optimal intelligence? What would that be? Elon Musk, obviously
HappMacDonald 5d ago
Better at predicting real world consequences from priors than we are :)
- unsungNovelty 5d ago
  We already know why most things happen. But beat around the bush. That is the problem. We don't care. When a severe security bug crops up, one postmortem tells where we messed up. Someone somewhere didn't do his job. Maybe they prioritised speed, money or due to management issue. But we know why. When an accident happens (Motorvehicles or otherwise), most of the time, if anyone in the chain of events did their job, it could've been avoided. But we ignored the policies, standards and laws set in place to avoid just that. There can't be a better example than Boeing for this. When it comes to natural disasters, we know how to fix most of it. But we as human race just don't want to. Hell, we knew how to freakin avoid Covid which came out of the blue. 6ft apart, wash hands and mask. We couldn't agree to that to keep us ALIVE! IMO, we don't want new technologies, new innovations, new laws in most cases. We just needs to apply what we know already - PROPERLY! But that ain't fancy. All I could remember is that tweet where musk has a $2M reward for the invention of a carbon capture device or something around those lines. And someone replied to that tweet saying, it's trees. Trees do this.
  - gary_0 5d ago
    Another big one is climate change. Ask the ASI how to solve that, and it's just going to say make a huge push for more renewables like solar, and strongly disincentivize GHG emissions. We already knew we had to do that. But the people in charge of the economy don't wanna.

avaer 5d ago

What local maxima are you seeing? That would imply that progress has stalled, which doesn't appear to be the case where I'm looking. Also, RSI is obviously guided. If only guided by "it's not giving results so we'll try something else". To require that RSI happens in a black box for it to count would be arbitrary, and also not how anyone is going to do it.

genidoi 5d ago
> RSI is obviously There is no publicly known reference example of RSI, we have no idea how it works or what it does to the trajectory of progress?

kovek 5d ago

Would the loss at the global maxima be much lower that the loss at most local maxima?

aaron695 4d ago

[dead]

mspn 4d ago

1. A lot of problems are inherently high-dimensional, so you're often more likely to run into saddle points 2. Some problems don't even have a bounded global maximum, a true optimal A more concerning bottleneck is diminishing returns: even with recursive improvement, capability gains could end up being only logarithmic (or worse). Even current systems seem capable of these diminishing returns only/mostly.

tim333 4d ago

>not seeing ... finds the true optimal That kind of assumes there is an optimal. It seems quite likely not so - that there will always be room for improvement in AI like we don't have a true optimal chess or go programs. They just get better and go up a bit on the ELO scale over time. like https://huggingface.co/spaces/lmarena-ai/chatbot-arena https://huggingface.co/spaces/lmarena-ai/chatbot-arena has a score for chatbots - 1506 for Claude Fable 5 at the moment for example. I guess the test for RSI is if it can make the number go up. It'll probably go through phases of self improvement being a bit rubbish, self improvement combined with human help being best which is probably where we are now, and humans not needed perhaps in the future.

variadix 4d ago
There is an optimal chess program and there is an optimal AI algorithm. The former is merely practically uncomputable, the latter is theoretically uncomputable.