8 ms·

This feels like the chimpanzee with a power drill. An agent is honestly just brute-force search, but guided.

by covi 6mo ago

deleted 6mo ago
[deleted]
chaos_emergent 6mo ago
Human-driven research is also brute-force but with a more efficient search strategy. One can think of a parameter that represents research-search-space-navigation efficiency. RL-trained agents will inevitably optimize for that parameter. I agree with your statement insomuch as the value of that efficiency parameter is lower for agents than humans today. It's really hard to imagine that they __won't__ exceed the human value for that efficiency parameter rather soon given that 1. there are plenty of scalar value functions that can represent research efficiency, of which a subset will result in robust training, and 2. that AI labs have a massive incentive to increase their research efficiency overall, along with billions of dollars and really good human researchers working on the problem.
- viccis 6mo ago
  >Human-driven research is also brute-force but with a more efficient search strategy No it's not. Is there anything to back that up? There's a creative aspect to human research that I've yet to see with gen AI. All it does is regurgitate stuff and get some "new" ideas via the latent space of the distribution it models. But a generative model cannot by definition create anything new. Just estimate its data well enough that it can sample it well enough to fake novelty.
groby_b 6mo ago
Is there anything in the research space that doesn't fit "brute-force search, but guided"? All of science is "gather inputs, make hypothesis, test, analyse" on repeat. There's plenty to critique in the particular guidance approach, but the overall method is the same.
gwern 6mo ago
Except the power drill isn't being used to make a better chimpanzee.
red75prime 6mo ago
Your "brute-force search, but guided" feels like oxymoron. How does it differ from "guided search"?

This feels like the chimpanzee with a power drill. An agent is honestly just brute-force search, but guided.

by covi 6mo ago

This feels like the chimpanzee with a power drill. An agent is honestly just brute-force search, but guided.

deleted 6mo ago

[deleted]

chaos_emergent 6mo ago

Human-driven research is also brute-force but with a more efficient search strategy. One can think of a parameter that represents research-search-space-navigation efficiency. RL-trained agents will inevitably optimize for that parameter. I agree with your statement insomuch as the value of that efficiency parameter is lower for agents than humans today. It's really hard to imagine that they __won't__ exceed the human value for that efficiency parameter rather soon given that 1. there are plenty of scalar value functions that can represent research efficiency, of which a subset will result in robust training, and 2. that AI labs have a massive incentive to increase their research efficiency overall, along with billions of dollars and really good human researchers working on the problem.

viccis 6mo ago
>Human-driven research is also brute-force but with a more efficient search strategy No it's not. Is there anything to back that up? There's a creative aspect to human research that I've yet to see with gen AI. All it does is regurgitate stuff and get some "new" ideas via the latent space of the distribution it models. But a generative model cannot by definition create anything new. Just estimate its data well enough that it can sample it well enough to fake novelty.

groby_b 6mo ago

Is there anything in the research space that doesn't fit "brute-force search, but guided"? All of science is "gather inputs, make hypothesis, test, analyse" on repeat. There's plenty to critique in the particular guidance approach, but the overall method is the same.

gwern 6mo ago

Except the power drill isn't being used to make a better chimpanzee.

red75prime 6mo ago

Your "brute-force search, but guided" feels like oxymoron. How does it differ from "guided search"?