6 ms·
ResearchAgent: Iterative Research Idea Generation Using LLMs
- not-chatgpt 2y agoCool idea. Never gonna work. LLMs are still generative models that spits out training data, incapable of highly abstract creative tasks like research. I still remember all the GPT-2 based startup idea generators that spits out pseudo-feasible startups.
- llm_trw 2y agoThey just need to be better at it than humans, which is a rather low bar when you go beyond two unrelated fields.
- growthwtf 2y agoI think that ship has sailed, if you believe the paper (which I do). LLMs are already super-human at some highly abstract creative tasks, including research. There are numerous examples of LLMs solving problems that couldn't be found in the training data. They can also be improved by using reasoning methods like truth tables or causal language. See Orca from Microsoft for example.
- bigyikes 2y agoIgnoring the “spits out training data” bit which is at best misleading, it’s interesting that you use the word “abstract” here. I recently followed Karpathy’s GPT-from-scratch tutorial and was fascinated with how clearly you could see the models improving. With no training, the model spits out uniformly random text. With a bit of training, the model starts generating gibberish. With further training, the model starts recognizing simple character patterns, like putting a consonant after a vowel. Then it learns syllables, and then words, and then sentences. With enough training (and data and parameters, of course) you eventually yield a model like GPT-4 that can write better code than many programmers. It’s not always that clear cut, but you can clearly observe it moving up the chain of abstraction as the training loss decreases. What happens when you go even bigger than GPT-4? We have every reason to believe that the models will be able to think more abstractly. Your “never gonna work” comment flies in the face of exponential curve we find ourselves on.
- ethanwillis 2y agoIf we keep extrapolating eventually GPT will be omniscient. I really can't think of any reason why that wouldn't be the case, given the exponential curve we find ourselves on.
- esafak 2y agoHow do you know you're not on a logistic curve? Don't you think costs and the availability of training data might impose some constraints?
- deleted 2y ago[deleted]
- dragonwriter 2y agoWith real world phenomena that have resource constraints anywhere, a good rule of thumb is: if it looks like an exponential curve, walks like an exponential curve, and quacks like an exponential curve, it’s definitely a logistic curve
- HeatrayEnjoyer 2y agoThe entire universe is training data.
- esafak 2y agoIt is, but we -- humans, and computers -- are limited in our ability to learn from it. We both learn more easily from structured data, like textbooks.
- inference-lord 2y agoI think they're being factitious?
- ethanwillis 2y agoI am. And I think it says a lot about the state of things that many people think I'm being completely serious.
- deleted 2y ago[deleted]
- ramraj07 2y agoI have asked chat GPT to generate hypotheses on my PhD topic that I know every single piece of existing literature about and it actually threw out some very interesting ideas that do not exist out there yet (this was before they lobotomized it).
- ta988 2y agoDid you try with the API directly? I've had great results with my own prompts, much less so with the chatgpt one.
- voxl 2y ago> (this was before they lobotomized it) Of course, of course. Because god forbid anyone be able to reproduce your suggestion. Funnily enough I tried the same and have the exact opposite experience.
- CuriouslyC 2y agothey don't just spit out training data, they generalize from training data. They can look at an existing situation and suggest lines of experimentation or analysis that might lead to interesting results based on similar contexts in other sciences or previous research. They're undertrained on bleeding edge science so they're going to falter there but they can apply methodology just fine.
- krageon 2y agoWhen you're this confident and making blanket statements that are this unilateral, that should tell you you need to take a step back and question yourself.
- pedalpete 2y agoI've found where LLMs can be useful in this context is around free-associations. Because they don't really "know" about things, they regularly grasp at straws or misconstrue intended meaning. This, along with the volume of language (let's not call it knowledge) result in the LLMs occasionally bringing in a new element which can be useful.
- gotts 2y agoCan you list some examples where free-associations from LLM were useful to you?
- bongodongobob 2y agoAssume free-associations = hallucinations. Assume hallucinations are exactly what makes LLMs useful and your question can be rephrased as "Can you list some examples where LLMs were useful to you?"
- ec109685 2y agoHallucinations are lies. So not the same thing.
- bongodongobob 2y agoI'm using hallucination to mean "not exactly the thing", not outright lying. So maybe the "truth" is "My socks are wet." A hallucination could be "My socks are damp."
- littlestymaar 2y agoAll lies aren't useless, some can be insightful even when blatantly wrong in themselves (for instance: taken literally every scientific model is a lie). I can definitely see how an LLM hallucinating can helps fostering creativity (the same way psychedelics can), even if all they say is bullshit.
- malux85 2y ago
- barathr 2y agoThis strikes me as similar to Cargo Cult Science. https://calteches.library.caltech.edu/51/2/CargoCult.htm https://calteches.library.caltech.edu/51/2/CargoCult.htm https://metarationality.com/upgrade-your-cargo-cult https://metarationality.com/upgrade-your-cargo-cult
- KhoomeiK 2y agoA group of PhD students at Stanford recently wanted to take AI/ML research ideas generated by LLMs like this and have teams of engineers execute on them at a hackathon. We were getting things prepared at AGI House SF to host the hackathon with them when we learned that the study did not pass ethical review. I think automating science is an important research direction nonetheless.
- srcreigh 2y agoThat’s pretty wild. What was the reason behind failing ethics review?
- robbomacrae 2y agoI'm generally a proponent of AI and LLM but to me the decision was the right one. You are tasking people with implementing an idea generated by an algorithmic model with (I'm guessing) zero oversight that might have very little training that teaches it the importance of coming up with ideas worth implementing. Some may be more useful than others so it won't be fair from an accomplishment or motivation point of view. Imagine you've already invested time going to this event and want to win the prize/credit but to do so you have to implement a plugin that makes webpages grayscale because of a random idea generator. Maybe some people would find that interesting but others would see it as wasting their time.
- golol 2y agoAs long as all participants are well-informed then there is absolutely no ethical issue...
- rsfern 2y agoHow do you make sure the participants are well informed? What if an idea suggested by a model turns out to be dangerous to implement, but nobody at the hackathon has quite the relevant experience to notice?
- UncleOxidant 2y agoThe ideas aren't the hard part.
- tokai 2y agoThis. Any researcher should, over a lunch, be able to generate more idea than can be tackled in a life time.
- falcor84 2y agoThe fact that a human expert can also do it doesn't mean the AI isn't valuable. Even if you just consider the monetary aspect, those few API calls would definitely be cheaper than buying the researcher lunch. But the big benefit is being able to generate those ideas immediately and autonomously every time there's new data.
- passwordoops 2y agoI think what they are saying is that idea generation is not a pain point and not really worth solving. Taking ideas and making them happen... that's the hard part where an artificial agent could come in much more handy
- kordlessagain 2y agoThe number of the ideas has nothing to do with the quality of the ideas. Some ideas a gold, many aren’t.
- fpgamlirfanboy 2y agoTell that to PhD advisor that took credit for all my work because they were his ideas (at least so he claimed).
- passwordoops 2y agoUnfortunately the good ones who do not steal credit are few and far between. Current incentives select for this behaviour. Not just in academia, but about everywhere. Go to any meeting and state the obvious fact that "any idiot can have an idea. Making it happen is the tough part" then watch how the decision makers react
- SubiculumCode 2y agoIn some fields of research, the amount of literature out there is stupendous, and with little hope of a human reading, much less understanding the whole literature. Its becoming a major problem in some fields, and I think, in some ways, approaches that can combine knowledge algorithmically are needed, perhaps llms.
- wizzwizz4 2y agoTraditionally, that's what meta-analyses and published reviews of the literature have been for.
- SubiculumCode 2y agoeven so.
- deegles 2y agoIt would be fun to pair this with an automated lab that could run experiments and feed the results into generating the next set of ideas.