8 ms·

John Jumper: AI is revolutionizing scientific discovery [video]

124 points by sandslash 1y ago

malux85 1y ago
First jump that computers gave us : speed. With excess of speed came the ability to brute force many problems. Next jump given by AI (not LLMs specifically, I mean “machine learned systems” in general) is navigation. Even with large amounts of speed some problems are still impractically large, we are using AI to better explore that space, by navigating it smarter, rather than just speeding through it combinatorially.
- whatever1 1y ago
  No evidence so far that "AI" has improved our general optimization capabilities. At all. Still at the top of the benchmarks of integer optimization by huge margin are the traditional usual suspects. Same in constraint programming and SAT.
  - lomase 1y ago
    If you only know how to use a hammer, everything looks like a nail.
    - whatever1 1y ago
      Combinatorics will always be a (tough) nail regardless of what a random HN commentator thinks. Bring any tool you wish, but the problem is very well defined and very real.
  - hodgehog11 1y ago
    Here is some evidence for you then: https://arxiv.org/abs/2411.00566 https://arxiv.org/abs/2411.00566 Not published just yet are experiments for finding solutions to mathematical problems traditionally found with SAT solvers, at much larger scale than was previously possible.
    - whatever1 1y ago
      This is just meta heuristic relying on local search :facepalm: You could call it artificial ant colony optimization. People come up with such ideas all the time. Sorry, but nothing groundbreaking here.
      - hodgehog11 1y ago
        Um, okay? Isn't that how most optimisation involving AI is supposed to go? Perhaps some much needed context. Mathematicians are not stupid; we are very much aware of all the existing forms of genetic optimisation algorithms, cross entropy method, etc. Nothing works on these problems, or at least not well at scale. As I said, state of the art for many of these was SAT-related. The problem is that the heuristic used for exploring new solutions always required very careful consideration as the naive ones rarely worked well. Here, the transformer is proving effective at searching for good heuristics, far more so than any other existing technique. In this sense, it is achieving far, far. far better performance in optimisation than previous approaches. That is a breakthrough, at least for us mathematicians. If this doesn't constitute improvements in optimisation, I don't know what does. Saying it's "just meta heuristic relying on local search" is akin to saying these tasks are "just optimisation". If it's so procedural, why weren't we making ground on these things before? Also, by the way, a :facepalm: is not exactly the pinnacle of academic rebuttal, no matter how wrong I could have been.
        whatever1 1y ago
        Apologies I didn’t mean to be cocky / dismissive. It’s just that the paper cited is no different than any other paper in the meta-heuristic community. Some idea for guiding the local search. Some limited sample results. No promises on bounds or generalizability of the method. If this is ground breaking, then every legitimate meta heuristic paper in the past 50 years was also ground breaking. I will change my mind if I see a wide set of benchmark results where it consistently beats or is even head-to-head with the SoTA. Then we would know that we have a game changer.
        1y ago
  - malux85 1y ago
    > No evidence so far that "AI" has improved our general optimization capabilities. At all. Uh, ok, I didn't claim that. At All. Deep Learning machine learned features have definitely helped us (meaning my company) over hand engineered features, allowing us to navigate our problem space significantly faster
bgwalter 1y ago
[flagged]
- layoric 1y ago
  Thank you for this context, should be the title IMO..
  - signatoremo 1y ago
    In the same vein, any link to Gary Marcus’s blog posts should be labeled “AI hater”, don’t you think?
    - bgwalter 1y ago
      People do that here, and mostly they are not downvoted or flagged (though sometimes they are). But the important part is that the funding for "AI" skeptics is practically zero, whereas the funding for "AI" boosters is basically unlimited.
  - layoric 1y ago
    For clarity, all the parent said was John Jumper is the current director at Google DeepMind. Which I think is useful context, not sure why flagged..
- lomase 1y ago
  [flagged]
  - lomase 1y ago
    [flagged]
    - dang 1y ago
      HN doesn't have a Youtube channel, and please don't cross into personal attack. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
some_guy_nobel 1y ago
NVIDIA published the Illustrated Evo2 a few days ago, walking through the architecture of their genetics foundation model: https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/ https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/ It's nice to see more and more labs using ai for drug discovery, something truly net positive for society.
- hodgehog11 1y ago
  As someone who works in the field, it really doesn't feel like more money (proportionally speaking) is going to this. A little bit is done here and there for PR. The number that are working on net positive applications for AI is still shockingly low compared to everything else.
the__alchemist 1y ago
I am reposting something along the lines of a flagged and dead comment: This would be lend more credibility to the premise AI is revolutionizing scientific discovery if it came from someone who's Nobel (or work in general) were in a non-AI-centered domain. This is not a critique of his speech or points, but I think the lead implied by the (especially Youtube) title would hit harder if it came from someone whose work wasn't AI-centered. Jumper's work is the poster child of AI success in science; this isn't about a new domain being revolutionized by it. I will throw out an idea I've been thinking about recently about a far less ambitious idea, but related: Amber (MD package) provides Force Field names and partial charges for a number of small organic molecules in their GeoStd set. I believe these come from its Antechamber program. Would it be possible to infer useful FF name and Partial charge for arbitrary organic molecules using AI instead, trained on the GeoStd set data?
- ants_everywhere 1y ago
  > This would be lend more credibility to the premise AI is revolutionizing scientific discovery if it came from someone who's Nobel (or work in general) were in a non-AI-centered domain. No it wouldn't. I've seen anti-AI people try to make this sort of argument repeatedly and it doesn't make any sense. It's an attempt to smuggle in an ad-hominem. It's relying on the fact that people who hate AI also hate people who work in AI.
  - bgwalter 1y ago
    It is not at all an ad hominem. Disclosure of interests and conflicts for interest were assumed to be declared in the open even a decade ago. Carter sold his peanut farm to avoid conflicts of interest, Trump launched a coin pump & dump on his first day. If a Nobel Prize winner works for a corporation, that should be disclosed (the original title contained "Nobel Prize Laureate" instead of "DeepMind Director"). But I suppose that in the current age where everyone just wants to get rich these courtesies no longer matter.
    - ants_everywhere 1y ago
jgalt212 1y ago
I see this sort of work as a natural extension of Combinatorial Chemistry or bootstrapping and Monte Carlo methods in stats. https://en.wikipedia.org/wiki/Combinatorial_chemistry https://en.wikipedia.org/wiki/Combinatorial_chemistry
epolanski 1y ago
I'll share something as a former solar researcher. Scientific progress is heavily influenced by how many bodies you can throw at a problem. The more experiments you can run, with more variety and angles the more data you can get, the higher the likelihood of a breakthrough. Several huge scientist are famous not because they are geniuses, but because they are great fundraisers and can have 20/30/50 bodies to throw at problems every year. This is true in virtually any experimental field. If LLMs can be de facto another body then scientific progress is going to sky rocket. Robots also tend to be more precise than humans and could possibly lead to better replication. But given that LLMs cannot interact with the real world I don't see that happening anytime soon.
- bonoboTP 1y ago
  > But given that LLMs cannot interact with the real world What type of interaction do you envision? Could a non-domain-expert, but somewhat trained person provide a bridge? If the LLM comes up with the big ideas and tells a human technical assistant to execute (put the vial here, run the 3D printer with this file, put the object there, drive in a screw), would that help? But dexterous robots are getting more and more advanced, see CoRL demos right now.
  - marbro 1y ago
    Can these robots move a chess piece from one square to another?
    - ragequittah 1y ago
      Probably: https://www.senserobotchess.com/ https://www.senserobotchess.com/
  - kjkjadksj 1y ago
    Someone needs to evaluate the big ideas spat out by the llm is the big issue. Lab work can already be automated. And bs holders are even cheaper than an automated machine.
  - 1y ago
NedF 1y ago
Awful title, great video. Three points jumped out 1) "really when you look at these machine learning breakthroughs they're probably fewer people than you imagine" In a world of idiots, few people can do great things. 2) External benchmarks forced people upstream to improve We need more of these. 3) "the third of these ingredients research was worth a hundredfold of the first of these ingredients data." Available data is 0 for most things.
- hodgehog11 1y ago
  > We need more of these. > Available data is 0 for most things. I would argue that we need an effective alternative to benchmarks entirely given how hard they are to obtain in scientific disciplines. Classical statistics has gone very far by getting a lot out of limited datasets, and train-test splits are absolutely unnecessary there.
  - bobmarleybiceps 1y ago
    I kind of dislike the benchmarkification of AI for science stuff tbh. I've encountered a LOT of issues with benchmark datasets that just aren't good... In a lot of cases they are fine and necessary, but IMO, the standard for legit "success" in a lot of ML for science applications should basically be "can this model be used to make real scientific or engineering insights, that would have been very difficult and/or impossible without the proposed idea." Even if this is a super high bar, I think more papers in ML for science should strive to be truly interdisciplinary and include an actual science advancement... Not just "we modify X and get some improvement on a benchmark dataset that may or may not be representative of the problems scientists could actually encounter." The ultimate goal of "ml for science" is science, not really to improve ML methods imo
- esjeon 1y ago
  Yeah the title was the worst and the most disgusting part, and I'm saying this in totally positive sense. As soon as you catch that "DeepMind" name there, you know it's really about science and a crafted model, not a wannabe general intelligence.
hodgehog11 1y ago
Something else to add is mathematical discovery. There is a team that is very close to solving the Navier-Stokes Millenium Prize problem: https://deepmind.google/discover/blog/discovering-new-solutions-to-century-old-problems-in-fluid-dynamics/ https://deepmind.google/discover/blog/discovering-new-soluti... The cynists will comment that I've just been sucked in by the PR. However, I know this team and have been using these techniques for other problems. I know they are so close to a computationally-assisted proof of counterexample that it is virtually inevitable at this point. If they don't do it, I'm pretty sure I could take a handful of people and a few years and do it myself. Mostly a lot of interval arithmetic with a final application of Schauder that remains; tedious and time-consuming, but not overly challenging compared to the parts already done.
- PontifexCipher 1y ago
  This is not just PR and is very interesting. However, in my view, (and from a quick read of the paper) this is actually a very classical method in applied math work: - Build a complex intractable mathematical model (here, Navier-Stokes) - Approximate it with a function approximator (here, a Physics Informed Neural Network) - Use the some property of function approximator to search for more solutions to the original model (here, using Gauss-Newton) In a sense, this is actually just the process of model-based science anyway: use a model for the physical world and exploit the mathematics of the model for real-world effects. This is very very good work, but this heritage goes back to polynomial approximation even from Taylor series, and has been the foundation of engineering for literal centuries. Throughout history, the approximator keeps getting better and better and hungrier and hungrier for data (Taylor series, Chebyshev + other orthogonal bases for polynomials, neural networks, RNNs, LSTMs, PINNs, <the future>). You didn't say anything to the contrary, and neither did the original video, but it's very different than what some other people are talking about in this thread ("run an LLM in a loop to do science the way a person does it"). Maybe I'm just ranting at the overloading of the term AI to mean "anything on a GPU".
  - hodgehog11 1y ago
    This is absolutely true, but it still makes use of the advantages and biases of neural networks in a clever way. It has to, because computationally-assisted proofs for PDEs with singularities is incredibly difficult. To me, this is not too similar from using them as heuristics to find counterexamples, or other approaches where the implicit biases pay off. I think we do ourselves a disservice to say that "LLMs replacing people" = "applications of AI in science". I also wouldn't say this is entirely "classical". Old, yes, but still unfamiliar and controversial to a surprising number of people. But I get your point :-).
Inviz 1y ago
I have a mildly psychotic friend who think that he uncovered the secrets to everything with AI. Quantum theory and Jungian archetypes, together with 4 dimensions - great mix
- cantor_S_drug 1y ago
  Let's say the "Secrets of the Universe" broadly consists of Graph of 100 "abstract" interconnected concepts. The concepts have to be abstract because it is describing everything. It has to be limited in number because we cannot be endlessly chasing the definitions till we reach the levels of atoms. Is it possible to get glimpse of that Graph just toying with abstract ideas. The exact nodes / concepts used in the graph maybe different (depending on field) but the structure will be isomorphic. It has to be discoverable in any field since we started with the assumption that the Graph is "Secret of the Universe" so it should apply to any subset as well and should be discoverable from that subset. This is like analytic functions where knowing its derivatives in a small enough interval can lead us to the exact function.
- Hilift 1y ago
  AI really is good at finding new viruses, due to simple DNA sequences look like noise to humans. But then you may have created a different problem.
tim333 1y ago
On the same topic of AI helping scientific discovery there was this tweet yesterday https://x.com/DeryaTR_/status/1972115494787338484 https://x.com/DeryaTR_/status/1972115494787338484 >...noticed an email from one of my PhD students sent more than eight years ago, outlining a highly complex immune cell experiment that would run for several weeks and asking me to make corrections >...Incredibly, GPT-5 Pro would have been as good as, if not better than, me at making these corrections, interpretations, analyses, and follow-up experiment suggestions! The experiment would also have yielded better results thanks to more precise planning... Maybe the era of AI speeding things is upon us. Maybe not so long till AIs are helping make better AIs?
- _heimdall 1y ago
  If and when these tools become self-improving we have absolutely no idea what comes next. Maybe its utopia, maybe its akin to an Eliezer Yudkowsky prediction, who knows. Regardless of the specific outcome, its a huge gamble with effectively unlimited risk.
  - cwmma 1y ago
    The architecture of the current models where learning is a separate and very expensive process make runaway self improvement seem like something that will require a bunch of breakthroughs in order to happen.
    - _heimdall 1y ago
      That seems reasonable, but I'm not sure if we really know yet. If current models are given direction and control to change how the next model is trained or architected it seems plausible that they could stumble into such a breakthrough. The current LLM approach makes huge assumptions, including that training only on text prediction is enough to simulate true intelligence. That may or may not be a valid assumption, but it could be enough for the LLM to make one seemingly small change that ends up running away from us faster than we would realize.
      - 1y ago

124 points by sandslash 1y ago

First jump that computers gave us : speed. With excess of speed came the ability to brute force many problems. Next jump given by AI (not LLMs specifically, I mean “machine learned systems” in general) is navigation. Even with large amounts of speed some problems are still impractically large, we are using AI to better explore that space, by navigating it smarter, rather than just speeding through it combinatorially.

whatever1 1y ago
No evidence so far that "AI" has improved our general optimization capabilities. At all. Still at the top of the benchmarks of integer optimization by huge margin are the traditional usual suspects. Same in constraint programming and SAT.
- lomase 1y ago
  If you only know how to use a hammer, everything looks like a nail.
  - whatever1 1y ago
    Combinatorics will always be a (tough) nail regardless of what a random HN commentator thinks. Bring any tool you wish, but the problem is very well defined and very real.
- hodgehog11 1y ago
  Here is some evidence for you then: https://arxiv.org/abs/2411.00566 https://arxiv.org/abs/2411.00566 Not published just yet are experiments for finding solutions to mathematical problems traditionally found with SAT solvers, at much larger scale than was previously possible.
  - whatever1 1y ago
    This is just meta heuristic relying on local search :facepalm: You could call it artificial ant colony optimization. People come up with such ideas all the time. Sorry, but nothing groundbreaking here.
    - hodgehog11 1y ago
      Um, okay? Isn't that how most optimisation involving AI is supposed to go? Perhaps some much needed context. Mathematicians are not stupid; we are very much aware of all the existing forms of genetic optimisation algorithms, cross entropy method, etc. Nothing works on these problems, or at least not well at scale. As I said, state of the art for many of these was SAT-related. The problem is that the heuristic used for exploring new solutions always required very careful consideration as the naive ones rarely worked well. Here, the transformer is proving effective at searching for good heuristics, far more so than any other existing technique. In this sense, it is achieving far, far. far better performance in optimisation than previous approaches. That is a breakthrough, at least for us mathematicians. If this doesn't constitute improvements in optimisation, I don't know what does. Saying it's "just meta heuristic relying on local search" is akin to saying these tasks are "just optimisation". If it's so procedural, why weren't we making ground on these things before? Also, by the way, a :facepalm: is not exactly the pinnacle of academic rebuttal, no matter how wrong I could have been.
      - whatever1 1y ago
        Apologies I didn’t mean to be cocky / dismissive. It’s just that the paper cited is no different than any other paper in the meta-heuristic community. Some idea for guiding the local search. Some limited sample results. No promises on bounds or generalizability of the method. If this is ground breaking, then every legitimate meta heuristic paper in the past 50 years was also ground breaking. I will change my mind if I see a wide set of benchmark results where it consistently beats or is even head-to-head with the SoTA. Then we would know that we have a game changer.
        1y ago
- malux85 1y ago
  > No evidence so far that "AI" has improved our general optimization capabilities. At all. Uh, ok, I didn't claim that. At All. Deep Learning machine learned features have definitely helped us (meaning my company) over hand engineered features, allowing us to navigate our problem space significantly faster

bgwalter 1y ago

[flagged]

layoric 1y ago
Thank you for this context, should be the title IMO..
- signatoremo 1y ago
  In the same vein, any link to Gary Marcus’s blog posts should be labeled “AI hater”, don’t you think?
  - bgwalter 1y ago
    People do that here, and mostly they are not downvoted or flagged (though sometimes they are). But the important part is that the funding for "AI" skeptics is practically zero, whereas the funding for "AI" boosters is basically unlimited.
- layoric 1y ago
  For clarity, all the parent said was John Jumper is the current director at Google DeepMind. Which I think is useful context, not sure why flagged..
lomase 1y ago
[flagged]
- lomase 1y ago
  [flagged]
  - dang 1y ago
    HN doesn't have a Youtube channel, and please don't cross into personal attack. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

some_guy_nobel 1y ago

NVIDIA published the Illustrated Evo2 a few days ago, walking through the architecture of their genetics foundation model: https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/ https://research.nvidia.com/labs/dbr/blog/illustrated-evo2/ It's nice to see more and more labs using ai for drug discovery, something truly net positive for society.

hodgehog11 1y ago
As someone who works in the field, it really doesn't feel like more money (proportionally speaking) is going to this. A little bit is done here and there for PR. The number that are working on net positive applications for AI is still shockingly low compared to everything else.

the__alchemist 1y ago

I am reposting something along the lines of a flagged and dead comment: This would be lend more credibility to the premise AI is revolutionizing scientific discovery if it came from someone who's Nobel (or work in general) were in a non-AI-centered domain. This is not a critique of his speech or points, but I think the lead implied by the (especially Youtube) title would hit harder if it came from someone whose work wasn't AI-centered. Jumper's work is the poster child of AI success in science; this isn't about a new domain being revolutionized by it. I will throw out an idea I've been thinking about recently about a far less ambitious idea, but related: Amber (MD package) provides Force Field names and partial charges for a number of small organic molecules in their GeoStd set. I believe these come from its Antechamber program. Would it be possible to infer useful FF name and Partial charge for arbitrary organic molecules using AI instead, trained on the GeoStd set data?

ants_everywhere 1y ago
> This would be lend more credibility to the premise AI is revolutionizing scientific discovery if it came from someone who's Nobel (or work in general) were in a non-AI-centered domain. No it wouldn't. I've seen anti-AI people try to make this sort of argument repeatedly and it doesn't make any sense. It's an attempt to smuggle in an ad-hominem. It's relying on the fact that people who hate AI also hate people who work in AI.
- bgwalter 1y ago
  It is not at all an ad hominem. Disclosure of interests and conflicts for interest were assumed to be declared in the open even a decade ago. Carter sold his peanut farm to avoid conflicts of interest, Trump launched a coin pump & dump on his first day. If a Nobel Prize winner works for a corporation, that should be disclosed (the original title contained "Nobel Prize Laureate" instead of "DeepMind Director"). But I suppose that in the current age where everyone just wants to get rich these courtesies no longer matter.
  - ants_everywhere 1y ago

jgalt212 1y ago

I see this sort of work as a natural extension of Combinatorial Chemistry or bootstrapping and Monte Carlo methods in stats. https://en.wikipedia.org/wiki/Combinatorial_chemistry https://en.wikipedia.org/wiki/Combinatorial_chemistry

epolanski 1y ago

I'll share something as a former solar researcher. Scientific progress is heavily influenced by how many bodies you can throw at a problem. The more experiments you can run, with more variety and angles the more data you can get, the higher the likelihood of a breakthrough. Several huge scientist are famous not because they are geniuses, but because they are great fundraisers and can have 20/30/50 bodies to throw at problems every year. This is true in virtually any experimental field. If LLMs can be de facto another body then scientific progress is going to sky rocket. Robots also tend to be more precise than humans and could possibly lead to better replication. But given that LLMs cannot interact with the real world I don't see that happening anytime soon.

bonoboTP 1y ago
> But given that LLMs cannot interact with the real world What type of interaction do you envision? Could a non-domain-expert, but somewhat trained person provide a bridge? If the LLM comes up with the big ideas and tells a human technical assistant to execute (put the vial here, run the 3D printer with this file, put the object there, drive in a screw), would that help? But dexterous robots are getting more and more advanced, see CoRL demos right now.
- marbro 1y ago
  Can these robots move a chess piece from one square to another?
  - ragequittah 1y ago
    Probably: https://www.senserobotchess.com/ https://www.senserobotchess.com/
- kjkjadksj 1y ago
  Someone needs to evaluate the big ideas spat out by the llm is the big issue. Lab work can already be automated. And bs holders are even cheaper than an automated machine.
- 1y ago

NedF 1y ago

Awful title, great video. Three points jumped out 1) "really when you look at these machine learning breakthroughs they're probably fewer people than you imagine" In a world of idiots, few people can do great things. 2) External benchmarks forced people upstream to improve We need more of these. 3) "the third of these ingredients research was worth a hundredfold of the first of these ingredients data." Available data is 0 for most things.

hodgehog11 1y ago
> We need more of these. > Available data is 0 for most things. I would argue that we need an effective alternative to benchmarks entirely given how hard they are to obtain in scientific disciplines. Classical statistics has gone very far by getting a lot out of limited datasets, and train-test splits are absolutely unnecessary there.
- bobmarleybiceps 1y ago
  I kind of dislike the benchmarkification of AI for science stuff tbh. I've encountered a LOT of issues with benchmark datasets that just aren't good... In a lot of cases they are fine and necessary, but IMO, the standard for legit "success" in a lot of ML for science applications should basically be "can this model be used to make real scientific or engineering insights, that would have been very difficult and/or impossible without the proposed idea." Even if this is a super high bar, I think more papers in ML for science should strive to be truly interdisciplinary and include an actual science advancement... Not just "we modify X and get some improvement on a benchmark dataset that may or may not be representative of the problems scientists could actually encounter." The ultimate goal of "ml for science" is science, not really to improve ML methods imo
esjeon 1y ago
Yeah the title was the worst and the most disgusting part, and I'm saying this in totally positive sense. As soon as you catch that "DeepMind" name there, you know it's really about science and a crafted model, not a wannabe general intelligence.

hodgehog11 1y ago

Something else to add is mathematical discovery. There is a team that is very close to solving the Navier-Stokes Millenium Prize problem: https://deepmind.google/discover/blog/discovering-new-solutions-to-century-old-problems-in-fluid-dynamics/ https://deepmind.google/discover/blog/discovering-new-soluti... The cynists will comment that I've just been sucked in by the PR. However, I know this team and have been using these techniques for other problems. I know they are so close to a computationally-assisted proof of counterexample that it is virtually inevitable at this point. If they don't do it, I'm pretty sure I could take a handful of people and a few years and do it myself. Mostly a lot of interval arithmetic with a final application of Schauder that remains; tedious and time-consuming, but not overly challenging compared to the parts already done.

PontifexCipher 1y ago
This is not just PR and is very interesting. However, in my view, (and from a quick read of the paper) this is actually a very classical method in applied math work: - Build a complex intractable mathematical model (here, Navier-Stokes) - Approximate it with a function approximator (here, a Physics Informed Neural Network) - Use the some property of function approximator to search for more solutions to the original model (here, using Gauss-Newton) In a sense, this is actually just the process of model-based science anyway: use a model for the physical world and exploit the mathematics of the model for real-world effects. This is very very good work, but this heritage goes back to polynomial approximation even from Taylor series, and has been the foundation of engineering for literal centuries. Throughout history, the approximator keeps getting better and better and hungrier and hungrier for data (Taylor series, Chebyshev + other orthogonal bases for polynomials, neural networks, RNNs, LSTMs, PINNs, <the future>). You didn't say anything to the contrary, and neither did the original video, but it's very different than what some other people are talking about in this thread ("run an LLM in a loop to do science the way a person does it"). Maybe I'm just ranting at the overloading of the term AI to mean "anything on a GPU".
- hodgehog11 1y ago
  This is absolutely true, but it still makes use of the advantages and biases of neural networks in a clever way. It has to, because computationally-assisted proofs for PDEs with singularities is incredibly difficult. To me, this is not too similar from using them as heuristics to find counterexamples, or other approaches where the implicit biases pay off. I think we do ourselves a disservice to say that "LLMs replacing people" = "applications of AI in science". I also wouldn't say this is entirely "classical". Old, yes, but still unfamiliar and controversial to a surprising number of people. But I get your point :-).

Inviz 1y ago

I have a mildly psychotic friend who think that he uncovered the secrets to everything with AI. Quantum theory and Jungian archetypes, together with 4 dimensions - great mix

cantor_S_drug 1y ago
Let's say the "Secrets of the Universe" broadly consists of Graph of 100 "abstract" interconnected concepts. The concepts have to be abstract because it is describing everything. It has to be limited in number because we cannot be endlessly chasing the definitions till we reach the levels of atoms. Is it possible to get glimpse of that Graph just toying with abstract ideas. The exact nodes / concepts used in the graph maybe different (depending on field) but the structure will be isomorphic. It has to be discoverable in any field since we started with the assumption that the Graph is "Secret of the Universe" so it should apply to any subset as well and should be discoverable from that subset. This is like analytic functions where knowing its derivatives in a small enough interval can lead us to the exact function.
Hilift 1y ago
AI really is good at finding new viruses, due to simple DNA sequences look like noise to humans. But then you may have created a different problem.

tim333 1y ago

On the same topic of AI helping scientific discovery there was this tweet yesterday https://x.com/DeryaTR_/status/1972115494787338484 https://x.com/DeryaTR_/status/1972115494787338484 >...noticed an email from one of my PhD students sent more than eight years ago, outlining a highly complex immune cell experiment that would run for several weeks and asking me to make corrections >...Incredibly, GPT-5 Pro would have been as good as, if not better than, me at making these corrections, interpretations, analyses, and follow-up experiment suggestions! The experiment would also have yielded better results thanks to more precise planning... Maybe the era of AI speeding things is upon us. Maybe not so long till AIs are helping make better AIs?

_heimdall 1y ago
If and when these tools become self-improving we have absolutely no idea what comes next. Maybe its utopia, maybe its akin to an Eliezer Yudkowsky prediction, who knows. Regardless of the specific outcome, its a huge gamble with effectively unlimited risk.
- cwmma 1y ago
  The architecture of the current models where learning is a separate and very expensive process make runaway self improvement seem like something that will require a bunch of breakthroughs in order to happen.
  - _heimdall 1y ago
    That seems reasonable, but I'm not sure if we really know yet. If current models are given direction and control to change how the next model is trained or architected it seems plausible that they could stumble into such a breakthrough. The current LLM approach makes huge assumptions, including that training only on text prediction is enough to simulate true intelligence. That may or may not be a valid assumption, but it could be enough for the LLM to make one seemingly small change that ends up running away from us faster than we would realize.
    - 1y ago