5 ms·

Deep learning and machine learning don’t work. Quantitative math will always prevail, as it always has. Unfortunately, mathematical research isn’t there yet. We

Deep learning and machine learning don’t work. Quantitative math will always prevail, as it always has. Unfortunately, mathematical research isn’t there yet. We don’t have models for vision, audition and linguistics. Neuroscience and psychology are in their infancy, a good analogy would compare these fields to where physics was pre-Newton, Galileo era of understanding. I suspect that in the decades to come, these fields will influence mathematics the same way physics influenced calculus. Physics historically had a huge influence on math, in the coming century it will be neuroscience and psychology, in linking brains to behavior, and the quantitative laws that allow brains to give rise to minds.

intuitionist 7y ago
> Deep learning and machine learning don’t work. Quantitative math will always prevail, as it always has. What do you think machine learning is, if not “quantitative math”? Deep learning is just linear algebra and calculus, and things like random forests are even simpler mathematically.
- acdc4life 7y ago
  Machine learning is glorified curve fitting. DL isn't even mathematically sound, back propagation has no proof of convergence. Quantitative math is about extracting natural laws, and mapping them to mathematical structures. You could use DL to predict planetary motion, and get pretty good at it. But this isn't a quantitative understanding of the world. You didn't learn anything. Physics in contrast has the laws of motion and gravitation. You can directly model arbitrary planets. Moreover, you can model arbitrary rigid bodies, from cars to space shuttles. Your ML, DL random forrest etc. all use math, sure. But so did the Keplarian models of motion. You aren't qualitatively deducing math that governs the world, but forcing an arbitrarily chosen mathematical structure to your data.
  - deleted 7y ago
    [deleted]
  - intuitionist 7y ago
    If we’re throwing out anything that doesn’t have a proof of convergence as “not mathematically sound,” you can kiss fluid mechanics goodbye, as well as lots of other subfields of physics that rely on partial differential equations.
QuackingJimbo 7y ago
I agree that DL/ML is destined to fail in domains like this but can you expand on this reasoning? What exactly do you mean by "quantitative math" (I haven't heard this phrase used in this way before)? And what were the equivalents of DL/ML for physics before calculus?
- acdc4life 7y ago
  Quantitative math might be the wrong word, maybe applied math? In quantitative finance, you make quantitative models about the world, and build math that realize those assumptions and understanding. A simple example: options are a great financial trading instrument that you can model mathematically, the simplest being the Black-Scholes model. You can imply things like volatility of the stock price based on the price of the option to get a better understanding of what a risk neutral market is thinking, and compare that to the actual market distribution. >And what were the equivalents of DL/ML for physics before calculus? This is a good question. Before Newtonian calculus, and the laws of gravitation, people were building very complicated conic models (ie. eclipses, parabolas etc.) to get better and better prediction of planetary motion. A lot of parametric math came out of this, with many sophisticated models getting better and better, giving these astronomers an illusion of progress. However, Newton's insight was that motion is connected to mass, and this insight was the basis of how to derive the laws of motion, which gave us the laws of gravitation (F = (Gm1m2/r^2)). This insight eliminated the previous Keplarian models of motion, because you were now able to predict the motion of arbitrary rigid bodies using very simple math (we teach this in highschool). Ofcourse, Newtonian motion has its limitations that's why we have quantum physics and Einstien's relativity theory. But for practical technological applications, Newtonian physics on its own gets you incredibly far. Where is ML/DL? It would be akin to Keplarian elliptical motion. More realistically however, it's closer to aether theory of light, and will go the way of GOFAI. This stuff isn't grounded in modelling any scientific observation. Moreover, they are mathematically useless. Back propagation doesn't converge, and why should you fit your data to an arbitrary mathematical structure? In practise, DL/ML doesn't work at all, you will be much more successful by modelling your problem mathematically. For example, consider an automobile manufacturer, which has all kinds of moving parts in their planes. They typically model each part mathematically (ie. gear x under goes exponential time decay), and imply their parameters using rigours test data. Then you use some sort of an empirical statistical model to predict the failure. I've seen deep learning companies come and fall flat on their face trying to beat the accuracy of these deterministic systems. Those guys needed a lot of data, and GPUs. I'm not even criticizing the fact that DL is a black box. It's worse, it's inferior to everything out there on every metric imaginable. These mathematical models in contrast have been in production for decades, with yearly updates, and they run in real time with little historical data, they are fully understandable and they beat every method we know of. This isn't the first time multi layer perceptrons gained hype. They didn't work in the 80s, or 90s or the 2000s, they don't work now. The math behind DL is the same that we had in the 80s, they just called it multi layer perceptron. None of the ideas in modern ML/DL are new, all these ideas like reinforcement learning, GANs etc.
bduerst 7y ago
>Deep learning and machine learning don’t work. Quantitative math will always prevail, I have a neural net onboard my phone which automatically detects songs offline and tells me what they are. Is that semantically 'quantitative math' and not machine learning?
- acdc4life 7y ago
  Quantitative math, or applied math isn't based on fitting data to an arbitrary mathematical structure. It's looking at real life, and deriving the mathematical laws that govern what you see. You could have a neural net predict planetary motion. However, it doesn't know jack shit about physics. >I have a neural net onboard my phone which automatically detects songs offline and tells me what they are. MP3 uses something called psycho acoustics, which is a quantitative model on human perception, which is used to eliminate frequencies that can't be heard based on this model. Your neural network doesn't tell you what features make songs distinct, it's not a quantitative model at all, but a black box heuristic on what the important features are superficially. If actual mathematicians worked on this problem, I guarantee you they'd do a better job, and their models would work on a commadore64, with real time training. Moreover it would tell you things like who is singing, if it's a live performance, which concert it was.
  - sonnyblarney 7y ago
    " If actual mathematicians worked on this problem, I guarantee you they'd do a better job" No, this is wrong. Some of the most brilliant people in the world have been working on image recognition, voice recognition etc. and AI is crushing all of their work. "Your neural network doesn't tell you what features make songs distinct, it's not a quantitative model at all" - it doesn't matter at all if our objective is detecting the song. Neither does the mp3 compression algorithm.
    - acdc4life 7y ago
      >Some of the most brilliant people in the world have been working on image recognition, voice recognition etc. and AI is crushing all of their work. This is very true. I take my stronger statements back, MAINSTREAM mathematicians attempting this problem are all wrong, and have been wrong for 50 years. But you do need the right theory, and the right math that realizes this theory. "AI" is superficially beating the work in computer vision. Computer vision is complete bogus. The gabor filters, fourier transfroms etc. are all wrong conceptually. The known methods do abysmally on basic tasks like object recognition, texture segmentation etc. But they keep trying it. I would take this one step further: computer vision, audio and NLP researchers have been stuck in a rut for the past 50 years. DL is beating THEIR math, but this is because of data and computation speed, not because of any insights. But DL is also wrong, and giving you an illusion of progress. Both of these things are doomed to go the way of GOFAI. I can go into great detail and carefully explain why MAINSTREAM contemporary ideas in math for vision, audition and language are completely wrong, and have been wrong for 50 years. What is the right model? Like I mentioned before, the right ideas are emerging, neural networks will dominate, just not DL.
sonnyblarney 7y ago
If you mean for medical diagnosis, maybe, but you do realize that NN/AI is totally state of the art for many tasks? NLP, image recognition etc. ? AI is in a hype bubble right now surely, but it's a 'very real' thing that's going to infiltrate a lot of areas.
- acdc4life 7y ago
  >you do realize that NN/AI is totally state of the art for many tasks? Being state of the art doesn't imply that these things will solve these problems. In ML terms, how do you know that NN/AI isn't a local maxima that we need to jump out of? All NLP systems are joke. Sure replace Watson with DL, might perform better on Jeopardy. But in real conversations? Forget it. I wouldn't bet on these things. NN will win, but not the back propagation, ReLu, sigmoid or whatever pseudo science that is the current buzzword. There is 50 years worth of understanding in actual neuroscience and cognitive modelling that no one has paid attention to, and new design principles are emerging that will influence mathematics.
  - chillacy 7y ago
    Because DL isn't great at NLP (and may never solve "real conversations" aka AGI), it's worthless? It's the best performing tool we have for NLP, image recognition, etc. Is it a local maxima? Probably. But it's out there solving real problems nonetheless. We'll capture all the gains we can and then move on after.
  - sonnyblarney 7y ago
    "Being state of the art doesn't imply that these things will solve these problems. " I suggest you are misinformed about the state of AI. AI is currently ahead of all other approaches in many fields. It's lead to quite a number of practical advances and breakthroughs. The 'best examples' are those that I described, but there are many more. Your comments indicate I think some ignorance on the issue - I think I see the point you are trying to make but I also submit that you're not aware of what AI is doing today. 'Self driving cars' would be impossible without AI today, for example. The vision systems depend on AI it's a breakthrough without which we simply wouldn't have the tech.
opsiprogram 7y ago
I guess it depends on... what "work" means. So I worked on Deep Networks for quantum chemistry (I'm not a physicist or chemist but) I can tell you people were ecstatic about the possibility that the approximations that the neural nets come up with might get closer to real physics than current approximations right now. This w/o any needed advancements. Some challenges are so difficult in these areas that approximations are the best that are possible. It's kind of similar to drug discovery now.. like if there are models which can help narrow down potential molecules / targets, that has tremendous potential even if the system needs to be double checked by a person. So it's hard to see "don't work" as anything but buzzy. BUT I will agree with you neuroscience will help develop our understanding of cognition. I just wanted to blast these other applications, because I think people get this idea that AI has to be AI for anything interesting to happen... but there are really niche applications where people don't think these tools are experimental. And what you describe may already be happening, Geoff Hinton's critique of modern deep nets seems to be a call to get more biological. (Thinking of capsules nets).

intuitionist 7y ago

> Deep learning and machine learning don’t work. Quantitative math will always prevail, as it always has. What do you think machine learning is, if not “quantitative math”? Deep learning is just linear algebra and calculus, and things like random forests are even simpler mathematically.

acdc4life 7y ago
Machine learning is glorified curve fitting. DL isn't even mathematically sound, back propagation has no proof of convergence. Quantitative math is about extracting natural laws, and mapping them to mathematical structures. You could use DL to predict planetary motion, and get pretty good at it. But this isn't a quantitative understanding of the world. You didn't learn anything. Physics in contrast has the laws of motion and gravitation. You can directly model arbitrary planets. Moreover, you can model arbitrary rigid bodies, from cars to space shuttles. Your ML, DL random forrest etc. all use math, sure. But so did the Keplarian models of motion. You aren't qualitatively deducing math that governs the world, but forcing an arbitrarily chosen mathematical structure to your data.
- deleted 7y ago
  [deleted]
- intuitionist 7y ago
  If we’re throwing out anything that doesn’t have a proof of convergence as “not mathematically sound,” you can kiss fluid mechanics goodbye, as well as lots of other subfields of physics that rely on partial differential equations.

QuackingJimbo 7y ago

I agree that DL/ML is destined to fail in domains like this but can you expand on this reasoning? What exactly do you mean by "quantitative math" (I haven't heard this phrase used in this way before)? And what were the equivalents of DL/ML for physics before calculus?

acdc4life 7y ago
Quantitative math might be the wrong word, maybe applied math? In quantitative finance, you make quantitative models about the world, and build math that realize those assumptions and understanding. A simple example: options are a great financial trading instrument that you can model mathematically, the simplest being the Black-Scholes model. You can imply things like volatility of the stock price based on the price of the option to get a better understanding of what a risk neutral market is thinking, and compare that to the actual market distribution. >And what were the equivalents of DL/ML for physics before calculus? This is a good question. Before Newtonian calculus, and the laws of gravitation, people were building very complicated conic models (ie. eclipses, parabolas etc.) to get better and better prediction of planetary motion. A lot of parametric math came out of this, with many sophisticated models getting better and better, giving these astronomers an illusion of progress. However, Newton's insight was that motion is connected to mass, and this insight was the basis of how to derive the laws of motion, which gave us the laws of gravitation (F = (Gm1m2/r^2)). This insight eliminated the previous Keplarian models of motion, because you were now able to predict the motion of arbitrary rigid bodies using very simple math (we teach this in highschool). Ofcourse, Newtonian motion has its limitations that's why we have quantum physics and Einstien's relativity theory. But for practical technological applications, Newtonian physics on its own gets you incredibly far. Where is ML/DL? It would be akin to Keplarian elliptical motion. More realistically however, it's closer to aether theory of light, and will go the way of GOFAI. This stuff isn't grounded in modelling any scientific observation. Moreover, they are mathematically useless. Back propagation doesn't converge, and why should you fit your data to an arbitrary mathematical structure? In practise, DL/ML doesn't work at all, you will be much more successful by modelling your problem mathematically. For example, consider an automobile manufacturer, which has all kinds of moving parts in their planes. They typically model each part mathematically (ie. gear x under goes exponential time decay), and imply their parameters using rigours test data. Then you use some sort of an empirical statistical model to predict the failure. I've seen deep learning companies come and fall flat on their face trying to beat the accuracy of these deterministic systems. Those guys needed a lot of data, and GPUs. I'm not even criticizing the fact that DL is a black box. It's worse, it's inferior to everything out there on every metric imaginable. These mathematical models in contrast have been in production for decades, with yearly updates, and they run in real time with little historical data, they are fully understandable and they beat every method we know of. This isn't the first time multi layer perceptrons gained hype. They didn't work in the 80s, or 90s or the 2000s, they don't work now. The math behind DL is the same that we had in the 80s, they just called it multi layer perceptron. None of the ideas in modern ML/DL are new, all these ideas like reinforcement learning, GANs etc.

bduerst 7y ago

>Deep learning and machine learning don’t work. Quantitative math will always prevail, I have a neural net onboard my phone which automatically detects songs offline and tells me what they are. Is that semantically 'quantitative math' and not machine learning?

acdc4life 7y ago
Quantitative math, or applied math isn't based on fitting data to an arbitrary mathematical structure. It's looking at real life, and deriving the mathematical laws that govern what you see. You could have a neural net predict planetary motion. However, it doesn't know jack shit about physics. >I have a neural net onboard my phone which automatically detects songs offline and tells me what they are. MP3 uses something called psycho acoustics, which is a quantitative model on human perception, which is used to eliminate frequencies that can't be heard based on this model. Your neural network doesn't tell you what features make songs distinct, it's not a quantitative model at all, but a black box heuristic on what the important features are superficially. If actual mathematicians worked on this problem, I guarantee you they'd do a better job, and their models would work on a commadore64, with real time training. Moreover it would tell you things like who is singing, if it's a live performance, which concert it was.
- sonnyblarney 7y ago
  " If actual mathematicians worked on this problem, I guarantee you they'd do a better job" No, this is wrong. Some of the most brilliant people in the world have been working on image recognition, voice recognition etc. and AI is crushing all of their work. "Your neural network doesn't tell you what features make songs distinct, it's not a quantitative model at all" - it doesn't matter at all if our objective is detecting the song. Neither does the mp3 compression algorithm.
  - acdc4life 7y ago
    >Some of the most brilliant people in the world have been working on image recognition, voice recognition etc. and AI is crushing all of their work. This is very true. I take my stronger statements back, MAINSTREAM mathematicians attempting this problem are all wrong, and have been wrong for 50 years. But you do need the right theory, and the right math that realizes this theory. "AI" is superficially beating the work in computer vision. Computer vision is complete bogus. The gabor filters, fourier transfroms etc. are all wrong conceptually. The known methods do abysmally on basic tasks like object recognition, texture segmentation etc. But they keep trying it. I would take this one step further: computer vision, audio and NLP researchers have been stuck in a rut for the past 50 years. DL is beating THEIR math, but this is because of data and computation speed, not because of any insights. But DL is also wrong, and giving you an illusion of progress. Both of these things are doomed to go the way of GOFAI. I can go into great detail and carefully explain why MAINSTREAM contemporary ideas in math for vision, audition and language are completely wrong, and have been wrong for 50 years. What is the right model? Like I mentioned before, the right ideas are emerging, neural networks will dominate, just not DL.

sonnyblarney 7y ago

If you mean for medical diagnosis, maybe, but you do realize that NN/AI is totally state of the art for many tasks? NLP, image recognition etc. ? AI is in a hype bubble right now surely, but it's a 'very real' thing that's going to infiltrate a lot of areas.

acdc4life 7y ago
>you do realize that NN/AI is totally state of the art for many tasks? Being state of the art doesn't imply that these things will solve these problems. In ML terms, how do you know that NN/AI isn't a local maxima that we need to jump out of? All NLP systems are joke. Sure replace Watson with DL, might perform better on Jeopardy. But in real conversations? Forget it. I wouldn't bet on these things. NN will win, but not the back propagation, ReLu, sigmoid or whatever pseudo science that is the current buzzword. There is 50 years worth of understanding in actual neuroscience and cognitive modelling that no one has paid attention to, and new design principles are emerging that will influence mathematics.
- chillacy 7y ago
  Because DL isn't great at NLP (and may never solve "real conversations" aka AGI), it's worthless? It's the best performing tool we have for NLP, image recognition, etc. Is it a local maxima? Probably. But it's out there solving real problems nonetheless. We'll capture all the gains we can and then move on after.
- sonnyblarney 7y ago
  "Being state of the art doesn't imply that these things will solve these problems. " I suggest you are misinformed about the state of AI. AI is currently ahead of all other approaches in many fields. It's lead to quite a number of practical advances and breakthroughs. The 'best examples' are those that I described, but there are many more. Your comments indicate I think some ignorance on the issue - I think I see the point you are trying to make but I also submit that you're not aware of what AI is doing today. 'Self driving cars' would be impossible without AI today, for example. The vision systems depend on AI it's a breakthrough without which we simply wouldn't have the tech.

opsiprogram 7y ago

I guess it depends on... what "work" means. So I worked on Deep Networks for quantum chemistry (I'm not a physicist or chemist but) I can tell you people were ecstatic about the possibility that the approximations that the neural nets come up with might get closer to real physics than current approximations right now. This w/o any needed advancements. Some challenges are so difficult in these areas that approximations are the best that are possible. It's kind of similar to drug discovery now.. like if there are models which can help narrow down potential molecules / targets, that has tremendous potential even if the system needs to be double checked by a person. So it's hard to see "don't work" as anything but buzzy. BUT I will agree with you neuroscience will help develop our understanding of cognition. I just wanted to blast these other applications, because I think people get this idea that AI has to be AI for anything interesting to happen... but there are really niche applications where people don't think these tools are experimental. And what you describe may already be happening, Geoff Hinton's critique of modern deep nets seems to be a call to get more biological. (Thinking of capsules nets).