6 ms·

It's ironic that the much more significant ultimate success of deep learning happened despite a lack of government funding, if Hinton is to be believed. The 90

by ponow 1y ago

It's ironic that the much more significant ultimate success of deep learning happened despite a lack of government funding, if Hinton is to be believed. The 90s were a neural net winter, and success required faster computation, a private success. I lose zero sleep at the prospect that there would be zero government robotics research funding. If the advantages are there, profit seekers will find a way. We must stop demonizing private accumulations of capital, "ending" billionaires and "monopolies" that are offering more things at lower cost. Small enterprises cannot afford a Bell Labs, a Watson Research, a Deep Mind, a Xerox PARC, etc.

jpeloquin 1y ago
Once something has a predictable ROI (can be productized and sold), profit seekers will find a way. The role of publicly funded research is to get ideas that are not immediately profitable to the stage that investors can take over. Publicly funded research also supports investor-funded R&D by educating their future work force. The provided examples do not clearly support the idea that industry can compensate for a decrease in government-funded basic research. Bell Labs was the product of government action (antitrust enforcement), not a voluntary creation. The others are R&D (product development) organizations, not research organizations. Of those listed, Xerox PARC is the most significant, but from the profit-seeking perspective it's more of a cautionary tale since it primarily benefited Xerox's competitors. And Hinton seems to have received government support; his backpropagation paper at least credits ONR. As I understand it, the overall deep learning story is that basic research, including government-funded research, laid theoretical groundwork that capital investment was later able to scale commercially once video games drove development of the necessary hardware.
regularization 1y ago
Hinton and his students studied for years on US (and then Canadian) government grants. The year Alexnet came out, Nvidia was awarded tens of millions by DARPA for Project Osprey. It's an odd historical revisionism where from Fairchild to the Internet to the web to AI, government grants and government spending are washed out of the picture. The government funded AI research for decades.
- absolutelastone 1y ago
  I think their point is the billions in private investment which preceded those millions. I think this is a common issue in computer science, where credit is given to sexy "software applications" like AI when the real advances were in the hardware that enabled them, which everyone just views as an uninteresting commodity.
  - Frost1x 1y ago
    I wonder if it deals more with the approachability of software applications. If I even begin to think I’d compete with NVIDIA delivering similar hardware, I’d very quickly realize I was an idiot. Meanwhile as a single individual, there is still a reasonable amount of commercial markets of software I really do have some chance at tackling or competing against. As software complexity rises it’s becoming far less tractable than it was in say the 90s but there are still areas individuals and small sums of capital can enter. I think that makes the sector alluring in general. Hardware is just in general capital intensive, not even including all the intellectual capital needed. So it’s not that it’s uninteresting or even a commodity to me, it’s just a stone wall that whatever is there is there and that’s it in my mind.
    - absolutelastone 1y ago
      That difference in difficulties is kind of the point. Imagine, as an extreme, a company makes a machine with certain functions performed based on which button combinations you press. A second company gets a patent for using the first company's machine for doing various tasks by pressing various button combinations, which are new uses of the machine no one had thought of yet. Now the second company has all the bargaining power in the market and so gets giant margins, despite doing a tiny fraction of the work it takes to make those tasks possible. I wonder if our current system ended up this way because it is the most efficient in terms of specialization, or because the patent system drove things in this direction where the people last dealing with customers (i.e., those making the software layer) have the best info of what tasks the customers want to do with their computers, and hence patent the solutions first. Leaving hardware vendors no choice but to serve the software monopolies (one after another since the 80's).
standardUser 1y ago
You are suggesting unilateral disarmament. Allowing other nations, not all of them friendly, to take the lead in science and technology as they continue to fund their own research and poach our best and brightest.

by ponow 1y ago

jpeloquin 1y ago

Once something has a predictable ROI (can be productized and sold), profit seekers will find a way. The role of publicly funded research is to get ideas that are not immediately profitable to the stage that investors can take over. Publicly funded research also supports investor-funded R&D by educating their future work force. The provided examples do not clearly support the idea that industry can compensate for a decrease in government-funded basic research. Bell Labs was the product of government action (antitrust enforcement), not a voluntary creation. The others are R&D (product development) organizations, not research organizations. Of those listed, Xerox PARC is the most significant, but from the profit-seeking perspective it's more of a cautionary tale since it primarily benefited Xerox's competitors. And Hinton seems to have received government support; his backpropagation paper at least credits ONR. As I understand it, the overall deep learning story is that basic research, including government-funded research, laid theoretical groundwork that capital investment was later able to scale commercially once video games drove development of the necessary hardware.

regularization 1y ago

Hinton and his students studied for years on US (and then Canadian) government grants. The year Alexnet came out, Nvidia was awarded tens of millions by DARPA for Project Osprey. It's an odd historical revisionism where from Fairchild to the Internet to the web to AI, government grants and government spending are washed out of the picture. The government funded AI research for decades.

absolutelastone 1y ago
I think their point is the billions in private investment which preceded those millions. I think this is a common issue in computer science, where credit is given to sexy "software applications" like AI when the real advances were in the hardware that enabled them, which everyone just views as an uninteresting commodity.
- Frost1x 1y ago
  I wonder if it deals more with the approachability of software applications. If I even begin to think I’d compete with NVIDIA delivering similar hardware, I’d very quickly realize I was an idiot. Meanwhile as a single individual, there is still a reasonable amount of commercial markets of software I really do have some chance at tackling or competing against. As software complexity rises it’s becoming far less tractable than it was in say the 90s but there are still areas individuals and small sums of capital can enter. I think that makes the sector alluring in general. Hardware is just in general capital intensive, not even including all the intellectual capital needed. So it’s not that it’s uninteresting or even a commodity to me, it’s just a stone wall that whatever is there is there and that’s it in my mind.
  - absolutelastone 1y ago
    That difference in difficulties is kind of the point. Imagine, as an extreme, a company makes a machine with certain functions performed based on which button combinations you press. A second company gets a patent for using the first company's machine for doing various tasks by pressing various button combinations, which are new uses of the machine no one had thought of yet. Now the second company has all the bargaining power in the market and so gets giant margins, despite doing a tiny fraction of the work it takes to make those tasks possible. I wonder if our current system ended up this way because it is the most efficient in terms of specialization, or because the patent system drove things in this direction where the people last dealing with customers (i.e., those making the software layer) have the best info of what tasks the customers want to do with their computers, and hence patent the solutions first. Leaving hardware vendors no choice but to serve the software monopolies (one after another since the 80's).

standardUser 1y ago

You are suggesting unilateral disarmament. Allowing other nations, not all of them friendly, to take the lead in science and technology as they continue to fund their own research and poach our best and brightest.