8 ms·

As one of the comments on reddit posts - it's not just big tech companies, but also entire university teams which feel the goalposts moving miles ahead all of a

As one of the comments on reddit posts - it's not just big tech companies, but also entire university teams which feel the goalposts moving miles ahead all of a sudden. Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already...

ChuckNorris89 4y ago
>Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already... In fairness most PhD topics people work on these days, outside of the select few top research universities in the world, are obsolete before they begin. At least from what my friends in the field tell me.
- Yoric 4y ago
  Anecdata of one: I finished my PhD about 20 years ago in programming language theory. I created something innovative but not revolutionary. Given how slowly industry is catching up on my domain, it will probably take another 20-30 years before something similarly powerful makes it into an industrial programming language. Counter-anecdata of one: On the other hand, one of the research teams of which I've been a member after my PhD was basically inventing Linux containers (in competition with other teams). Industry caught up pretty quickly on that. Still, academia arrived first. edit Rephrased to decrease pedantism.
  - nemaar 4y ago
    > something as powerful as what I created Could you give us more detail? It sounds intriguing.
    - Yoric 4y ago
      I developed a new static analysis (a type system, to be precise) to guarantee statically that a concurrent/distributed system could fail gracefully in case of (D)DoS or other causes of resource exhaustion. Other people in that field developed comparable tools to statically guarantee algorithmic space or time complexity of implementations (including the good use of timeouts/resource sandboxes if necessary). Or type system-level segregation between any number of layers of classified/declassified information within a system. Or type systems to guarantee that binary (byte)code produced on a machine could find all its dependencies on another machine. Or type systems to prove that an algorithm was invariant with respect to all race conditions. Or to guarantee that a non-blocking algorithm always progresses. Or to detect deadlocks statically. etc. All these things have been available in academia for a long time now. Even languages such as Rust or Scala, that offer cutting edge (for the industry) type systems, are mostly based on academic research from the 90s. For comparison, garbage-collectors were invented in the 60s and were still considered novelties in the industry in the early 2000s.
      - codethief 4y ago
        Is there a good resource (a review paper maybe?) to get an overview over such programming language / type system topics?
        Yoric 4y ago
        I can't think of any from the top of my head. Perhaps looking at the proceedings of ICFP and POPL can help?
simonh 4y ago
I'm not too worried about that. We don't actually understand fully how LLMs function internally, so research on how language works and how to process it is still useful in advancing our understanding. It may not lead to products that can compete with GPT, but PhDs aren't about commercialisation, they're about advancing human knowledge.
- oldgradstudent 4y ago
  > We don't actually understand fully A touch of understatement.
echelon 4y ago
All these people don't understand how hireable and desirable they are now. They need to get out of academia and plugged into AI positions at tech companies and startups. Their value just went up tremendously, even if their PhD thesis got cancelled. Easily millionaires waiting to happen. --- edit: Can't respond to child comment due to rate limit, so editing instead. > That is not how it works at all. Speak for yourself. I'm hiring folks off 4chan, and they're kicking ass with pytorch and can digest and author papers just fine. People stopped caring about software engineering and data science degrees in the late 2010's. People will stop caring about AI/ML PhDs as soon as the challenge to hire talent hits - and it will hit this year.
- goethes_kind 4y ago
  That is not how it works at all. You won't get hired if you don't have the academic pedigree in the first place. That means a completed Ph.D and good publications in good journals.
  - Yoric 4y ago
    Hired in academia? Sure. Hired in industry. That's the opposite. I've had a friend who had to hide that they had a PhD to be hired...
    - goethes_kind 4y ago
      I guess we are living in two different universes. Any job ad for an ML role or ML adjacent role says Ph.d required or Ph.d preferable. Maybe it is also a matter of location. I am in Germany. For a plain SWE role a Ph.d might be a disadvantage here too, but for anything ML related it is mandatory from what I can see.
      - visarga 4y ago
        In my hiring experience as an interviewer, 90% of candidates with PhD or not will actually have mediocre grasp on ML. It is a rare happy day when I get a good candidate. We interview for months for one hire. I got to interview candidates worldwide so I've seen people from many countries.
sgt101 4y ago
Perhaps - but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4 Will this effect the job market (both academic and commercial) for these folks? It's very hard to say. Clearly lots of value will be generated by the new generation of models. There will be a lot of catchup and utilisation work where people will want to have models in house and with specific features that the hyperscale models don't have (for example constrained training sets). I'm wondering how many commercial illustrators have had their practices disrupted by Stable Diffusion? Will the same dynamics (what ever they are) apply for the use of LLM's?
- hn_throwaway_99 4y ago
  > but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4 Pretty hard disagree. Even if your NLP PhD topic is looking at hypotheses on underlying processes about how languages work (and LLMs can't give you this insight), 9 times out of 10 it's with an eye for some sort of "applicability" of this for the future. GPT-4 just cut off the applicability parts of this for huge swaths of NLP research.
goethes_kind 4y ago
This is where it pays off to be researching something completely esoteric rather than something immediately applicable. I mostly scoffed at such research in the past, but now I see the value of it. The guy researching QML algorithms for NLP is not panicking yet, I think.
pyuser583 4y ago
Isn’t that the sort of thing advisors are supposed to caution against? And aren’t PhDs supposed have a theoretical underpinning?
paganel 4y ago
> Imagine working on your PhD on chat bots To be honest, that's a sh*tty PhD to begin with, it doesn't bring anything good and really worthwhile to the world, quite the contrary.
jll29 4y ago
We are just building a chatbot to assist students that study (that study AI itself, to be precise). We could not "just use ChatGPT" because the answers have to be correct, or students could take legal action against the university in situations where they'd give an answer in an exam that the chatbot gave them and that might be wrong. It's exciting for sure to compare e.g. ChaptGPT with other approaches, for a set of controlled benchmark questions such as: -- snip -- Q: What is the A* algorithm? A: The A* algorithm is a popular heuristic search algorithm used in pathfinding and graph traversal. It is an extension of Dijkstra's algorithm, but uses a heuristic function to guide the search towards the goal node, which can make it more efficient than Dijkstra's algorithm in certain situations. The algorithm works by maintaining a priority queue of nodes to be explored, where the priority of each node is the sum of its actual cost from the starting node (g-score) and an estimate of the cost to reach the goal node (h-score), based on a heuristic function. The heuristic function must be admissible, meaning it never overestimates the actual cost to the goal node. At each step, the algorithm selects the node with the lowest priority from the queue and expands it, generating its neighboring nodes and computing their priority scores. The algorithm then adds these nodes to the queue if they have not been explored before or if a better path to them has been found. If the goal node is reached, the algorithm terminates and returns the path from the starting node to the goal node. If the queue becomes empty before the goal node is reached, then there is no path from the starting node to the goal node. Overall, the A* algorithm is a powerful and widely used algorithm that has many applications in fields such as robotics, video games, and logistics. -- snip --
- aimbivalent 4y ago
  This is variation on the common "it has to be perfect" argument. I don't really buy it. Right now, GPT 4 would earn a top 10% SAT score, implying that it can give very high quality answers on a range of scholarly topic. This output is basically free. Nobody that's beholden to even mild economic pressures is going to pay for an expensively constructed 100% solution if they can have the 99% solution for free.

ChuckNorris89 4y ago

>Imagine working on your PhD on chat bots since start of 2022. Your entire PhD topic might be irrelevant already... In fairness most PhD topics people work on these days, outside of the select few top research universities in the world, are obsolete before they begin. At least from what my friends in the field tell me.

Yoric 4y ago
Anecdata of one: I finished my PhD about 20 years ago in programming language theory. I created something innovative but not revolutionary. Given how slowly industry is catching up on my domain, it will probably take another 20-30 years before something similarly powerful makes it into an industrial programming language. Counter-anecdata of one: On the other hand, one of the research teams of which I've been a member after my PhD was basically inventing Linux containers (in competition with other teams). Industry caught up pretty quickly on that. Still, academia arrived first. edit Rephrased to decrease pedantism.
- nemaar 4y ago
  > something as powerful as what I created Could you give us more detail? It sounds intriguing.
  - Yoric 4y ago
    I developed a new static analysis (a type system, to be precise) to guarantee statically that a concurrent/distributed system could fail gracefully in case of (D)DoS or other causes of resource exhaustion. Other people in that field developed comparable tools to statically guarantee algorithmic space or time complexity of implementations (including the good use of timeouts/resource sandboxes if necessary). Or type system-level segregation between any number of layers of classified/declassified information within a system. Or type systems to guarantee that binary (byte)code produced on a machine could find all its dependencies on another machine. Or type systems to prove that an algorithm was invariant with respect to all race conditions. Or to guarantee that a non-blocking algorithm always progresses. Or to detect deadlocks statically. etc. All these things have been available in academia for a long time now. Even languages such as Rust or Scala, that offer cutting edge (for the industry) type systems, are mostly based on academic research from the 90s. For comparison, garbage-collectors were invented in the 60s and were still considered novelties in the industry in the early 2000s.
    - codethief 4y ago
      Is there a good resource (a review paper maybe?) to get an overview over such programming language / type system topics?
      - Yoric 4y ago
        I can't think of any from the top of my head. Perhaps looking at the proceedings of ICFP and POPL can help?

simonh 4y ago

I'm not too worried about that. We don't actually understand fully how LLMs function internally, so research on how language works and how to process it is still useful in advancing our understanding. It may not lead to products that can compete with GPT, but PhDs aren't about commercialisation, they're about advancing human knowledge.

oldgradstudent 4y ago
> We don't actually understand fully A touch of understatement.

echelon 4y ago

All these people don't understand how hireable and desirable they are now. They need to get out of academia and plugged into AI positions at tech companies and startups. Their value just went up tremendously, even if their PhD thesis got cancelled. Easily millionaires waiting to happen. --- edit: Can't respond to child comment due to rate limit, so editing instead. > That is not how it works at all. Speak for yourself. I'm hiring folks off 4chan, and they're kicking ass with pytorch and can digest and author papers just fine. People stopped caring about software engineering and data science degrees in the late 2010's. People will stop caring about AI/ML PhDs as soon as the challenge to hire talent hits - and it will hit this year.

goethes_kind 4y ago
That is not how it works at all. You won't get hired if you don't have the academic pedigree in the first place. That means a completed Ph.D and good publications in good journals.
- Yoric 4y ago
  Hired in academia? Sure. Hired in industry. That's the opposite. I've had a friend who had to hide that they had a PhD to be hired...
  - goethes_kind 4y ago
    I guess we are living in two different universes. Any job ad for an ML role or ML adjacent role says Ph.d required or Ph.d preferable. Maybe it is also a matter of location. I am in Germany. For a plain SWE role a Ph.d might be a disadvantage here too, but for anything ML related it is mandatory from what I can see.
    - visarga 4y ago
      In my hiring experience as an interviewer, 90% of candidates with PhD or not will actually have mediocre grasp on ML. It is a rare happy day when I get a good candidate. We interview for months for one hire. I got to interview candidates worldwide so I've seen people from many countries.

sgt101 4y ago

Perhaps - but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4 Will this effect the job market (both academic and commercial) for these folks? It's very hard to say. Clearly lots of value will be generated by the new generation of models. There will be a lot of catchup and utilisation work where people will want to have models in house and with specific features that the hyperscale models don't have (for example constrained training sets). I'm wondering how many commercial illustrators have had their practices disrupted by Stable Diffusion? Will the same dynamics (what ever they are) apply for the use of LLM's?

hn_throwaway_99 4y ago
> but normally you'll have a narrowly defined and very specific technical topic/hypothesis that you're working on, and many/most of these aren't going to be closed off by ChatGPT4 Pretty hard disagree. Even if your NLP PhD topic is looking at hypotheses on underlying processes about how languages work (and LLMs can't give you this insight), 9 times out of 10 it's with an eye for some sort of "applicability" of this for the future. GPT-4 just cut off the applicability parts of this for huge swaths of NLP research.

goethes_kind 4y ago

This is where it pays off to be researching something completely esoteric rather than something immediately applicable. I mostly scoffed at such research in the past, but now I see the value of it. The guy researching QML algorithms for NLP is not panicking yet, I think.

pyuser583 4y ago

Isn’t that the sort of thing advisors are supposed to caution against? And aren’t PhDs supposed have a theoretical underpinning?

paganel 4y ago

> Imagine working on your PhD on chat bots To be honest, that's a sh*tty PhD to begin with, it doesn't bring anything good and really worthwhile to the world, quite the contrary.

jll29 4y ago

We are just building a chatbot to assist students that study (that study AI itself, to be precise). We could not "just use ChatGPT" because the answers have to be correct, or students could take legal action against the university in situations where they'd give an answer in an exam that the chatbot gave them and that might be wrong. It's exciting for sure to compare e.g. ChaptGPT with other approaches, for a set of controlled benchmark questions such as: -- snip -- Q: What is the A* algorithm? A: The A* algorithm is a popular heuristic search algorithm used in pathfinding and graph traversal. It is an extension of Dijkstra's algorithm, but uses a heuristic function to guide the search towards the goal node, which can make it more efficient than Dijkstra's algorithm in certain situations. The algorithm works by maintaining a priority queue of nodes to be explored, where the priority of each node is the sum of its actual cost from the starting node (g-score) and an estimate of the cost to reach the goal node (h-score), based on a heuristic function. The heuristic function must be admissible, meaning it never overestimates the actual cost to the goal node. At each step, the algorithm selects the node with the lowest priority from the queue and expands it, generating its neighboring nodes and computing their priority scores. The algorithm then adds these nodes to the queue if they have not been explored before or if a better path to them has been found. If the goal node is reached, the algorithm terminates and returns the path from the starting node to the goal node. If the queue becomes empty before the goal node is reached, then there is no path from the starting node to the goal node. Overall, the A* algorithm is a powerful and widely used algorithm that has many applications in fields such as robotics, video games, and logistics. -- snip --

aimbivalent 4y ago
This is variation on the common "it has to be perfect" argument. I don't really buy it. Right now, GPT 4 would earn a top 10% SAT score, implying that it can give very high quality answers on a range of scholarly topic. This output is basically free. Nobody that's beholden to even mild economic pressures is going to pay for an expensively constructed 100% solution if they can have the 99% solution for free.