7 ms·

>They're still terrible at logical reasoning. 2 years ago a machine that understands natural language and is capable of any arbitrary, free-form logic or probl

by ux-app 3y ago

>They're still terrible at logical reasoning. 2 years ago a machine that understands natural language and is capable of any arbitrary, free-form logic or problem solving was pure science fiction. I'm baffled by this kind of dismissal tbh. >but LLMs are never going to go into a recursive loop of self-improvement never is a long time.

deleted 3y ago
[deleted]
leonidasv 3y ago
Two years ago we already had GPT-2, that was capable of some problem solving and logic following. It was archaic, sure, it produced a lot of gibberish, yes, but if you followed OpenAI releases closely, you wouldn't think that something like GPT3.5 was "pure science fiction", it would just look as the inevitable evolution of GPT-2 in a couple of years given the right conditions.
- ux-app 3y ago
  that's pedantic. switch 2 years to 5 years and the point still stands.
  - edgyquant 3y ago
    No it isn’t. Even before transformers people were doing cool things with LSTMs and RNNs before that. People following this space haven’t really been surprised by any of these advancements. It’s a straight forward path imo
- canjobear 3y ago
  In hindsight it’s an obvious evolution, but in practice vanishingly few people saw it coming.
  - leonidasv 3y ago
    Few people saw it coming in just two years, sure. But most people following this space were already expecting a big evolution like the one we saw in 5-ish years. For example, take this thread: https://news.ycombinator.com/item?id=21717022 https://news.ycombinator.com/item?id=21717022 It's a text RPG game built on top of GPT-2 that could follow arbitrary instructions. It was a full project with custom training for something that you can get with a single prompt on ChatGPT nowadays, but it clearly showcased what LLMs were capable of and things we take for granted now. It was clear, back then, that at some point ChatGPT would happen.

by ux-app 3y ago

deleted 3y ago

[deleted]

leonidasv 3y ago

Two years ago we already had GPT-2, that was capable of some problem solving and logic following. It was archaic, sure, it produced a lot of gibberish, yes, but if you followed OpenAI releases closely, you wouldn't think that something like GPT3.5 was "pure science fiction", it would just look as the inevitable evolution of GPT-2 in a couple of years given the right conditions.

ux-app 3y ago
that's pedantic. switch 2 years to 5 years and the point still stands.
- edgyquant 3y ago
  No it isn’t. Even before transformers people were doing cool things with LSTMs and RNNs before that. People following this space haven’t really been surprised by any of these advancements. It’s a straight forward path imo
canjobear 3y ago
In hindsight it’s an obvious evolution, but in practice vanishingly few people saw it coming.
- leonidasv 3y ago
  Few people saw it coming in just two years, sure. But most people following this space were already expecting a big evolution like the one we saw in 5-ish years. For example, take this thread: https://news.ycombinator.com/item?id=21717022 https://news.ycombinator.com/item?id=21717022 It's a text RPG game built on top of GPT-2 that could follow arbitrary instructions. It was a full project with custom training for something that you can get with a single prompt on ChatGPT nowadays, but it clearly showcased what LLMs were capable of and things we take for granted now. It was clear, back then, that at some point ChatGPT would happen.