4 ms·

No, it is `n * 3`. The challenge is simple: starting from "1", we must reach "17" by combining the operations `x * 3`, `x * 2` and `x + 5`. What is embarrassing

No, it is `n * 3`. The challenge is simple: starting from "1", we must reach "17" by combining the operations `x * 3`, `x * 2` and `x + 5`. What is embarrassing is that Gemini manages to read the formal jargon and understand the challenge just fine. Yet it fails to combine these operations to get from "1" to "17", which is something most 10 yo kids would be able to do.

qsort 3y ago
Yeah, seems like I got that right. That graduate-level course must have worked. But then: (f (g (h (g z)))) results in: ((((1 * 2) + 5) * 2) * 3) = ... not 17? while it would work if the type of f was corrected. Or, again, am I missing something?
- deleted 3y ago
  [deleted]
- deleted 3y ago
  [deleted]
- pierrebai 3y ago
  You are right and I had the same reaction. The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5. Is it not ironic that the supposedly test of AGI is flawed and its human designer fail to see it and denied it when presented with facts. Maybe the test designer is hallucinating just as much as those LLM? :)
  - mahogany 3y ago
    It's pretty amusing and it is not the first time I've seen this. Random example: https://news.ycombinator.com/item?id=38387168 https://news.ycombinator.com/item?id=38387168 It's a little scary that it can be so hard to evaluate the correctness of these LLMs even when we are paying close attention and looking for mistakes. Or maybe the scary part is that we can become biased when we want to believe.
  - LightMachine 3y ago
    Oh fuck. Well, in my defense, nobody is claiming I'll design fusion reactors and cure cancer by 2027
    - famouswaffles 3y ago
      No, but we at least acknowledge you a general intelligence like all humans. I'm not sure when agi - artificial general intelligence began to mean anything other than artificial and generally intelligent. agi may as well be God, the bars some people have.
  - scarmig 3y ago
    > The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5. Isn't that (h(g(h z)))? And, FWIW, at least in my test, Gemini gets that in its final answer, though it failed in the two other drafts: https://g.co/gemini/share/c922e7ef62aa https://g.co/gemini/share/c922e7ef62aa ChatGPT sputters: https://chat.openai.com/share/25abbf47-2ed4-4635-a351-90a9a6cfbc65 https://chat.openai.com/share/25abbf47-2ed4-4635-a351-90a9a6... (ETA more Gemini testing suggests its correct answer was a one-off)
- LightMachine 3y ago
  Oh, lol, you're right. Seems like I'm dumber than both AIs. GPT-4 mixed up `h` and `f`, so it also got it wrong, so this is a draw and both AIs (and, apparently, myself) are terrible at reasoning. Guess we're not curing cancer with computers anytime soon :')
  - motoxpro 3y ago
    so much for "Anyone familiar with the syntax / jargon should be able to answer this specific problem in ~5 seconds of thinking, though. And I mean it, even a 10yo kid should..."

qsort 3y ago

Yeah, seems like I got that right. That graduate-level course must have worked. But then: (f (g (h (g z)))) results in: ((((1 * 2) + 5) * 2) * 3) = ... not 17? while it would work if the type of f was corrected. Or, again, am I missing something?

deleted 3y ago
[deleted]
deleted 3y ago
[deleted]
pierrebai 3y ago
You are right and I had the same reaction. The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5. Is it not ironic that the supposedly test of AGI is flawed and its human designer fail to see it and denied it when presented with facts. Maybe the test designer is hallucinating just as much as those LLM? :)
- mahogany 3y ago
  It's pretty amusing and it is not the first time I've seen this. Random example: https://news.ycombinator.com/item?id=38387168 https://news.ycombinator.com/item?id=38387168 It's a little scary that it can be so hard to evaluate the correctness of these LLMs even when we are paying close attention and looking for mistakes. Or maybe the scary part is that we can become biased when we want to believe.
- LightMachine 3y ago
  Oh fuck. Well, in my defense, nobody is claiming I'll design fusion reactors and cure cancer by 2027
  - famouswaffles 3y ago
    No, but we at least acknowledge you a general intelligence like all humans. I'm not sure when agi - artificial general intelligence began to mean anything other than artificial and generally intelligent. agi may as well be God, the bars some people have.
- scarmig 3y ago
  > The correct answer should have been: (f (h (g (h z)))) AKA ((1 + 5) * 2) + 5. Isn't that (h(g(h z)))? And, FWIW, at least in my test, Gemini gets that in its final answer, though it failed in the two other drafts: https://g.co/gemini/share/c922e7ef62aa https://g.co/gemini/share/c922e7ef62aa ChatGPT sputters: https://chat.openai.com/share/25abbf47-2ed4-4635-a351-90a9a6cfbc65 https://chat.openai.com/share/25abbf47-2ed4-4635-a351-90a9a6... (ETA more Gemini testing suggests its correct answer was a one-off)
LightMachine 3y ago
Oh, lol, you're right. Seems like I'm dumber than both AIs. GPT-4 mixed up `h` and `f`, so it also got it wrong, so this is a draw and both AIs (and, apparently, myself) are terrible at reasoning. Guess we're not curing cancer with computers anytime soon :')
- motoxpro 3y ago
  so much for "Anyone familiar with the syntax / jargon should be able to answer this specific problem in ~5 seconds of thinking, though. And I mean it, even a 10yo kid should..."