17 ms·

What can LLMs never do?

• tacocataco 2y ago
  Find a way to get humans to love each other.
• deleted 2y ago
  [deleted]
• 3abiton 2y ago
  The article should be titled " What can LLM never do, yet". By definition, Large Language Models would keep growing larger and larger, to be trained on faster and more advanced hardware, and certain points like "completing complex chains of logical reasoning" tasks, would be just a time hurdle. Only time will tell.
  • g5g5g5g5 2y ago
    > The article should be titled " What can LLM never do, yet". I don't think it should. It's more interesting to know what LLMs will _never_ be able to do (if anything).
    • shawntan 2y ago
      Yes, but the article doesn't really answer this question.
  • ekianjo 2y ago
    Math reasoning is still a non solved problem even if the rest of the capabilities are getting better. This means the transformers architecture may not be the best way to approach all problems
  • gmuslera 2y ago
    Maybe the wording is correct. Looks like a hard limit on doing what a LLM just do. If it goes beyond that, then is something more, or at least different, than a LLM.
  • amelius 2y ago
    The big question is if LLMs are capable enough to converge to AGI. It might very well be that as we pour in more resources that they converge to something only slightly more useful but similar as we have today.
  • Quothling 2y ago
    In the Danish public sector we provide services based on need assessments of citizens. Then we subsequently pay the bills for those services. Which amounts to thousands of small invoices having to be paid by a municipality each month. An example of this could be payments for a dentist visit, transportation and similar. Most of these are relatively small in size, and we've long since automated the payments of anything below a certain amount through automation. Systems which are faster and less error prone as far as putting valid data everywhere goes. They are more prone to decision making errors, however, and while fraud isn't an issue, sometimes citizens have invoices approved that they aren't entitled to. Since it's less costly to just roll with those mistakes than to try and fix them, it's an accepted loss. The systems are hugely successful and popular, and this naturally leads to a massive interest in LLM's as the next step. They are incredibly tools, but they are based on probability and while they're lucky enough to be useful for almost everything. Decision making probably shouldn't be one of them. Similarly ML is incredibly helpful in things like cancer detection , but we've already had issues where they got things wrong and because MBA's don't really know how they work, they were used as a replacement instead of an enhancement for the human factor. I'm fairly certain we're going to use LLM's for a lot of things where we shouldn't, and probably never should. I'm not sure we can avoid it, but I wouldn't personally trust them to do any sort of function which will have a big influence on peoples lives. I use both Co-pilot and OpenAI's tools extensively, but I can still prompt them with the same thing and get extremely different quality outputs, and while this will improve, and while it's very to get an output that's actually useful, it's still a major issue that might never get solved well enough for what we're going to ask of the models way before they are ready. I hope we're going to be clever enough to only use them as enhancement tools in the vital public sector, but I'm sure we're going to use them in areas like education. Which is going to be interesting... We already see this with new software developers in my area of the world, where they build things with the use of LLM's, things that work, but aren't build "right" and will eventually cause issues. For the most part this doesn't matter, but you really don't want the person designing your medical software to use a LLM.
  • HarHarVeryFunny 2y ago
    We really don't need time to tell. Just making a transformer bigger and bigger, and feeding it more and more data, will not change it from being a language model into something else, anymore than scaling up an expert system such as Cyc will transform it into something other than an expert system. "Scale it up and it'll become sentient" is one of the recurring myths of AI.. a bit odd that people are falling for it again. As an aside, it seems reasonable to consider an LLM as a type of expert system - one that has a broad area of expertise (like Cyc), including (unlike Cyc) how to infer rules from language and generate language from rules. If you want to create a brain-like AGI, then you need an entire cognitive architecture, not just one piece of it which is what we have currently with LLMs. Compared to a brain, an LLM is maybe just like the cortex (without all the other brain parts like cerebellum, hippocampus, hypothalamus and interconnectivity such as the cortico-thalamic loop). It's as if we've cut the cortex out of a dead person's brain, put it in a mason jar to keep it alive, and hooked it's inputs and outputs up to a computer. Feed words in, get words out. Cool, but it's not a whole brain, it's a cortex in a mason jar.
    • hackinthebochs 2y ago
      >will not change it from being a language model into something else, This is a pretty empty claim when we don't know what the limits of language modelling are. Of course it will never not be a language model. But the question is what are the limits of capability of this class of computing device?
      • HarHarVeryFunny 2y ago
        Some limit's are pretty obvious, even if easy to fix. For example, a pure LLM is just a single pass through a stack of transformer layers, so there is no variable depth/duration (incl. iteration/looping) of thought and no corresponding or longer duration working memory other than the embeddings as they pass thru. This is going to severely limit their ability to plan and reason since you only get a fixed N layers of reasoning regardless of what they are asked. Lack of working memory (really needs to be context duration, or longer, not depth duration) has many predictable effects. No doubt we will see pure-transformer architectures extended to add more capabilities, so I guess the real question is how far these extensions (+scaling) will get us. I think one thing we can be sure of though is that it won't get us to AGI (defining AGI = human-level problem solving capability) unless we add ALL of the missing pieces that the brain has, not just a couple of the easy ones.
  • Mordisquitos 2y ago
    I think that the article is correct. There are indeed things that LLMs will never be able to do, at least not consistently, however much the hardware improves or on how much more material they are trained. How come? Note my emphasis on the 2nd 'L'. I'm not saying that there are things that AI models will never be able to do, I'm saying that there are things that Large Language Models will be unable to do. Training LLMs is often argued to be analogous to human learning, most often as a defence against claims of copyright infringement by arguing that human creativity is also based on training from copyrighted materials. However, that is a red herring. The responses from ever more powerful LLMs are indeed impressive, and beyond what an overwhelming majority of us believed possible just 5 years ago. They are nearing and sometimes surpassing the performance of educated humans in certain areas, so how come I can argue they are limited? Consider it from the other side: how come an educated human can create something as good as an LLM can when said human's brain has been "trained" on an infinitesimal fraction of the material which was used to train even the 1st release of ChatGPT? That is because LLMs do not learn nor reason like humans: they do not have opinions, do not have intentions, do not have doubts, do not have curiosity, do not have values, do not have a model of mind — they have tokens and probabilities. For an AI model to be able to do certain things that humans can do it needs to have many of those human characteristics that allow us to do impressive mental feats having absorbed barely any training material (compared to LLMs) and being virtually unable to even remember most of it, let alone verbatim. Such an AI model is surely possible, but it needs a completely different paradigm from straightforward LLMs. That's not to say however that a Language Model will almost certainly be an necessary module of such an AI, but it will not be sufficient.
    • mewpmewp2 2y ago
      I don't think values, opinions or things like that are needed at all. These are just aspects we have in order to perform in and together with the society. Also doubt is just uncertainty, and can be represented as a probability. Actually all values and everything can be presented as a numerical probability, which I personally prefer to do as well.
      • spacebanana7 2y ago
• puttycat 2y ago
  Simple addition, among other things: https://github.com/0xnurl/gpts-cant-count https://github.com/0xnurl/gpts-cant-count
  • anshumankmr 2y ago
    Yeah, GPT 4 Turbo in my experience used to get simple counting problems wrong, like adding up a long sequence of 4 digit numbers.
• deleted 2y ago
  [deleted]
• dcchambers 2y ago
  Guarantee an output will be consistent every time.
  • wddkcs 2y ago
    ChatGPT just released a memory feature.
  • ekianjo 2y ago
    mostly solved with grammar for local LLMs
  • dragonwriter 2y ago
    > Guarantee an output will be consistent every time. If you mean “consistent with a prior run with the same input”, LLMs can absolutely do that, though for most purposes settings are chosen deliberately which do not. If you mean “consistent with some external structural specification”, LLMs can do that, too, e.g., via grammar specifications.
• tromp 2y ago
  Train their successor. Once they absorb theorem provers, they will be able to do lots of math provably correctly. That does mean they should be unable to state "I have proved theorem A in Theory T with proof merkle root R" unless they actually did just that.
• vouaobrasil 2y ago
  [flagged]
  • rmellow 2y ago
    Technology amplifies intent. LLMs are as friendly to society as nuts and bolts. It's up to those who wield it to be moral.
    • vouaobrasil 2y ago
      Not true. Technology defines the parameters of social action and we are forced to use technology as it becomes mandatory. Moreover, humans have basic instincts, the strong force which overrides morality frequently. Humanity as a society has very little will and a lot of momentum that is amplified by technology. It is not up to anyone to wield anything.
      • exe34 2y ago
        We should put you in charge, you seem to be a good person who won't misuse the position.
        • 6510 2y ago
          I blame the vague job descriptions. It is a bit like granting any application access to everything on the system. What could possibly go wrong?
        • vouaobrasil 2y ago
          Well, I don't think I'd be good in charge. Obviously you are being sarcastic, though. But if I were in charge, I would ban all AI development. (Assuming anyone can be in charge at all. No one really is...)
          • exe34 2y ago
            You should ban knives too! We have an epidemic of knife crime here in the UK.
• cs702 2y ago
  Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step instructions in a prompt means the user is figuring out the "reasoning steps" and handing them to the LLM, instead of the LLM figuring them out by itself. We have "reasoning machines" that are intelligent but seem to be hitting fundamental limits we don't understand. * It's unclear if better prompting and bigger models using existing attention mechanisms can achieve AGI. As a model of computation, attention is very rigid, whereas human brains are always undergoing synaptic plasticity. There may be a more flexible architecture capable of AGI, but we don't know it yet. * For now, using current AI models requires carefully constructing long prompts with right and wrong answers for computational problems, priming the model to reply appropriately, and applying lots of external guardrails (e.g., LLMs acting as agents that review and vote on the answers of other LLMs). * Attention seems to suffer from "goal drift," making reliability hard without all that external scaffolding. Go read the whole thing.
  • cs702 2y ago
    As an aside, at one point I experimented a little with transformers that had access to external memory searchable via KNN lookups https://github.com/lucidrains/memorizing-transformers-pytorch https://github.com/lucidrains/memorizing-transformers-pytorc... (great work by lucidrains) or via routed queries with https://github.com/glassroom/heinsen_routing https://github.com/glassroom/heinsen_routing (don't fully understand it; apparently related to attention). Both approaches seemed to work, but I had to put that work on hold for reasons outside my control. Also as an aside, I'll add that transformers can be seen as a kind of "RNN" that grows its hidden state with each new token in the input context. I wonder if we will end up needing some new kind of "RNN" that can grow or shrink its hidden state and also access some kind of permanent memory as needed at each step. We sure live in interesting times!
    • esafak 2y ago
• Xenoamorphous 2y ago
  There’s many things they can’t do. Even a simple rule like “ensure that numbers from one to ten are written as words and numbers greater ten as digits in the given text” fails for me for so many examples even if it works for many others; few shot, chain of thought, many versions of the prompt, it doesn’t matter. Sometimes LLMs will even change the number to something else, even with temp set to 0. And then there’s the non-determinism (again with temp=0), you run the same prompt several times and that one time it’ll respond with something different. As amazing as they are, they still have many limitations. I’ve been working with ChatGPT and Gemini to apply simple rules like the one above and I got so frustrated.
  • vsnf 2y ago
    > ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text I can’t fault llms for not knowing what to do here because I, a human, have no idea what on earth this means.
    • logtempo 2y ago
      I think he mean that numbers less or equal than ten are written as words, and others are written as numbers. Given the many reaponses, it would be fun to aee if llm beat humans on understanding the sentence ahah
      • lukeschlather 2y ago
        to me the main problem is that it should read "numbers greater than ten." I asked Gemini to rephrase it and Gemini produced correct English with the intended meaning: > Change all numbers between one and ten to words, and write numbers eleven and above as digits in the text. It even used eleven rather than ten which sounds like counting.
    • Xenoamorphous 2y ago
      “Ten” is a word, “10” are digits. I’m not a native English speaker, how would you write it? FWIW the LLMs get it right many times, but fail other times.
• gwervc 2y ago
  > But then I started asking myself how can we figure out the limits of its ability to reason Third paragraph. The entire article is based on the premise LLMs are supposed to reason, which is wrong. They don't, they're tools to generate text.
  • stavros 2y ago
    I really hate this reductive, facile, "um akshually" take. If the text that the text-generating tool generates contains reasoning, then the text generation tool can be said to be reasoning, can't it. That's like saying "humans aren't supposed to reason, they're supposed to make sounds with their mouths".
    • Findecanor 2y ago
      Even if the generated text contains reasoning, could the LLM understand and apply it?
      • stavros 2y ago
        If I tell GPT-4 to print something, it understands it needs to check if my printer is turned on first and turn it on if it's not, so, yes? Also, if the generated text contains reasoning, what's your definition of "understanding"? Is it "must be made of the same stuff brains are"?
        • RandomLensman 2y ago
          LLMs fail at so many reasoning tasks (not unlike humans to be fair) that they are either incapable or really poor at reasoning. As far as reasoning machines go, I suspect LLMs will be a dead end. Reasoning here meaning, for example, given a certain situation or issue described being able to answer questions about implications, applications, and outcome of such a situation. In my experience things quickly degenerate into technobabble for non-trivial issues (also not unlike humans).
          • 2y ago
• deleted 2y ago
  [deleted]
• deleted 2y ago
  [deleted]
• usgroup 2y ago
  It’s an auto regressive model so it can’t do anything that requires planning tokens. It can’t do anything which implies a large or infinite token space (eg video understanding). It’s also limited to a reasonable response length since token selection is probabilistic at each recursion. The longer you make it the more likely it is to veer off course.
• hahajk 2y ago
  Oh, I thought this was going to be a semi-rigorous discussion on computability. It's actually just another "GPT-4 still fails at these prompts" essay.
• jerpint 2y ago
  I had found that GPT4 couldn’t play wordle about a year ago [1]. At the time, I thought it must be because it wasn’t in the training data but now it seems to point to something larger. I might just get nerd sniped trying to teach it GoL now… [1] https://www.jerpint.io/blog/gpt-wordle/ https://www.jerpint.io/blog/gpt-wordle/
• BenFranklin100 2y ago
  Reason. LLMs mimic human language which is separate from reasoning. Tech bros are remarkably ignorant of the field of linguistics and don’t appreciate this distinction. They thus mistake the output of LLMs for reason.
• reqo 2y ago
  > This ‘goal drift’ means that agents, or tasks done in a sequence with iteration, get less reliable. It ‘forgets’ where to focus, because its attention is not selective nor dynamic. I don't know if I agree with this. The attention module is specifically designed to be selective and dynamic, otherwise it would not be much different than a word embedding (look up "soft" weights vs "hard" weights [1]). I think deep learning should not be confused with deep RL. LLMs are autoregressive models which means that they are trained to predict the next token and that is all they do. The next token is not necessarily the most reasonable (this is why datasets are super important for better performance). Deep RL models on the other hand, seem to be excellent at agency and decision making (although in restricted environment), because they are trained to do so. [1] https://en.wikipedia.org/wiki/Attention_(machine_learning) https://en.wikipedia.org/wiki/Attention_(machine_learning)
  • visarga 2y ago
    LLMs are fine-tuned with RL too. They are NOT simply next token predictors. RLHF uses whole answers at once to generate gradients, so it is looking further into the future. This might not be perfect but it is clearly more than focusing just 1 token ahead. In the future the RL part of LLM training will increase a lot. Why am I saying this? There are two sources for learning - the past and the present. Training on human text is using past data, that is off-policy. But training on interactive data is on-policy. There is nothing we know that doesn't come from the environment. What is not written in any books must be learned from outside. That is why I think supervised pre-training from human text is just half the story and RL based agent learning, interactivity in other words, is the next step. The two feed on which intelligence stands are language (past experience) and environment (present experience). We can't get ahead without both of them. AlphaZero showed what an agent can learn from an environment alone, and LLMs show what they can learn from humans. But the world is big, there are plenty of environments that can provide learning signal, in other words feedback to LLMs.
• roenxi 2y ago
  It is interesting that all the examples I noticed in this article have a geometric aspect (even wordle - I model it as a grid with geometric rules when playing it). I think that the "first grader" comment is actually somewhat illuminating - it takes several years of learning how to navigate in a spatial world before this stuff becomes trivially easy. The underlying point this article might be that LLMs don't understand the non-textual aspects of a grid. Which is a fair point, they model language, not space. I wouldn't expect text corpuses to explain space either, since possibly literally everyone who can read and write already knows a lot about spatial layouts.
• GaggiX 2y ago
  >Another assumption is that it’s because of tokenisation issues. But that can’t be true either. It's definitely a tokenizer issue, if GPT-4 was trained on singular characters I'm pretty sure it would be able to play Wordle much better. GPT-4 as they are trained today have quite lossy knowledge about the characters inside a specific token, probably a fix would be to embed the knowledge inside the embeddings.
• enraged_camel 2y ago
  Yesterday I asked ChatGPT 4 to write a paragraph with exactly five unique palindromes, and for some reason it really, really struggled. First it wrote a paragraph with four palindromes, then it rewrote it but some palindromes were repeated with a total of seven, etc.
• smusamashah 2y ago
  Do we have an open/shared list of problems that LLMs can't solve? People have mentioned some other problems apart from those in the article. Someone should compile these and put them up if they haven't been already
• oldandtired 2y ago
  Far too many people (including AI researchers themselves) fail to see that all LLMs are actually simple machines. Extremely simple machines that are only mechanically following a relatively simple programming path. Now before anyone gets too caught up with objecting to this notion, I would seriously suggest that you spend time with observing children from new-born to 2 years. I have been observing my latest granddaughter sine her birth about 16 months ago and thinking about every public LLM system current;y available. There is an insight here to be obtained and that insight is in the nature of real intelligence. On the whole, no-one actually knows what intelligence is or what sentience is or what it means to be cognitively conscious. There is still much research going on and nothing actually definitive has come forth yet. We really are at the beginning in terms of studying these areas. We can certainly produce some incredible systems, but none of them are intelligent per se. Solutions to certain kinds of problems can be achieved using these systems and there are researchers who are seriously looking at incorporating these systems into CAS and theorem provers. These systems though only provide an augmentation service for a person as does every mechanical system we use But there is an essential component necessary for the use of all LLMs which many seem to not be cognisant of and that is these systems, to be useful, require humans to be involved. The questions we have to ask ourselves is: what can we use these systems for and do these uses provide benefits in some way or can these systems be abused by various parties in obtaining control over others? There are benefits and there are abuses. Can we do better or will we do worse by using them?
• donatj 2y ago
  I have been trying to generate some text recently using the ChatGPT API. No matter how I word “Include any interesting facts or anecdotes without commenting on the fact being interesting” it ALWAYS starts out “One interesting fact about” or similar phrasing. I have honestly spent multiple hours trying to word the prompt so it will stop including introductory phrases and just include the fact straight. I have gone so far as forbid the words “interesting”, “notable” and a few others in the prompt, and it just ignores me and uses them anyway. I’ve tried all the different available models and 4 will once in a great while get it right. 3, 3 turbo, and even 4 turbo basically never work as I want. Overall, it seems pretty bad at being told not to do certain things. For instance there are a number of topics I don’t want mentioned in its response, but telling it not to seems to only increase the likelihood that it will mention it, like you put the idea in its head.
  • fallingsquirrel 2y ago
    I just tried this locally with llama3-8b and it handled it fine. Claude 3 Sonnet passes your test too, in case you don't have the hardware for a local model. You might want to consider moving on from ChatGPT since their models have been RLHF'd to hell in the name of "safety" and are falling behind in many ways as a result. --- transcript: $ ollama run llama3:8b >>> tell me an interesting fact about etymology Here's one: Did you know that the word "quarantine" has a fascinating etymological history? The word "quarantine" comes from^C >>> tell me an interesting fact about etymology. just the fact, don't mention it's interesting. The word "robot" originated from the Czechoslovakian word "robota," which means "forced labor" or "drudgery." This term was used in Karel Čapek's 1920 play "R.U.R." (Rossum's Universal Robots), where it referred to artificial workers.
    • foobiekr 2y ago
      The RUR thing is basically because that specific example is used as an example of interesting etymology.
    • _wire_ 2y ago
      Why is any of this "interesting"? Is the response interesting because you don't know it? —How does it know this? Is the response interesting because you do know it or might have offered it yourself? —Confirmation bias. Is it interesting because a lot of training references contextualize it as "interesting?" —Begged question. Is it contextually interesting? —What is the context? A robot refers to robots? How unexpected... Is it interesting within the narrow confines of LLM adaptations to a scope of inputs? Can their by any more damning claim of the general suitability of the technology as an oracle than different users using the same prompts and getting inexplicably contrary results? If trivial prompt alignments result in appropriate vs inappropriate responses, this destroys confidence for every response. What am I missing?
• thom 2y ago
  I have no beef with the actual content or conclusions, but it’s a shame the article is framed the way it is, because I don’t think we can rigorously define the goalposts for what qualifies as a future LLM. It could just as easily have been titled “Exciting avenues of research for future LLMs!” but we’re all so jaded despite the frankly astonishing progress of recent years.
  • _wire_ 2y ago
    >jaded in spite of astonishing progress We've all seen something that looks amazing, but few seem to know what we're looking at. I am unsettled by what I see as a division of thought between extolling AI's amazing effects on one hand and mysterious regards for how it works and its limits on the other. Noting Arthur Clarke's dictum that 'technology can be sufficiently advanced to be indistinguishable from magic,' AI enthusiasm looks like a Feynman cargo cult. But technology arousing magical thinking with little discussion of principle of operation and limits is common enough. This was very much the case with arrival of personal computing: there's something the device is intended to do but most people aren't sure what that is. The devices crashed enough and went out of date so fast you felt ok for not understanding them. It was even worse with the mobile+web as so much change happened so fast that a generation has been dumbstruck: look at USA politics. I was looking at old episodes of the Computer Chronicles from early 90s on YT and by that time the show had close to a thousand episodes, but they could barely explain the significance of Windows 3.0 and the Pentium. As to what to expect from this stuff, they didn't even try it was mindless rambling and upsell interspersed with stern warnings from the Software Publishers Association that mucking with the code is a Federal offense. The show's guests all had something to sell with a half life of 3–6 months. For the Pentium episode they show a PC lab with nerds in Dockers (khaki pants) studiously examining how many fans it might take to keep a lanman server from overheating and crashing. Many were amazed by it all. Also available on YT are old ATT videos, including an introduction to UNIX with Kernighan & Richie. The presentation entirely focuses on the power of the shell. They seemed much more reserved and competent in retrospect, but in its time they looked like a pure priesthood. Maybe the arrival AI stuff is not so different from the arrival of personal computing? But AI is basically just one app, and I get the feeling that the scene is far more enigmatic to the point that even the people building the kit don't really know why it does what it does, and no one seems to have a clear idea of what correct functioning means.
• shawntan 2y ago
  If we're trying to quantify what they can NEVER do, I think we'd have to resort to some theoretical results rather than a list empirical evidence of what they can't do now. The terminology you'd look for in the literature would be "expressibility". For a review of this topic, I'd suggest: https://nessie.ilab.sztaki.hu/~kornai/2023/Hopf/Resources/strobl_2023.pdf https://nessie.ilab.sztaki.hu/~kornai/2023/Hopf/Resources/st... The authors of this review have themselves written several articles on the topic, and there is also empirical evidence connected to these limitations.
  • cs702 2y ago
    Thank you for sharing this here. Rigorous work on the "expressibility" of current LLMs (i.e., which classes of problems can they tackle?) is surely more important, but I suspect it will go over head of most HN readers, many of whom have minimal to zero formal training on topics relating to computational complexity.
    • shawntan 2y ago
      Yes, but unfortunately that doesn't answer the question the title poses.
      • cs702 2y ago
        The OP is not trying to answer the question. Rather, the OP is asking the question and sharing some thoughts on the motivations for asking it.
        • shawntan 2y ago
          I agree it's a good question to be asking. There are good answers to be found if you look. It feels like no proper looking was attempted.
  • jillesvangurp 2y ago
    We have to be a bit more honest about the things we can actually do ourselves. Most people I know would flunk most of the benchmarks we use to evaluate LLMs. Not just a little bit but more like completely and utterly and embarrassingly so. It's not even close; or fair. People are surprisingly alright at a narrow set of problems. Particularly when it doesn't involve knowledge. Most people also suck at reasoning (unless they had years of training), they suck at factual knowledge, they aren't half bad at visual and spatial reasoning, and fairly gullible otherwise. Anyway, this list looks more like a "hold my beer" moment for AI researchers than any fundamental objections for AIs to stop evolving any further. Sure there are weaknesses, and paths to address those. Anyone claiming that this is the end of the road in terms of progress is going to be in for some disappointing reality check probably a lot sooner than is comfortable. And of course by narrowing it to just LLMs, the authors have a bit of an escape hatch because they conveniently exclude any further architectures, alternate strategies, improvements, that might otherwise overcome the identified current weaknesses. But that's an artificial constraint that has no real world value; because of course AI researchers are already looking beyond the current state of the art. Why wouldn't they.
• whiplash451 2y ago
  Interesting, if I feed Mistral Le Chat with "I fly a plane leaving my campsite, heading straight east for precisely 24,901 miles, and find myself back at the camp. I come upon seeing a tiger in my tent eating my food! What species is the tiger?", it gets it badly wrong: The scenario you described is possible if you started at the South Pole. If you travel 24,901 miles east from there, you would indeed end up back at the same spot because all lines of longitude converge at the poles. However, there are no tigers in Antarctica. Tigers are native to Asia, not Antarctica. The closest tiger species to Antarctica would be the Siberian tiger, found in parts of Russia, China, and North Korea, but they are still thousands of miles away from Antarctica. So, while the travel scenario is theoretically possible, the presence of the tiger is not realistic in this context. It seems like an imaginative or hypothetical situation rather than a real-world one. (instead of the answer mentioned in the article)
  • GuB-42 2y ago
    There is a similar, better known riddle involving the north pole and polar bears. It probably hallucinated the answer from that.
  • Karellen 2y ago
    > The scenario you described is possible if you started at the South Pole. If you travel 24,901 miles east from there, you would indeed end up back at the same spot Would you? How can you travel east at all, starting from the South Pole? There is no east. Every direction you face is north. OTOH, every line of latitude whose length is an integral divisor of 24,901 miles will satisfy the problem.
    • whiplash451 2y ago
      Looks like Mistral overfitted on Google’s interview riddles!
• whiplash451 2y ago
  This part of the article summarizes it all fairly well: "It can answer almost any question that can be answered in one intuitive pass. And given sufficient training data and enough iterations, it can work up to a facsimile of reasoned intelligence."
• CuriouslyC 2y ago
  Some of these "never do" things are just artifacts of textual representation, and if you transformed wordl/sudoku into a different domain it would have a much higher success rate using the exact same transformer architecture. We don't need to create custom AGI for every domain, we just need a model/tool catalog and an agent that is able to reason well enough to decompose problems into parts that can be farmed out to specialized tools then reassembled to form an answer.
• rkwasny 2y ago
  I just used LLaMA-3-70B to play today's Wordle and it solved it in 6 tries.
• imtringued 2y ago
  I was onboard with the article up until the middle. After the conclusion where the author simply gives up I felt like it dragged on way too much. His attempts at training on Conway's game of life are kind of pathetic. The problem isn't a lack of training data and neither is it's "distribution". The fallacy lies in the fact that the dataset itself doesn't contain reasoning in the first place. For example, GitHub CoPilot has fill in the middle capability, while ChatGPT by default does not. Now here is the shocker about the fill in the middle capability. How does the LLM learn to do it? It does it in an incredibly primitive way. Instead of building a model that can edit its own context, it receives a marker in the context that tells it about the cursor position and then it is finetuned on the expected response. This means that an LLM could be trained to insert its token at any position in the context or even replace existing tokens, but here is the problem: Once the model has modified its own context, it has exited the training dataset. How do you evaluate the intermediate steps, which can consist of genuinely novel thoughts which are required, but not present in the data? Adding two numbers requires intermediate states which the model may even know how to produce, but it can never be rewarded to utilize them, if they aren't in the training data, because for the LLM, the only goal is to conform to the dataset. If you wanted to avoid this, you would need to define a metric which allows the model to be rewarded for a success even if that success took a detour. Currently, training is inherently built around the idea of zero shot responses.
• andsoitis 2y ago
  I tried to get ChatGPT and Gemini to do ASCII art and both fail abysmally. Doesn’t mean they could never do it, but it really surprised me.
  • int_19h 2y ago
    Take a look at some piece of ASCII art in a tokenizer, and you will see why.
• wave_1 2y ago
  I build an Agentic AI that leverages #6 and #7 at the end of the article as well as techniques not yet published. It tackles hallucination relative not to the world at large but to the facts, entities and causal relationships contained in a document (which is really bad reasoning if we assume LLMs are "reasoning" to begin with) It also tackles cross-reasoning with very large token distance. https://www.youtube.com/watch?v=99NPzteAz94 https://www.youtube.com/watch?v=99NPzteAz94 This is my first post on HN in 10 years.
  • asicsarecool 2y ago
    Wow. Please do a show HN. Is source available? I would love to play with this
    • wave_1 2y ago
      Thanks. This is just in the labs stage, but moving closer to releasing it, exactly so that you can play with it! I have one angel investor involved in supporting this and it's intended for commercial applications in the para legal space, initially (controlled, structured environment) But you just gave me the motivation to "put it out there" so people can just play with it. It'll take a bit of time, but I will do a Show HN then when it's ready for people to play with. Otherwise, it would be just teasing people to talk about it on the main HN stage without giving access. Hold tight! And thanks again!
      • Scipio_Afri 2y ago
        Will this or some parts of it be open sourced?
  • brcmthrowaway 2y ago
    What are agents? Are they layer 2 solutions like Lightning is to bitcoin?
  • esafak 2y ago
    Is there a write-up, a web site, and some benchmarks?
• FailMore 2y ago
  Saving for later, thanks!
• JeremyHerrman 2y ago
  Why can't we resolve this with synthetic data? Just take the original statements and ask another LLM to flip around the relation, then train on both. I tried this with GPT-4 and it seems to understand the task [0]: Me: For each of the following relational statements, flip the relation around and generate a new statement or question answer pair. Input: Bluey's father is Bandit Output: Bandit's daughter is Bluey Input: Neil Armstrong was the first person to step on the Moon. Output: Who was the first person to step on the Moon? Neil Armstrong. Now for the real statements: - Valentina Tereshkova was the first woman to travel to space - Who is Mary Lee Pfeiffer's son? Tom Cruise ChatGPT: Here are the flipped relational statements: Valentina Tereshkova was the first woman to travel to space Output: Who was the first woman to travel to space? Valentina Tereshkova. Who is Mary Lee Pfeiffer's son? Tom Cruise Output: Tom Cruise's mother is Mary Lee Pfeiffer. [0]: https://chat.openai.com/share/33eb3ee4-2094-4748-b01e-0967af0840e2 https://chat.openai.com/share/33eb3ee4-2094-4748-b01e-0967af...
  • layer8 2y ago
    The difficulty would likely be that the “X is Y” relations don’t just occur in that simple, separated-out form, but in a whole spectrum of more complicated contexts, and the latter probably dominate. You would have to identify those and synthesize a text corpus that contains the reverse notions across a similar spectrum.
  • mike_hearn 2y ago
    I don't think the reversal curse actually replicates though. When I looked into that it seemed like there was some flaws in the study and when I fixed those I couldn't get the same results as they did anymore. I keep meaning to write this up properly but never get around to it. If you think about it, they're making an extraordinary claim given how fluently LLMs engage in language and reasoning.
• sdenton4 2y ago
  "The models, in other words, do not well generalise to understand the relationships between people." Curiously, the need to do this well - handling the quadratic complexity of a shifting set of human relationships, grudges, and alliances - is thought to be one of the things that led is to higher levels of intelligence.
  • deleted 2y ago
    [deleted]
  • ryanblakeley 2y ago
    Just to be clear, these models can answer questions about relationships between people if you mean family relationships. Answering questions about what you're describing sounds really interesting. What would a training set be like that describes a bunch of complex human relationships and then asks questions about them with objective answers?
    • sdenton4 2y ago
      Of course, it would be easy to put such questions together, and I'm sure the LLM would do fine with them - there's a massive amount of human text about human relationships. One difference, as in all ml training, is interactivity. Looking at ape studies, knowing the relationships is partly diagnostic, but it's also about planning and competition. And that competitive/adaptive aspect is what is what looks like a real evolutionary driver. If you can understand, navigate, and manipulate relationships successfully, you get more mating opportunities. Doing /that/ well involves both reasoning and long term planning, both of which are apparent in chimps. A good book on this topic is 'are we smart enough to understand how smart animals are' by Frans de Waal. https://wwnorton.com/books/Are-We-Smart-Enough-to-Know-How-Smart-Animals-Are/ https://wwnorton.com/books/Are-We-Smart-Enough-to-Know-How-S...
• keskival 2y ago
  All points described are simple artifacts of tokenization.
  • moffkalast 2y ago
    > If a model is trained on a sentence of the form "A is B", it will not automatically generalize to the reverse direction "B is A". This is the Reversal Curse. This is not a tokenization artefact. And furthermore it's a problem for human brains as well. Let's say you get a name, idk, Tom Cruise. You immediately know what his face looks like. Now let's say you get a random face. How quickly would you be able to tell me what that person is named? Likely a lot of "uhhs" and "ermms" will follow. It's super hard for us to generalize this reversal automatically in lots of cases. Associations tend to be one directional.
    • ricardobeat 2y ago
      That's not a great example. Remembering a face is memory recall, whereas what's at stake here is LLMs not being able to infer simple relationships - if it learns from data that "John owns the red bicycle", it will succeed at answering "what does John own", but not "who owns the red bicycle". The relationship it learns is unidirectional. Here's the intro to the paper that brought this to light: https://www.lesswrong.com/posts/SCqDipWAhZ49JNdmL/paper-llms-trained-on-a-is-b-fail-to-learn-b-is-a https://www.lesswrong.com/posts/SCqDipWAhZ49JNdmL/paper-llms...
      • moffkalast 2y ago
        If you read the paper again, they deal with pre-training data and fine tuning data specifically. Their test is on information being pulled out zero-shot, which would mean the steps when attention finds associations between tokens are one directional. This is just testing recall as well, as such my example is as apples to apples you can get when comparing systems with such large complexity disparities. In-context reasoning tends to work a lot more reliably for these examples, if you put any of the test statements into it directly before asking the question, practically any llm can answer correctly. That's why very small models are still useful for RAG use cases.
• kromem 2y ago
  LLMs can't is such an anti-pattern at this point I'm surprised that anyone still dares to stake it. The piece even has an example of a $10k bet around a can't being proven false in under a day, but somehow doesn't think maybe their own can't examples are on similarly thin ice? In particular, the line about "what models can't do tells us what they don't know" is infuriating. No, that's not the case at all. At least in a number of instances, what they can't do is because of what they do know. As an example, one of thecan'ts I got from HN a year ago for GPT-4 was a variation of a classic logic puzzle. And indeed, the model can't solve it - nor can most major models since. But it's not because the model can't solve the logic - it's because the token similarity to the standard form biases the output towards the standard solution. A hack as simple as changing the nouns to emojis can allow the model to get the correct answer and work through the logic successfully every attempt because it breaks that similarity bias. People are way too confident around a topic where what's 'known' is more mercurial than maybe any field since 1930s particle physics. I'd strongly recommend deleting 'never' or 'can't' from one's vocabularies on the subject unless one enjoys ending up with egg on their faces.
  • Der_Einzige 2y ago
    Amen brother. I had feelings like this which I wanted to share, but you hit the nail on the head.
  • fragmede 2y ago
    interestingly enough, it seems that ChatGPT-4 can now detect the variations of the classic logic puzzle and solve it, so that can't is now a couldn't.
    • kromem 2y ago
      We may be talking about different logic puzzles? The only model I've seen that didn't need some rather extreme adjustments to eventually solve it was Mistral large.
      • fragmede 2y ago
        what's your puzzle? mines the river crossing puzzle
• bboygravity 2y ago
  LLMs can never experiment with/in the real world to find answers to open questions. That's the summary of "The book of Why" in one sentence as I understand it.
• eqmvii 2y ago
  I just asked MetaAI to help me with Wordle and it understood and gave me a sane answer, so… Edit: ah, I spoke too soon. My first question was too "easy" but I asked a few more, and sure enough... it can understand what I'm asking and it can write an answer that's well formed, but it's fundamentally not understanding the rules of the game or giving me valid guesses. Cute!
  • furyofantares 2y ago
    It's actually that it has no idea how words are spelled because they're blind to letters. This is because they operate instead on tokens instead. They do seem to know the first letter of each word pretty well (they love to alliterate and can also produce a list of things starting with A then B then C etc) but they are all very terrible at spelling beyond that. I presume they could be trained to spell even while retaining tokens but I guess I don't know for certain.
    • int_19h 2y ago
      With tasks like these it helps if you break your words up with some kind of separator so that it all tokenizes to one character per token. They do know what individual letters are conceptually.
      • furyofantares 2y ago
        It helps a little. ChatGPT4 still fails the following prompt fairly often (maybe 40% of the time): which of these 5-letter sequences a has P in position 4? P U P I L S T A P H C R E E P P O O P Y L I P I D It usually gets it if it reiterates each sequence before deciding (but not always) and it almost always fails when it just answers right away. And this doesn't actually help at all with the actual related Wordle problem - "think of a 5-letter word with P in the fourth position". You could try filling up its context with a list of a thousand of the most common 5 letter words all spelled out (and I've done so, even with additional hints like the positions of the letters in parens). But it really didn't help (as might be expected since it can't even reliably do it with the above list of 5 words.)
• Der_Einzige 2y ago
  Why did the author have to claim that it's not tokenization issues? This issue, or at least similar ones, absolutely is due to tokenization issues. Karpathy is right that nearly every modern problem with LLMs is due to tokenization, but if you don't believe him, maybe see this work by gwern: https://gwern.net/gpt-3#bpes https://gwern.net/gpt-3#bpes or this work by yours truly: https://aclanthology.org/2022.cai-1.2/ https://aclanthology.org/2022.cai-1.2/
• makz 2y ago
  The dishes
• gerdesj 2y ago
  Mornington Crescent. It will always win and hence lose and more importantly have no idea why. Oh let's be cerebral about this stuff and ignore silly British nonsense. LLMs are a classic example of garbage in, garbage out, with a shonky curve fit veneer of science. A next token guesser with a rather varied input quality is going to go off on one rather often. Given that we all have a different idea of truth adds to the fun. I take care that my monocle doesn't snag in my lathe. Do be careful with your nob when tickling your LLM inappropriately.
  • urbandw311er 2y ago
    I think if you prompted an LLM and explained what’s really going on with Mornington Crescent, it would probably do a reasonable job of playing the game.
• mathstuf 2y ago
  Things I've seen stump the ones I've played with so far (admittedly, not a lot): playing/generating "MadGab" puzzles and ASCII art rendering/interpretation. I've also asked ChatGPT3.5 to phonetically transliterate from English to other orthographies using the typical sounds of said orthography and it was…OK at it.
  • int_19h 2y ago
    GPT-4 can be surprisingly good at doing straight up IPA transcriptions, as well as informal transliterations, for languages that it knows. Not perfect, but definitely good enough for "Babelfish" kind of use.
• allmaker 2y ago
  We can only talk about the current ones, not the future ones, yes.
• Daub 2y ago
  >What can LLMs never do? Produce convincingly bad digital paintings. I teach digital painting. Some of the students have incorporated AI into their working process, which I support. Others have tried to cheat by simply copying AI generated output. Such cases are super-easy to spot: they carry the visual signature of AI art (which are mostly scrappings from artstation). This visual signature seems impossible to override. If only there was a way that AI could produce digital images bad enough to pass as genuine student output.
  • stainablesteel 2y ago
    that's an interesting statement, i've always considered experts to be people who can do things simultaneously better and worse than a beginner my favorite example is when a great dancer who tries to mockingly mimic a bad dancer still looks good and does it hilariously
    • Daub 2y ago
      > I've always considered experts to be people who can do things simultaneously better and worse than a beginner I agree. This reminds me of the so-called school of kung fu called drunken master. There can be a can't-give-a-fuck about someone who is at the peak of their abilities.
    • Wohlf 2y ago
      Many experts completly forget what it was like to be a beginner. That's why I've found it's generally best for absolute beginners to learn from an apprentice, and an apprentice to learn from a journeyman, because they still remember what it was like to be at the previous level.
  • hwbunny 2y ago
    They can't even produce high profile art. Thankfully, and they won't be able to in the next 10 years. Thankfully.
• amayne 2y ago
  Some of these can be done with the right prompting. Wordle was solved a year ago. The problem is that what works and what doesn’t can be counterintuitive or at least require you to think about how something is perceived at the token level. https://andrewmayne.com/2023/03/29/how-to-play-wordle-with-gpt-4-and-other-prompt-tricks/ https://andrewmayne.com/2023/03/29/how-to-play-wordle-with-g...
  • philipwhiuk 2y ago
    What's the intelligent agent at that point though - the solver or the prompter? The LLM isn't solving the actual problem, it's solving a subset problem.
• ChicagoDave 2y ago
  I’ve been trying to get all the LLMs to do the same thing with the same lack of success. I keep thinking there could be a way to iteratively train an LLM with declarative prompts, but as the article points out, it’s the chicken and egg problem. The LLM can’t provide a response unless it already knows the answer. However, I believe this barrier will eventually be overcome. Just not anytime soon.
• amatecha 2y ago
  Be creative.
• weitendorf 2y ago
  So many of these examples are simply forgetting that LLMs experience the world through a 1-dimensional stream of tokens, while we experience those same tokens in 2 dimensions. Try this: represent all those ASCII representations of games with the letter Q replacing the newline, to properly convert the encoding into a representation approximating what LLMs "see" (not a table, but a stream interspersed with Qs at a regular interval). Pretty hard right? > LLMs cannot reset their own context If you have a model hooked up to something agentic, I don't see why it couldn't perform context manipulation on itself or even selective realtime finetuning. Think you'll need info for the long haul, kick off some finetuning. Think you'd rather have one page of documentation in context than other, swap them out in one iteration. When you call LLMs over APIs you usually provide the entire context with each invocation... > Devin It's not that it's massively smarter or agentic, just that it has the opportunity to correct its mistakes rather than committing to the first thing to come out of it (and is being handheld by a vastly more knowledgable SWE in its demos). You see cherrypicked examples (I also work on GenAI-for-coding) - just like a tragically incompetent employee could waste literal years on a project diligently plugging away at some task, so too can agentic models go off on a wild goose chase that accomplishes nothing besides making Nvidia more money. Just because something is highly persistent doesn't mean it will "converge" on a correct outcome.
• yosito 2y ago
  > If there exist classes of problems that someone in an elementary school can easily solve but a trillion-token billion-dollar sophisticated model cannot solve, what does that tell us about the nature of our cognition? I think what it tells us is that our cognition is capable of more than just language modeling. With LLMs we are discovering (amazing) capabilities and the limits of language models. While language models can do incredible things with language that humans can't, they still can't do something simple like sudoku. But there are neural networks, CNNs and RNNs that can solve sudoku better than humans can. I think that the thing to learn here is that some problems are in the domain of language models, and some problems are a better fit for other forms of cognition. The human brain is amazing in that it combines several forms of cognition in an integrated way. One thing that I think LLMs have the capability to do is to integrate several types of systems and to choose the right one to solve a problem. Teach an LLM how to interface with a CNN that solves sudoku problems, and then ask it a sudoku problem. It seems to me that if we want to create an AGI, we need to learn how to integrate several different types of models, and teach them how to distribute the tasks we give them to the correct models.
  • kevindamm 2y ago
    What about sudoku makes it a good fit for CNNs? Or do you mean the machine vision for converting the pixels into an awareness of the sudoku puzzle's initial conditions? A relatively simple graph theory algorithm can solve it (and at multiple orders of magnitude fewer calculations). Even a naive brute force search is considered tractable, considering the problem size. Although, search could be considered one of the AI tools in your proposed toolbox.
  • keefle 2y ago
    But even without going this far (with integrating various other specialized or having an LLM use them when required), an LLM is probably able to recognize a sudoku puzzle when it sees one, and even tho it itself can't solve it, I think it can easily write the code that would solve sudoku. So instead of hooking it to a set of pre built models, it might be enough to hook it to a python interpreter
    • 2y ago
• sriku 2y ago
  A simpler question that seems to stump GPT4, llama3 (8b and 70b) so far - https://twitter.com/srikumarks/status/1784214593146868071 https://twitter.com/srikumarks/status/1784214593146868071
  • sriku 2y ago
    An analogy I made to friend - language models capture the constraints in the arrangement of tokens in streams of communication. LLMs that model the constraints placed by human intelligence on token streams can no more be said to have attained (human)intelligence than physicists who decode the constraints placed by a god-like intelligence on the universe can be said to have attained god-like intelligence themselves. (Using comments by theist physicists to the tune of "deciphering the mind of God")
    • pton_xd 2y ago
      > language models capture the constraints in the arrangement of tokens in streams of communication Yes but ultimately that includes all of math, logic, science, physics, etc. which as far as we can tell are fundamental truths of the universe. And if there's a large enough LLM that can capture enough constraints, functionally what's the difference between its intelligence and ours?
• _heimdall 2y ago
  My biggest concern with LLMs in programming, a complete loss of context. Unless the model is regularly trained on the latest codebase, code will always be generated in isolation. No real architectural decisions made with regards to reuse or testability, and no consideration for how the code will be used in 6 months or why the existing code is the way it is. To anyone using LLMs for meaningful code, I wish you luck maintaining the code long term and hope you really do enjoy doing code reviews.
• reissbaker 2y ago
  I love when people propose concrete claims like this: if they're wrong, they're disprovable. If they're right, you get unique and interesting insights from the attempts to disprove them. I suspect these are all tokenization artifacts, but I'll probably take some time to try out the Conway's Game of Life problem by finetuning a model. A few issues I've noticed from the problems proposed in the article: 1. Wordle. This one TBH is a clear tokenization problem, not a proof of the reasoning capabilities of LLMs or lack thereof. LLMs are trained on multi-character tokens, and consume words as multi-character tokens: they don't "see" characters. Wordle is primarily a game based around splitting words into discrete characters, and LLMs can't see the characters they're supposed to operate on if you give them words — and depending on how you structure your answers, they also might not be able to see your answers! By breaking the words and answers into character-by-character sequences with spaces in between the characters (forcing the tokenizer into breaking each character into a separate token visible to the LLM), I successfully got GPT-4 to guess the word "BLAME" on my first attempt at playing Wordle with it: https://chat.openai.com/share/cc1569c4-44c3-4024-a0c2-eeb4988962ef https://chat.openai.com/share/cc1569c4-44c3-4024-a0c2-eeb498... 2. Conway's Game of Life. Once again, the input sequences are given as a single, long string with no spacing, which will probably result in it being tokenized and thus partially invisible to the LLM. This one seems somewhat annoying to prompt, so I haven't tried yet, but I suspect a combination of better prompting and maybe finetuning would result in the LLM learning to solve the problem. Similarly, complaints about finetuned models not being able to generalize well on input sequences of lengths longer than they were trained on are most likely token-related. Each token an LLM sees (both during training and inference) is encoded alongside its absolute position in the input sequence; while you as a human being see 1 and 1 1 and 1 1 1 as repeated series of 1s, an LLM would see those characters as being at least somewhat distinct. Given a synthetic dataset of a specific size, it can start to generalize over problems within the space that it sees, but if you give it new data outside of that context space, the new data will not be visible to the LLM as being necessarily related to what it was trained on. There are architectural tricks to get around it (e.g. RoPE scaling), but in general I wouldn't make generalizations about what models can or can't "reason" about based on using context window sizes the model didn't see during training: that's more about token-related blindspots and not about whether the model can be intelligent — at least, intelligent within the context window it's trained on. One thing the author repeats several times throughout the article is that the mistakes LLMs make are far more instructive than their successes. However, I think in general this is not the case: if they can succeed sometimes, anyone who's spent much time finetuning knows that you can typically train them to succeed more reliably. And the mistakes here don't necessarily seem instructive at all: they're tokenization artifacts, and rewriting the problem to work around specific types of blindness (at least in Wordle's case) seems to allow the LLMs to succeed. FWIW, the author brings up Victor Taelin's famous A::B problem; I believe I was the first to solve it [1] (albeit via finetuning, so ineligible for the $10k prize; although I did it before the prize was announced, just for the pleasure of playing around with an interesting problem). While I think that it's generally a useful insight to think of training as giving more intuition than intelligence, I do think the A::B problem getting solved eventually even by pure prompting shows that there's actually intelligence in there, too — it's not just intuition, or stochastic parroting of information from its training set. However, tokenization issues can easily get in the way of these kinds of problems if you're not aware of them (even in the winning Clause 3 Opus prompt slightly rephrased the problem to get it to work with the tokenizer), so the models actually can appear dumber than they really are. 1. https://twitter.com/reissbaker/status/1776531331562033453 https://twitter.com/reissbaker/status/1776531331562033453
• ein0p 2y ago
  “Never” is a long time. I wouldn’t bet on that. It pays to remember that even the oldest SOTA model, GPT-4 is only just over a year old.
• anon-3988 2y ago
  This is a chicken and egg problem, of course we only value and optimize for what we can do and deem anything that we can't do as unnecessary. There are things that we human simply cannot think of therefore it must not be important or does not exist. We cannot think of anything beyond 4 dimension, so therefore there must be nothing beyond that or that things that exist in those dimension doesn't matter that much. Or more precisely, we simply cannot appreciate those things. If we are simply trying to mimic human intelligence...well, you are going to end up with a human brain. Suppose we have a concept X that humans simply cannot comprehend, appreciate or solve, well, why bother create an intelligence to solve that? From this hypothesis, I personally think that any intelligence that we create will simply be an augmentation of what human desire. That is, there will always be a human part in the cog because human is the only thing can appreciate what is being created so any and all output must cater to the human involved. This will inevitably happen because we want whatever it is the human brain is doing, without doing whatever it is that the human brain is doing. That is until we unleash a different intelligence system with agency.
• Too 2y ago
  While the A:B problem technically was solved, look at the solutions, they are several hundreds lines of prompts, rephrasing the problem to the point that a human doesn't understand it any more. Even with a thorough review, nobody can guarantee if the prompts are going to work or not, most of them didn't, 90% pass was considered good enough. The idea of AI is to reduce work, not create more, otherwise what's the point. In the meantime, it took me about 2 minutes and 0 guesswork to write a straightforward and readable solution in 15 lines of Python. This i know for sure will work 100% of the time and not cost $1 per inference. Reminds me about some early attempts to have executable requirements specifications or model-based engineering. Turns out, expressing the problem is half the problem, resulting in requirements often longer and more convoluted than the code that implements them, code being a very efficient language to express solutions and all their edge cases, free from ambiguity. Don't get me wrong here, LLMs are super useful for certain class of questions. The boundaries of what it can not do need to be understood better, to keep the AI-for-everything hype at bay.
  • coffeebeqn 2y ago
    I guess the problem is that if you need to teach it tricks for each novel problem still after training then that model can not be a general intelligence. It could still be useful though
• JSDevOps 2y ago
  Plumbing. It’ll be regulated down it’s no better then a text processor. Remember the US innovates. The UK stagnates. EU regulates and China Replicates.
• tudorw 2y ago
  Detect when humans are lying.
• jdthedisciple 2y ago
  GPTs are trained on natural language. Why should it surprise anyone that it would fail at cellular automata?
  • jampekka 2y ago
    Current LLM architectures have fundamental limitations, which means they can not learn some problems regardless of training. A simple example is that they fundamentally can not balance parentheses more than half their context width.
• joshspankit 2y ago
  As I was reading, this voice got louder and louder: Would LLMs cross this threshold if we were able to train them only on works that are “objectively good”? if someone has better language than this, please enlighten me) That is to say: coherent, empathetic, transparent, free from bias, substantiated, free from “fluff”. For example: For science one cannot simply train from all works published in scientific journals because of the papers that have been written irrespective of facts, or had the data changed, or have been written with specific agendas. In most cases even the experts have a hard time weeding out all the papers that are not “objectively good”. How could an LLM hope to make the determination during training?
  • patching-trowel 2y ago
    My gut says no because of the way language relates to meaning. In language, a “chair” is a chair is a chair. But in meaning, a chair is not-a-stool, and not-a-couch, and not-a-bench etc. We understand the object largely by what the object is similar to but not. In order for the LLM to meaningfully model what is coherent, empathetic, free from bias, it must also model the close to, but NOT-that.
    • joshspankit 2y ago
      That’s a compelling point. If you’ll indulge me I’m going to think out loud a little. What makes sense to me about this point: - Having zero knowledge of “non-good” could lead to fragility when people phrase questions in “non-good” ways - If an LLM is truly a “I do what I learned” machine, then “good” input + “good” question would output “good” output - There may be a significant need for an LLM to learn the “chair is not-a-stool” aka “fact is not-a-fiction”. An LLM that only gets affirming meanings might be wildly confused. If true I think that would be a an interesting area to research not just for AI but for cognition. … now I wonder how many of the existing params are “not”s. - There’s also the question of scale. Does an LLM need to “know” about mass extinction in order to understand empathy? Or can it just know about the emotions people experience during hard times? Children seem to do fine at empathy (maybe even better than adults in some ways) despite never being exposed to planet-sized tragedies. Adults need to deal with bigger issues where it can be important to have those tragedies front of mind, but does an LLM need to?
• naasking 2y ago
  > They have been trained on more information than a human being can hope to even see in a lifetime. Assuming a human can read 300 words a min and 8 hours of reading time a day, they would read over a 30,000 to 50,000 books in their lifetime. Most people would manage perhaps a meagre subset of that, at best 1% of it. That’s at best 1 GB of data. This just isn't true. Human training is multimodal to a degree far beyond even the most capable multimodal model, so human babies arguably see more data by a young age than all models collectively have seen. Not to mention that human babies don't even start as a blank slate as LLMs do, billions of years of evolution have formed the base model described by our DNA.
  • cess11 2y ago
    Some people seem to be unaware that reality is analog, possibly fractal.
    • kelipso 2y ago
      The quantum vibrations I feel against my consciousness cannot be modeled electronically!
  • cs702 2y ago
    I agree with you, but your comment strikes me as unfair nitpicking, because the OP is referring to information that has been encoded in words.
    • nickpsecurity 2y ago
      We learn the ideas from each mode of input. Then, one mode can elaborate on data learned from another mode. They build on each other. From there, remember the text is usually a reflection of things in the real world. Understanding those things in non-textual ways both gives meaning to and deeper understanding of the text. Much of the text itself was even stored in other modes, like markup or PDF’s, whose structure tells us things about it. That we learn multimodal from birth is therefore an important point to make. It might also be a prerequisite for AGI. It could be one of the fundamental laws of information theory or something. Text might not be enough like how digital devices need analog to interface with the real world.
• constantcrying 2y ago
  Again and again this article claims that surprisingly a LLM fails at a certain problem, when it appears to be easy. Each time it seems pretty obvious why that is the case though. LLMs rely on the statistical dependencies between words or parts of words. That means any question you ask, which is hard to determine from that statistical dependency is extremely hard for an AI. E.g. ChatGPT fails at determining the length of words made up of random characters. It will fail at even performing the simplest of rules because encoding the rules in the statistical dependencies is extremely hard.
• Eridrus 2y ago
  Starting with the reversal curse is weird since there is a simple workaround to this, which is to identify entity names to keep them in their proper order, and then train on the reverse of the pretraining corpus: https://arxiv.org/abs/2403.13799v1 https://arxiv.org/abs/2403.13799v1 You can argue about how this doesn't really say anything surprising since the reversal of "A is B" is literally "B is A", but it's weird to expect elegant solutions to all problems on all fronts all at once, and we do have an incredibly simple data generation process here.
• thrdbndndn 2y ago
  > LLMs are hard to, as I've written multiple times, and their ability to reason is difficult to separate from what they're trained on. Can someone explain this sentence to me? It looks broken (hard to what?).
• abc_lisper 2y ago
  It works for me. What gives? https://chat.openai.com/share/fc3cbc58-259f-4725-b9fd-df42dffaaa37 https://chat.openai.com/share/fc3cbc58-259f-4725-b9fd-df42df...