6 ms·
The article says that LLMs don't summarize, only shorten, because... "A true summary, the kind a human makes, requires outside context and reference points. Sh
by sobiolite 1y ago
The article says that LLMs don't summarize, only shorten, because...
"A true summary, the kind a human makes, requires outside context and reference points. Shortening just reworks the information already in the text."
Then later says...
"LLMs operate in a similar way, trading what we would call intelligence for a vast memory of nearly everything humans have ever written. It’s nearly impossible to grasp how much context this gives them to play with"
So, they can't summarize, because they lack context... but they also have an almost ungraspably large amount of context?
- kayodelycaon 1y agoThey can’t summarize something that hasn’t been summarized before.
- naikrovek 1y agothey can, they just can't do it well. at no point does any LLM understand what it's doing.
- kayodelycaon 1y agoIt can produce something that looks like a summarization based on similarly matching texts. Depending how unique the text is determines how accurate the summarization is likely to be.
- kblissett 1y agoIf you think they can't do this task well I encourage you to try feeding ChatGPT some long documents outside of its training cutoff and examining the results. I expect you'll be surprised!
- timmg 1y agoAbout a year ago, I gave a film script to an LLM and asked for a summary. It was written by a friend and there was no chance it or its summary was in the training data. It did a really good -- surprisingly good -- job. That incident has been a reference point for me. Even if it is anecdotal.
- pc86 1y agoI'm not as cynical as others about LLMs but it's extremely unlikely that script had multiple truly novel things in it. Broken down into sufficient small pieces it's very likely every story element was present multiple times in the LLM's training data.
- Spivak 1y agoI'm not sure I understand the philosophical point being made here. The LLM has "watched" a lot of movies and so understands the important parts of the original script it's presented with. Are we not describing how human media literacy works?
- pc86 1y agoI'm not making a philosophical point. The earlier comment is "I updated a new script and it summarized it," I was simply saying the odds of that script actually being new is very slim. Even though obviously that script or summaries of it do not exist in their entirety in the training data, its individual elements almost certainly do. So it's not really a novel (pun unintended?) summarization.
- BobaFloutist 1y agoThe point is that if you made a point to write a completely novel script, with (content-wise, not semantically) 0 DNA in it from previous movie scripts, with an unambiguous but incoherent and unstructured plot, your average literate human would be able summarize what happened on the page, for all that they'd be annoyed and likely distressed by how unusual it was; but that an LLM would do a disproportionately bad job compared to how well they do at other things, which makes us reevaluate what they're actually doing and how they actually do it. It feels like they've mastered language, but it's looking more and more like they've actually mastered canon. Which is still impressive, but very different.
- deleted 1y ago[deleted]
- originalcopy 1y agoI'd like to see some examples of when it struggles to do summaries. There were no real examples in the text, besides one hypothetical which ChatGPT made up. I think LLMs do great summaries. I am not able to come up with anything where I could criticize it and say "any human would come up with a better summary". Are my tasks not "truly novel"? Well, then I am not able, as a human, to come up with anything novel either.
- tovej 1y agoYou can reconcile these points by considering what specific context is necessary. The author specifies "outside" context, and I would agree. The human context that's necessary for useful summaries is a model of semantic or "actual" relationships between concepts, while the LLM context is a model of a single kind of fuzzy relationship between concepts. In other words the LLM does not contain the knowledge of what the words represent.
- neerajsi 1y ago> In other words the LLM does not contain the knowledge of what the words represent. This is probably true for some words and concepts but not others. I think we find that llms make inhuman mistakes only because they don't have the embodied senses and inductive biases that are at the root of human language formation. If this hypothesis is correct, it suggests that we might be able to train a more complete machine intelligence by having them participate in a physics simulation as one part of the training. I.e have a multimodal ai play some kind of blockworld game. I bet if the ai is endowed with just sight and sound, it might be enough to capture many relevant relationships.
- btown 1y agoIt's an interesting philosophical question. Imagine an oracle that could judge/decide, with human levels of intelligence, how relevant a given memory or piece of information is to any given situation, and that could verbosely describe which way it's relevant (spatially, conditionally, etc.). Would such an oracle, sufficiently parallelized, be sufficient for AGI? If it could, then we could genuinely describe its output as "context," and phrase our problem as "there is still a gap in needed context, despite how much context there already is." And an LLM that simply "shortens" that context could reach a level of AGI, because the context preparation is doing the heavy lifting. The point I think the article is trying to make is that LLMs cannot add any information beyond the context they are given - they can only "shorten" that context. If the lived experience necessary for human-level judgment could be encoded into that context, though... that would be an entirely different ball game.
- entropicdrifter 1y agoI agree with the thrust of your argument. IMO we already have the technology for sufficient parallelization of smaller models with specific bits of context. The real issue is that models have weak/inconsistent/myopic judgement abilities, even with reasoning loops. For instance, if I ask Cursor to fix the code for a broken test and the fix is non-trivial, it will often diagnose the problem incorrectly almost instantly, hyper-focus on what it imagines the problem is without further confirmation, implement a "fix", get a different error message while breaking more tests than it "fixed" (if it changed the result for any tests), and then declare the problem solved simply because it moved the goalposts at the start by misdiagnosing the issue.
- ratelimitsteve 1y agoI think the differentiator here might not be the context it has, but the context it has the ability to use effectively in order to derive more information about a given request.
- jchw 1y agoI think the real takeaway is that LLMs are very good at tasks that closely resemble examples it has in its training. A lot of things written (code, movies/TV shows, etc.) are actually pretty repetitive and so you don't really need super intelligence to be able to summarize it and break it down, just good pattern matching. But, this can fall apart pretty wildly when you have something genuinely novel...
- gus_massa 1y agoHumans too. If I were too creative writing the midterm, most of my students would fail and everyone would be very unhappy.
- card_zero 1y agoThat's exams, not humanity.
- BobaFloutist 1y agoThat's because midterms are specifically supposed to assess how well you learned the material presented (or at least directed to), not your overall ability to reason. If you teach a general reasoning class, getting creative with the midterm is one thing, but if you're teaching someone how to solve differential equations, they're learning to the very edge of their ability in a given amount of time, and you present them with an example outside of what's been described, it kind of makes sense that they can't just already solve it. I mean, that's kind of the whole premise of education, that you can't just present someone with something completely outside of their experience and expect them to derive from first principles how it works.
- throwway120385 1y agoI would argue that on a math midterm it's entirely reasonable to show a problem they've never seen before and test whether they've made the connection between that problem and the problems they've seen before. We did that all the time in upper division Physics.
- usefulcat 1y agoI think "context" is being used in different ways here. > "It’s nearly impossible to grasp how much context this gives them to play with" Here, I think the author means something more like "all the material used to train the LLM". > "A true summary, the kind a human makes, requires outside context and reference points." In this case I think that "context" means something more like actual comprehension. The author's point is that an LLM could only write something like the referenced summary by shortening other summaries present in its training set.
- jjaksic 1y agoBut "shortening other summaries from its training set" is not all an LLM is capable off. It can easily shorten/summarize a text it had never seen before, in a way that makes sense. Sure, it won't always summarize it the same way a human would, but if you do a double blind test where you ask people whether a summary was written by AI, a vast majority wouldn't be able to tell the difference (again this is with a completely novel text).
- cainxinth 1y agoThis can be solved with prompting. You can say: “summarize this document, but don’t just recap, give me the big picture” or anything to that effect.