8 ms·
Strong agree. For every time that I'd get a better answer if the LLM had a bit more context on me (that I didn't think to provide, but it 'knew') there seems to
by corry 11mo ago
Strong agree. For every time that I'd get a better answer if the LLM had a bit more context on me (that I didn't think to provide, but it 'knew') there seems to be a multiple of that where the 'memory' was either actually confounding or possibly confounding the best response.
I'm sure OpenAI and Antropic look at the data, and I'm sure it says that for new / unsophisticated users who don't know how to prompt, that this is a handy crutch (even if it's bad here and there) to make sure they get SOMETHING useable.
But for the HN crowd in particular, I think most of us have a feeling like making the blackbox even more black -- i.e. even more inscrutable in terms of how it operates and what inputs it's using -- isn't something to celebrate or want.
- mbesto 11mo ago> For every time that I'd get a better answer if the LLM had a bit more context on me If you already know what a good answer is why use a LLM? If the answer is "it'll just write the same thing quicker than I would have", then why not just use it as an autocomplete feature?
- Nition 11mo agoThat might be exactly how they're using it. A lot of my LLM use is really just having it write something I would have spent a long time typing out and making a few edits to it. Once I get into stuff I haven't worked out how to do yet, the LLM often doesn't really know either unless I can work it out myself and explain it first.
- cruffle_duffle 11mo agoThat rubber duck is a valid workflow. Keep iterating at how you want to explain something until the LLM can echo back (and expand upon) whatever the hell you are trying to get out of your head. Sometimes I’ll do five or six edits to a single prompt to get the LLM to echo back something that sounds right. That refinement really helps clarify my thinking. …it’s also dangerous if you aren’t careful because you are basically trying to get the model to agree with you and go along with whatever you are saying. Gotta be careful to not let the model jerk you off too hard!
- Nition 11mo agoYes, I have had times where I realised after a while that my proposed approach would never actually work because of some overlooked high-level issue, but the LLM never spots that kind of thing and just happily keeps trying. Maybe that's a good thing - if it could think that well, what would I be contributing?
- svachalek 11mo agoYou don't need to know what the answer is ahead of time to recognize the difference between a good answer and a bad answer. Many times the answer comes back as a Python script and I'm like, oh I hate Python, rewrite that. So it's useful to have a permanent prompt that tells it things like that. But myself as well, that prompt is very short. I don't keep a large stable of reusable prompts because I agree, every unnecessary word is a distraction that does more harm than good.
- brookst 11mo agoBecause it's convenient not having to start every question from first principles. Why should I have to mention the city I live in when asking for a restaurant recommendation? Yes, I know a good answer is one that's in my city, and a bad answer is on one another continent.
- fluidcruft 11mo agoFor example when I'm learning a new library or technique, I often tell Claude that I'm new and learning about it and the responses tend to be very helpful to me. For example I am currently using that to learn Qt with custom OpenGL shaders and it helps a lot that Claude knows I'm not a genius about this
- cubefox 11mo agoAnecdotally, LLMs also get less intelligent when the context is filled up with a lot of irrelevant information.
- taejavu 11mo agoThis is well established at this point, it’s called “context rot”: https://research.trychroma.com/context-rot https://research.trychroma.com/context-rot
- cubefox 11mo agoYeah, though this paper doesn't test any standard LLM benchmarks like GPQA diamond, SimpleQA, AIME 25, LiveCodeBench v5, etc. So it remains hard to tell how much intelligence is lost when the context is filled with irrelevant information.
- awesome_dude 11mo agoIf I find that previous prompts are polluting the responses I tell Claude to "Forget everything so far" BUT I do like that Claude builds on previous discussions, more than once the built up context has allowed Claude to improve its responses (eg. [Actual response] "Because you have previously expressed a preference for SOLID and Hexagonal programming I would suggest that you do X" which was exactly what I wanted)
- logicallee 11mo agoit can't really "forget everything so far" just because you ask it to. everything so far would still be part of the context. you need a new chat with memory turned off if you want a fresh context.
- awesome_dude 11mo agoI mean I am telling you what has actually worked for me so far - and being a NLP the system (should) understand what that means... as should you...
- baq 11mo agoLLMs literally can’t forget. If it’s in the context window, it is known regardless of what you put in the context next. That said, if the ‘pretend forget’ you’re getting works for you, great. Just remember it’s fake.
- awesome_dude 11mo agoLike I said, the AI does exactly what I intend for it to do. Almost, as I said earler, like the AI has processed my request, realised that I am referring to the context of the earlier discussions, and moved on to the next prompt exactly how I have expected it to Given the two very VERY dumb responses, and multiple people down voting, I am reminded how thankful I am that AI is around now, because it understood what you clearly don't. I didn't expect it to delete the internet, the world, the universe, or anything, it didn't read my request as an instruction to do so... yet you and that other imbecile seem to think that that's what was meant... even after me saying it was doing as I wanted. /me shrugs - now fight me how your interpretation is the only right one... go on... (like you and that other person already are) One thing I am not going to miss is the toxic "We know better" responses from JUNIORS
- chaostheory 11mo agoBoth of you are missing a lot of use cases. Outside of HN, not everyone uses an LLM for programming. A lot of these people use it as a diary/journal that talks back or as a Walmart therapist.
- deleted 11mo ago[deleted]
- gordon_freeman 11mo agoWalmart therapist?
- sshine 11mo agoAs in cheap.
- chaostheory 11mo agoPeople use LLMs as their therapist because they’re either unwilling to see or unable to afford a human one. Based on anecdotal Reddit comments, some people have even mentioned that an LLM was more “compassionate” than a human therapist. Due to economics, being able to see a human therapist in person for more than 15 minutes at a time has now become a luxury. Imo this is dangerous, given the memory features that both Claude and ChatGPT have. Of course, most medical data is already online but at least there are medical privacy laws for some countries.
- SecretDreams 11mo agoThis is exactly why the two use cases need to be delineated.
- brookst 11mo agoI'm pretty deep in this stuff and I find memory super useful. For instance, I can ask "what windshield wipers should I buy" and Claude (and ChatGPT and others) will remember where I live, what winter's like, the make, model, and year of my car, and give me a part number. Sure, there's more control in re-typing those details every single time. But there is also value in not having to.
- brulard 11mo agoI would say these are two distinct use cases - one is the assistant that remembers my preferences. The other use case is the clean intelligent blackbox that knows nothing about previous sessions and I can manage the context in fine detail. Both are useful, but for very different problems.
- helloplanets 11mo agoI'd imagine 99% of ChatGPT users see the app as the former. And then the rest know how to turn the memory off manually. Either way, I think memory can be especially sneakily bad when trying to get creative outputs. If I have had multiple separate chats about a theme I'm exploring, I definitely don't want the model to have any sort of summary from those in context if I want a new angle on the whole thing. The opposite: I'd rather have 'random' topics only tangentially related, in order to add some sort of entropy in the outout.
- sheepscreek 11mo agoGood point. I almost wish for an anonymous mode with chat history.
- crucialfelix 11mo agoAll those moments will be lost in time, like tears in rain.
- philmont 11mo agoDo Androids Dream of Electric Sheep? Soon.
- tom_m 11mo agoNah, they don't look at the data. They just try random things and see what works. That's why there's now the whole skills thing. They are all just variations of ideas to manage context basically. LLMs are very simply text in and text out. Unless the providers begin to expand into other areas, there's only so much they can do other than simply focus on training better models. In fact, if they begin to slow down or stop training new models and put focus elsewhere, it could be a sign that they are plateauing with their models. They will reach that point some day after all.