6 ms·
It actually still scares the hell out of me that this is the way even the experts 'program' this technology, with all the ambiguities rising from the use of nat
by zevv 2y ago
It actually still scares the hell out of me that this is the way even the experts 'program' this technology, with all the ambiguities rising from the use of natural language.
- Terr_ 2y agoLLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole until you get what you want, enough of the time, temporarily.
- ljlolel 2y agosame as with asking humans to do something
- ImHereToVote 2y agoWhen we do prompt engineering for humans, we use the term Public Relations.
- manuelmoreale 2y agoThere’s also Social Engineering but I guess that’s a different thing :)
- TeMPOraL 2y agoNo, that's exactly the thing - it's prompt injection attacks on humans.
- Bluestein 2y ago... or - worse even - something you think is what you want, because you know not better, but happens to be a wholy (or - worse - even just subtly partially incorrect) confabulated answer.-
- brookst 2y agoAs a product manager this is largely my experience with developers.
- visarga 2y agoWe all use abstractions, and abstractions, good as they are to fight complexity, are also bad because sometimes they hide details we need to know. In other words, we don't genuinely understand anything. We're parrots of abstractions invented elsewhere and not fully grokked. In a company there is no single human who understands everything, it's a patchwork of partial understandings coupled functionally together. Even a medium sized git repo suffers from the same issue - nobody understands it fully.
- brookst 2y agoWholeheartedly agree. Which is why the most valuable people in a company are those who can cross abstraction layers, vertically or horizontally, and reduce information loss from boundaries between abstractions.
- Terr_ 2y agoWell, hopefully your developers are substantially more capable, able to clearly track the difference between your requests versus those of other stakeholders... And they don't get confused by overhearing their own voice repeating words from other people. :p
- criddell 2y agoIt probably shouldn't be called prompt engineering, even informally. The work of an engineer shouldn't require hope.
- sirspacey 2y agoThis is the fundamental change in the concept of programming From computer’s doing exactly what you state, with all the many challenges that creates To is probabilistically solving for your intent, with all the many challenges that creates Fair to say human beings probably need both to effectively communicate Will be interesting to see if the current GenAI + ML + prompt engineering + code is sufficient
- mplewis 2y agoNah man. This isn’t solving anything. This is praying to a machine god but it’s an autocomplete under the hood.
- Loughla 2y agoHonestly, this sort of programming (whether it's in quotes or not) will be unbelievably life changing when it works. I can absolutely put into words what I want, but I cannot program it because of all the variables. When a computer can build the code for me based on my description... Holy cow.
- VirusNewbie 2y agoif this doesn't work well with super high level languages, why would it work really well with LLMs?
- Loughla 2y agoI can have a conversation with LLM's. they can walk me through the troubleshooting without prior knowledge of programming languages. That seems like a massive advantage.
- deleted 2y ago
- spiderfarmer 2y agoIt still scares the hell out me that engineers think there’s a better alternative that covers all the use cases of a LLM. Look at how naive Siri’s engineers were, thinking they could scale that mess to a point where people all over the world would find it a helpful tool that improved the way they use a computer.
- spywaregorilla 2y agoDo you have any evidence to suggest the engineers believed that?
- pb7 2y ago13 years of engineering failure.
- ec109685 2y agoThe technology wasn’t there to be a general purpose assistant. Much closer to reality now and I have found finally Siri not to be totally terrible.
- cj 2y agoMy overall impression using Siri daily for many years (mainly for controlling smart lights, turning Tv on/off, setting timers/alarms), is that Siri is artificially dumbed down to never respond with an incorrect answer. When it says “please open iPhone to see the results” - half the time I think it’s capable of responding with something but Apple would rather it not. I’ve always seen Siri’s limitations as a business decision by Apple rather than a technical feat that couldn’t be solved. (Although maybe it’s something that couldn’t be solved to Apple’s standards)
- spiderfarmer 2y agoThe original founders realised the weakness of Siri and started a machine learning based assistent which they sold to Samsung. Apple could have taken the same route but didn't.
- miki123211 2y agoKeep in mind that this is not the only way the experts program this technology. There's plenty of fine-tuning and RLHF involved too, that's mostly how "model alignment" works for example. The system prompt exists merely as an extra precaution to reinforce the behaviors learned in RLHF, to explain some subtleties that would be otherwise hard to learn, and to fix little mistakes that remain after fine-tuning. You can verify that this is true by using the model through the API, where you can set a custom system prompt. Even if your prompt is very short, most behaviors still remain pretty similar. There's an interesting X thread from the researchers at Anthropic on why their prompt is the way it is at [1][2]. [1] https://twitter.com/AmandaAskell/status/1765207842993434880?lang=en https://twitter.com/AmandaAskell/status/1765207842993434880?... [2] and for those without an X account, https://nitter.poast.org/AmandaAskell/status/1765207842993434880?lang=en https://nitter.poast.org/AmandaAskell/status/176520784299343...
- MacsHeadroom 2y agoAnthropic/Claude does not use any RLHF.
- teqsun 2y agoIs that a claim they've made or has that been externally proven?
- cjbillington 2y agoWhat do they do instead? Given we're not talking to a base model.
- tqi 2y agoSupposedly they use "RLAIF", but honestly given that the first step is to "generate responses... using a helpful-only AI assistant" it kinda sounds like RLHF with more steps. https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback https://www.anthropic.com/research/constitutional-ai-harmles...