8 ms·
Perhaps the solutions(s) needs to be less focusing on output quality, and more on having a solid process for dealing with errors. Think undo, containers, git,
by emn13 1y ago
Perhaps the solutions(s) needs to be less focusing on output quality, and more on having a solid process for dealing with errors. Think undo, containers, git, CRDTs or whatever rather than zero tolerance for errors. That probably also means some kind of review for the irreversible bits of any process, and perhaps even process changes where possible to make common processes more reversible (which sounds like an extreme challenge in some cases).
I can't imagine we're anywhere even close to the kind of perfection required not to need something like this - if it's even possible. Humans use all kinds of review and audit processes precisely because perfection is rarely attainable, and that might be fundamental.
- deleted 1y ago[deleted]
- techpineapple 1y agoBut, assuming this is a general thing not just focused on say software development, can you make the tooling around creating this easier than defining the process itself? Everyone loosely speaking sees the value in test driven development, but often I think with complex processes, writing the test is harder than writing the process.
- RicoElectrico 1y agoI want to make a simple solution where data is parsed by a vision model and "engineer for the unhappy path" is my assumption from the get-go. Changing the prompt or swapping the model is cheap.
- herval 1y agovision models are also faulty, and some times all paths are unhappy paths, so there's really no viable solution. Most of the times, swapping the model completely randomizes the problem space (unless you measure every single corner case, it's impossible to tell if everything got better or if some things got worse...
- _bin_ 1y agoThe biggest issue I’ve seen is “context window poisoning”, for lack of a better term. If it screws something up it’s highly prone to repeating that mistake. It then makes a bad fix that propagates two more errors, the says, “Sure! Let me address that,” repeating to poorly fix those rather than the underlying issue (say, restructuring code to mitigate.) It is almost impossible to produce a useful result, far as I’ve seen, unless one eliminates that mistake from the context window.
- instakill 1y agoI really really wish that LLMs had an "eject" function - as in I could click on any message in a chat, and it would basically start a new clone chat with the current chat's thread history. There are so many times where I get to a point where the conversation is finally flowing in the way that I want and I would love to "fork" into several directions from that one specific part of the conversation. Instead I have to rely on a prompt that requests the LLM to compress the entire conversation into a non-prose format that attempts to be as semantically lossless as possible; this sadly never works as in ten did [sic].
- theblazehen 1y agoYou can use LibreChat which allows you to fork messages: https://www.librechat.ai/docs/features/fork https://www.librechat.ai/docs/features/fork
- tough 1y agoGoogle UI supports branching and delete someone recently made a blog post about how great it is
- dfilppi 1y ago[dead]
- ModernMech 1y ago> Perhaps the solutions(s) needs to be less focusing on output quality, and more on having a solid process for dealing with errors. Think undo, containers, git, CRDTs LLMs are supposed to save us from the toils of software engineering, but it looks like we're going to reinvent software engineering to make AI useful. Problem: Programming languages are too hard. Solution: AI! Problem: AI is not reliable, it's hard to specify problems precisely so that it understands what I mean unambiguously. Solution: Programming languages!
- Workaccount2 1y agoWith pretty much every new technology, society has bent towards the tech too. When smartphones first popped up, browsing the web on them was a pain. Now pretty much the whole web has phone versions that make it easier*. *I recognize the folly of stating this on HN.
- LtWorf 1y agoNo it's still a pain. There's apps that open links in their embedded browser where ads aren't blocked. So I need to copy the link and open them in my real browser.
- mdaniel 1y agoOr my other favorite trap: an embedded browser where I'm not authenticated. Great, now I have to roll the dice about pasting a password in your "trust me, bro" looking login page because I cannot see the URL and the autofill is all "nope"
- otabdeveloper4 1y ago> LLMs are supposed to save us from the toils of software engineering Well, cryptocurrency was supposed to save us from the inefficiences of the centralized banking system. There's a lesson to be learned here, but alas our sociiety's collective context window is less than five years.