5 ms·
using the term "engineering" for writing a prompt feels very unserious
by leshow 1y ago
using the term "engineering" for writing a prompt feels very unserious
- morkalork 1y agoFor real. Editing prompts bares no resemblance to engineering at all, there is no accuracy or precision. Say you have a benchmark to test against and you're trying to make an improvement. Will your change to the prompt make the benchmark go up? Down? Why? Can you predict? No, it is not a science at all. It's just throwing shit and examples at the wall in hopes and prayers.
- echelon 1y ago> Will your change to the prompt make the benchmark go up? Down? Why? Can you predict? No, it is not a science at all. Many prompt engineers do measure and quantitatively compare.
- morkalork 1y agoMe too but it's after the fact. I make a change then measure, if it doesn't I roll back. But it's as good as witch craft or alchemy. Will I get I get gold with this adjustment? Nope, still lead. Tries variation #243 next
- a2dam 1y agoThis is literally how the light bulb filament was discovered.
- MegaButts 1y agoAnd Tesla famously described Edison as an idiot for this very reason. Then Tesla revolutionized the way we use electricity while Edison was busy killing elephants.
- SrslyJosh 1y agoLots of things have been discovered by guess-and-check, but it's a shit method for problem solving. You wouldn't use guess-and-check unless A) you don't know enough about the problem to employ a better method or B) the problem space can be searched quickly enough that using a better method doesn't really matter.
- yowlingcat 1y agoI think that the only reason guess-and-check fails is when you don't know enough to actually check. Maybe this is why we disagreed on a previous comment. I think guess and check is generally the best way to solve problems, so long as your checks well-designed (which to be fair, does require understanding the problem) and you incorporate the results from the check back into further guesses, and you can afford to brute force it (which is statistically the most common problems big and small I've had to solve). In a lot of ways, there's nothing fancy about that, it's just the scientific method -- the whole point which is that you don't really need to "know" about the problem a priori to get to the truth, and that you can recompile all dependent knowledge from scratch through experimentation to get to it. In practice it feels really expensive but it's also where the best insights come from -- experimentally re-deriving common knowledge often exposes where its fractures and exceptions are in clear enough detail to translate it into novel discovery.
- yawnxyz 1y agoupdating benchmarks and evals is something closer to test engineering / qa engineering work though
- dwringer 1y agoIsn't this basically the same argument that comes up all the time about software engineering in general?
- leshow 1y agoI have a degree in software engineering and I'm still critical if its inclusion as an engineering discipline, just given the level of rigour that's applied to typical software development. When it comes to "prompt engineering", the argument is even less compelling. Its like saying typing in a search query is engineering.
- klntsky 1y agogoogling pre-LLMs was a required skill. Prompting is not just for search if you build LLM pipelines. Cost commonly can be easily optimized 2x if you know what you are doing.
- leshow 1y agosomething being a skill does not mean it is an engineering discipline. engineering is the application of science or math to solve problems. writing prompts doesn't fit that definition.
- SrslyJosh 1y agoI think a more fundamental aspect of engineering is that it involves well-defined processes with predictable results, and prompting doesn't check either box.
- ozim 1y agoBecause your imagination stopped at chat interface asking for funny cat pictures. There are prompts to be used with API an inside automated workflows and more to it.
- vunderba 1y agoI came across a pretty amusing analogy back when prompt "engineering" was all the rage a few years ago. > Calling someone a prompt engineer is like calling the guy who works at Subway an artist because his shirt says ‘Sandwich Artist.’ All jokes aside I wouldn't get to hung up on the title, the term engineer has long since been diluted to the point of meaninglessness. https://jobs.mysubwaycareer.eu/careers/sandwich-artist.htm https://jobs.mysubwaycareer.eu/careers/sandwich-artist.htm
- theanonymousone 1y agoWhy would I have a problem calling that guy a sandwich engineer? https://en.wikipedia.org/wiki/Audio_engineer https://en.wikipedia.org/wiki/Audio_engineer
- wiseowise 1y agoBecause you’ll hurt ops huge ego. God forbid you put the godly title of ENGINEER near something so trivial as sandwich.
- guappa 1y agoIt's cute that you think that being a sound engineer is something you can pick up in a few minutes, while it requires knowledge of acoustics, electronics, music theory and human perception.
- theanonymousone 1y agoIt's your understanding of what I wrote, not what I meant and far from it.
- kovac 1y agoIT is where words and their meanings come to die. I wonder if words ever needed to mean something :p
- theanonymousone 1y agoI understand your point, but don't we already have e.g. AWS engineers? Or I believe SAP/Tableau/.. engineers?
- guappa 1y agoWell in USA they have "sales engineers", which in my experience are people who have no clue how the thing they're supposed to sell works.
- ndriscoll 1y agoI went into software instead, but IIRC sales and QA engineers were common jobs I heard about for people in my actual accredited (optical) engineering program. A quick search suggests it is common for sales engineers to have engineering degrees? Is this specifically about software (where "software engineers" frequently don't have engineering degrees either)?
- guappa 1y agoIn my (software) organisation, sales engineers were not aware of the fact that after entering a command on a linux terminal you must press enter for it to work. They were also unaware of the fact that if you create a filename with spaces you must then escape/quote it for it to work. They requested this important information to be included in the user manual (the users being sysadmins at very large companies).
- Timwi 1y agoI've always wondered why so many instruction pages for software meticulously include all key presses like Enter. This explains a lot
- cbm-vic-20 1y agoIn the places I've worked, sales engineers are similar to consultants. They work with the sales team to produce a demo for a prospective customer. They need to have development chops to do any customizations, produce realistic sample data, and need to understand the architecture of the product to make a compelling demo. They also need to have the social skills to answer technical questions on-the-fly.
- guappa 1y ago
- liampulles 1y agoAbsolutely. It's not appropriate to describe developers in general either. That fight has been lost I think and that's all the more reason to push against this nonsense now.