4 ms·

I wonder if "prompt engineering" will be replaced by improvements to LLMs within the next couple years and this is just a bad DX step gap along the journey

by eddsh1994 4y ago

dpflan 4y ago
It pretty much seems like it, as this is the infancy of LLM UX.
awb 4y ago
Communication is basically prompt engineering. You can say whatever you like and live with the results, or try to phrase your communication in a way that elicits the kind of response you want from the other party. We’ve engineered it away with computers where we learn a programming language or the rules of an operating system and the acceptable input is translated into a deterministic result. But natural language is much more creative and open ended.
13years 4y ago
Prompt engineering is really more like the art of exploring the emergent behavior of the LLMs. Unlike traditional software that has known features and capabilities. We build a model and then we literally don't know what it does until we explore it.
- saliagato 4y ago
  wow
simonw 4y ago
I wrote a response to this argument here: https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-engineering/ https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-e... Short version: I think prompt engineering will be an important skill for a very long time. Using these systems to the maximum of their (currently unknown) potential will always involve expert knowledge!
- eddsh1994 3y ago
  Thanks!
alphanullmeric 4y ago
Prompt “engineering” currently reeks of the A in STEAM
ravenstine 4y ago
As long as language is used, prompt engineering will always be a thing. The term "prompt engineering" is just a fancy way of describing being a good communicator.
- crooked-v 4y ago
  A good communicator, but also a good sophist. For example, I discovered pretty quick that I could twist the ChatGPT 4 guardrails into knots, even without "DAN" prompts, just by feeding it a lot of philosophical nonsense about semantics and meaning.
ahzhou 4y ago
Advances seem to make some of the hacks less necessary (such as CoT behaving better with newlines), but precisely specifying what you want will always be useful.

I wonder if "prompt engineering" will be replaced by improvements to LLMs within the next couple years and this is just a bad DX step gap along the journey

dpflan 4y ago

It pretty much seems like it, as this is the infancy of LLM UX.

awb 4y ago

Communication is basically prompt engineering. You can say whatever you like and live with the results, or try to phrase your communication in a way that elicits the kind of response you want from the other party. We’ve engineered it away with computers where we learn a programming language or the rules of an operating system and the acceptable input is translated into a deterministic result. But natural language is much more creative and open ended.

13years 4y ago

Prompt engineering is really more like the art of exploring the emergent behavior of the LLMs. Unlike traditional software that has known features and capabilities. We build a model and then we literally don't know what it does until we explore it.

saliagato 4y ago
wow

simonw 4y ago

I wrote a response to this argument here: https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-engineering/ https://simonwillison.net/2023/Feb/21/in-defense-of-prompt-e... Short version: I think prompt engineering will be an important skill for a very long time. Using these systems to the maximum of their (currently unknown) potential will always involve expert knowledge!

eddsh1994 3y ago
Thanks!

alphanullmeric 4y ago

Prompt “engineering” currently reeks of the A in STEAM

ravenstine 4y ago

As long as language is used, prompt engineering will always be a thing. The term "prompt engineering" is just a fancy way of describing being a good communicator.

crooked-v 4y ago
A good communicator, but also a good sophist. For example, I discovered pretty quick that I could twist the ChatGPT 4 guardrails into knots, even without "DAN" prompts, just by feeding it a lot of philosophical nonsense about semantics and meaning.

ahzhou 4y ago

Advances seem to make some of the hacks less necessary (such as CoT behaving better with newlines), but precisely specifying what you want will always be useful.