7 ms·
I'm not a full time "prompt engineer" but it's a small role in the work that I do for my business. Prompt engineering is kinda like a basic form of data scienc
by binarymax 3y ago
I'm not a full time "prompt engineer" but it's a small role in the work that I do for my business.
Prompt engineering is kinda like a basic form of data science. You have a dataset and some manually labelled results, and you hypothesize on prompts and try to improve on some metric. You'd be surprised how the tiniest of changes will alter a result. Thinks like a hyphen, semicolon, or capitalization can sway the metric. It's very finicky and very annoying.
- msp26 3y agoTokenization errors are also really, really annoying to deal with. e.g. why does the JSON output have silly whitespace/quotation sometimes? Obviously it's because the first token the model output was `{` and not `{"` like it should be. Obviously.
- binarymax 3y agoTotally. When using GPT I exclusively use function calling, which is far more reliable for JSON output. When using other models I don't even bother with strict JSON because of the error rate - and instead opt for HTML which is more forgiving when parsing.
- msp26 3y agonono, the same thing happens with OpenAI's JSON mode and function calling. It does output parsable JSON very reliably but it comes out mangled with a bunch of whitespace sometimes. GPT-4-turbo's output context window is limited to 4096 so fixing this is relevant. You can use logit_bias for it.