5 ms·
I don’t like it. Structures responses really do not work well with LLMs at all. They are one of the biggest causes of issues with tool calling right now.
by youre-wrong3 1mo ago
I don’t like it. Structures responses really do not work well with LLMs at all. They are one of the biggest causes of issues with tool calling right now.
- sscaryterry 1mo agoThen you're doing it wrong :)
- maleldil 1mo agoAgree with sibling. If you're getting severely deteriorated results with structured output, you're probably doing something wrong. There's been some research on the impact of structured outputs on results distribution, and there are tradeoffs, but "do not work well at all" doesn't match the experience at large.
- JonChesterfield 1mo agoYou prefer having the harness execute any markdown that looks like it might be a tool call? I had a bad time getting that to work reliably whereas a grammar in the sampler gets it right every time.