5 ms·
Not the original commenter but I’ve done this too with Pydantic AI (actually the library does it for you). See “Streaming Structured Output” here https://ai.pyd
by richin13 1y ago
Not the original commenter but I’ve done this too with Pydantic AI (actually the library does it for you). See “Streaming Structured Output” here https://ai.pydantic.dev/output/#streaming-structured-output https://ai.pydantic.dev/output/#streaming-structured-output
- tough 1y agoThanks yes! Im aware of structured outputs, llama.cpp has also great support with GBNF and several languages beyond json. I've been trying to create go/rust ones but its way harder than just json due to all the context/state they carry over