You might also find Semantic Streaming interesting. It's t he same concept but applied to llm token streaming. It's used in BAML (the ai framework). https://www

You might also find Semantic Streaming interesting. It's t he same concept but applied to llm token streaming. It's used in BAML (the ai framework). https://www.boundaryml.com/blog/semantic-streaming https://www.boundaryml.com/blog/semantic-streaming I'm one of the developers of BAML.