4 ms·

> the model has to re-run the generation, and the client has to start receiving tokens from scratch again. I don't understand. The payload can be designed to h

> the model has to re-run the generation, and the client has to start receiving tokens from scratch again. I don't understand. The payload can be designed to have sequence number. In case of reconnect, send the last known sequence number. Sounds like a application level protocol problem and not transport. Am I missing something? The pub/sub mentioned in the article essentially does the same thing.

medbrane 9mo ago
Indeed, the reconnect behavior is described in the protocol and the server will simply resume from the requested sequence id.
petcat 9mo ago
The blog author is confusing SSE the protocol itself, with how the application is typically implemented. SSE is great and can trivially be implemented in a way that allows history, catch-up, and resuming. The "Pub/Sub" mentioned at the end of the exact application of SSE that the author wants.
- verdverm 9mo ago
  Exactly, I use ADK (https://google.github.io/adk-docs/runtime/ https://google.github.io/adk-docs/runtime/) and it handles all that bookkeeping for me. If my client disconnects, the engine continues to run until the turn is up, recording all events. We need more frameworks like ADK that handle the bookkeeping and give us great abstractions for building agentic systems.
  - embedding-shape 9mo ago
    If you don't have a proper grasp of what is the transport, what the is the protocol and what is your application protocol, I think chucking in libraries to try to help often makes things too complicated. You still would need to understand the differences and nuances.
    - verdverm 9mo ago
      yup, I have recently started saying "building blocks over batteries included" particularly as it comes to people trying to sell me on their Agent "framework", which amounts to little more than some well built tools and prompts, but pigeon holes me into how they think about solving certain issues in the agentic space, based on how things work today. If I go out 2 years, do I have to wait for the "framework" to realize their ideas are now out-of-touch and wait for them to course correct, or have I selected a framework that allows me to easily experiment, evaluate, and adjust any technique, with an ecosystem of building blocks for both the provider and user side of what I am building
- embedding-shape 9mo ago
  > The blog author is confusing SSE the protocol itself, with how the application is typically implemented Yeah, pretty common misunderstand among us self-taught developers who at one point never came across things like the OSI model (https://en.wikipedia.org/wiki/OSI_model https://en.wikipedia.org/wiki/OSI_model) or similar before, that we confuse what layer things actually happens at.

medbrane 9mo ago

Indeed, the reconnect behavior is described in the protocol and the server will simply resume from the requested sequence id.

petcat 9mo ago

The blog author is confusing SSE the protocol itself, with how the application is typically implemented. SSE is great and can trivially be implemented in a way that allows history, catch-up, and resuming. The "Pub/Sub" mentioned at the end of the exact application of SSE that the author wants.

verdverm 9mo ago
Exactly, I use ADK (https://google.github.io/adk-docs/runtime/ https://google.github.io/adk-docs/runtime/) and it handles all that bookkeeping for me. If my client disconnects, the engine continues to run until the turn is up, recording all events. We need more frameworks like ADK that handle the bookkeeping and give us great abstractions for building agentic systems.
- embedding-shape 9mo ago
  If you don't have a proper grasp of what is the transport, what the is the protocol and what is your application protocol, I think chucking in libraries to try to help often makes things too complicated. You still would need to understand the differences and nuances.
  - verdverm 9mo ago
    yup, I have recently started saying "building blocks over batteries included" particularly as it comes to people trying to sell me on their Agent "framework", which amounts to little more than some well built tools and prompts, but pigeon holes me into how they think about solving certain issues in the agentic space, based on how things work today. If I go out 2 years, do I have to wait for the "framework" to realize their ideas are now out-of-touch and wait for them to course correct, or have I selected a framework that allows me to easily experiment, evaluate, and adjust any technique, with an ecosystem of building blocks for both the provider and user side of what I am building
embedding-shape 9mo ago
> The blog author is confusing SSE the protocol itself, with how the application is typically implemented Yeah, pretty common misunderstand among us self-taught developers who at one point never came across things like the OSI model (https://en.wikipedia.org/wiki/OSI_model https://en.wikipedia.org/wiki/OSI_model) or similar before, that we confuse what layer things actually happens at.