7 ms·

When you discuss caching, are you talking about caching the LLM response on your side (what I presume) or actual prompt caching (using the provider cache[0])? C

• imsh4yy 1y ago
  I think I need to make this a bit more clear. I was mostly referring to caching the tools (sub-agents) if they are a pure function. But that may be a bit too speicific for the sake of this post. i.e. you have a query that reads data that doesn't change often, so you can cache the result.
  • adastra22 1y ago
    It seems very doubtful to me that every query would be literally the same (e.g. same hash), if these are plain text descriptions of the subset task.
    • imsh4yy 1y ago
      I mean that depends on how you define the "input" for the tool. Some can be very deterministic like an enum, boolean, number, etc.
    • cma 1y ago
      The task can be something like summarize each source file. Many files might not change every time.