esafranchik

6 ms·

1. 1.▲
   by esafranchik 4mo ago

   Wouldn't NDCG/token results vary wildly depending on the agent's query and the number of returned items? e.g. agents often run `grep -m 5 "QUERY"` with different queries, instead of one big grep for all items.
2. 2.▲
   by esafranchik 4mo ago

   Two follow-ups: 1) How do you compare accuracy? by checking if the answer is in any of the returned grep/bm25/semble snippets? 2) How do you measure token use without the agent, prompt, and tools?
3. 3.▲
   by esafranchik 4mo ago

   Is the benchmark measuring one-shot retrieval accuracy, or Coding agent response accuracy?
4. 4.▲
   Show HN: Cush – curl your shell, an HTTP tunnel for AI agents(github.com)

   3 points by esafranchik 5mo ago|0 comments
5. 5.▲
   by esafranchik 5mo ago

   Does this work with any tool calls that make an HTTP request? e.g. calling `curl` directly vs writing a script to make the request, then calling it
6. 6.▲
   by esafranchik 5mo ago

   Have you noticed an relationship between recall and the number of files/memories?
7. 7.▲
   by esafranchik 7mo ago

   The new API2MCP
8. 8.▲
   by esafranchik 2y ago

   Hello HN! I built Cubyc to manage my ML research in grad school. It lets you store all your experiment metadata with cloud-based repo providers like GitHub, GitLab, and Bitbucket. Plus, you can directly use SQL to dive into your runs. I kep
9. 9.▲
   Show HN: I open-sourced the library I use to track ML experiments with GitHub(docs.cubyc.com)

   5 points by esafranchik 2y ago|2 comments