6 ms·

Hi, great tool! I've made https://uithub.com https://uithub.com 2 months ago. Its speciality is the fact that seeing a repo's raw extract is a matter of changi

Hi, great tool! I've made https://uithub.com https://uithub.com 2 months ago. Its speciality is the fact that seeing a repo's raw extract is a matter of changing 'g' to 'u'. It also works for subdirectories, so if you just want the docs of Upstash QStash, for example, just go to https://uithub.com/upstash/docs/tree/main/qstash https://uithub.com/upstash/docs/tree/main/qstash Great to see this keeps being worthwhile!

Arcuru 2y ago
That looks awesome. You didn't mention it but uithub.com also has an API, I can definitely see myself using this for a new tool.
helsinki 2y ago
I wonder why nobody uses jsonl format to represent an entire codebase? It’s what I do and LLMs seems to prefer it. In fact, an LLM suggested this strategy to me. Uses less characters, too.
- addaon 2y ago
  Are you suggesting that there's a correlation between what input formats provide best performance for an LLM input, and what sequence of tokens the same LLM outputs when prompted about what input formats provide best performance? Why would that be?
  - TeMPOraL 2y ago
    Why wouldn't that be? We've had several generations of LLMs since ChatGPT took the world by storm; current models are very much aware of LLMs that came before them, as well as associated discussions on how to best use them.
  - wwoessi 2y ago
    I don't think it's much difference, but I've read that Markdown codeblocks (or YAML, or XML) is better for code than JSON, for example: https://aider.chat/2024/08/14/code-in-json.html https://aider.chat/2024/08/14/code-in-json.html I think it makes sense. YAML is shorter and easier to read, Markdown codeblocks have no added syntax between the lines compared to normal code. But JSON vs JSONL I can't come up with any big advantages for the LLM, it's mostly the same.
- wwoessi 2y ago
  You can use JSON using the accept parameter of the API. The url structure remains the same. It also supports YAML and I found that's easiest to read for LLMs. Previous example but in JSON: https://uithub.com/upstash/docs/tree/main/qstash?accept=application%2Fjson https://uithub.com/upstash/docs/tree/main/qstash?accept=appl... Is there any reason to prefer JSONL besides it being more efficient to edit? I'm happy to add it to my backlog if you think it has any advantages for LLMs

Arcuru 2y ago

That looks awesome. You didn't mention it but uithub.com also has an API, I can definitely see myself using this for a new tool.

helsinki 2y ago

I wonder why nobody uses jsonl format to represent an entire codebase? It’s what I do and LLMs seems to prefer it. In fact, an LLM suggested this strategy to me. Uses less characters, too.

addaon 2y ago
Are you suggesting that there's a correlation between what input formats provide best performance for an LLM input, and what sequence of tokens the same LLM outputs when prompted about what input formats provide best performance? Why would that be?
- TeMPOraL 2y ago
  Why wouldn't that be? We've had several generations of LLMs since ChatGPT took the world by storm; current models are very much aware of LLMs that came before them, as well as associated discussions on how to best use them.
- wwoessi 2y ago
  I don't think it's much difference, but I've read that Markdown codeblocks (or YAML, or XML) is better for code than JSON, for example: https://aider.chat/2024/08/14/code-in-json.html https://aider.chat/2024/08/14/code-in-json.html I think it makes sense. YAML is shorter and easier to read, Markdown codeblocks have no added syntax between the lines compared to normal code. But JSON vs JSONL I can't come up with any big advantages for the LLM, it's mostly the same.
wwoessi 2y ago
You can use JSON using the accept parameter of the API. The url structure remains the same. It also supports YAML and I found that's easiest to read for LLMs. Previous example but in JSON: https://uithub.com/upstash/docs/tree/main/qstash?accept=application%2Fjson https://uithub.com/upstash/docs/tree/main/qstash?accept=appl... Is there any reason to prefer JSONL besides it being more efficient to edit? I'm happy to add it to my backlog if you think it has any advantages for LLMs