xyc

6 ms·

1.
by xyc 1y ago
I'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.
2.
by xyc 1y ago
For TypeScript you can refer to https://github.com/modelcontextprotocol/typescript-sdk/blob/... There isn't much documentation available right now but you can ask coding agent eg. Claude Code to generate
3.
by xyc 1y ago
Great to see more local AI tools supporting MCP! Recently I've also added MCP support to recurse.chat. When running locally (LLaMA.cpp and Ollama) it still needs to catch up in terms of tool calling capabilities (for example tool call
4.
by xyc 1y ago
It's a protocol that doesn't dictate how you are calling the tool. You can use in-memory transport without needing to spin up a server. Your tool can just be a function, but with the flexibility of serving to other clients.
5.
by xyc 1y ago
recurse.chat + M2 max Mac
6.
by xyc 1y ago
I recently discovered toolhive which is pretty handy too https://github.com/stacklok/toolhive
7.
by xyc 1y ago
If you are on a Mac, give https://recurse.chat/ a try. As simple as download the model and start chatting. Just added the new multimodal support in LLaMA.cpp.
8.
by xyc 1y ago
Actually this is a good way to find product ideas. I placed a query in Grok to find posts about what people want, similar to this. Then it performs multiple searches on X including embedding search, and suggested people want stuff like tama
9.
by xyc 1y ago
It seems that this is possibly not necessary, since LLaMA.cpp already integrates Jinja with CPP implementation (through minja)
10.
by xyc 2y ago
Check out https://recurse.chat/
11.
by xyc 2y ago
The fact that there's no alternative implementation of SQLite also seems to play a part in preventing standardization of WebSQL. https://www.w3.org/TR/webdatabase/ "The specification reached an impasse:
12.
by xyc 2y ago
If anyone on macOS wants to use llama.cpp with ease, check out https://recurse.chat/ . Supports importing ChatGPT history & continue chats offline using llama.cpp. Built this so I can use local AI as a daily driver.
13.
by xyc 2y ago
You can get a release binary from https://github.com/ggerganov/llama.cpp/releases too.
14.
by xyc 2y ago
Made tool use work! check out demo here: https://x.com/chxy/status/1861684254297727299
15.
by xyc 2y ago
sharing the messy code here just for funsies: https://gist.github.com/xyc/274394031b41ac7e8d7d3aa7f4f7bed9
16.
by xyc 2y ago
Just tried out the puppeteer server example if anyone is interested in seeing a demo: https://x.com/chxy/status/1861302909402861905 . (Todo: add tool use - prompt would be like "go to this website and screensh
17.
by xyc 2y ago
^ asked the question in the discussion: https://github.com/modelcontextprotocol/specification/discus...
18.
by xyc 2y ago
Thanks for the pointers! Will do. I've fired up https://github.com/modelcontextprotocol/inspector and the code looks helpful too. I'm looking at integrating MCP with desktop app. The spec ( https://
19.
by xyc 2y ago
Superb work and super promising! I had wished for a protocol like this. Is there a recommended resource for building MCP client? From what I've seen it just mentions Claude desktop & co are clients. SDK readme seems to cover it a b
20.
by xyc 2y ago
With Claude you barely had to learn the language this days as you just need to prompt, but SQLite column is an interesting idea.
21.
by xyc 2y ago
If you are interested in no config setup for local LLM, give https://recurse.chat/ a try (I'm the dev). The app is designed to be self-contained and as simple as you can imagine.
22.
by xyc 2y ago
Shameless plug: If you are on a Mac, check out RecurseChat: https://recurse.chat/ A few outstanding features: - Fast: Import ChatGPT history, loads thousands of conversations at once. - Floating Chat: spotlight / ChatG
23.
by xyc 2y ago
Regarding chat history, I've been thinking that people should have ownership over chat history. We are migrating the chats towards SQLite, so your data is going to be a timeless format like files* - SQLite has long term support through
24.
by xyc 2y ago
Give https://recurse.chat/ a try - I'm the developer. One particular advantage over alternative apps is importing ChatGPT history and speed of the app, including full-text search. You can import your thousands of conv
25.
by xyc 2y ago
what's your experience with sqlite-vec? I'm considering using sqlite-vec in addition to/or replace qdrant vector db for a project (recurse.chat), since I'm moving all the data to sqlite. I love everything SQLite so far,
26.
by xyc 2y ago
Neat. I automatically upvote / like when one builds <not-a-game> with a game engine.
27.
by xyc 2y ago
UX of the website is honestly underrated, and I think if every website has an HN/Craigslist mode it'd be so awesome. I just wished for additional functionalities and a brief search seems to indicate people resort to external websi
28.
by xyc 2y ago
Most likely someone will make desktop/mobile clients for popular websites with minimalist aesthetics. Last time I tried a couple but didn't find it useful.
29.
by xyc 2y ago
The website is great, fast and slick. but sometimes I wished there were notifications when I needed it. Of course a mute everything option would be handy as well. Like for this post it'd be faster if I can click a button to see the res
30.
Ask HN: What Desktop HN client do you use?
5 points by xyc 2y ago|17 comments

More ›