6 ms·

I'm currently working on an open-source engine [1] exactly for this purpose. If anyone wants to try or has any suggestions, I'm happy to listen :) 1. https://g

I'm currently working on an open-source engine [1] exactly for this purpose. If anyone wants to try or has any suggestions, I'm happy to listen :) 1. https://github.com/stfurkan/bitgpu https://github.com/stfurkan/bitgpu 2. https://aidekin.com https://aidekin.com --> this is one of my projects that's currently using bitgpu engine

alex7o 2mo ago
Hay man, I really want a bonsai style embeddings or re-ranking model for you think sth like that is feasible?
- stfurkan 2mo ago
  Hi :) There are some smaller embedding and re-ranking models (I am currently using some of them on my aidekin project) but having it like bonsai models will be great to have.
mgoetzke 2mo ago
Interesting. But trying the demo with 27G model and simple 8+5 did not work. Guessing there is some work to do still ``` calculate({"expression":"g"}) → error: only numbers, + - * / and parentheses are allowed error: No user query found in messages. What is 8+5? Use the calculator get_time({}) → Mon Jul 20 2026 10:36:28 GMT+0200 (Central European Summer Time) error: No user query found in messages. hi Hello! 2 tokens · 6.2 tok/s · prefill 4563 ms · confidence 87% which tools do you know about ? I'myepic know about the following are two tools I am aware of course, I currently i know about I know about me know I am aware of course I know I know I am aware of the following calculate({"expression":" I know I am aware of know I am aware of I am aware I know I know know "}) → error: only numbers, + - * / and parentheses are allowed error: No user query found in messages.```
- stfurkan 2mo ago
  Thank you for trying it :) I just added the 27B support 1 hour ago. Let me check and try to fix it. All other models should work as expected but if you have issues with them too, I can take a look.
  - stfurkan 2mo ago
    Hi again :) I updated the engine and 27B model should work in the demo[1]. We can still consider it as experimental but I am trying to make it better and efficient. 1. https://stfurkan.github.io/bitgpu/examples/chat.html https://stfurkan.github.io/bitgpu/examples/chat.html
luca-3 2mo ago
This is great for high-privacy use cases. If you could add an agentic loop that can call tools within the website using the user’s session, that would be great. Then the user can chat with an AI about sensitive information in the most secure way possible. Also, do you have data on the technical requirements for this? If someone uses an old phone, e.g., does it still work?
- stfurkan 2mo ago
  Hi, the engine itself already has tool calling capabilities but if you're asking for the aidekin I didn't add tool calling there on purpose. It needs webgpu support but I haven't had a chance to test with different devices.
EGreg 2mo ago
Yes, I’m very interested in exploring this stuff. Want to connect and discuss it? https://calendly.com/safebotsai https://calendly.com/safebotsai
- stfurkan 2mo ago
  Hello, I sent you a LinkedIn connection request. We can chat first and schedule a meeting. Thanks
espetro 2mo ago
Cool stuff! I see you're targeting two verticals here (runtime on top of ONNX + TS library), I'm curious did you consider simply creating a AI SDK Provider (https://ai-sdk.dev/providers/community-providers/custom-providers https://ai-sdk.dev/providers/community-providers/custom-prov...) instead of crafting a separate library? Besides it, how are you dealing with the harness the model can use on web? I'm currently building https://github.com/bolojs/bolo https://github.com/bolojs/bolo to extend the base harness agents can use on web and I'm interested in hearing from your experience on crafting tools for bitgpu
- stfurkan 2mo ago
  Thank you :) I wanted to create it from zero myself so I can control it better. e.g. I started with ONNX support but now it support gguf models too. It might be a good idea to add some popular connectors in the future. Good luck with your project :)

alex7o 2mo ago

Hay man, I really want a bonsai style embeddings or re-ranking model for you think sth like that is feasible?

stfurkan 2mo ago
Hi :) There are some smaller embedding and re-ranking models (I am currently using some of them on my aidekin project) but having it like bonsai models will be great to have.

mgoetzke 2mo ago

Interesting. But trying the demo with 27G model and simple 8+5 did not work. Guessing there is some work to do still ``` calculate({"expression":"g"}) → error: only numbers, + - * / and parentheses are allowed error: No user query found in messages. What is 8+5? Use the calculator get_time({}) → Mon Jul 20 2026 10:36:28 GMT+0200 (Central European Summer Time) error: No user query found in messages. hi Hello! 2 tokens · 6.2 tok/s · prefill 4563 ms · confidence 87% which tools do you know about ? I'myepic know about the following are two tools I am aware of course, I currently i know about I know about me know I am aware of course I know I know I am aware of the following calculate({"expression":" I know I am aware of know I am aware of I am aware I know I know know "}) → error: only numbers, + - * / and parentheses are allowed error: No user query found in messages.```

stfurkan 2mo ago
Thank you for trying it :) I just added the 27B support 1 hour ago. Let me check and try to fix it. All other models should work as expected but if you have issues with them too, I can take a look.
- stfurkan 2mo ago
  Hi again :) I updated the engine and 27B model should work in the demo[1]. We can still consider it as experimental but I am trying to make it better and efficient. 1. https://stfurkan.github.io/bitgpu/examples/chat.html https://stfurkan.github.io/bitgpu/examples/chat.html

luca-3 2mo ago

This is great for high-privacy use cases. If you could add an agentic loop that can call tools within the website using the user’s session, that would be great. Then the user can chat with an AI about sensitive information in the most secure way possible. Also, do you have data on the technical requirements for this? If someone uses an old phone, e.g., does it still work?

stfurkan 2mo ago
Hi, the engine itself already has tool calling capabilities but if you're asking for the aidekin I didn't add tool calling there on purpose. It needs webgpu support but I haven't had a chance to test with different devices.

EGreg 2mo ago

Yes, I’m very interested in exploring this stuff. Want to connect and discuss it? https://calendly.com/safebotsai https://calendly.com/safebotsai

stfurkan 2mo ago
Hello, I sent you a LinkedIn connection request. We can chat first and schedule a meeting. Thanks

espetro 2mo ago

Cool stuff! I see you're targeting two verticals here (runtime on top of ONNX + TS library), I'm curious did you consider simply creating a AI SDK Provider (https://ai-sdk.dev/providers/community-providers/custom-providers https://ai-sdk.dev/providers/community-providers/custom-prov...) instead of crafting a separate library? Besides it, how are you dealing with the harness the model can use on web? I'm currently building https://github.com/bolojs/bolo https://github.com/bolojs/bolo to extend the base harness agents can use on web and I'm interested in hearing from your experience on crafting tools for bitgpu

stfurkan 2mo ago
Thank you :) I wanted to create it from zero myself so I can control it better. e.g. I started with ONNX support but now it support gguf models too. It might be a good idea to add some popular connectors in the future. Good luck with your project :)