5 ms·

I would love to have a shit load of small (27B dense. 35B MoE) agents running locally and looking at and ingesting every bit of data about me, my life and what

I would love to have a shit load of small (27B dense. 35B MoE) agents running locally and looking at and ingesting every bit of data about me, my life and what I get up to see what sort of correlations it finds. Give a coding agent access to a data lake of events and let it build up its own analytics tooling to extract and draw out information from that data, and present it to me as daily/weekly/monthly summaries.

Havoc 5mo ago
That's definitely doable. Planning similar except more webscraping / newsfeed / monitoring like. I've got 3x SBCs that can run the Gemma 4 26B MoE on NPU. Around 4W extra power, 3 tokens a second...so that can hammer away at tasks 24/7 without moving the needle on electricity bill
- 2001zhaozhao 5mo ago
  I wonder if some investment firms are already doing this internally at a large scale. (Probably.)
  - Havoc 5mo ago
    They are - I’ve seen it. They just use APIs though. There is very little interest within them to do the model engineering and inference in house.
2001zhaozhao 5mo ago
This was along my lines of thinking at one point as well. Though I'm now more interested in having it experiment autonomously on my software projects overnight.
deleted 5mo ago
[deleted]