LuxBennu

6 ms·

1.
Show HN: CtxGov – see what instructions your AI agent inherits before it runs(github.com)
3 points by LuxBennu 3mo ago|0 comments
2.
Show HN: CtxGov – drop in AI memory files, get an influence-boundary report(ctxgov.github.io)
2 points by LuxBennu 3mo ago|0 comments
3.
Show HN: CtxGov – a local claim firewall for AI memory claims(ctxgov.github.io)
2 points by LuxBennu 3mo ago|0 comments
4.
Show HN: CtxVault – receipts for AI context, not another memory store(ctxvault.github.io)
1 points by LuxBennu 4mo ago|0 comments
5.
Show HN: CtxVault – local receipts for AI context handoffs(github.com)
3 points by LuxBennu 4mo ago|0 comments
6.
by LuxBennu 5mo ago
Chatgpt for Excel is still an office add-in running in the same sandbox though. strongpigeon described the exact bottleneck upthread, process boundary crossings, context.sync() roundtrips that take seconds on web. That's a platform lim
7.
Making prompts longer did not help. Making the task contract explicit did(signaldepth.ai)
1 points by LuxBennu 5mo ago|0 comments
8.
by LuxBennu 5mo ago
Yeah sorry that was unclear on my part. I chunk at the endpoint level, whisper itself obviously processes 30s windows. The memory/latency thing I was referring to is more about processing longer files end to end through the pipeline, n
9.
by LuxBennu 5mo ago
Oh nice, the pyannote coreml port is interesting. Last time I looked at pyannote it was pytorch only so getting it to run efficiently on apple silicon was kind of a pain. Does the coreml version handle diarization or just activity detection
10.
by LuxBennu 5mo ago
Ah that makes sense, quadratic scaling is brutal. So with 96gb i'd probably get somewhere around 4-5k total sequence length before hitting the wall, which is still pretty limiting for anything multimodal. Do you do any gradient checkpo
11.
by LuxBennu 5mo ago
I run whisper large-v3 on an m2 max 96gb and even with just inference the memory gets tight on longer audio, can only imagine what fine-tuning looks like. Does the 64gb vs 96gb make a meaningful difference for gemma 4 fine-tuning or does it
12.
by LuxBennu 5mo ago
Yeah that makes sense, chunking on silence would sidestep the latency issue pretty cleanly. I've been running it through a basic fastapi wrapper so it just takes whatever audio blob gets thrown at it, no chunking logic on the server si
13.
by LuxBennu 5mo ago
I've been running whisper large-v3 on an m2 max through a self-hosted endpoint and honestly the accuracy is good enough that i stopped bothering with cleanup models. The bigger annoyance for me was latency on longer chunks, like anythi
14.
by LuxBennu 6mo ago
that tracks with what i've noticed practically. shorter prompts feel basically the same between llama.cpp metal and what i'd expect from native mlx, but once context gets longer the overhead starts showing up. would be interesting
15.
by LuxBennu 6mo ago
Roughly 8-12 token/s on generation depending on context length. Prompt processing is faster obviously. Haven't benchmarked it super carefully though, just eyeballing the llama.cpp output.
16.
by LuxBennu 6mo ago
yeah fair point, it's definitely model dependent. i've had good results with qwen but tried it on a smaller mistral variant once and the output quality dropped noticeably even at q8 for both. the speed hit from mixed types hasn&#x
17.
by LuxBennu 6mo ago
good overview of the architecture side but worth mentioning there's another axis that stacks on top of all of this: you can quantize the kv cache itself at inference time. in llama.cpp you can run q8 for keys and q4 for values and it c
18.
by LuxBennu 6mo ago
Thanks! Turns out structural signals get you surprisingly far. An LLM catches more, but speed is the feature.
19.
by LuxBennu 6mo ago
I ran this on my own prompt history and three things surprised me. found 3 API keys buried in copy-pasted stack traces (`reprompt privacy`). 35% of my agent sessions had error loops -- the agent retrying the same failing approach 3+ times (
20.
Show HN: Reprompt – Analyze what you type into AI tools, not what they output(github.com)
3 points by LuxBennu 6mo ago|3 comments
21.
by LuxBennu 6mo ago
Already running qwen 70b 4-bit on m2 max 96gb through llama.cpp and it's pretty solid for day to day stuff. The mlx switch is interesting because ollama was basically shelling out to llama.cpp on mac before, so native mlx should mean b
22.
by LuxBennu 6mo ago
Sadly I have the issue on a new m5 air. I have a 60hz 4k work monitor and two high refresh 4k gaming displays. The 60hz pairs fine with either gaming monitor, but the two gaming ones together and one just doesn't get recognized. Spent
23.
by LuxBennu 6mo ago
This is true for prohibitions but claude.md works really well as positive documentation. I run custom mcp servers and documenting what each tool does and when to use it made claude pick the right ones way more reliably. Totally different ou
24.
by LuxBennu 6mo ago
yeah that's a good way to put it. the "felt good in the moment" framing is basically the whole problem. the reward model was trained on human preferences and humans preferred the agreeable answer, so now that's what you
25.
by LuxBennu 6mo ago
i tested this pretty extensively actually. built a pipeline that asks the same question rephrased across multiple turns and tracks how much the model shifts based on user tone. even when you tell it to be critical, the moment the user pushe
26.
by LuxBennu 6mo ago
this is exactly how i use it too. i have a few custom MCP servers running on a mac mini homelab, one for permission management, one for infra gateway stuff. the key thing i learned is keeping CLAUDE.md updated with what each MCP server actu
27.
by LuxBennu 6mo ago
OpenClaw adapter was straightforward since it uses the same JSON session format. For agent-generated prompts, I haven't specifically benchmarked agentic workflows yet. The repetition metric detects n-gram repetition within a single pro
28.
Show HN: Reprompt – Score your AI coding prompts with NLP papers(github.com)
13 points by LuxBennu 6mo ago|2 comments
29.
by LuxBennu 6mo ago
Interesting collection. The architecture differences show up in surprising ways when you actually look at prompt patterns across models. Longer context windows don't just let you write more, they change what kind of input structure wor
30.
by LuxBennu 6mo ago
Your code map compresses signal on the context side. Same principle applies on the prompt side: prompts that front-load specifics (file, error, expected behavior) resolve in 1-2 turns. Vague ones spiral into 5-6. 1M context doesn't cha

More ›