om8

5 ms·

1.
by om8 1mo ago
Why int4? There are a lot of superior 4 bit formats like nf4 from bitsandbytes.
2.
by om8 2mo ago
> So how can I regain my ability to focus? Getting diagnosed with ADHD and taking meds did the trick for me. Not sure how relevant this is, since inability to focus was always an issue for me.
3.
by om8 2mo ago
This project needs webgpu -- I did it on cpu about a year ago. My demo uses 2 bit quantization to run llama3 models on any device with enough ram. https://galqiwi.github.io/aqlm-rs/
4.
by om8 3mo ago
> manufacturing companies making the flimsiest, cheapest, plastic crap to save 1/3 of a cent on every mop they produce. Designed to work for the least amount of time before needing replaced We live in a world with such companies, an
5.
by om8 3mo ago
Oh, yes, you are right about watts. My bad
6.
by om8 3mo ago
> volt-amperes are technically the same unit as watts volt-amperes are joules
7.
by om8 3mo ago
Cool way to think about GWh/year: 1 GWh/year = (10 ** 9) / 24 / 365.25 / (10 ** 6) MW = 0.11 MW 70 GWh/year = 8 MW 1755 GWh/year = 200 MW 252 GWh/year = 29 MW
8.
by om8 5mo ago
cargo/uv/go have lock files though
9.
by om8 5mo ago
https://docs.vllm.ai/en/v0.20.0/api/vllm/model_executor/laye... `vllm.model_executor.layers.quantization.turboquant` > The technique implemented here consists of the scalar case of the HIGGS quan
10.
by om8 6mo ago
These are very different media types with very different goals.
11.
by om8 7mo ago
Is there a way to disable it? Sometimes I value agent not having knowledge that it needs to cut corners
12.
by om8 9mo ago
https://steveklabnik.com/writing/does-unsafe-undermine-rusts...
13.
by om8 11mo ago
... Yet.
14.
by om8 11mo ago
FYI you can run just `uvx pdm`
15.
by om8 1y ago
Also got it, found this thread by googling "ycombiinator"
16.
by om8 1y ago
Also got it, found this thread by googling "ycombiinator"
17.
by om8 1y ago
Have a similar project. Also written in rust, runs in a browser using web assembly In-browser demo: https://galqiwi.github.io/aqlm-rs Source code: https://github.com/galqiwi/demo-aqlm-rs
18.
by om8 1y ago
https://web.archive.org/web/20250914222638/http://ewaste.fka...
19.
by om8 1y ago
Oh, ok. Thanks, I agree
20.
by om8 1y ago
Why? They acquired books, that’s what they do
21.
by om8 1y ago
Good rationalism includes empiricism though
22.
by om8 1y ago
llama.cpp is a mess and ollama is right to move on from it
23.
by om8 1y ago
Of course it is. GPT-5 is one of the most anticipated things in AI right now. To live up to the hype, it needs to be a reasoning model.
24.
by om8 1y ago
It’s unfortunate that llama.cpp’s code is a mess. It’s impossible to make any meaningful contributions to it.
25.
by om8 1y ago
Sure, but integrated graphics usually lacks vram for LLM inference.
26.
by om8 1y ago
To have a gpu inference, you need a gpu. I have a demo that runs 8B llama on any computer with 4 gigs of ram https://galqiwi.github.io/aqlm-rs/about.html
27.
by om8 1y ago
I'm currently in Russia trying to get a US visa for my CS PhD. Because I do CS, I got into a thing called administrative processing. For 95% of people, it takes days -- weeks, tops. Because of the colour of my passport, it has already
28.
by om8 1y ago
I believe he knows
29.
by om8 1y ago
What would you do if some of the deps started to have conflicts in them? Also, what are your plans for migration when you'll need to move from one os version to another? Implicit solutions like yours have lower cost of entrance, but la
30.
by om8 1y ago
I have a demo that runs llama3-{1,3,8}B in browser on cpu. It can be integrated with this thing in the future to be fully local https://galqiwi.github.io/aqlm-rs

More ›