wolfgangK

5 ms·

1.
by wolfgangK 27d ago
Isn't ChatGPT benchmaxxing, then ? Responding "hmm…" isn't actually responding and latency should time to first relevant phoneme.
2.
by wolfgangK 27d ago
I have zero interest in world knowledge for my LLMs but this got me wondering : are there RAGs for that kind of data ? How could a LLM like DeepSeek-v4-flash-vision-exp accurately answer you question with an indexed database of labeled land
3.
by wolfgangK 1mo ago
Indeed ! LLM are creative writers, not journalists. Relying on overfitting for factual accuracy in not tenable. I don't understand why grounded RAG with judges in not the norm.
4.
by wolfgangK 1mo ago
You understand that drones are used for precision strikes instead of indiscriminate bombing, right ? It's the opposite of "sponsoring a genocide" (if there actually ever was one happening…)
5.
by wolfgangK 1mo ago
Everything is food and air. There is no such thing a "human labor" without food and air to sustain it. I'm not sure what your point is.
6.
by wolfgangK 2mo ago
BTW, I don't understand why having the driver converting AC to DC inside the LED spots is the default. For a new house, it seems to make sense to me to have at least one external driver for a ceiling of spots, if not one per floor (not
7.
by wolfgangK 3mo ago
I presume this works (will work) also for JupyterLite that is based on Pyodide ? Would be great if it helped getting the latest OpenCV-python version [0] and it's dnn goodies being available on a zero-install client side Notebook ! [0]
8.
by wolfgangK 3mo ago
OpenCV being in the list of Pyodide modules [0] was the biggest boon for my online teaching experience because remotely dealing with install woes (corporate proxies & cie) was a show stopper for regular Python. I'm hoping that they
9.
by wolfgangK 6mo ago
Nice ! My most pressing request for VSS would be efficient binary vectors : is this on the table ?
10.
by wolfgangK 1y ago
Only those who don't care/know about prompt processing speed are buying Macs for LLM inference.
11.
by wolfgangK 1y ago
For LLM inference, I don't think the PCIe bandwidth matters much and a GPU could improve greatly the prompt processing speed.
12.
by wolfgangK 1y ago
Indeed, recent Flash Attention is a pain point for non CUDA.
13.
by wolfgangK 1y ago
The idea is presumably that you would "sell" at an artificially low price.
14.
by wolfgangK 1y ago
> The Soviets had the […]first woman,[…] That is quite the claim !
15.
by wolfgangK 1y ago
You forgot the "/s", or do you actually believe that it's capitalism's fault is a mother taking care of her children is "unpaid labor" ?
16.
by wolfgangK 1y ago
"is hard" ≠ "sucks"
17.
by wolfgangK 1y ago
Most interesting ! Would you mind sharing the prompt and the resulting CLAUDE.md file ? Thx !
18.
by wolfgangK 2y ago
IMO, it would be more interesting to have a 3-way comparison of price/performance between DeepSeek 671b running on : 1. M3 Ultra 512 2. AMD Epyc (which Gen ? AVX512 and DDR5 might make a difference in both performance and cost , Gen 4
19.
by wolfgangK 2y ago
DeepSeek is not a model.Which model did you use (v3 ? R1 ? a distillation ?) at which quantization ?
20.
by wolfgangK 2y ago
Nice ! Is it possible to connect to an in browser DB like WASM DuckDB https://duckdb.org/docs/api/wasm/overview.html or https://github.com/babycommando/entity-db ? That would be most use
21.
by wolfgangK 2y ago
It seems that this aims to refutes claims for inaction with facts about spending money. However, the high speed rail project or homelessness management seem to show that in California, $$$ spent doesn't always imply that the problem is
22.
by wolfgangK 2y ago
How do we know that this extension can be trusted ?
23.
by wolfgangK 2y ago
«Unfortunately, I have only seen 3 models, 3B or over, handle RAG.» I would love to know which are these 3 models, especially if they can perform grounded RAG. If you have models (and their grounded RAG prompt formats) to share, I'm ve
24.
by wolfgangK 2y ago
Just played a bit with it. Were you working with ASCII ? This example didn't work for you ? https://github.com/jfalcou/eve/blob/a141ba93048bb2916c2157a9...
25.
by wolfgangK 2y ago
Counterpoint : your message is not synthetic data and will contribute to lots of LLMs saying the same. Many such cases ? (It seems to me obvious that a fgrep would sanitize synthetic data obtained from competitors.)
26.
by wolfgangK 2y ago
DeepSeek v3 can run on CPU & RAM : https://www.reddit.com/r/LocalLLaMA/comments/1hqidbs/deepsee... Epyc Gen4 and 12 memory channels of DDR5 @4800 should give you 7 to 9 t/s.
27.
by wolfgangK 2y ago
I don't think that Python would be the right language for such low-level performance maxxing endeavor. I would have picked C++ but t was eye opening for me to see how rust enabled such low level optimization, so I'm grateful for t
28.
by wolfgangK 2y ago
Amazingly thorough ! I love how the author leaves no stone unturned. I had no idea you could do the kind of low level efficiency shaving in Rust. I wonder how a C++ implementation with https://github.com/jfalcou/eve wo
29.
by wolfgangK 2y ago
Most interesting ! Amazing job at optimizing various parts of the task. It seems that being an MoE with 'only' 37B active params per token would put it within the reach of CPU & RAM inference for the lucky hobbyist with an Epy
30.
by wolfgangK 2y ago
For training, doesn't checkpoint saving make high reliability a moot point ? Why pay for 99.99999? uptime when you can restart your training from last/best model ?

More ›