DrPhish

24 ms·

1.
by DrPhish 9d ago
Every speaker is also a microphone
2.
by DrPhish 25d ago
>Computer building on those days was much more unfriendly Anyone who heard “black-to-black, back-to-back” just a little too late would agree with you (and probably has mild PTSD) PC building has become MUCH safer over the decades
3.
by DrPhish 3mo ago
When you watch on vhs or laserdisc the loss of resolution only bothers you til the movie sucks you in. At that point it’s irreverent because your eyeballs are not watching a long sequence of pretty still pictures, but rather your brain is w
4.
by DrPhish 3mo ago
I redid everything that matters in my house/homelab with DAC cables for exactly that reason. Order of magnitude difference in watts and heat
5.
by DrPhish 8mo ago
Wouldn’t you also need to include the Ancient Greek phryctoriae military fire signalling system by that logic? It probably wasn’t the first, at that.
6.
by DrPhish 8mo ago
Very “futurological congress” thought
7.
by DrPhish 8mo ago
Also s4nake, the concept in a 4k binary from the demoscene circa 2013 https://www.pouet.net/prod.php?which=61035
8.
by DrPhish 9mo ago
Just use a commercial signage display
9.
by DrPhish 10mo ago
It’s not a process monitor, really, but to me the AWS Lightsail monitor tab feels like this. The “sustainable” line hits me right in the OCD to keep me grinding on cpu usage of the workload to keep extra spend at zero.
10.
by DrPhish 1y ago
Model back doors feel like baseless fearmongering. Something like https://rentry.org/IsolatedLinuxWebService should provide a good guarantee of privacy and security.
11.
by DrPhish 1y ago
I have this as well, but run a heavily locked down and isolated BIND server with NSD and Unbound for external authoritative and internal caching DNS respectively. Its easy to feed an RBL to unbound to do pi-hole type work, I use pf to trans
12.
by DrPhish 1y ago
Thanks, it was a bit of a gamble at the time (lots of dodgy ebay parts), but it paid off. R1 starts at about 10t/s on an empty context but quickly falls off. I'd say the majority of my tokens are generating around 6t/s. Some
13.
by DrPhish 1y ago
No, I’m running the unquantized 120b
14.
by DrPhish 1y ago
2xEPYC Genoa w/768GB of DDR5-4800 and an A5000 24GB card. I built it in January 2024 for about $6k and have thoroughly enjoyed running every new model as it gets released. Some of the best money I’ve ever spent.
15.
by DrPhish 1y ago
Its also easy to do 120b on CPU if you have the resources. I had 120b running on my home LLM CPU inference box in just as long as it took to download the GGUFs, git pull and rebuild llama-server. I had it running at 40t/s with zero eff
16.
by DrPhish 1y ago
Thanks Daniel. I know you upload them, but I was hoping for some solid numbers on your dynamic q8 vs a naive quant. There doesn't seem to be anything on either of those links to show improvement at those quant levels. My gut feeling is
17.
by DrPhish 1y ago
I generally download the safetensors and make my own GGUFs, usually at Q8_0. Is there any measurable benefit to your dynamic quants at that quant level? I looked at your dynamic quant 2.0 page, but all the charts and graphs appear to cut of
18.
by DrPhish 1y ago
Trees are pure carbon. I have heard a number of weak “yeah, but…” arguments that try to diminish the fact, but a central, common sense thesis remains. If we are truly worried about climate change and are unable to curb our consumption, then
19.
by DrPhish 1y ago
Here’s a demoscene prod from 2013 that executes a similar idea in 4 kilobytes https://m.pouet.net/prod.php?which=61035
20.
by DrPhish 1y ago
I can second this. I’ve been using R1 to both straight up generate mermaid as well as making custom mermaid syntax generators for dynamic diagramming
21.
by DrPhish 1y ago
In my opinion GPT-SoVITS is the best if you can put in the effort. I'm still using v2 since the output is so good. Its also the best multilingual one in my testing on Japanese inputs.
22.
by DrPhish 2y ago
Smokeping is an amazing and underrated resource as a network health metric and diagnostics tool. If you have a network monitoring or asset system you can export IP addresses from, you should use a small glue script to automatically build a
23.
by DrPhish 2y ago
That's very nice and compact. I do the same with a short bash script, but wrap each file in triple-backticks and attempt to put the correct language label on each eg: Filename: demo.py ```python ...python code here... ```
24.
by DrPhish 2y ago
The build guide index page is newer, but to be fair, the mikubox rentry is from Oct 6, 2023. If that isn't "ancient" in terms of AI workstation build guides, then I don't know what is.
25.
by DrPhish 2y ago
This is just a limited recreation of the ancient mikubox from https://rentry.org/lmg-build-guides Its funny to see people independently "discover" these builds that are a year plus old. Everyone is sleeping on the
26.
by DrPhish 2y ago
"Being in possession of a contraband Chinese Artificial Intelligence" is honestly one of the most cyberpunk things I can imagine. I hadn't felt like I lived in the future until now, honestly.
27.
by DrPhish 2y ago
“you can get a dual EPYC server with 768GB RAM - CPU inference only at around 6-8 tokens/sec.” This is what I run at home. I built it just over a year ago and have run every single model that has been released.
28.
by DrPhish 2y ago
The Nano only has 4GB VRAM and DS-R1 is 671B FP8 parameters (equivalent to 671GB model size). You need something with about 800GB to run the full model with context. You'd still need 400GB to even run a half-sized Q4 quant of R1, so th
29.
R1 671B at Home – EPYC Inference Build Guide(rentry.co)
3 points by DrPhish 2y ago|0 comments
30.
by DrPhish 2y ago
Making your own ggufs is trivial: https://rentry.org/tldrhowtoquant/edit It's a bit harder when they've provided the safetensors in FP8 like for the DS3 series, but these smaller distilled models appear to be

More ›