dzr0001

5 ms·

1.
by dzr0001 4mo ago
And making TLS more difficult, especially for HA systems. Guess you would just need one cert for 127.0.0.1 for all local services.
2.
by dzr0001 4mo ago
My token throughput is much better using vLLM-mlx on my M2 ultra than llama.cpp. It might be worth a shot to give it a try.
3.
by dzr0001 8mo ago
I think there's a fundamental difference between programming language repos and package repositories like the official RPM, deb, and ports trees. These (typically) operating system repos have oversight and are tested to work within a s
4.
by dzr0001 1y ago
I suspect that there's some marketing component at play here. People who do not own but observe devices making seemingly unnecessary noises might perceive these devices as premium. Think about the various beeps that occur when locking
5.
by dzr0001 1y ago
I did a quick scan of the repo and didn't see any reference to Ray. Would this indicate that llm-d lacks support for pipeline parallelism?
6.
by dzr0001 1y ago
It does if you need pipeline parallelism across multiple nodes.
7.
by dzr0001 2y ago
Unfortunately, this data is harder to find than it should be. For instance, just looking at Kioxia, which I've found to be very performant, their datasheets for the CD series drives don't mention write latency at all. Blocks and F
8.
by dzr0001 2y ago
What drive is this and does it need a trim? Not all NVMe devices are created equal, especially in consumer drives. In a previous role I was responsible for qualifying drives. Any datacenter or enterprise class drive that had that sort of la
9.
by dzr0001 3y ago
I have the original M1 air that I got the day it released in 2020. In a typical week, I will let my battery discharge to less than 10% twice and recharge it to 100%. I've logged 458 cycles and lost 11% of my capacity. Not too bad.
10.
by dzr0001 6y ago
We used to have a pub rate of about 200k msgs/s, from about 400 producers all to a single exchange and had similar issues. However, we were able to mitigate this by using lazy queues. This worked fine until things got behind and then w