p12tic

11 ms·

1.
by p12tic 6mo ago
> Only a matter of time for local models to reach Opus level. We are 1 or at most 2 years behind that and Anthropic knows that. Can confirm. Kimi K2.5 is pretty intelligent and most of the time there's no difference between Opus and
2.
by p12tic 6mo ago
> In the last few years California runs 100% renewable on many days (and growing) every year. How many is "many days"? Gas is still used for at least one fifth of electricity. https://app.electricitymaps.com/map
3.
by p12tic 8mo ago
> not ... web/db servers, lightweight stuff like that. They scale very well for web and db servers as well. You just put lots of containers/VMs on a single server. AMD EPYC has a separate architecture specifically for such work
4.
by p12tic 1y ago
Depends on a server. This test got 79W idle for _two socket_ E5 2690-V4 server. https://www.servethehome.com/lenovo-system-x3650-m5-workhors...
5.
by p12tic 1y ago
The problem is with the form factor, not the server hardware per-se. If one buys regular ATX motherboard that accepts server CPUs and fits it in regular ATX case, then there's lots of space for a relatively silent CPU air cooler. 2690
6.
by p12tic 1y ago
Better build a single workstation - less noise, less power usage and the form factor is way more convenient. A budget of $3000 can buy 128 cores with 512GB of RAM on a single regular EATX motherboard, a case, a power supply and other access
7.
by p12tic 1y ago
> Today it's a bit more complicated when you have servers with 100+ cores as an option for under $30k (guestimate based on $10k CPU price). If one can buy used, then previous generation 128C 256T epyc server is less than $5k. For ho
8.
by p12tic 1y ago
That's just an artifact of Intel disabling ECC on consumer processors. There's no reason for ECC to have significantly higher power consumption. It's just an additional memory chip per stick and a tiny bit of additional logic
9.
by p12tic 1y ago
We both agree. Batch size 1 is only relevant to people who want to run models on their own private machines. Which is the case of OP.
10.
by p12tic 1y ago
All of this is for batch size 1.
11.
by p12tic 1y ago
State of the art of local models is even further. For example, look into https://github.com/kvcache-ai/ktransformers , which achieve >11 tokens/s on a relatively old two socket Xeon servers + retail RTX 4090 GPU
12.
by p12tic 1y ago
Incorrect. https://en.wikipedia.org/wiki/USB_hardware#USB_Power_Deliver... is a good start about the subject: "PD-aware devices implement a flexible power management scheme by interfacing with the power source thr
13.
by p12tic 1y ago
For all intents and purposes cache may not exist when the working set is 17B or 109B parameters. So it's still better that less parameters are activated for each token. 17B parameters works ~6x faster than 109B parameters just because
14.
by p12tic 2y ago
Most laptops are severely limited by heat dissipation. So it's normal that performance is much worse. The CPU cannot stay in turbo as long and must drop to lower frequencies sooner. On longer benchmarks they CPU starts throttling due t
15.
by p12tic 2y ago
Container security boundary can be much stronger if one wants. One can use something like https://github.com/google/gvisor as a container runtime for podman or docker. It's a good hybrid between VMs and containers
16.
by p12tic 2y ago
One can use something like https://github.com/google/gvisor as a container runtime for podman or docker. It's a good hybrid between VMs and containers. The container is put into sort of VM via kvm, but it does not
17.
by p12tic 4y ago
Seems like that's true: https://learn.microsoft.com/en-us/sharepoint/redirect-known-...
18.
by p12tic 4y ago
It's complicated, memory accesses can really block for relatively long periods of time. Consider that regular memory access via cache takes around 1 nanosecond. If the data is not in top-level cache, then we're looking at roughly
19.
by p12tic 5y ago
That would be a significant downgrade. Threadripper CPUs top out at 64 cores / 128 threads.
20.
by p12tic 5y ago
I agree that just buying the most expensive monitor is waste of resources. On the other hand, I think price shouldn't even come into the picture when deciding on basic things, such as the number of monitors or whether to choose 4K moni
21.
by p12tic 5y ago
It's strange when monitor costs are being discussed while they are the primary tool for software development. If software developer produces $100k/year value, then it's worth spending several thousands of dollars for every pe
22.
by p12tic 5y ago
It uses a custom widget toolkit. Adding touchpad gesture support is certainly doable, but it would benefit only single application, so we haven't prioritized that so far.
23.
by p12tic 5y ago
Touchpad gesture implementation in X server has been released in version 21.1 Wayland touchpad gesture implementation in Qt widget framework has been released in version 6.2.0 X11 touchpad gesture implementation in Gtk widget framework has
24.
by p12tic 5y ago
Fedora and Gnome is only one desktop environment and one widget toolkit. For example, Qt-based apps didn't have touchpad gestures at all anywhere until my work on Wayland gestures landed this year. It is true though that if one limits
25.
by p12tic 5y ago
Interesting, this feature may only be for touchscreens because two-finger swipes are registered as scrolls on Wayland. This will indeed need further work. What does work though is two-finger pinch gesture to zoom in/out of a web page.
26.
by p12tic 5y ago
Most applications will use either Gtk or Qt widget libraries, so a lot of similarity of how applications behave already exists. I don't think it's possible to make Gtk and Qt themselves behave identically at this point. For exampl
27.
by p12tic 5y ago
For now we only focus on touchpads. I think that if we're successful in delivering touchpad improvements then we will gain credibility and trust that could be useful when working on other input devices.
28.
by p12tic 5y ago
In short - there are no binaries and it's relatively hard to compile these manually, so I recommend to wait until the Linux distributions picks these projects up. This usually takes around 6 to 12 months.
29.
by p12tic 5y ago
You need to wait up to a year for the distributions to pick up the code into their default installs. Installing bleeding edge window managers, widget toolkits and similar software yourself is more difficult than it's worth for a normal
30.
by p12tic 5y ago
Hi, I'm the developer behind this effort. I can answer any questions you have.

More ›