gajjanag

8 ms·

1.
by gajjanag 3d ago
> The entire essay is about stopping development until it can be made more safe, with specific ideas on how to do that. No, the essay does not talk about "stopping development". That's a very clever sleight of hand made in
2.
by gajjanag 3d ago
> The idea that open-weight models are more efficient seems unfounded. https://artificialanalysis.ai/ intelligence vs cost per task disagrees with this statement.
3.
by gajjanag 23d ago
> You will typically find your staff and higher level engineers rarely are actually writing code themselves. Many so called staff+ engineers are basically glorified Google Docs engineers, or management who want to call themselves technic
4.
by gajjanag 4mo ago
> Communication matters most when you're dealing with cross-org concerns and those that master it are usually the more friendly and pleasant ones. I don't agree with the second one, but agree with the first. Throughout my corpo
5.
by gajjanag 5mo ago
> Thanks for that! It is worth noting that taking advantage of the post-rotation distribution I again feel this claim is too strong. Rotations have been used in information theory/wireless communications for decades at this point, w
6.
by gajjanag 5mo ago
Wow, yes - you are completely correct (read through the note in detail now). Though, as your paper also notes, the quantizer values themselves aren't fundamentally novel to either paper. Lloyd Max scalar quantizers have been studied f
7.
by gajjanag 5mo ago
There are also more papers on similar themes. For example, TurboQuant makes use of QJL (quantized Johnson Lindenstrauss transformations). One of the first papers to characterize the QJL and in fact the rate distortion tradeoff for quantized
8.
by gajjanag 5mo ago
TurboQuant is known across the industry to not be state of the art. There are superior schemes for KV quant at every bitrate. Eg, SpectralQuant: https://github.com/Dynamis-Labs/spectralquant among many, many papers. &g
9.
by gajjanag 5mo ago
The bigger challenge is GPU/NPU. Branches for fast vs accurate path get costlier, among other things. On CPU this is less of a cost. Most published libm on GPU/NPU side have a few ULP of error for the perf vs accuracy tradeoff. Eg
10.
by gajjanag 7mo ago
> That's why you need to put your scope The problem is, "scope" is often equated to "how many people worked in my empire" rather than "how much business value did my work X generate". The two things are
11.
by gajjanag 10mo ago
>80%-90% or so of real life vectorization can be achieved in C or C++ just by writing code in a way that it can be autovectorized. Yep. I was pleasantly surprised by the autovectorization quality with recent clang at work a few days ago.
12.
by gajjanag 11mo ago
Welcome to the brave new world these days: 1 - Very few people conduct "proper scholarship", and fail to trace ideas back to their original inception and cite them correctly. This happens time and again in deep learning, where 30+
13.
by gajjanag 1y ago
> I don't think there are many (or any) upsides to the well documented downsides. C++ template metaprogramming still remains extremely powerful. Projects like CUTLASS, etc could not be written to give best performance in as ergonom
14.
by gajjanag 1y ago
As others have pointed out, these phenomena are well known to many folks across companies in the AI infra space. It doesn't really break new ground. This article is a good exposition of the basic strategies though. What I would have lo
15.
by gajjanag 1y ago
I guess you have never worked with a slow induction cooktop. Literally we had to spend 15 minutes more for cooking things on induction compared with our previous apartment's gas connection. Maybe they are better now but it is certainly
16.
by gajjanag 1y ago
+1 - there are just so many Asian recipes that can not be done anywhere near as easily on induction stovetops (high heat from direct flame for flatbreads, etc). Plus a whole bunch of cookware doesn't work with induction (clay pots, non
17.
by gajjanag 1y ago
There is a vast number of sysctl in xnu that have not really been re-examined in over 15 years. Many tunings date back to the spinning rust drive era (for example). There are plenty of examples like this. Disclaimer: I worked at Apple and p
18.
by gajjanag 1y ago
The big problem is a bunch of folks actually take these things seriously and use it as an excuse to freeze the junior hiring pipeline. At the senior levels this is not actually believed by the powers that be, since a bunch of hiring is stil
19.
by gajjanag 1y ago
> Large corporations believe anyone is replaceable. This is definitely true. By design, large corporations are structured so that there is no single point of failure. > Again I am an IC & don’t see/hear any extra work done fo
20.
by gajjanag 2y ago
Same. The compensation is substantially better at FAANG, but in terms of actual on the ground work being rewarded, almost never the case. Meta-work (lots of "cross functional" documents, alignment meetings, sync ups with senior te
21.
by gajjanag 2y ago
This is much more nuanced now. See Apple "Private Cloud Compute": https://security.apple.com/blog/private-cloud-compute/ ; they run a lot of the larger models on their own servers. Fundamentally it is mo
22.
by gajjanag 2y ago
Maybe on a particular model/dataset but extremely unlikely in general. Again, like another commenter pointed out: if you truly believe it isn't that hard we would love to hire you at Meta ;)
23.
by gajjanag 3y ago
Our group works on some of this stuff at Meta, and we have a pretty good diversity of backgrounds - high performance computing (the bulk), computer systems, compilers, ML engineers, etc. We are hiring. Feel free to DM me to learn more.
24.
by gajjanag 3y ago
lmkd (low memory killer daemon) works fairly differently off of a different set of signals and different policy. But yes, conceptually they try to achieve the same goal. I also do not know if Android combines system libraries into one big f
25.
by gajjanag 3y ago
A couple of additional points on how the "low-RAM" works: 1 - https://www.lifewire.com/understanding-compressed-memory-os-... : Apple devices have support for memory compression, see https://opensource.
26.
by gajjanag 3y ago
Umm, Apple still sells devices with just 1 GB of RAM on them ;)
27.
by gajjanag 4y ago
Same, doing it once out of my own curiosity to see how the corporate machine works. Not doing it again - seeing first hand how it is due to managerial incompetence more than anything else. The "reward" ratio is just not worth it:
28.
by gajjanag 4y ago
> based on "impact" rather than arbitrary metrics Umm, from whatever I have seen in big tech "impact" is also fairly arbitrary. It all is based on how cozy one is with one's manager, skip manager, and so on. More
29.
by gajjanag 4y ago
> A page will be loaded in if any part of it is useful. Given that functions will be laid out more or less randomly throughout a shared library, and programs use a randomly scattered subset of the functions, I think its safe to say that
30.
by gajjanag 4y ago
By the way, https://github.com/FALCONN-LIB/FALCONN contains a really good LSH implementation. Also see https://www.mit.edu/~andoni/LSH/ if you want to know more about the research literature.

More ›