newhouseb

5 ms·

1.
by newhouseb 2y ago
A more extreme version of the ADC/DAC projects mentioned here is my (old) project that built a Bluetooth transceiver using just an FPGA and an antenna (and a low pass filter if you don't want the FCC coming after you): https:&#x2
2.
by newhouseb 3y ago
The short but unfulfilling answer is that this is because the DOCSIS standard historically has allocated a much broader frequency range for DL than UL. Unlike other forms of communication (like cell) that can use similar frequency ranges fo
3.
by newhouseb 3y ago
Huge fan of RASP et al. If you enjoy this space, might be fun to take a glance at some of my work on HandCrafted Transformers [1] wherein I hand-pick the weights in a transformer model to do long-handed addition similar to how humans learn
4.
by newhouseb 3y ago
I'm a little skeptical of the paper in question: https://arxiv.org/pdf/2301.00250.pdf If you look at the training and test set info: > We report results on two protocols: (1) Same layout: We train on the traini
5.
by newhouseb 3y ago
Right, there are many folks (dozens of us!) yelling about logit processors and building them into various frameworks. The mostly widely accessible form of this is probably BNF grammar biasing in llama.cpp: https://github.com&#x2F
6.
by newhouseb 3y ago
You still charge through the USB-C / Thunderbolt ports (and use a USB-C to USB-C cable with the provided power brick to do so). The MagSafe plug is just for convenience (and perhaps slightly higher wattage than is normally specced for
7.
by newhouseb 3y ago
An example from an earlier comment of mine on a different thread (assuming I've understood correctly): > let's say we had a grammar that had a key "healthy" with values "very_unhealthy" or "moderately_h
8.
by newhouseb 3y ago
I think this is likely a consequence of a couple of factors: 1. Fancy token selection w/in batches (read: beam search) is probably fairly hard to implement at scale without a significant loss in GPU utilization. Normally you can batch
9.
by newhouseb 3y ago
Also important to call out that anytime you have a freeform string it's pretty much an open invitation for the LLM to go completely haywire and run off into all sorts of weird tangents. So these methods are best used with other heurist
10.
by newhouseb 3y ago
The way LLMs work is they output probabilities for every _token_, so you don't really need to backtrack you can just always pick a token that matches the provided grammar. That said, you might want to do something like (backtracking) b
11.
by newhouseb 3y ago
I think you mean "run as part of the sampler," the tokenizer (and tokenization) is fixed for a given model. The sampler blob would basically: 1. Modify the output token probabilities to fit any arbitrary use case 2. Perhaps do tri
12.
by newhouseb 3y ago
I've been trying to wrap my head around this and my layman understanding is that there's an assumption (but maybe not baked into any requirements/standard) that use of the hardware key is locked behind either a biometric chec
13.
by newhouseb 3y ago
Modern LTE/5G provides pretty accurate location out of the box without GPS so if you really wanted to hide you would really need to just shut off the entire modem. Fortunately, the PinePhone has hardware switches to do this!
14.
by newhouseb 3y ago
This is great! I built a house and DIY'ed this because our site was 2 hours away from where we were living at the time. It was clear (our) GC was not used to this because they were constantly telling us that things were happening when
15.
by newhouseb 3y ago
AidKit | Remote | Full-time | https://aidkit.org | $130k-180k + equity | Social Impact / GovTech | TypeScript AidKit runs the largest guaranteed income programs in the country. We replace convoluted workflows of glued toget
16.
by newhouseb 3y ago
But only for the whole generation. So if you want to constrain things one token at a time (as you would to force things to follow a grammar) you have to make fresh calls and only request one token which makes things more or less impractical
17.
by newhouseb 3y ago
Good point. Backtracking is certainly possible but it is probably tricky to parallelize at scale if you're trying to coalesce and slam through a bunch of concurrent (unrelated) requests with minimal pre-emption.
18.
by newhouseb 3y ago
As someone also building constrained decoders against JSON [1], I was hopeful to see the same but I note the following from their documentation: The model can choose to call a function; if so, the content will be a stringified JSON obje
19.
by newhouseb 3y ago
Yeah, they used to be called SmartyStreets and then dropped the Streets. Not the greatest branding call.
20.
by newhouseb 3y ago
Quota management is indeed nonsensical. We serve large cash assistance programs that have stampedes of people applying all at once where we needed decent geocoding to determine eligibility. We were exceeding the default 50qps quota. I asked
21.
by newhouseb 3y ago
Thinking about this more, softmax is also a form of normalization that could likely contribute to this phenomenon.
22.
by newhouseb 3y ago
Oh good catch! The author defines a layer norm layer but then... comments it out in the actual implementation (I missed the fact that it was commented out). So that answers my second question of what happens without it. Anecdotally in my ow
23.
by newhouseb 3y ago
The algorithm learned here actually makes a lot of sense when you spend more time understanding how transformers typically work. Namely: once you include layer normalization your model is more or less forced to find ways to represent absolu
24.
by newhouseb 3y ago
Where do you draw the line? Is type checking in any domain harmful because it acts a crutch for your mental model of how your code works? One could similarly extrapolate this to any static analysis in any language.
25.
by newhouseb 3y ago
Interesting, do you have any references or examples? Some brief googling around hasn't found anything like this. The fact that overloading was an issue makes me think that TF1 was doing something different because Typescript generic ty
26.
by newhouseb 3y ago
It's a great question. I don't really have a horse in this race as long as whatever wins is maximally ergonomic. I think as long as the DSL is Turing complete such that you could "compute" on tensor shapes then we win. T
27.
by newhouseb 3y ago
We're talking about the streaming API which streams generated text token by token, not the normal one-shot API. I have no insider knowledge but would agree with your intuition on the normal API.
28.
by newhouseb 3y ago
This is a great thread, thanks! Somehow I missed it when looking for prior art. When I initially started implementing this I was hung up on similar concerns. For example in GPT2/PotatoGPT the MLP player is 4x the width of the residual
29.
by newhouseb 3y ago
I'm excited about this for probably different reasons than most: I think Typescript could be a more ergonomic way to develop ML models than Python because you can automatically infer and check tensor dimensions while you are writing c
30.
by newhouseb 3y ago
This is slick -- It's not explicitly documented anywhere but I hope OpenAI has the necessary callbacks to terminate generation when the API stream is killed rather than continuing in the background until another termination condition h

More ›