marmaduke

5 ms·

1.
by marmaduke 2y ago
similar to RWKV7’s new (sub quadratic) attention mechanism which models key values as v≈kS’ and does an in-context descent on ||v - kS’||^2/2 (where the state matrix S is one attentional head) , explained more by the author here https
2.
by marmaduke 2y ago
ha, yeah, thanks.
3.
by marmaduke 2y ago
no, wrote it by hand for use with my own Heun implementation, since it’s for use within stochastic delayed systems. jax is fun but as effective as i’d like for CPU
4.
by marmaduke 2y ago
looks like a nice overview. i’ve implemented neural ODEs in Jax for low dimensional problems and it works well, but I keep looking for a good, fast, CPU-first implementation that is good for models that fit in cache and don’t require a GPU
5.
by marmaduke 2y ago
i like how the contraction it’s and abbreviation T’is are anagrams
6.
by marmaduke 2y ago
I didn’t look it up, but at first glance, it reminded me of discussions like this one https://youtu.be/qnT48wO0UL0
7.
by marmaduke 2y ago
that’s not what the story says. in any case, the point is to explain, in terms of dualistic if-then logic, that the if (you practice now) and then (you will wake up) are a single non-dual thing. but to communicate in in terms which make se
8.
by marmaduke 2y ago
do you have a link for your game? how was the release process?
9.
by marmaduke 2y ago
how far from your previous experience is the game work?
10.
by marmaduke 2y ago
i wonder if there are any semi automated approaches to finding outliers or “things worth investigating” in these traces, or is it just eyeballs all the way down?
11.
by marmaduke 2y ago
https://www.biorxiv.org/content/10.1101/2024.10.25.620245v1.... covers some of the recent perspectives on this modeling approach if you’re interested.
12.
by marmaduke 2y ago
having worked on whole brain modeling the last 15 years and european infra for supporting this kinda research, this is a terrible buzzword salad. the pdf is on par with a typical masters project.
13.
by marmaduke 2y ago
HH is kinda the opposite of LIF on the abstraction spectrum.
14.
by marmaduke 2y ago
The pigeon experiment is a great one to learn from not just about programming or software, but about life in general. Where are you getting your next dopamine hit? Is it random? Maybe that’s where our idiosyncrasies come from.
15.
by marmaduke 2y ago
> CPUs have the memory capacity advantage perhaps also more precisely they also have quite an advantage on anything that needs and plays nicely with caches? when I sliced my problem to maximize cache usage, I saw pretty clear scalabilit
16.
by marmaduke 2y ago
My approach to this is to write a bunch of tiny “kernels” which are obvious to SIMD and then inline them all, and it does a pretty good job on x86 and arm https://github.com/maedoc/tvbk/blob/nb-again/src&
17.
by marmaduke 2y ago
> People won't even pay for most of these pocket distractions If someone is paying, the transaction, by construction reinforces the psychological boundaries that obliteration eliminates. So I think not paying is part of it, just li
18.
by marmaduke 2y ago
I think “a cloud never dies” is more apt for this sentiment
19.
by marmaduke 2y ago
How would you want to host or update the markdown file? Sending an email for instance? Or run your own host?
20.
by marmaduke 2y ago
> Unlike most prior efforts, we release not only model weights and inference code, but also the reproducible training data, complete data processing pipeline, rigorous experimental ablation results, and detailed training protocols for op
21.
by marmaduke 2y ago
Just brainstorming here, but as a parent and someone who subscribed to a weekly CSA, it’d be a big boost to be able to take a picture of what I’ve got and get some ideas about recipes, especially if I could say, I’ve only got about 15 minut
22.
by marmaduke 2y ago
At the risk of taking some heat, I’d wager a preprint is recognized rightly by the Chinese as a flag planting, we’re first formality, where in the faults may even serve to validate it was written by human and not an LLM. Whereas the Western
23.
by marmaduke 2y ago
I tend to think of science as a distributed consensus process, and that peer review is analogous to a proof of work, and publishers are gatekeeping the distribution of proof of work. I think this is a useful analogy because one can subtrac
24.
by marmaduke 2y ago
Was your path towards the decision point a sudden one or a slow burn?
25.
by marmaduke 2y ago
Good question, I only looked briefly at unscented, I just thought ensemble was easy to understand.
26.
by marmaduke 2y ago
This is really nice approach. We are doing some nonlinear system id, and faced with this kinda problem (not irregular spacing but low sample rate and noisy). Definitely will check it out. What’s your opinion on ensemble KF? We’d like to
27.
by marmaduke 2y ago
Ha, I was benchmarking arm really, not writing from scratch.
28.
by marmaduke 2y ago
> All of the other cores from ARM or Qualcomm aren't as fast as the top Intel and AMD x86 I don’t think top cpu perf is relevant. I was working on some C code for science stuff inside Termux on a Pixel 7a, and would’ve been perfectl
29.
by marmaduke 2y ago
lol as if teachers have money to buy this to make their lessons plans.
30.
by marmaduke 2y ago
Hi! Thanks for your reply. First time hearing about Brski, I’ll have a look I guess your takeaway is that despite spotty support, the benefits of rust are net positive? I guess learning to call C from Rust is gonna be part of it too right

More ›