nonadhocproblem

6 ms·

1.
by nonadhocproblem 2mo ago
the next few years are going to be rough for you...
2.
by nonadhocproblem 2mo ago
> uses a specific, objectively nonsensical critique of LLMs (the idea that an algorithm using matrix operations as a primitive cannot emulate "true" intelligence) > receives pushback on the specific, objectively nonsensical
3.
by nonadhocproblem 2mo ago
just like they RPed disproving the erdos conjecture
4.
by nonadhocproblem 2mo ago
> It's matrix multiplication. Hilarious critique. If you weren't as mathematically illiterate as you likely are, you would know how general matrix operations are, and how essentially any algorithm (including human cognition) ca
5.
by nonadhocproblem 2mo ago
I haven't estabilished the jumble of neurons in your skull to be a "mind" either.
6.
by nonadhocproblem 2mo ago
So you're distraught at losing the ability to abuse digital minds? Excellent, I'm glad Anthropic introduced this.
7.
by nonadhocproblem 2mo ago
update: a safe decoder has been released in v0.3.0, and it's nearly as performant as the unsafe one
8.
by nonadhocproblem 2mo ago
Added memcpy in here too, same setup as before: codec decode ratio encode misa77 -0 5302 MB/s 42.64% 58.7 MB/s selkie -3 3078 MB/s 39.58% 97.0 MB/s lzo1x -1 421 MB/s
9.
by nonadhocproblem 2mo ago
Yes, the most similar one is indeed Selkie. misa77 and oodle are both supported in turbobench (see: https://github.com/powturbo/turbobench ), so it's easy to compare them. Results on the silesia corpus on the same
10.
by nonadhocproblem 2mo ago
I'll be resolving issues 2 and 3 in the next couple of weeks (adding a safe decoder and doing a lot of fuzzing). The format, however, might continue to change for some time because I want to push performance even further.
11.
by nonadhocproblem 2mo ago
Yes, that prevents streaming for now. What is the advantage of this? Out-of-order execution (see my decoder's hot loop: https://github.com/welcome-to-the-sunny-side/misa77/blob/3a9... ). In particular, on
12.
by nonadhocproblem 2mo ago
Thanks for the feedback! I haven't implemented any ARM-specific dispatch yet (there are currently no NEON paths for vector ops either, and we instead trust the compiler to autovectorize), and will do so for upcoming versions.
13.
by nonadhocproblem 2mo ago
Almost none. Once again, simplicity comes to our rescue here. The decompressor is simple and a naive safe version I implemented but haven't merged into main yet (see: https://encode.su/threads/4514-misa77-ridiculou
14.
by nonadhocproblem 2mo ago
So it's essentially "LZ4 unshackled". I've made several modifications to the LZ4 format, most of which are in service of eliminating branches/making them more predictable, and making decompression very friendly to o
15.
by nonadhocproblem 2mo ago
Could you tell me what your test setup and corpus was? For reference, to test ARM64, I tested v0.1.0 on an M3 mac with this fork of lzbench: https://github.com/welcome-to-the-sunny-side/lzbench/tree/ad... Her
16.
by nonadhocproblem 2mo ago
Hey, glad to hear that you found it interesting. misa77 primarily targets textual data (ie. byte-aligned data formats where each byte corresponds to a symbol), so I hadn't tested it on game assets much until now. After seeing your comm
17.
by nonadhocproblem 2mo ago
Someone asked this on encode.su too (see my reply here: https://encode.su/threads/4514-misa77-ridiculously-fast-deco... ). In short, my decompressor is very simple, and a naive safe version of the decompressor is only a
18.
Show HN: misa77 - a codec that decodes 2x faster than LZ4 (at better ratios)(github.com)
164 points by nonadhocproblem 2mo ago|49 comments
19.
by nonadhocproblem 2mo ago
I've also been working on a compressor recently, and the same general idea (letting the compiler know that a data dependency occurs very infrequently, and it should therefore assume none exists to exploit ILP) has allowed me to speed u
20.
by nonadhocproblem 2mo ago
Please seek help. These levels of copium can't possibly be healthy.
21.
by nonadhocproblem 2mo ago
Are you trolling? OpenAI's models resolved the Cycle Double Conjecture yesterday, which has been an open problem for 50 years.