fwessels

5 ms·

1.
by fwessels 4y ago
disclaimer: I am one of the co-authors pf the code as well as founder
2.
Sneller: Building a SQL VM in AVX-512 Assembly (With Golang)(github.com)
7 points by fwessels 4y ago|1 comments
3.
by fwessels 4y ago
Thank for for the feedback, that is nice to hear. And as for the business question, we plan on launching a Sneller Cloud offering. (Sneller founder here)
4.
by fwessels 4y ago
Sneller founder here: we do not have any non-AVX code so we cannot compare directly against that. But generally speaking our code always works on 16 lanes in parallel per core, so that gives a huge speed-up.
5.
by fwessels 4y ago
Albeit not using gas, you may want to check out https://github.com/SnellerInc/sneller -- it has about ~250 primitives written in AVX-512. (Sneller founder)
6.
by fwessels 6y ago
We have added both a cost and power consumption comparison to the blog post (including AMD EPYC as well), here is a snippet: | Instance type | m6g.16xlarge | c5.18xlarge | c5.24xlarge | c5a.24xlarge | | Architecture | G
7.
by fwessels 6y ago
As per last week's announcement from AWS about the availability of AMD EPYC cpus, we have repeated the “single socket” test for the EPYC cpu as well. The updated chart is included in the post or you can find it here: https:/&#x2F
8.
by fwessels 9y ago
Check out the graph at https://github.com/fwessels/HashCompare that compares different hashing techniques. And also we've done some predictions for collisions, see: https://github.com/fwessels&#x2F
9.
by fwessels 9y ago
Yes, you can, but just naming the return variable takes care of this.
10.
by fwessels 9y ago
No, it is not an IR code such as in Java or .Net. At compile time it is translated into opcodes for the corresponding architecture that you are compiling for.
11.
by fwessels 9y ago
That is a good question but have never tried it. However, there is nothing "special" about the assembly as ported by c2goasm, so it should equally well as any other (hand-crafted) assembly.
12.
by fwessels 10y ago
S3 data transfers costs are an issue -- that's why you can host minio yourself at any hosting company, and save significantly (multiple times) on data transfer and storage costs.
13.
by fwessels 10y ago
Nice blog, and the tool would have helped indeed. And also, memory allocation is not the most obvious topic in Go/Golang assembly...
14.
by fwessels 10y ago
The Golang assembly does not support all instructions, we've created a tool (see https://github.com/minio/asm2plan9s ) to assist in translating instructions into their opcode equivalents.
15.
by fwessels 10y ago
No we did not compare to SHA512 (since we don't need it). I would expect SHA512 to be 2x faster compared to the SHA256 software version, but that is still way slower than the ARM SHA extensions accelerated version.
16.
by fwessels 10y ago
No, there are no algorithm changes (otherwise the results would be different), it is just taking advantage of the ARM SHA accelerations when they are available.
17.
by fwessels 10y ago
No, compared to an ASIC it will still be significantly slower.
18.
by fwessels 10y ago
Go didn't help specifically, it is just that this package makes support for the ARM SHA instructions available for Golang.
19.
Time Machine for directories? Directory Versioning using s3git(github.com)
2 points by fwessels 10y ago|0 comments
20.
by fwessels 10y ago
The library is not yet open sourced but will be soon. Some internals about how s3git uses the BLAKE2 hashing algorithm are described in https://github.com/s3git/s3git/blob/master/BLAKE2.md which may be o
21.
by fwessels 10y ago
That is correct (see also earlier reply), it is aimed at really large repos with primarily non-text blobs backed up cloud storage like S3, but doing so in the spirit of git and hence the name.
22.
by fwessels 10y ago
Thanks for the links. However s3git more applies the git philosophy/way of working to Cloud Storage rather than being compatible at a binary level.