haxen

15 ms·

1.
Does "vpmovzxbd" scare you? Here's why it doesn't have to(questdb.io)
1 points by haxen 2y ago|0 comments
2.
by haxen 3y ago
Can you provide more details on how that would work? Given that the input is CSV rows and not an array of integers alone, and the fact that getting to the next row is dependent on finding the delimiter in the current row.
3.
by haxen 3y ago
I think it would work if that was the only code in the loop. But the loop spans several more nontrivial operations, including hashtable insertion.
4.
by haxen 3y ago
> maybe 4 cycles worth in this optimized version? It's quite a bit more than that, just the code discussed in the post is around 20 instructions, and there's a bunch more concerns like finding the delimiter between the name and
5.
by haxen 3y ago
I wonder what you mean here. What code exactly would get auto-vectorized? Parsing the temperature surely not, since it's not in an array of fixed-size fields.
6.
by haxen 3y ago
There are many variations of the original code used in different solutions. Many of them return the temperature like the variant used in the post, but they split out the part that finds the decimal dot into a separate function. Then you can
7.
by haxen 3y ago
The temperature fields are interleaved with name fields, so I don't think you'd get any extra benefit from SSE. Also, since the temperature field is variable-sized, it would probably not pay off even if it was stored by column. SS
8.
by haxen 3y ago
This post explains a piece of code that appeared on the recent One Billion Row Challenge, it parses a variable-layout string whose bytes are packed into a single 64-bit variable, and doesn't use any if statements to achieve it. The str
9.
by haxen 3y ago
It was known, but the requirement was that the program keep working for an arbitrary keyset that conforms to the specified rules (up to 10,000 unique keys, each up to 100 bytes in length).
10.
by haxen 3y ago
I didn't bother to try, so not sure. There would probably be some challenges and I don't see how I'd accomplish it without some branch instruction in the custom hasher.
11.
by haxen 3y ago
Running without RAM cache would be a great followup to this challenge. I think a time around 2-3 seconds should be achievable. But, it would be highly sensitive to the hardware setup and how well the disk-reading code is placed on cores rel
12.
by haxen 3y ago
I used to benchmark a lot on an enterprise-grade SSD 10 years ago, and that was already at 2 GB/s. Today, even my laptop's SSD supports multiple GB/s. But you're right about the contest -- each program was measured five
13.
by haxen 3y ago
Thanks for the praise Gunnar, but we all owe it to you for organizing it, and especially sticking through thick and thin when it took off, and needed lots of attention to evaluate everyone and maintain a level playing field!
14.
by haxen 3y ago
The reason the custom hashtable wins out isn't something generally applicable. For the very specific dataset used in the challenge, the hash function could be radically simplified, to just a single multiply and rotate left. To be fair,
15.
by haxen 3y ago
I actually wrote a Rust version as well, and yes, it was far easier to write, far less code (although not incorporating all the tricks), completely safe, and pretty fast -- but still 2x slower than my end result in Java. HashMap was quite a
16.
by haxen 3y ago
Interestingly enough, that was my first idea. But when you consider the tiny keyset size, it would be hard to beat two machine instructions to calculate the hash + a single array lookup.
17.
by haxen 3y ago
Cool, I see mfiguiere linked to my recent blog post! Let me share a few words about it... I took part in the One Billion Row challenge (1BRC). It was a lot of fun, but also a great learning experience. People came up with some pretty incred
18.
Billion Events per Second at Millisecond Latency: NEXMark at Giga-Scale(jet-start.sh)
21 points by haxen 6y ago|0 comments
19.
by haxen 6y ago
Here are some examples where user consent is undisputed: ride hailing, bicycle rental, street navigation, running/biking/sailing contests, location-sensitive searches. These are the kinds of applications for which Hazelcast Jet of
20.
by haxen 6y ago
This statement comes from our benchmarking work: https://jet-start.sh/blog/2020/06/09/jdk-gc-benchmarks-part1 The point is that Jet can track several million distinct keys, even on a single machine, and
21.
by haxen 6y ago
An Apache Beam Runner is already implemented in Jet: https://beam.apache.org/documentation/runners/jet/ Beam is just an API layer with different backing implementations. But you don't typically use Beam
22.
by haxen 6y ago
Hazelcast Jet will get an SQL API soon, and we're actively considering first-class support from other languages as well.
23.
by haxen 6y ago
I concur with your point about the programming model and style, but I do also maintain that "cooperative" vs. "preemptive" is not about that difference. It is a technical difference on how the system interleaves threads
24.
by haxen 6y ago
I think the distinction is pretty clear: either the mechanism requires cooperation by the application thread (which typically initiates the yield at a compiled-in, predefined point), or it doesn't and the runtime environment preempts i
25.
by haxen 6y ago
"processes voluntarily yield control periodically or when idle or logically blocked in order to enable multiple applications to be run concurrently." This tells me virtual threads (without forced preemption) are cooperative.
26.
by haxen 6y ago
Without the very last point, forced preemption, they would indeed be cooperative because not calling any blocking method would make them non-cooperative. This is exactly the same as within the "colored" subspace in a language that
27.
by haxen 6y ago
> you are actually going to allocate both the parent and child. You're going to allocate a single memory block that contains all the state of that object, which of course includes the superclass state. But that has nothing to do wit
28.
by haxen 6y ago
http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part2.h... Yes it can and actually it's quite low-hanging fruit on the JVM. GC safepoints are already there, you just have hook into the mechanism.
29.
by haxen 6y ago
>They do explicitly yield Arguably, virtual threads also explicitly yield by calling one of the blocking methods in the JDK. This is very similar to putting all the bottom-level suspendable functions into the Kotlin standard library. &gt
30.
by haxen 6y ago
A month ago we benchmarked against the master of the ZGC repo. There wasn't a big difference.

More ›