8 ms·

Here's my implementation in Go, which runs in under 5 seconds. It doesn't use anything too obscure, only the built-in maps and no external libraries. It's also

Here's my implementation in Go, which runs in under 5 seconds. It doesn't use anything too obscure, only the built-in maps and no external libraries. It's also the fastest solution I've tested on my M3 Pro Mac. Eager to see what beats it! https://gist.github.com/corlinp/176a97c58099bca36bcd5679e68f9708 https://gist.github.com/corlinp/176a97c58099bca36bcd5679e68f...

bufferoverflow 3y ago
Look at the leaderboard for the fastest solutions https://github.com/gunnarmorling/1brc#results https://github.com/gunnarmorling/1brc#results The fastest is currently at 12 seconds (not on M3 Pro though).
guss_bro 3y ago
The fastest java solution from the current leaderboard runs within 2.6 seconds in my brand new M3 Mac.
- corlinpalmer 3y ago
  Is that `./calculate_average_royvanrijn.sh`? It runs in 6.7s for me. I would love it if you could run my solution and compare!
dvko 3y ago
There’s an implementation in C which should run in well under 2 seconds on your M3. https://github.com/dannyvankooten/1brc https://github.com/dannyvankooten/1brc
- corlinpalmer 3y ago
  Under 2 seconds seems rather impossible on my machine since the disk maxes out at 5.1 GB/s. This one ran in 7.4s on my M3: `bin/analyze measurements.txt 30.34s user 16.28s system 629% cpu 7.406 total`
  - dvko 3y ago
    On most Linux distributions and when the file is mmap'd, if you run it a second time the data will still be in RAM and not have to be read from disk. This gets the runtime down to 1.1s for this AMD 2950x (https://github.com/gunnarmorling/1brc/discussions/46#discussioncomment-8028029 https://github.com/gunnarmorling/1brc/discussions/46#discuss...). With SIMD and certain assumptions about the input this can seemingly be further reduced to well under a second, eg see https://github.com/gunnarmorling/1brc/discussions/138 https://github.com/gunnarmorling/1brc/discussions/138.

bufferoverflow 3y ago

Look at the leaderboard for the fastest solutions https://github.com/gunnarmorling/1brc#results https://github.com/gunnarmorling/1brc#results The fastest is currently at 12 seconds (not on M3 Pro though).

guss_bro 3y ago

The fastest java solution from the current leaderboard runs within 2.6 seconds in my brand new M3 Mac.

corlinpalmer 3y ago
Is that `./calculate_average_royvanrijn.sh`? It runs in 6.7s for me. I would love it if you could run my solution and compare!

dvko 3y ago

There’s an implementation in C which should run in well under 2 seconds on your M3. https://github.com/dannyvankooten/1brc https://github.com/dannyvankooten/1brc

corlinpalmer 3y ago
Under 2 seconds seems rather impossible on my machine since the disk maxes out at 5.1 GB/s. This one ran in 7.4s on my M3: `bin/analyze measurements.txt 30.34s user 16.28s system 629% cpu 7.406 total`
- dvko 3y ago
  On most Linux distributions and when the file is mmap'd, if you run it a second time the data will still be in RAM and not have to be read from disk. This gets the runtime down to 1.1s for this AMD 2950x (https://github.com/gunnarmorling/1brc/discussions/46#discussioncomment-8028029 https://github.com/gunnarmorling/1brc/discussions/46#discuss...). With SIMD and certain assumptions about the input this can seemingly be further reduced to well under a second, eg see https://github.com/gunnarmorling/1brc/discussions/138 https://github.com/gunnarmorling/1brc/discussions/138.