amunra__

7 ms·

1.
by amunra__ 2y ago
Article author here.. Instrumenting would have been 100% just as feasible. Ironically it would have been more work. For context, our DB is highly optimised for ingestion (millions of rows / second) and adding any high resolution metric
2.
by amunra__ 3y ago
Maybe we should write a blog post, though there's gotta be one out there for this already. The short of it is: Learn C. Learn your system calls. Learn JNI. Learn about com.sun.misc.Unsafe. Learn about the disruptor pattern. Learn how t
3.
by amunra__ 3y ago
Maven, see the rust-maven-plugin we wrote for this. It's opensource.
4.
by amunra__ 3y ago
I'd say both Rust and C++ trade blows when it comes to expressiveness. You know Rust already, so I'm not going to try sell you how powerful macros can be (see SQLx's ability to compile-time check SQL queries). And indeed C++
5.
by amunra__ 3y ago
The rust-maven-plugin we wrote indeed also supports JNA for these simple use cases. https://github.com/questdb/rust-maven-plugin Compared with JNA, JNI is indeed more complex, but it's faster and has more features
6.
by amunra__ 3y ago
We have an initialisation step as soon as we load the jni lib that takes care of this. Given that this gets done before any other threads are started, I don't think there'd be an issue. Good point :-)
7.
by amunra__ 3y ago
I'd say expressiveness of C++ with productivity of Java. Rust is indeed not easy to learn. A little example. Yesterday I was making changes to our C++ client library and I wanted to improve the example in our documentation. We use a de
8.
by amunra__ 3y ago
When the time is right. There's finally new APIs coming in the Java space that will make native-code interop easier and more reliable. Our open source database edition can also be used embedded though, so we can only upgrade at the pac
9.
by amunra__ 3y ago
I'm the author of the blog post. The focus of the article really is about JNI in Rust. I see most questions are about "Why did you not use X language instead?", so let me try and address this. To answer the "Why not just
10.
by amunra__ 3y ago
All the library does is facilitate the build process. I think there's possibly some automation that can be added once that whole stuff stabilizes properly. From Rust it's already possibile to automate exposing a C header via cbind
11.
by amunra__ 3y ago
Hello, Here at QuestDB we're beginning to write part of our database code in Rust as JNI native code extensions. Since all of our Java code is built with Maven we found we needed an easier way to invoke `cargo` during our normal build
12.
Show HN: rust-maven-plugin: Compile Rust JNI libraries in Java Maven projects(github.com)
4 points by amunra__ 3y ago|4 comments
13.
by amunra__ 4y ago
Hi, I'm the original author of the QuestDB Python client library and benchmark. It all started when we had one of our users needing to insert quite a bit of data into our database quickly from Pandas. They had a dataframe that took 25
14.
Inserting 1.8M Rows/s from Pandas into QuestDB with Arrow, Rust and Cython(github.com)
1 points by amunra__ 4y ago|1 comments
15.
by amunra__ 4y ago
We actually do have a web based demo which is https://demo.questdb.io/ preloaded with millions of rows of data. That one focuses on SQL queries though. The notebook in https://play.questdb.io/ offers a more
16.
by amunra__ 4y ago
Yes. We're pretty good with large volumes of data. Eventually all local drives fill up though. When ingesting data we partition data by time. By default we partition by day. This give you the flexibility to detach partitions, store the
17.
by amunra__ 4y ago
Most databases store the latest state of something. We don't. We ingest events. After all, life is a function of time :-) The whole world ticks and we take those ticks and store them. If part of your application tracks anything happeni
18.
by amunra__ 4y ago
Hi, I'm Adam Cimarosti, one of the core engineers at QuestDB. We built play.questdb.io to make it easy for anyone to try our database. No installation. There's a Jupyter Lab notebook, data, sample code, queries and graphs. We&#x27