petermattis

7 ms·

1.
by petermattis 6y ago
Cgo exists because goroutine stacks are small and grow on demand. C/C++ expect large fixed size stacks. While you can technically call into C++ without going through cgo (see https://github.com/petermattis/fastcgo
2.
by petermattis 6y ago
Direct IO is on our radar, though when I experimented with enabling direct IO in RocksDB it only hurt CockroachDB benchmarks. This was several years ago. I believe newer releases of RocksDB have made improvements in this area.
3.
by petermattis 6y ago
> Is Pebble/CockroachDB capable of saturating the backplane with requests in parallel? Yes. > Does it multiplex a single query by dispatching smaller requests to a thread-pool? Yes, though it depends on the query. Trivial queries
4.
by petermattis 6y ago
Pebble and FoundationDB are apples and oranges. Pebble is per-node KV storage engine. FoundationDB is a distributed multi-modal database. Internally, FoundationDB uses a library like Pebble for the per-node data storage. I think at one poin
5.
by petermattis 6y ago
Yes. So far performance was worse in experiments, and the durability improvements are questionable because it is extremely difficult to get a clear understanding of the durability semantics of direct IO. If you can find a pointer to clear d
6.
by petermattis 6y ago
Damned for using a unique name (CockroachDB), and damned for using an innocuous one. PS PebblesDB was a research project and is dead as far as I know.
7.
by petermattis 6y ago
Pebble does not currently implement MultiGet as CockroachDB did not use RocksDB's MultiGet operation. CockroachDB can use multiple nodes to process a query by decomposing SQL queries along data boundaries and shipping the query parts t
8.
by petermattis 6y ago
As far as I'm aware, the fsync/fdatasync sharp edges are around what happens after an fsync/fdatasync failure. My understanding is that you can't rely on anything. The only sane option is to crash the process and attempt
9.
by petermattis 6y ago
> So I understand the rationale for writing your own storage layer and think this is an awesome project, but there's something missing for me. One of the issues Peter brings up is they've come across a number of serious bugs in
10.
by petermattis 6y ago
The TLDR is that the GC did cause problems so we had to avoid it for the block cache. Luckily we were able to do so without exposing the complexity in the API. Not for the faint of heart. Don't try this at home kids.
11.
by petermattis 9y ago
Performance is "ok". CockroachDB 1.0 won't be winning benchmarks, but that was never the goal. We have seen near linear scaling from 3-64 nodes on a uniformly random write workload. And the new distributed SQL execution engin
12.
by petermattis 9y ago
(Cockroach Labs co-founder here) This is a really hard question to answer because there are so many different scenarios to test. Even if you restrict the question to KV operations a simple answer isn't forthcoming. For example, if you
13.
by petermattis 9y ago
(Cockroach Labs co-founder here) CockroachDB also stores SQL metadata inside of the KV store, but that metadata is also gossiped around the cluster (i.e. it is replicated to every node) so that SQL execution almost never has to read it from
14.
by petermattis 10y ago
Hi Chris, we describe CockroachDB as a SQL database because that is what we're aspiring to. The missing functionality (i.e. joins) is on our near-term roadmap.
15.
by petermattis 10y ago
(employee/founder here) Re joins: we have top men working on it right now. Joins are definitely an interesting problem for a distributed database.