4 ms·

FoundationDB has been growing as my favorite database lately. Even though it is only key-value store. Out of curiosity: what are the scale limits of Foundation

by Nican 1y ago

FoundationDB has been growing as my favorite database lately. Even though it is only key-value store. Out of curiosity: what are the scale limits of FoundationDB? What kind of issues would it start to have? For example, being able to store all of Discord messages on it? I see blog posts of Discord moving to Scylla and ElasticSearch, but I wonder if there would be any difficulties here.

heipei 1y ago
ScyllaDB discontinued it's free and open source version, so I personally wouldn't build anything new on it.
hardwaresofton 1y ago
Note that FDB can support other paradigms on top of KV https://foundationdb.github.io/fdb-record-layer/SQL_Reference.html https://foundationdb.github.io/fdb-record-layer/SQL_Referenc... Also IIRC Apple uses FDB at tremendous scale: https://read.engineerscodex.com/p/how-apple-built-icloud-to-store-billions https://read.engineerscodex.com/p/how-apple-built-icloud-to-...
eatonphil 1y ago
There are a lot of strict limits so AFAIK everyone uses FoundationDB for fast, consistent, highly-available metadata while doing replication/storage of actual data elsewhere (such as in S3). https://apple.github.io/foundationdb/known-limitations.html https://apple.github.io/foundationdb/known-limitations.html That is to say it's more like part of your solution and not the entire stack on its own.
- Nican 1y ago
  Yeah. I read over that page, and everything seems sensible. The only part that is not well explained is that "FoundationDB has been tested with databases up to 100 TB".
  - eatonphil 1y ago
    In their data modelling pages they mention you should break up rows into separate keys per column. Or separate keys per field in a document. This is indeed how many databases model rows on a distributed kv store. So this might be how they achieved 100TB. However you still have the issue of any single key-value needing to be in their limit. (But it's not like people typically store enormous blobs in Postgres or MySQL either I think?)
  - Dave_Rosenthal 1y ago
    The documentation is woefully out of date, sadly. Despite the code being in active development no one is touching the public docs. Though I don’t know for sure, that limitation was probably written something like 10 years ago.

by Nican 1y ago

heipei 1y ago

ScyllaDB discontinued it's free and open source version, so I personally wouldn't build anything new on it.

hardwaresofton 1y ago

Note that FDB can support other paradigms on top of KV https://foundationdb.github.io/fdb-record-layer/SQL_Reference.html https://foundationdb.github.io/fdb-record-layer/SQL_Referenc... Also IIRC Apple uses FDB at tremendous scale: https://read.engineerscodex.com/p/how-apple-built-icloud-to-store-billions https://read.engineerscodex.com/p/how-apple-built-icloud-to-...

eatonphil 1y ago

There are a lot of strict limits so AFAIK everyone uses FoundationDB for fast, consistent, highly-available metadata while doing replication/storage of actual data elsewhere (such as in S3). https://apple.github.io/foundationdb/known-limitations.html https://apple.github.io/foundationdb/known-limitations.html That is to say it's more like part of your solution and not the entire stack on its own.

Nican 1y ago
Yeah. I read over that page, and everything seems sensible. The only part that is not well explained is that "FoundationDB has been tested with databases up to 100 TB".
- eatonphil 1y ago
  In their data modelling pages they mention you should break up rows into separate keys per column. Or separate keys per field in a document. This is indeed how many databases model rows on a distributed kv store. So this might be how they achieved 100TB. However you still have the issue of any single key-value needing to be in their limit. (But it's not like people typically store enormous blobs in Postgres or MySQL either I think?)
- Dave_Rosenthal 1y ago
  The documentation is woefully out of date, sadly. Despite the code being in active development no one is touching the public docs. Though I don’t know for sure, that limitation was probably written something like 10 years ago.