lxpz

7 ms·

1.
by lxpz 7mo ago
Garage does actually support CORS (via PutBucketCORS). If there is anything specific missing, would you be wiling to open an issue on our issue tracker so that we can track it as a feature request?
2.
by lxpz 9mo ago
Very interesting, thank you. It would probably make sense for most tables but not all of them because some are holding large CRDT values.
3.
by lxpz 9mo ago
I had a very rapid look at these two papers, it looks like none of them allow the implementation of compare-and-swap, which is required for if-match / if-none-match support. They have a weaker definition of a "transaction". W
4.
by lxpz 9mo ago
Fair enough, we will work on making the documentation clearer.
5.
by lxpz 9mo ago
So the thing is, different KV stores have different trade-offs, and for now we haven't yet found one that has the best of all worlds. We do recommend SQLite in our quick-start guide to setup a single-node deployment for small/mode
6.
by lxpz 9mo ago
Garage already shards data by itself if you add more nodes, and it is indeed a viable path to increasing throughput.
7.
by lxpz 9mo ago
If I make certain assumptions and you respect them, I will give you certain guarantees. If you don't respect them, I won't guarantee anything. I won't guarantee that your data will be toast either.
8.
by lxpz 9mo ago
They are not solving the same problem. Syncthing will synchronize a full folder between an arbitrary number of machines, but you still have to access this folder one way or another. Garage provides an HTTP API for your data, and handles int
9.
by lxpz 9mo ago
We were not able to get good enough performance compared to LMDB. We will work on this more though, there are probably many ways performance can be increased by reducing load on the KV store.
10.
by lxpz 9mo ago
If you have replication, you can lose one of the replica, that's the point. This is what Garage was designed for, and it works. Erasure coding is another debate, for now we have chosen not to implement it, but I would personally be ope
11.
by lxpz 9mo ago
It does not, at least not for a small local dev server. I believe RAM usage should be around 50-100MB, increasing if you have many requests with large objects.
12.
by lxpz 9mo ago
If you know of an embedded key-value store that supports transactions, is fast, has good Rust bindings, and does checksumming/integrity verification by default such that it almost never corrupts upon power loss (or at least, is always
13.
by lxpz 9mo ago
Thank you for your feedback, we will take it into account.
14.
by lxpz 9mo ago
I talked about the meaning of the Jepsen test and the results we obtained in the FOSDEM'24 talk: https://archive.fosdem.org/2024/schedule/event/fosdem-2024-3... Slides are available here: https:/&#
15.
by lxpz 9mo ago
Read-after-write consistency : yes (after PutObject has finished, the object will be immediately visible in all subsequent requests, including GetObject and ListObjects) Conditionnal writes : no, we can't do it with CRDTs, which are th
16.
by lxpz 9mo ago
Losing a node is a regular occurrence, and a scenario for which Garage has been designed. The assumption Garage makes, which is well-documented, is that of 3 replica nodes, only 1 will be in a crash-like situation at any time. With 1 crashe
17.
by lxpz 1y ago
If I can reassure you about Garage, it's not at all abandoned. We have active work going on to make a GUI for cluster administration, and we have applied for a new round of funding for more low-level work on performance, which should k
18.
by lxpz 1y ago
Doesn't Haskell do this to some degree with the IO monad? Functions that are not supposed to do IO directly simply have a more specific type signature, like taking in a stream and returning a buffer for example.
19.
by lxpz 2y ago
https://openwrt.org/toh/tp-link/er605_v2?s[]=er605v2 This one doesn't have WiFi, just 5 Ethernet ports. It can be flashed openwrt with a very reasonable amount of tweaking. It's actually quite powerful a
20.
by lxpz 2y ago
This is due to SQLite calling fsync a lot by default, to be on the safe side. You can use pragma journal_mode = "wal" and pragma synchronous = "off" and it should be much faster, without risking corruption if your server
21.
by lxpz 2y ago
Yes! My desktop PC is almost 10 years old at this point and it works perfectly, I've upgraded the CPU to an i7-4790K (the best of that generation) and added some RAM (32G total) which makes it very good for most of my work and even for
22.
by lxpz 3y ago
I think as long as your construction is deterministic, both options are possible: you can either add the intermediate nodes that have only one children, or you can skip adding them and allow links between non-consecutive levels. In that sec
23.
by lxpz 3y ago
As the main author of the Merkle Search Tree paper, I'm really glad to see the design become more known and start to be used. I did not go on and build a practical system that made use of the structure, although I originally came up wi
24.
Dual-booting NixOS and Alpine Linux without root partitions(adnab.me)
3 points by lxpz 3y ago|0 comments
25.
by lxpz 4y ago
Very interesting project and impressive work, thanks for sharing! Can you talk a bit about how data is replicated between nodes when Stalwart is run in clustered mode, and what kind of data integrity/resilience properties we have when
26.
by lxpz 4y ago
For back-end systems and distributed systems, I'd love to work on a new OS based on a microkernel and capabilities. Ditch Linux+Docker+Kubernetes, replace that with a microkernel that has a minimal set of modules to multiplex storage a
27.
by lxpz 5y ago
> Can you do conditional writes? If by that you mean something like compare-and-swap or test-and-set, then no, that would require consensus number > 1 [0] > Can old versions of a register be compacted to reduce metadata space? Yes,
28.
by lxpz 5y ago
I mean, sure, but we're operating under the premise that we don't want to give out our data to a cloud provider and instead want to store data ourselves. Half of the post is dedicated to explaining that, so pretending that you can
29.
by lxpz 5y ago
Then you didn't look close enough. Have a look at the .drone.yml to see the entry points, we have both unit tests and integration tests that run at each build.
30.
by lxpz 5y ago
You probably want at least some redundancy otherwise you're at risk of losing data as soon as a single one of your hard drive fails. However if you're interested in maximizing capacity, you probably need erasure coding, which Ga

More ›