9 ms·
I wonder if Amazon's new managed EFS service would make sense? It's exposed as a NFS mount to the OS. The claims are: - Up to thousands of Amazon EC2 instances
by Rezo 10y ago
I wonder if Amazon's new managed EFS service would make sense? It's exposed as a NFS mount to the OS. The claims are:
- Up to thousands of Amazon EC2 instances, from multiple AZs, can connect concurrently to a file system.
- Data is stored redundantly across multiple AZs.
- Low, consistent latency.
- Multiple GBs per second.
- sytse 10y agoEFS is pretty great. But for our use case there are a few drawbacks: 1. It doesn't scale to the size we need for all repositories, so we still need sharding. 2. It is expensive, $300/TB-month 3. You're still sending all traffic over the network, we prefer the latency of local storage we can achieve by developing Gitaly https://gitlab.com/gitlab-org/gitaly https://gitlab.com/gitlab-org/gitaly
- jamesmiller5 10y agoWhy develop Gitaly instead of using consensus (etcd, zookeeper, etc.) for writes to fast storage like ssds and the normal git binary? As far as I can tell it's adding a lot of complexity by caching high level RPC calls but still doesn't really address multi-master read after write consistency or coordination.