6 ms·
It's a complex question. I had experience of working with ~60petabytish system back in 2016, and there a lot of things to cover (not only storage): * network a
by itroot 5y ago
It's a complex question. I had experience of working with ~60petabytish system back in 2016, and there a lot of things to cover (not only storage):
* network access - do you have data that will be accessed frequently, and with high traffic? You need to cover this skewed access pattern in your solution.
* data migration from one node to another, etc...
* ability to restore quickly in case of failure.
I would suggest to:
* use some open-source solution on top of the hosted infrastructure (Hetzner or similar is a good choice)
* bring in a seasoned expert to analyze your data usage/storage patterns, maybe there are some other ways to make storage more cost effective, that simply moving out of AWS S3.