5 ms·

how is it different than s3?

• rco8786 3y ago
  Sounds like they provide a caching layer in between "Every blob uploaded is stored in-memory to provide fast access — and then blobs that are not read frequently are eventually automatically moved to disk for low-cost, long-term storage. When you make a new read request to a blob in disk, it is reloaded back to memory, providing you the fastest possible access again." For a developer-focused tool, it strikes me as odd that they wouldn't just use the term "caching" though?
  • jmarbach 3y ago
    Yes, I agree with your point. It reminds me a little of the ongoing debate over CDN vs Edge. However, one difference we offer is that eventually our 'cache' will flush to a persistent storage layer - so you don't have to think about managing your memory or disk resources. Data not used for weeks ends up in low-cost object storage - further saving you from high-cost memory storage costs and the repetitive task of performing this archiving operation yourself.
    • rco8786 3y ago
      > eventually our 'cache' will flush to a persistent storage layer What happens if the server goes down before the flush?
      • jmarbach 3y ago
        When a blob is saved, first I write it to Postgres (https://neon.tech https://neon.tech), so that ensures there's a persisting backup. However, it's typically a waste of money to store infrequently accessed blobs in disk with Postgres over months and years. After 4-6 weeks data is offloaded to object storage - so that you benefit from low long term storage costs. The lifecycle of a blob works out roughly something like as follows, based on last read date: < 30 mins ago - In-Memory: Cloudflare CDN < 30 days ago - In-Disk: Redis Auto-Tiering Memory/Disk combination - In-Disk: Postgres > 30 days ago - Object Storage: Backblaze B2