6 ms·

It's interesting that Netflix decided to write this rather than using Amazon's own DynamoDB. I wonder specifically if DynamoDB was too expensive (as I have foun

by jlafon 12y ago

It's interesting that Netflix decided to write this rather than using Amazon's own DynamoDB. I wonder specifically if DynamoDB was too expensive (as I have found), or was there some other reason?

jetblackio 12y ago
This is completely different than DynamoDB. Essentially this is a framework for sharding or horizontally scaling multiple datastores (MySQL, Memcache, Redis, etc).
- jlafon 12y ago
  That makes sense. I had only seen the Memcache & Redis datastores being used (and both being comparable to DynamoDB).
- amelius 12y ago
  Why would you use multiple datastores? Isn't it much more difficult to atomically store data this way? Assume a transaction succeeds in store 1, and fails in store 2, you'd need to roll back the transaction in store 1 after it has committed (!)
  - earless1 12y ago
    Because there is no such thing as a one size fits all datasotre. Some data may make way more sense stored in a relational database, while some data makes more sense as key:value pairs. Some data may be infrequently accessed and stored to disk while others are stored in memory for quick access. Trying to make a single datastore work for many use cases will cause more pain than it is worth.
  - michaelmior 12y ago
    You wouldn't likely use multiple data stores for the same data. That is, you probably wouldn't have a single system which stores data in both Redis and MySQL. However, you can imagine different use cases for each of these stores and you can potentially increase operational efficiency by having a common layer which handles the distribution of data for each store.
fizx 12y ago
This is the sort of project you do to get cross-region replication on your existing data stores, many of which look more like PostgreSQL than KV stores.
grogers 12y ago
Keep in mind that they define DC := AWS region Rack := AWS AZ That should tell you that they need multi-region availability, something that DynamoDB will likely never provide.

by jlafon 12y ago

It's interesting that Netflix decided to write this rather than using Amazon's own DynamoDB. I wonder specifically if DynamoDB was too expensive (as I have found), or was there some other reason?

jetblackio 12y ago

This is completely different than DynamoDB. Essentially this is a framework for sharding or horizontally scaling multiple datastores (MySQL, Memcache, Redis, etc).

jlafon 12y ago
That makes sense. I had only seen the Memcache & Redis datastores being used (and both being comparable to DynamoDB).
amelius 12y ago
Why would you use multiple datastores? Isn't it much more difficult to atomically store data this way? Assume a transaction succeeds in store 1, and fails in store 2, you'd need to roll back the transaction in store 1 after it has committed (!)
- earless1 12y ago
  Because there is no such thing as a one size fits all datasotre. Some data may make way more sense stored in a relational database, while some data makes more sense as key:value pairs. Some data may be infrequently accessed and stored to disk while others are stored in memory for quick access. Trying to make a single datastore work for many use cases will cause more pain than it is worth.
- michaelmior 12y ago
  You wouldn't likely use multiple data stores for the same data. That is, you probably wouldn't have a single system which stores data in both Redis and MySQL. However, you can imagine different use cases for each of these stores and you can potentially increase operational efficiency by having a common layer which handles the distribution of data for each store.

fizx 12y ago

This is the sort of project you do to get cross-region replication on your existing data stores, many of which look more like PostgreSQL than KV stores.

grogers 12y ago

Keep in mind that they define DC := AWS region Rack := AWS AZ That should tell you that they need multi-region availability, something that DynamoDB will likely never provide.