12 ms·

Why You Need a Strategic Data Service

27 points by jasonkolb 12y ago

abekarpinski 12y ago
Predictive analytic is the thing I'm most interested in right now. Trying to use past data to predict the success of future product launches. I can say that having your data decoupled will save you some major headaches later on.
VonIgelfeld 12y ago
Now I understand the value of data, but what most companies miss is the value of consumer privacy. We like our privacy and will pay for it if we need to.
- ianfernz 12y ago
  Typical data capture and analytics isn't like Prism. Seriously, you overestimate your own importance. Facebook doesn't care what you individually do, it's all about collecting statistics about everyone so they can better reach the groups you're a part of.
  - hengheng 12y ago
    I know, and I'd rather the service be paid for by myself than by a secret service.
mdda 12y ago
Where did the footnotes go?
k0 12y ago
Great topic Jason. It seems the footnote links [2], [3], etc are missing something...I'm getting "File Not Found"/custom 404.
- cylentwolf 12y ago
  I had the same issue and was coming here to post the same thing.
jrullmann 12y ago
The author says that different data stores are good at different things, so we need to use multiple data stores in our application. He proposes that a data service layer can abstract these implementations, making it easier to swap out data stores as needed. I think that's a good idea. Separating our applications from data store specifics is a big reason why we use ORMs and ODMs today. However, I think there are challenges with this polygot data store architecture that he doesn't address. Each addition requires due diligence to understand its CAP trade-offs (which the author mentions briefly), scalability and performance characteristics, how to configure, etc. These are non-trivial concerns even for a single database. It's important to consider these challenges when building out a data store or data service. I'd propose an architecture where the data services layer itself exposes different data models to the application, all of which are persisted in a single data store. Given that many data stores use a key-value store under the covers anyway, translating the specific data model down to a single, persistent data store would simplify operations while exposing the desired data model to the application. (As a caveat, this multi-model approach requires ACID transactions to ensure strict consistency when translating between data models.) This approach provides operational simplicity with just one data store and application efficiency by exposing the "right" data model API. Full disclosure: I'm an engineer at FoundationDB, a database that provides ACID-compliant polygot data models on a single key-value data store (http://www.foundationdb.com http://www.foundationdb.com).

27 points by jasonkolb 12y ago

Predictive analytic is the thing I'm most interested in right now. Trying to use past data to predict the success of future product launches. I can say that having your data decoupled will save you some major headaches later on.

VonIgelfeld 12y ago

Now I understand the value of data, but what most companies miss is the value of consumer privacy. We like our privacy and will pay for it if we need to.

ianfernz 12y ago
Typical data capture and analytics isn't like Prism. Seriously, you overestimate your own importance. Facebook doesn't care what you individually do, it's all about collecting statistics about everyone so they can better reach the groups you're a part of.
- hengheng 12y ago
  I know, and I'd rather the service be paid for by myself than by a secret service.

mdda 12y ago

Where did the footnotes go?

k0 12y ago

Great topic Jason. It seems the footnote links [2], [3], etc are missing something...I'm getting "File Not Found"/custom 404.

cylentwolf 12y ago
I had the same issue and was coming here to post the same thing.

jrullmann 12y ago

The author says that different data stores are good at different things, so we need to use multiple data stores in our application. He proposes that a data service layer can abstract these implementations, making it easier to swap out data stores as needed. I think that's a good idea. Separating our applications from data store specifics is a big reason why we use ORMs and ODMs today. However, I think there are challenges with this polygot data store architecture that he doesn't address. Each addition requires due diligence to understand its CAP trade-offs (which the author mentions briefly), scalability and performance characteristics, how to configure, etc. These are non-trivial concerns even for a single database. It's important to consider these challenges when building out a data store or data service. I'd propose an architecture where the data services layer itself exposes different data models to the application, all of which are persisted in a single data store. Given that many data stores use a key-value store under the covers anyway, translating the specific data model down to a single, persistent data store would simplify operations while exposing the desired data model to the application. (As a caveat, this multi-model approach requires ACID transactions to ensure strict consistency when translating between data models.) This approach provides operational simplicity with just one data store and application efficiency by exposing the "right" data model API. Full disclosure: I'm an engineer at FoundationDB, a database that provides ACID-compliant polygot data models on a single key-value data store (http://www.foundationdb.com http://www.foundationdb.com).