7 ms·

FTA: "...I decided to migrate the existing system into a single table setup." "Alternative Proposed Solutions: MySQL Partitioning and Redis" I'm surprised h

by dstorrs 14y ago

FTA: "...I decided to migrate the existing system into a single table setup." "Alternative Proposed Solutions: MySQL Partitioning and Redis" I'm surprised he didn't consider at Mongo, Couch, etc.

derwiki 14y ago
My coworkers and I have absolutely zero experience with Mongo, Couch, etc. We do have experience with MySQL and Redis (see https://github.com/causes/mock_redis https://github.com/causes/mock_redis), and I feel "safer" keeping important data in something as mature as MySQL.
- whalesalad 14y ago
  I've heard so many stories of people moving away from MongoDB once they reach scale like yours, I don't blame you.
- bsg75 14y ago
  Is there a pattern for putting multiple Redis "collectors" in front of a single RDBMS aggregate / history store?
james4k 14y ago
One of the shockers I came across with MongoDB is that each instance of a key takes up memory. There is no form of a symbol table for the keys, so this means a huge amount of data overhead if each 'row' of data uses keys at all, which they likely do. No one just uses arrays.
- taligent 14y ago
  Not really sure what you mean about the rows given that it is JSON but anyway. I think what you are referring to is the tokenization of field names: https://jira.mongodb.org/browse/SERVER-863 https://jira.mongodb.org/browse/SERVER-863
  - bsg75 14y ago
    As in an INFORMATION_SCHEMA collection ?
  - james4k 14y ago
    Right, what I really meant was a document. I was trying to relate it to SQL, but I may have made it more confusing than anything. That issue is what I was referring to, and it would be a good step forward. However, it's a shame you have the overhead of field names in the first place. I understand why it is like it is, being schemaless, and I suppose in terms of scaling it isn't a huge issue, as the overhead scales linearly which is manageable. But, in most cases, it's still huge compared to the size of the data itself. I'm not sure how they'll fix it..and I don't know much about other schemaless DBs, but perhaps some sort of pattern recognization would be appropriate. Now that MongoDB has lots of funding for research, it will be interesting to see what they come up with.

by dstorrs 14y ago

FTA: "...I decided to migrate the existing system into a single table setup." "Alternative Proposed Solutions: MySQL Partitioning and Redis" I'm surprised he didn't consider at Mongo, Couch, etc.

derwiki 14y ago

My coworkers and I have absolutely zero experience with Mongo, Couch, etc. We do have experience with MySQL and Redis (see https://github.com/causes/mock_redis https://github.com/causes/mock_redis), and I feel "safer" keeping important data in something as mature as MySQL.

whalesalad 14y ago
I've heard so many stories of people moving away from MongoDB once they reach scale like yours, I don't blame you.
bsg75 14y ago
Is there a pattern for putting multiple Redis "collectors" in front of a single RDBMS aggregate / history store?

james4k 14y ago

One of the shockers I came across with MongoDB is that each instance of a key takes up memory. There is no form of a symbol table for the keys, so this means a huge amount of data overhead if each 'row' of data uses keys at all, which they likely do. No one just uses arrays.

taligent 14y ago
Not really sure what you mean about the rows given that it is JSON but anyway. I think what you are referring to is the tokenization of field names: https://jira.mongodb.org/browse/SERVER-863 https://jira.mongodb.org/browse/SERVER-863
- bsg75 14y ago
  As in an INFORMATION_SCHEMA collection ?
- james4k 14y ago
  Right, what I really meant was a document. I was trying to relate it to SQL, but I may have made it more confusing than anything. That issue is what I was referring to, and it would be a good step forward. However, it's a shame you have the overhead of field names in the first place. I understand why it is like it is, being schemaless, and I suppose in terms of scaling it isn't a huge issue, as the overhead scales linearly which is manageable. But, in most cases, it's still huge compared to the size of the data itself. I'm not sure how they'll fix it..and I don't know much about other schemaless DBs, but perhaps some sort of pattern recognization would be appropriate. Now that MongoDB has lots of funding for research, it will be interesting to see what they come up with.