5 ms·

Wherein Stonebraker misunderstands distributed systems engineering completely.

ora600 16y ago
I understood what he said as: Solving the problems of distributed systems is incredibly hard and not necessary if all you need is scalability and high availability. Better think of your problems in terms of engineering trade-offs and not distributed theory and understand what you are giving up and what this gains you. When you decide to give up consistency, your application can no longer assume consistency ever. Giving up availability in case of network partition means few extra minutes of downtime a year. I don't think he completely misunderstood distributed systems - I think he decided to completely side-step the entire field.
- jhugg 16y ago
  Nice summary.
- moonpolysoft 16y ago
  > Solving the problems of distributed systems is incredibly hard and not necessary if all you need is scalability and high availability. Once one scales beyond a single node the system becomes distributed. Then by definition one must deal with distributed systems problems in order to achieve scale beyond the capabilities of a single node. > When you decide to give up consistency, your application can no longer assume consistency ever. Giving up availability in case of network partition means few extra minutes of downtime a year. Depends on the application. Giving up availability might mean cascading failures throughout your entire application. For instance if the datastore is unavailable for writes then any kind of queueing systems built around the DB (a common design pattern) run the risk of overflow during the downtime. And I would make the argument that once an application scales beyond a single datacenter it cannot help but give up strict consistency under error conditions. > I don't think he completely misunderstood distributed systems - I think he decided to completely side-step the entire field. If he didn't misunderstand them then he is purposefully ignoring the hard problems. Which is worse?
  - ora600 16y ago
    > Once one scales beyond a single node the system becomes distributed. Then by definition one must deal with distributed systems problems in order to achieve scale beyond the capabilities of a single node I meant that one doesn't need to solve the general problem of distributed systems. Sharding is a common way to scale avoiding most of the problems generally associated with distributed systems. Scaling within LAN is easier than across data centers. You can assume no malicious traffic between your servers and suddenly solving the byzantine generals problem is far easier. Purposefully ignoring really hard problems can be a very good engineering practice.
    - ieure 16y ago
      Cool story, bro.
    - benblack 16y ago
      I purposefully ignored the tornado and so it did not hit my datacenter, tear off a section of the roof, kill all power sources, and drench my servers. Hard problems: solved.
      - sophacles 16y ago
        Thanks! My app was in that datacenter too. I mean, it had replicated mongodb instances, and well balanced app servers, and nodes going away have no discernable affect on users. Turns out tho, that with all that distributed engineering, I didn't find out that the hosting company doesn't put your nodes in various data-centers. That tornado would have taken down my service during peak hours. I know you will try to write that off as a "you get what you diserve" but I challenge you to go ask people if their apps would survive a tornado to the data-center. Many of them will say "sure its in the cloud!" Then drop the killer question on them... "How many different data centers are your nodes running on right now". Most will say "i dont know". Some will say "My host has many data centers" (note this doesn't answer the question). A few will actually have done the footwork. Also, the scenario you describe is as easily mitigated with hot failovers and offsite backups. This probably qualifies as distributed engineering, but only is only the same as the above discussions in the most pedantic senses.

I understood what he said as: Solving the problems of distributed systems is incredibly hard and not necessary if all you need is scalability and high availability. Better think of your problems in terms of engineering trade-offs and not distributed theory and understand what you are giving up and what this gains you. When you decide to give up consistency, your application can no longer assume consistency ever. Giving up availability in case of network partition means few extra minutes of downtime a year. I don't think he completely misunderstood distributed systems - I think he decided to completely side-step the entire field.

jhugg 16y ago
Nice summary.
moonpolysoft 16y ago
> Solving the problems of distributed systems is incredibly hard and not necessary if all you need is scalability and high availability. Once one scales beyond a single node the system becomes distributed. Then by definition one must deal with distributed systems problems in order to achieve scale beyond the capabilities of a single node. > When you decide to give up consistency, your application can no longer assume consistency ever. Giving up availability in case of network partition means few extra minutes of downtime a year. Depends on the application. Giving up availability might mean cascading failures throughout your entire application. For instance if the datastore is unavailable for writes then any kind of queueing systems built around the DB (a common design pattern) run the risk of overflow during the downtime. And I would make the argument that once an application scales beyond a single datacenter it cannot help but give up strict consistency under error conditions. > I don't think he completely misunderstood distributed systems - I think he decided to completely side-step the entire field. If he didn't misunderstand them then he is purposefully ignoring the hard problems. Which is worse?
- ora600 16y ago
  > Once one scales beyond a single node the system becomes distributed. Then by definition one must deal with distributed systems problems in order to achieve scale beyond the capabilities of a single node I meant that one doesn't need to solve the general problem of distributed systems. Sharding is a common way to scale avoiding most of the problems generally associated with distributed systems. Scaling within LAN is easier than across data centers. You can assume no malicious traffic between your servers and suddenly solving the byzantine generals problem is far easier. Purposefully ignoring really hard problems can be a very good engineering practice.
  - ieure 16y ago
    Cool story, bro.
  - benblack 16y ago
    I purposefully ignored the tornado and so it did not hit my datacenter, tear off a section of the roof, kill all power sources, and drench my servers. Hard problems: solved.
    - sophacles 16y ago
      Thanks! My app was in that datacenter too. I mean, it had replicated mongodb instances, and well balanced app servers, and nodes going away have no discernable affect on users. Turns out tho, that with all that distributed engineering, I didn't find out that the hosting company doesn't put your nodes in various data-centers. That tornado would have taken down my service during peak hours. I know you will try to write that off as a "you get what you diserve" but I challenge you to go ask people if their apps would survive a tornado to the data-center. Many of them will say "sure its in the cloud!" Then drop the killer question on them... "How many different data centers are your nodes running on right now". Most will say "i dont know". Some will say "My host has many data centers" (note this doesn't answer the question). A few will actually have done the footwork. Also, the scenario you describe is as easily mitigated with hot failovers and offsite backups. This probably qualifies as distributed engineering, but only is only the same as the above discussions in the most pedantic senses.