7 ms·

Services, or even microservices, are more of a strategy to allow teams to scale than services or products to scale. I think thats one of the biggest misconcepti

by scient 3y ago

Services, or even microservices, are more of a strategy to allow teams to scale than services or products to scale. I think thats one of the biggest misconceptions for engineers. On the other end you have the monorepo crew, who are doing it for the same reasons. On your note about resiliency and scale - its always a waste of money until shit hits the fan. Then you really pay for it.

dns_snek 3y ago
How do you square that with the fact that shit usually hits the fan precisely because of this complexity, not in spite of it? That's my observation & experience, anyway. Added bits of "resiliency" often add brand new, unexplored failure points that are just ticking time bombs waiting to bring the entire system down.
- bluGill 3y ago
  Not adding that resiliency isn't the answer though - it just means known failures will get you. Is that better than the unknown failures because of your mitigation? I cannot answer that. I can tell you 100% that eventually a disk will fail. I can tell you 100% that eventually the power will go out. I can tell you 100% that even if you have a computer with redundant power supplies each connected to separate grids, eventually both power supplies will fail at the same time - it just will happen a lot less often than if you have a regular computer not on any redundant/backup power. I can tell you that network cables do break from time to time. I can tell you that buildings are vulnerable to earthquakes, fires, floods, tornadoes and other such disasters). I can tell you that software is not perfect and eventually crashes. I can tell you that upgrades are hard if any protocol changed. I can tell you there is a long list of other known disasters that I didn't list, but a little research will discover. I could look up the odds of the above. In turn this allows calculating the costs of each mitigation against the likely cost of not mitigating it - but this is only statistical you may decide something statistically cannot happen and it does anyway. What I cannot tell you is how much you should mitigate. There is a cost to each mitigation that need to be compared to the value.
  - camgunz 3y ago
    Absolutely yeah, these things are hard enough to test in a controlled environment with a single app (e.g. FoundationDB) but practically impossible to test fully in a microservices architecture. It's so nice to have this complexity managed for you in the storage layer.
- troupe 3y ago
  > How do you square that with the fact that shit usually hits the fan precisely because of this complexity The theoretical benefit may not be what most teams are going to experience. Usually the fact that microservices are seen as a solution to a problem that could more easily be solved in other much simpler ways, is a pretty good indication that any theoretical benefits are going to be lost through other poor decision making.
- Eridrus 3y ago
  Microservices almost always increase the amount of partial failures, but if used properly can reduce the amount of critical failures. You can certainly misapply the architecture, but you can also apply it well. It's unsurprising that most people make bad choices in a difficult domain.
  - Tainnor 3y ago
    Fault tolerance doesn't necessarily require microservices (as in separate code bases) though, see Erlang. Or even something like Unison. But for some reason it seems that few people are working on making our programming languages and frameworks fault tolerant.
    - Eridrus 3y ago
      Because path dependence is real so we're mostly building on top of a tower of shit. And as computers got faster, it became more reasonable to have huge amounts of overhead. Same reason that docker exists at all.
camgunz 3y ago
> Services, or even microservices, are more of a strategy to allow teams to scale than services or products to scale. I've never really understood why you couldn't just break up your monolith into modules. So like if there's a "payments" section, why isn't that API stabilized? I think all the potential pitfalls (coupling, no commitment to compatibility) are there for monoliths and microservices, the difference is in the processes. For example, microservices export some kind of API over REST/GraphQL/gRPC which they can have SDKs for, they can version them, etc. Why can't you just define interfaces to modules within your monolith? You can generate API docs, you can version interfaces, you can make completely new versions, etc. I just feel like this would be a huge improvement: - It's so much more engineering work to build the service handler scaffolding (validation, serialization/deserialization, defining errors) - You avoid the runtime overhead of serialiation/deserialization and network latency - You don't need to build SDKs/generate protobufs/generate clients/etc. - You never have the problem of "is anyone using this service?" because you can use code coverage tools - Deployment is much, much simpler - You never have the problem of "we have to support this old--sometimes broken--functionality because this old service we can't modify depends on it". This is a really undersold point: maybe it's true that microservice architectures let engineers build things without regard for other teams, but they can't remove things without regard for other teams, and this dynamic is like a no limit credit card for tech debt. Do you keep that service around as it slowly accretes more and more code it can't delete? Do you fork a new service w/o the legacy code and watch your fleet of microservices grow ever larger? - You never have the problem of "how do we update the version of Node on 50 microservices?"
- mjr00 3y ago
  > You never have the problem of "how do we update the version of Node on 50 microservices?" And instead you have the problem of "how do we update the version of Node on our 10 million LOC codebase?" Which is, in my experience, an order of magnitude harder. Ease of upgrading the underlying platform versions of Node, Python, Java, etc is one of the biggest benefits of smaller, independent services.
  - naasking 3y ago
    Upgrading the platform also happens at least 10x less frequently, so that math doesn't necessarily work out in your favour though.
manicennui 3y ago
Except that most people build microservices in a way that ignores the reality of cloud providers and the fact that they are building (more) distributed systems, and often end up with lower resiliency.
danielovichdk 3y ago
Microservices is more about organisation than it is about technology. And that is why developers have so much trouble getting it right. They can't without having the organisational fundamentals in place. It is simply not possible. The architectural constraints of microservices will show the organisational weaknesses in a much higher rate because of the pressure it puts on having the organisation be very strict about ownership, communication and autonomy. The takes a higher level of maturity as an organisation to enable the benefits of microservies, which is also why most organisations shouldn't even try. Stop all the technical nonsense because it won't solve the root cause of the matter. It's the organisation. Not the technology

by scient 3y ago

dns_snek 3y ago

How do you square that with the fact that shit usually hits the fan precisely because of this complexity, not in spite of it? That's my observation & experience, anyway. Added bits of "resiliency" often add brand new, unexplored failure points that are just ticking time bombs waiting to bring the entire system down.

bluGill 3y ago
Not adding that resiliency isn't the answer though - it just means known failures will get you. Is that better than the unknown failures because of your mitigation? I cannot answer that. I can tell you 100% that eventually a disk will fail. I can tell you 100% that eventually the power will go out. I can tell you 100% that even if you have a computer with redundant power supplies each connected to separate grids, eventually both power supplies will fail at the same time - it just will happen a lot less often than if you have a regular computer not on any redundant/backup power. I can tell you that network cables do break from time to time. I can tell you that buildings are vulnerable to earthquakes, fires, floods, tornadoes and other such disasters). I can tell you that software is not perfect and eventually crashes. I can tell you that upgrades are hard if any protocol changed. I can tell you there is a long list of other known disasters that I didn't list, but a little research will discover. I could look up the odds of the above. In turn this allows calculating the costs of each mitigation against the likely cost of not mitigating it - but this is only statistical you may decide something statistically cannot happen and it does anyway. What I cannot tell you is how much you should mitigate. There is a cost to each mitigation that need to be compared to the value.
- camgunz 3y ago
  Absolutely yeah, these things are hard enough to test in a controlled environment with a single app (e.g. FoundationDB) but practically impossible to test fully in a microservices architecture. It's so nice to have this complexity managed for you in the storage layer.
troupe 3y ago
> How do you square that with the fact that shit usually hits the fan precisely because of this complexity The theoretical benefit may not be what most teams are going to experience. Usually the fact that microservices are seen as a solution to a problem that could more easily be solved in other much simpler ways, is a pretty good indication that any theoretical benefits are going to be lost through other poor decision making.
Eridrus 3y ago
Microservices almost always increase the amount of partial failures, but if used properly can reduce the amount of critical failures. You can certainly misapply the architecture, but you can also apply it well. It's unsurprising that most people make bad choices in a difficult domain.
- Tainnor 3y ago
  Fault tolerance doesn't necessarily require microservices (as in separate code bases) though, see Erlang. Or even something like Unison. But for some reason it seems that few people are working on making our programming languages and frameworks fault tolerant.
  - Eridrus 3y ago
    Because path dependence is real so we're mostly building on top of a tower of shit. And as computers got faster, it became more reasonable to have huge amounts of overhead. Same reason that docker exists at all.

camgunz 3y ago

> Services, or even microservices, are more of a strategy to allow teams to scale than services or products to scale. I've never really understood why you couldn't just break up your monolith into modules. So like if there's a "payments" section, why isn't that API stabilized? I think all the potential pitfalls (coupling, no commitment to compatibility) are there for monoliths and microservices, the difference is in the processes. For example, microservices export some kind of API over REST/GraphQL/gRPC which they can have SDKs for, they can version them, etc. Why can't you just define interfaces to modules within your monolith? You can generate API docs, you can version interfaces, you can make completely new versions, etc. I just feel like this would be a huge improvement: - It's so much more engineering work to build the service handler scaffolding (validation, serialization/deserialization, defining errors) - You avoid the runtime overhead of serialiation/deserialization and network latency - You don't need to build SDKs/generate protobufs/generate clients/etc. - You never have the problem of "is anyone using this service?" because you can use code coverage tools - Deployment is much, much simpler - You never have the problem of "we have to support this old--sometimes broken--functionality because this old service we can't modify depends on it". This is a really undersold point: maybe it's true that microservice architectures let engineers build things without regard for other teams, but they can't remove things without regard for other teams, and this dynamic is like a no limit credit card for tech debt. Do you keep that service around as it slowly accretes more and more code it can't delete? Do you fork a new service w/o the legacy code and watch your fleet of microservices grow ever larger? - You never have the problem of "how do we update the version of Node on 50 microservices?"

mjr00 3y ago
> You never have the problem of "how do we update the version of Node on 50 microservices?" And instead you have the problem of "how do we update the version of Node on our 10 million LOC codebase?" Which is, in my experience, an order of magnitude harder. Ease of upgrading the underlying platform versions of Node, Python, Java, etc is one of the biggest benefits of smaller, independent services.
- naasking 3y ago
  Upgrading the platform also happens at least 10x less frequently, so that math doesn't necessarily work out in your favour though.

manicennui 3y ago

Except that most people build microservices in a way that ignores the reality of cloud providers and the fact that they are building (more) distributed systems, and often end up with lower resiliency.

danielovichdk 3y ago

Microservices is more about organisation than it is about technology. And that is why developers have so much trouble getting it right. They can't without having the organisational fundamentals in place. It is simply not possible. The architectural constraints of microservices will show the organisational weaknesses in a much higher rate because of the pressure it puts on having the organisation be very strict about ownership, communication and autonomy. The takes a higher level of maturity as an organisation to enable the benefits of microservies, which is also why most organisations shouldn't even try. Stop all the technical nonsense because it won't solve the root cause of the matter. It's the organisation. Not the technology