6 ms·

Hi! Author of the article here. The core concern is not about the capabilities of the compute abstraction being used (bare metal, containers or functions) or

Hi! Author of the article here. The core concern is not about the capabilities of the compute abstraction being used (bare metal, containers or functions) or testing OS capabilities. The aim is to validate mitigations which are in place to counter turbulent scenarios (For example: massive spike in traffic, network outage, dependency is down, etc). These scenarios generally originate outside the given system. These kind of questions should be asked and systematically validated (quoting the article): * Have you tested how the system behaves when the underlying instances have a sustained CPU spike? * Is the system behavior understood under different stress? * Is there sufficient monitoring? * Have the alarms been validated? * Are there any countermeasures implemented? For example, is auto-scaling set up, and does it behave as expected? Are timeouts and retries appropriate?

fxtentacle 6y ago
I believe we just have a rather different approach here. "Have you tested how the system behaves when the underlying instances have a sustained CPU spike?" Since dedicated boxes are cheap, I'd just buy 5x the CPU resources that I reasonably need and call it a day. If there ever is a more than 5x traffic spike, then docker will prevent it from being a noisy neighbor, so the affected services will just become slower than usual. But even a 10x traffic multiplier would just produce a 2x slowdown, which should be tolerable for most users. I agree that on clouds you want to save costs by only booking what you need. But bare metal, you can usually afford to keep spare capacity around all the time. As such, I wouldn't plan for the system to behave well under stress. I'd try to always have enough resources around so that stress never happens. At the end of the day, this seems like a developer time vs. resource costs trade-off and for most companies, developers are sparse and resources are plentiful, so they'll have a very different trade-off from big FAANG companies. "For example, is auto-scaling set up, and does it behave as expected?" If your system is usually 90% idle, I wonder if you'll ever need that auto-scaling. Also, I'd say my customers can endure it if page load time goes up from 100ms to 200ms. So in my opinion, there is little need for auto-scaling for most companies.
- neo01124 6y ago
  No, there is no different approach. You are misunderstanding what is being addressed in the article. This is not about bare metal vs cloud or autoscaling vs no-autoscaling/overscaling or developer time vs resource costs. The article talks about injecting failures at various points in the system, understanding how the system behaves under this stress, putting in counter-measures for the resulting problems, and eventually re-running this to validate those counter-measures.
  - fxtentacle 6y ago
    I did understand what the article is about. But you only need to worry about failure under stress if you are driving your system close to the hardware's limit. In the virtual cloud world, that is common, because you rent the cheapest instance that will be big enough. In the bare metal world, that is rarely the case, because you usually get a Ryzen with 16+ cores and 128+ GB of RAM. In that case, there's no point in checking what will happen to your 200 MB web app if there's a CPU spike. It'll be just fine because the hardware can handle 10x the load without a hickup. Similarly, if your page load time is dominated by internet latency, it doesn't matter if your CPU needs a few more ms to spit out the page HTML. So there, a 2x CPU usage increase will be barely noticeable to the user.
- ses1984 6y ago
  >"Have you tested how the system behaves when the underlying instances have a sustained CPU spike?" You didn't really address this question, you addressed a different question, which is a traffic spike. >Also, I'd say my customers can endure it if page load time goes up from 100ms to 200ms. So in my opinion, there is little need for auto-scaling for most companies. 100ms to 200ms average? What about the tail? Your app might go from P99 - 500ms to P95 - timeout. That's when you'll lose customers.
  - fxtentacle 6y ago
    If the underlying hardware is a bare metal server, it won't magically turn slow and have a CPU spike. That problem is caused noisy neighbor and kind of exclusive to clouds. Well, with the 2x example, my app might get from a 1s P99 to a 2s P99 which feels slow, but is still doable. Again, those timeouts are usually introduced by cloud infrastructure. For example, if you use nginx outside of Heroku, it won't have a 30s timeout for file downloads.

fxtentacle 6y ago

I believe we just have a rather different approach here. "Have you tested how the system behaves when the underlying instances have a sustained CPU spike?" Since dedicated boxes are cheap, I'd just buy 5x the CPU resources that I reasonably need and call it a day. If there ever is a more than 5x traffic spike, then docker will prevent it from being a noisy neighbor, so the affected services will just become slower than usual. But even a 10x traffic multiplier would just produce a 2x slowdown, which should be tolerable for most users. I agree that on clouds you want to save costs by only booking what you need. But bare metal, you can usually afford to keep spare capacity around all the time. As such, I wouldn't plan for the system to behave well under stress. I'd try to always have enough resources around so that stress never happens. At the end of the day, this seems like a developer time vs. resource costs trade-off and for most companies, developers are sparse and resources are plentiful, so they'll have a very different trade-off from big FAANG companies. "For example, is auto-scaling set up, and does it behave as expected?" If your system is usually 90% idle, I wonder if you'll ever need that auto-scaling. Also, I'd say my customers can endure it if page load time goes up from 100ms to 200ms. So in my opinion, there is little need for auto-scaling for most companies.

neo01124 6y ago
No, there is no different approach. You are misunderstanding what is being addressed in the article. This is not about bare metal vs cloud or autoscaling vs no-autoscaling/overscaling or developer time vs resource costs. The article talks about injecting failures at various points in the system, understanding how the system behaves under this stress, putting in counter-measures for the resulting problems, and eventually re-running this to validate those counter-measures.
- fxtentacle 6y ago
  I did understand what the article is about. But you only need to worry about failure under stress if you are driving your system close to the hardware's limit. In the virtual cloud world, that is common, because you rent the cheapest instance that will be big enough. In the bare metal world, that is rarely the case, because you usually get a Ryzen with 16+ cores and 128+ GB of RAM. In that case, there's no point in checking what will happen to your 200 MB web app if there's a CPU spike. It'll be just fine because the hardware can handle 10x the load without a hickup. Similarly, if your page load time is dominated by internet latency, it doesn't matter if your CPU needs a few more ms to spit out the page HTML. So there, a 2x CPU usage increase will be barely noticeable to the user.
ses1984 6y ago
>"Have you tested how the system behaves when the underlying instances have a sustained CPU spike?" You didn't really address this question, you addressed a different question, which is a traffic spike. >Also, I'd say my customers can endure it if page load time goes up from 100ms to 200ms. So in my opinion, there is little need for auto-scaling for most companies. 100ms to 200ms average? What about the tail? Your app might go from P99 - 500ms to P95 - timeout. That's when you'll lose customers.
- fxtentacle 6y ago
  If the underlying hardware is a bare metal server, it won't magically turn slow and have a CPU spike. That problem is caused noisy neighbor and kind of exclusive to clouds. Well, with the 2x example, my app might get from a 1s P99 to a 2s P99 which feels slow, but is still doable. Again, those timeouts are usually introduced by cloud infrastructure. For example, if you use nginx outside of Heroku, it won't have a 30s timeout for file downloads.