17 ms·

I'm not familiar with HAProxy, but with other load balancers. I've been looking into it as a replacement for the hardware load balancers we use. Why do you ha

I'm not familiar with HAProxy, but with other load balancers. I've been looking into it as a replacement for the hardware load balancers we use. Why do you have to reload haproxy? When you update the configuration? I reload nginx all the time (nginx -s reload) and I'm not sure if that is a true zero-downtime reload either. Interesting hack nonetheless (stopping SYNs.)

fxx2 11y ago
Reloading HAProxy is part of the SmartStack architecture, IIRC. SmartStack/synapse does polling on say, ZooKeeper or Docker or whatever, generates a new HAProxy configuration and then reloads it.
- e12e 11y ago
  That's an important detail, as it explains why the tests aren't entirely flawed: Testing 10 reloads every second really distorts the numbers, assuming a reload every hour, or every few hours is more realistic. But if reloads are part of what amounts to doing live testing on a large part of live traffic -- then the exercise makes more sense. And to be sure; being able to do ten reloads every second with few ill effects enables different, more nimble systems engineering. But if we assume 2000 requests per second, per box - fighting ~100 reset connections a day (assuming two ha-proxy reconfigs) doesn't really seem worth the effort - packet loss and other outages would probably(?) domminate anyway.
  - eropple 11y ago
    > Testing 10 reloads every second really distorts the numbers, assuming a reload every hour, or every few hours is more realistic. It depends on what you do. I've seen shops (successfully and, IMO, correctly) scaling AWS instances for services with a threshhold of every fifteen minutes, and I've seen Mesos clusters dynamically spinning up web instances much more nimbly than that (think every two minutes under spiky load--the instances would come up in five seconds, so it didn't hurt to down them).
    - e12e 11y ago
      Well, once every 120 seconds is still quite a leap from 10 every second...
      - eropple 11y ago
        Sure, but if it doesn't work there, I don't trust it to work if, say, a piece of my scheduler goes nuts and suddenly is upping and downing containers every few seconds. The problem remains, it's just not as acute and still must be fixed.
Thaxll 11y ago
You should read that, it's very interesting: http://nginx.org/en/docs/control.html http://nginx.org/en/docs/control.html
fideloper 11y ago
Right the configuration is loaded into memory, and you need to reload it to get any changes in. I believe it's API let's you enable/disable existing configured servers but not dynamically add or remove them.
jfroma 11y ago
It is. Nginx launch a new worker while the other with the old config is still running, the master redirects new traffic to the worker and keeps the old worker until all previous requests has been handled. Once the old worker has finished the master process close it or when it time outs. It works even for updating nginx binaries. It is a very useful approach and I use it all time as well. I implemented the same in node by using the cluster api [1]. [1]: http://joseoncode.com/2015/01/18/reloading-node-with-no-downtime/ http://joseoncode.com/2015/01/18/reloading-node-with-no-down...

fxx2 11y ago

Reloading HAProxy is part of the SmartStack architecture, IIRC. SmartStack/synapse does polling on say, ZooKeeper or Docker or whatever, generates a new HAProxy configuration and then reloads it.

e12e 11y ago
That's an important detail, as it explains why the tests aren't entirely flawed: Testing 10 reloads every second really distorts the numbers, assuming a reload every hour, or every few hours is more realistic. But if reloads are part of what amounts to doing live testing on a large part of live traffic -- then the exercise makes more sense. And to be sure; being able to do ten reloads every second with few ill effects enables different, more nimble systems engineering. But if we assume 2000 requests per second, per box - fighting ~100 reset connections a day (assuming two ha-proxy reconfigs) doesn't really seem worth the effort - packet loss and other outages would probably(?) domminate anyway.
- eropple 11y ago
  > Testing 10 reloads every second really distorts the numbers, assuming a reload every hour, or every few hours is more realistic. It depends on what you do. I've seen shops (successfully and, IMO, correctly) scaling AWS instances for services with a threshhold of every fifteen minutes, and I've seen Mesos clusters dynamically spinning up web instances much more nimbly than that (think every two minutes under spiky load--the instances would come up in five seconds, so it didn't hurt to down them).
  - e12e 11y ago
    Well, once every 120 seconds is still quite a leap from 10 every second...
    - eropple 11y ago
      Sure, but if it doesn't work there, I don't trust it to work if, say, a piece of my scheduler goes nuts and suddenly is upping and downing containers every few seconds. The problem remains, it's just not as acute and still must be fixed.

Thaxll 11y ago

You should read that, it's very interesting: http://nginx.org/en/docs/control.html http://nginx.org/en/docs/control.html

fideloper 11y ago

Right the configuration is loaded into memory, and you need to reload it to get any changes in. I believe it's API let's you enable/disable existing configured servers but not dynamically add or remove them.

jfroma 11y ago

It is. Nginx launch a new worker while the other with the old config is still running, the master redirects new traffic to the worker and keeps the old worker until all previous requests has been handled. Once the old worker has finished the master process close it or when it time outs. It works even for updating nginx binaries. It is a very useful approach and I use it all time as well. I implemented the same in node by using the cluster api [1]. [1]: http://joseoncode.com/2015/01/18/reloading-node-with-no-downtime/ http://joseoncode.com/2015/01/18/reloading-node-with-no-down...