18 ms·

Another issue is launching behind feature flags often skips integration/e2e testing, which couldn't possibly test with every combination of every flag. So your

Another issue is launching behind feature flags often skips integration/e2e testing, which couldn't possibly test with every combination of every flag. So your release which is "safer" behind a flag is actually untested until it's launched.

locknitpicker 1mo ago
> So your release which is "safer" behind a flag is actually untested until it's launched. It's only untested if your intention is to not test it. There is nothing preventing you from testing a feature flag in e2e tests. In fact, testing is perhaps the primary reason why user overrides are supported by feature flag systems.
- willsmith72 1mo ago
  how? if you have 100 feature flags in your system each with 3 potential states, which states do you run in your e2es?
  - deathanatos 1mo ago
    This sort of combinatorical complexity only really exists in theory. The overwhelmingly common case is that FF A and FF B are independent, and can be successfully tested independently. You test FF A on & off, FF B on & off; your tests grow linearly with your FF set (which you should be pruning as soon as the feature is deployed). Also, since the original comment stated, > often skips integration/e2e testing They're probably responding with the expectation of this being compared to doing no testing at all. Merely assuming your FFs are independent, and testing them as such, is greater than nothing at all, even if the rare correlation might exist. And where a correlation might exist, well, that's what your judgement & expertise is for. Absolutely test that if you find yourself in that case.
    - willsmith72 1mo ago
      [dead]
  - locknitpicker 1mo ago
    > how? if you have 100 feature flags in your system each with 3 potential states, which states do you run in your e2es? You're going way out of your way to imagine problems where there are none. Feature flags are ephemeral and work as an ad-hoc release toggled at runtime. You create a feature flag, you commit the changes you need to commit behind the feature flag. Before switching the feature flag you run tests in preparation for the feature flag release, an finally you flip the flag. After that point, you either rollback the feature flag or you remove the feature flag. This is not rocket science.
    - willsmith72 1mo ago
      just replied above. i wish it was my imagination, it's reality

locknitpicker 1mo ago

> So your release which is "safer" behind a flag is actually untested until it's launched. It's only untested if your intention is to not test it. There is nothing preventing you from testing a feature flag in e2e tests. In fact, testing is perhaps the primary reason why user overrides are supported by feature flag systems.

willsmith72 1mo ago
how? if you have 100 feature flags in your system each with 3 potential states, which states do you run in your e2es?
- deathanatos 1mo ago
  This sort of combinatorical complexity only really exists in theory. The overwhelmingly common case is that FF A and FF B are independent, and can be successfully tested independently. You test FF A on & off, FF B on & off; your tests grow linearly with your FF set (which you should be pruning as soon as the feature is deployed). Also, since the original comment stated, > often skips integration/e2e testing They're probably responding with the expectation of this being compared to doing no testing at all. Merely assuming your FFs are independent, and testing them as such, is greater than nothing at all, even if the rare correlation might exist. And where a correlation might exist, well, that's what your judgement & expertise is for. Absolutely test that if you find yourself in that case.
  - willsmith72 1mo ago
    [dead]
- locknitpicker 1mo ago
  > how? if you have 100 feature flags in your system each with 3 potential states, which states do you run in your e2es? You're going way out of your way to imagine problems where there are none. Feature flags are ephemeral and work as an ad-hoc release toggled at runtime. You create a feature flag, you commit the changes you need to commit behind the feature flag. Before switching the feature flag you run tests in preparation for the feature flag release, an finally you flip the flag. After that point, you either rollback the feature flag or you remove the feature flag. This is not rocket science.
  - willsmith72 1mo ago
    just replied above. i wish it was my imagination, it's reality