6 ms·

I think it is irresponsible to put tools like this into the hands of non-technical users. The lack of statistical Significance or recommended sample size will c

I think it is irresponsible to put tools like this into the hands of non-technical users. The lack of statistical Significance or recommended sample size will cause a lot of people to make uninformed decisions.

paraschopra 16y ago
Yes, I noticed lack of statistical significance. If the developer is reading this, I would seriously recommend adding statistical significance. It should be quite easy to implement. I had made on excel sheet for calculating statistical significance so you can use those formulas directly: http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-significance-calculator-spreadsheet-in-excel/ http://visualwebsiteoptimizer.com/split-testing-blog/ab-test... That said, I commend the team in bringing easy to use split testing solution for Wordpress!
- javery 16y ago
  Perfect, that will make it into the next release.
javery 16y ago
That's a more irresponsible stance to take - witholding tools and knowledge because you fear people are ignorant is a sure fire way to keep those people ignorant. Part of our goal with this plugin is to educate users about these things - look for much more of that coming in the future.
- paraschopra 16y ago
  Are you trying to say it is better to have A/B testing without statistical significant as compared no A/B testing at all? I'd disagree with you completely and since you are taking this stance, I am actually worried now about recommending the plugin. Not caring about statistical significance is equivalent to taking decisions at random. You may better flip a coin, why bother A/B testing? You are letting your users take decisions not knowing if those are real or just random.
  - arcware 16y ago
    Keep in mind that this is just step one of many we'd like to take with this plugin. It's not that we don't care about statistical significance - it just didn't make it into this version, so we'll work to get that in there soon. We will also be adding more in the FAQ tab that talks about these sorts of issues where users could possibly make decisions too soon instead of letting the tests gather more data.
- ccollins 16y ago
  I am absolutely not suggesting you withhold tools and knowledge, but that you should give them more tools tools that fully inform them. When you show "Improvement: 25%" after say, 20 page views - that is simply inaccurate.
  - javery 16y ago
    Fair enough - it is definitely something we are going to add very quickly.

paraschopra 16y ago

Yes, I noticed lack of statistical significance. If the developer is reading this, I would seriously recommend adding statistical significance. It should be quite easy to implement. I had made on excel sheet for calculating statistical significance so you can use those formulas directly: http://visualwebsiteoptimizer.com/split-testing-blog/ab-testing-significance-calculator-spreadsheet-in-excel/ http://visualwebsiteoptimizer.com/split-testing-blog/ab-test... That said, I commend the team in bringing easy to use split testing solution for Wordpress!

javery 16y ago
Perfect, that will make it into the next release.

javery 16y ago

That's a more irresponsible stance to take - witholding tools and knowledge because you fear people are ignorant is a sure fire way to keep those people ignorant. Part of our goal with this plugin is to educate users about these things - look for much more of that coming in the future.

paraschopra 16y ago
Are you trying to say it is better to have A/B testing without statistical significant as compared no A/B testing at all? I'd disagree with you completely and since you are taking this stance, I am actually worried now about recommending the plugin. Not caring about statistical significance is equivalent to taking decisions at random. You may better flip a coin, why bother A/B testing? You are letting your users take decisions not knowing if those are real or just random.
- arcware 16y ago
  Keep in mind that this is just step one of many we'd like to take with this plugin. It's not that we don't care about statistical significance - it just didn't make it into this version, so we'll work to get that in there soon. We will also be adding more in the FAQ tab that talks about these sorts of issues where users could possibly make decisions too soon instead of letting the tests gather more data.
ccollins 16y ago
I am absolutely not suggesting you withhold tools and knowledge, but that you should give them more tools tools that fully inform them. When you show "Improvement: 25%" after say, 20 page views - that is simply inaccurate.
- javery 16y ago
  Fair enough - it is definitely something we are going to add very quickly.