5 ms·

Show HN: Bandit Algorithm for iOS, Android and Unity

24 points by splitforce 12y ago

broodsugar 12y ago
Getting Splitforce already at the test stages of development is one of the smartest things a game studio can do.
- splitforce 12y ago
  True, but why? Activision spends over $10M annually on employing an analytics team of about a dozen PhDs to build Rubin causal models to be able to show the right thing at the right time to players. That kind of budget makes sense for a $1 billion title like Call of Duty, but where does that leave the other 99%? The fact is that even for the companies that don't employ a full-time team of statisticians, if you're running your app or game like a true business you should be leveraging data to make better business decisions. Bandit algorithms like those we're proposing just let you do that at a fraction of the cost ;-)
0x0 12y ago
That custom scroll handler which overrides the native ios mobilesafari scroll handler is quite the annoying distraction. :-/
- splitforce 12y ago
  Thanks for the heads up - looking into it...
iosnoob 12y ago
As a splitforce user this is pretty cool.. biggest issue is having to wait for conclusive results, whatever speeds this up and auto prunes away the losing options is welcome.
- splitforce 12y ago
  Thanks, we think it's pretty cool too! And you've totally hit on why this is awesome - by automatically funneling users towards better-performing variations you can focus your resources on validating what matters and cut down on the deadweight loss associated with a traditional A/B. For more on this here's a recommended read: http://stevehanov.ca/blog/index.php?id=132?utm_medium=referral http://stevehanov.ca/blog/index.php?id=132?utm_medium=referr...
  - orasis 12y ago
    Do you guys do any sort of logistic regression for optimizing numeric values? That would be really cool.
    - splitforce 12y ago
      Right now auto-optimization only supports binary goal types. But we support A/B testing with Time and Quanity (numeric) goal types as well, so working on a solution to automate those types of experiments.
ikura 12y ago
Have seen a lot of these services, but this is definitely worth checking out. I think Multi-armed bandits are the way to go with optimisation, and just won Startup Asia's hackathon using this technique for a tinder-style app. A lot of people talk about learning from their customers, but few know how to really do it - understanding the math behind what's going on gives me much better confidence when deploying this type of service... Great work sharing it in this way.
orasis 12y ago
For the DIY crowd, implementing this on your own is a straightforward (and fun) exercise. I used the Javascript Bayesian Bandit library found here: https://github.com/omphalos/bayesian-bandit.js/blob/master/bayesian-bandit.js https://github.com/omphalos/bayesian-bandit.js/blob/master/b... I then used it to create a job that runs every 5 minutes on parse.com that updates an 'order' field based on the 5 minute sample. Then, from the client side, I simply do a query sorted by the 'order' field and I get the top variant for that 5 minute interval. Works like a charm!

24 points by splitforce 12y ago

broodsugar 12y ago

Getting Splitforce already at the test stages of development is one of the smartest things a game studio can do.

splitforce 12y ago
True, but why? Activision spends over $10M annually on employing an analytics team of about a dozen PhDs to build Rubin causal models to be able to show the right thing at the right time to players. That kind of budget makes sense for a $1 billion title like Call of Duty, but where does that leave the other 99%? The fact is that even for the companies that don't employ a full-time team of statisticians, if you're running your app or game like a true business you should be leveraging data to make better business decisions. Bandit algorithms like those we're proposing just let you do that at a fraction of the cost ;-)

0x0 12y ago

That custom scroll handler which overrides the native ios mobilesafari scroll handler is quite the annoying distraction. :-/

splitforce 12y ago
Thanks for the heads up - looking into it...

iosnoob 12y ago

As a splitforce user this is pretty cool.. biggest issue is having to wait for conclusive results, whatever speeds this up and auto prunes away the losing options is welcome.

splitforce 12y ago
Thanks, we think it's pretty cool too! And you've totally hit on why this is awesome - by automatically funneling users towards better-performing variations you can focus your resources on validating what matters and cut down on the deadweight loss associated with a traditional A/B. For more on this here's a recommended read: http://stevehanov.ca/blog/index.php?id=132?utm_medium=referral http://stevehanov.ca/blog/index.php?id=132?utm_medium=referr...
- orasis 12y ago
  Do you guys do any sort of logistic regression for optimizing numeric values? That would be really cool.
  - splitforce 12y ago
    Right now auto-optimization only supports binary goal types. But we support A/B testing with Time and Quanity (numeric) goal types as well, so working on a solution to automate those types of experiments.

ikura 12y ago

Have seen a lot of these services, but this is definitely worth checking out. I think Multi-armed bandits are the way to go with optimisation, and just won Startup Asia's hackathon using this technique for a tinder-style app. A lot of people talk about learning from their customers, but few know how to really do it - understanding the math behind what's going on gives me much better confidence when deploying this type of service... Great work sharing it in this way.

orasis 12y ago

For the DIY crowd, implementing this on your own is a straightforward (and fun) exercise. I used the Javascript Bayesian Bandit library found here: https://github.com/omphalos/bayesian-bandit.js/blob/master/bayesian-bandit.js https://github.com/omphalos/bayesian-bandit.js/blob/master/b... I then used it to create a job that runs every 5 minutes on parse.com that updates an 'order' field based on the 5 minute sample. Then, from the client side, I simply do a query sorted by the 'order' field and I get the top variant for that 5 minute interval. Works like a charm!