I've used A/B and multivariate testing at multiple companies. It can be incredibly helpful in optimizing a product. I would suggest however to focus on buildi

I've used A/B and multivariate testing at multiple companies. It can be incredibly helpful in optimizing a product. I would suggest however to focus on building a solid base product first and then optimizing it. For web the google web optimizer works ok and its free. There is a Boston startup called Sitespect who I've used in the past as well, which has a more robust platform. I can introduce you to them if you want.