Sometimes, like many other men trying to dress to impress, I will stand before my wife--or even one of my children--with two different ties.
"Which one do you like better: this one or this one?" I'll ask, giving them a choice between colors, patterns, or often both. Depending on the nature of the meeting I'm going to, I may follow up with questions like, "Which one says 'knowledgeable' or 'creative' to you?"
This is not hardcore A/B testing by any standard, but it's all I've got and, at the moment, Android developers don't have it much better. Although there are plenty of tools in the market to help them A/B test different features or functions within their app, conducting A/B tests on the app itself in the Google Play store has traditionally been something of a crapshoot. Consumers may like what they read about an app, and the screenshots may compel them to download, or they might not.
According to several sources, including The Next Web, Google may unveil a program that brings A/B testing capabilities to Play for the first time later this month at its I/O conference. Though this could disrupt third-party services that offer a sort of app store profile page replica for developers, it could also fundamentally change the way apps and mobile games are launched and marketed.
I suspect there is often a rush, particularly for indie devs, simply to get the app out the door and onto Google Play, with the hope that its features alone will be enough to sell it. Of course, as app stores get more crowded, this is obviously not enough. That has driven more developers to consider mobile ads, paid keywords and other tactics to drive installs. Though the success rates on any of these things aren't guaranteed, they are at least areas where developers have a modicum of control. With more control over app profile pages on Google Play, it will be interesting to see to what extent it changes the mix of promotional vehicles they use.
Then again, the I/O rumors may turn out to be just that. Even if they're true, Google may need to introduce more than just the tools to do A/B testing. What's the right approach to choosing things like the amount of text, the nature of images and the number of options you try? What amount of data is sufficient to make a decision? Is this something developers should do once and then leave alone, or re-introduce with new versions of an app or new competition within a category?
One would think Google has some expertise in-house to assist developers in conducting their own A/B testing. Sharing those best practices could be a nice complement to whatever it may announce at I/O 2015. A/B testing is great, but as developers may soon learn, it takes some knowledge and experience to ensure your app can pass those tests with flying colors. --Shane