The Importance of A/B Testing

Guest Posts

January 13, 2022

This is a guest post by Dave Bell, CEO and co-founder of Gummicube

You have successfully optimized your app following the best practices of App Store Optimization and current trends, now what? As the trends change over time, so too should your metadata and creatives. Experimentation of metadata and creative elements (then implementing the winning variations) becomes imperative to perform and maintain visibility amongst competitors.

Developers should look to test variations of current copy and creatives, and learn from the findings to create new iterations for testing. Creating iterations that incorporate seasonal language and/or imagery may also be an impactful change.

Historically, only Google Play had the ability to A/B test up to three variants against the current listing. However, with the launch of Apple’s Product Page Optimization feature, a doorway has been opened to developers on iOS.

Testing Metadata and Creative Sets

When creating the variations for testing, it is crucial to ensure that the copy or creative sets that will be tested are following general best practices.

Copy Best Practices:

The app’s copy, whether it be the short description, full description or screenshot text, should be easy to read and understand.

  • Screenshot copy should be clear, concise, and relate directly to what the UI is showing.
  • Description copy should be optimized for readability, but still include relevant keywords for your targeted demographic.

Creative Best Practices:

The app’s creative set, whether that be the icon, screenshots, video, or feature graphic, should follow general best practices as well.

  • Screenshots should always show an accurate representation of what the app will show a user.
  • Screenshot order should depict the app’s core feature at the beginning, followed by additional screenshots.
  • Icons should have one or two key elements and remain clutter-free. It is a small space, so more detail does not make it better.
  • Videos should create interest as well as accurately show a user what to expect when they interact with the app.

Identifying Keyword Trends

When creating a metadata-focused A/B test, it is important to keep in mind relevant keyword trends that may have an impact on conversion, all year round as well as seasonally. As an example, all year round a shopping app would want to target key items they sell. For seasonal optimizations such as Black Friday, shopping apps would want to utilize terms surrounding “doorbusters” and “best deals”. Keyword trends vary significantly between app genres, and being keenly aware of relevant features and seasonal events that directly pertain to the app can influence the success of your A/B testing.

Setting Up Elements to Test

With A/B testing, it can be tempting to take all of your best ideas and implement them all into an experiment to see if it outperforms the current listing.However, in doing so, you will not know which element that you changed made the most impact, so when beginning a new test, you will not know what to keep and what to create new iterations of.

The testing process should be a gradual, iterative process that will only test one element at a time to gauge what exactly is impacting conversion. For example, when testing a short description, a developer may look to see if emphasizing combat (e.g. “fight against forces of darkness”) or emphasizing social aspects (e.g. “team up with friends”) yields better results.

This process is not exclusive to metadata. Creatives – such as screenshots and icons – can also be tested to see what element is leading to higher user conversion. For creative testing, a developer can test general elements, such as background color or emphasized copy, or they can test more specific elements, such as specific characters to identify what yields the best results.

With this iterative, one-change-at-a-time method of testing, your conversion ratio can not only inform you about simple copy changes users prefer, but also what parts of your app they’re more interested in.

Seasonal Events & Elements to Test

There is no exact science behind which type of app should highlight which seasonal events. However, developers can gain an understanding as to if seasonal events, language and imagery do in fact have an impact on user conversion rates.

To give an example, a dating app may not benefit from testing different creatives or metadata during Halloween as a mobile game which came out with “Spooktacular levels” would.  Instead, a dating app may benefit more from utilizing testing during the Valentine’s Day seasonality. Put shortly, it is important to understand the genre and the user base of the app to know what seasonality is worth testing, when are optimal times to test, and most importantly when to leave things as they are. When in doubt, a simple test can provide valuable insight.

A/B Testing in the App Stores

When preparing for testing, it is important to know what elements are allowed to be tested in the iOS App Store and the Google Play Store.

iOS App Store

As of December 7th Product Page Optimization has launched for all apps within the iOS App Store. This allows developers to test creative assets alongside the currently live listing. One thing to note is that metadata, such as the subtitle, cannot be tested at this time.

For testing on iOS, Screenshots will not need to be included in the build, however icons will.

Google Play Store

The Google Play Store allows much more liberty. A developer can test creative elements, but can also test variations of the short description and full description. It is important to note that the title cannot be tested.

For testing on Google Play, no element has to be included into the build.

Benefits of A/B Testing

A/B testing offers app managers and developers a means to test the waters with their users, without having to commit to entirely new metadata or creative sets that may lead to a drop in KPIs such as conversion rate and tap-through rate. With A/B testing, we can confidently test new ideas, discover what users are responding to, and use the results to improve on key metrics for the app.

Put simply, A/B testing helps everyone involved in the app marketplace. Developers get a chance to try out new things in a low risk environment, and users have a chance to be introduced to additional facets of an app that they may have not seen before.


With the implementation of Product Page Optimization, this testing is now available within both stores, albeit limited to creatives only within the iOS App Store.

Consistently updating metadata and creative elements piece by piece, testing these iterations, and measuring performance to provide insights for future iterations is the key to a successfully optimized app.

About the Author

Dave Bell is the Co-Founder and Chief Executive Officer of Gummicube. In this role, Dave is responsible for overseeing the business strategy for Gummicube, driving growth and market development. Dave is a pioneer of the mobile entertainment industry with more than 15 years of experience in publishing, marketing, and distributing mobile apps and games across carrier, direct-to-consumer and app store channels.

More about Guest Posts

Leave a Reply

Your email address will not be published. Required fields are marked *