What you need to know about Apple’s Product Page Optimization – WWDC21
App Store Optimization (ASO)
October 13, 2022
A/B testing has been available on the Google Play Store since 2015, 6 years later, it’s finally coming to the App Store. At the beginning of June, Apple launched the 2021 edition of its WWDC during which they announced their own version of A/B testing: product page optimization (coupled with the introduction to product page customization).
What is Apple’s product page optimization?
Product page optimization is a new A/B testing feature that will launch with the arrival of iOS 15 in the Fall. For years now, developers have had to rely on external A/B testing for the App Store. Finally, Apple is launching native A/B testing, directly on their platform.
In what they call product page optimization you will be able to create up to 3 different treatments for your test. This testing aims at users who find your landing page organically through searching or browsing in the App Store. On each treatment page, you will have the possibility to try a different icon, a different app preview video, and different screenshots, and your test can run for up to 90 days. Just like product page customization, each preview video, and screenshot you want to test will have to go through App Review, but you won’t need to submit a new version of your app (if you want to test an icon, you will apparently require to update the app in order to propose a test icon).
You’ll also have the ability to run tests only in specific locations. Meaning that if your app is localized in several languages, you can choose to run your test only in one (or several) of those. You can also choose the percentage of your audience that will see the treatments, if you have several treatments running in the same test the percentage you choose will be evenly split between the different variations.
App Analytics will allow you to measure the impressions, downloads, and conversion rate on each page, showing you which treatment performs better and could replace your default product page.
If one of your treatment pages includes a test icon, when the user downloads from that test page, they will have your test icon on their home screen (to ensure a consistent experience according to Apple). For this to be possible you will need to include the variant icon assets in your app binary (including all sizes for the applicable devices and the 1024×1024 version for the App Store).
The performance analysis will rely on Bayesian statistics, showcasing the impact of the changes you’re testing. Even though your test can last 90 days, you can also stop it anytime you want.
A long-awaited native testing option for the App Store
Until now, app developers had to rely on the use of 3rd-party tools to test their screenshots and videos on iOS. However, even with specialized tools, it was impossible to include all the store’s traffic channels in the test. This native testing could be the solution to this issue, according to Apple, it would be seamlessly included in the store traffic, and you will have the opportunity to choose which percentage of users you want to redirect to your testing page. This new feature will be connected to the new and improved version of App Analytics, allowing you to analyze the results of your testing directly in the App Store.
Let’s compare what we currently know about Apple’s A/B testing with Google’s long-running testing platform. So far, Apple has announced the same maximum number of variants (3 possible variations on each platform), the main difference seems to be that the Google Play Store allows developers to test alternative versions of your app listing’s text elements (in addition to the graphic elements), you can only test different media on the App Store (the icon, the screenshots, and the video preview).
What we can assume is that Apple is making a distinction between organic traffic and external traffic, offering different solutions tailored for each of them: product page customization will better target external audiences and product page optimization will help you make the best landing page for the users finding your app organically within the App Store.
Will it be worth the wait?
While Apple’s version of A/B testing will allow you to test different app icons, there is no mention of testing for titles, subtitles, or other metadata. So you won’t be able to test the textual elements of your product page. Also, while Apple introduced product page customization and product page optimization in the same video, you won’t be able to use their testing tool with the new customization feature. Any customized landing page you create for your app can’t go through A/B testing.
Although they made it very clear that you can create 3 variants per test, they haven’t disclosed whether you’ll be able to run several tests at the same time. For example, Google Play Experiments allows a maximum of five local experiments per listing (or one global experiment). Even though Apple spoke about testing only on certain localizations, they didn’t disclose whether or not you could run separate tests on separate locations at the same time.
Another question that arises is reliability. AppTweak notes that Google Play Experiments is notoriously famous for their below standard 90% confidence interval (where usually a 95% confidence interval is considered the minimum for A/B testing). We’ll just have to wait and see how Apple’s new feature compares to Google’s.
With the announcement of both custom product pages (to improve referral traffic) and product page optimization (more focused on organic traffic), Apple made it clear that they are putting the spotlight on improving traffic on their platform.
What did you think of Apple’s announcement? Will you try its native A/B testing when it comes out? Tell us in the comments!
- Mobile Video Monthly #38 – November 2023 - 5 December 2023
- Disturbing ads, a new trend for mobile gaming creatives? - 28 November 2023
- The Power of Holiday Marketing in Boosting Mobile Game Engagement - 21 November 2023
Thankyou so much for sharing nice information