Asodesk Blog / App Marketing & Product Management / Article

Essential Things You Need to Know About the Google Play Experiments Tool

Buse Kanal
Buse Kanal
Buse Kanal
Buse Kanal
Buse is the Marketing Specialist of ShyftUp. She is a marketing enthusiast with a huge interest in mobile apps. She was responsible for marketing in startups and contributed to many projects regarding branding and communication for well-known brands. She is thrilled by the technology and optimization.
All articles by author
Published: 14.07.2022
Updated: 11.08.2022
6 min read

The Google Play Experiments Tool is among the most effective ways to increase your app’s conversion rate. However, there are still marketers who are new to this tool. Some marketers also don’t know that there are new updates that affect the performance of their experiments.

In this post, let’s take a look at how Google Play Experiments work, the recent updates you need to know about, and other important details to ensure you get your app working optimally.

Continue reading to learn more about Google Play Store Listing Experiments.


What are app listing experiments

Google Play Store allows you to perform experiments to determine the best way to design your app page. Experiments involve A/B testing where you can compare various versions of your store listing page and identify which of those versions lead to the best results.

What are the benefits of app listing experiments

Running experiments in the Google Play Store provides many invaluable benefits, which include the following:

  • Identify the best graphics and texts that you should include in your app listing.
  • Increase app installs and user retention.
  • Learn more about various groups of your users based on their language and locality.
  • Gain more insights into your users’ behavior.

According to ShyftUp, you can better take advantage of the app listing experiments when you combine them with other App Search Optimization campaigns and strategies.

What tests can you do with listing experiments

As mentioned above, Google experiments are primarily conducted with A/B testing.

There are two types of experiments that you can run on Google: the global and localized.

With global, you can A/B test your graphics in your app’s default language. You can test your app’s promo video, icon, graphic, and screenshots. All global users are included in the experiment.

On the other hand, localized can test not only graphics but also text. You can test in up to five languages.

How to run experiments on Google Play

You conduct your experiments in the Google Play Console. 

Go to Store listing experiments and from there, you can then set up your test. Follow the on-screen instructions and your test should be up and running.

The interface of the Google Play Experiments in Google Play Console
The Google Play Experiments Tool. Source:

Once you’re done with your test, you can review and apply results based on your findings.

Best practices in conducting listing experiments

The Google Play experiments are a powerful tool to help you improve your app’s performance. However, to ensure you fully take advantage of this tool, you need to remember the following:

  • The best way to take advantage of store listing experiments is by testing app icons, screenshots, and videos.
  • If you want to obtain the most accurate results, consider testing one app page element at a time.
  • Google advises that you run your experiment for at least seven days. This will also help you measure data coming from both weekends and weekdays.
  • Regularly perform experiments and revisit your assets to ensure your app page is still designed optimally for changes that may occur over time.
  • Be sure to turn on email notifications for your experiments. This ensures that you will be regularly informed as soon as new updates are available with regard to your test.

Which changes were made to  Google Play Store listing experiments

On December 7, 2021, Apple announced that it will release the new Product Page Optimization (PPO) feature for its developers together with iOS 15.

The PPO feature is somewhat similar to the store listing experiments, which Google already made available to its app developers many years ago.

As a response to the PPO of Apple, Google has made some improvements and changes to its store experiments in 2022. The creation of experiments now comes with three steps:

  1. Store listing
  2. Experiment goals
  3. Variant configuration

Let’s go through each of them in more detail.

Store listing

Under the store listing, you will be able to configure which specific app you want to involve in your experiment.

Aside from that, you would then choose the name of your test, the experiment type, and whether it would be “global” or “localized”.

Experiment goals

Among the three, step number two has seen the biggest changes. Google gave this aspect a bigger focus — and rightly so.

Here are some of the most important updates you’ll find under the Experiment goals step:

Target metric

In the previous version of the experiments, you can only specify the percentage of your users who will be included in the test. Now, you can add more specific details.

Under the Target metric, you can choose between the two:

  • Retained first-time installers — this is the default and recommended setting. It allows you to measure the number of users who downloaded your app and kept it for at least a day.
  • First-time installers — you would choose this setting if you wanted to gauge the number of users who installed your app regardless of whether they installed it for a day or not.
Choose up to 5 variants in the Google Play Experiments tool
Create Experiments in the Google Play Experiment tool


The variants are simpler. You can choose up to five variants. While it is good to test as many variants as possible, experts recommend you only test one variant at a time to get a more conclusive result.

Experiment audience

You can choose a percentage between 1 and 50. This is the size of the available users who would be included in the test. It is highly recommended that you include 50% of your audience in the experiment to get more accurate data. The larger the sample size, the better.

Minimum detectable effect

With this feature, you can specify the necessary percentage of the minimum difference between control and variants to determine the winner of your test.

You can choose between 0.5% to 3.0%. The lower the percentage, the more users you need to complete your test.

Confidence level

As its name implies, the confidence level tells Google how much confidence you want to have in your test results. Obviously, the higher the confidence level, the better.

You can choose between 90% and 99%. While it is preferable to have a higher confidence level, if you ever choose 99%, your test would run longer than usual. 

Nevertheless, having a high confidence level would lower the chance of getting a false positive.

Variant configuration

In the third step, Variant configuration, you’ll be able to see and determine the duration of your experiment.

You can add specific details to your Google Play Experiment tool
Conditions in the Google Play Experiment tool

As seen on the screenshot above, you’ll see the “Estimated retained first-time installers” or “First-time installers” required, depending on which target metric you chose.

You’ll also find the “Estimated days to completion,” which is based on the number of your installers.

Those were the most significant changes made in the Google Play Experiments tool.

The new updates increase your level of control over how the test runs. Furthermore, the changes made by Google will lead to more statistically accurate and reliable results.

This is what setting up experiments looked like in the past:

How the Google Play Experiments looks in the past
Previous targeting in the Google Play Expleriments tool

Since changes were introduced, the setup looks like this:

Choose up to 5 variants in the Google Play Experiments tool
Create Experiments in the Google Play Experiment tool

What to remember when conducting Google Play Experiments

Google Play Store listing experiments are highly valuable in identifying the best variants of your app page. To ensure you run an effective Google Play experiment, keep the following in mind:

  1. Focus your experiments on screenshots, icons, and videos. The visual elements of your app product page play a major role in your conversion rate.
  2. Run your experiments for at least one week
  3. Opt to test one asset at a time
  4. Subscribe to email notifications to receive updates about your experiments
  5. When you run your experiments, you’ll receive a lot of data. You can combine the data with user acquisition tools from ShyftUp.

Google’s experiments can perform A/B testing, but these are not enough. With ShyftUp, you can run app store optimization services with ease, leading to better app visibility, conversion rates, and ultimately, income.

Also read:
Julia Suliagina
Julia Suliagina
Julia Suliagina
Julia Suliagina
Asodesk Content Manager. She writes newsletters and articles in blog. Make all complex topics in ASO easy and interesting.
All articles by author
9 min read

Failing to recoup app advertising costs: what can I do?

High cost per install, poor conversion, and low ROAS are a nightmare for any UA specialist. We will explain how you can solve these problems and walk you through the most relevant approach to UA used by top advertisers in 2023. Read more

Marina Trypolskaya
Marina Trypolskaya
Marina Trypolskaya
Marina Trypolskaya
Marina is a mobile app marketing professional with a decade of expertise in App Store Optimization.
All articles by author
9 min read

How to use AI for app marketing

In 2023, app publishers can leverage AI for marketing to improve their app rankings and conversion rates in the app stores, create effective ad campaigns, and ultimately drive more downloads and revenue. In this article, we will explore the various ways in which AI can enhance ASO, UA, monetization, and Retention and give you practical examples of how to use AI for marketers. Read more

Julia Suliagina
Julia Suliagina
Julia Suliagina
Julia Suliagina
Asodesk Content Manager. She writes newsletters and articles in blog. Make all complex topics in ASO easy and interesting.
All articles by author
12 min read

8 App Marketing Trends for 2023: experts’ opinions

If you would like to create an app promotion strategy that outperforms competitors in 2023, it’s crucial to consider current market trends. We have compiled insights on app marketing trends for 2023, sourced from industry experts including Thomas Petit, Lorenzo Rossi (REPLUG), Johannes von Cramon, and Ryan Thorpe (Reflectly ApS). Read more