Turbine Games Consultinghttps://turbine.games Game Product Management, Analytics and MonetizationTue, 05 Dec 2023 22:38:48 +0000en-US hourly 1 https://wordpress.org/?v=6.9.4https://turbine.games/wp-content/uploads/2020/07/TGC\_square-111x111.jpgTurbine Games Consultinghttps://turbine.games 3232The A/B Testing Playbook For Mobile Games, Part 2: Statistical Significance Testinghttps://turbine.games/2023/12/04/the-a-b-testing-playbook-for-mobile-games-part-2-statistical-significance-testing/ https://turbine.games/2023/12/04/the-a-b-testing-playbook-for-mobile-games-part-2-statistical-significance-testing/#respondMon, 04 Dec 2023 18:21:00 +0000https://turbine.games/?p=1063In the first article in this series: The A/B Testing Playbook for Mobile Games, Part 1: Structuring the Experiment, we discussed the different types of A/B testing (also known as “split-testing”), how to properly structure experiments, and the most common …

The post The A/B Testing Playbook For Mobile Games, Part 2: Statistical Significance Testing first appeared on Turbine Games Consulting.

]]>In the first article in this series: The A/B Testing Playbook for Mobile Games, Part 1: Structuring the Experiment, we discussed the different types of A/B testing (also known as “split-testing”), how to properly structure experiments, and the most common mistakes my team has seen in our work with 90+ mobile game studios.

In this article, we’ll dig into the mechanics of the one statistical tool that we use most frequently in split-test analysis.

** Caution**: Statistical significance testing can be very dangerous when used without an adequate understanding of why and how it works. As a product manager (not a statistician) by trade with over 750 split-tests under my belt, I have intimate experience with being on the wrong side of these mistakes . So, in this article I’ll attempt to provide the necessary background on the statistics while keeping things as simple as possible for the non-statisticians out there (including yours truly).

Statistical Significance, Made Simple(ish)

When split-testing changes to our game, what we really want to know is:

Unfortunately, predicting the future is a messy business.

The best we can do is use the behavior of a small player sample, paired with statistical analysis, to place bets about the potential behavior of the broader population (i.e. all current and future players) once exposed to our change. This helps us make product choices with a higher rate of success than if we operated on intuition alone.

To gain valuable insights from split-testing, we need to set up experiments correctly, and then analyze them correctly (a nontrivial task!):

Step 1: Execute a split-test. Expose different player samples to different treatments of your game, let them play for a predefined period of time, then measure differences in average KPI (or lever metric) between player samples.

Step 2: IF we observe meaningful KPI differences between our player samples, we can use two statistical tools to predict the true KPI change for the broader population in response to our game changes:

• Statistical Significance Testing , and
• Probability Distributions

Key Terms to Know

• Player sample: A relatively small, randomly selected group of players whose behavior we will analyzedirectly, typically to make predictions about the behavior of the broader population.

• Broader population: An unmeasurable group that includes ALL current and future players. In inferential statistics, we analyze a small sample of players in order to make inferences or predictions about the behavior of the broader population.

• Treatments: Often called “variants.” The different variations of your game that you wish to split-test. Typically you will have a control group, and one or more alternate treatments.

• Control group: The set of players who experience no game changes (in other words, they experience the “normal” version of the game), and whose KPI we analyze against those of the player sample in order to detect a change.

• KPI: (Key Performance Indicator) A small group of summary metrics that serve as final aggregates (typically averages) for diverse types of behavior. The canonical examples are ARPU (average revenue per user) and dX Retention (day 1, 7, 30 retention).

Know Thy Enemy: Sampling Error

Within any randomly chosen player sample, and behind any KPI expressed as an average (as most are), lurks a LOT of internal variation in player behavior.

Most confounding is the case of IAP spend in F2P games, where it is typical for a random sample of 1,000 players to contain 950 players with $0 spend, such that ARPU is shaped by a mere handful of minnows, dolphins and whales. For example:

Presented as an average, this entire sample’s ARPU (average revenue per user) is $3.70. However, this is a tremendous and uncomfortable amount of variation to bury in an average. Because two individual users’ behavior can be so wildly different ($0 vs. $2,000), the random selection of players, for our small sample, has a dramatic impact on the sample average.

If a second whale (say a $1,000 player) was randomly included in our 1000-player sample, sample ARPU would increase by a whopping 54% to $5.70, not because of any game change we made, but merely due to random sampling error.

Sampling error, for our purposes, can be defined as the difference between the KPI (e.g. ARPU) observed in a small player sample, and the true KPI for the broader population the sample is meant to represent.

Sampling error, results from the fact that, and to the extent that, a small sample of players overweights or underweights certain behaviors depending on who is or isn’t randomly included in the sample. If the broader population has 1% whales, and your sample randomly happens to contain 2%, you are likely to experience significant sampling error due to the overrepresentation of whales and their spend in the sample. This is also why increasing your sample sizes reduces the impact of sampling error. If we have a 10,000-player sample with 10 whales, then including one additional whale in the random sample is less distorting than was the case in our 1,000-player sample.

So, if we run an experiment, hoping to lift ARPU, what might give us confidence that an ARPU lift observed in the experimental treatment (relative to the control treatment) was real, and not the result of sampling error?

To confidently say that our product changes caused the observed ARPU increase, we’d want to observe a positive ARPU change much larger, and/or more broadly distributed across players in the sample, than could reasonably be expected from sampling error.

Another way of framing this: we’re looking for a signal that so greatly exceeds the level of likely noise that it convinces us that the KPI change we observed is almost certainly the direct result of our game changes.

To compare signal to noise, we’ll need to use a statistical significance test.

Welch’s T-Test

If you research statistical significance testing, you’ll find an intimidating number of specialized methods and techniques.

Luckily, for mobile game split-testing, we can nearly always rely on Welch’s T-Test , aka the “ unequal variances t-test.”

Why? This test is appropriate for any split-test where all the following are true:

1. we randomly assign users to independent groups,
2. we’re comparing KPI, expressed as averages, between samples, and
3. we’re not certain that all samples have identical size and variance (defined below)

The Welch’s (unequal variances) t-test can be easily performed in Excel or Google Sheets, but let’s stay focused on comprehension for now.

“t”, the Test Statistic

Welch’s T-Test compares two data samples and generates a “t” value, often called a “test statistic”.

The “test statistic” is an abstract concept (bordering on pedantic to non-statisticians), but I find it useful to think of t as a simple ratio comparing statistical signal to noise.

For example:

• Signal: difference in D30 ARPU between two player samples
• Noise: “variance”: the random variation in spend values amongst the players in both samples.

t is “strength of the difference observed, net noise”.

Let’s use another example to make this more concrete.

Analysis Example with Dummy Test Data

As a hypothetical exercise, let’s use Welch’s T-Test to evaluate an ARPU data set from a hypothetical experiment.

This data reflects a typical, painfully skewy F2P ARPU distribution where only 3% of players spend, and ARPU is heavily influenced by a small number of whales.

In our example experiment data above, Treatment B had 50% higher ARPU than the control group . Let’s use Welch’s t-test to compare that positive signal to the level of noise in the data so that we can determine how excited we should be about the observed results.

First we will calculate the difference in means between the samples for the KPI we are measuring (ARPU). Then, to fill in the denominator in the equation (“noise”), we will calculate the standard error using the standard deviations from our two samples.

As we can see above, Welch’s t-test yielded a t-value of 1.55 for our experiment.

What the Heck is a t-Value?

The t statistic has one job: to measure strength of evidence against the null hypothesis.

The null hypothesis is the possibility that, in the broader population, the true ARPU lift is zero.

In other words, the null hypothesis states that there’s “nothing to see here” because, after considering the level of ARPU variance within our samples, the observed ARPU difference between the control and variant treatments can easily be explained by sampling error.

So if the null hypothesis is true, we should not expect the 50% ARPU lift to be “reproducible”

(to produce similarly exciting results) in either a repeat experiment, OR in a full rollout to the broader population.

So, how do we use the t statistic to measure the strength of our evidence against the null hypothesis?

The T-Distribution

To use the t statistic to evaluate the null hypothesis, we unfortunately must contend with yet another abstraction: a probability curve called a t distribution.

A t-distribution is a probability distribution that helps us find the probability of getting a t value (i.e  signal to noise ratio) this extreme, under the null hypothesis.

Where does this curve come from? Well, actually, from a Head Brewer at Guiness who was just trying to make a better beer, but that’s a story for another day.  For our purposes, just understand that, the t curve is always the same for any given sample size (e.g. 1,000 users), such that the t-distribution curve essentially functions as a lookup table for P, given t.

Nerd bonus: **** The t-distribution curve is always the same shape, except that it grows narrower and taller as sample size increases. With sample sizes in the thousands of users though, the change in shape is hardly detectable.

But I digress… what’s important is that we use our t-value (1.55 in our example), and the t probability distribution curve, to look up the value of P for our experiment.

P is the probability that, under the null hypothesis, sampling error could still cause us to get a t-value this high (or higher).

Wait, What the Heck is a P-Value?

Right, let’s try again… P is the probability of observing a KPI change as large and as distributed (across players, instead of just concentrated on 1-2 whales), as the one we observed, entirely due to sampling error.

If our observed KPI change were due to sampling error, this would be bad for us, since it would suggest that our experimental change would fail to actually lift ARPU when rolled out to the broader population.

This probability P is also called the “ P value.”  In our case, the t-distribution for our sample size, with a t-value of 1.55, yields a P-value of 0.12 or 12%.

Nerd bonus: **** The t probability distribution curve above illustrates the probabilities of getting different t values under the null hypothesis. The x-axis is t. The y-axis is the probability at each point on the curve. The total area under the curve represents 100% of possible outcomes. As you can see, higher t values are less and less probable (lower y-axis values). The area under the curve in both directions, beyond our t values of t = 1.55 or t = -1.55  is the P value for our experiment = 12%.

Mega-nerd bonus: **** Curious why we must evaluate t = -1.55 when our t-value was positive 1.55? That’s a particularly tortuous question, but the short answer is that we’re treating this as a “two-sided” analysis, and looking for values “as extreme” as 1.55, which implies distance from 0 in both positive and negative directions.

So, What can we conclude from P = 12%?

So, what p = 12% explicitly tells us is: if our product changes have no true KPI impact, we’d still have a ~ 12% chance of observing an ARPU lift this extreme due to sampling error alone.

And, at the risk of beating a dead horse, this would suggest a 12% chance that the cause of our ARPU bump was (perhaps) catching an extra whale or two in our random sample of 1,000 players.

More explicitly, p = 12% projects a 12% chance of getting a t-value of 1.55 or higher due to sampling error. So if we ran this experiment 100 times (with player samples of similar size and variance) but we changed the experiment so that the variant and control groups had identical game treatments (i.e. no game changes), we should still expect to get a t value >= 1.55 in 12 out of those 100 experiments just by dumb luck.

Scientific convention is to look for p values of 5% or lower to declare an experimental outcome to be statistically significant.  So, our 12% wouldn’t strictly qualify, and it follows that the 50% ARPU lift we observed can’t be considered statistically significant in the formal sense.

BUT, a p-value of 12% does imply that the variant is significantly more likely than not to have the higher “true” ARPU, and therefore I would caution against automatically rejecting the variant.

To make a sensible decision, we’ll want to weigh other considerations alongside the p-value to make a comprehensive decision. This is where experience both running experiments and understanding your users can be invaluable.

To thoughtfully conclude this experiment, I might consider the following:

• P-value:  P = 12% implies it’s more likely than not that the variant has higher true ARPU, BUT…

• Technical risk: How risky is it that we screw something up (e.g. introduce bugs) when rolling this out?

• Player sentiment risk: Does this change run the risk of upsetting players?

• New vs. existing users: How might this change impact newer vs. elder players differently? Do we need to test both?

• Probability distribution of effect size: How much lift should we expect, and does it justify the risks? (see next article!)

• Assessing known priors:

  • Have we tested similar changes in the past? How were the results?
  • Have our competitors made this change? How confident are we that they tested it and proved it successful, and how similar are our players?

• Level of trust in experiment: How confident are we in the reliability of how we structured the experiment, and collected data?

This is where making a decision about whether and how to roll out a tested change is as much art as science. The p-value is a valuable (and essential) input to this decision, but by no means should it be the only consideration. Trust me on this one, as I’ve learned this the hard way through hundreds of split-tests on live games.

What P doesn’t tell us: How large is our effect?

One limitation of the P value is that it tells us very little about what magnitude of effect sizes we can expect in the broader population.

This is limiting, because, as game developers, what we really want to know is:

Suppose that in a split-test we observe an ARPU lift of 33%, and a p-value of 5%, suggesting that this is a “statistically significant impact.”

A p-value of 5% does not suggest that we should expect a 33% ARPU lift when rolling out this treatment.

Instead, by producing a p-value of 5%, the (large) observed 33% ARPU lift helped convince us that the treatment is very likely to produce a higher ARPU in the broader population.

But how much higher? Suppose we wanted an ARPU lift of at least 10% to justify the risk of proceeding with the variant. How might we find the probability of a >= 10% true lift?

This brings us to a discussion of our second statistical tool, which we’ll discuss in the next article: effect sizes!

In the next article in this series, The A/B Testing Playbook for Mobile Games, Part 3: Probability Distribution of Effect Size, we’ll go deep on our second statistical tool for analyzing split-test results.

As always, if you have questions, comments, or critiques, I want to hear them! And if you’re interested in deploying rapid and reliable split-testing for your game, I would love to help: matt@turbine.games.

If you enjoyed this article, you might also like my series on F2P IAP Merchandising best practices:

• The IAP Merchandising Playbook, Part 1: Special Offers That Sizzle
• The IAP Merchandising Playbook, Part 2: Visual Hierarchy
• The IAP Merchandising Playbook, Part 3: MVP Store Design

Need help with your game? Email me at matt@turbine.games, or book a time on my calendar.

]]>https://turbine.games/2023/12/04/the-a-b-testing-playbook-for-mobile-games-part-2-statistical-significance-testing/feed/0The A/B Testing Playbook For Mobile Game Growth, Part 1: Structuring the Experimenthttps://turbine.games/2023/02/10/the-a-b-testing-playbook-for-mobile-game-growth-part-1-structuring-the-experiment/ https://turbine.games/2023/02/10/the-a-b-testing-playbook-for-mobile-game-growth-part-1-structuring-the-experiment/#respondFri, 10 Feb 2023 03:20:14 +0000https://turbine.games/?p=1037About our articles and guides In our work on 100+ games with 80+ game teams, there are some challenges that we encounter again and again. Over time, our experiences naturally evolve into generalizable best-practices. In these articles, I attempt to …

The post The A/B Testing Playbook For Mobile Game Growth, Part 1: Structuring the Experiment first appeared on Turbine Games Consulting.

]]>About our articles and guides

In our work on 100+ games with 80+ game teams, there are some challenges that we encounter again and again. Over time, our experiences naturally evolve into generalizable best-practices.

In these articles, I attempt to document these best-practices and make them available to everyone!, not just the few core clients that we serve.

About this article

The focus of THIS article series will be Split-Testing (also known as A/B Testing), in service of KPI improvement for live, mobile free-to-play games.

As with many specialized topics, split-testing is covered in countless articles and guides at a superficial level, but in-depth coverage of the topic is surprisingly scant . Most in-depth  articles seem to cater to online marketers and conversion rate optimization for landing pages or ads. Compared to conversion testing, mobile game feature testing is typically more complex, has different pitfalls and requires a different approach. That approach is the focus of this article series. This article series is written for mobile game product leaders, and should help empower them to split-test product changes in any area of interest: features, tuning, pricing, art, merchandising, and UI/UX.

This is not light reading material. This article will likely bore most to tears , save for the relatively few, fellow mobile game product managers out there.

For lighter reading, check out one of our more popular articles, like “How to Fail at Mobile Games!”

For those still with us…

To do the topic justice, we will need to cover experimental design and statistical analysis in some depth, as these topics are the most opaque, least-well understood, and most likely to trick us into drawing the wrong conclusions from our tests. As a product manager (and not a statistician) by trade with over 600 (product and app store) split-tests under my belt, I have intimate experience with being on the wrong side of these mistakes.

While I hope that you find this article useful, please don’t hesitate to comment or offer advice or critical feedback. I invite statisticians out there to question (and help improve) our methodology. Because we (at Turbine) run hundreds of product split tests per year, I will  update this article as our approach continues to mature, and will post to announce any updates.

Of course, the best way to learn our split-testing methodology is by doing it with us, on your own live game! While doing so, you’ll be improving product retention and monetization – 20-400% ARPU lifts are typical. DM me if interested.

With that said, let’s dig in.

What IS Split-Testing?

Split-Testing is an experimental method where we do the following:

1. Show different versions of a product to different groups of users, then…
2. Measure performance of the groups (on key metrics), then…
3. Use statistical tools to predict which version will perform better for ALL users…
4. with specific, measurable degrees of certainty.

WHEN Should We Split a Game Change or Feature?

1. Split-testing product changes (changes to the game itself, not just the ads or app store assets) is only realistic for live products with significant user volume (say, 1,000+ installs per week or 3,000+ WAU).

2. Even then, because split-testing requires significantly more effort and attention than simply releasing features normally, it should be used judiciously. It is not a free ride.

3. When it comes to product changes, split-testing is generally worth considering when you have reason to believe the impact of your changes will be large, and risky (impact could be negative for some of your variants).

For more detail, this flowchart below may be helpful.

What metric(s) should we target?

The short answer? (1) A lever metric that the experiment targets directly (defined below), (2) dX retention (X varies with experiment, as we’ll later discuss), and (3) dX ARPU as a proxy for LTV.

The slightly longer answer:

Ultimately, the goal of most (if not all) experiments is to increase player LTV.

Experiments targeting retention do so with the intent to increase LTV. Without driving LTV, retention’s impact on business outcomes is quite limited (aside from hypothetical virality and some impact on appstore search rank).

The relationship between retention and LTV is complicated and inconsistent. Sometimes, improving LTV comes at the cost of retention, e.g. when tightening an economy or showing ads more frequently. Conversely, making an economy more generous can improve retention, at the cost of LTV.

For this reason, we recommend always evaluating dX retention and dX ARPU as primary KPI for any split-test.

Realistically, for some experiments, you won’t budget enough users to power statistically significant IAP results (doing so is expensive), but that doesn’t mean you should ignore dX ARPU data. You might still learn something and, given the primacy of ARPU for business outcomes, I’d argue that it always deserves consideration.

Besides LTV (ARPU as proxy) and retention, most experiments need a third metric, which we call the lever metric. This is the metric the experiment directly manipulates, with the hopes of having downstream impact on retention and/or LTV.

A simple example: imagine an experiment testing the placement of a daily reward in the game store, for purposes of increasing daily store traffic, and IAP LTV as a byproduct. In this experiment, an appropriate lever metric might be total store visits by dX, or average unique daily store visitors by dX.

And, of course, the two other primary metrics for evaluation would be dX ARPU and some retention checkpoint (D14, D30, etc.), as appropriate for the experiment.

On Terminology:  A/B Testing or Split-Testing?

These terms are nearly identical. ‘ A/B Testing ’ is a more popular term, but often implies an experiment with only two variants: A and B.

The term ‘ split-testing ’ allows for any number of variants, and is therefore our preferred term.

Stacked Split-Testing, and Siloed vs. Blended approaches

In our work with clients, Turbine nearly always ‘stacks’ multiple split-tests, run in parallel, to get more experiments done in less time. There are two different ways to approach this: Siloed testing and Blended testing.

Siloed Split-Testing (in stacks) is our preferred term for

1. running a stack of several experiments concurrently…
2. where the experiments have a single, shared control group , and…
3. where each user enrolled is subjected to only one of the experiments.

Benefits of ‘Siloed-Stacked Testing:’

• Practically speaking, stacking experiments gets more experiments done, in less time.
• Sharing a control group across multiple experiments reduces user costs.

Analysis: During analysis, we compare all variants to the shared control using our statistical tools, which we’ll discuss at length in our next article.

Alternatively, Blended Stacked Split-Testing is our preferred term for

1. running a stack of several experiments concurrently…
2. where each experiment independently assigns its own control group, and
3. where each user is simultaneously enrolled in ALL experiments in the stack .

Benefits of Blended Stacked Testing:

• Stacking gets more experiments done in less time.
• You can assess intersections of winning variants, before deployment.
• You can enroll many more users into each experiment than is possible with siloed stacking. In most cases this benefit can outweigh any additional noise caused by blending.

Analysis: During analysis, we assess each experiment’s variants individually vs. their own control.

After finding winning variants across experiments, and before rollout, we can look at the intersections of these winners as a gut check for any negative interaction effects.

Below is an example, showing how a single user is randomly placed into one variant for every experiment in the stack. Below, the user shown is placed in the control groups for experiment 1 and experiment 3, but is exposed to variants in Experiment 2 and Experiment N.

Answer: According to our research and experience, no.  With random assignment of users to experimental variants, and with each experiment only modifying / controlling  one variable (while letting all others remain random), the background ‘noise’ of player differences arising from other experiments (1) approaches zero impact with large enough sample sizes, (2) isn’t materially different from the myriad, random differences already present (player play preferences, spend preferences, device type, geography, and numerous others), and (3) is usually offset by the benefit of having many more users enrolled in each experiment.

That said, some experiments can produce noisy results when blended, e.g. three experiments all targeting the same narrow metric like D0 IAP CV.  This can be countered by adding more players to the stacked experiment, or un-stacking them.

Finally, be careful not to blend experiments whose changs are mutually exclusive (i.e. where the winners of each experiment can’t ultimately be combined). Mutually exclusive product changes should either be siloed, or made independent variants of a single experiment.

When should we use Blended, Stacked Testing?

At Turbine, we used to run Siloed Stacked tests exclusively. However, over the last 12 months, we have gradually learned to embrace Blended Stacked testing.

As of today, we have become big fans, primarily due to the large gains in efficiency, but also the added ability to catch unexpected interaction effects.

For more details, here’s our current thinking regarding which approach to use:

Should we test New or Existing users?

For games with existing, valuable player-bases, we usually err on the side of caution and test new users first, and then existing .

However, if we know that risk to existing users is low, we will likely opt to test on both in parallel, for efficiency.

Here’s a flowchart of our thought process for new vs. existing user testing:

How long should we run experiments?

Turbine usually assesses experiments at average user d30 (for new installers) or average user ‘test-d30’ (for existing users: 30 days after the first day each user has their first active day within the experiment). When we need to be extra careful about retention risk, we will run to average user d60.

Note that ‘average user d30’ means that the average player age in the cohort is 30 days from install. Because installs occur over a period of time, some users will of course be younger, and some older, at the time of analysis.

For more detail, a flowchart:

How many users do we need?

This is a critical question when considering test structure, and one of the areas where mistakes are most common.

Getting this right is a question of statistics, so to do it justice we will circle back after covering the statistical tools used to analyze split test results.

We’ll cover this topic in the next article in the series.

Should we exclude any users?

A critical step in experiment design is the definition of inclusion criteria: what users need to do to ‘qualify’ for inclusion in final analysis, and which users will be excluded.

As a simple example, imagine you are testing the impact of a change to a screen.

It obviously wouldn’t make sense to include in the final analysis users who never reached or saw the screen in question!  Including them would only add truly random noise to your data, and weaken the ability of your experiment to isolate the impact of your specific changes.

Lastly, inclusion criteria should be defined before the experiment starts , so that you aren’t later tempted to fit the rules to the data.

Most Common Split-Testing Mistakes

Last but not least, here are the mistakes we see most commonly, both in test structure and analysis (covered in next article):

Structural mistakes

• Split-testing too early in the product lifecycle, when you should be ‘building’ (adding well-proven features) instead of ‘optimizing’
• Split-testing product changes that do not need testing (just make the change!)
• Testing changes that are too small to make a measurable difference, e.g. the mythical ‘button color’ experiment
• Assigning too few users to an experiment, particularly for IAP testing
• Not instrumenting lever metrics to aid in validation of results, and just relying on a tentpole KPI like ARPU
• Not blending and stacking experiments when reasonable
• Not doing proper QA on the experiment setup
• Accidentally exposing elder / existing users to an experiment intended for new users

Analysis mistakes

• Relying on black-box systems (like web-based tools) without fully understanding / doing the statistical analysis yourself
• Relying simply on aggregate metrics (like ARPU) without statistical significance testing and other tools
• Not committing to a specific date to measure & determine results (e.g. D30); calling the experiment on a random date when you see the result you want
• Optimizing for statistical perfection (99% confidence!) over speed and volume of high-ROI tests. This is likely to happen if you let your data scientists drive the process

Key takeaways from Part 1:

• A/B Testing and Split-Testing are essentially synonymous, but we prefer the latter.
• Split-Testing is appropriate for teams with live products that have user volume (at minimum I’d say 1000+ weekly installs or 3000+ WAU (for existing user tests).
• Multiple experiments can be ‘stacked’ on top of each other and run in parallel for efficiency gains, either in ‘Siloed’ or ‘Blended’ fashion. We generally prefer blended, stacked testing for maximum efficiency.
• Experiments can target New Users, Existing Users, or both simultaneously, depending on how careful we want to be to protect existing users’ experience
• We generally recommend performing analysis at avg. user D7, D30 and/or D60, and deciding in advance which age will be decisive.

In the next article in this series, The A/B Testing Playbook for Mobile Games, Part 2: Statistical Significance Testing, we’ll go deep on statistical tools for test analysis, and answer some tricky but frequently asked questions.

PS:  The best way to learn our full split-testing methodology is by doing it for real, with us, on YOUR live game! Turbine’s monetization growth servic e focuses on rapid-split testing, and typically yields ARPU gains of 20-400%. Our approach is informed by work with over 80 F2P game teams and over 600 experiments. If interested to learn more, DM me!

If you enjoyed this article, you might also like my series on F2P IAP Merchandising best practices:

Need help with your game? Email me at matt@turbine.games, or book a time on my calendar.

]]>https://turbine.games/2023/02/10/the-a-b-testing-playbook-for-mobile-game-growth-part-1-structuring-the-experiment/feed/0The IAP Merchandising Playbook, Part 3: MVP Store Design (V2.0)https://turbine.games/2021/11/09/the-iap-merchandising-playbook-part-3-free-to-play-iap-monetization-design-stack-v1-0/ https://turbine.games/2021/11/09/the-iap-merchandising-playbook-part-3-free-to-play-iap-monetization-design-stack-v1-0/#respondTue, 09 Nov 2021 01:57:42 +0000https://appturbine.co/?p=1006Wireframes and best practices for building a minimal viable product for your in-game mobile free-to-play game store

The post The IAP Merchandising Playbook, Part 3: MVP Store Design (V2.0) first appeared on Turbine Games Consulting.

]]>February 2022 NOTE: This article and wireframes were updated to V2.0. Included:

• Refinements born of split-test data / ongoing iteration
• Added Landscape wireframes

In the first two articles in this series, The IAP Merchandising Playbook Part 1 and Part 2 , we focused on the effective visual merchandising of individual offers.

Now, in Part 3, I’ll outline and wireframe what I believe to be the minimum viable store, product and merchandising stack for a new F2P game product.

Why write this?

I often use these articles to efficiently answer common questions we get from our clients. While the previous two articles focused on how to make individual offers more effective, this article’s F2P MVP Store Design addresses a lot of common IAP monetization questions with a single, coherent design solution, plus F.A.Q.

F2P MVP Store Design

The F2P MVP Store Design V2.0 below includes:

• Store Layout, in portrait (and now landscape!)
• The Starting Offers and Products that new players will see
• Pricing and Discount recommendations
• How to merchandise products outside of the store.

It does not include:

• Landscape wireframes (if this would be valuable to you, leave a request in comments!)
• Offers introduced after D1 (a topic for a future article!)
• Multiple currency / conversion support (e.g. gems to gold). For casual games, I recommend on-demand conversion via modal popup.
• Tabbed store support for midcore or core games.

Finally, the F2P MVP Store Design embodies specific Design Goals, enumerated below, and the visual guidelines from Part 1 and Part 2 of the playbook. To fully understand the problems being solved by this design, I recommend reviewing these links first!

How do I use the F2P MVP Store Design?

Should my store be portrait or landscape? In nearly all cases, scrolling store UI is most effective in the direction of the longer axis of the screen, so as to maximize the number of onscreen products (facilitating comparison) and minimizing scrolling, for screens of any aspect ratio. In other words, portrait games’ stores should generally scroll vertically, and landscape games’ horizontally.

Casual Games in Portrait Mode: For those of you working on a portrait-mode, casual title with a single-currency economy, the wireframes / design above can be used out of the box. Just forward this article to your Product Manager or UI Designer!

Everything in the wireframes including numerical values can be used directly, except for the coin and consumable payload for each product.

For example, your economy may not (and probably shouldn’t) value 100 coins at $1.00. So, using the base USD value for currency and consumables in your game, just calculate and replace the payload shown for each IAP product based on the IAP prices and bonus values shown in the design.

Midcore / Core Games: For midcore and core games, I’d recommend using this design as your foundation for a simple, tabbed store, with tabs organizing products into clear compartments by use case / player intent.

Any other questions, just ask in the comments!

Design Goals

The F2P MVP Store Design and its components should be quite familiar to F2P game developers.

Where I have deviated slightly from convention, I did so in service of the following design goals, in order of importance:

1. Don’t overwhelm the eye: Ruthlessly minimize visual clutter!
2. Facilitate confident decision w/ minimal effort:
   1. Communicate information as succinctly as possible
   2. Organize info hierarchically (see Part 1 and Part 2 of this series)
   3. At a glance, UI guides player to best product for them
   4. Player can quickly audit ALL products with minimal effort / scrolling
3. Leverage a first-conversion product: Typically, a high-value, low-price starter offer
4. Leverage recurring revenue: Weekly subscription with free trial is a must
5. Cater to whales in 1st session with a juicy $100 option

If you don’t like or agree with the approach in the wireframes, it’s likely that your goals or priorities differ from these!

Next, I’ll dig a bit deeper into the three major components of the design, to provide a bit of additional color and rationale.

Step 1:  Six Base Currency Products

When testing product-market fit, you should be squarely focused on D30 retention improvements, with minimal bandwidth for monetization features.

At this early stage, your fledgling store should look something like the above: Six basic hard currency price points, and nothing else that requires development effort. Porting a more robust store from another game is fine of course, as long as it doesn’t siphon any effort away from your singular focus on D30 retention.

Why six price points? It is the minimum that achieves a reasonable price spread, and the 3×2 matrix keeps the store clean and tight, in service of Design Goals #1 (don’t overwhelm the player) and #2 (see all products with minimal scrolling).

Step 2:  Four Starting Bundles

Once your game has reached its D30 retention target, and you are ready to build out your IAP monetization stack, my next recommendation is to fill out the store with four starting bundles, starting at $2.99 and ending at $99.99. The $2.99 and $99.99 are given top (4x) value, to achieve Design Goals #3 (first conversion device) and #5 (juicy whale offer).

All four offers can be available to new players on install, and the Starter Pack popup can be first displayed during or (more typically) immediately after the FTUE.

Each of the four offers are purchasable only once, but the design foregoes any expiration timers. This omission is a minor design concession to reduce scope, as we don’t want our starting offers expiring (leaving an offer void) until we have a schedule of new offers to backfill (a topic for another article: Store Design V2.0!). Until then, the starting offers meet our design goals while creating the appropriate level of upsell pressure.

All four of these bundles include 1) popup reminder / merchandising as shown above in C. Bundle Popup (simply swap the product data on the popup design), and 2) Title Screen merchandising, as shown in A. Home Screen.

Step 3:  Two Subscription Products

If your game uses interstitial and/or banner ad monetization, you’ll need a remove-ads product. My recommended solution?  An auto-renewing subscription, which leverages both the conversion benefit of the free trial and the ARPU benefit of auto-renewal. For your MVP I recommend just two choices: 1) weekly and 2) annual, for simplicity (see Design Goals #1 and #2).

“If our game doesn’t have forced ads, do we still need a subscription?”

First, and particularly for casual games that aren’t ROAS-positive, I would strongly recommend testing the use of forced ads with new users, to see if the ARPU benefit outweighs the retention loss.

Secondly, when it comes to subscriptions, my general recommendation is that ALL games test a subscription product, in order to leverage its powerful one-two punch of free trial + auto-recurring revenue. So, if you can’t offer ‘remove ads’ as the tentpole benefit, you’re not off the hook! Just substitute another benefit that’s appropriate for your game economy.

Key Takeaways

For Casual game developers, these wireframes can serve as a starting point for your store and offer merchandising design.

For everyone else: If not directly useful, the wireframes can serve as a more- robust example of the visual merchandising principles from Part 1 and Part 2 of this series.

That’s it for now!  In the next article, The IAP Merchandising Playbook, Part 4 we’ll wireframe Store V3.0, which includes progression offers, timed sales, just-in-time sales, and features specific to midcore games. As always, we’ll keep designs as lean as possible, with the hope of delivering “maximum ARPU with minimum effortTM.”

^ Editor’s note: Not a real trademark… yet

Need help with your game’s performance?  Contact matt@appturbine.co to see if we can help!

——————————————————-

Thanks for reading! If you found this interesting, useful, or even disagreeable, don’t hesitate to message me through LinkedIn or email at matt@appturbine.co

Other mobile game product management articles you might find interesting:

The IAP Merchandising Playbook – series:

1. [Special Offers that Sizzle](/content/2020/10/09/the-iap-merchandising-playbook-part-1-special-offers-that-sizzle/ "https://appturbine.co/2020/10/09/the-iap-merchandising-playbook-part-1-special-offers-that-sizzle/"/index.html)
2. Visual Hierarchy

Mobile Game Design: [Getting More With Less: 6 High-Yield Design Patterns](/content/2020/04/22/getting-more-with-less-6-high-yield-design-patterns-for-f2p-games/ "https://appturbine.co/2020/04/22/getting-more-with-less-6-high-yield-design-patterns-for-f2p-games/"/index.html)

Mobile Game Monetization: Do you know your game’s spend depth?

How to Fail at Mobile Game Product Management – series:

1. Start Without a Business Case
2. Undisciplined Innovation
3. [Vague Product Goals](/content/2019/10/09/how-to-fail-in-f2p-mobile-games-part-iii-vague-product-goals/ "https://appturbine.co/2019/10/09/how-to-fail-in-f2p-mobile-games-part-iii-vague-product-goals/"/index.html)

Mobile Game Analytics – series:

1. DON’T Measure Everything
2. Just Track These 6 Behaviors
3. What You Need to Know About Your Users

Cheers!

-Matthew

]]>https://turbine.games/2021/11/09/the-iap-merchandising-playbook-part-3-free-to-play-iap-monetization-design-stack-v1-0/feed/0The IAP Merchandising Playbook, Part 2: Visual Hierarchyhttps://turbine.games/2020/11/17/the-iap-merchandising-playbook-part-2-visual-hierarchy/ https://turbine.games/2020/11/17/the-iap-merchandising-playbook-part-2-visual-hierarchy/#respondTue, 17 Nov 2020 00:34:41 +0000https://appturbine.co/?p=1001Top-grossing mobile games use visual hierarchy to guide players' purchases. Here's your playbook for visual hierarchy in IAP stores!

The post The IAP Merchandising Playbook, Part 2: Visual Hierarchy first appeared on Turbine Games Consulting.

]]>In the first article in this series: The IAP Merchandising Playbook, Part 1 we provided a simple checklist for ‘Captivating’ Special Offer Merchandising:

In this article, we’ll take a closer look at one particular item in our checklist: #5 ‘Effective Visual Hierarchy.’

What is Visual Hierarchy?

When viewing a collection of distinct visual elements, our eyes naturally gravitate to large, bold, high-contrast, or animated elements first, and to small, lower-contrast elements last.

By being more deliberate in how we structure information, we can guide the user’s experience and streamline their purchase decision.

As an exercise, look at the offer below, imagining yourself as a player seeing it for the first time. Where do your eyes go first?  In what order do you 1) see items, and 2) actually process the information they convey?

Above: from Might and Magic RPG: In what order does your eye perceive and process elements on this screen? Why?

Above: My eyes perceive elements roughly in this order

There is no right answer of course, but for me the visual journey goes something like the above.

The large, bright hero image attracts my attention first (a good thing!). From here, my eye is drawn to nearby high-contrast elements: “Limited!” and “This pack can only be purchased once every 14 days!” which, to me, are not highly relevant to my immediate purchase decision. A time-limit would be more meaningful.

Next, my eyes land on the green call-to-action button: “Purchase: $14.99,” briefly before jumping to the very high-contrast  ‘tap X to close’ button to the upper right, which seems to dare me to close the popup before considering further information.

If the goal is to help me make a purchase decision quickly with minimal effort, the information here isn’t organized optimally.

Let’s redesign it!

Redesigning for Visual Hierarchy

As a rule of thumb, I recommend arranging visual hierarchy into a maximum of three ‘tiers.’

Above, we have created an information hierarchy with three distinct tiers through deliberate use of contrast and size. Each tier in the hierarchy has a corresponding text size: 1) Header, 2) Body and 3) Fine Print.

Our intent is to help the user consume information in the following order:

• Tier 1: HEADERS: Hero image, product title, purchase button. We want players to see these first.

• Tier 2: BODY: Key product value propositions: “Get these daily in your inbox!,” “Limited!,” resource icons and quantities. We expect that nearly all players will read and process this information before making a purchase decision.

• Tier 3: FINE PRINT: Additional information that is optional and not critical-path in the purchase decision for most users. For this offer, fine print includes text labels for each resource (if icons alone don’t do the trick), the ‘more info’ button and the x / close button. These elements are here if needed, but definitely don’t need to attract attention.

Above: Generally, the items deserving ‘Tier-1’ treatment are hero image, product title and purchase Button.

Above: ‘Tier-2’ elements generally include 1-2 Value Statement(s), resource icons and quantities.

Above: ‘Tier-3’ is for non-critical supporting information (i.e. fine print) and the close button. The information is there in case players need it, but most won’t. Regarding the close button, most players reflexively know where to find it, so it needn’t attract attention.

Let’s use this clear definition of ‘Effective Visual Hierarchy’ to evaluate some more offers, both good and bad!

Do this!

In addition to meeting all of our key criteria, this offer from Dirt Bike Unchained is a particularly good example of a deliberate, three-tiered visual hierarchy.  Most importantly, Tier-1 / Header size is reserved for our most important three elements: the (large) hero image, product title and purchase button, and all other elements take a back seat at Tiers 2-3.

Above: The most prominent elements here are hero image, title and purchase button / price.

Above: In this offer, Tier-2 sizing is used for the ‘10X Value’ label, and offer expiration timer.

Above: In this offer, Tier-3 sizing is used for the Value Statement, and bundle components. While I would usually put components at Tier-2, it works in this case because this ‘bit of everything’ offer is aimed at new players, who are likely to consider the ‘10X Value!’ statement more immediately relevant than the specific quantities of resources included.

Don’t do this!

The in-store offers shown above are crisp, colorful and relatively uncluttered. But, because they lack large, compelling hero images, it becomes very difficult to create effective visual hierarchy.

Because the hero images (the gift-wrapped boxes, at left) are small, the bundle components must grow to fill the remaining space. Their size and contrast here arguably elevate them to ‘header’ treatment alongside the title, purchase button and hero image. The large slash-through price ($15.99) is also given header treatment, and the bright, fully-saturated colors used for the large offer containers and title banners give these non-informational elements more visual weight than warranted.

Putting all of this together, we have 8-9 distinct, ‘loud’ visual elements all competing for attention at the same (top) level in the visual hierarchy.

My eye doesn’t know where to go, and I feel a subtle but significant pressure to escape this screen!  Now where’s that close button…

Do this!

While imperfect, each of the offers in the above example ( AFK heroes) gets a pass for our key criteria. The large, aspirational and animated hero images do their job well.

Let’s focus on the visual hierarchy and get specific with potential improvements. I would de-emphasize the background image on the right half of each offer to help the component icons read more clearly.

If I’m being picky, the expiration time needs more margin on the left, and font sizes could change to emphasize price, discount % and quantity, and to de-emphasize expiration time and purchase limit.

Don’t do this!

The in-store offers from Langunis come close, but fall short for two reasons:  1) the hero images are too small, AND identical for all offers (tsk. tsk.), and 2) visual hierarchy:

The offer contents at left are given too much visual prominence, and so they overpower the  hero image or purchase button. Again, with all elements shouting for attention at similar volume, my eye doesn’t know where to go first, and I’m tempted to close the store to avoid investing more effort.

The examples thus far have all been offers in the scrollable store list. Let’s look at some modal popup offers!

Special Offers: Modal Popups

Review: a modal popup is a popup window that appears on top of an existing interface while keeping the background interface visible but non-functional.

Once again, I’ll pick on Just Kill Me 3.

Don’t do this!

Above, the modal popup offer from Just Kill Me 3 suffers from the same issues as their in-store offers. One improvement here is the attempted inclusion of a hero image (the cat at top-right), but this asset is borrowed from gameplay, and not particularly exciting or aspirational.

Visual hierarchy: despite the lack of margins, the visual hierarchy is reasonably well organized, with the title banner and purchase button benefiting from ‘header’ treatment.

However, the close button is much larger and more prominent than necessary, an issue that I see frequently with popup offers. Treat the close button as Tier 3: fine print!

Do this!

The above example ( Empires & Puzzles), satisfies all key criteria in our checklist.

Visual Hierarchy: note the dramatic difference between header and fine print sizes, and the use of header treatment for title and purchase button. The ‘body’ font size is used for the most important value statement: “410% Value!” The animated hero image (top) and animated glows (below) are a nice finishing touch. Note that players can tap each pack component for more information.

Don’t do this!

The popup offer above fails on most of our criteria. The use of scaled-up versions of existing game UI icons in place of unique hero image assets is a missed opportunity to convey value.

Visual hierarchy: the myriad of high-contrast, similarly sized text elements (without adequate space between them) creates a feeling of visual overwhelm. All text elements in the top half of the offer seem to be fighting for my attention, shouting with similar intensity in the same header size.

As a player, my eye doesn’t know where to go, so I’m likely to close the popup immediately. It will take a bit of effort (read: too much) to get my bearings and understand what’s important. Is the blue ribbon with the number ‘250’ really the most important piece of information, as suggested by its size and contrast? Is the expiration timer more important than the purchase button?

Do this!

Angry Birds 2’s Kickstarter Pack uses a small hero image that is not unique to this offer (we’ll discuss efficiency and scalability later in this article series). Because their hero image is small, the offer components must compensate, and so are larger than I would normally advise. That one gripe aside, their use of high-quality assets, effective visual hierarchy, slick animations and clear value statement are excellent design choices.

Visual Hierarchy: note the deliberate use of three distinct font sizes: header-sized fonts for title and purchase button, body-sized font for resource quantities, and fine-print size for the value statement and expiration timer.

Don’t do this!

Above, the welcome offer from Fishing Clash has no hero image, and the four large icons representing the four resource types show visual degradation at this size. Inadequate margins between the header, value statement, and the top of the four gold frames create an unnecessary feeling of clutter.

Visual hierarchy: the visual hierarchy focuses the eye on four, monolithic saturated gold rectangles, which somewhat overshadow their contents and leave little space for anything else in the offer. Header font size is given to the product title and purchase button (good), but also to the component quantities (not good). The medium body text size is used for virtually everything else, including value statement, component titles, expiration time, value multiplier and purchase limit.

This offer exhibits another issue I see frequently: overuse of hard-edged nested boxes. This tends to happen when large hero images are absent, and the designer (or engineer in some cases) responds by filling the abundant space with a telescoping series of rectangles. Notice the very heavy borders around the purchase button and each of the four gold rectangles, and the many hard, high-contrast lines that separate the pack title, value statement, and component titles. The heaviness of these high-contrast lines draws undue attention, adding to the visual clutter, and making the offer feel rigid and stiff, like a spreadsheet.

Do this!

Though they stop short of using animation or vfx, RAID: Shadow Legends does a superb job of achieving consistently high quality across a seemingly endless variety of offers.

An important takeaway here: to be effective, a hero image doesn’t necessarily need to show or even directly relate to the offer contents themselves. Great if it can, but, if not, then any thematically relevant, exciting and unique art asset can do the job.

Visual hierarchy: my eyes go first to the huge hero image and high-contrast, bright, double-walled purchase button. Note the use of body size font for the product price, while giving the button art itself the header treatment.

The offer makes deliberate use of three very distinct font sizes. In this case, header text size is used for the product title (top) and value statement (bottom left), and the fine print size is appropriately small, as it should be! The value statement is perfect for a starter pack.

Finally, notice how the product components are split into two tiers; at top, the larger anchors(blue and purple shards), and at bottom, the four smaller kickers. Given their contrast and size, the anchor shards are arguably given header treatment, which leads to my only gripe with this offer:

There are arguably four different tiers here, which makes the visual journey a bit muddier than I’d like.

I’m being picky, but here is how I would change it.

Before:

After:

Above: We have clarified the visual hierarchy by 1) Enlarging the title and changing to white font, 2) demoting the value statement, ‘Best Value for New Players’ to ‘body’ text size, 3) making pack components smaller, so as to not overpower the hero image, title and purchase button, and 4) demoting “Limit: 0/1” to fine print text size.

—

Key takeaways

When it comes to Visual Hierarchy, using our three-tier system can prevent the most frequent and egregious mistakes I see in offer layout. Using a large hero image, product title and purchase button as the basis for your visual layout solves much of the problem by forcing you to use smaller icons and fonts for everything else.

With this three-tier system, and the checklist from Part 1: Special Offers that Sizzle, you now have two clear, simple, and easy-to-use tools to guide your game’s IAP merchandising!

In Part 3 of The IAP Merchandising Playbook Part 3: MVP Store Design, we’ll dive into product discounts and offer segmentation.

——————————————————-

If you enjoyed this article, you might also like:

How to Fail in F2P Mobile Games, Part 1: Start without a Business Case

How to Fail in F2P Mobile Games, Part 2: Undisciplined Innovation

How to Fail in F2P Mobile Games, Part 3: Nebulous Product Goals

Need help with your game? Contact me!

]]>https://turbine.games/2020/11/17/the-iap-merchandising-playbook-part-2-visual-hierarchy/feed/0The IAP Merchandising Playbook, Part 1: Special Offers that Sizzlehttps://turbine.games/2020/10/09/the-iap-merchandising-playbook-part-1-special-offers-that-sizzle/ https://turbine.games/2020/10/09/the-iap-merchandising-playbook-part-1-special-offers-that-sizzle/#respondFri, 09 Oct 2020 23:05:51 +0000https://appturbine.co/?p=987In mobile games, IAP Merchandising is an area where best-practices are well established. Here's your checklist for creating IAP Special Offers that sell!

The post The IAP Merchandising Playbook, Part 1: Special Offers that Sizzle first appeared on Turbine Games Consulting.

]]>

About our articles and guides

In our work on 80+ games with 35+ game teams, there are some challenges that we encounter again and again. Over time, our experiences naturally evolve into generalizable best-practices.

In my articles, I strive to document these best-practices and make them available to everyone!, not just the few core clients that we serve.

Above: How we think about these articles. Let us know which topics most interest you!

About this article

In THIS article series I will focus on IAP Merchandising, an area where best-practices are particularly tangible, well-established, and almost certain to get you the best results!

And yet, these best-practices don’t seem to be documented anywhere… let’s fix that.

If the term ‘IAP Merchandising’ is unfamiliar to you, I’d recommend you first skim: ‘ What is IAP Merchandising?’

If you follow the recommendations that follow, your team can save time and effort while avoiding the most common mistakes. Perhaps, like us at Turbine Games Consulting, you’ll even come to regard IAP Merchandising, and IAP Monetization more broadly, as one of the ‘easier’ parts of F2P game development.

Now, I certainly don’t mean to suggest that effective merchandising requires NO effort. Art assets still need to be created, stores wireframed, products designed, discounts tested and so on. But if we follow a playbook, effort and results are fairly predictable, with few surprises along the way!

Why start this series with Special Offers?

Opening our IAP Merchandising discussion with Special Offers helps us focus first on the fundamentals: how to most effectively present a single product to your players. Later in this article series, we’ll zoom out and tackle broader, structural topics like store layout, segmentation and discounts.

Let’s get started!

Best Practice #1 Special Offers should be ‘Captivating’

Offers have one job

If we expect our players to exchange real money for virtual (read: NOT REAL) products, we’d better do everything we can to make them look and feel valuable!

In your battle to captivate players via IAP merchandising, special offers are the most powerful tool in your arsenal. While your vanilla, in-store base products must also convey value, special offers have a higher calling: to be irresistable. The most effective offers evoke palpable feelings of excitement, opportunity, urgency and loss-aversion.

Many games miss the mark here.

When they do, a common cause is the developers’ failure to internalize and match the standard of quality set by their top-grossing competitors. The most egregious merchandising I’ve seen came from teams who didn’t habitually analyze top-grossing games.

This is an unnecessarily risky choice, because players’ perception of value is the primary driver of their monetization decisions, and this perception is regulated by the games they play most often.

So, your IAP products need to look as appealing as your top competitors’.

Two Types of Special Offers

The two most common places (“placements”) where special offersare sold, are:

1. In-Store Product List
2. Modal Popup Banner

In-Store Product List Offers

The In-Store Product List presents special offers next to basic products, facilitating the side-by-side comparisons that drive urgency.

Above: in the scrollable, in-store product list, medium banners are used to elevate and differentiate ‘special offers’ from the smaller, gridded ‘base products.’ This store layout is ubiquitous for games across the Casual – Core spectrum.

Modal Popup Offers

The Modal Popup(or ‘screen takeover’) is the second place where special offers are merchandised.

A modal popup puts the interface into a ‘mode’ where the main window, while still visible in the background, has its functionality and buttons temporarily disabled. The popup becomes the front window and has the focus.

In F2P Games, modal popups generally occupy two-thirds of the screen. This provides enough room to make a captivating pitch for your IAP product while still leaving some background visible for context.

Your Checklist for Creating ‘ Captivating’ Special Offers

“Your IAP products need to look as appealing as your top competitors”.

This gets us in the ballpark but, as a measurable standard, it still falls short for two reasons:

1. ‘As appealing as’ is still uncomfortably subjective
2. We should aim for better than ‘appealing!’ I’d say: ‘captivating.’

So, we’ll enumerate 7 key criteria for ‘Captivating’ Special Offers.

Here is your checklist:

Mandatory:

1. Generous size: To effectively communicate that they are special, special offers need significant screen real-estate. For modal pop-ups, use at least 2/3 of the screen. In the scrollable store, use ‘medium banner’ or larger size.
2. Killer Hero Image: A special offer product should have a large, beautiful keyart asset that makes the offer feel exciting and aspirational. This is often dubbed the ‘ Hero Image’ (meaning ‘important image’, not ‘image of a hero’).
3. Retina-quality art assets : Do NOT over-compress your offer assets. They should appear fully retina resolution, with crisp edges, and no visible distortion.
4. Uncluttered: Having too much text, too many distinct visual elements, or inadequate margins creates visual clutter that detracts from both the clarity of your pitch and the perception of product value. This is a big personal pet peeve of mine!
5. Effective Visual Hierarchy: The user should consume information in the correct order, most important to least, to minimize friction and effort. For most offers this means allowing the product title, purchase button, and hero image to carry the show, while making other elements smaller in size and lower in contrast. We’ll explore Visual Hierarchy in-depth in the next article!

Optional Bonus:

6. Bonus: Clear Value Statement(where appropriate): Explain to the player, succinctly, why they should care about this offer. Provide benefits, e.g. ‘get ahead of the competition!’ as opposed to features, e.g ‘get x of this, y of that.’ While I consider this mandatory for certain offers, particularly those targeting new users, it is optional for offers that target experienced users, or offers in very simple casual economies.
7. Bonus:  Animated elements: Animated sparkles, glows, and other vfx are a nice finishing touch, and a clear visual marker that helps further distinguish special offers from base products.

Now that we’ve provided the checklist, let’s apply it to some actual offers!

First, an example of what NOT to do.

Don’t do this!

In Just Kill Me 3, these in-store offers fail to meet our criteria.

1. Generous Size: ✓‘Medium Banner’ or larger: Just Kill Me uses banners of adequate size, but doesn’t allocate space effectively within them (see #4).
2. Killer Hero Image: X The unremarkably small icon serving as hero image fails to generate excitement.
3. Crisp, Retina-quality art assets: X All art assets look overly compressed, with a lack of crispness and clarity.
4. Uncluttered: X All three offers in Just Kill Me 3 are too busy. The offer shown above has 37 words, 6 different text treatments (fonts, sizes, colors), and inadequate margins (space between elements). The overall effect is a sloppy look that detracts from perceived value.
5. Effective Visual Hierarchy:  OK  Despite the clutter, the offer does a reasonable job of emphasizing the title and purchase button.
6. Bonus: Clear Value Statement (where appropriate):  ✓ The three offers in Just Kill Me 3 are complex and different enough to warrant short value statements, particularly since they are aimed at beginners. ‘Create a supremely powerful Pet deck!’ is a clear and concise one, provided that you can find it amongst the clutter. ‘Great value!’ would ideally be communicated elsewhere via a specific bonus or discount amount.
7. Bonus: Animated elements: X None used.

Do this!

Above: The top offer (“Vanquisher Pack”) meets all five mandatory criteria in our list. Art is crisp, text is minimal, margins adequate, and layout cleanly emphasizes the title, aspirational hero image, and purchase button. Additional info is available via the small question-mark button.

The other two offers (“Ascension” and “Brawler”) fall a bit short due to their less-aspirational hero images, but this is likely part of an intentional effort to highlight the first offer.

Don’t do this!

The above offers, from Bricks Breaker Quest, are another illustration of how offers without hero images can fall flat. Even if I want the items being offered (in either pack), the offer presentations aren’t making any effort to add or communicate value, interest or excitement.

To me, the offers say: “Hey you could buy this stuff, if you really want to, I guess.”

Additionally, (if I’m being picky) the containers for the ‘Bonus’ items in each offer are too small, leaving insufficient margin around their contents, and the decision to hide the purchase prices makes it difficult to compare the offers.

Do this!

The in-store special offers above from Word Serenity do a great job of illustrating the essential five items in our checklist.

1. Generous Size: ✓ The two offers use the standard ‘medium banner’ size for in-store special offers. The size difference clearly differentiates the offers from the three basic products below.
2. Killer Hero Image: ✓ These large, lustrous hero images are the critical ingredient that takes these offers from unremarkable to aspirational!
3. Crisp, Retina-quality art assets: ✓ Hero images and other assets are crisp and don’t suffer from any visible compression or distortion.
4. Uncluttered: ✓ Word count is low (8), distinct visual elements are few (11 by my count), margins are adequate, and there is plenty of negative (empty) space, making it easy to quickly digest the offer.
5. Effective Visual Hierarchy: ✓ The glorious hero image(s), purchase button, and offer title are the largest and highest-contrast elements, and thus draw the eye first. Supporting messages, including the resource quantities, icons, and ‘best seller’ badge are deliberately de-emphasized via smaller size.
6. Bonus: Clear Value Statement(where appropriate): N/A Not necessary here, as the special offers’ contents are very similar, save for differences in price and volume.
7. Bonus:  Animated elements: X. Not present, but would have been a nice touch.

—

Key takeaways

Special Offers need to look valuable enough to generate excitement. Thisisn’t the time or place to skimp on art, layout or overall visual polish. Aim for captivating and don’t settle for less.

As we’ve shown above, top-grossing games have established a clear structure for building captivating offers. Let’s benefit from their R&D efforts!

By sticking to the checklist, you can maximize the effectiveness of your special offers while minimizing effort and uncertainty.

—

In the next article in this series, The IAP Merchandising Playbook, Part 2: Visual Hierarchy, we’ build on these fundamentals with a deep dive into Visual Hierarchy for Special Offers, then apply these principles to an analysis of modal pop-up offers from Casual, Midcore and Core F2P titles.

——————————————————-

If you enjoyed this article, you might also like my “How to Fail” series:

How to Fail in F2P Mobile Games, Part 1: Start without a Business Case

How to Fail in F2P Mobile Games, Part 3: Vague Product Goals

Need help with your game? Contact me!

]]>https://turbine.games/2020/10/09/the-iap-merchandising-playbook-part-1-special-offers-that-sizzle/feed/0What is ‘IAP Merchandising?’https://turbine.games/2020/10/09/what-is-iap-merchandising/ https://turbine.games/2020/10/09/what-is-iap-merchandising/#respondFri, 09 Oct 2020 22:41:33 +0000https://appturbine.co/?p=983Successful free-to-play mobile games use effective IAP Merchandising. But what does that actually mean, and what are its components?

The post What is ‘IAP Merchandising?’ first appeared on Turbine Games Consulting.

]]>In my work with 35+ clients, ‘IAP Merchandising’ is a topic that I raise frequently, in service of improving IAP monetization.

But, I’ve found this term to be unfamiliar to many devs.

If that’s you, read on!

IAP Monetization Design

We’ll start with a more familiar concept: IAP Monetization Design.

If we define IAP Monetization Design as encapsulating all considerations related to IAP monetization in a F2P game, then we can divide the discipline into three areas:

1. The Game Economy Model
2. IAP Product Design
3. IAP Merchandising

1. The Game Economy Model

A Game Economy Model is the foundation of every successful game and answers three key questions: 1) how players progress through content, 2) what barriers gate or slow this progression, and 3) how spending real money helps players overcome those barriers more quickly.

The basic economy model can be expressed that simply: ‘gameplay progression works like X, is gated by Y, and players can overcome gates by purchasing premium products A, B or C.’

For example, in Candy Crush, the fundamental Saga Economy Model is:

Progression: Players progress by beating levels in a linear sequence.

Gates: Players lose lives on level failure. When empty, players must wait for lives to replenish. Pain is felt when progress is blocked by losses, and when players must wait for lives.

Premium Products:

• Boosters and Extra Moves help players beat difficult levels.
• Refill Lives or Infinite Lives allow players to bypass the wait.

2. IAP Product Design

The game economy model lays a mechanical foundation that allows for a set of premium ‘experience enhancers,’ like powerups, currencies and other items.

Combining these items and currencies in different amounts, and at various price points, results in a list of IAP products (bundles) to be sold in your game. Any discounted bundles (special offers) should also be in this list.

3. IAP merchandising

Given the list of IAP products and prices, IAP merchandising does the job of defining when, where, how, and to whom these products are actually sold.

Imagine that you have a stockroom full of toys to sell, but no store yet, and no labels, packages, price tags, artwork or sales copy. This is where IAP merchandising comes in.

IAP merchandising includes:

• Visual design

  • Product appearance (art)
  • Store appearance
  • Value propositions, communicated via text and art

• User Experience design

  • Segmentation: routing different offers to different player segments
  • Appearance and ‘feel’ of premium currencies, consumables and collectables
  • Store UX, layout, organizational structure
  • Where, when and how products are shown outside the store

Key takeaway:  Given a list of IAP products, IAP merchandising is the discipline that defines when, where, how (including appearance), and to whom these products are shown, both inside and outside the store.

To learn more about IAP Merchandising, check out The IAP Merchandising Playbook, Part I: Special Offers that Sizzle

——————————————————-

If you enjoyed this article, you might also like:

[How to Fail in F2P Mobile Games, Part 1: Start without a Business Case](/content/2019/06/07/how-to-fail-part-1-start-without-a-business-case/ "How to Fail, Part 1: Start Without a Business Case"/index.html)

[How to Fail in F2P Mobile Games, Part 2: Undisciplined Innovation](/content/2019/08/22/how-to-fail-in-f2p-mobile-games-part-ii-undisciplined-innovation/ "How to Fail in F2P Mobile Games, Part II: Undisciplined Innovation"/index.html)

[How to Fail in F2P Mobile Games, Part 3: Vague Product Goals](/content/2019/10/09/how-to-fail-in-f2p-mobile-games-part-iii-vague-product-goals/ "How to Fail in F2P Mobile Games, Part III: Vague Product Goals"/index.html)

Need help with your game? Contact me!

]]>https://turbine.games/2020/10/09/what-is-iap-merchandising/feed/0Getting more with less: 6 high-yield design patterns for F2P gameshttps://turbine.games/2020/04/22/getting-more-with-less-6-high-yield-design-patterns-for-f2p-games/ https://turbine.games/2020/04/22/getting-more-with-less-6-high-yield-design-patterns-for-f2p-games/#respondWed, 22 Apr 2020 22:04:13 +0000http://appturbine.co/?p=673“We have tons of ideas, but what should we do first?” Having worked with over 30 game teams as a product consultant, I’ve been surprised by how often developers fail to frame the prioritization problem correctly. As a result, I’ve seen …

The post Getting more with less: 6 high-yield design patterns for F2P games first appeared on Turbine Games Consulting.

]]>“We have tons of ideas, but what should we do first?”

Having worked with over 30 game teams as a product consultant, I’ve been surprised by how often developers fail to frame the prioritization problem correctly. As a result, I’ve seen teams spend months agonizing over important product decisions as they repeatedly second-guess themselves.

A simple application of the appropriate perspective (which will sound absurdly obvious) can make short work of these problems by shining a spotlight on which initiatives deserve priority.

The key question is:

Where can we create the most value, with the least effort?

Why? At the risk of stating the obvious, it’s because:

Profit Margin = Business Success

In any enduringly-profitable business, behind the flashy headlines touting ‘exceptional leadership’ and ‘game-changing ideas’, you’ll find the relentless pursuit of a decidedly less sexy goal: how to cut costs while increasing value.

The delta between the value generated and the effort required to deliver it is a company’s profit margin.

But, in an efficient, competitive market, this margin will get squeezed from both ends, as 1) input costs to produce the product or service are driven upward, and 2) price competition decreases what customers are willing to pay.

Our hyper-competitive F2P game industry is a particularly acute example, because:

• Everyone wants in. Because the mobile games industry is massive, sexy, and highly profitable (for a fortunate few), its siren song lures many investors, who then bankroll your newest competitors.
• Competition drives up UA costs: User acquisition / ad marketplaces are real-time and hyper-efficient, quickly driving CPI to the line where profit margins disappear.
• Demand for talent increases development costs: Limitless demand for experienced developers, paired with efficient job marketplaces, empower developers to demand higher compensation.
• Customers now expect free, high-quality games: F2P business models place downward pressure on what customers expect to pay for entertainment.

If we want to survive in F2P games, these competitive realities only leave us with two viable paths:

1. C reate a differentiated product that can’t be easily cloned, or
2. Deliver value with higher efficiency than your rivals .

In reality this is a false choice; the extreme level of competition in our industry demands that we do both. That said, strategy #1, defensibility and differentiation, is a topic for another article. The rest of this article focuses on #2: ‘ Deliver value with higher efficiency than your rivals .’

Finding Efficiencies in Game Design

To survive in this competitive environment, game designers and developers must be able to create products that are 1) adequately differentiated, 2) polished and high-quality, and that 3) offer as much value (entertaining, retentive content) as possible, all while 4) tightly controlling development scope to minimize costs.

The four goals above are by no means a trade secret, but in my experience creative teams tend to focus on the first two goals (novelty and quality) and neglect the second two (highest value possible at lowest cost).

After all, to make the best choice among many options, we need to do more than compare their impact on product quality and value.

Our north star must be return on investment .

A simple rubric for prioritization:

A deliberate but simple process like the one below can help move teams in the direction of high-yield thinking.

Simply list the initiatives your team is considering and then fill in the yellow columns.

Estimating Benefit: To arrive at a useful, apples-to-apples comparison, I recommend framing all potential improvements (whether to retention, to monetization, or otherwise) as improvements to player lifetime value (LTV).

Estimating Effort: When estimating effort, I recommend choosing one of two approaches: either an arbitrary effort scale from 1 to 10, or an estimate of the number of actual weeks of development time.

If you feel like you’re making wild guesses, don’t despair — your ability to estimate benefit and effort will improve with experience. You can also improve estimates (and fast-track learning) by sourcing independent estimates from your team, then debating the results as a group.

The final estimates needn’t be perfect, just directionally correct, to point you in the direction of the ‘right decisions’ given the information available. The result of this exercise is a simple, back-of-napkin ROI calculation per feature. And, in my experience, you’ll have a few low-effort, high-yield candidates at the top of your list.

If, on the other hand, you find yourself short of ideas, here are some to consider.

Six Common, High-Yield Design Patterns

What follows are six, low effort, high-impact design patterns from successful, contemporary F2P games.

Pattern #1: Have players replay content at higher difficulty.

Take something that already exists (e.g. Campaign or game levels), duplicate it, crank up the difficulty and add appropriate rewards.

Marvel Strike Force and RAID Shadow Legends will often simply duplicate the same content 6, 8, 10, even 20 times at increasing levels of difficulty.

Summoners War’s Campaign can be played through three times, on Normal, Hard and Hell difficulty.

RAID Shadow Legends’ dungeons re-use the same content 20 times, by ramping difficulty and rewards, and by changing the elemental affinities of the enemies.

Another common, but more casual approach to content re-use / replay is the 3-star scoring system. Two very different games, Candy Crush and Clash of Clans, both use this to get more mileage out of each game level, by incentivizing players to replay earlier levels to achieve higher scores or more difficult goals.

Across many F2P genres, games with Single Player Campaigns use 1–3 Star Scoring to incentivize replay of existing level content. Above: Candy Crush, Clash of Clans

Combining both patterns: RAID’s PVE Campaign levels have 1–3 star scoring, and the entire campaign can be played through 4 times (escalating difficulty).

Pattern #2: Reuse content, with different rules / restrictions.

Content can also be duplicated and repurposed simply by changing the rules or requirements, in a manner that forces the player to make fundamental changes to their strategy.

My favorite example, due to its effectiveness and, frankly, brashness, comes (again) from RAID Shadow Legends.

RAID’s Faction Wars Feature re-purposes the same Campaign sixteen times, with each instance limiting hero selection to one of the game’s 16 factions.

RAID created a new, 21-level campaign, then duplicated it sixteen times. Sixteen campaigns for the price of one!

Pattern #3: Build an endless runway of ‘meta’ goals that leverage, stretch and re-use existing content.

Once built, an achievements or progressive quest system is very high-leverage, allowing designers to churn out content with very little effort.

When looking for high-ROI content ideas, data-driven content (i.e. content that can be authored in a spreadsheet, without additional art or code) is a great place to look!

Fortnite’s Battle Pass is a glorified Achievements System:

Merge Dragons (above) has three independent, linear quest vectors (hundreds of quests each) that guide players through, and extend, the base content.

RAID’s progressive quest system (above) uses both immediate and longer-term rewards to drive players to progress through, and re-engage with (i.e. farm), existing content.

Pattern #4: Create incentives for players to repeatedly play (or ‘farm’) existing content.

The grind is real.

Games that structure core gameplay and metagame incentives such that players are willing to play the same content over and over have a massive advantage.

Pokemon GO players spend hours, weeks, months happily repeating the same behaviors, over and over, without the need for additional content.

In the typical Slots progression arc, players repeatedly spin on the same slot machines, over and over, to slowly gain XP and unlock later machines.

In RAID, players repeat the same battles tens, hundreds, thousands of times to farm the XP, currencies, or equipment that they need to reach that next milestone.

Pattern #5: Collection, Fusion Pyramids

The fusion pyramid is a mechanic first popularized by eastern, core-leaning character collector RPGs like Summoners War, Epic Seven, RAID, and Idle Heroes.

In the popular RPG, Summoners War, a 1-star character must be fully leveled-up, then fused with 3 other 1-star characters to create a 2-star character, which then must be leveled-up and fused with two 2-star characters to create a 3-star character, and so on. This inverted pyramid shape creates an exponential number of tasks for players to complete, and does so with very few game assets.

Recently, fusion pyramids have undergone a casual rebrand, by replacing complex inventory systems with more casual merge mechanics.

Merge Dragons is the quintessential example, in which merge chains require that players generate and merge three of an item to create an item one step higher on the chain. Thus, creating the 10th item on the chain requires performing 3 to the 10th power (59,000!) merges.

Above, one of many merge chains in Merge Dragons. With just 10 art assets, the chain pictured generates 3¹⁰ or 59,000 tasks for players to perform.

Pattern #6: Extend existing level-up vectors, or add new ones!

As detailed in my previous article on Free-To-Play Economy Spend Depth, if your game has characters, weapons, or other collectables that can be leveled-up individually, increasing their level caps is an incredibly efficient way to increase your content runway and spend depth.

Similarly, adding new level-up vectors to collectable items produces exponentially higher numbers of goals for players to chase.

In recent work with a client, I compared their game’s goal (and spend) depth to that of Empires & Puzzles:

The client’s game (above/left) has an economy suffering from too little dimensionality. Rather than adding any more incremental cards or abilities (expensive), we could instead double the level cap on cards (to 60) and add a new level-up vector (30 levels) to each of the 3 abilities per card.

This gives each of the 100 cards 60 level-ups (100 * 60 = 6000 goals), and 90 ability levels (9000 goals), for a total of 15,000 goals to pursue, all for a relatively modest development effort.

Note that these are mostly data and tuning changes only, which, while certainly requiring time, thought and testing, are orders of magnitude less costly than the addition of new incremental content (e.g. brand new cards or game modes).

So, while substantially lower effort than the development of 50 new cards, the proposal above could have seven times greater impact (11,700 vs. 1,650) in terms of improving goal depth and LTV.

Wrap-Up

Depending on your F2P product’s genre, the examples above may or may not directly apply to your game. If they don’t, how can you apply these ideas right away? At your next roadmap review, just ask:

“How can we multiply what we already have, before building something brand new ?”

Or:

What can we do that could produce big results , but costs very little to try ?”

]]>https://turbine.games/2020/04/22/getting-more-with-less-6-high-yield-design-patterns-for-f2p-games/feed/0How to Fail in F2P Mobile Games, Part III: Vague Product Goalshttps://turbine.games/2019/10/09/how-to-fail-in-f2p-mobile-games-part-iii-vague-product-goals/ https://turbine.games/2019/10/09/how-to-fail-in-f2p-mobile-games-part-iii-vague-product-goals/#respondWed, 09 Oct 2019 21:58:00 +0000http://appturbine.co/?p=670Previously, in “How to Fail in F2P Mobile Games…” In Part II of “How to Fail in F2P Mobile Games” we touched on three distinct approaches to product and feature design: Adapting Proven Solutions, Forward-Innovation, and Pure (blank slate) Design. Now, in …

The post How to Fail in F2P Mobile Games, Part III: Vague Product Goals first appeared on Turbine Games Consulting.

]]>

Previously, in “How to Fail in F2P Mobile Games…”

In Part II of “How to Fail in F2P Mobile Games” we touched on three distinct approaches to product and feature design: Adapting Proven Solutions, Forward-Innovation, and Pure (blank slate) Design.

Now, in Part III, we’ll discuss how a clear understanding of and consistent focus on our key product goals can shed light on these difficult, daily choices.

Let’s return to this fundamental, critical question:

“Should we copy a feature (or product) from our competitor, improve on a competitor’s design, or invent our own approach?”

– F2P Game Designers, every day of the week.

I have three recommended rules for approaching this problem:

Rule #1: Focus ALL Innovation on 2–4 Key Features, representing ~30% of total scope. Use Proven Solutions, unabashedly, for the other 70%.

My recommendation: Choose 2–4 major game features or areas, comprising roughly 30% of your game feature scope, as your “ Key Innovation Targets. “ These features should be highly valuable to your target players, and the key focus of your marketing and product positioning.

As a general guideline, I recommend a 70/30 split between emulating Proven Solutions and innovation, with a clear line between the two.

For the remaining (70%) of your game feature scope, I recommend tightly conforming to Proven Solutions. By saying “no” to innovation outside of your KITs (Key Innovation Targets), you limit non-KIT design time to < 30% of your total budget, leaving 70% for your KITs. To stack the odds in your favor, designers should choose feature references (games) carefully in order to pluck Proven Solutions that are as plug-and-play as possible.

When done properly, your designers should expect to spend 70% of their time creating, testing, and refining the innovative 30% (2–4 KIT features) of the game.

Most designers and teams will appreciate having clear objectives, and the breathing room to focus on KITs. Your players will benefit from this approach too, as they will already be familiar with 70% of the game systems and only need to learn, at most, the innovative 30%. Don’t underestimate this benefit, as heavy tutorialization creates agony for both your players AND your development timeline.

Note that this paradigm does not reduce the time that you spend innovating. We just choose to direct all of that effort toward those 2–4 Key Innovation Targets — the features that will become our primary product differentiators — and we consciously agree to leverage Proven Solutions for the rest (in order to meet our launch date).

The real objective here is a bright line leaving NO room for ambiguity, creating a massive level of clarity for the development team, execs and other stakeholders. By consciously identifying all of the areas in which we will NOT innovate, we are able to prevent unnecessary conversations, quicken meetings, speed development, reduce risk, and laser-focus the creativity and imagination of the entire team on the 2–4 key areas that matter most.

Rule #2: ALL other innovation goes to the backlog

Great! So now you have defined your Key Innovation Targets: the 2–4 unique features that will be most meaningful to your target audience, and most critical to your game’s ability to achieve product-market fit.

After some necessary due diligence, your next task will be to deliver a test product to that market, with those 2–4 innovative features, as quickly and inexpensively as possible.

This test launch will confirm whether your hypothesis for product-market fit was valid, i.e. whether there exists a sizeable market of users craving your 2–4 innovative features enough to 1) grok your key differentiators in a targeted ad, 2) download your game, and 3) stick around and engage with it.

Said more simply, the ONLY goal at this stage is to find out if your 2–4 KIT features resonate strongly enough with your target market to justify further investment.

Sidebar: In most cases, the answer will be no, which is why you want to get to this answer quickly (more on this in the next article)!

All features aside from your Key Innovation Targets can be thought of as “Supporting Features” which, for the purposes of this initial product-market fit testing, simply need to ‘exist’ (and to not ‘completely suck’) while allowing players to have a reasonable initial experience with your KITs.

These Supporting Features are likely to include the typical F2P fodder: store, offers, currencies, leaderboard systems, quests, daily rewards, item and inventory management, notifications and announcements, mailbox, chat, player profiles and so on. Because nearly all of these features exist as Proven Solutions in other top-grossing games, and because players are generally quite comfortable with the status quo, your team should be unabashed in copying them.

BUT, what if your team has credible, creative ideas on how to innovate on and/or improve these Supporting Features too?

Let me be clear:

Do not waste innovation budget on Supporting Features prior to product-market fit testing.

If you thought, for example, that an innovative leaderboard design was critical to players, then you would have chosen leaderboards as one of your Key Innovation Targets.

But, you didn’t. You instead decided, explicitly, that innovation in other areas would be more meaningful to players.

So, capture all of your team’s great ideas, but put them in the feature backlog for now.

If your product’s KITs resonate strongly with the market, there will be plenty of time for further innovation and polish.

If your KITs don’t resonate, innovative and/or polished Supporting Features aren’t the problem, and won’t save you. The target market, or the Key Innovation Targets will need to change until you find product-market fit.

Rule #3: Get your team 100% aligned.

To reap the rewards of focused innovation and rapid testing, your team needs to be 100% clear on your 2–4 Key Innovation Targets, and fully bought-in on the deferral of all other innovation until after product-market fit testing.

In my experience, achieving and maintaining this alignment can be quite challenging, given developers’ natural inclination to analyze, critique, create and improve!

“If we backlog all of our good ideas, the product will be ‘average’ and fail product-market fit testing!”

– Hypothetical designer

“Plus, why WOULDN’T we improve a feature if we know we can do it better?”

– Hypothetical designer

These are common sentiments that come from a good place: wanting to make a great, successful product. However, they are based on some incorrect assumptions, and can be quite harmful to products and teams.

1. “The market responds STRONGLY to innovative Supporting Features.” RARELY TRUE. Designers are accustomed to focusing on small details, and can always find ways to further perfect products. As a recovering perfectionist, I’ve learned over the years that the broad market judges a product by very different criteria than game designers. Generally speaking, new players (particularly in the casual and hyper-casual markets) will form a quick, initial impression given a product’s theme, genre, and core mechanic, and then decide whether to stick around. While polishing all features to a high shine will move the needle for lifetime value, we’ve repeatedly seen that casual and hyper-casual players will happily engage with largely ‘unpolished’ games if they resonate with the core mechanic and premise. More on this in the next article!
2. “We always have time for more innovation.” WE DON’T! If we want 70% of designer time focused on our KITs, we’ve only left 30% for Supporting Features — the bare minimum for implementing Proven Solutions. Any deviation from Proven Solutions requires unscheduled time, which will come at the expense of our KITs: not a good tradeoff. Keep the good ideas flowing, but put them on the backlog until you’ve proven product-market fit
3. “Innovation only costs designer-time.” FALSE. Innovation is expensive and its costs are borne by the whole team. It requires implementation, tuning, QA, and communication overhead; innovation also dramatically increases the risk that features need rework, meaning additional wave(s) of unplanned support and effort. Welcome to crunch time!
4. “Different always means better.” BUT, DOES IT? Again, we must make a first-order distinction between KITs and Supporting Features. It’s true that a game needs a few differentiators to stand out. It does not follow that ALL game features will benefit from differentiation. A Proven Solution is (definitionally) proven, and any amended version has yet to be. So while it’s certainly possible that changes or improvements could yield meaningful gains, it’s more likely to yield no results, or even negative results relative to the Proven Solution, while costing precious time.
5. “But, [X specific feature] is worth an exception.” HIGHLY UNLIKELY. Yes, a small innovation here or there could certainly make the product slightly ‘better.’ But, the actual KPI benefits are likely to be undetectably small. These benefits likely pale in comparison to the incredible clarity and efficiency that a bright-line rule of ‘all other polish and innovation goes to backlog’ brings to the team, process and product.

Wrapping Up

To summarize: for most projects, I recommend choosing 2–4 Key Innovation Targets as product differentiators for your initial test release, and using Proven Solutions for everything else.

By focusing your efforts on the few features that will truly define your product’s success or failure, you avoid wasted effort, unnecessary bugs, meetings, and overhead. You mitigate the possibility of late-nights and crunch time. Everyone (your players, your developers, your investors) stand to benefit.

In the next article, we’ll discuss how to bring a product to market for the first time!

]]>https://turbine.games/2019/10/09/how-to-fail-in-f2p-mobile-games-part-iii-vague-product-goals/feed/0Glu Mobilehttps://turbine.games/2019/09/27/glu-mobile/ https://turbine.games/2019/09/27/glu-mobile/#respondFri, 27 Sep 2019 10:54:44 +0000http://appturbine.co/?p=619‘Glu hired Matthew to work on one of our most important gaming properties…Matthew came up to speed, seemingly instantly, and has been providing deep insights into the user funnel and game economy. Most impressive is Matthew’s versatility… If you have …

The post Glu Mobile first appeared on Turbine Games Consulting.

]]>‘Glu hired Matthew to work on one of our most important gaming properties…Matthew came up to speed, seemingly instantly, and has been providing deep insights into the user funnel and game economy. Most impressive is Matthew’s versatility… If you have a skills gap anywhere in the analytics to product management spectrum, I highly recommend Matthew!’

The post Glu Mobile first appeared on Turbine Games Consulting.

]]>https://turbine.games/2019/09/27/glu-mobile/feed/0How to Fail in F2P Mobile Games, Part II: Undisciplined Innovationhttps://turbine.games/2019/08/22/how-to-fail-in-f2p-mobile-games-part-ii-undisciplined-innovation/ https://turbine.games/2019/08/22/how-to-fail-in-f2p-mobile-games-part-ii-undisciplined-innovation/#respondThu, 22 Aug 2019 21:52:00 +0000http://appturbine.co/?p=666The F2P Designer’s Dilemma Like many mobile game PMs and designers, I cut my teeth in premium console game production. Down in the trenches, creative decision-making flowed freely, and we were blissfully ignorant of and unperturbed by broader business realities …

The post How to Fail in F2P Mobile Games, Part II: Undisciplined Innovation first appeared on Turbine Games Consulting.

]]>The F2P Designer’s Dilemma

Like many mobile game PMs and designers, I cut my teeth in premium console game production. Down in the trenches, creative decision-making flowed freely, and we were blissfully ignorant of and unperturbed by broader business realities and performance metrics.

My first few years in games were spent in sound effects design, and the work was as close to ‘pure creativity’ as one could imagine. I recall spending three whole days creating a single sound-effect for an Xbox horror/action game (for vampiric, blood-powered dual sub-machine guns). Those were the days!

F2P changed all of that. The ‘try free’ business model, and broad availability on Facebook, mobile and other ubiquitous platforms opened the floodgates to a colossal new market, a blue ocean of non-gamers who might have never installed a game in the ‘pay first / console’ paradigm. This new market of players preferred more casual, bite-sized experiences to the heavily crafted, polished, 3D epics that dominated the console market.

This new, untapped market expanded rapidly and, by 2009, most of the best job opportunities in game development were in the emerging F2P space. It was perhaps inevitable that, sooner or later, most console developers would make the jump.

For some, the transition from console to F2P was a bit jarring. Where premium games tended to optimize for creativity, immersion and quality of experience above all else, F2P’s mandate was ruthlessly pragmatic: to focus on speed, effectiveness and agility in product design and development, in service of driving players to repeatedly and compulsively engage in the specific, measurable behaviors that the business model demanded.

For many designers and developers fortunate enough to experience the “high art” of premium product design, the subjugation of the old values of originality, depth and quality to F2P’s new mandates: pragmatism, speed to market, and KPI measurement came as quite a culture shock.

Effective F2P developers must be both creative and metrics-driven.

With all that said…

I would venture that ALL F2P designers, whether reared in the console ethos or not, frequently find themselves squeezed by the competing demands of creativity and pragmatism.

Consider the following question:

“Should we copy a feature (or product) from our competitor, improve on a competitor’s design, or invent our own approach?” — F2P Game Designers, every day of the week.

This question reminds me of the alleged Mark Pincus (Zynga) quote, which achieved notoriety due to its… inelegant… handling of the same uncomfortable question.

“I don’t [**#%$] want innovation,” (an ex-employee recalls Pincus saying). “You’re not smarter than your competitor. Just copy what they do until you get their numbers.”

Was Pincus wrong? What’s a well-intentioned game designer to do?

In this article, I’ll seek to reframe this question from my perspective as both a product manager and designer, so that perhaps we can bring a bit more confidence and clarity to our decisions about whether to innovate, “copy” a competitor, or aim somewhere in between.

Let’s examine these choices in more detail.

Three Types of Design Solutions

First, we’ll define ‘design solution’ as “a design for a particular system that meets our game’s specific requirements.”

For example,

• Design problem: “Our live, casual game is launching a “Leagued Tournaments” feature, and needs a design for a scoring system that supports 1) promotion of successful players to the next league, 2) demotion of unsuccessful or inactive players, and 3) rewards that improve with league advancement.”
• Design solution: “Our proposed design for MVP launch of Leagued Tournaments is based tightly on Gardenscapes’ Leagued Tournaments, with the following minor tweaks needed to adapt it to our game model: #1, #2, #3 etc.”

When choosing how to approach a particular design problem, I’ve seen three main paths, which differ in 1) whether we research competitors’ solutions, and 2) to what degree we innovate. And over the years, I have learned (often the hard way) to be very explicit about these choices, because innovation is expensive (time, effort, distraction) and adds risk to our projects.

1. Adopt a Proven Solution

For most design problems, adopting a Proven Solution is by far the most effective approach.

Definition: When our design problem is not unique, we can quickly review solutions from other top-grossing games, and choose a solution that is highly likely to work very well in our game with minimal modification. In nearly no time at all, we have an 80/20 solution. And, because F2P live game development is an iterative process, we can always make future improvements if needed.

That last point is worth repeating: We can always make future improvements if needed.

For example, ‘daily login rewards’ is a ubiquitous, proven solution to the problem: “We need a simple mechanism to help drive a bit more retention and stickiness.” And because MOST top-grossing games use similar variants of this solution, we can have a high level of confidence that a nearly identical solution will meet our needs.

Pros:

• Proven Solutions are low-risk and low-cost, having benefited from plenty of iteration and testing at scale.
• They are easy to communicate to engineering.
• Players are already familiar with them.

Cons:

• Proven Solutions won’t meet players’ desire for originality, so we’ll need to solve that elsewhere (via our Key Innovation Targets, which I’ll cover in the next article).

Summary: When your design problem is not unique, it’s smart to start with Proven Solutions, conserving your resources for the more difficult problems ahead.

2. Forward-Innovating Solutions

Definition: When warranted, we can attempt to improve upon a Proven Solution from other top-grossing games. Or, after digesting the competitors’ approaches, we could seek to create a wholly new, alternative solution that we believe will perform even better.

But either approach – improving upon prevailing solutions or inventing your own – is likely to consume an outsized share of your project budget.

For this reason, such innovation should be used sparingly, and only on a few of your most important features. From my experience, innovative designs are generally only appropriate 1) during initial development, to support a major product differentiator or 2) after the initial launch of a product or feature, to further improve its performance.

Outside of these two scenarios, I will generally challenge teams to play it safe and walk the beaten path (Proven Solutions).

Pros:

• Forward-Innovating Solutions can serve as major product differentiators.
• They can improve player experience, KPI, or both.

Cons:

• Innovation is risky. You are betting that your new, un-tested solution is superior to those of (usually multiple) high-grossing competitors, staffed by smart people like yourself. Statistically speaking, this is probably not a good bet!
• High quality, innovative solutions are very expensive and time-consuming to develop, and typically require iteration and tuning. So, you can only afford to innovate in a few key areas.

Summary: Always review your competitors’ solutions first. Innovation is expensive, so choose your battles. Save your innovation budget for your Key Innovation Targets.

3. Pure Design Solutions (start with a blank slate)

Pure Design is great for out-of-box thinking, inefficient in other cases.

Definition: Many designers approach design problems, by default, with the generative ‘bottom-up’ method. They start with a blank sheet of paper and end with a boutique design, meticulously crafted and born of the designer’s imagination.

The defining characteristic of this approach is that it skips an audit of how successful competitors have handled similar design problems.

To many designers, this feels like the intuitive, ‘right’ approach to design. After all, isn’t a designer’s job to… dream up new designs? I can certainly relate, as in my early years as a F2P designer I too succumbed to this logical fallacy:

“As a capable game designer, why copy what I can design myself?” — Novice F2P Game Designer

The problem is, in the vast majority of cases, this ‘Pure Design’ approach is the least effective of the three.

Why? Because most design problems aren’t unique, and have therefore already been thoroughly solved by your (numerous, smart, and successful) competitors.

And, to the same extent that your Pure Design solution is new and original, it is also hypothetical, unproven, and first-generation, and will therefore most likely underperform a mature, proven solution, while taking you more time (and cost) to develop. Not a great tradeoff!

All that said, Pure Design CAN be a great initial exercise, to generate broadly creative ideas, illuminate opportunities, and have a good bit of fun. BUT, it is no substitute for also learning from your competitors before settling upon a design solution.

Out-of-box thinking via Pure Design can be useful in some contexts, woefully inefficient in others.

Pros:

• With Pure Design, it’s possible (however unlikely) to stumble onto superior, innovative solutions.
• Unconventional designs are likely to please novelty-seeking players, provided that the solutions work well and can be easily learned by players.
• Many designers intuitively love this approach — it can be motivating to the team and good for morale.
• As a standalone exercise, Pure Design can generate useful creative ideas!

Cons:

• Competitors’ solutions can provide a great deal of information about what works, what doesn’t and what unanticipated problems may surface. Ignore them at your peril.
• If your design problem is not unique (the dominant case), then you’re likely to invest significant time creating and implementing a design that’s inferior to a Proven Solution already used by your competitors!

Summary: In F2P games, Pure (blank page) Design is a great exercise for generating creative ideas. Just don’t forget to ALSO thoroughly absorb the ideas and solutions of your successful competitors as well. In practice, I’d only recommend using Pure Design as a standalone exercise, for ideation around your Key Innovation Targets.

Three approaches to product / feature design. The RED path is not recommended.

]]>https://turbine.games/2019/08/22/how-to-fail-in-f2p-mobile-games-part-ii-undisciplined-innovation/feed/0