At a glance: Combine Level Analytics with A/B tests to execute on strategies that aim to improve your game engagement.
After analysing data to understand the user behavior and some key problems with the level design, it’s time to turn it into action with A/B test, allowing you to make changes and validate your hypothesis with ease. Level engagement will be the key success factor for your game.
Use case 1: Custom difficulty level to improve retention
Step 1: identify levels that have low Retention Rate and high Abandonment Rate
In this example: Level 3 has some trouble where we can see a sharp drop of Retention Rate.
Step 2: Zoom in on Number of fail at this level
Focus on Level 3, we can group "Users per No. of Fails" and see that users are trying multiple times but still not passing game level 3, indicating that the motivation is high but somehow it’s too difficult to complete.
Looking into the "Level Fails" reasons, the majority of level fails was due to a lack of space, where they still had plenty of time to play the game.
Step 3: Take action with level difficulty A/B Test
Set up the A/B test on the Level 3 with the following:
Hypothesis:
If a user can pass level 3, then they can continue to play our game and increase its LTV.
Key Metric:
Retention and abandonment rate at latter level (level 10).
Supporting Metric:
Engagement and time to retry
How:
Set up audience for A/B test: Only those who finished Level 2
Set up the A/B Test on Firebase
Control: Keep the same board
Variant: Make the board bigger, or give the board more space
Leverage A/B test tool from Cost Center to validate result
Use case 2: Custom Monetisation strategy with IAA and IAP
Step 1: Find the opportunity for a highly engaged level
Level 19 has strong average level start per user, indicating strong motivation of users of accomplishing this challenge
Step 2: Zoom in to the User fail data
Using the number of failed dashboards for level 19, we can conclude that user are keen to get over this level to continue playing the game, however they cannot do it successfully.
Step 3: Understand the monetisation strategy at this level
Force ads (Interstitial, collapsible banner, banner ads,...) are still dominant for level 19, related to the times users spend in the game and their retry number.
However, the contribution of Rewarded videos is low, indicating the opportunity for us to show relevant offer supporting users get over the challenge
Regarding the IAP, we can see the missed opportunities at level 19 where there are no IAP happening.
Step 4: Find insight to show relevant offer
“Out of Space” is still the biggest reason for failure, however we can see bigger share of “Out of Time”, indicating new challenge that users are not familiar with before
Step 5: Set up the A/B test to maximise revenue at level 19
Set up the A/B test on the level 19 with the following:
Hypothesis
Bby offering more relevant offers via Rewarded Video and IAP, we can increase the LTV of users.
Key Metric
LTV of rewarded video and IAP
Supporting Metric
engagement and time to retry
How
Set up audience for A/B test: Only for those who started Level 19
Set up the A/B Test on Firebase
Control: Keep the monetisation as now
Variant 1: Reduce Force Ads frequency after 3rd retry and increase IAP & Rewarded video offer
View Rewarded video get 1 booster
Purchase IAP to get 10 boosters
Leverage A/B test tool from Cost Center to validate results