Since 2001, Netherlands-based gaming company GameHouse has quickly picked up industry shifts. 10 years ago, they started tailoring their stories to female gamers, and began collaborating with television writers to craft narrative-driven games. “What sets us apart is the depth of the story that we go into, which is quite high for a video game,” says Sharath Kowligi, Director of Ad Monetization at GameHouse. Five years ago, they revamped their strategy for an increasingly mobile-first world; and three years ago, GameHouse launched its first free-to-play game on mobile.
GameHouse’s first free-to-play game, Heart’s Medicine - Doctor’s Oath, managed to engage 10x more players than their previous, premium games. This influx of users forced GameHouse to reassess whether it was using the best monetization solution. They relied on testing for answers but their approach only gave approximated revenue numbers and made data-based decisions difficult. To tackle this, GameHouse tried out AdMob’s new mediation A/B test tool, and ran a test between waterfall mediation and bidding. The tool showed that bidding grew total revenue by 12% and provided new possibilities in experimentation.
A need for data-driven testing
GameHouse’s biggest challenge was that their existing testing strategy was complicated and didn’t provide clear results. In fact, the company didn’t have a proper way to test advertising revenue strategies. Instead, they would turn on a specific monetization solution for a few weeks, track changes, and disable it in place of another solution. To test ad revenue in regions, the team would enable a solution in one region while disabling it in another, and measure changes manually.
However, different geographies and timelines introduced disparities in revenue spikes. For example, it was difficult to account for regional events, national holidays, or cultural or political occurrences that could skew results. “We needed a better way to A/B test, since what we were currently doing was not a proper test,” says Sharath. Although the team behind Doctor’s Oath were strong advocates for experimentation, they didn’t have a proper internal solution — leaving GameHouse to rely on outdated testing practices. Recognizing the need for a new solution, Sharath kept an eye out for new features and soon found one in its long-time partner, AdMob.
Straightforward setup and an easy roll out
In August 2019, Sharath learned about AdMob’s new A/B test solution and was eager to start the testing process. The biggest concern for the team was having to change the game’s code — which can be time consuming and cause potential issues to the game. “The best thing about the mediation A/B test tool is that it required no change to our binary code since we already had the latest version of the SDK. This was great because getting engineering bandwidth is always a challenge, and the conversations around integrating new things can be circular and slow,” mentions Sharath. In fact, GameHouse did not have to pull any extra engineers in order to set up the test. “I could essentially set up the tool on my own because of the way it's built, and it took no time,” Sharath says.
With the latest SDK in place, Sharath turned the A/B test toggle “on” and selected his demand sources. The goal was to test between bidding and regular waterfall mediation, and see which solution provided a higher revenue boost without manual manipulation. “There’s always going to be some amount of hand holding required with a new tool, but AdMob’s support team was incredibly patient,” he says, “the instructions were really clear and the overall test was easy to implement.”
Sharath’s perception of risk for this new tool was low, so the test was turned on for iOS users in all 40 of their markets — with 50% of users on bidding and 50% on a regular waterfall. For three weeks, GameHouse tracked match rate, effective cost per thousand impressions (eCPMs), and total revenue uplift.
Clear data simplifies decision-making
The results were clear: Bidding significantly increased total revenue by 12%, with eCPM and match rate staying steady. The tool had projected a 13% uplift with bidding, which closely matched the actual results. But the revenue boost was part of a much larger win for the team. “Bidding and A/B testing are both powerful tools; but speaking from a long-term point of view, A/B testing is going to be the biggest gamechanger. To be able to quickly and easily test, and get guaranteed results is fantastic. We’ve loved the opportunity to learn and be a part of something cutting edge”, says Sharath.
As a result of their findings, GameHouse has moved to bidding completely on iOS and are planning to run the same test on Android. GameHouse is happy to have a tool that can run tests quickly and provide actionable results. “Mediation A/B testing has gotten us thinking about what else we would like to test in ad monetization now that this is possible,” says Sharath. “The experimental process is what helps us get better.”
