Video Transcript
0:00 | Adam Creamer
Today with us on this session, we have Shane evans from another partner of ours here at kobiton called game driver, and Shane is going to talk all about mobile gaming because we know mobile is hard and mobile gaming is even harder and Shane and his team know how to help with that. So, Shane, the floor is yours?
0:18 | Shane Evans
Thanks, Adam. Let me just make sure I got everything set up here.
0:28 | Shane Evans
Maybe let me know if you can see my screen, that would be fantastic.
0:35 | Adam Creamer
Yep. You are good to go.
0:38 | Shane Evans
Okay, great. So hi, everyone, my name is Shane evans. I’m one of the co founders at game driver. And if you joined us recently for a webinar, that was just game driver and kobiton. This might be a bit of a refresher, but we felt it was important enough that we wanted to come and share this sort of content again to get people talking about or thinking about mobile testing and gaming, which is our forte, not specifically mobile but gaming in general. And this partnership that we have with kobiton is really kind of a first of its kind and we’re really excited about working together with kobiton and therefore, we wanted to come out and discuss that a bit again. So I’m going to jump right in and talk a bit about mobile testing, specifically mobile testing and gaming, and why it’s important a bit of the market dynamics and what brought us to working together with kobiton. Some of the special sauce that’s provided by that relationship. And then we’ll close it off with a bit of a demo and see how that goes. All right. And of course, Adam and jump in at any time. With any questions. I do like to keep it interactive. So if I’m not looking at the chat, if something pops up, that seems relevant to what we’re talking about, please jump in. So just a few quick facts about the mobile gaming market. Many people don’t realize. But gaming as a whole is the largest entertainment segment in the world. It’s larger than digital music. It’s larger than movies. It’s larger than all the professional sports combined. So it’s an incredibly large market that’s pandemic, proof. It’s recession proof and really continues to exceed its expectations year over year with very few exceptions. Now, the largest part of that market has become mobile gaming. And that might be surprising to some of you gamers who are watching. I’m one of them and gamers historically have treated mobile gamers as not real gamers but the stats and the revenues clearly show that mobile gaming is in fact the most gamer sort of segment of the market that it represents 50 percent of the 190,000,000,000 dollar gaming market overall. And it is the fastest growing segment. Estimates are anywhere between 204 100,000,000,000 dollars for mobile gaming alone by 20 27 depending on who you ask. And the top three markets in this space are China, India and the us. But by revenue, it’s actually China us and Japan. So it’s quite significant. And within that, and there’s millions of apps between the iOS and android app. Stores there’s literally millions of apps to choose from, but over 1,400 apps in 20 22 generated over 10,000,000 dollars each of the remainder 224 surpassed 100,000,000 dollars in revenue and 10 of those went over a 1,000,000,000. So this is not just fun for fun’s sake, but it is in fact big business. And so there’s a lot on the line. There’s a lot at stake. Mobile gaming is also the most inclusive of all entertainment mediums and not just gaming. And it includes representation from all age groups, genders, backgrounds. You name it. According to newzoo, there was 2,000,000,000 players globally in 20 22, and that number is expected to grow to three point 5,000,000,000 by 20 25. So over 50 percent increase. Mobile games are equally almost equally popular among men and women specifically 53 percent Ish of the overall market is men and the other 46 percent are women and people of all ages play mobile games, pretty standard bell curve with, the average gamer sitting around 36 years old. But more than half of players make are players in this space are over the age of 35. And finally, I wanted to point out how fierce the competition is in this space. It’s incredibly fierce. As of 20 23, there’s over 700,000 games alone in the app stores with about 500,000 on Google play and 200,000 on the apple app store. I know I’d said millions earlier in, this sort of factors out a lot of the clones and non monetized games they’re called hyper casuals. In some cases, quite a significant market size. The saturation is significant. And so it’s quite important that you stand out in this space that you stand separate from the rest of the market in terms of feature and gameplay, but also in terms of quality, which is of course why we’re here today. There’s several options available in any category, but they have similar entertainment value. Players will always choose the game with least amount of issues, least amount of bugs and gamers are, very picky for those of us who play games. If something’s not working. I mean, there’s plenty of options, but I want to spend my time on something. It’s got to be something of high quality simple as that… some of the challenges of developing modern games in this space. So there’s quite a lot. But just to touch on a few, the rising user expectations as mobile device hardware improves, there’s this expectation for games to boast near console or even PC like graphics. We’re talking high resolution, textures, realistic lighting, gameplay, the detailed character models are becoming increasingly the norm. And so a couple of years ago, if you were to compare mobile games like flappy bird or crossy road or something like that to console games, there was no comparison. But these days the lines are starting to blur between what we can do with the phone in our pocket and what we can do with the computer or the console that we plug into our TV. We’ve got more complex game mechanics, diverse gameplay elements requiring multifaceted user testing such as multiplayer titles, fortnite pubg are some of the largest titles in this space. And Ar and VR applications like pokemon go that make use of the devices, lidar and camera and Ar capabilities, features like leaderboards, friend systems, chat. So no longer are we bound to testing or playing a game by ourself on our mobile device, it’s become a much more social and connected experience. And lastly, in this space, there’s more regular updates than any other. In fact, console and PC gaming, unlike any other medium, see constant sort of updates and content. So things like fortnite that I mentioned will receive a seasonal updates as well as bug fixes, and that sort of thing. But these games tend to last for many years. And so gamers are expecting of regular content updates. But on mobile, it’s actually much more common. So seasonal can mean, you know, two to three months in some cases. But on mobile devices, the games tend to be updated much more frequently sometimes in months or even weeks. And this sort of constant iteration and improvement. So testing for bugs and compatibility for every single release. Every update is super critical. In this space. We’re also dealing with multiple monetization models. So you’ve got everything from direct purchases which are becoming increasingly less popular over time. As, you know, the past, you could buy a game and it would be yours in perpetuity. In fact, some of those games have been pulled from the market. Game developers are finding that those are not sustainable models. If you reach a certain number of users and you’re not increasing, then revenue tends to dry up. So they’ve switched to models like sort of paid advertisements where, you know, you play through a level and you get an ad. If you want to get a power up or a little extra time, you get an ad, things like in app purchases for things like items and cosmetics are very popular and virtual currencies. Where if you’re looking to get a specific thing or a capability in that game, you got to buy some coins and those coins translate into something that you can exchange for extra power ups or extra. Lives et cetera. In most cases, these are actually not standalone. And so you’ve got multiple sort of mixed modes of monetization to test when you’re creating these games. And it’s critical that these work. There’s been several examples. Recently, I don’t want to mention any names and shame anyone. But we talked to customers who suddenly from one release to the next, their monetization platform wasn’t working and they literally lost millions of dollars because of that. And so a small piece of functionality in any other environment in this case can mean sort of a critical loss of revenue. And lastly one that’s near and dear to the hearts of many here is the mobile device fragmentation. So this is a tale as old as time. In terms of mobile testing. I’ve been in testing and test automation for well over a decade now approaching two. And as long as I’ve been in this space and mobile has been a thing. The challenge has been to test on all of the major devices, of course, but also the devices in different regions, the different form factors, the different capabilities. And it’s not like in some other spaces where you’ve got one target console to work with or PCS, or sort of this general thing that you need to work from mobile devices are unique, but they also tend to last many years. So you’ve got devices that are several years old still in the market and you’ve got the latest and greatest that you need to keep up with. So this sort of matrix of testing just increases exponentially when you consider the variety of mobile devices that are out there and need to be tested specifically in gaming where things like performance, the device capabilities that we talked about earlier. All of these things change very quickly in mobile. So a game that worked on your phone two or three years ago might not work with the latest updates or if you’ve got a brand new game and you’ve got an older phone, you might be out of luck. So mobile gaming is very difficult as a market. It’s difficult for developers and it’s difficult for testers. It’s really in this case where we’re in need of a new approach for testing. We’ve got mobile games that are unique. We’ve got different form factors, different capabilities in each of the devices, but the games themselves are much more difficult to test. So if you’re comparing to like business applications, some of the other speakers have mentioned, you know, there’s much less sensitivity to the performance of the device, the refresh rates or the frames per second that you’re being able. To view or interact with the device itself. These can all affect gameplay significantly. It’s often an always on experience with the monetization models that I mentioned. These games are ad supported. They’ve got online stores that are exchanging real money. They’re multiplayer and they’re using real world or augmented reality capabilities, which is a unique capability to mobile and mobile gaming. Specifically, mobile games are very responsive. It’s very dynamic. It’s not like a form based application where you’ve got, you know, banking for example, you want to perform a transfer and yeah, there’s going to be timeouts and session ids and that sort of thing. But in gaming, you’ve got to respond very quickly. So, for testing a game that needs to be very responsive, it becomes very difficult even in manual testing. How do you achieve that sort of responsiveness and repeatability is key there. So being able to do it once is great. But you need to be able to do this for every device, every form factor, each, and every release, et cetera.
13:13 | Adam Creamer
Hey, Shane, I wanted to jump in here real quick.
13:15 | Shane Evans
Yeah, go ahead.
13:17 | Adam Creamer
You’re talking about like device fragmentation and the performance of those devices in the world. Like how do you guys approach? Like the multiplayer gaming? In that instance? Like if you’re on an iPhone, I’m on a pixel three, like how do you ensure those experiences are similar?
13:34 | Shane Evans
Yeah. So game driver which I’ll jump into briefly here. I’ll just throw up the slide while we’re talking one of our sort of core capabilities is being able to interact with the game as if you were a real user. And so that means providing inputs like a real user would, if it’s touch tap, you know, you’re moving the device around or you’ve got some sort of peripheral plugin as some games do support. Our goal is to test as if the user was testing but in a more automated consistent manner and that includes mobile or sorry multiplayer testing. So in a scenario where, and we have many customers doing this where we’re interacting with different devices and different platforms, we can run the same tests across those different platforms without modification. It’s really just we’re connecting to an android device. We’re connecting to an iOS device. We’re connecting to a PC or console and running the same tests in parallel so that we’re able to interact between these. And the automated approach allows you to even coordinate between the different, you know, users that are being executed during those tests. Great question. Thank you. So really our goal since 2018 has been to introduce this sort of scalable reusable approach to automated testing. And it’s the first of its kind in that we’re able to support multiple game engines similar to selenium, for those of you familiar with selenium and appium, and the sort of standardized approach to testing browsers and mobile devices. We took the same approach with game engines. So developers familiar with unity can quickly change gears from unity to unreal and develop and work with game driver tests in much the same way. And we recently added these other engines, unity being our first. But unreal and godot were added earlier in 20 22. We just had a major release like yesterday. And so we’re very proud to support all of the latest versions of unreal and unity as part of that. And for those of you unfamiliar, unity and unreal combined make up about 80 percent of the gaming market. There’s a lot of engines out there that a lot of sort of bespoke or niche engines, one which we support called godot which is an open source community, loved engine. And game engines are where we create these experiences. So unlike websites which can be crafted in multiple different languages and frameworks, game engine is how the games are made. So we’ve based our solution on sort of open standards which allows us to test very rapidly in those types of environments that our customers are expecting using tools like Jenkins and GitHub. And end unit for execution and that sort of thing, which means that we’re easily portable to all the different platforms as well. So you can see some listed here, nintendo switch, xbox, etc. But today we’re focusing on mobile… very quickly how gamedriver works. We drop a small tool, an agent in the game itself. One of our sort of core design philosophies was making no changes to the game itself in order to facilitate automated testing. So you drop a small agent into the game and then you build it for whatever platform that you’re looking just for some of those are shown on the right. And this allows us to control the game, but also to interrogate and manipulate objects at runtime. So we’re able to control things within the game so that we can set up these sort of complex scenarios and execute tests as a user would. But also in a way that’s more efficient. So we’re not just recording and replaying every touch and tap on a device which we can also do. But we’re able to set up a scene and run a test, move to the next scene, run another test. So very consistent approach to doing that. And as you can see with some of the code that’s shown on the bottom left, there, it’s quite simple. We’re not talking about application development. We’re talking about steps. And so in this example, we’ve got like wait for an object and then the object’s name and then click on that object and then wait a second or so and then perform a check. Are these things equal? Or as expected, take a screenshot and move on to the next step. Now, one of the things that we were challenged with when we were looking at partnerships in this space specifically in mobile was who to work with. There’s a lot of companies out there who have different sort of capabilities. And it was important for us to partner with somebody that could easily work in the environment that we’re forced to work in which is the game engine. And there was a number of factors involved there. So things like the scalability, which is key, the performance of the devices. As I mentioned, being able to operate on those devices in a very responsive manner. And being able to see those games being played in a very responsive manner was key for us. So we looked at the entire market and we spent about a year and a half evaluating the different potential partnerships. And we came to the conclusion that kobiton was best suited to support those needs. One of the other things that I didn’t mention about gaming in general but specifically in mobile gaming is how secretive this market is. I joke with some of our customers that it’s easier for us to get into a department of defense contractor relationship than it is to get into a gaming studio. In some cases, they’re very secretive. They don’t want to leak anything out into the wild to say, you know, this is what we’re working on because there’s a chance that intellectual property might be stolen and then their edge, their differentiators can be lost. And this happens quite a bit. And so, it was also very important for us that whoever we partner with is able to work in that environment. So, not only with the cloud scalability but to be able to work sort of on Prem with customers in their environment, if that sort of makes sense. And I’ll come back to that in a second. But I wanted to very quickly demonstrate. How are we doing for time? By the way?
19:54 | Adam Creamer
We have about just under 10 minutes left.
19:57 | Shane Evans
That’s great. So, I’m going to… bring up a game very quickly. Let me know if you can see my screen?
20:12 | Adam Creamer
Yes, mobile tennis xxi.
20:15 | Shane Evans
Yeah, yeah. So this is a game like any other, but I just wanted to quickly demonstrate how this sort of testing works and I know the sharing doesn’t show all of the windows that pop up here. So I’ll just be very cognizant of the capabilities here. But, you know, you can imagine this is a unity game, but I’m working on a mobile sort of platform environment where I’m interacting very quickly with the scene. This is a tennis game, right? So you’ve got to be able to swipe up and respond to the actions or inputs very quickly. And so as I mentioned earlier, this is key for us. This is kind of our baseline for, and here, I’m trying to play a game while I’m talking for partnering with a mobile device, cloud vendor such as kobiton. So this is sort of the key initial capability that we’re after I won’t get into too much detail here because it gets a little bit technical. But the game driver in this environment in unity, you know, provide tools like being able to record inputs and generate our test steps from that. We’ve got, you know, unique tools like the object explorer, which allows us to traverse scenes and find objects and export those, you know, very quickly. So that as a tester, I’m able to find the things that I’m looking for. I can run my tests, I can create, modify my tests very easily. And so we’ve designed game driver with some key kind of capabilities in mind that allow us to sort of empower testers and developers to develop and maintain those tests very easily. And I know this is maybe a different environment for some. So I won’t spend a whole lot of time here but jumping… over to kobiton for a second, you know, talk a bit about the architecture. There. We’ve got back to the slides. There we go. And so kobiton gave us a couple of unique capabilities that, you know, I mentioned a bit earlier for us, it was about being able to scale number one. And so, you know, having that scalability was key. But also the performance that I mentioned. So kobiton is the only platform that we’re aware of that is able to provide the 30 frames per second and millisecond latency, the ability to interact with a game such as that tennis game in a very sort of dynamic manner without loss of fidelity or the latency that’s going to cause it to be near impossible to test in that environment. And so that was key, the flexibility of environments that I mentioned. So being able to test not only in the cloud, using commodity devices, but being able to stand up an environment on Prem, for many of our customers, testing in the cloud just isn’t an option. And so for them, it’s yes, we want these capabilities. We want to be able to manage testing, but we want to be able to do it in our own environment. And kobiton stood out in that. So, we’re again very pleased with the partnership that we’ve been able to form as a result of that… some of the other capabilities that I, you know, maybe touched on a lot of these here with like the consistency provided by game driver test automation, being able to do multi platform testing across different devices, form factors, versions, et cetera, ease of use without getting into too much detail. We believe game driver is the easiest solution on the market. And we’ve heard time and again from our customers that that’s true. And then things that kobiton provides, like the security, the speed and performance, and the scalability, which we couldn’t do on our own. So this really is an ideal relationship for us and our customers as well.
24:13 | Shane Evans
And I think the last thing I wanted to share before we open it up for questions was just how easy it is to get it to work together. But I’ll stop sharing briefly. This is actually something we did not get to show in our joint… webinar recently. So I was very excited to have a chance to be able to show it again. Now, this is the kobiton platform. I’m not sure how often this has been shared today, but this is a game driver test that was run recently in this environment. When I hit play here, you’ll see the same game that I was playing on my local machine playing in this environment. And you can see even through the sharing the high fidelity, the crispness of the replay, the, you know, the 30 frames per second, the ability to respond very quickly. And this was a collaboration effort on ours and kobiton’s part where we were able to define like, you know, here’s what’s missing. In order to run tests in this environment, we need you to be able to put the test as close to the physical device as possible. And the kobiton team was able to do that for us very quickly. And you can see the results here. We’re able to respond within, you know, sub second response to, you know, returning that ball back and forth for this test, which is just really key. We get all kinds of powerful information for our developers here. If something’s not working, we’ve got the device logs. We’ve got, you know, test report that’s generated from the execution itself. We’ve got the video recording and there’s more capabilities that are going to be added to this such as the session explorer that is available for other types of applications as well. But that’s near future stuff. And then of course, we’ve got the Jira integration, which is fantastic. A lot of our customers. You know, they say this is great. First question is how do I scale after we build in, some test automation capabilities. Second question is okay. How do I get this back to our developers? What information can I give them in order to solve problems after we’ve run these automated tests? And so, this really provides a sort of complete solution from building the test automation to executing whether it’s in the public or private cloud? And then getting that information back to developers so that they can quickly iterate and fix any issues that arise. And with that, I’ll stop for any questions Adam.
26:38 | Adam Creamer
Yeah. We, I had a private question in that I threw up in the Q and a pane. So I hope this is what, they meant. But you kind of mentioned the growing demographics of the gaming space. And I think the opening keynote we saw today even touching on accessibility a little bit like how do you guys approach or use inclusivity in your approach to mobile gaming testing?
26:58 | Shane Evans
Yeah. And, and ours, is sort of a generic solution but we’ve talked to a lot of companies that have applied that in a way that I think solves some of these problems. One of the challenges with accessibility, is the automation piece and how do you make sure that the different standards are being met? Automation allows for that very easily rather than having people go through, and check, are we meeting all of the standards that were mentioned? The W3 C standards, and others? There was another government one. I can’t remember the name. Offhand. It was like a number sequence… but you can very easily code those things into your automated tests so that each and every release you’re checking those things. It also helps with localization. If you think about, a game where you have a lot of different translation or different dialogues that need to be translated from one language to another, it would take, you know, just a linear amount of time for individuals to go through all of that. But you can put automation behind it very quickly and say, here’s, the base language here’s, the expected result for each of the translated languages. Now go through and play through those menus, those options, and check that each fits what it needs to and that it doesn’t bleed outside of the boxes that it looks correct? That sort of thing. This is very easily automated. And one of those things that, you know, we tell our customers all the time. Our goal is not to eliminate manual testing for gaming. In particular, you need manual testing. You need people to play it and say, is this fun? Is this the experience that we were looking to achieve? But we believe that automation really helps to unlock that. It helps to free up your resources from doing the very mundane repetitive things that aren’t adding a lot of value so that they can test those things that are more valuable. And so localization testing for accessibility. Those are those are some of the low hanging fruit that we recommend.
28:57 | Adam Creamer
Excellent. No, that’s awesome to hear. And I know we’re coming up on time. It looks like we got about 50 seconds left. But one more quick question for you. Is there any good documentation courses or videos on how to get started with you guys that you would recommend? I know you have the booth here with us today as well, but, yeah, feel free to plug away. Yeah.
29:15 | Shane Evans
Actually, we did a course recently on testgill com, which is pretty popular platform in the community. And so there’s a free course there. It’s me talking in this hour or this room for about two and a half hours, but it is free break, kind of breaks down, the different aspects, of testing with game driver, and it’s something that you can just revisit, you know, as needed until, you understand what needs to be done there. So, it’s, again, test testgill com, under the courses tab and you can find it there.
29:45 | Adam Creamer
Excellent. Yeah, we’ll make sure to link that in your booth as well. So thank you, Shane, very much appreciated.
29:52 | Shane Evans
Thanks, Adam.
