Mobile Testing for Gaming at Scale

Video Transcript

0:00:19.242,0:00:22.442

Today with us on this session, we have Shane Evans

0:00:22.442,0:00:25.232

from another partner of arts here at Kobiton.

0:00:25.232,0:00:28.292

Called game driver and Shane is gonna talk all about

0:00:28.292,0:00:31.842

Mobile gaming because we know mobile hard and mobile gaming is even harder.

0:00:31.842,0:00:35.282

And shane and his team know how to help with that so Shane.

0:00:35.282,0:00:42.592

Four yours. Thanks, Adam. And just make sure I got everything set up here.

0:00:47.412,0:00:50.662

And maybe let me know if you can see my screen

0:00:50.662,0:00:57.682

that would be fantastic. Yep. You are good to go.

0:00:57.682,0:01:01.272

Okay. Great. So hi everyone.

0:01:01.272,0:01:05.312

My name is Shane Evans. One of the c founders at Game driver

0:01:05.312,0:01:09.132

And if you joined us recently for Webinar

0:01:10.022,0:01:13.922

that was just game driver in C. This might be a bit of a refresher.

0:01:13.922,0:01:16.572

But we felt it was important enough

0:01:16.572,0:01:17.992

that we wanted to

0:01:17.992,0:01:22.542

come and share this sort of content again to get people talking about

0:01:22.542,0:01:24.302

they’re thinking about

0:01:24.302,0:01:28.742

mobile testing and gaming, which is our forte, Not not specifically

0:01:28.742,0:01:30.502

mobile but gaming in general,

0:01:30.502,0:01:35.182

And this partnership that we have with Ko ton is really kind of a first of its kind

0:01:35.182,0:01:38.862

and we’re really excited about working together with C and

0:01:38.862,0:01:42.172

And therefore, we wanted to to come out and and discuss that of it and again,

0:01:43.092,0:01:46.772

I’m gonna jump right in and talk a bit about mobile testing.

0:01:46.772,0:01:50.372

What in in specifically, mobile testing and gaming why it’s important.

0:01:50.372,0:01:54.022

A bit of bit of the market dynamics and what brought

0:01:54.022,0:01:59.822

us to working together with C ton, some of the the special sauce that’s

0:02:00.382,0:02:03.322

provided by that relationship. And then we’ll

0:02:03.322,0:02:07.032

close it off with a a bit of a demo and see how that goes.

0:02:07.622,0:02:13.402

Right. And, of course, Adam jump in at any time with any questions.

0:02:13.402,0:02:16.812

I do like to keep it interactive. So if I’m not looking at the chat if

0:02:16.812,0:02:19.222

something pops up that seems

0:02:19.222,0:02:20.872

relevant to what we’re talking about.

0:02:20.872,0:02:21.812

Please jump in.

0:02:22.932,0:02:27.872

So just a few few quick facts about the mobile gaming market.

0:02:28.632,0:02:30.782

Many people don’t realize, but but

0:02:30.782,0:02:33.772

Gaming as a whole is the largest entertainment

0:02:33.772,0:02:38.762

segment, in the world. It’s larger than digital music, it’s larger than movies.

0:02:38.762,0:02:43.662

It’s larger than all the professional sports combined. So it’s a incredibly large

0:02:43.662,0:02:46.892

market that’s pandemic proof. It’s recession proof,

0:02:47.922,0:02:52.642

and and really continues to exceed its expectations year over year.

0:02:52.642,0:02:55.112

Was very few exceptions.

0:02:55.852,0:02:58.062

The largest part of that market,

0:02:58.062,0:03:00.012

has become mobile gaming.

0:03:00.012,0:03:03.832

And that might be surprising to some of you gamers who are are watching.

0:03:03.832,0:03:05.262

I’m one of them.

0:03:05.262,0:03:10.232

And gamers historically have treated mobile gamers as not real gamers.

0:03:10.392,0:03:11.632

With the the

0:03:11.632,0:03:15.882

stats and them the revenues clearly show that mobile gaming is, in fact,

0:03:15.882,0:03:19.012

the most gamer sort of segment of the market.

0:03:19.012,0:03:23.982

That it represents fifty percent of of the hundred and ninety billion dollar gaming market overall.

0:03:24.122,0:03:25.992

And it is the fastest growing segment.

0:03:25.992,0:03:29.822

Estimates are anywhere between two hundred and

0:03:29.822,0:03:33.142

four hundred billion dollars for mobile gaining a loan and by

0:03:33.142,0:03:35.242

twenty twenty seven depending on who you you ask,

0:03:35.612,0:03:39.472

and the top three markets in this space are

0:03:39.472,0:03:41.092

China india in the Us.

0:03:41.092,0:03:44.152

But by revenue, it’s actually China, Us and Japan.

0:03:44.152,0:03:45.712

So it’s it’s quite significant.

0:03:46.102,0:03:47.522

And within that,

0:03:47.522,0:03:50.402

there’s millions of apps between the…

0:03:51.922,0:03:55.612

Ios and Android app stores

0:03:55.612,0:04:00.312

there’s literally millions of of apps to choose from. But but over fourteen hundred apps

0:04:00.312,0:04:04.662

in twenty twenty two generated over ten million dollars each.

0:04:04.662,0:04:10.552

Of the remainder two hundred and twenty four surpassed a hundred million dollars in revenue.

0:04:10.552,0:04:14.082

And ten of those went over a billion. So this is this is not just

0:04:14.082,0:04:16.622

fun for fun sake, but it is… In fact,

0:04:16.622,0:04:20.202

big, big business. And so there’s a lot lot on the line. There’s a lot of stake.

0:04:21.862,0:04:26.652

Mobile beginning is also the most inclusive of all entertainment mediums and not just gaming.

0:04:26.652,0:04:31.542

And includes representation from all age groups, genders, backgrounds, you name it.

0:04:31.542,0:04:39.072

According to New zoo, there was two billion players globally in twenty twenty two. And that number is expected to grow two three point five billion

0:04:39.072,0:04:42.262

twenty twenty five. So over fifty percent increase.

0:04:42.982,0:04:47.302

Mobile games are equally almost equally popular among men and women.

0:04:47.652,0:04:50.502

Specifically fifty three percent ish.

0:04:50.502,0:04:53.312

The overall market is men.

0:04:54.152,0:04:58.672

Forty six percent are women, and people of all ages play mobile games.

0:04:59.892,0:05:03.682

Pretty standard bell curve with the the average gamer

0:05:03.682,0:05:07.102

sitting around thirty six years old, but more than half of

0:05:07.312,0:05:08.792

players make a… Or

0:05:08.792,0:05:16.752

players in the space are over the age of thirty five. And finally I wanted to point out how

0:05:16.752,0:05:19.032

Fierce the competition is in this space.

0:05:19.032,0:05:23.462

It’s incredibly fierce. As of twenty twenty three, there’s over seven hundred thousand games low

0:05:23.462,0:05:24.982

in the app stores with

0:05:24.982,0:05:26.872

about five hundred thousand on Google,

0:05:26.872,0:05:31.512

play and two hundred thousand on on the app app. Sorry. I know it said millions earlier in this

0:05:31.512,0:05:34.522

this sort of factors out a lot of their clones and and

0:05:34.522,0:05:37.842

non monetized games, we call hyper casual,

0:05:37.842,0:05:41.212

in some cases, So Quite significant

0:05:41.212,0:05:44.542

market size, the the saturation is significant.

0:05:44.542,0:05:47.312

And some… So it’s it’s quite important.

0:05:47.312,0:05:49.722

That you stand out in this space.

0:05:49.722,0:05:55.042

That you stand us separate from from the rest of the market in terms

0:05:55.042,0:05:58.682

a feature in gameplay but also in terms of quality, which is of course, why we’re here today.

0:05:58.682,0:06:01.882

There’s several options available in any category.

0:06:01.882,0:06:06.852

They have simulated entertainment value players will always choose the game with lease amount of issues, lease, amount of bugs.

0:06:06.852,0:06:08.472

And gamers are very, very picky. And

0:06:08.472,0:06:13.442

for those of us who who play games. If something’s not working, I mean, there’s plenty of options. But

0:06:13.442,0:06:16.772

I wanna spend my time on something, it’s gotta be something of high quality.

0:06:16.772,0:06:18.022

Simple as that

0:06:18.492,0:06:23.462

Some of the challenges of developing modern games.

0:06:23.462,0:06:25.262

In this space. So

0:06:25.262,0:06:26.412

There’s quite a lot,

0:06:26.412,0:06:27.912

but just to touch on a few

0:06:28.242,0:06:30.182

the rising user expectations, the

0:06:30.182,0:06:35.292

as mobile device hardware improves, there’s this expectation

0:06:35.292,0:06:39.352

for games to boast near console or even Pc light graphics. Which

0:06:39.352,0:06:41.132

talking high resolution textures,

0:06:41.132,0:06:43.342

realistic lighting, gameplay,

0:06:43.342,0:06:47.612

the detailed character models are becoming increasingly the norm. And so the

0:06:47.612,0:06:50.762

a couple years ago, if you were to compare mobile games

0:06:50.762,0:06:56.402

bikes, you know, Pla Bird or or Cross road or something like that to console games, there was no comparison

0:06:56.402,0:06:58.932

But these days, the the

0:06:58.932,0:07:01.672

the lines are starting to blur between what

0:07:01.672,0:07:06.592

can do with the phone in our pocket and what we can do is the computer over the console that we put plug into our Tv.

0:07:06.592,0:07:10.212

We’ve got more complex game mechanics.

0:07:10.212,0:07:14.942

Diverse gameplay elements requiring multifaceted user testing such as multiplayer titles.

0:07:14.942,0:07:18.282

Fortnite, Pub g or some of the largest titles in the space.

0:07:18.282,0:07:21.382

And Air and Vr applications like Pokemon go,

0:07:21.382,0:07:26.122

that make use of the devices, Lidar and camera and Ar capabilities.

0:07:27.362,0:07:31.492

Features like leader boards, Friends systems, chat. So so no longer are we

0:07:31.492,0:07:35.642

down to testing a or playing a game by yourself on our mobile device

0:07:35.642,0:07:37.452

it’s become a much more social.

0:07:37.452,0:07:42.992

And connected experience. And lastly, in this space, there is more

0:07:42.992,0:07:44.992

regular updates, than any other

0:07:44.992,0:07:47.892

In fact, console and Pc gaming

0:07:47.892,0:07:50.812

unlike any other medium

0:07:50.812,0:07:52.932

see constant

0:07:52.932,0:07:57.152

sort of updates and content. So things like Fortnite that I mentioned

0:07:57.152,0:08:01.732

we’ll receive a a seasonal updates as well as bug fixes and that sort of thing. Do these games

0:08:01.732,0:08:03.842

tend to last for many, many years.

0:08:04.712,0:08:07.152

And so gamers are are

0:08:07.152,0:08:11.442

expecting of regular content updates. But on mobile, it’s actually much more

0:08:11.442,0:08:14.332

comments of seasonal can mean,

0:08:14.902,0:08:19.122

you know, two to three months in some cases. But on mobile devices, the the games tend to be up

0:08:19.122,0:08:22.292

did much more frequently. Sometimes in months or even weeks, did

0:08:22.292,0:08:25.802

in this sort of constant iteration and improvement

0:08:25.802,0:08:28.702

So testing for for

0:08:28.702,0:08:32.052

bugs and compatibility for every single release, every update

0:08:32.052,0:08:33.922

is super critical in this space.

0:08:37.092,0:08:40.842

We’re also dealing with multiple monetization models. So

0:08:41.612,0:08:45.742

you’ve got everything from direct purchases, which are becoming

0:08:45.742,0:08:50.362

increasingly less popular, over time is as

0:08:50.362,0:08:53.912

you know, the past you could buy a game, and it would be yours

0:08:53.912,0:08:55.392

in perpetuity.

0:08:55.392,0:08:58.022

In fact, some of those games have been pulled from the market.

0:08:59.112,0:09:04.082

Game developers are finding that those are not sustainable models. If you reach a certain

0:09:04.082,0:09:09.052

number of users and and you’re not increasing, then then revenue tends to dry up. So they’ve switched to

0:09:09.052,0:09:13.462

models like of paid advertisements where

0:09:13.462,0:09:15.812

know you place through a level and you get an ad, you

0:09:15.812,0:09:19.382

if you wanna get a power up or a little extra time, you get an ad

0:09:19.382,0:09:22.642

Things like app purchases for

0:09:22.642,0:09:27.132

things like items and and cosmetics are are are very popular. And virtual currencies,

0:09:27.132,0:09:30.212

where if you’re you’re looking to get a a specific thing,

0:09:30.212,0:09:34.402

or capability in that game. You gotta buy some coins and those coins

0:09:34.402,0:09:35.492

translate into

0:09:35.492,0:09:39.252

you know, something that you can exchange for extra power ups or extra lives, etcetera.

0:09:39.702,0:09:42.722

In most cases, these are actually not standalone.

0:09:42.722,0:09:47.632

And so you’ve got multiple sort of mix modes of monetization

0:09:47.632,0:09:49.982

to test when you’re your

0:09:49.982,0:09:55.882

creating these games, And it’s critical that these work. There’s been several examples recently. I don’t wanna

0:09:55.882,0:09:58.712

I mention any names and and shame anyone, but

0:09:58.712,0:10:01.372

we talked to customers who suddenly

0:10:01.372,0:10:03.672

from one release to the next, their

0:10:03.672,0:10:08.612

monetization platform wasn’t working. And they literally lost millions of dollars because of that

0:10:08.612,0:10:11.632

And so a a small piece of functionality,

0:10:11.632,0:10:14.532

in in any other environment. In this case can mean

0:10:14.532,0:10:16.312

sort of the critical loss of revenue.

0:10:18.712,0:10:22.422

And lastly, one that’s near and dear to the the hearts of

0:10:22.422,0:10:26.592

of many here is the mobile device fragmentation. So

0:10:27.432,0:10:30.482

This is a tale is all this time in terms of mobile test.

0:10:30.482,0:10:35.392

I’ve been in in testing and test automation for for well over a decade now approaching two to

0:10:36.362,0:10:40.152

And as long as as I’ve been in this space and mobile been a thing,

0:10:40.152,0:10:42.782

the challenge has been to

0:10:42.782,0:10:46.632

test on all of the major devices, of course, but also the

0:10:46.632,0:10:50.462

the the devices in different regions, the the different form factors,

0:10:50.462,0:10:52.272

the different capabilities,

0:10:52.272,0:10:55.422

it’s not like in in some other their spaces where

0:10:55.422,0:10:58.442

you know, you’ve got one target console to work with or

0:10:58.442,0:11:02.352

Pcs or sort of this general thing that you you need to to work from.

0:11:02.352,0:11:07.292

Mobile devices are are unique, but they also tend to last many, many years. So

0:11:07.292,0:11:12.262

you’ve got devices that are several years old still in the market and you’ve got the latest

0:11:12.262,0:11:14.982

greatest that you need to keep up with. So so this

0:11:14.982,0:11:16.952

sort of matrix of of testing

0:11:16.952,0:11:21.712

just increases exponentially when you consider the the variety of mobile devices.

0:11:21.712,0:11:23.442

Better are out there and need to be tested.

0:11:23.442,0:11:28.262

Specifically in gaming where things like performance, the device

0:11:28.262,0:11:30.152

capabilities that we talked about earlier.

0:11:30.152,0:11:32.952

All of these things change very quickly in in mobile.

0:11:32.952,0:11:35.012

So worked

0:11:35.012,0:11:35.932

on your phone on

0:11:35.932,0:11:37.402

two or three years ago, you know

0:11:37.402,0:11:40.842

might not work with the latest updates or

0:11:40.842,0:11:42.502

if you’ve got a brand new game,

0:11:42.502,0:11:44.162

and you’ve got an older phone,

0:11:44.162,0:11:45.122

you might be at lot. You

0:11:45.122,0:11:49.432

So mobile gaming is very difficult.

0:11:49.432,0:11:54.072

As a market. It’s difficult as a for for developers and it’s difficult for for

0:11:54.072,0:11:59.862

testers, it’s really in this case where we’re in need of a new approach for testing.

0:11:59.862,0:12:02.362

We’ve got mobile games that are are unique. We’ve got we’ve

0:12:03.212,0:12:08.242

different form factors, different capabilities in each of the devices.

0:12:08.242,0:12:12.792

But the games themselves are much more difficult to test. So if you’re you’re comparing to, like, business app

0:12:12.792,0:12:17.002

locations, some of the the other speakers have mentioned.

0:12:17.002,0:12:20.902

You know, there’s much less sensitivity to the performance of the device.

0:12:20.902,0:12:24.822

The refresh rates or the the frames per second

0:12:24.822,0:12:25.492

thank your

0:12:25.912,0:12:29.112

being able to view or interact with the device itself.

0:12:29.112,0:12:32.252

These can all affect gameplay significantly.

0:12:33.092,0:12:35.392

It’s often an always on experience.

0:12:35.392,0:12:40.212

With the monetization models that I mentioned, these these games are ad supported.

0:12:40.212,0:12:43.232

They’ve got online stores that are exchanging

0:12:43.232,0:12:45.602

money, they’re multi multiplayer, and they’re using

0:12:45.602,0:12:49.932

real world or Mean take augmented reality capabilities,

0:12:49.932,0:12:54.912

which is a unique capability to to mobile and mobile gaming specifically.

0:12:55.712,0:13:00.082

Mobile games are very responsive. It’s very dynamic. It’s not like a form based

0:13:00.082,0:13:01.962

application where you’ve got

0:13:16.474,0:13:18.624

It becomes very difficult

0:13:18.624,0:13:22.454

even in manual testing, how do you how do you achieve that sort of response of this?

0:13:22.454,0:13:26.854

And repeat ability is is key there. So being able to do it once it’s great

0:13:26.854,0:13:30.704

you need to be able to do this for every device, every form factor, each and every release.

0:13:30.704,0:13:31.564

Etcetera.

0:13:31.564,0:13:36.104

Hey, Shane. About to jump in here real quick. Go ahead, question. Because I know

0:13:36.104,0:13:39.844

you’re talking about, like, device fragmentation and the performance of those

0:13:39.844,0:13:42.394

devices in the world? Like, how do you guys

0:13:42.674,0:13:47.304

approach, like the multiplayer gaming in that instance. Like, if you’re on

0:13:47.304,0:13:52.114

and iphone fifteen, I’m on a Pixel three, like, how do you ensure those experiences are similar.

0:13:52.114,0:13:56.454

Yeah. So game driver which I’ll I’ll jump into

0:13:56.454,0:13:58.984

briefly here. I’ll just throw up the slide where we’re talking.

0:13:58.984,0:14:03.564

One of our our sort of core capabilities is being able to interact with

0:14:03.564,0:14:05.644

the game as if you are a real user.

0:14:05.644,0:14:10.494

And so that means providing inputs like… A real user would if it’s touched

0:14:10.494,0:14:12.254

tap, you know, your your you

0:14:12.594,0:14:16.174

moving the device around or you’ve got some sort of peripheral plug in,

0:14:16.174,0:14:17.724

some games do support.

0:14:17.724,0:14:22.454

Our goal is to to test as if the user was testing, but in a more automated

0:14:22.454,0:14:26.234

consistent matter. And and that includes mobile or sorry, multiplayer testing,

0:14:26.574,0:14:29.284

So in a scenario where and we have

0:14:29.284,0:14:30.934

many customers doing this.

0:14:30.934,0:14:35.514

Where we’re interacting with different devices and different platforms,

0:14:35.514,0:14:38.724

We can run the same test across those different platforms.

0:14:38.724,0:14:43.664

Without modification, it’s really just we’re connecting to an Android device. We’re connecting to an ios device.

0:14:43.664,0:14:45.844

For connecting to a Pc or console for

0:14:45.844,0:14:48.244

and running the same test. In parallel,

0:14:48.244,0:14:52.174

that we’re able to interact between these. And the automated approach allows you to

0:14:52.174,0:14:54.894

to even coordinate between the different…

0:14:54.894,0:14:57.184

You know, users that are are

0:14:57.184,0:14:59.484

being executed during those tests.

0:14:59.484,0:15:03.374

Great question. Thank you.

0:15:03.374,0:15:05.244

So really, our

0:15:05.244,0:15:08.344

goal since twenty eighteen has been to introduce this sort of

0:15:08.954,0:15:12.714

scalable, reusable approach to automated testing

0:15:12.714,0:15:14.634

and it’s the first of its kind.

0:15:14.634,0:15:17.374

And that we’re able to support multiple game engines.

0:15:17.374,0:15:19.584

Similar to s millennium

0:15:19.584,0:15:22.274

for those of you familiar with s and Api,

0:15:22.274,0:15:26.154

in the sort of standardized approach to testing

0:15:26.154,0:15:29.834

browsers and and mobile devices. We took the same approach

0:15:29.834,0:15:31.824

with game engines. So

0:15:31.824,0:15:34.964

developers familiar with Unity can

0:15:34.964,0:15:36.114

you know, quickly, you

0:15:36.114,0:15:40.244

change gears from from Unity to and unreal and and develop and and work with

0:15:40.244,0:15:42.574

game driver tests in much the same way.

0:15:42.574,0:15:46.604

And we recently added these

0:15:46.604,0:15:50.154

these other engines you’d are gonna be being our first, but

0:15:50.154,0:15:56.424

unreal and Gi, we’re added earlier in twenty twenty two. We just had a a major release like yesterday.

0:15:56.424,0:16:00.144

And so we’re very proud to support all of the the latest versions of

0:16:00.144,0:16:02.654

unreal unity is part of that.

0:16:03.114,0:16:04.954

And for those of you unfamiliar,

0:16:04.954,0:16:09.384

unity and a real combined make up about eighty percent of the the gaming market.

0:16:09.384,0:16:13.454

There’s a lot of engines out there there’s a lot of sort of bespoke or niche

0:16:13.454,0:16:17.654

engines, one, which we support called God which is an open source.

0:16:17.654,0:16:23.854

Community loved, engine, and game engines are where we create these experiences. So unlike,

0:16:23.854,0:16:26.874

you know web websites which can be crafted in

0:16:26.874,0:16:28.974

multiple different languages and and

0:16:28.974,0:16:32.774

and frameworks, game engine is how the games are made.

0:16:32.774,0:16:37.384

So we’ve based our solution on sort of open standards, which allows us

0:16:37.384,0:16:38.804

to to test

0:16:38.804,0:16:42.424

very rapidly and in those types of environments that our customers are

0:16:42.424,0:16:45.444

expecting, using tools like jenkins and github and

0:16:45.444,0:16:48.254

and end unit for execution and that sort of thing.

0:16:48.254,0:16:51.904

Means that we’re easily portable to all the different platforms as well.

0:16:51.904,0:16:55.404

You can see some listed here. Nintendo Switch, xbox, etcetera,

0:16:55.404,0:16:57.064

But today, we’re focusing on mobile.

0:16:59.514,0:17:02.234

Very quickly, how Game beverage works.

0:17:03.314,0:17:06.334

We drop a small tool at

0:17:06.334,0:17:10.314

an agent in the the game itself, one of our our sort of core

0:17:10.314,0:17:14.144

design philosophies was was making no changes to the game itself.

0:17:14.144,0:17:18.744

Order to facilitate automated testing. So you drop a small agent into the game,

0:17:18.744,0:17:21.014

then you build it for whatever platform that you’re

0:17:21.014,0:17:23.914

you’re looking just for. Some of those are are shown on the right. You’re

0:17:23.914,0:17:28.884

And this allows us to control the game, but also to inter

0:17:28.884,0:17:32.824

and manipulate objects at run runtime. We’re able to control things.

0:17:32.824,0:17:36.224

Within the games that we can set up these sort of complex scenarios.

0:17:36.224,0:17:40.564

And execute tests as as a user would, but also in a way that’s more efficient.

0:17:40.564,0:17:41.384

So we’re not just so

0:17:41.384,0:17:43.184

know, recording and replaying

0:17:43.184,0:17:46.934

every touch and tap on a device, which we can also do

0:17:46.934,0:17:49.084

we’re able to set up a scene

0:17:49.084,0:17:53.004

you know, run a test, move to the next scene, run another test. So very

0:17:53.004,0:17:57.314

consistent approach to to doing that. And as you can see with some of the the code that’s

0:17:57.314,0:18:00.334

shown on the bottom left there, It’s quite simple.

0:18:00.334,0:18:05.174

We’re not talking about application development. We’re talking about, you know, steps. And so

0:18:05.174,0:18:09.244

in this example, we’ve got, like, wait for an object and then the the object’s name.

0:18:09.244,0:18:10.854

And then click on that object.

0:18:10.854,0:18:14.414

And then wait a second or so And then perform a check. Are these things

0:18:14.414,0:18:17.934

equal as expected, take a screenshot, and move on to the next step.

0:18:17.934,0:18:20.214

Now

0:18:20.214,0:18:22.974

one of the things that we were challenged with

0:18:22.974,0:18:27.634

when we were looking at partnerships in this space, specifically mobile,

0:18:27.634,0:18:31.104

was who to work with. There’s a lot of companies out with there

0:18:31.104,0:18:34.254

who have different sort of capabilities and and

0:18:34.944,0:18:38.144

it was important for us to to partner with somebody that

0:18:38.144,0:18:42.694

could easily work in the environment that

0:18:42.694,0:18:45.914

that we’re forced to working, which is the the game engine. So

0:18:45.914,0:18:49.444

there was a number of factors involved there. So things like the scalability

0:18:49.444,0:18:52.404

which is kate the performance,

0:18:52.404,0:18:55.904

of the devices, as I mentioned, being able to to operate

0:18:55.904,0:19:00.414

on those devices in a very responsive manner and being able to see

0:19:00.414,0:19:04.954

those those games being played in a very responsive matter was was key for us. So

0:19:04.954,0:19:09.024

we looked at the entire market and we spent about a year and a half.

0:19:09.024,0:19:12.944

Evaluating the different potential partnerships, and we

0:19:12.944,0:19:16.564

came to the conclusion that Kobe tong was best suited to support those needs.

0:19:16.564,0:19:21.534

One of the other things that I I didn’t mention about gaming in general, but but specifically,

0:19:21.694,0:19:24.694

in in mobile gaming is how secretive this market is.

0:19:24.694,0:19:26.094

A joke with

0:19:26.094,0:19:31.554

some of our customers that it’s easier for us to get into a department of defense

0:19:31.554,0:19:36.014

contractor relationship, than it is to get into a gaming studio in some cases.

0:19:36.014,0:19:39.834

They’re very secretive. They don’t wanna leak anything out into the wild to say,

0:19:39.834,0:19:42.914

you know, this is what we’re working on because it… There’s a chance that that

0:19:42.914,0:19:45.604

intellectual property might be stolen and then they’re they’re

0:19:45.604,0:19:49.684

their edge their their cheater can be lost. And this has quite a bit.

0:19:50.104,0:19:50.924

And so and

0:19:50.924,0:19:53.414

It was also very important for us that

0:19:53.414,0:19:56.404

whoever we partner with is able to work in that environment.

0:19:56.404,0:19:59.494

Not only with the cloud scalability, but to be able to work

0:19:59.494,0:20:02.644

sort of on prem with customers in their environment.

0:20:02.644,0:20:05.604

If that totally makes sense.

0:20:05.604,0:20:08.684

You know, I’ll come back to that in a second, but I wanted to

0:20:08.684,0:20:11.964

to very quickly demonstrate how we doing for time by the way?

0:20:11.964,0:20:16.014

We have about just under ten minutes left. Let’s

0:20:16.014,0:20:19.854

Great. So I’m going to

0:20:21.904,0:20:23.584

bring up a game very quickly,

0:20:25.194,0:20:28.414

Let me know if you can see my screen

0:20:29.804,0:20:37.284

Yes. Mobile tennis twenty one. Yeah. Yeah. So this is this is a game like any other

0:20:37.284,0:20:41.444

I just wanted to quickly demonstrate how this sort of testing works

0:20:41.444,0:20:46.264

And then know the sharing doesn’t show all of the the windows that pop up here. So I’ll just

0:20:46.264,0:20:50.124

being very cognizant of of the capabilities here. But

0:20:50.124,0:20:54.524

you know, you can imagine this is a unity game, but I’m working in a on a mobile

0:20:54.524,0:20:59.004

sort of platform environment where I’m I’m interacting very quickly

0:20:59.004,0:21:00.174

with the scene?

0:21:00.174,0:21:04.334

This is a tennis game. Right? So you gotta be able to swipe up and and respond to

0:21:04.334,0:21:07.714

to the actions or inputs very quickly.

0:21:07.714,0:21:11.834

And so as I mentioned earlier, this was key for us. This is kind of our baseline for

0:21:11.834,0:21:15.274

And here I’m trying to play a game while I’m talking. For for

0:21:15.274,0:21:18.384

partnering with a mobile device,

0:21:18.384,0:21:21.544

cloud vendor, such as C. So

0:21:21.544,0:21:26.364

this is sort of the the key initial capability that we’re after

0:21:26.564,0:21:30.184

I won’t get into too much detail her because it gets a little bit tech…

0:21:30.184,0:21:33.874

But the game driver in this environment in in Unity.

0:21:33.874,0:21:37.384

Provide tools like being able to record your inputs,

0:21:37.384,0:21:39.454

and generate our test steps from that

0:21:39.454,0:21:44.304

We’ve got, you know, unique tools like the the object Explorer, which allows us to

0:21:44.304,0:21:47.834

traverse scenes and find objects and and export those

0:21:47.834,0:21:50.244

know, very quickly so that as a tester

0:21:50.244,0:21:53.024

I’m able to to find the things that I’m looking for.

0:21:53.564,0:21:57.934

I can run my my tests or I can create, modify my tests very easily.

0:21:57.934,0:22:00.984

And so we’ve we’ve designed game driver with with some

0:22:00.984,0:22:04.624

some key kinda of capabilities in mind that allows allow us to

0:22:05.334,0:22:09.844

sort of empower testers and developers to to

0:22:09.844,0:22:12.494

develop and maintain those tests very easily,

0:22:12.494,0:22:17.324

I know this is maybe a different environment for some. So I won’t spend a whole lot of time here, but

0:22:19.014,0:22:22.654

jumping over to Kobe tong for a second.

0:22:22.654,0:22:25.414

Talk a bit about the architecture there.

0:22:26.304,0:22:28.194

We’ve got

0:22:28.194,0:22:30.544

Back to the slides,

0:22:32.804,0:22:36.034

There we go. And so C gave us

0:22:36.034,0:22:38.954

a couple of unique capabilities that

0:22:39.994,0:22:42.134

I mentioned a bit earlier, you

0:22:42.134,0:22:46.764

For us, it was about being able to scale number one. And so

0:22:46.764,0:22:51.674

you know, having that scalability was was key.

0:22:51.674,0:22:56.104

But also the performance that I mentioned. So Cope is the only platform that

0:22:56.104,0:22:59.634

that we’re aware of that that is able to provide the thirty french

0:22:59.634,0:23:03.794

second and millisecond latency, the ability to to interact with a game sets

0:23:03.794,0:23:05.514

just that tennis game in a very

0:23:05.514,0:23:07.704

sort of dynamic manner without

0:23:07.704,0:23:11.334

loss of fidelity or or the latency that’s gonna cause it to be

0:23:11.334,0:23:14.844

they’re impossible to to test in that environment.

0:23:14.844,0:23:17.104

And so that was key.

0:23:17.104,0:23:22.074

The the flexibility of environments that I mentioned, so being able to test not only

0:23:22.074,0:23:25.184

in the cloud using commodity devices, but being able to

0:23:25.184,0:23:27.314

to stand up an environment on prem,

0:23:27.314,0:23:28.954

for many of our customers,

0:23:28.954,0:23:32.874

testing in the cloud, just what isn’t an option. And so for them, it’s it’s yes. We want this

0:23:32.874,0:23:37.044

these capabilities, we wanna be able to manage testing, but we wanna be able to do it in our own environment

0:23:38.494,0:23:42.454

ten C stood out in that. So we’re again, very, very

0:23:42.454,0:23:46.464

pleased with the the partnership that we’ve been able to form as a result of that.

0:23:48.544,0:23:56.594

Some of the other capabilities that I I know, maybe touched on a lot of these here, Would like the consistency provided by

0:23:56.594,0:24:00.774

game driver test automation, being able to do multi platform test

0:24:00.774,0:24:04.574

across different devices, form factors, versions, etcetera, ease of use,

0:24:04.574,0:24:08.974

without getting it into us detail, we we believe game drivers is the easiest solution.

0:24:08.974,0:24:10.494

On the market and we’ve

0:24:10.494,0:24:13.664

heard time and again from our customers. That’s that’s true.

0:24:13.664,0:24:18.104

And then things that C provides like the security

0:24:18.104,0:24:22.834

the speed and performance and the scalability, which we couldn’t do on our own. This really is an ideal

0:24:22.834,0:24:25.454

relationship for us and our customers as well.

0:24:30.304,0:24:32.534

But I think

0:24:32.534,0:24:36.364

The last thing I wanted to share before we we open it up for questions was

0:24:36.364,0:24:39.774

just how easy it is to get it to work together, but also I’ll

0:24:39.774,0:24:44.964

stop sharing briefly. This is actually something we we

0:24:44.964,0:24:46.904

did not get to show and our… Did

0:24:49.044,0:24:53.434

our joint webinar recently, so I was very excited to have a chance to be able to show it again

0:24:53.434,0:24:58.304

No. This is the Covid platform. I’m not sure how how often this has been shared today.

0:24:59.034,0:25:03.944

But this is a game driver test that was was run recently in this environment

0:25:03.944,0:25:07.484

And I hit play here, you’ll see the same game that I was playing

0:25:07.484,0:25:09.684

on my local machine,

0:25:09.684,0:25:13.184

playing in in this environment, and you can see

0:25:13.184,0:25:16.204

even through the the sharing the the high fidelity, the

0:25:16.204,0:25:18.374

miss of the the replay of

0:25:18.374,0:25:21.314

you know, the thirty frames per second, the the ability to

0:25:21.314,0:25:22.924

to respond very quickly

0:25:22.924,0:25:27.824

And this was a a collaboration effort on on ours and and C times part.

0:25:27.824,0:25:30.684

We were able to define, like,

0:25:30.684,0:25:32.214

yeah Here here’s what’s missing. Yeah

0:25:32.214,0:25:33.754

in order to to run

0:25:33.754,0:25:35.314

tests in this environment,

0:25:35.314,0:25:39.694

need you to be able to to put the test as close to the physical device as possible.

0:25:39.694,0:25:43.034

The C tan team was able to to do that for us very quickly.

0:25:43.034,0:25:47.744

You can see the results here. We’re we’re able to respond within, you know, sub second response

0:25:47.744,0:25:52.914

to you know, returning that ball back and forth for this test, which is just

0:26:08.959,0:26:12.729

To this such as the session explorer that is available for for other

0:26:12.729,0:26:15.059

types of applications as well. But… That’s

0:26:15.059,0:26:19.479

near future stuff. And then, of course, we’ve got the Jira integration, which

0:26:19.479,0:26:22.619

fantastic a lot of our customers. You know, they say this is great.

0:26:22.619,0:26:25.519

First question is, how do I scale after we

0:26:25.519,0:26:28.089

build in some some test automation capabilities.

0:26:28.089,0:26:29.819

Second question is okay, How do I get this

0:26:29.819,0:26:34.009

back to our developers. What information can I give them in order to solve problems

0:26:34.009,0:26:42.899

after we’ve run these automated tests? And so this this really provides a sort of complete solution from from building the test automation, to executing

0:26:42.899,0:26:45.079

whether it’s in the public or private cloud,

0:26:45.079,0:26:49.699

and then getting that information back to developers so that they can quickly iterate

0:26:49.699,0:26:51.539

and fix any issues that arise.

0:26:52.169,0:26:55.169

And with that, I’ll stop for any questions, Adam?

0:26:55.169,0:26:59.419

Yeah. We had a private question that I threw up in the Q and A paints. I hope

0:26:59.419,0:27:01.959

this is what they they meant. But

0:27:01.959,0:27:06.929

you kinda mentioned the growing demographics of the ga space and I think the opening keynote we

0:27:06.929,0:27:10.879

saw today, even touching on accessibility a little bit. Like, how do you guys approach

0:27:10.879,0:27:15.469

or use inc exclusivity in your approach to mobile gaming testing.

0:27:15.469,0:27:19.509

Yeah. And and ours is is sort of a generic solution, but we’ve talked

0:27:19.509,0:27:23.999

a lot of companies that have applied that in a way that I think Solves

0:27:23.999,0:27:26.719

some of these problems.

0:27:26.719,0:27:28.739

One of the challenges with

0:27:28.739,0:27:33.709

accessibility is is the automation piece. And how do you

0:27:33.909,0:27:37.229

make sure that that the different standards are being met.

0:27:37.229,0:27:39.579

Automation allows for that very easily.

0:27:39.579,0:27:42.809

Rather than than having people go through and and check

0:27:42.809,0:27:49.549

Are we meeting all of the standards that were mentioned the W3C standards and and and others. There’s another government

0:27:49.549,0:27:50.039

wanna it

0:27:50.039,0:27:53.549

can’t remember the name of offhand. It it’s like, a number

0:27:53.549,0:27:57.979

sequence. But you can very easily code those things

0:27:57.979,0:28:02.079

into your automated test, so that each and every release, you’re you’re checking those things.

0:28:02.079,0:28:06.089

Also helps with localization,

0:28:06.089,0:28:11.059

if you think about a a game where you have a lot of different

0:28:11.289,0:28:16.199

translation or different dialogues that need to be translated from from one language to another.

0:28:16.199,0:28:17.609

It would take

0:28:18.219,0:28:21.989

just a linear amount of time for individuals to go through all of that.

0:28:21.989,0:28:25.489

But you can put automation behind it very quickly and say here’s here’s

0:28:25.489,0:28:29.619

the base language, here’s the expected result for each of the translated languages.

0:28:29.619,0:28:33.029

Now go through and play through those menus, those options,

0:28:33.029,0:28:34.799

and check that each

0:28:34.799,0:28:39.769

fits what it needs to and that it doesn’t bleed outside of the boxes that it looks correct. That sort of thing.

0:28:39.769,0:28:41.509

This is very easily automated.

0:28:41.509,0:28:43.339

And one of those things that

0:28:43.339,0:28:45.139

You, we tell our customers all the time.

0:28:45.139,0:28:49.599

Our goal is not to eliminate manual testing. For gaming in particular, you need manual tests

0:28:49.599,0:28:51.369

you need people to play it and say,

0:28:51.369,0:28:54.809

is this fun Is this the experience that we were looking to achieve.

0:28:54.809,0:28:59.379

But we believe that automation really helps to unlock that.

0:28:59.379,0:29:03.119

It helps to free up your resources from doing the very mundane, repetitive,

0:29:03.119,0:29:05.259

things that aren’t adding a lot of value,

0:29:05.259,0:29:08.819

so that they can test those things that are more valuable. And so so

0:29:08.819,0:29:13.789

localization, testing for accessibility. Those are those are some of the low hanging fruit that we recommend

0:29:13.789,0:29:16.599

Excellent. No. That’s awesome to hear. And I know we’re

0:29:16.599,0:29:21.539

coming up on time. It looks like we got about fifty seconds left, but one more quick question for you,

0:29:21.659,0:29:25.459

Is there any good documentation, courses or videos

0:29:25.459,0:29:30.429

on how to get started with you guys that you would recommend. I know you have the booth here with us today as well, but

0:29:30.429,0:29:32.179

Yeah. Feel free to plug away.

0:29:32.179,0:29:36.669

Yeah. Actually, we did a a course recently on test skill dot com.

0:29:36.669,0:29:39.689

Which is pretty popular platform in the community.

0:29:39.689,0:29:43.189

And so there’s a free course there. It’s me talking in this hour or this

0:29:43.189,0:29:47.469

room for about two and a half hours. But it is free. Right it kind of breaks down the

0:29:47.469,0:29:50.809

different aspects of testing with game driver and it’s

0:29:50.809,0:29:53.629

it’s something that you can just revisit

0:29:53.629,0:29:55.389

know, as needed

0:29:55.389,0:29:58.709

until you you understand what needs to be done there. So it’s

0:29:58.709,0:30:02.559

Again, test test skill dot com under the courses tab and you can find it there.

0:30:02.559,0:30:06.659

Excellent. Yeah. We’ll make sure to link that in your booth as well.

0:30:06.659,0:30:09.289

So thank you, Shane very much. Much appreciate it.

0:30:09.799,0:30:16.810

Excellent. Have a good one. Have a good one.