Accelerating Testing for Faster, More Reliable Releases

Video Transcript

0:00:09.257,0:00:10.637

Discussion with

0:00:10.637,0:00:12.657

some of our partners here.

0:00:12.657,0:00:15.827

At Ko who are sponsoring the event as well.

0:00:15.827,0:00:20.607

So our channel lead Sydney, whoever will be leading the discussion.

0:00:20.607,0:00:25.457

Along with, some of the partners from App and apt,

0:00:25.457,0:00:29.257

So with that, I am gonna bring them to the state.

0:00:29.257,0:00:32.587

Exact. Sydney, floor Yours.

0:00:32.587,0:00:35.357

Hey. Good morning, everybody.

0:00:35.357,0:00:38.677

Thank you so much for joining us. So welcome to our…

0:00:38.677,0:00:42.897

Final discussion it. We are

0:00:42.897,0:00:46.147

gonna be talking about accelerating mobile app delivery and test

0:00:46.147,0:00:50.067

No. Let me introduce you to our Panel today.

0:00:50.067,0:00:53.447

So we have it Johnson. You as the c founder and Ceo at

0:00:53.447,0:00:59.037

five We have Karen, who is the Sv of product, mobile Devops and

0:00:59.037,0:01:01.317

security solutions from Aft,

0:01:01.317,0:01:04.107

And we have our very own Lindsay. He was our Head of customer sits.

0:01:04.447,0:01:10.537

You’re at C time. So Johnson, can you give us a little bit of a brief overview about

0:01:11.907,0:01:16.697

Yeah. Have Sydney. I’m sure if I is a Ai

0:01:16.697,0:01:18.887

powered risk based testing solution,

0:01:18.887,0:01:22.657

what that employees is when developers changed application,

0:01:22.657,0:01:24.937

we’re able to determine where that change is made,

0:01:24.937,0:01:27.697

order to selecting and execute just a few tests

0:01:27.697,0:01:29.697

associated with that change area,

0:01:29.697,0:01:33.047

So rather running a hundred percent of your mobile appian test.

0:01:33.047,0:01:36.017

We’re able just focus on the top five percent of test.

0:01:36.017,0:01:39.487

And still catching the functional aggression progressions in that change set.

0:01:39.487,0:01:42.317

So optimize Crm pipelines, ultimate

0:01:42.317,0:01:44.887

mi test drums, get feedback,

0:01:44.887,0:01:47.487

faster, two developers to find these functional regression

0:01:47.487,0:01:48.867

earlier. To help

0:01:48.867,0:01:50.817

increase get to more time.

0:01:50.817,0:01:54.277

Awesome. Thank you.

0:01:54.767,0:01:56.767

Karen, can tell us a little bit about App?

0:01:57.287,0:02:02.017

Yes. Thank you, Cindy and and Kobitonfor this opportunity.

0:02:02.017,0:02:06.927

So App dom as a company, our mission is to protect every mobile app in the world

0:02:06.927,0:02:08.177

and the people who use and

0:02:08.177,0:02:10.307

these apps in their lives network.

0:02:10.307,0:02:17.088

As a technology is a platform for building, testing, leasing and monitoring all

0:02:18.157,0:02:19.897

in the dev Ci pipeline,

0:02:19.897,0:02:23.967

So our customers are able to accelerate the delivery of mobile app.

0:02:23.967,0:02:25.677

Security, anti fraud,

0:02:25.677,0:02:28.427

anti malware, anti cheat anti bot,

0:02:28.427,0:02:31.438

in other in in N apps.

0:02:31.438,0:02:43.077

Awesome. And Lindsay, let’s hear a little bit about Kobiton. Well, if you guys have not planned on hearing a lot about Kobiton today, we enable better mobile to

0:02:43.077,0:02:49.657

three my, in our customer success function. We’re very interested in learning more about what our

0:02:49.657,0:02:53.047

customers are trying to achieve with their specific mobile apps, customers

0:02:53.047,0:02:56.637

and defining new milestones for them to conquer.

0:02:56.637,0:03:00.967

An understanding where Covid can be a tool in their tool belt to help them achieve it.

0:03:00.967,0:03:08.597

Awesome. So before we jump right in with some of the the questions that were submitted online,

0:03:08.597,0:03:13.138

We do want to make note that in the chat, if you have any questions for our panel list.

0:03:13.138,0:03:18.108

Please enter them in the chat and at the very end, I will run through any of those.

0:03:18.647,0:03:20.877

So to kick us off, Karen.

0:03:20.877,0:03:25.507

How have you seen the mobile app delivery landscape evolve in the recent years?

0:03:25.507,0:03:28.177

So mainly what we’re seeing is…

0:03:28.178,0:03:31.647

The speed of delivery, the speed of the ability to

0:03:31.647,0:03:35.987

create new mobile apps in spite of all the new frameworks,

0:03:35.987,0:03:38.977

they’re out there and available in spite of aldi.

0:03:38.977,0:03:42.737

New operating systems? For Android Ios,

0:03:42.737,0:03:45.537

and all the specific functionality,

0:03:45.537,0:03:49.388

that make mobile apps sticky and interesting to use

0:03:49.388,0:03:50.917

We see that

0:03:50.918,0:03:55.797

developers are a lot of pressure to deliver these features as well as

0:03:55.797,0:03:56.847

Keep up quiz

0:03:56.847,0:04:01.818

the security challenges. And there are a number of them because security tax are just

0:04:02.767,0:04:06.927

with Jenny, Bo attacks if things up at a faster rate.

0:04:06.928,0:04:12.677

Are many marketplaces out there, where hackers will take the

0:04:12.678,0:04:16.987

so now where did they produce and sell them to other hackers.

0:04:16.987,0:04:19.638

The hackers are using the same tools

0:04:20.618,0:04:23.718

developers. And and make games

0:04:25.377,0:04:25.997

taps and

0:04:25.997,0:04:28.057

type. The new malware that they deliver

0:04:28.057,0:04:30.107

even more power. So

0:04:30.107,0:04:31.757

are a lot of challenges that

0:04:31.757,0:04:33.807

the mobile app developer’s space,

0:04:34.647,0:04:35.997

as well as

0:04:35.997,0:04:37.878

you know, be able to deliver

0:04:40.138,0:04:44.457

Awesome. And

0:04:45.477,0:04:49.867

There we go. That sounds a little bit better. I apologize. I had a little bit of noise coming through.

0:04:49.868,0:04:52.188

Johnson, Quick question for yale.

0:04:52.188,0:04:57.158

How does effective testing contribute to faster app delivering and and how might

0:04:57.158,0:05:00.178

testing become a bottleneck.

0:05:00.178,0:05:06.928

Yeah. So I was gonna like to kinda flip this a little bit with ineffective testing,

0:05:06.928,0:05:09.398

So when you have ineffective testing,

0:05:09.987,0:05:12.207

kitchen bugs late in the process

0:05:12.207,0:05:13.517

again, the production,

0:05:13.517,0:05:16.767

you know, pushing it back to release deadlines,

0:05:16.767,0:05:19.357

So the reason why, you know,

0:05:19.357,0:05:20.798

we can’t start shifting

0:05:21.877,0:05:25.557

So faster out delivery is because no, they had this small idea in cons waterfall.

0:05:25.557,0:05:29.948

I apologize. Hoa landscaping just came right of time of this session.

0:05:29.948,0:05:32.747

So I I completely migrate into a different room right now, but

0:05:32.747,0:05:36.487

there’s a big weed wack right on outside my door. So Apologize we hear

0:05:36.487,0:05:38.777

not tree or something my that market gets the way.

0:05:38.777,0:05:39.827

Yeah.

0:05:39.827,0:05:42.297

What we’re talking about is

0:05:42.297,0:05:46.897

accelerate at delivery, and effective testing and helping you to deliver that.

0:05:46.897,0:05:51.807

And automation is just one piece of the puzzle right there. You also have manual test.

0:05:52.477,0:05:56.467

The more you can automate the more you can actually automate that process and kick off.

0:05:56.467,0:05:59.567

The faster you can potentially find those regression early in the cycle.

0:05:59.567,0:06:02.257

And one of the key things that we find is that

0:06:02.257,0:06:05.767

We ask is are people running their tests as part of the Pipeline?

0:06:05.767,0:06:09.247

These very often you have these Ui integration test suites

0:06:09.247,0:06:15.408

turning outside the C five. Because they’re too big to be put into the Ci pipeline to be part of that hope build process.

0:06:15.987,0:06:18.917

So one area that, you know, we find and help

0:06:18.918,0:06:20.647

overcome that bottleneck, you

0:06:20.647,0:06:22.967

is by just selecting which test irrelevant

0:06:22.967,0:06:24.688

given a pro change basis,

0:06:24.688,0:06:28.877

So you can bring that full Ui feedback into the Ci pipeline so that…

0:06:28.877,0:06:33.517

Developers and testers. You get that full regression feedback by only running maybe just a smart subset

0:06:33.517,0:06:34.377

set a task. Set

0:06:34.377,0:06:36.428

Which is very compelling it helps.

0:06:36.428,0:06:38.157

Accelerate, go to market time,

0:06:39.577,0:06:42.847

Yeah. Let’s let’s double down there a little bit.

0:06:42.847,0:06:47.817

What what testing strategies and practices are most effective in accelerating the

0:06:47.817,0:06:50.937

developments, like like the development cycle. The

0:06:50.938,0:06:55.097

Yeah. Honestly, just… The the

0:06:55.097,0:06:59.057

It always like, you know, the feedback loop and the developers, the Qa developers,

0:06:59.057,0:07:02.007

we can somehow mers to be on top of each other,

0:07:02.007,0:07:04.687

there would be just a very strong line of communication.

0:07:04.688,0:07:07.507

When you’re gonna have the highest velocity for the whole team.

0:07:07.507,0:07:10.577

And the way to do that is to not have the two as silent.

0:07:10.577,0:07:15.517

And that’s where the, you know, the beauty of the Ci pipeline and automated testing really kinda of comes to the play.

0:07:15.517,0:07:20.437

Because if you can take that automation testing and bringing it closer and closer and closer shifting and left,

0:07:20.438,0:07:23.138

the top of the developer, you’d have less

0:07:23.138,0:07:24.888

issues with requirements,

0:07:24.888,0:07:26.567

less issues down the road.

0:07:26.567,0:07:29.857

Because there’s a just a strong feedback channel between testing,

0:07:29.857,0:07:32.087

and development based on top of each jump

0:07:32.477,0:07:34.397

it’s really hard to achieve because… And

0:07:34.397,0:07:38.347

if you build out this big Ui, you know, mobile testing suite,

0:07:39.198,0:07:42.367

hours to run on a given change.

0:07:42.368,0:07:45.787

So this without Share for us speak personally,

0:07:45.787,0:07:47.377

is basically a weight just

0:07:47.377,0:07:50.418

pick which test are relevant given the change area. So

0:07:50.418,0:07:52.717

If a developer may change the log on button,

0:07:52.717,0:07:56.138

you’re just running a test assistant load bundle. You’re not running tests.

0:07:56.138,0:07:59.007

In the count section or user a profile section.

0:07:59.007,0:08:01.267

Just running the test a series log button. You’re

0:08:01.267,0:08:02.657

you’re running any, you know email,

0:08:02.658,0:08:05.207

a percent… A very small percentage test

0:08:05.207,0:08:09.107

get that regression feedback to developers on a per commit basis.

0:08:09.107,0:08:10.867

And while you’re also doing that,

0:08:10.868,0:08:12.727

You’re also running less test.

0:08:12.727,0:08:15.727

You’re, you know, beforehand, you’re running all these tests in parallel.

0:08:15.727,0:08:18.827

Percent of your test and you have a lot of servers to keep time down.

0:08:18.827,0:08:21.627

Now you’re just running maybe five percent or ten percent of test.

0:08:21.627,0:08:23.507

And getting that feedback faster.

0:08:23.507,0:08:26.087

We’re also saving a lot of money on infrastructure at the same time. We’re

0:08:26.087,0:08:28.258

And the add

0:08:28.908,0:08:33.847

a different viewpoint from what John said said. I love what he talked about in putting the development

0:08:33.847,0:08:37.147

team and the Qa team together on top of one another.

0:08:37.147,0:08:40.737

And thinking about the cycle that a code… Like a piece of code go

0:08:40.737,0:08:43.337

from from creation to release.

0:08:43.337,0:08:46.707

If you are creating the ability to understand what

0:08:46.708,0:08:51.677

could be a defect at a functional level during that development life

0:08:51.678,0:08:55.208

cycle and then allowing Qa to do what they actually do best.

0:08:55.208,0:08:58.987

Look for things that are not instantly obvious,

0:08:58.988,0:09:02.547

that could go wrong that could break within your mobile app

0:09:02.547,0:09:04.698

or how a user may

0:09:05.458,0:09:09.377

move around your app and interact with certain components to it.

0:09:09.377,0:09:14.078

To find things that they need to consider to reiterate back into the testing process

0:09:14.078,0:09:17.788

before bug gets released into defect… Into code.

0:09:19.687,0:09:21.218

That makes sense,

0:09:21.218,0:09:26.187

So Lindsey, let’s stick with you for a moment. How does automation and

0:09:26.187,0:09:28.528

enhance our mobile app testing.

0:09:29.757,0:09:32.698

So I always tell people as human beings were limited.

0:09:32.698,0:09:36.317

I can only do so many tasks at a given time and I only get

0:09:36.317,0:09:40.377

so many hours in a day, even if someone figures out how to not have to sleep

0:09:40.377,0:09:45.348

And so if I have a partner and I call that partner Ai who

0:09:45.348,0:09:50.137

has the ability to work alongside me. And to be able to double check

0:09:50.138,0:09:51.958

I work, I’m gonna be more efficient.

0:09:51.958,0:09:55.968

And I believe one of the greatest examples is of that is

0:09:55.968,0:09:58.987

something that we offer at C and accessibility test

0:09:58.988,0:10:02.968

thing because I as as it maybe a functional tester in the

0:10:02.968,0:10:06.887

app who is clicking through and ensuring that I’m running my test case

0:10:06.888,0:10:09.908

I’m not focusing in on is the color contrast on this

0:10:09.908,0:10:12.927

page, something that goes against guidelines,

0:10:12.928,0:10:17.057

that our accessibility team may be concerned about, when we finally get this

0:10:17.057,0:10:19.307

this set of code all the way to them

0:10:19.307,0:10:22.627

All the way to them normally at the end of the testing process.

0:10:22.627,0:10:25.647

I’m not, and I don’t have the time and ability to be focused

0:10:25.647,0:10:29.477

on that. But if I have Ai running in the background while I’m working through my

0:10:29.478,0:10:33.638

functional test, whether they are automated or manual, and I’m reviewing

0:10:33.638,0:10:38.247

everything at the end of my session in session explorer. And I have a little hand that’s a yellow dot

0:10:38.247,0:10:41.337

this says, hey, there’s something here you may wanna take a look at.

0:10:41.337,0:10:45.047

I wasn’t having to look for it, and I can quickly send that

0:10:45.047,0:10:49.718

to my accessibility team or even my developers and say, hey, this is something that we need to be

0:10:49.718,0:10:57.107

considering as a design change or a functionality change that could protect our our organization,

0:10:57.107,0:11:02.367

from particular lawsuit. And I think that that is something that expands my ability

0:11:02.368,0:11:06.258

as a functional tester to that that is something that can

0:11:06.917,0:11:09.917

speed up the the path of delivery.

0:11:11.567,0:11:17.677

Okay. Keeping on the topic of automation and Ai,

0:11:17.678,0:11:20.997

This is a question for the for the entire the entire group here.

0:11:20.997,0:11:24.857

If anyone has any insight. But are there any specific tools or technologies

0:11:24.857,0:11:29.578

that any of your companies offer in regards to Ai and mobile testing?

0:11:29.888,0:11:32.958

Karen? Yeah. So

0:11:32.958,0:11:36.837

that’s actually something that is part of the

0:11:36.837,0:11:39.287

the base of A technology,

0:11:39.287,0:11:41.737

what we do is automate,

0:11:41.738,0:11:46.437

the delivery of protections for mobile apps and what that means is we look

0:11:46.437,0:11:47.958

Yeah. The app.

0:11:47.958,0:11:50.908

After the app has is loaded to our platform,

0:11:51.587,0:11:55.547

we look at the protections that the customer wants to add.

0:11:55.547,0:11:58.247

Say it’s encryption or modification.

0:11:58.247,0:11:59.638

And we match

0:12:00.167,0:12:02.517

and this is our technology doing this.

0:12:02.517,0:12:05.437

Matching the protections,

0:12:05.718,0:12:07.167

that people want to add,

0:12:07.167,0:12:09.357

to the app itself.

0:12:09.357,0:12:12.698

And optimizing how those protections are added,

0:12:12.698,0:12:16.187

there’s a Mic services architecture, and we’re simply

0:12:16.187,0:12:19.427

mapping the plugins that are needed

0:12:19.428,0:12:21.667

to be added in to the app

0:12:21.667,0:12:26.537

in a automated way, so that developers don’t have to code, They don’t have to use Sdk.

0:12:26.537,0:12:28.377

It’s all optimized for them.

0:12:28.377,0:12:32.807

And in that process, not only are we optimizing how the protections are added.

0:12:32.807,0:12:37.567

We’re also optimizing for the size of the app after protection. We know that a lot of

0:12:37.567,0:12:39.357

Organizations are concerned about.

0:12:39.357,0:12:40.587

It increased size. Yeah. It

0:12:40.587,0:12:46.257

Got it. It. Care with you too.

0:12:46.257,0:12:51.227

How can companies like App and app and C, you know, all work together to

0:13:06.666,0:13:07.796

Been added, I’ve

0:13:08.396,0:13:09.856

may shut down,

0:13:09.856,0:13:13.325

you know, the app or as the app is being tested,

0:13:13.326,0:13:17.346

The apple shut down because these security protections are in there. So

0:13:17.346,0:13:20.115

For example, you may have

0:13:20.116,0:13:23.166

protection against the use of. Now

0:13:23.166,0:13:26.825

in your testing automation process, you might have

0:13:26.826,0:13:30.635

been using an. And because

0:13:30.636,0:13:34.906

app don’t detects that the emulator is being used, it’ll shut down the app. Now

0:13:34.906,0:13:36.346

what we’re doing with

0:13:36.346,0:13:38.596

Ko and we’re really excited about.

0:13:38.596,0:13:42.195

Is basically making sure that we know

0:13:42.196,0:13:44.355

that’s happening. So we’re trusting

0:13:44.356,0:13:48.166

Cob, so to speak. So that when Covid is

0:13:48.166,0:13:51.096

being test is testing an app secured app

0:13:51.096,0:13:56.006

App dome says, I know C is testing me. Therefore, I trust it and I will not shut down

0:13:56.006,0:13:58.006

and let Ga Down continue to test.

0:13:58.006,0:14:00.366

And so I think that’s really exciting

0:14:00.366,0:14:02.625

because it is making

0:14:03.186,0:14:11.856

the testing process that more intelligent. It’s, like you were saying, Lindsay, it’s a Another sort of part of the process where you have intelligence

0:14:11.856,0:14:16.505

that is a partner in automating this process.

0:14:16.506,0:14:25.056

The second thing that I think that’s really exciting. This goes back to appointment z that you made as well as Johnston, you made about Ai and that is we have

0:14:25.056,0:14:27.975

really interesting security events.

0:14:27.976,0:14:32.946

That we’re capturing in apt. And those security events

0:14:33.126,0:14:36.386

can be very informative during the test

0:14:36.386,0:14:39.525

process and when presented in context,

0:14:39.526,0:14:42.665

for a particular app for a particular test,

0:14:42.666,0:14:46.095

can be very useful in testing

0:14:46.096,0:14:48.216

you know, the functionality of the app.

0:14:48.886,0:14:52.055

So… In

0:14:52.056,0:14:55.916

like to double down there again. You know, what are some of the benefits of integrating different

0:14:55.916,0:14:58.496

tools and services in the in the process, tools

0:14:58.496,0:15:01.966

Yes. So there are a a number of benefits the first

0:15:01.966,0:15:05.885

is, you know, going along the theme of, okay, we have now visibility

0:15:05.886,0:15:10.216

forty the end user, whether that end user is

0:15:10.216,0:15:18.745

tester, you know, a test automation person or whether the person is a a security person. So in the operations part of the Devops

0:15:18.746,0:15:22.895

life cycle, having this continuous security

0:15:22.896,0:15:23.826

is important is

0:15:23.826,0:15:28.345

obviously, for security. But definitely in the development part of cycle,

0:15:28.346,0:15:35.166

enabling the Testing, automation person to have that ability is important as well. And so

0:15:35.166,0:15:38.035

what you can see then are

0:15:38.036,0:15:42.756

know, in in the automation, being able to have

0:15:42.756,0:15:45.326

the the data,

0:15:45.326,0:15:47.645

in a form that

0:15:47.646,0:15:51.686

the persona, the specific individual can see

0:15:51.686,0:15:55.865

and that means they can see what apps, Ios version are being attacked,

0:15:55.866,0:16:02.196

and apply and understand how to deal with, the security events that are coming up,

0:16:03.086,0:16:07.245

and and that is specific also to the device.

0:16:07.246,0:16:08.705

That. They’re testing.

0:16:10.226,0:16:13.276

That makes sense. What I hear from a lot of

0:16:13.276,0:16:17.886

customers is understanding vulnerabilities, whether their defect

0:16:17.886,0:16:21.236

or security, breach opportunities,

0:16:21.236,0:16:26.115

by device and Os and understanding where that impacts their customer base.

0:16:26.116,0:16:29.616

That can also… That knowledge can be power for them to

0:16:29.616,0:16:32.636

understand where they need to be able to spend additional development

0:16:32.636,0:16:36.856

effort or additional testing efforts and ensuring that they are sharing an

0:16:36.856,0:16:40.405

up that vulnerability. And in order to do that,

0:16:40.406,0:16:43.575

many times you need to know that you have a tool set

0:16:43.576,0:16:46.626

that can work flawlessly through the entire process.

0:16:46.626,0:16:51.325

When you see teams who are either working as individuals and silos or their

0:16:51.326,0:16:56.535

teams, are t off from one another, that additional time and

0:16:56.536,0:17:01.086

effort that it takes to communicate what’s being done or what needs to be done is time lost

0:17:01.086,0:17:06.056

from release or time loss from testing to ensure that we aren’t releasing issues.

0:17:06.235,0:17:09.705

And so the importance of ensuring that you have a tech stack that works

0:17:09.706,0:17:14.376

towards your end goal and that all of those components are perfectly seamless

0:17:14.376,0:17:17.855

seamlessly tied together can be the difference between

0:17:17.856,0:17:20.525

being excellent at mobile app delivery.

0:17:21.295,0:17:25.215

Getting to the finish line at some point, whether that’s on time or delayed.

0:17:25.215,0:17:27.396

Exactly. And so you

0:17:27.396,0:17:30.235

our basically verifying security enforcement is part of

0:17:30.235,0:17:34.125

single automated standard process, and then

0:17:34.126,0:17:39.036

in that way you can eliminate the need for say, split testing paths, like you’re talking about and verifying

0:17:39.356,0:17:40.836

and advanced and

0:17:40.836,0:17:42.865

you at use experience or do I handling?

0:17:42.866,0:17:47.775

When these attacks happen. Great great

0:17:47.775,0:17:49.916

the discussion. Also, I apologize

0:17:49.916,0:17:54.206

Johnson, it seems to be your day at my house as well. So I I apologize in advance.

0:17:54.206,0:17:57.646

Johnson coming back to you real fast, how do you

0:17:57.646,0:18:00.435

strike a balance between, you know, delivering,

0:18:00.946,0:18:05.076

between delivering apps quickly and ensuring a high quality user experience.

0:18:06.346,0:18:07.616

Yeah. So, yeah

0:18:07.616,0:18:10.045

we like to say like… When we come them

0:18:10.045,0:18:13.556

into a new user base then. We’re gonna test your probably a hundred times more

0:18:13.556,0:18:17.056

more frequently than he would have at any given time.

0:18:17.056,0:18:18.685

What we’re doing is we’re just

0:18:18.686,0:18:20.495

focusing on the change barrier,

0:18:20.496,0:18:22.626

Now this is our Google and meta

0:18:22.626,0:18:25.315

test. They built as a proprietary and house

0:18:25.315,0:18:27.435

test platforms,

0:18:27.436,0:18:31.696

And what they do is, during the day, with from butts seats, the developers are watching,

0:18:31.696,0:18:34.065

you get a smart subset given our change.

0:18:34.065,0:18:38.995

And then where essentially is you studying a very small amount risk.

0:18:38.996,0:18:41.356

When saying Ai to choose which tests,

0:18:41.356,0:18:44.065

to select given a developer change rate.

0:18:44.065,0:18:45.666

Now that Ai model.

0:18:45.666,0:18:49.516

You talk to anybody in the market. Ai is never a hundred percent accurate.

0:18:49.516,0:18:56.836

I don’t, if they were do then run the other one. Ai is only gonna be able to provide so much accuracy than you given time Is it so much

0:18:56.836,0:18:59.125

inner workings and fine tuning software forth.

0:18:59.126,0:19:03.476

So what we’re doing is we’re considering a very small controlled amount of risk.

0:19:03.476,0:19:06.185

I’m gonna get you a lot of background.

0:19:06.186,0:19:10.755

That’s need by essentially

0:19:10.755,0:19:13.016

in inserting a very small element of risk,

0:19:13.016,0:19:15.616

To get ten x output,

0:19:15.616,0:19:23.125

And then to just run more subsets of the day and the change errors. And then at a depending on our test strategy,

0:19:23.126,0:19:24.856

you run your full test suite,

0:19:24.856,0:19:29.375

nightly or every on the night or potentially on the weekends. So most enterprises that we work with,

0:19:31.676,0:19:34.725

Got the week. When production developers are working. Got

0:19:34.726,0:19:38.465

and then on whole weekend they fill that full regression. So you kinda have that balancing active

0:19:38.466,0:19:40.806

very fast feedback to developers,

0:19:40.806,0:19:45.375

but also a catch all safeguard before, you know, releasing into the next stage development.

0:19:45.525,0:19:50.176

Releasing it to into production. So you kinda get the best of both worlds.

0:19:50.176,0:19:53.495

Faster delivery, faster feedback and also

0:19:53.496,0:19:56.036

still like catch all in case of any leakage.

0:19:59.025,0:20:05.826

Yeah. Karen what in that same conversation, You know, what challenges are organizations facing in the devops

0:20:05.826,0:20:09.076

life cycle with respect to security and testing.

0:20:09.076,0:20:11.665

Yeah. That’s a great question. And we’d

0:20:11.666,0:20:14.326

touched on a few of those elements earlier,

0:20:14.326,0:20:19.295

But if we divide up the devops ops, I cycle into dev and ops and look at

0:20:19.295,0:20:23.315

the development part first. In the development part,

0:20:23.315,0:20:24.565

you know, there are

0:20:24.565,0:20:25.866

a number of things that

0:20:25.866,0:20:29.235

organizations would do before

0:20:30.036,0:20:32.935

they actually do and add the security part.

0:20:32.935,0:20:33.995

Into the product?

0:20:33.996,0:20:36.016

Into the app and

0:20:36.016,0:20:39.435

and that includes vulnerability testing, includes scanning,

0:20:39.435,0:20:44.406

It’s basically where they’re identifying the security issues in the app

0:20:44.406,0:20:48.826

Now the challenge is once those issues have been identified,

0:20:48.826,0:20:53.495

developers have to manually code in the fixes or resolve the issues, and that takes

0:20:53.496,0:20:54.266

time,

0:20:54.266,0:20:55.545

and could delay the re

0:20:55.545,0:21:00.935

release. So the organization in some cases, instead of dealing with those delays,

0:21:00.935,0:21:02.536

we’ll waiver the issue away.

0:21:02.536,0:21:07.065

Him that only increases the risk of the app once it’s released.

0:21:07.065,0:21:09.696

Now if the developer does code in,

0:21:09.696,0:21:12.315

the security fixes

0:21:12.315,0:21:16.235

and the app can’t be tested in an automated way like we talked about earlier,

0:21:16.235,0:21:19.485

again, the app what

0:21:19.485,0:21:22.926

shut down, you know, because it’s not recognizing

0:21:22.926,0:21:27.145

that it is being tested in an automated way. And so this is again

0:21:27.145,0:21:31.665

another issue where The automation team must to go back and manually test

0:21:31.666,0:21:36.155

which would delay the relief. So there are number places in the development part the cycle

0:21:36.156,0:21:40.355

that break down or that are challenging and slow down

0:21:40.355,0:21:44.806

Today The second part of the cycle operations also has

0:21:44.806,0:21:49.126

a number of issues. And those issues have to deal with

0:21:49.126,0:21:53.075

having visibility in monitoring security.

0:21:53.076,0:21:54.475

So in

0:21:54.475,0:21:55.955

In this part of the cycle,

0:21:55.956,0:21:59.235

you’re looking at operation that

0:21:59.235,0:22:03.715

when an issue was identified, the same sort of

0:22:03.716,0:22:06.735

problems occur, where once it’s identified,

0:22:06.735,0:22:10.205

It goes back to the developer the developer has to code in, a fix,

0:22:10.206,0:22:12.005

if they’re unable to

0:22:12.005,0:22:16.975

then the issue is wave away. So how do you identify these issues

0:22:16.975,0:22:22.575

and then automate the remediation in a seamless way and

0:22:22.576,0:22:26.356

and those are the big challenges that we see organizations face today.

0:22:29.126,0:22:32.175

Sounds good. Yep.

0:22:32.176,0:22:37.145

Guys, I just wanna thank you all for your participation today. We do have

0:22:37.145,0:22:41.515

coming up on our the end of our session. So we do have some questions from the audience.

0:22:41.516,0:22:46.275

What are some mobile testing tools and resources that can be used

0:22:46.275,0:22:48.126

to speed up the testing process?

0:22:48.126,0:22:50.165

Especially when times are tight

0:22:50.435,0:22:52.615

So this is open forum, Anybody?

0:22:52.616,0:22:57.765

One is… Wanna grab that one. I think you have to ask yourselves

0:22:58.766,0:23:03.225

what why the speed is speed the most important thing to your organization

0:23:03.225,0:23:04.885

in your your mobile app.

0:23:04.886,0:23:08.446

Because there is an opportunity to say we need to slow down if

0:23:08.446,0:23:13.285

quality is the most important factor to your end users on your application.

0:23:13.855,0:23:18.825

The best way to achieve speed is to to create more testing. And so that’s

0:23:18.826,0:23:24.955

through in human interaction and manual testing and then you have to bring in more people.

0:23:24.956,0:23:28.216

Or you can can learn to automate. And

0:23:28.216,0:23:32.286

you know, one of the beautiful things about working for Cob is we offer different solutions

0:23:32.286,0:23:36.536

We have a script list option that can allow you to automate test.

0:23:36.536,0:23:37.405

And then

0:23:39.306,0:23:41.955

run that or on revisit cross,

0:23:41.956,0:23:45.306

other devices that can sometimes help create or inject,

0:23:45.306,0:23:50.275

speed into a process for an organization that’s struggling to figure out the direction they wanna take

0:23:50.275,0:23:53.175

work mobile automation,

0:23:53.176,0:23:56.315

But outside of that, it’s it really comes down to understanding work.

0:23:56.315,0:24:00.475

Speed and quality you need to meet together when it comes to an individual least.

0:24:00.475,0:24:04.605

Yeah. And to kinda piggyback off that, you hear a lot of new tools that check

0:24:04.605,0:24:07.595

Gp, spinning up a lot of… Is it easier than ever Gp

0:24:07.596,0:24:09.325

to write automated tests.

0:24:09.326,0:24:13.786

And we’re kinda of focused on the automation test side of the house. And we’ve seen companies

0:24:13.786,0:24:17.166

you know, go from a thousand test to ten thousand tests to spend of three months.

0:24:17.166,0:24:19.635

A couple of years ago, that would’ve have taken three years.

0:24:19.636,0:24:23.255

So the problem the issue is that you could be you could spin up a lot of tests

0:24:23.255,0:24:26.455

gonna make sure that the tests are actually testing with their intended test.

0:24:26.456,0:24:29.475

And validated the actual outcomes in the requirements.

0:24:29.475,0:24:32.225

So it’s more than ever to kind of hone in on…

0:24:32.225,0:24:36.045

You know, not necessarily just getting a massive amount of tests,

0:24:36.045,0:24:40.065

coverage, but also just building the right type of test framework to actually deliver

0:24:40.065,0:24:41.435

you intended to be delivering?

0:24:41.435,0:24:43.885

And choose carefully there.

0:24:43.886,0:24:50.525

And this is why I’m so excited about in particular the abdomen and C, integration is that

0:24:50.525,0:24:54.635

and integrations is that we’re able to really automate

0:24:54.636,0:25:00.366

The development of security features, the testing of those security features,

0:25:00.366,0:25:03.785

through this integration, this pre built integration that we have

0:25:03.786,0:25:09.225

into that is a huge lift in the process, a huge acceleration

0:25:09.225,0:25:12.455

is because they don’t have to go back and forth, do anything manually.

0:25:12.456,0:25:14.906

Whether it’s development or testing,

0:25:14.906,0:25:17.485

and they don’t have to go back and just…

0:25:17.485,0:25:21.775

And often oftentimes companies will make this choices to increase risk.

0:25:21.775,0:25:26.616

Bye. Not adding security or not doing the testing. Right? They can have it all.

0:25:26.616,0:25:27.905

And still

0:25:27.906,0:25:28.876

deliver the app on time.

0:25:28.876,0:25:35.835

Yep. Well, thanks guys so much for sharing and in participating in our panel today. We are

0:25:35.835,0:25:39.906

coming up on the end. So we’re gonna wrap it up. I would love note

0:25:57.632,0:26:02.262

And it really depends on the project that you’re really working to build out. But

0:26:02.262,0:26:04.091

At the end the day, you wanna get

0:26:04.092,0:26:08.512

Out door. And you went out out the door with a good user experience.

0:26:08.512,0:26:12.432

So just plan those requirements, building out the foundation and kind of

0:26:12.432,0:26:13.972

really started hone

0:26:13.972,0:26:17.172

see where you can’t automate, but also respect where you can’t automate

0:26:17.172,0:26:21.422

keep the manual side because It’s always gonna be a manual component. And that’s just one of the core competencies season

0:26:21.422,0:26:26.351

strength of Covid is that you really kinda get that robust manual element of testing within

0:26:26.351,0:26:30.731

And then if you wouldn’t know for the case that you can automate, how can you

0:26:30.732,0:26:33.202

get those Ui testing for taking on five to ten minutes,

0:26:33.202,0:26:36.431

sooner and earlier and closer to the developer.

0:26:36.432,0:26:39.212

Because that’s ultimately the one who’s gonna be pushing that release

0:26:39.212,0:26:44.181

pushing machine it out. So if you can get, you know, testing on top of development,

0:26:44.182,0:26:46.532

and really so they’re not too solid organizations and

0:26:46.532,0:26:50.091

but really just speaking one language, that’s when it really be really powerful

0:26:50.092,0:26:54.242

we see the future outlook of testing as when a developer makes a change

0:26:54.242,0:26:57.951

is their coverage for this test? Is there coverage for this change?

0:26:57.952,0:27:02.922

Because now there are tools or soon tools, they’re gonna be able to write out test

0:27:02.922,0:27:06.121

on change air, they don’t necessarily have automation test coverage.

0:27:06.121,0:27:15.232

And that’s a really exciting journey that we’re going on. But as I mentioned earlier, with E, Suddenly, the Ai could be spinning a bunch of tests of this change area that has nothing to do with anything.

0:27:15.232,0:27:19.462

So you always got be really on top of it. They make sure that these newly introduced test

0:27:19.462,0:27:21.701

are relevant for the recalls and ambitions.

0:27:21.702,0:27:24.892

Yes. And I would

0:27:24.892,0:27:28.331

add then build on it with saying there’s the data.

0:27:28.331,0:27:32.432

That is so important to the automation tester and to…

0:27:32.432,0:27:35.822

Any organization, having that information

0:27:35.822,0:27:41.521

increases the the the value of that testers user experience. It also increases

0:27:41.522,0:27:46.432

you know, the value of the end user’s experience, but certainly, having that data at your fingertips in

0:27:46.432,0:27:51.162

context, makes know the test automation person that much powerful whether it is

0:27:51.162,0:27:54.272

Like that’s when you’re saying a manual or a automated.

0:27:54.502,0:27:59.311

Testing. And I would say it’s really important to understand.

0:27:59.312,0:28:03.081

Your your partners and making sure your partners understand

0:28:03.081,0:28:04.742

the importance of your app,

0:28:04.742,0:28:09.141

your app is not vanilla. You don’t want it to be something that may or may not

0:28:09.141,0:28:14.112

sit on someone’s phone. There’s a business impact to that and understanding

0:28:14.112,0:28:16.751

the incremental changes that you can

0:28:16.752,0:28:21.712

can introduce through the ever changing landscape of mobile app development.

0:28:21.712,0:28:24.911

Can help you get faster. It can help increase your quality. It

0:28:24.912,0:28:29.641

you can help drive the Roi that your mobile app delivery can build for your business. You

0:28:29.641,0:28:32.762

or can create efficiencies for the people who use it,

0:28:32.762,0:28:37.622

and understanding the why behind it and working together, you can have all of that.

0:28:37.622,0:28:39.071

A reasonable period of time.

0:28:42.092,0:28:45.111

Well, thank you guys so much for joining us today

0:28:45.112,0:28:48.012

Lindsey Karen Johnson, it has been a pleasure.

0:28:48.012,0:28:50.952

And we really look forward to the rest of the summit.

0:28:50.952,0:28:57.402

And And this that’s enough. We’re actually gonna be pushing people to the booth right now. Our first

0:28:57.402,0:29:00.211

free of the days you at the store

0:29:00.212,0:29:04.482

Aaron Johnston, K and or of our speaking so far.

0:29:04.482,0:29:06.991

Head over there, or be happy to

0:29:06.992,0:29:11.301

answer them and and we couldn’t do this without the support of our partners and sponsor

0:29:11.302,0:29:12.232

or so.

0:29:12.232,0:29:15.431

First break of a day, go give them a little love in the booths.

0:29:15.432,0:29:17.762

And we’ll see you guys back here in about

0:29:17.762,0:29:19.741

fifteen minutes for some of our breakouts.

0:29:20.752,0:29:24.212

Thank you. Thank you. Bye.