Accelerating Testing for Faster, More Reliable Releases

Video Transcript

0:00 | Phone Caller #5
Hello, everyone. We are on to the next session here. It is a panel discussion with some of our partners here at Kobiton who are sponsoring the event as well. So our channel lead, Sydney Weber will be leading the discussion along with some of the partners from AppSureify and AppDome. So with that, I am going to bring them to the stage. Sydney, the floor is yours.

0:28 | Phone Caller #1
Good morning, everybody.

0:30 | Phone Caller #4
Thank you so much for joining us. So welcome to our panel.

0:33 | Phone Caller #2
Discussion today.

0:37 | Phone Caller #5
And we.

0:37 | Phone Caller #4
are going to be talking about accelerating mobile app delivery and testing. So, let me introduce you to our panelists today. So we have Johnson, who is the co founder and CEO of AppSureify. We have Karen who is the SVP of product, mobile DevOps and security solutions from AppDome. And we have our very own Lindsey, who is our head of customer success here at Kobiton. So Johnson, can you give us a little bit of a brief overview about AppSureify?

1:06 | Phone Caller #2
Yeah. I’m sure. If I is a AI powered risk based?

1:12 | Phone Caller #4
Testing solution, What that means is when.

1:15 | Phone Caller #2
developers change the application, we’re able to determine where that change was made and auto select and execute just a few tests associated with that change area. So, rather than running 100 percent of your mobile Appium tests, we’re able to just focus on the top five percent of tests and still catching the functional regressions in that change set. So optimizing CI pipelines, optimizing test runs to get feedback faster to developers, to find these functional regressions earlier to help increase our get to market timing?

1:47 | Phone Caller #4
Awesome. Thank you, Karen. Can you tell us a little bit about Appdome?

1:52 | Phone Caller #1
Yes, thank you, Sydney and Kobiton for this opportunity. So, Appdome is a company, Our mission is to protect every mobile app in the world and the people who use these mobile apps in their lives and at work As a technology, Appdome is a platform for building testing, leasing and monitoring all mobile app protections in the DevOps CIC pipeline. So our customers are able to accelerate the delivery of mobile app, security, anti fraud, anti malware, anti cheat, anti bot and other defenses in Android and iOS apps.

2:28 | Phone Caller #4
Awesome. And Lindsey, let’s hear a little bit about Kobiton?

2:32 | Phone Caller #3
Well, if you guys have not planned on hearing a lot about Kobiton today, we’re in Able Bettering Mobile Delivery. And in our customer success function, we’re very interested in learning more about what our customers are trying to achieve with their specific mobile apps and defining new milestones for them to conquer and understanding where Kobiton can be a tool in their tool belt to help them achieve it.

2:57 | Phone Caller #4
Awesome. So before we jump right in with some of the questions that were submitted online, we do want to make note that in the chat, if you have any questions for our panelists, please enter them in the chat. And at the very end, I will run through any of those. So to kick us off, Karen, how have you seen the mobile app delivery landscape evolve in the recent years?

3:21 | Phone Caller #1
So mainly what we’re seeing is the speed of delivery, the speed of the ability to create new mobile apps in spite of all the new frameworks that are out there and available in spite of all the new operating systems for Android and iOS, and all the specific functionality that We see that developers are under a lot of pressure to deliver these new features as well as keep up with the security challenges. And there are a number of them because security attacks are just going to happen With Gen AI. Those attacks are just going to happen at a faster rate. There are many marketplaces out there where hackers, those will take the malware that they’re producing.

4:09 | Phone Caller #4
And sell them to other hackers.

4:11 | Phone Caller #2
The hackers are using the same tools that developers use in making these new.

4:19 | Phone Caller #1
Hacks, the new malware that they deliver even more powerful. So there are a lot of challenges that the mobile app developers face as well as, you know.

4:31 | Phone Caller #2
Being able to deliver these new apps.

4:37 | Phone Caller #4
Awesome. There we go. That sounds a little better. I apologize. I had a little bit of noise coming through Johnson, quick question for you. How does effective testing contribute to faster app delivering? And how might testing become a bottleneck?

4:55 | Phone Caller #2
Yeah. So I always kind of like to kind of flip this a little bit with ineffective testing. So when you have ineffective testing, you know, you’re catching bugs late in the process, they get into production, you know, pushing back release deadlines. So the reason why, you know, we kind of start shifting to faster app delivery is because, you know, they have this whole idea concept of waterfall. And I apologize, Hoa, landscaping just came right at the time of this session. So I completely migrated to a different room right now but there’s a big weed whacker right outside my door. So I apologize if you hear a tree or something like that against the wall… What we’re talking about is accelerating app delivery and effective testing and helping you to deliver that. And automation is just one piece of the puzzle. Right there. You also have manual testing. But the more you can automate, the more you can actually automate that process and kick off, the faster you can potentially find those regressions earlier in the cycle. And one of the key things that we find is that we ask is, are people running their tests as part of the CICD pipeline? Because very often you have these UI and integration test suites, they’re running outside the CI pipeline because they’re too big to be put into the CI pipeline to be part of that whole build process. So one area that we find to help overcome that bottleneck is by just selecting which tests are relevant, given a per change basis. So you can bring that full UI feedback into the CI pipeline, so that developers and testers can get that full regression feedback by only running maybe just a smart subset of tests which is very compelling and helps accelerate go to market timing?

6:35 | Phone Caller #4
Yeah. Let’s double down there a little bit What testing strategies and practices are most effective in accelerating the development cycle, The development cycle?

6:46 | Phone Caller #2
Yeah, honestly, it just, we always like the feedback loop and the developers, the QA’s developers. If we can somehow merge those to be on top of each other and there would be just a very strong line of communication and that’s when you’re going to have the highest velocity for the whole team. And the way to do that is to not have the two as siloed. And that’s where the beauty of the CI pipeline and automated testing really kind of comes into play because if you can take that automation testing and bring it closer and closer, shifting it left the top of the developer, you can have less issues with requirements. You can have less issues down the road because there’s just a strong feedback channel between testing and development basically being on top of each other. And that’s really hard to achieve. Because if you build out this big UI, mobile testing suite that can take hours to run on a given change. So this with AppShare for us speaking personally is basically a way to just pick which tests are relevant given the change area. So if a developer makes a change to the logon button, you’re just running the test associated logon button, You’re not running tests in the account section or the user profile section. You’re just running the test associated logon button. So you’re running maybe, you know, a percent, a very small percentage of tests. So you can get that regression feedback to developers on a per commit basis. And while you’re also doing that, you’re also running less tests, you know, beforehand, you’re running all these tests in parallel, 100 percent of your tests, but you have a lot of servers to keep time down. Now, you’re just running maybe five percent or 10 percent of tests and getting that feedback faster while also saving a lot of money on infrastructure at the same time.

8:22 | Phone Caller #3
And to add a different viewpoint from what Johnson said, I love what he talked about in putting the development team and the QA team together on top of one another and thinking about the cycle that a piece of code goes from creation to release. If you are creating the ability to understand what could be a defect at a functional level during that development lifecycle and then allowing QA to do what they actually do best Look for things that are not instantly obvious that could go wrong, that could break within your mobile app or how a user may Move around your app and interact with certain components to it to find things that they need to consider to reiterate back into the testing process before a bug gets released into defect into code.

9:15 | Phone Caller #4
That makes sense. So Lindsey, let’s stick with you for a moment. How does automation and AI enhance our mobile app testing?

9:25 | Phone Caller #3
So I always tell people as human beings we’re limited. I can only do so many tasks at a given time and I only get so many hours in a day even if someone figures out how to not have to sleep. And so if I have a partner and I call that partner AI who has the ability to work alongside me and to be able to double check my work, I’m going to be more efficient. And I believe one of the greatest examples of that is something that we offer here at Kobiton in accessibility testing Because I, as a, as it may be a functional tester in the app who is clicking through and ensuring that I’m running my test case. I’m not focusing in on is the color contrast on this page? Something that goes against guidelines that our accessibility team may be concerned about when we finally get, this set of code all the way to them all the way to them. Normally, at the end of the testing process, I’m not, and I don’t have the time and ability to be focused on that. But if I have AI running in the background while I’m working through my functional tests, whether they are automated or manual, and I’m reviewing everything at the end of my session in Session Explorer. And I have a little hand raised that’s a yellow dot that says, hey, there’s something here you may want to take a look at. I wasn’t having to look for it and I can quickly send that to my accessibility team or even my developers and say, hey, this is something that we need to be considering as a design change or a functionality change that could protect, our organization from particular lawsuit. And I think that is something that expands my ability as a functional tester to, that is something that can speed up, the path of delivery.

11:07 | Phone Caller #4
Okay. Keeping on the topic of automation and AI. This is a question, for, the entire group here. If anyone has any insight. But are there any specific tools or technologies that any of your companies offer in regards to AI and mobile testing? Karen? Yeah. So that’s actually a.

11:29 | Phone Caller #1
Something that is part of, the base of AppFilm’s technology, What we do is automate the delivery of protections for mobile apps. And what that means is we look at the app after the app has been loaded to our platform and we look at the protections that the customer wants to add, Say it’s encryption or obfuscation, and we match. And this is our AI technology doing this matching the protections that people want to add… to the app itself and optimizing how those protections are added. So, there’s a microservices architecture, and we’re simply mapping the plugins that are needed to be added to the app in an automated way so that developers don’t have to code, they don’t have to use SDKs. It’s all optimized for them. And in that process, not only are we optimizing how the protections are added. We’re also optimizing for the size of the app after protections are added. Since we know that a lot of organizations are concerned about an increased size in the app.

12:39 | Phone Caller #4
Got it. Karen, let’s stick with you too. How can companies like AppSherify and AppDome, and Kobiton, you know, all work together to streamline the mobile app delivery and testing process?

12:50 | Phone Caller #1
Right. So there’s a couple of things The first is, you know, in the testing process, when you have a secured app, oftentimes the security protections that have been added may shut down, you know, the app or as the app is being tested, The app will shut down because these security protections are in there. So for example, you may have protection against the use of emulators. Now in your testing automation process, you might have been using, you know, an emulator, And because AppDome detects that the emulator is being used, it’ll shut down the app. Now, what we’re doing with Kobiton and we’re really excited about is basically making sure that we know that’s happening. So we’re trusting Kobiton so to speak. So that when Kobiton is being test, is testing an AppDome, secured app. Dome says, I know Kobiton is testing me, therefore, I trust it and I will not shut down and let Kobiton continue to test. And so I think that’s really exciting because it is making the testing process that more intelligent. And like you were saying, Lindsay, it’s a, it’s another sort of part of the process where you have intelligence that, is a partner, you know, in automating this process. The second thing that I think that’s really exciting. This goes back to a point Lindsey that you made as well as Johnston you made about AI. And that is we have really interesting security events that we’re capturing in AppDome. And those security events can be very informative during the testing process. And when presented in context for a particular app for a particular test can be very useful in testing, you know, the functionality of the app.

14:46 | Phone Caller #4
So, and I’d like to double down there again, you know, what are some of the benefits of integrating different tools and services, in the process?

14:54 | Phone Caller #1
Yeah. So there are a number of benefits. The first is, you know, going along that theme of okay, we have now visibility for, the end user, whether that end user is a tester, you know, a test automation person or the person is a security person. So in the operations part of the DevOps life cycle, having this continuous security is important obviously for security but definitely in the development part of the cycle, enabling the testing automation person to have that visibility is important as well. And so what you can see then are, you know, in the automation, being able to have, the data in a form that the persona the specific individual can see and that means they can see what apps by OS version are being attacked and apply the, and understand how to deal with the security events that are coming up. And that is specific also to the device that they’re testing.

16:06 | Phone Caller #3
That makes sense. I think what I hear from a lot of our customers is understanding vulnerabilities, whether they’re defects or security breach opportunities by device and OS, and understanding where that impacts their customer base. That can also, that knowledge can be power for them to understand where they need to be able to spend additional development effort or additional testing efforts in ensuring that they are shoring up that vulnerability. And in order to do that many times, you need to know that you have a tool set that can work flawlessly through the entire process. When you see teams who are either working as individuals in silos or their teams are truncated off from one another, that additional time and effort that it takes to communicate what’s being done or what needs to be done is time lost from release or time lost from testing to ensure that we aren’t releasing issues have a tool set that can work flawlessly through the entire process. And so, the importance of ensuring that you have a tech stack that works towards your end goal, and that all of those components are perfectly seamlessly tied together. Can be the difference between being excellent at mobile app delivery and getting to the finish line at some point, whether that’s on time or delayed.

17:21 | Phone Caller #1
Exactly. And so you are basically verifying security enforcement as part of a single automated standard process. And then in that way you can eliminate the need for say split testing paths like you’re talking about and verifying any advanced UI user experience or UI handling when these attacks happen.

17:43 | Phone Caller #4
Great love. The discussion also, I apologize Johnson. It seems to be yard day at my house as well. So I apologize in advance Johnson, coming back to you real fast. How do you strike a balance between, you know, delivering between delivering apps quickly and ensuring a high quality user experience?

18:02 | Phone Caller #2
Yeah. So we like to say like when we come into the, into a new user base that we’re going to test your product 100 times more frequently than you would have at any given time. Because what we’re doing is we’re just focusing on the change area. Now, this is how Google and Meta test, they build these in house test platforms. And what they do is during the day when the product is, when the butts are in seats developers are working, you get a smart subset given their change area. And then Well that essentially is doing you’re inserting a very small element of risk. You’re inserting AI to choose which test to select. Given a developer change area. Now that AI model, you can talk to anybody in the market. AI is never 100 percent accurate. If they ever do then run the other way. AI is only going to be able to provide so much accuracy at any given time because it’s so much inner workings and fine tuning and so forth. So what we’re doing is we’re inserting a very small component… controlled amount of risk. I’m getting a lot of background noise. I don’t know if that’s me or right now, But essentially inserting a very small element of risk to get 10 x output And then to just run smart subsets throughout the day in the change areas. And then depending on your test strategy, you run your full test suite nightly or every other night or potentially on the weekends. So most enterprises that we work with, they’re running smart subsets throughout the weekday throughout the week when production developers are working. And then on the weekend they film that full regression. So you kind of have that balancing act of very fast feedback to developers, but also a catch all safeguard before releasing it to the next stage of development or releasing it into production. So you kind of get the best of both worlds, Faster delivery, faster feedback. And also still that catch all in case of any leakage of let ups. Mm. Hmm.

19:54 | Phone Caller #4
Yeah, Karen, in that same conversation, what challenges are organizations facing in the DevOps lifecycle with respect to security and testing?

20:05 | Phone Caller #1
Yeah, that’s a great question. And we touched on a few of those elements earlier. But if we divide up the DevOps lifecycle into dev and ops and look at the development part first in the development part, there are a number of things that organizations will do before they actually do and add the security part into the product into the app. And that includes vulnerability testing. It includes scanning path testing. It’s basically where they’re identifying the security issues in the app. Now, the challenge is once those issues have been identified, developers have to manually code in the fixes or resolve the issues. And that takes time and could delay the release. So the organization, in some cases instead of dealing with those delays will waiver the issue away. And that only increases the risk of the app once it’s released. Now, if the developer does code in the security fixes, and the app can’t be tested in an automated way. Like we talked about earlier. Again, the app will shut down because it’s not recognizing that it is being tested in an automated way. And so this is again another issue where the automation team has to go back and manually test which would delay the release. So there are a number of places in the development part of the cycle that break down or that are challenging and slow down… Today. The second part of the cycle operations also has a number of issues. And those issues have to deal with having visibility and monitoring security. And so in this part of the cycle, you know, you’re looking at operation when an issue is identified, the same sort of problems occur where once it’s identified, it goes back to the developer. The developer has to code in A fix. If they are unable to, then the issue is wavered away. So how do you identify these issues and then automate the remediation in a seamless way. And those are the big challenges that we see organizations face today.

22:27 | Phone Caller #4
Sounds good. Yeah Guys. I just want to thank you all for your participation today. We do have we’re coming up on the end of our session. So we do have some questions from the audience. What are some mobile testing tools and resources that can be used to speed up the testing process? Especially when times are tight. So this is open forum. Anybody want to grab that one?

22:51 | Phone Caller #3
I think you have to ask yourselves what, why the speed Is speed? The most important thing to your organization and your mobile app? Because there is an opportunity to say we need to slow down if quality is the most important factor to your end users on your application. But the best way to achieve speed is to create more testing And so that’s either through human interaction and manual testing. And then you have to bring in more people or you can learn to automate. And one of the beautiful things about working for Kobiton is we offer different solutions. We have a scriptless option that can allow you to automate tests and then be able, to run that or on revisits across other devices that can sometimes help create or inject speed into a process for an organization that’s struggling to figure out the direction they want to take for mobile automation. But outside of that, it really comes down to understanding where speed and quality need to meet together when it comes to an individual release.

23:57 | Phone Caller #2
Yeah. And to kind of piggyback off that you hear a lot of new tools with ChatGPT spinning up a lot of. Is it easier than ever to write automated tests? And we’re kind of focused on the automation test side of the house? And we’ve seen companies, you know, go from a 1,000 tests to 10,000 tests in the span of three months A couple years ago. That would have taken three years. So the problem, the issue is that you can be, you can spin up a lot of tests but you got to make sure that the tests are actually testing what they’re intended to test and validate the actual outcomes and requirements. So it’s more than ever to kind of hone in on, you know, not necessarily just getting a massive amount of tests coverage, but also just building the right type of test framework to actually deliver what you intended to be delivering and choose carefully there.

24:40 | Phone Caller #1
And this is why I’m so excited about in particular, the AppDome and Kobiton integration. Is that and integrations is that we’re able to really automate The development of security features, the testing of those security features through this integration, this pre built integration that we have. And so that is a huge lift in the process, a huge acceleration of the development process because they don’t have to go back and forth do anything manually whether it’s development or testing And they don’t have to go back. And just And oftentimes companies will make this choice just to increase risk by not adding security or not doing the testing, right? They can have it all and still deliver the app on time.

25:27 | Phone Caller #4
Yeah. Well, thanks guys so much for sharing and participating in our panel today. We are coming up on the end. So we’re going to wrap it up. I would love Lindsay, Johnson, Karen, any parting words or tips you may have to test smarter or more securely for our audience today, Johnson, we can start with you.

25:48 | Phone Caller #2
Yeah. You know, there’s so many great tools out there in the market right now And it really depends on the project that you’re really working to build out. But at the end of the day, you want to get out the door And you went out the door with a good user experience. So just planning those requirements, building out the foundation and kind of really starting to hone in on see what you can automate, but also respect what you can’t automate and keep the manual side because there’s always going to be a manual component And that’s just one of the core competencies and strengths of Kobiton is that you really kind of get that robust manual element of testing within it. And then if you, for the cases you can automate, how can you get those UI tests which can take, you know, five to 10 minutes sooner and earlier and closer to the developer? Because that’s ultimately the one that’s going to be pushing that release, pushing it out. So if you can get, you know, testing on top of development and really, so they’re not two solid organizations but really just speaking one language that’s going to really be really powerful because we see the future outlook of testing as when a developer makes a change. So is there coverage for this test? Is there coverage for this change? Because now there are tools or soon to be tools. They’re going to be able to write out tests on change area that don’t necessarily have automation test coverage. And that’s a really exciting journey that we’re going on. But as I mentioned earlier with ChatGPT, suddenly the AI could be spending a bunch of tests for this change area that has nothing to do with anything. So you always got to be really on top of it to make sure that these newly introduced tests are relevant for your goals and ambitions.

27:20 | Phone Caller #4
Yes. And I would just add.

27:22 | Phone Caller #1
to then build on it with saying there’s the data that is so important to the automation tester and to any organization having that information increases the value of that tester’s user experience. It also increases, you know, the value of the end user’s experience. But certainly having that data at your fingertips in context makes you know the test automation person that much more powerful whether it is like Dustin was saying a manual or a automated test.

27:53 | Phone Caller #3
And I would say it’s really important to understand your partners and making sure your partners understand the importance of your app. Your app is not vanilla. You don’t want it to be something that may or may not sit on someone’s phone. There’s a business impact to that. And understanding The incremental changes that you can introduce through that ever changing landscape of mobile app development, can help you get faster. It can help increase your quality. It can help drive the Roi that your mobile app delivery can build for your business, or it can create efficiencies for the people who use it And understanding the why behind it and working together, you can have all of that within a reasonable period of time.

28:39 | Phone Caller #4
Well, thank you guys so much for joining us today. Lindsay, Karen Johnston. It has been a pleasure.