Making the Most of Your Tech Stack: Tips & Tricks for Better Mobile Testing

Video Transcript

0:18 | Billy Cabrera
hi folks. If you are just joining, Billy Cabrera, here we’re going to be starting in just a few moments. Want to make sure everyone has a chance to join and get this party started. Hey, Adam. How are you doing?

0:33 | Adam Creamer
Doing well. How about yourself, Billy?

0:36 | Billy Cabrera
Excited. I think we’re going to have a good turnout today across all the sessions. So I’m pumped on that.

0:42 | Adam Creamer
Absolutely. Yeah. So, sorry for being a couple seconds late. It’s what happens when you’re jumping from one session to the next at a live event. But, yeah, Billy, this is your time to show the audience some mobile testing tips and tricks. And if you guys have questions as he goes along, feel free to throw them in the chat or the Q a pane, and we’ll grab them as we go.

1:04 | Billy Cabrera
Awesome. Thank you, Adam, folks. If you haven’t met me or kobiton before, I want to make sure that everyone has a chance to meet the team behind the team. So shout out to Adam for getting this all together. On the marketing side. I am Billy Cabrera. I am your senior solutions engineer or sales engineer here at kobiton. My background is in the martech SaaS, space of the world where I also spend some time with accessibility, which is something that we’re going to talk about. It’s certainly near and dear to my heart. But today, this session is dedicated to making the most of your tech stack. With respect to mobile application testing. We’re going to cover some tips and tricks. I’m going to get into our platform to show you live what that solution can look like. Because if you’re here, you want to learn how to make the most of your efforts in mobile across your tech stack, you may be starting out, you may be a veteran of mobile application testing, but everyone’s a little different, right? You might have a marketing oriented app or maybe even offer support through your app, or you might make all of your revenue through your app like Uber, for example. But regardless, it’s incredibly important to ensure mobile success through testing from start to finish across all of your releases. And today, we really are going to get into some tips and tricks that’ll help you develop this mobile testing strategy. Now at kobiton, you’re going to hear me say this a little bit, but we like to refer to your journey in mobile testing with the mobile maturity model. And really where we’re going to start the conversation is step one, right? You obviously have ambitions to get your app out in the best way faster and with as many with as minimal amounts of issues or maybe performance issues. And again, you might already have your app out in the wild that’s great, right? No matter where you are, you’re here talking to us because you want to get better at this mobile topic. Now, since you might have an app or you’re building one, one of the first steps that we like to talk about is really finding a platform that’s not going to slow you down. Now, if you’re just starting out this mobile testing, maybe a little early on in your journey with mobile maturity, what you really want to do is come to one of these conclusions. You might have already come to them, right? Whether simulators or emulators, or a platform that provides that is the move for you or maybe you want real devices with real battery or other performance metrics baked into that, that’s huge, right? Because you want to be able to mirror what goals you have with the platform that can support that. There’s so many in this space that I know it might be a little bit overwhelming. So, one of the great ways that we talk about this growth or this journey at kobiton is really to find the scope of what you’re testing for. And that’s huge and it’s a huge issue as well or challenge because there are a lot of sort of pitfalls right there’s. So much device fragmentation. You have so many of these device vendors, all of them with different software that comes with the devices that might cause issues down the line. You don’t want those types of challenges. Obviously, that’s why we’re testing for them. And really again those KPIs, full functionality across multiple operating systems can help dictate what exactly you’re going to be testing for. So finding these KPIs, how do you define success? And really is that your developers or QA team’s responsibility, finding the platform and defining that scope. Are these first two major tips and tricks for better mobile testing. Now that’s often going to be for those earlier mobile testing, sort of frameworks that you might be still building out. Now, once you have those metrics and you have the platform that can support that testing, let’s talk about what that can look like. And this is where we’re going to use our home brand kobiton’s platform to be able to achieve these individual types of results. So I’m going to start sharing my screen here. And the first thing you’re going to see is a device farm. Now at kobiton we have real physical devices that allow us to be able to curate the types of devices. And the scope that represents your business’s needs, right? You know, one thing that we’ve found for example, in the latam region is that it’s all android, right? You want to be able to mirror the test scope to the types of devices that you want to be able to test on. Now, these device farms of course can shrink. They do change over time, right? You know, that the newest iPhone just came out, you might want to make sure that you’re staying on top of the latest and greatest. But again, that’s where scope really comes into play where maybe you have a lot of tickets in Jira from bugs that were found on older versions of android, for example, or maybe there are older devices that you actually have a lot of customers on. So again, that scope is going to be reflected in that device farm for you to be able to do that type of manual testing. So, let’s talk about manual testing today’s. Guinea pig is going to be my favorite little S 21 with a five G chip in it. We’re going to watch this bad boy. Now, remember when we’re doing manual testing, there are some creature comforts that you want to look for and that you want to be able to expect of the platform that you use. Now, I’m not going to go through every single possible test case that you could look for. But again, you want to be able to be comprehensive in your testing. Now that can range from maybe just simple touch or pinches and zoom gestures on these devices. But you want to be comprehensive. Again, that also means that you want to be a little bit more technical. You can start getting pretty fancy by maybe updating the device location. If you wanted to test this device manually to see how it behaves in different regions, maybe in a different country, for example, that’s a great way for you to be again comprehensive in your scope. You also might want to do some highfalutin things like biometric authentication, or maybe you simply just want to scan a QR Code. And these are the types of challenges that again, the breadth of devices that you’re going to run into should all be tested equally. And even if it’s just scanning a QR Code, you want a platform that supports that testing, maybe with something like image injection. Thinking about that again is going to further you along in that mobile maturity model that we talk about getting you closer to a really mature and well lubricated testing outfit. Let me take a quick pause here and check for questions in there, Adam, are there any questions that we need to address before moving forward?

9:26 | Adam Creamer
Nothing just yet that I’m seeing, but I’ll let you know once something comes through.

9:30 | Billy Cabrera
Awesome. Hey, Derek. Love you, man. Good to see you on here as well. Saw that one come through. So moving right along, we’re really in the nitty gritty of it, right? We know a little bit about the scope, we know about finding the platform that supports that testing scope. We talked about real physical devices, Guinea pig of ours. This S 21 is a real physical device. It resides just like I do in Atlanta and it physically exists within our data center that’s cool. Now, if you’ve gotten this far in your testing outfit, you know, that you might have some specific metrics that you’re using to define success. If we look to the right of our platform right off the bat, we are probably getting a lot of attention at the scroll here. And those are the actual logs or the exhaust pipe really of this S 21 that I have up running. Now that log readout is a great way for you to do a lot of investigation as to what’s working or what isn’t working with this specific test. You know, when we’re looking at success criteria or those specific KPIs, maybe your application writes to a log that lets you know of a specific function that succeeded. Or maybe you’re looking for a specific output, being able to find those, whether it is in the middle of this actual session, this manual session or later down the road is super important. The exact same way that maybe pulling in some of these metrics is also equally important, right? You want to make sure that your device, your application is also not choking your actual users’ phones. So being able to plug into the exhaust pipe that’s huge, you know, what we’re looking to achieve is again comprehensive breakdown of the success criteria in the platform. And what we’re looking at here is a collection of those device stats that you are able to get. Because again real life devices, simulators and emulators aren’t going to give you the breadth and the depth that you may want from a real physical device. So just a quick breakdown. We are obviously looking at memory cpu. We have network, we have actual battery drain and even temperatures. So here at kobiton, you might hear us say this a 1,000,000 times but we want to be able to recreate device in hand experiences because we’re able to give you those exhaust pipe type metrics. You’re absolutely able to see what’s going on and what’s going on under the hood specifically. Now, just for the folks that may not be familiar with kobiton. Obviously, I’m testing on a real device. I want to install an app. Very quickly, I’m just going to do this on the fly using our own homemade app called wingman. Part of the reason I’m doing this is just sort of represent what a manual test might look like and why this experience matters so much. And again, why your platform really needs to be able to bring the results that matter to your business here. We have wingman installed as a representative for this manual test. All we’re going to do is tap, get started, tap next. And then, you know what? I’m going to type in my phone number and that will complete the test. If this represents a good sort of manual test, you will know. Because now we’re going to talk a little bit more about how can we expand and what should we consider with these manual type tests because I’m going to talk about automating that next. But first we want to talk a little bit about this manual test. Some other criteria that we’re going to want to look at which is going to be held in kobiton within our session overview and our session explorer, where we get a breakdown of everything that happened in that manual test. We are able to check out a quick video on it. That’s great. We can see the specific apps that were installed, maybe something about the actual device that we were talking about or using a lot of the meat and the bones lives in our explorer. Now, our explorer is going to take every single gesture or test and align it to a timeline down here at the bottom where again we are looking for specific moments in that test as you remember, I tapped, get started on the actual app. That is not a real device. That is again really just the output of that manual test that I just created. You can see here this red dot is where I tested or where I tapped, or where the gesture was received on the actual app. And then very quickly we have accessibility built in. This is huge not only because I personally love accessibility but because it’s an important dimension of that success criteria. Now if you were lucky enough to actually get into one of the first sessions we had, I believe it was a keynote, Adam, we had the director of accessibility for Microsoft, that dude was absolutely brilliant in discussing not just the importance but the need to sort of shift left with accessibility testing and to make sure that that’s included as well in your grand scope. Again, that’s why your platform matters so much. We’re going to talk a little bit more about accessibility in just a moment. But what I like to talk about at this point is to make sure that you don’t forget it. This is something that you want to get in your sites as soon as possible. And again, letting that sort of success criteria or metric help dictate the type of platform that you’re going to be using to conduct that test. Now, kobiton again just get to brag on home team for just a minute. We do have the ability to be able to generate these types of results where that accessibility concern is fully considered. So this is where color contrast issues with the two colors that might be presented on a specific element are going to be able to be surfaced and presented very easily for your team whether it’s developers or the actual QA team touch target size. Again, you don’t want to come back, right? You don’t want to have a ticket dictate where accessibility fits into your testing criteria. You want to shift left and test early and test often and certainly have a platform that can support you doing that. I see that we’re almost out of time that’s OK, we still have a few more things to talk about and it’s the big elephant in the room about holy cow Billy, that was really cool. But it’s still very manual. How am I going to drive the maturity of my mobile testing? So that, yes, I am including accessibility, but how do I scale that out? If we’re thinking about the permutations of devices, scope that can dictate your testing? There’s a lot, right? If we’re just looking at iOS, maybe there’s folks using iOS and then we have all the newest ones. Now for each one, maybe there’s six of those. I am not a mathematician but I already know that the matrix grows with again the scope of the type of testing that you want to do especially with all these devices out there. So humans have leaned on automation. Now, again, this is so important for your team to be able to have a platform that supports that. Because for the folks that are further along in the maturity, you might already have something like appium… you might already be using something like appium to do your mobile testing. Appium’s awesome. We’re so familiar with it. But you also again want to be able to leverage a platform that doesn’t butt it out, right? That doesn’t compete for its attention, that more complements what you might be using there. And that’s where again kobiton is able to provide a platform for you to be able to do that without writing a lot of code because that is another teammate, right? And again, is that a QA teammate or is it a developer? Who’s running this automation? And kobiton is able to give yes for both answers? Because these steps that we saw earlier, right? I installed the app I tapped, get started next, and I even typed in my phone number, all of that. If that represents something that is the baseline test for all of your devices, that’s great. But scripting it or creating automation from it. Is not as my mom said as easily as blowing glass and making a cup out of it. It’s way harder. I don’t know why she said that she wasn’t a glass blower, but here we are. Kobiton does have the answer to that solution. And that brings us to one of the last things I want to talk about is driving automation throughout everything that we just saw. Everything that we just saw is presented here where each step is its own test step that we are now translating into an automated script. Now, there’s two ways to do it. There is super easy export functionality of an appium script that means that if install get started next is the core of what you want to test on other devices. It’s as simple as exporting that appium script. And again, the importance of a platform that plays well with your environment. And your tech stack comes to show its head again here where you obviously want to be able to use this script in the language or the framework that you are most familiar with, you are absolutely able to do that here. And if you are not an appium household, that’s still fine because we have the ability to rerun this test across any of the devices in our device farm. If you remember that grid showing all of our devices are now available here where you know, what that install get started next needs to get slapped across other devices. It’s as easy as selecting which devices you want to be able to test on. Now with one more click of the button down here, rerun, I have the ability to automate those steps without again writing a line of code. Now, when we’re thinking about how that fits in to your tech stack, there’s a few ways to be able to absorb it. Again, you’re going to hear a lot about kobiton in just more in one second. But again, let me take a quick pause here. I want to make sure we’re not leaving anybody behind Adam, anything from the chat or anything else.

22:03 | Adam Creamer
Yeah, there was one question that came in that you kind of just touched on. But I think it goes, you can go a little deeper. Is essentially, is kobiton based on appium?

22:13 | Billy Cabrera
That’s a great question. So I’m going to take just two seconds to talk about this. I didn’t mean to close that window. I’ll start sharing. Again when we’re talking about how kobiton is built and really what we’re able to provide when it comes to appium or that type of testing. We actually created our own appium outfit that we call exium. Now, exium is running the exact same script that you are seeing on the left with your standard appium outfit. And what you’re going to see is that exium is going to complete that test two to three times faster than appium. Now, to answer your question… exium is built off of the exact same selenium web selectors as appium. Exium really is just appium stripped of all that software bloat that might make it run slower. And as you can see here our exium, I’m sorry, the script running on our exium server just completed and spoiler alert, we still have maybe, 40 50 more seconds before appium completes that same script. So not only are you getting that functionality, that familiarity with appium, and again, the selenium web selectors and endpoints, you’re getting to be able to run it substantially faster and ostensibly a much easier time to resolution because you’re getting the results also much faster within our platform. I think we still have 20 more seconds on this clock before it actually finishes just to show you how much faster our exium, which again is really just appium stripped of all its bloat and baggage before even it finishes here. So hopefully that helped answer your question. We’re going to dive right back into. I see that we got just about five more minutes here. We’re going to dive back into sort of automation and really where kobiton as a platform can provide value there. So if you’re keeping track, we went from finding the scope, letting that help dictate what kind of devices you want to test on. We obviously answered the question of we want performance KPIs and that exhaust pipe data. So we want real devices? We want a platform that allows us to be able to test for the criteria that matches our KPIs and success measures. And then how can we do that at scale with automation? We’re able to export an appium script directly from kobiton here. You’re also able to rerun those steps in a part of our platform that we call scriptless that you just saw. And so far as test coverage, taking those steps, hitting all of your devices and the last thing I’m going to show you is a part of our platform that surfaces those results in a way that again, blends well with your tech stack because you might’ve noticed Jira tucked in all around our platform. We play well within azure devops, so on and so forth. But the results, right? Cool. We got testing at scale. We also want the results to be able to provide value to get you to your finish line and these releases much faster. And this is where I’m going to leave you. If anybody is familiar with script based or automation based testing. It is so brittle, anything can break it. And a lot of things, simple things do break it, namely, something like this false positive that we’re looking at here, right? We were looking for this paragraph, it was rendered on this galaxy S, 20 or 21 and we were hoping for that on that pixel six that I chose to test on. As you can see, this isn’t really an issue, right? It takes, kobiton has found that it takes anywhere from, 30 minutes to about an hour just for your QA engineers or your dev, your dev team to be able to even identify, and surface again a result of a failed condition. But it’s just again a false positive. This is where the platform matters. You want to be able to see what was originally rendered on the, right? The pixel six running android 12 is throwing a false positive. The platform matters here again, where we’re able to annotate these results directly from one platform where, where you might want to set a new baseline, you can skip this. You also have the ability to plug right into, your cicd pipeline. So, let’s throw out a ticket with all the information possible to those developers, if it’s their, if it’s their responsibility. Again, the platform that blends seamlessly in your, in your tech stack that’s the real pro tip folks, get the tool that, that not just achieves what you’re looking to do, but also helps you grow and mature within your mobile application testing framework. Whew, that was a lot. I get it. I’m here for emotional support and also, to answer any questions. I am not going anywhere. I’m, happy to hang out here as well. I know that we have, some more great sessions coming down the pipe for you, but, yeah, thank you for joining again. I’m Billy Cabrera, I am here with, mr, Adam Kramer, and, we’re here to be able to answer any questions regarding, mobile application testing, automating it, manual testing, real devices. You know, any, anything else that, we can certainly help the conversation, grow and educate. We’re, we’re here. So thank you so much, for that, Adam, anything you wanted to jump in for?

28:48 | Adam Creamer
No, just, thank you very much. If there are any questions that you guys have, feel free to come to our, the kobiton virtual booth or find Billy directly in the chat pane here. Yeah, I think there’s a lot, to think about when you look at scaling, automation and collaboration and all that fun stuff in mobile, and this was very helpful. Thank you, Billy.

29:08 | Billy Cabrera
Of course. Thank you for having me.

29:11 | Adam Creamer
All right, off to the next breakout sessions for everyone, we have a discussion with, bluebeam as well as one of our customers, weight watchers, so, pick one of those, and I will see you on the other side.

29:24 | Billy Cabrera
Oh, Gilbert, I saw you come in just at the last minute, with 48 more seconds to go, is there any possibility of testing a specific device model with a different OS? Absolutely, when we go about, sort of picking, what to test, and what exactly is, within scope, this is again, how easy kobiton is able to make that. If, again, I have my device farm with all the devices that I want. You could see the different OS versions in the center of the screen, if I wanted to, maybe test those steps on android, 10 devices. I can select those specifically. You could, organize that based off of, the type of device, the resolution, or a specific bundle. So, you absolutely have the ability, to test the specific device model with a different OS, the only gotcha on that one, Gilbert, if you’re still with me is, is that, you know, iOS tests for iOS, android tests for android. So that’s the only, that’s the only type testing limits. And that’s not really kobiton that’s just how we do it, all together. So if there aren’t any more questions, Gilbert, if you want, feel free to reach out to me, start a chat or maybe send me an email. I’m more than happy to have this conversation. So, thanks for everyone. Please stick around there’s. A lot of great stuff to talk about and to learn and, have a great day.