Empowering Quality: Why Quality is Everyone’s Responsibility in Mobile Testing

Video Transcript

0:00 | Nikita Patel
Hello, everyone. Thank you for joining me today. My name is Nikita and I’m a test engineer at Medtronic. I’m here to talk about how we can all work together to ensure quality in mobile testing. Quality in mobile testing is essential to ensure a positive user experience and successful mobile application. However, we all know that it comes with a lot of challenges and this responsibility shouldn’t just lie on the shoulders of a test engineer or a couple of developers. There should be a shared endeavor and everyone should be involved in this. What can you expect in today’s session?

0:44 | Nikita Patel
I’m going to take you a little bit on my journey how I started as a test engineer. I will talk a little bit about the importance of quality. I will also share some of the challenges that we come across as test engineers on mobile. I will talk about some missed quality and what does that look like in real life? I will also be talking about the main topic Which is quality being the shared effort or endeavor. And we’ll also look at some of the methodologies that I implemented at Medtronic which helped me get everyone on board and think about quality when we had big releases.

1:32 | Nikita Patel
This is about my journey. I started my journey as a validation tester in 2013 and I was working for a pharma company. So I was involved in testing systems which were used for drug trials. That means I had to be, really diligent. There was no room for any error, any mistakes. Any bugs could have a serious consequence on the overall outcome of the trial, right? And this was quite stressful. To be honest. And this attitude I carried to the next place that I went to, I, this pressure followed me at various roles that I played in different companies and it put, it led to a lot of anxiousness. It also had, it gave me a lot of self blame when something went wrong and it felt like a constant battle me down. However when I joined Medtronic, it was a little bit different. The concept of shared quality was introduced to me. However I was a little bit puzzled by that concept. I was wondering if quality was made. Everyone’s responsibility. Does this mean that my role is no longer valid? Is this meaning? Does this mean that I’m going to be made redundant or something in long run if everyone starts testing and everyone thinks about quality? Like what does this mean for… me? This is something that was going through my head but over a period of time, I realized how liberating and fulfilling this was, it allowed me to contribute to more than just testing. It allowed me to contribute to the overall process, right? From the very beginning all the way towards the end. Me and I’m here to share like how that could be achieved if you haven’t already implemented a quality, a shared concept of quality and how we can all start thinking about in our cross functional teams. But first let’s delve into the importance of quality.

3:52 | Nikita Patel
We all know when we achieve quality standards on mobile, that means it. And it unlocks plethora of benefits. There’s cascades of benefits like client satisfaction, there’s product reputation, long term success, where app store rating, we probably have good revenue coming in promising clients. All these are just few examples. You all know that there’s a lot more to this than just a few handful of benefits that are mentioned on the slide. With the benefits comes the challenges, right? While quality is paramount, ensuring mobile application presents ensuring quality on mobile application, we all know it comes with challenges. Let’s explore some of these challenges together.

4:53 | Nikita Patel
The road to mobile app quality is paved with hurdles. Device. There’s so many things that we need to think about right there’s. Device fragmentation, different operating system. We have to think about the risk factors that is involved and we have to have good quality risk matrix. And finally replicating real world scenarios or user scenarios and environment can be tricky. Let’s turn our attention to potential consequences of missing the quality mark on mobile. Well, there are a lot of challenges. There are a lot of things that could happen when we miss quality. I have just listed a few of them. We could potentially lost customers. We could get bad reputation, tarnished reputation.

5:51 | Nikita Patel
We could lose a lot of money. We could fail to meet user’s requirement And there could be some serious consequences of that, right? So let’s look at some of these consequences and bugs that made to real world and how that affected the users. So I’ve just illustrated some examples of missed quality in real world mobile app scenarios. The first one being Omnipod V, which is an automated system which is used by diabetic patients to automatically receive insulin. And their system had a bug which failed to recognize the decimal point when entering without. It had a leading zero. So that means the insulin was getting administered into patients 10 times more, 100 times more than the intended amount. And FDA had to step in and recall that version. Another example I have given is about breach of privacy. Babylon Health had this serious bug where Babylon Health is now acquired by E Med, but it does private consultation and therapy and GP. And all that. So With this bug, patient A was seeing medical record and consultation notes of patient B. Now imagine the consequence of that, like how would a user or patient feel They would probably feel? Absolutely wonderful, right? Another bug is about the banking bug which is called Brockwell Banking Bug. It was a malware bug which stole users’ credentials and that led to a loss of a lot of money. So these are just a handful of examples that I have mentioned here that could negatively impact the user experience. And in worst case scenarios, it could also lead to safety hazards. Now that we’ve discussed the significance of quality, let’s transition into our main topic, how we can cultivate a culture of shared responsibility and how can we achieve that in our organization?

8:45 | Nikita Patel
Every stage of software development from conception to delivery contributes to quality and breakdown at any step or any level can negatively impact the outcome. Fostering a shared responsibility culture can ensure consistent quality. I hope we all have an understanding that quality is not a singular responsibility and it’s a shared effort across the entire organization. Let’s compare what a role of tester looks like in a traditional environment and when quality is made, as everyone’s responsibility. To the left is the role of testers in traditional environment. To the right is the quality with the whole team approach. When quality is in traditional view, quality engineers are looked as solely responsible for quality. Whereas with the whole team approach, everyone thinks about quality as being their responsibility. In the traditional view, test engineers would probably be seen as gatekeepers which could lead to potentially a blame culture sort of environment. With the whole team approach. All team members actively participate in ensuring quality. With the traditional view. Another point that I’ve made here is testing is often seen as a separate phase at the end of development lifecycle. And testers are often relegated towards the very end. With the whole team approach. Testing is an integrated phase, integrated process which happens throughout the whole development starting from very early stages. And a tester… Testers are focused more on meeting users requirement and needs rather than just adhering to specification. And with the whole team approach testers are also involved from the very beginning until the end. In a lot of teams where you and I have worked in the past or you’re currently working testers are only involved when a ticket is moved to in progress column or is put to a ready for test column, right? And a lot of people don’t have that sort of involvement or say in the process. And don’t have that don’t have that collaboration with the whole team?

11:48 | Nikita Patel
With the traditional view testers, as I already mentioned that don’t have a lot of direct involvement in the early stage of product development, which leads to side approach. But with the whole team approach testers are involved all the way from the start to the end. And the testing doesn’t stop At a stop or start at in progress. And until done, we think about testing by shifting left by shifting right by shifting literally everywhere. And this is commonly called as continuous testing. We’re also doing cross functional collaboration and We break down silos, basically when we work in this whole team approach model.

12:48 | Nikita Patel
Let’s look at how individual contributors in our cross functional teams can start thinking about how they can contribute to this whole team approach and think about quality from their roles. First and foremost, product managers and owners, Product managers and owners can start thinking about good quality requirements. They can start prioritizing and maintaining quality. They can also set and align quality goals with the test engineer and the rest of the team And ensure there’s good collaboration and functional communication happening amongst the team. And if, for whatever reason a developer is not involved, get them involved. If the tester is not involved in the communication, make sure that they think about all of these communication at every step of development. The second one is our developers, They can start thinking about quality by writing firstly, good quality code. I’m sure they already write good quality code, but then they can always think about, you know, testability, right? If they’re writing good quality code. But if the code that is written is not automatable, then there’s no point, right? So think about testability, think about maintainability of that code. Think about pair programming, Think about a peer review on their PRs by the developers, by the testers, And, yeah… and also think about having a lot more unit test coverage.

14:44 | Nikita Patel
When it comes to designers, they have a huge impact on design decisions on user experience and design and the quality. So make sure that your designers know that they’re thinking about the product and designers are, have Collaborated come to a common understanding of what the user’s requirements are. The same knowledge is shared with developers, is shared with test engineers. I have noticed that in a lot of my previous role that I wasn’t involved in the designing process, I wasn’t involved in a lot of design related discussions. And when everything was implemented and the ticket was in progress or it was… in ready for test column, I would look at the designs, and I realized that some of the states were not Hubbard. We didn’t have designs for like what happens when internet connection is lost? What happens when a message that we’re trying to send gets failed? So the, error states was something that didn’t think that the developers didn’t think about or the designers didn’t think about. And we as test engineers, we’re so wired to think about like happy path and also the sad path, right? So it becomes really important for us to be involved from early on so that there’s no delay If we find out about these things or at a state. This is just one example… that I have I’m sharing. But if we were, if I was involved early on, I would have spoken about that could have been mitigated. We would have had it implemented and we wouldn’t have wasted a little bit of time that we did closer to the release. We, as testers, we play obviously a vital role. And this does not mean when I say quality being everyone’s responsibility. This does not mean we’re replacing our independent testing because we’re really good at it. And we bring that expertise in our team right Along with that. We should start thinking about things beyond bug finding… rather than.

17:16 | Nikita Patel
Rather than just thinking about functional testing or doing acceptance testing, how can we all contribute a little bit more to the overall process by shifting left by shifting right? By doing continuous testing In order to, In order to let’s also understand what is needed to foster a quality driven workforce? We probably need like good and clear measurable and quality, good quality standards. We would need open communication channels where everyone is open And feeling comfortable to talk about important and crucial things. We have to ensure that everyone’s feeling empowered to think about quality. And that could be done by running a lot of workshops like definition of done, what it means like to pick up a ticket, What is readiness in our team looks like When a developer picks up a ticket. So these are things that we can start having conversation with our wider team and get more feedback early on.

18:42 | Nikita Patel
I touched base on continuous testing in the previous slide. And this slide captures the essential sense of continuous testing that can be done by everyone in our team. This is a slide. This is a model that I have picked up from Diane Ashbaugh’s link With the continuous testing. We have a continuous testing offers several advantages. First is early identification and resolution of bugs, improved quality. And we have a lot faster feedback loop. So testing shouldn’t just be done at a ticket level. We can start thinking about testing at every step of development by shifting left by shifting right and continuously testing throughout the whole process.

19:39 | Nikita Patel
Shift left. I know, it’s a word that’s been used so much lately and.

19:53 | Nikita Patel
Every developmental effort involves numerous unknowns that surface as developmental progresses And team learned new fact. If this learning happens much later in the process, the underlying issues will significantly impact the solution and significant rework and delay will result. However if learning takes place much earlier or is shifted left, the problem reveals themselves sooner, enabling corrective action with minimum impact. This earlier focus on quality offers several advantages. It definitely reduces rework, It definitely by catching bugs early on, we can fix them before they become more deeply embedded in the code which saves times and resources. It helps us improve the product quality by proactively addressing potential issues Early on. We can deliver higher quality product to our users, faster time to market. Shifting left can help us identify and fix issues sooner which can accelerate the delivery of our product to the market.

21:23 | Nikita Patel
Let’s see some of the testing methodologies in action. This is something that I experienced with the whole quality being everyone’s responsibility. I was like, how can i involve everyone, when we were doing some major releases? First and foremost was rebranding, we got it by our company was called digital surgery, which got acquired by metronic and everything. So that means everything had to be rebranded in our surgical training app. And So all the screen text font, everything had to be re rendered with the Mitra brand.

22:08 | Nikita Patel
In material branding guideline, and i looked at the, as a mobile test engineer, i looked at the automation and what was the coverage like? And i realized that we didn’t have all the fonts covered, like all the font style covered all the screen covered to ensure that everything were, if i were to run automated test and i would be confident with that. Just with that run, that means i had to do it manually. Now doing it manually on and testing all the screens, all the text, all the font, all the color was sounded a bit too much and doing it on both android and android, ios was a lot of effort. So i got involved everybody in our engineering department and i did an in house testing more like a crowd test. But in house and i in, i blocked one hour in people’s calendar. Everyone had voluntary participation. There was a little bit of a reward. The amount of feedback that i got in just one hour was insane. It would have taken me at least two to two weeks a sprint. That is how long our sprints are for me to get the same amount of feedback that i received in an hour. So that was a good quality of a good example of how we can all think about quality as everyone’s responsibility. Now, now we have aptly tool implementation. So this is slightly different. Now if we were to rebrand again, it wouldn’t have, i wouldn’t have to go through the whole… crowd testing or enhanced testing. But, i highly recommend it if you have a major release upcoming, then think about this approach, get everyone on board, get a lot of feedback and save a lot of your time. I did the same thing for translations. Translation is like we decided as metronix supports the globalization and localization. So we had to support our app in 16 different languages. Now, again testing it, testing our app in 16 different languages on ios and android, as a mobile test engineer was a lot of work. I did have help from other test engineers, but still it was a lot of work. And i was, i, we had some automated tests but not enough to have entire coverage. And yeah, think about missing translations and all of that. So we again, it… implemented this approach, got everyone on board blocked their calendar for an, hour and a bit and ran some crowd test in our staff gave them instruction what to look for, what is in scope, what is out of scope? This also helped a lot of people to understand how testing works? How can they start creating bugs in our system? If they at all come across the bug rather than just writing on a channel saying, hey, this is not working, right? So, it leveraged a lot of employees, to use our app and also gain a lot of knowledge and insight about different scale and features which they not necessarily, but which they wouldn’t necessarily go out of their way and think about those features and wouldn’t even know if they existed, right? If you are someone from the commercial team, if you’re someone from finance team, you wouldn’t necessarily know… how the app works. And this just gave a lot of insight. And it was so useful to get this feedback from everyone and have everyone involved. I did the same thing about, with live stream. I’m going to talk a little bit more about live stream in the next slide. So, live stream is something that has been generating buzz in the medtech industry, and we have it set to release by way soon, but we had our pilot release a little while ago as a mobile test engineer. My focus was just on a mobile, however, there’s a lot of integration. There’s a lot of communication… happening across platforms and different devices. So, I just realized that there’s a lot of things, that could be missed. And how can we, how can we mitigate that? Right? I, there’s a risk involved Me doing testing in silos, is not going to mitigate any of those risks that, we probably don’t even know about a lot of unknowns, Right? So, I got a lot of, other team, other testers involved and, we applied, ensemble testing where, where we switched roles as a driver… navigators, looked at different, clients. This just helped us, explore a lot, on different clients. It also made us think about what, feature is implemented, where, and how that is benefiting either mobile, either web, either, you know, platform, how everything is working. And, also, it uncovered like numerous unknowns, some of the missed requirements, some of the missed scenarios, We found a lot of bugs and, yeah, it just gave us a lot more confidence, in our testing, using the ensemble testing methodology. It, yeah, we uncovered a lot.

28:14 | Nikita Patel
Let’s look at some of the best practices that we can follow to foster this culture of quality. We can, think about promoting cross cultural, cross functional collaboration. We can, we should start emphasizing on, early testing and, feedback… Implementation of, Agile and DevOps principles, Invest in, automation and tooling, encouraging our team, with the continuous learning and improvement… Prioritizing user centric design and feedback… Establishing clear communication channels.

29:07 | Nikita Patel
This is my favorite slide. We’ll look at how a tester can transition and, how can they think about being a solo tester to a whole team approach? So, if I was a solo tester, I would probably think that quality assurance is just, testing is just my responsibility. But if I was working as a tester, if I was working in, an environment where quality is a shared endeavor, then I would think that quality is everyone’s responsibility and, fostering a collaborative mindset. How would I feel as a tester? If I was working As a solo tester? I probably feel isolated, overwhelmed and burdened to ensure like… everything is working as expected before a release… However with a shared quality, I would probably feel a lot more empowered. I would feel supported by the team working together, delivering high quality product In our team. If I, we don’t get to state where there’s like tested bottleneck, where there’s you know, bottleneck, of tickets which are blocked because of testing. If something that, we collaborate, we come together, we have strategies in place which, doesn’t let that happen. And everyone thinks about quality. If there’s something that, could be tested by, somebody, and if it’s just a string change and somebody else can test it, they, we have that discussion and everyone feels empowered to pick that ticket up and test it. However, for critical stuff. It’s nice for test engineers to be involved and, and bring the expertise on the table. What would we do as test engineers? We would just be focusing on our individual testing responsibility… As, as in the whole team approach, we’d probably be collaborating with the designer developer, product owner. We’d be conducting workshops and, constantly being quality advocate, constantly being quality coach and telling people how to think about good quality documents, good quality, code, good quality requirement, all of these things… In conclusion… continuous testing over testing towards the end, embracing all testing activities over automating everything, testing, what gives us value based on customer usage rather than testing everything. I always tell people, don’t be afraid to look at the data, look at the usage, how your features are being used by our customers, follow their journey, see what they do and how often do they use and visit that page on mobile, visit that screen on mobile, A whole team approach to testing over testing in silo, department, product coverage, over code coverage. So that’s it from me. And if you have any questions I am happy to answer.

33:06 | Cara Suarez
Thank you.

33:06 | Cara Suarez
That was a fantastic presentation. We actually already have quite a few questions that have come in through the chat. So I’ll go ahead and get started with that. One of the earlier ones that came in during your presentation was that you mentioned that quality engineering is seen as everyone’s responsibility at Medtronic. How is this philosophy implemented within your team?

33:36 | Nikita Patel
Thank you. Cara. Quality is literally quality being shared. Responsibility is embedded in our body and system And everyone thinks about quality being their responsibility. We have this collaborative culture where everyone feels empowered and developers will think about like quality code testers will think about quality test writing them executing them. We also get a lot of feedback. We ask a lot of feedback because I think feedback is so important. Otherwise we get complacent when we don’t have feedback. So all of these things I think just helps us improve continuously and continuously grow.

34:20 | Cara Suarez
That’s really interesting. A follow up to that is how do you measure the success of your testing efforts?

34:33 | Nikita Patel
Testing success can be measured. I think through a lot of different matrix. We can look at number of bugs in production. We can think about the bugs raised versus the bugs fixed. We can also get like client feedbacks or customer feedback. We can look at test coverage, our own test coverage. We have excellent integration In Jira with X Ray, which gives us dashboards. And that is something that we use to demonstrate it to our stakeholders, which helps us showcase like what have we achieved? What is our coverage looking like? So in case where we have good coverage, all passes all pass tests. That is somehow I would measure quality along with the other matrix.

35:22 | Cara Suarez
Okay, great answer From Jonathan Rivera earlier, you were kind of showing a slide where you had a lot of different stakeholders represented, And that’s when this question came in, It said, how can you organize the communication standpoint when dealing with multiplied tasks?

35:43 | Nikita Patel
I think I would, if it’s a product related task, I would probably have a communication channel in place where people feel either it could be general channel or a team specific channel where people feel comfortable to ask these questions. And we’re not working in silos, or we’re not having this conversation at individual level, and everyone in the team is aware what is happening. And so, I think I would encourage having these conversation at a team level in a communication channel where it’s important where everyone feels safe and everyone feels that they have voice and they feel empowered to speak about anything that they want to speak.

36:26 | Cara Suarez
Great. The next one is kind of like a two part question. So again, it’s a follow up from Jonathan Rivera. It says what’s the best way to explain about a bug? And then the second part of that is should a test, I think a tester take a screenshot and give a small detail about how he or she received the bug in the system?

36:49 | Nikita Patel
I think it depends on like who are you trying to explain this bug to, right? If you’re trying to explain this bug to a developer for an ongoing test that you are, for an ongoing testing that you’re executing at a story level, Then it could be just a recording and a little bit of details. But if it’s something that you have found in production, then I guess it requires a lot more in depth and you can probably attach if you have Tableau or if you have Firebase, if you have Embrace, then you can attach all of these logs and give a little bit more details to this so that people know where exactly things have gone wrong and do a little bit debugging before we take it to the developer or product owner. Does that answer your question?

37:43 | Cara Suarez
I think so, Jonathan, if you’re still out there, let us know if that answers your question, You could type it in back in the QA, or if you have a follow up, we welcome that too, And just a little housekeeping before we get to some additional questions For those of you in the audience. Remember the Q, A tab is where you can type in any questions that you have, and we’ll answer them. Now. Okay. We have an additional question from Adam Creamer. It says, how do you balance the need for thorough testing with the requirement for timely product releases as more teams take the responsibility for quality?

38:19 | Nikita Patel
That’s such a good question. And we have to ask that question to ourselves, right? Like what is thorough testing? And when is it enough? It can be, I think achieved by careful planning, We can think about implementing a risk based approach. We can obviously collaborate. I cannot stress enough, but we need to collaborate with our cross functional team and get their feedback, get our priorities in order and Also follow principles of Agile to break down each of these tasks into releasable items and stories. So I think when we achieve this and we’re able to release it and test it in time. I think that that’s thorough testing for me. That’s great. Okay.

39:11 | Cara Suarez
We’ll wait to see if there’s another minute or two. If there’s oh, yep. We got some more questions coming in. Another one just came in. How frequently do you release a code? And do you perform any beta testing to get feedback from users?

39:27 | Nikita Patel
We ship our code every two weeks at the end of the sprint. So we have a release at the end of every two weeks and we don’t do beta testing, but we do have user experience testing where we get like user feedback. We, we Test it with our app is like a surgical training app. So we get different doctors, different nurses to use our app and see what they, what kind of feedback we get. Like sometimes as I said, like Feedback is so important, We may develop something thinking that this is going to work fine, and everyone’s going to be amazed by this. But when a real user, when our end users are not able to use the product, then all our efforts is going to go down the drain.

40:19 | Cara Suarez
Jonathan did respond back while you were answering Sigar’s question, And he said, thanks Nikita. That helps a lot with the multiple hats that I’m currently doing So good. I’m glad Jonathan, we’re able to help. Oh good. We have another question that came in. It says, I always envy people in the healthcare domain based on the impact they can make in the community. Kudos. There. Nikita Question to you is, how do you go by test data setup when it comes to medical devices? How do you automate tests as part of a prerequisite before text execution Data is critical for your domain? That’s kind of a big question? Yeah.

41:03 | Nikita Patel
Yeah, that’s a big question. Luckily, our product doesn’t fall under medical devices. So I think I can dodge that question, But I think I would probably get an understanding by speaking to other people who know what kind of real data it looks like in production? And based on that, I would create my data sets and prerequisites based on that, if that makes sense?

41:34 | Cara Suarez
Yeah, no, I think that makes great sense. Thanks. Yep. They said it does make sense. Good.

41:42 | Nikita Patel
Okay.

41:47 | Cara Suarez
I know you’re probably all typing very quickly. So we’ll give it another moment or two.

41:55 | Nikita Patel
You?

41:55 | Cara Suarez
Okay… Good. We have some good dialogue in the QA for anyone wanting to watch.

42:11 | Nikita Patel
Thank you.

42:13 | Cara Suarez
Okay. Well, with that, oh good Questions still coming in. Three more, few more came in. Okay. Here we go. So first from Sunita, what are some examples of successful implementations of shared quality responsibility and mobile testing?

42:31 | Nikita Patel
I think I listed a few of them on my slide. And when I joined even, I didn’t know what the concept of shared quality was. When I got to know a little bit more about shared quality. I was like this is so nice. And the first thing when I was new, I’ll give you a little bit of example of how it felt liberating. When i joined. I, i wasn’t aware about the product completely and i missed something and that bug was slipped to production and a senior developer found that bug and he said that, you know, we need to do a fat patch fix and i apologized in a general channel which everyone’s like, no, no, it’s not your fault. Like it’s not your fault. It’s everyone’s fault. Everyone’s responsible for quality. And, you know, we should all have tested this and that was really liberating. So having that mindset in our team will help us grow as an individual. We wouldn’t just think about testing. You think about a lot more than just, you know, writing your, except your test scripts and executing them. You think about process improvement. You think about you, you’ll also have time to look at the data. You’ll also follow the trend. You will think about like how can we fault apply? Like some process automation? We can think about… what is automatable, what is not automatable? So all of these things, it’ll just give you a lot of time. And i think i may have deviated from your original question, but what was the original question?

44:10 | Cara Suarez
It was examples of successful implementations of shared quality responsibility in mobile testing.

44:18 | Nikita Patel
So a way this was just from a tester’s point of view, how it, how amazing it would feel as a test engineer. But developers can start think about writing good quality codes, product owners can think about good quality requirements. We can all meet together, do some ensemble testing, look at the same things. So these are just some of the examples that i can think about right now and share with you. But if you want more insight, i am happy to answer your question. If you email it to me.

44:54 | Cara Suarez
Yeah. And that’s all the time that we have for today. There’s still a few more, but we will send you some emails. Okay? Great.

45:02 | Nikita Patel
Thank you so much for joining all the questions.