Ensuring Quality in DevOps Culture

Video Transcript

0:00 | Sagar Mahendrakar
Welcome to Mobile Testing Summit. So my name is Sagar Mahendakkar and I am from India. I hold a bachelor’s degree and currently working with Red Hat. I have almost five years of experience in a quality engineer role. Mostly, I am responsible for a few tasks of a project involvement in sprint planning and releases. Also I automate test cases, integrate it with a CI CD pipeline and to make sure That our product is bug free. So today, I will share, I would like to share one of my experience. Like when I was working with previous organization, we used to follow waterfall methodology. And because of this, our development cycle was too long And we used to face some challenges like difficulties in accommodating change request because customer requirements keep on changing with time And it was difficult to accommodate any change request after the requirement specification phase was complete. Also in the waterfall model testing is typically done at the end of the development cycle. This means that a defect may not be discovered until late in the development process which can be expensive and time consuming to fix. But when I joined Red Hat where agile methodology was used, I got to know why agile is important and why agile is important for development and testing purposes And what makes it successful is DevOps. So let’s start with the today’s topic, ensuring quality in DevOps culture. So moving to agenda for the today’s call, Yeah… Let’s walk through from the agenda for the today’s session. So we will start with the introduction for DevOps and their importance. Then we will learn the relationship between QV and DevOps. Moving further, we will discuss the role of QV in DevOps and their responsibilities in brief. Then we can discuss more about challenges that QV face while working with DevOps. And we will have some deep knowledge about toolchain integration and their importance. At last, we will discuss more about skills required for QA to work with DevOps. And then we can have a conclusion and a QA session. Yep. So moving to the first slide of the presentation. Yep… Here it is. So what is DevOps? Devops combines development and operations to increase the efficiency, speed and security of software development and delivery. Compared to traditional processes. It allows organizations to serve customers better and compete more strongly in the market. In simple words. We can say, DevOps can be defined as alignment or development and IT operations with better communication and collaboration. There are a few reasons that why DevOps is necessary for modern businesses. So I have listed a few of them here. Like short development cycles, early error detection reduces operational cost, improves customer satisfaction. So we will start with the first one, short development cycles. So, DevOps allows developers to release their product much quicker because they no longer have to wait for approval from other teams. Instead, they can release new version as soon as they are ready for testing, which means that bugs can be found, bugs can be caught much earlier in the process and fixed before significant issue. Second is early error detection When teams work together on a project. When product, when teams work together on projects, There is more opportunity for errors than a team working on the same project but separately. So in DevOps environment, when one team makes an error in coding or testing, another team can detect it immediately and fix it before it causes any damage introduction. It leads to higher quality of product and fewer… problems down the road. Third one is reduces operational cost. So it’s hard for businesses to keep up with the pace of technology today. If you are not keeping up, you might as well close up shop because your competitors will leave you in the dust. One of the biggest reason why DevOps is important, it’s because it helps companies lower operational cost errors by increasing efficiency, which means they can spend less on servers, support staff and other resources while maintaining their competitive edge. So, last one, here, it is improved customer satisfaction. One of the important aspect of any business is keeping your customers happy errors. There is no better way to do this than by providing excellent customer service… which can be difficult for some companies. Devops can help make this easier for you and your team. By integrating your development team with your operations, Teams can create a seamless experience for both customers and external employees. It will increase customer satisfaction because they don’t have to wait long for their request to be answered. It will also keep things running smooth within your organization because everyone knows what they need to do next. And hope we got the basic and the importance of DevOps here. Now, we can move to the next slide of our presentation. Yeah, here it is. So here, we will discuss a relationship between QA, QE and DevOps. And where I exactly, QA feeds into the world of continuous integration dominated by DevOps. Well, pretty much in every release, a quality analyst is required to assure the robustness of the product. Especially in the world of DevOps, where the updates take place in the application almost on a weekly basis With so many frequent changes taking place. Outages look to be very common. But outages are supposed to break the life of the developers and the managers. Everyone needs a quick hotfix when the service of the product goes south, affecting more than the single customer. But an immediate fix needs to be thoroughly tested before it gets migrated into production Developers and product managers would want the job to be done in a van. This is where QA needs to set up and takes the responsibility of evaluating whether that immediate fix for one outage does not end up affecting any other functionality of the application. So just as much as developer is needed to get rid of the outage by a quick fix, a quality analyst is needed to prevent future outages to as many extent as possible Moving to next slide challenges faced by QV teams in DevOps. Several challenges may arise due to fundamental changes in workflow, collaboration and technology use such as integration Quality engineering in DevOps environment is not just about ensuring the quality… of the software but also about enhancing the processes that lead to the development of a high quality software product. Continuously Here, I will discuss some of the main challenges as Main challenges a QV might face while working with DevOps. So I have listed a few of them such as shift lift, speed versus quality, trade off, automated testing, expertise, keeping up the changes. So we will walk through with the first one shift lift and continuous testing. Devops emphasizes shifting late a lift, which means testing early and often in the development cycle. This requires the QV team to integrate testing seamlessly into the daily workflow of development, which can be a significant change from traditional models where testing was a distinct phase. Qv teams must… adapt by developing automated tests that can run effectively in every stage of the CI CD pipeline. Second one speed versus quality trade off DevOps prioritize speed and efficiency in delivering updates and new features. Qv teams are challenged to maintain a high standard of quality without slowing down the development and deployment process. This can sometimes create tension as the drive for rapid releases may compromise the thoroughness of automated testing expertise With the emphasis on automation in DevOps. Qv team must possess strong skills in automated testing. This includes the ability to write, maintain and understand automated testing scripts and frameworks, Developing the expertise requires time and… training, which can be challenging especially in the fast paced environments, Keeping up with changes In a DevOps culture. Changes occur rapidly and frequently, QV team must keep up with these changes in both the software being developed and the technologies used to develop and test it. This continuous learning and adoption can be demanding Moving to next slide… Yeah Challenges for QV in mobile DevOps. So several challenges QV face in mobile DevOps. So I listed a few of them Diverse device landscape, platform, fragmentation, specialized testing approach, variability in network condition, app store compliance. So mobile devices are used in environment with various… network conditions, Testing how an application performs across different networks like a three G5 G, four G and Wi Fi and how it handles network transaction. Like switching from LTE to Wi Fi or experiencing a dropped connection is critical for QV, but hard to simulate accurately performance benchmarking, Ensuring the app application performs well under various conditions like in the background during high CPU usage or when the battery is low Performance, benchmarking in a real world scenario is a significant QV challenge… Moving to next slide And yeah Roles and responsibilities in QV or QV in DevOps. The quality engineering role is more demanding than QA watch as it is more fundamentally focused on quality. The QA role demands that you have expertise in all aspects of DevOps which can be directly or indirectly related to product quality, including software testing, automation, testing, agile processes, CICD, etc, Toolchain integration. So, what is toolchain integration? A toolchain is a collection of tools and technologies that enables development and operations teams to collaborate throughout the entire development process, addressing key DevOps fundamentals such as continuous integration, continuous delivery automation and collaboration, Toolchain integration in DevOps and quality engineering is crucial for achieving the seamless automation and collaboration required to support rapid reliable software releases. A well integrated toolchain allows for continuous deployment testing and deployment, which are the cornerstones of the DevOps philosophy. Here, a detailed look at the importance of a toolchain and we can go in the brief of toolchain integration. I listed a few of the characteristics of toolchain integration like streamlined workflow, automation, enhanced collaboration, consistency and standardization and real time feedback. So here we go with the first one streamlined workflow, automation, Integration of tools in the DevOps environment, helps in creating a seamless workflow for developers, operations teams, and the quality engineers. Each stage of the software deployment lifecycle from planning… and coding to testing deployments, and monitoring can be automated. This automation can reduce the manual hands off between teams and minimize errors and speed up the process of the software delivery. Enhanced collaboration Toolchain integration facilitates better communication and collaboration among cross functional teams, Tools such as chatbot, integrated with issue tracking system, can automate status update and alerts. This keeps everyone in the loop and make it easier for team members to quickly to development or operational issues, and to ensure that everyone works from a unified platform with access to real time data. Here we go with the one consistency and standardization… Integrated tools help standardizing process across the development testing and the deployment phases. This standardization is crucial for maintaining, consistency in how software is developed tested and released, which in turn helps in managing complex systems and reducing unexpected behavior or discrepancy in a production environment, Real time feedback, Integrated tools provide continuous feedback and detailed metrics about every aspect. Of the software development lifecycle, For instance, dashboard tools that integrate with CI, servers testing frameworks, and deployment tools that integrate with CI, servers… testing frameworks and the deployment. This data is vital for making informed decisions about the product and process improvement. For quality engineering, integration of tools like source control, build, servers, deployment mechanism and testing frameworks, means they can ensure comprehensive, quality check. This integrated approach helps QV team to implement advanced testing strategies such as shift lift testing, performance testing, and security testing as a part of a deployment pipeline rather than the afterthoughts like difference between web DevOps and the mobile DevOps. So, yeah, so how it differs like we have a mobile application must be tested and optimized for the broader range of devices and operating systems. Unlike web applications that can be updated in real time, mobile applications require passing through app store reviews which introduces delays and additional planning for rollouts. Mobile development often requires native tools and environment which can complicate the CI CD pipeline Emulators and simulators are used. But testing on real devices is critical for ensuring the real world usability, which is less of a concern in web development. So, this was about the differences between the web DevOps and the mobile DevOps. So now moving to the next slide… And time to get to know the skills required for testers in DevOps. Yeah. So some important skills required like understanding software testing will enable you to spot and resolve software problems. Quickly, Knowledge of the DevOps technologies will enable you to organize and automate the software deployment and testing processes. One should have collaboration abilities with developers and other team members, Understanding how to automate testing and deploy code updates regularly is important Knowing how effectively CI CD pipeline works and it works is better. So this was the skills required for QA to work with the DevOps. But I know there are much skills… you might know, But here I have pointed some of the skills and it is a major skill. Yeah. So moving to the conclusion of this presentation and conclusion of the, and the evolution of the QA, The impact of DevOps on quality assurance testing strategies is transformative By embracing a culture of collaboration, automation and continuous testing organization can achieve the delicate balance between speed and quality in software development, QA teams play a vital role in the paradigm shift, contributing to the creation of a resilient reliable and high quality software in the dynamic DevOps environment. As the organization continues to evolve, the integration of DevOps principles into QA testing… technologies, will be fundamental to staying competitive in the ever changing landscape of technology. So what I think like QA should not limit to only for manual testing or automation. Testing, but they can grow by collaborating with different teams like DevOps team developers team. They have to make the communication strong and they need to suggest the improvements in the processes. By this. The will continue. There will be no separate teams like development team, QA team and the DevOps team. But we have to work as a whole team. And although just, we have to make sure our product is bug free. And by saying this thing, I would like to stop here And I hope you like the presentation and this will be a productive in your day to day life. And yep, thank you.

21:05 | Cara Suarez
Okay. Great. Well, excellent presentation, Sagar. We got a lot of questions and a lot of engagement from the audience. And I’m glad we have enough time to hopefully answer them. So let’s start. Yeah, let’s start with Brittany. She says mobile apps face strict rules from app stores. So how do you adapt your mobile QA and DevOps process to meet those challenges?

21:32 | Sagar Mahendrakar
Yes, Cara. So basically, app stores have different guidelines based on the operating system, General processes like dev uploads application on a store. Then there is automated scans to check malware and the functionality issues. Then there is a manual scan to check usability issues. And then developers get the status like app is accepted or rejected and the feedback To avoid these challenges. In an app store, we should follow app guidelines from the… of the product development Regularly. We can integrate a test built in CI CD pipelines to address any issues, implement a beta testings, to get feedback, from the group of the users and ensure that your application follows best practices for security including data encryption, security authentication mechanism, and protection against common vulnerabilities such as sql injection and cross site script.

22:38 | Cara Suarez
Wow. Yeah. You know, in your answer, you mentioned a little bit about beta testing for example, as a practice, i’d love to hear a little bit more about, you know, your experience there and maybe some learnings that you have at, from Red Hat.

22:55 | Sagar Mahendrakar
Yeah. So surely, so while working with the previous organization, we were testing some mobile application and we face many challenges like while testing a mobile application, we have to test it on our different platform with different networks. And we have to, we also face some challenges while testing mobile devices. Like there, we have to test mobile application when there is a connection lost or a network switch from lta to wi fi. So, we started utilizing cross platform testing frameworks like a fpm and espresso, which allows us to run our script on multiple operating systems. Also, we started manual testing on our real devices to test functionality and usability testings. And also, we use some simulators and emulators to reproduce the functionality on other devices as well.

23:55 | Cara Suarez
Yeah. I think that’s really interesting trying to tackle those challenges of like testing across diverse mobile platforms. I mean, you know, obviously real devices is, you know, we think at kobiton, we think that’s really an essential part of the process. So, i’m glad to hear, you know, that that’s a practice that you guys are relying on as well. We have another question here, ashkansa, i’m sorry, i think i said your name wrong, apologies it says, how do you handle frequent updates during mobile testing?

24:28 | Sagar Mahendrakar
Yeah. So answering to that question, like after each update, we focus on regression things to check that new code changes have not caused any harm to the existing functionality. We track all the changes in logs and communicate closely with the development team to understand the implication of each update is followed.

24:48 | Cara Suarez
Interesting. Okay. I see another one that’s coming in, it says what new trends or technologies do you think will reshape mobile qa practices in the near future?

25:05 | Sagar Mahendrakar
Yeah. So answering to that, we are hearing many things from related to ai and machine learning, ai and machine, ai and machine learning technologies are increasingly been integrated into testing tools like, and frameworks to automate a test case, to optimize the test coverage and identify patterns in the test data for more efficient defect detection and resolution. Also there is a one thing like a shift left testing which involves moving testing activities earlier in to the development process with the focus of with a focus on with a focus on preventing defects rather than just detecting. This approach emphasizes collaboration between developers, testers and other stakeholders to ensure quality built into the product from the outset. Also, there is one thing continuous testing and continuous delivery. So continuous testing extends the principle of continuous integration and continuous delivery to the testing process, enabling automated testing to be seamlessly integrated into the software development pipeline. This will allow for faster feedback loops, shorter release cycle and improve overall quality.

26:21 | Cara Suarez
One thing we talk about at Kobiton, a lot is how to improve people’s maturity in their processes, you know, around sort of achieving mobile excellence. If you were to give advice to somebody who’s in a testing organization, mobile tester, Maybe who has a little bit less mature processes, you know, than what you’re practicing at Red Hat, what advice would you give? You know, just very generally speaking for people to kind of up, their skills and improve their processes overall.

26:58 | Sagar Mahendrakar
Yeah. So, Cara, I would like to advise before starting testing, we should understand the requirement clearly because This involves understanding the application functionality. Then we should target the audience Based on the target audience. We should select a range of devices and operating systems. Then a detailed test plan that includes the scope type of testing to be performed, resources needed, timeline and risk assessment should be created. Then we should select a type of testing. There can be a functional usability, performance regression, security, many types of testing, required Testing types to be conducted are decided based on the project requirement. And also… we should keep the detailed records of the test case, results, bugs and fixes for analysis and improvement, And to identify any bugs, fix issues that are not covered in the test cases. We should use feedback And we should regularly ask for feedback with the help of beta testing or we can create any document from which we can get feedback from the real users or inner team members From the team members.

28:12 | Cara Suarez
Wow. Thank you. I think that, that’s like really valuable advice to like kind of take it back to basics and start with understanding, you know, all those various requirements before you kind of dive in and just start, you know, on the testing. So I would like to say we really only have about a minute left. I think we’ve answered the majority of the questions that came in through the Q. A, If there’s anyone out there that wants to chat in one real quick, we could probably take one. But other than that, maybe if you have some closing or concluding thoughts?

28:47 | Sagar Mahendrakar
Yes. Okay. This was a good experience to give a good talk in a mobile testing experience summit. I would like to participate many years from here as well And thank you for your coordination and cooperation.

29:01 | Cara Suarez
Oh, thank you so much. We really appreciate it. I’m glad it was a great experience for you. Take care.

29:07 | Sagar Mahendrakar
Yeah, thank you.