Chapter-9: Parallel Test Execution on Simulators and Emulators.

Chapter-9: Parallel Test Execution on Simulators and Emulators.

 

Up until now we’ve learned all the Appium basics, including how to extract elements and executing the tests on devices and emulators.

Although automation is great, in today’s fast moving world there is a constant demand to execute tests faster and on more devices. So the idea of executing these tests in a linear fashion (ie. one at a time) seems somewhat antiquated.

In this chapter we are going to discuss how you can leverage parallel test execution on simulators and emulators to test more, faster.

We will be using Java + TestNG, a great combination in the world of Automation Testing allowing us to run tests on parallel.

Before going to deep into parallelization details we will go through basics of TestNG.

  • TestNG:

    TestNG is a open source testing framework written in Java, featuring:

    • Annotations support, which reduces complexity at the class level.
    • It is written in Java and has clean Object Oriented features.
    • All tests can be run from one place via the testng.xml file that specifies the tests to be executed.
    • It has a grouping feature so you can group the test cases and can run them group wise.
    • Supports multi threading and parallel testing at the 1) method(test), 2) class and 3) test suite level.
    • Supports multi threading and parallel testing at the 1) method(test), 2) class and 3) test suite level.

In this chapter we will focus on the Parallel testing feature.

Let’s take a refresher on how you can execute test cases from a Test Class file in IntelliJ Idea. You will recall that you just need to select the Test Method Name > Right click on it > Run.

Figure-1: Run the test case.

Now before jumping into parallel testing, let’s revisit the following simple code for iOS we used in an earlier chapter. This is the code we will be using as an example for parallel execution:

After putting the above test script into a IntelliJ java project, we can execute the test cases. But now we will use the testng.xml method for execution.

Creation of testng.xml:

Unfortunately, by default testng.xml is not available to the project so we need to create it. But you can do it in 2 ways:

  • 1.) Manually create testng.xml:

    • a.) Specifying the Package name, to execute tests from whole package.

  • look liike
    • b.) Specifying the Test Class name, to execute tests for particular Test Class.

    • c.) Specifying the Test Method name, to execute a particular test case.

    • d.) Specifying the specific groups to be included or excluded. Here you need to set the group to Test case first like:

    • And after that you can create a testng.xml file to run the test cases for a sample group.

 

  • 2.) Automatically create testng.xml:

  • That may seem a little confusing right now since you don’t yet know the details of what all of that does. Hang in there, it’ll start making sense in a bit.

  • Creating it via IntelliJ involves the following steps:

    • Move to IntelliJ Idea project and open Preferences.
  • Figure-2: Open Preferences.

    • Now move to plugin section and click on Browse repositories… which will takes you to the Plugins dialog.
  • Figure-3: IntelliJ Plugins.

    • Search for “Create testng” type string and you will find the plugin named “Create TestNG XML”
  • Figure-4: Create TestNG XML plugin.

    • After installing the plugin, you need to restart IntelliJ IDEA.
  • Figure-5: Restart IntelliJ IDEA after installing plugin.

    • Now after restarting the IntelliJ IDEA, you are able to generate a TestNG xml file.
  • Figure-5: Create TestNGn XML.

    • In a fraction of a second testng.xml will be created and you will get the below confirmation modal dialog.
  • Figure-6: TestNG created successfully.

    • By default testng.xml will be created under the project root directory, so it won’t be identified by the java compiler at run/compile time. So you need to move it to test/resources.
  • Figure-7: Testngxml under test OR resources directory.

    • So as you can see in above screenshot, there are some pre-defined XML tags present. Here by default class tag has name=”testcases.IOSTestCases” attribute, which means TestNG will execute all test cases under this test class only. And as we have only one test case(sampleTestCase) is defined under testcases.IOSTestCases it will run/execute only one test case.

      You can also change the testng.xml and make it run at the package, class, method and group level.

Now How can you run the testng.xml?

Well, the answer is just a 2 step process:

    • 1) Right click on testng.xml
    • 2) Select Run

And that’s it, your tests under the IOSTestCases will start the execution sequentially.

Figure-8: Run the testngxml.

1) Parallel execution of Tests on iOS Simulator:

Our Goal is to execute a single test case(sampleTestCase) written in IOSTestCase.java on 3 iOS Simulators(iPhone 7, iPhone 8 and iPhone X) in parallel.

In order to achieve that we need to understand 2 important concepts:

  • 1) We need to manage Appium Server from Code:
    • Up until now we were using the Appium Desktop Application to start the Appium server. Normally, one Appium server is bound to one Appium Session (or you can say to one device/simulator), but now as we need to run the test case on 3 iOS simulators at the same time, we need 3 Appium servers running on three different ports. So now you can’t rely on the Appium Desktop Application as it will be able to run only one Appium Server.

      So the best option left to us is to create and run 3 Appium servers at runtime, and using Java you can easily create the Runtime Appium Server by mentioning the particular port. In our case we will need 3 different ports to start 3 different Appium Servers(or Sessions).

      Below is the code which will start and stop the Appium server for port 4725.

  • 2) We need to use parameters in Test Class and testng.xml
      • We need to start 3 Appium Servers on 3 different ports, and each device should be assigned to a single Appium Server with a unique WDA port.

        What is a WDA Port?

        It’s nothing but used to forward traffic from the Mac host to the iOS Simulator to real ios devices over USB

        This table may make it more clear:

        No. Appium Server Port Device Name (Simulator) WDA Local Port
        1 4725 iPhone 7 8100
        2 4726 iPhone 8 8200
        3 4727 iPhone X 8300

        Now we need to pass these values from testng.xml to our Test Class before all initialization of the WebDriver takes place. And we will use the TestNG @Parameters annotation for that.

        IOSTestCases.java

        testng.xml

        In the above example we are passing 3 values from testng.xml to the IOSTestClass.java:

      • 1) wda: Which will be passed to Desired capabilities IOSMobileCapabilityType.WDA_LOCAL_PORT
      • 2) deviceName: Which also would be passed to Desired capabilities of MobileCapabilityType.DEVICE_NAME
      • 3) port: It will be used to create the Appium server.

Now let’s come to the parallelization part. If you want to run test cases in parallel then you need to use attributes along with the tag.

  • parallel: It has a number of possible values such as tests, classes, methodandinstances.
  • If you want to run parallelization at then you can use parallel="tests"
  • thread-count: No of threads to execute in parallel. If you want to execute 5 test cases in parallel then thread-count="5"

To reach our Goal we need to execute test cases from IOSTestCases.java on 3 iOS simulators in parallel, so below is the testng.xml file we can use:

After adding this to testng.xml you just need to Right click on it and select Run option, and you will see the 3 iOS Simulators will open and each of them will execute test cases in parallel. Great!

We have discussed only one possible way to achieve parallelization. There are many other ways out there and you can also create your own.

2) Parallel execution of Tests on real iOS devices:

In the previous section we looked at test execution on iOS Simulators, but what if you want to execute tests on Real Devices? In the next chapter we will look at using a cloud testing service like Kobiton, but for now, let’s look at how you may need to run parallel tests using the real-devices on-hand.

The answer is pretty simple - we just need to pass UDID as a 4th parameter.

Let’s understand this by way of an example. Let’s say we have 2 real iOS Devices connected to our Mac host and we want to run the test cases on that in parallel. Please look at the table below for Device capability and Port information.

No. Appium Server Port Real Device Name UDID WDA Local Port
1 4725 John’s iPhone 2b6f0cc904d137be2e1730235f5664094b831186 8100
2 4726 iPhone d137be2e12b6f0cc90473031186235f5664094b8 8200
3 4727 iPhone X 137b30235f5664094b831186e22b6f0cc904de17 8300

IOSTestCases.java and testng.xml will look like below.

IOSTestCases.java

testng.xml

Everything else remains consistent.

4) Parallel execution of Tests on real Android Devices:

You just need to change the device name in testng.xml. Using $ adb devices you can get the connected real device names.

testng.xml

When you execute the above testng.xml you can get output result similar to the below image.

Figure 9: Parallel execution on Real Android Devices.

You can also find this example on our Github page as well: https://github.com/appiumbook/appiumbook/tree/master/Chapter9-Test%20Execution%20on%20Parallel%20simulators%20and%20emulators

So that was all about parallel execution on Appium using TestNG. As we discussed there are many other ways to achieve parallelization using TestNG, so using your custom logic you can even execute tests for both iOS and Android parallelly.

In the next chapter we’ll look at how you can use the Kobiton cloud testing service to execute your test scripts against real-devices in the cloud.

No Comments

Post A Comment