Introduction

iOS automation testing involves testing applications that are made for iOS devices only. Testing applications, in general, requires the tester to understand the purpose of the application, its technical requirements, and performance metrics to compare the expected behavior versus the actual behavior. The iOS family is a closed environment that is licensed to operate on Apple devices only. This makes it easier to test the application across the device range as opposed to Android applications that can be deployed into a much larger array of devices.

Looking to find the right automation testing framework for your team? We can help you.

Get in Touch with Kobiton

The iOS application needs to be tested both manually and using automation scripts. Bugs are inevitable in any application despite thoughtful designing. iOS automation testing is just as important as testing any application whether it is on iOS or Android platforms. Quality assurance activities must be performed right from the beginning of the application lifecycle. In iOS applications, since the devices are known, and the OS updates are well-planned, it is more convenient to perform testing on these devices. Automated testing on iOS applications has a multitude of benefits. Running automated test scripts helps save time and effort as it allows one test to run in parallel across multiple devices.

How is iOS Automation Testing Implemented?

An iOS application needs to be tested not only for its functionality but its UI/UX design too. The test cases must be applied to the various iDevices – viz, iPhone, iPad, iPad mini, iPad Pro, Apple Watch, Apple TV, iMac (of various screen sizes), iPod Touch, Macbooks, etc. – to ensure the UI/UX is being implemented as expected on devices of varied resolutions. Apart from testing on the array of devices where the application will be deployed, as a tester, you would also require to test out the application from every angle. Some other types of testing that are essential include unit testing, functional testing, visual UI testing, as well as end-to-end testing. Let’s take a moment to look at each type of testing:

1. Unit Testing: Unit testing is most often conducted by the developer itself. Unit tests are performed to confirm whether the source code of any specific section works as per the requirements, in isolation. This testing is done individually on a piece of code that the developer had developed once he merged his work into the application’s source code. The most common framework for performing unit tests is Apple’s XCTest, available within the XCode development environment, which offers iOS automation testing capabilities, as well.
2. Functional Testing: The process of testing all the functionalities thoroughly without prior knowledge of the source code is known as functionality testing. On iOS devices, we need to be cognizant of the haptic functionalities, such as soft-touch, 3D touch, taps, shakes, and rotations, etc. These inputs need to be considered when performing functional testing of the iOS application on various compatible devices. These test cases can be written manually, and later, to be automated into automation test scripts, so that the functionalities can be tested in every version release of the application and as part of regression testing as well. KIF (Keep It Functional), XCode UI Test, EarlGrey, and Calabash are some of the iOS automation testing tools supported by Apple for functional testing. We will learn more about these tools further in the article.
3. UI/UX Testing: One of the most important parts of iOS testing is the user interface and experience. The application’s user interface must function as expected when the following actions are performed. We can categorize the different aspects of UI testing as:Inputs: The inputs include soft touch, scrolling, long touch/3D touch, and buttons. Each of these inputs would perform specific functions which need to be tested meticulously. The buttons on the screen may differ in size, position, color, and font. Screen: The screen orientation of the application must be tested on its compatible devices. Screen UI testing must also include the screen resolution changes and adaptability according to the device. Soft Keyboard: The option to hide or use the keyboard which comprises emoji and symbols tabs is important to include and test. Hard Keys: Some applications also link with the physical buttons such as the power, home, and volume buttons. These hard keys play a major role in UI/UX testing as they provide accessibility and comfort for the user. Lists: Unlike its Android counterparts, iOS handles pop-ups differently. In iOS applications, lists are displayed when options need to be chosen instead of pop-ups. Testing iOS UI/UX is considerably different from testing those in Android applications. Owing to Apple’s closed ecosystem, the behavior and expectation of its devices are limited, which makes for a predictable testing process. Automating UI/UX testing for an iOS application can be done by utilizing XCUITest, which is baked right into Xcode and IDE environment and is restrictive to Objective C or Swift languages, or leveraging Appium automation framework that allows flexibility in which programming language and additional tools you see fit for your iOS automation testing.
4. Regression and End-to-End TestingiOS applications undergo multiple changes, bug fixes, and versions of releases. Throughout the lifetime of the application, the changes introduced to the application could affect the existing features of the application. For this reason, testers must perform regression testing making sure that the existing functionalities are working as expected, despite the changes made. Regression tests must be performed during each version release of the application. The QA team, in general, would prefer to automate these test scenarios due to their repetitive nature. Automating them can save the QA team, time, and effort allowing them to use their focus on writing and automating test cases for the new changes that will become part of the regression test suite in the next release. On the other hand, we have end-to-end testing where the application stack is tested completely from beginning to end. Here, we simulate the entire user experience from front-end UI/UX to backend server features and other network-related aspects. It is a part of integration testing where we can test if all the functionalities work in conjunction with one another, as expected. In cases like these, simulators are used to automate the end-to-end test scenarios if testing on the actual devices or all the compatible devices is not feasible. However, it is best practice to perform regression and end-to-end testing on real devices as it promotes confidence in testing as if the device was in a user’s hand. By utilizing iOS automation testing frameworks mentioned above (and later to be discussed), a quality engineer can achieve automated regression and end-to-end suites.

Challenges Faced in iOS Automation Testing

Having tight schedules for release is not uncommon when developing iOS applications. This gives the QA team less time to integrate the multitude of features that the application performs or to create new test scripts.

With the introduction of face-ID and fingerprint authentication methods, it can be harder to automate most of the test cases. Manual intervention is deemed necessary in such cases. It can be hard to automate the environments of dynamic carrier network conditions and related background processes which requires specialized expertise.

Some other challenges faced in automating iOS application testing include troubleshooting and debugging the test code due to lack of time. The absence of an adequate mobile testing lab can also prove arduous.

On top of that, the automation framework solutions for iOS automation testing, and for mobile application testing in general, is minimal compared to various frameworks and solutions provided for web applications.

Best Practices for iOS Testing

Real devices over Emulators/Simulators:

When it comes to UI/UX testing, we are required to test the application on all the compatible devices, but it can get quite expensive for executing tests on all the devices. This is where emulators or simulators come into play. Apart from the cost perspective, there is also the matter of debugging the application step-by-step which is not as easy to spot when testing with real devices.

On the contrary, the processing speed of testing with a real device can be quicker than working with emulators/simulators. Reliability plays a major role here, as well. Real devices can be more reliable when compared to emulators/simulators since simulating all kinds of user interactions can be tedious.

Although emulators and simulators are easily attainable to spin up, they cannot quite capture the scenarios where the device’s battery or storage is low or when the flow is interrupted by calls or notifications. In such instances, manually testing with real devices can be more satisfactory. Additionally, as mentioned above, real devices are preferable in executing regression and end-to-end testing to simulate a user’s actions as close as possible.

Rich reporting and Communication:

When it comes to application testing, taking screenshots or recording the screen for every step or action can be crucial in debugging and understanding the application’s actual behavior. On iOS devices, screenshots can be taken manually by pressing the power and home buttons together on older devices and pressing the power and volume up buttons on the latest devices. To record the screen, we can use QuickTime to record the iOS device screen, provided it is connected to a Mac machine using a lightning cable. Some tools offer automated screenshots and screen recordings, such as fastlane, which uses XCUITest to take screenshots. Additionally, there are iOS automation testing frameworks that also deliver rich reporting and communication during your automation and testing efforts.

Crash and Console Logs:

Crash logs are used to understand the root cause in case of application failure. In such instances, you need to catch the crash logs. The following steps can be taken to capture the crash logs:

1. Connect the iOS device with a Mac machine (iMac or Macbook).
2. Open the Menu bar by holding the Option (⌥) key.
3. Under Menu, navigate to Library/Logs/CrashReporter/MobileDevice
4. Here you will find the name of the iOS Device. Click on the folder.
5. You can find the logs here starting with the name of your AUT(application under test).

Console logs, on the other hand, display the complete information of the AUT (application under test) on the iOS device. To view the console logs, you will need to use the iTools application. Once you’ve downloaded the application, connect the device to the system where the iTools application is running and click on the Toolbox icon. The last step is to click on the Real-Time Log button which displays the console log.

SetUp and TearDown methods in iOS Automation Testing Frameworks:

In some frameworks like XCTest and Appium, we can use SetUp and TearDown methods to customize the state of the test cases according to your testing needs.

setUp(): This method can be used to set the initial state of each test case before the particular test method is run.

tearDown(): This method can be used to clean up after each test case or test method is complete.

These methods are essential for effective iOS automation testing using some common testing frameworks to avoid test failure due to cache.

Top iOS Automation Frameworks

• Appium:

Created by Selenium, the pioneering web application automation framework, Appium is the mobile application automation framework. Since within the same family, Appium utilizes the same Selenium WebDriver and HTTP protocol process to automate mobile applications, including iOS applications. It is an open-source platform, and it is not restricted by programming languages, as you can test the application in most of the commonly used languages, such as Java, C#, Python, Ruby, etc. Additionally, Appium can be integrated with any additional framework or tool needed to help facilitate mobile application automation. Thus, Appium offers a flexible way of setting up your testing needs as you see fit.One of the major advantages of Appium is that it works on any kind of mobile application, including web, native, and hybrid, which implies that we can use the same Appium code on both iOS and Android devices.

• XCTest/XCUITest

XCTest is Apple’s official testing framework to perform unit tests. XCUITest is a UI testing framework that is built upon the XCTest framework and includes supplementary classes that aid in UI testing such as UIAccessibility. Both frameworks can be written in either Objective C or Swift. Since it is native to iOS devices, it does not require any supporting software or packages to perform testing on the AUT (application under test).XCTest/XCUITest is faster than other automation frameworks for iOS applications. XCTest does not require additional layers of abstraction APIs and is therefore lightweight. XCode Test Recorder is another option if you decide to use XCTest as your automation framework. With the test recorder, it is easier to perform UI test scenarios as we can record and playback the executed UI tests

• EarlGrey

Google has put forward its own, open-source, and flexible, internal UI Testing framework named EarlGrey. It comes with its distinctive advantages such as:

• Synchronization Features with the UI, network requests and threads.
• User-like interaction which are app-level touch events that can perform swipes and taps.
• Visibility Checks that check if an element is visible before executing the test script.

• Calabash

Calabash is also an open-source framework that can perform testing in both iOS and Android platforms. To automate test cases in Calabash, the test cases must be written in Cucumber. Cucumber is a Behavior Driven Development framework that uses the Gherkin language for writing test scenarios in the ‘Given, When, Then’ format that makes it easier for the business analysts and stakeholders to understand the test scenarios without demanding any technical coding knowledge from them. Calabash can interact with native as well as hybrid applications. It consists of various beneficial features such as screenshot recording capabilities, and gestures.

• Detox

Detox is a tool that can be specifically used for end-to-end, UI automation testing needs. It supports JavaScript and is meant to work as a grey box testing framework that is easier to learn. Some of the advantages of Detox are:

• It helps reduce the flakiness of the application by monitoring the asynchronous elements in the UI. It runs the test cases as they would in production.

• As mentioned previously, Detox is a grey box testing tool that accesses the source code from the mobile applications, and hence, it is faster than other third-party iOS automation frameworks, excluding XCTest/XCUITest
• It is made to run in a continuous integration pipeline.
• OCMock

This framework is used to create stub objects in the iOS AUT. It can be implemented with a static library to develop iOS applications or as a framework used for OS X development. Its strengths include:

• It uses Objective-C so that most iOS application developers can develop the test cases too.
• It is also an open-source framework.
• It is easy to use by adding mock objects to unit tests.
• KIF (Keep It Functional)

Like most iOS automation testing frameworks out there, KIF is an open-source tool. It is an iOS native application, and a developer will need to add the KIF framework to the project. Some key benefits of KIF are listed below:

• The syntax is easy to understand and intuitive.
• It uses Objective C.
• Runs well on the command line of a CI pipeline.
• It can be integrated with XCode workflow effortlessly.
• It does not require any external dependencies, which many other testing frameworks do.

Conclusion

Performing iOS automation testing on iOS applications can be both interesting and challenging. As discussed above, it can be strenuous to keep up with the advancing technology, such as face-ID and fingerprint authentication, that have taken over the age-old PIN and password authentication methods. It is not easy, considering the time-restricting OS version upgrades that occur periodically, but fortunately, these upgrades are well-planned.

To summarize, it is crucial for the development and QA teams to put forth best practices to be well-equipped to handle any challenge that may arise. It is important to know when to use emulators/simulators and when the real devices need to be put to test. We need to make use of screen capturing and recording tools, as well as, crash and console logs to trace the steps of the test cases and to understand potential UI issues that may surface in order to fix them in a timely manner.

Now that we have discussed an overview of testing iOS applications, it is apparent that choosing the right automation framework, the right supplementary tools and methods, as well as testing devices (emulators/simulators/stubs) for your project team are vital in successfully testing the application. Based on the skillset of your team, choose the best framework to test your iOS application.

Looking for a mobile automation testing framework?