If we take another look at the mobile operating system usage, we see that the main mobile platforms iOS & Android combined have a higher world-wide usage than Windows and OS X. The current share is 51% for iOS & Android, compared to 41% of Windows and OS X.
These numbers should show you, that the mobile usage is THE main use case these days to connect to the internet. Both main mobile platforms provide an ecosystem called App Store to enrich the functionality of the modern smartphones with apps. The apps that can be installed on a mobile device can make use of the powerful hardware they are running on. For example, an app can make use of the build in sensors like GPS, Gyroscope, Ambient Light Sensor or Near Field Communication just to name a view. With the help of the sensors the app gets information about the current environment of the user and can provide a unique using experience to the user being in navigation, playing a game or the pay in a shop.
Mobile App Quality is Key To Success
But what about the quality of these apps? According to TechBeacon, more than 50% of mobile users will uninstall an app after first usage if the app is crashing or freezes. Furthermore, mobile users expect that an app is available for usage in 2 seconds. This shows that mobile users have a much higher expectations to mobile phones and especially to mobile apps. This poses a special challenge for mobile software development teams and especially to mobile testers.
The software industry is a fast-paced industry. The majority of companies are using agile methodologies to develop their products. For an agile software development team using SCRUM or KANBAN this means that after each iteration a shippable product is available and can be released to the customer.
This fast iteration is especially a unique challenge for mobile testers. Why? Once the packaged mobile app has been uploaded to the respective App Store and was installed by the user, a team can’t roll back the version in case a critical bug was found. The only way the team can fix the native mobile app is to detect the bug, fix it and push an immediate hotfix to the users. However, some app stores have a review process which can extend the time until the patched app reaches the customers.
Because of that, mobile testers should have a mobile testing checklist before the team will submit an app version to the app stores. With the help of the mobile testing checklist the tester and the whole team can see if they have missed an important step.
Mobile Testing Checklist
Here is a possible mobile testing checklist with a short explanation why each point is important to check before going live:
- All test cases or testing ideas have been executed for the release and are documented e.g. in a test case management tool like TestBench Cloud Services?
This is important to have especially in bigger companies where many teams are working on the product to not lose the quality overview.
- The app was tested against the most used operating system versions and most used mobile devices?
Create mobile device groups and prioritize the testing devices based on customer usage to downsize the effort in testing activities on different devices.
- The app was tested in different mobile networks depending on the use case scenario of the app, including offline mode?
Don’t make the mistake and test only in strong office Wi-Fi. Especially slow networks like EDGE or 3G can highlight possible bugs in the app.
- The app was tested against the supported sensors?
Have a list of all supported sensors available and make sure to test against different device manufacturer.
- The app was tested in portrait and landscape mode?
If the app supports both views, it’s important to take a look after the screen rotation happened. Sometimes there might be UI glitches.
- The existing automated checks have been executed and are green?
Also take a look at static checks on the CI systems for potential issues.
- The app has been approved by a design or interaction design colleague?
Especially helpful if new features and user scenarios have been implemented.
- The battery consumption of the new app release has been checked?
Compare the current live version side by side with the new release candidate. Perform the same actions and check battery consumption.
- The app store material like screenshots, text and releases notes are available?
Up-to date app store material gives the user the feeling the product is complete and that the company behind it cares about it.
- The app update has been tested to check that the update did not cause any issues?
Especially during the app update many things can go wrong. For example, the local database migration can fail and sensitive data might get lost. Or a user is getting logged out after the app update.
Note: This list is by no means complete but it might help you to remember important actions to do before submitting an app to the app stores. Depending on your app and the use case of the app, your own mobile testing checklist might look totally different.
Daniel Knott for TestBench Cloud Services