The mobile experience has transformed end-user expectations for good. If an app crashes or acts buggy more than twice, the abandonment rate for that app is higher than 80%. Strategies to meet such high expectations will shape how the IoT and more particularly the wearables market grows.
The user experience for a wearable like a smart watch is far different than a user experience on a tablet. An airline’s smart watch app most likely won’t be used for browsing flights, but instead for gate updates and check ins. Limited hardware, memory, and battery life also challenge IoT app designers to narrow the focus of an app, and concentrate on the quality.
Testing for quality of experience makes sense. But device fragmentation presents some significant hurdles to creating user experience testing environments fit for purpose. Look at the first wave of smart watches and their already different screen shapes, sizes and UIs. What’s more there are new devices hitting the market daily. But which are really relevant and will have staying power? Real devices are essential for testing. Emulators may work well in the beginning of a test cycle, but the holes that emulators miss just won’t do when releasing final products.
Here are some questions to consider when building a test environment:
- What percentage of device coverage is enough for you to ship an app? We at Perfecto Mobile recommend 95% device coverage, but for most that may be unrealistic. Will 30% coverage do? Probably not, but it’s a start. How about 50%, or 80%? It depends how important your mobile or IoT users are to you.
- What matrix of devices is most relevant for your end users? Poll them! You might be surprised to find that many of them are operating on older legacy devices, or prefer tablets vs. phones when using your specific app.
- What about networks? Does network compatibility impact your app? Does it need wifi? Thinking about this will strengthen the conditions of the test environment that you build.
- Who’s testing? Dev/test teams are located all over the world. To support them, build a cloud-based test environment. It facilitates sharing and collaboration much faster, cheaper, easier and more securely than device-in-hand methods, like shipping phones around or traveling with them.
The challenges that arise with IoT testing are similar to those of mobile, but with more complexity.
For instance, everything is connected. Look at a smart watch again. It connects to your phone and then becomes an extension of you. If something doesn’t work, it interrupts the entire connectivity process. There’s risk involved in not fully testing the quality of connectivity throughout all devices enough.
The questions posed are “does an IoT device or app work? Does it work well? Does it work all of the time?” These questions are so crucial to an end user experience, and the answer needs to be YES to all of them . Take a smart home. If my garage door is part of my smart home, and it works well 50% of the time, to me, it doesn’t work at all.
Answer these challenges by making sure IoT apps undergo sufficient functional and performance testing, then post-release monitoring is the answer to this challenge. Functional testing will determine if it works. Performance testing will tell how well it works, as in how quickly it responds, how it performs under varying network conditions, etc. Monitoring will show the stability of the app, because successful repeated usage is how the user experience will grow.
By Eran-Kinsbruner
Director of product marketing Perfecto Mobile
