End-to-end testing for the IoT ecosystem and the importance of multistage validation

Virendra Maheta of Volansys

The Internet of Things (IoT) is an intelligent technology between the real and the digital world. It is a system of interconnected devices which have the ability to collect and transfer data over a network beyond manual interference. Enterprises are getting leverage, says Virendra Maheta of Volansys, by implementing IoT Solutions in their business models to reduce time-to-market and boost productivity.

But as we know every technology has its own challenges. The Internet of Things also faces a major challenge in terms of its IoT testing. To build a world-class IoT product, end-to-end IoT solution need to undergo quality engineering thoroughly.

It means every component such as a sensor, cloud access, gateway, user interface, and their inter-connectivity need to be tested before delivery to the end-customer. This end-to-end IoT testing of every component broadly comes under multistage validation.

Let us understand the concept of multistage validation in more detail. An End-to-End IoT solution consists of multiple components like:

  1. User Access Component: Mobile Application or Web Application
  2. Cloud Infrastructure
  3. IoT Gateway
  4. IoT Embedded Devices/Sensors

Each of the above components plays a very critical role in the functioning of the IoT ecosystem which triggers the need for multistage validation. The multistage validation ensures that each component should perform its designated action as per the requirement.

It advocates the process of validating each component of the IoT ecosystem while doing the system testing with closed loop test where the forward path (from the mobile application to the IoT device) and reverse path (from IoT device to mobile application) is considered.

Use Case

IoT System for smart air conditioner

Let us say, a user wants to set the temperature of bedroom AC from mobile application while leaving from the office, then Air Conditioner sends the notification to mobile application when desired temperature is achieved.

The IoT ecosystem for the above use case contains the following components:

  • Mobile Application: The user can set the AC temperature on his/her mobile application.
  • User Access Cloud: The mobile application sends the temperature value on user access cloud using Rest APIs and also updates the database.
  • IoT Cloud and Gateway Device: IoT cloud delivers the “change temperature” commands to IoT gateway device installed at the user’s home.
  • Smart AC: The gateway device sends the desired temperature on the bedroom AC and the AC sends a notification on mobile application once the desired temperature is achieved.

For End-to-End IoT testing, the multi-stage validation plays a critical role as the verification at each component level is required to ensure the system functionality.

Stage – 1: The validation requires mobile application level to check the mobile application functionality. Correlating with the above example, the validation, in this case, would be whether the temperature of the AC is changed to the desired level or not.

Stage – 2: The validation requires user access cloud where the mobile application or web application access the cloud using the Cloud APIs. It is mandatory to ensure that the functional requirements meet at the APIs and database level. Also, make sure that the changes made by mobile application and which are reflected in the database are sent to the gateway device through IoT cloud logs validation. Correlating with the example, validation would be done to ensure that the APIs are working as expected and the changes made to the database for AC are as expected for the desired device.

Stage – 3: Here, the verification is needed at the gateway stage where the IoT cloud sends “temperature change” command using IoT communication protocol like MQTT, XMPP, XML, JSON. The validation is done to ensure that the correct message is received by the IoT gateway device and the message is getting forwarded to the intended end device via a communication protocol such as Zigbee, BLE, or Wi-Fi, whichever is available. Correlating with the given instance, the validation would be done to ensure that the temperature change action is made for intended AC device over a supported communication protocol.

Stage – 4: At last, the validation is required at end embedded device to ensure that the action received from the IoT gateway is reflected on the embedded device.

Corresponding to the above example, the validation would be done to make sure that the temperature gets set to the desired level on AC. The air conditioner sends the “desired temperature achieved” notification to the mobile via gateway and cloud. The mobile application, cloud, and gateway gets validated as soon as the mobile notification is generated from Smart AC.

Importance of the multistage IoT ecosystem testing

  1. It enables testing, validation, and verification of application architecture, integration between all the components and business requirements
  2. Able to catch the bugs at integration level and also find the issues at component level
  3. Enables solution testing with the perception of end user and real-time use cases.

The author is Virendra Maheta, project manager at Volansys Technologies. Virendra has 12+ years of experience in Quality Assurance for various domains like Embedded, Networking, IoT, Telecom/VOIP, ERP. He also has experience in Manual and Automation Testing Management.

Comment on this article below or via Twitter: @IoTNow_OR @jcIoTnow


9 IoT applications that will change everything

Posted on: September 1, 2021

Whether you are a future-minded CEO, tech-driven CEO or IT leader, you’ve come across the term IoT before. It’s often used alongside superlatives regarding how it will revolutionize the way you work, play, and live. But is it just another buzzword, or is it the as-promised technological holy grail? The truth is that Internet of

Read more

Which IoT Platform 2021? IoT Now Enterprise Buyers’ Guide

Posted on: August 30, 2021

There are several different parts in a complete IoT solution, all of which must work together to get the result needed, write IoT Now Enterprise Buyers’ Guide – Which IoT Platform 2021? authors Robin Duke-Woolley, the CEO and Bill Ingle, a senior analyst, at Beecham Research. Figure 1 shows these parts and, although not all

Read more

CAT-M1 vs NB-IoT – examining the real differences

Posted on: June 21, 2021

As industry players look to provide the next generation of IoT connectivity, two different standards have emerged under release 13 of 3GPP – CAT-M1 and NB-IoT.

Read more

IoT and home automation: What does the future hold?

Posted on: June 10, 2020

Once a dream, iot home automation is slowly but steadily becoming a part of daily lives around the world. In fact, it is believed that the global market for smart home automation will reach $40 billion by 2020.

Read more

IoT set to overtake cloud computing as primary Industry 4.0 technology, Inmarsat research reveals

Posted on: October 14, 2021

New research by Inmarsat, the provider of global mobile satellite communications, reveals that investment in the Internet of Things (IoT) is set to overtake cloud computing, next generation security, big data analytics and other digital transformation technologies in the near future.

Read more

IDTechEx looks at the setbacks and explores how to move forward

Posted on: October 14, 2021

Bill Gates backed a Belmont smart city in the Arizona desert little has happened beyond a land purchase. Authorities demand that the Colorado river’s diminishing water supply is unharmed. Arizona suffers historic water shortage. The Southwest and much of the West is suffering from an intense 22-year drought, resulting in increasingly low water levels, dry

Read more