WiMi Hologram Cloud Inc announced that it has developed IoT-LocalSense algorithm, which optimises the load balancing problem, improves the task localisation execution rate, reduces non-local execution and load imbalance, optimises resource utilisation, and further enhances the performance of IoT cluster systems.
In IoT computing environments, data scheduling involves distributing the input data of a job to various compute and storage nodes. If the data matching deviation is severe, it may lead to non-local execution of data scheduling, which increases the task execution time and resource consumption. At the same time, load imbalance may lead to overloading of some nodes and light loading of other nodes, which affects the overall performance of the system and resource utilisation efficiency.
The principle:
Data placement module: Through the processing capacity assessment of the IoT work nodes, the data placement algorithm is designed to reasonably distribute the input data of the job in the computing nodes and storage nodes. Meanwhile, considering the localisation of data, relevant data are placed near the computing nodes to reduce data transmission overhead and delay.
Data scheduling optimisation module: Optimise the data scheduling by using the data block storage location information to make it more likely that tasks will be executed in local nodes during execution, reducing the frequency of non-local execution. It also balances the load of each node in the cluster, ensures that tasks are evenly distributed throughout the cluster, and optimises the utilisation efficiency of system resources.
Advantages of the IoT-LocalSense algorithm:
Improving task localised execution rate: Through data placement algorithms and data scheduling optimisation, the IoT-LocalSense algorithm can improve the local execution rate of tasks on compute nodes. The local storage of relevant data enables tasks to access the data, reducing the need for data transfer and thus speeding up task execution.
Reducing non-local execution: The IoT-LocalSense algorithm puts the data required for non-local data scheduling into the local storage of the compute node in advance through the data prefetching method. This reduces the amount of time a task waits for non-local data transfers, thereby reducing the frequency of non-local execution and improving overall execution efficiency.
Considering data locality: The algorithm focuses on the locality of the data and places the relevant data in the vicinity of the computational nodes, which reduces the data transmission across the network, thus reducing the network transmission overhead and latency, and improving the overall system performance.
Optimised resource utilisation: By reducing non-local execution and optimising data scheduling, the IoT-LocalSense algorithm improves the efficient use of system resources. Tasks are executed more locally, reducing wasted resources and unnecessary load.
In IoT large-scale data processing scenarios, WiMi’s IoT-LocalSense algorithm can improve system performance and resource utilisation efficiency. In the real IoT computing system, the algorithm can be used as a core component of data scheduling optimisation to optimise the schedule of tasks and the distribution of data to improve the overall performance of the system. The performance of the IoT-LocalSense algorithm is compared with other data scheduling algorithms through system simulation experiments, and the algorithm excels in terms of task localisation execution rate and response time, which is better than traditional data scheduling optimisation algorithms.
WiMi’s IoT-LocalSense algorithm improves the performance and efficiency of IoT cluster systems by optimising data placement, data scheduling optimisation, and data prefetching, which increases task localisation execution, reduces non-local execution and load imbalance, and optimises resource utilisation. With the continuous development of IoT technology, the IoT-LocalSense algorithm will continue to be optimised and improved to provide data scheduling optimisation support for IoT computing.
Comment on this article below or via Twitter: @IoTNow_OR @jcIoTnow
