The IoT data deluge in industry and manufacturing

“Effective data storage management is a critical component of the IoT ecosystem,” says David Keegan, Group CEO of DataQube Global. Internet of Things (IoT), from a top-level standpoint, refers to a network of physical devices such as embedded sensors, driverless vehicles, smartphones/tablets, wearables, or home appliances, that create and share information without human intervention. Even though there is currently a strong drive towards IoT and digitisation, the concept has been around for the last 10 years at least, with interconnected devices and applications prevalent in industry and consumables.

What has recently changed is the augmented capabilities of said devices, faster comms networks, the standardisation of communication protocols and more affordable IT, which is giving the IoT phenomena a turbocharge. As such, it is transforming operational processes and product lifecycles across a range of markets and applications. That said, the detailed level of information current IoT devices are capable of capturing should be empowering manufacturers to leverage the benefits of Industry 4.0 to operate truly automated production lines/assembly lines, but this isn’t happening as quickly as you might expect. Whilst some of the barriers may be cultural or finance-related, a much bigger barrier, in many instances, is highly intelligent devices versus substandard data handling and storage management infrastructure. 

An unavoidable consequence of IoT and the devices and applications it powers, is the colossal amount of constantly changing data that is generated as a result. This data needs to be processed in real-time if meaningful conclusions are to be drawn and swift decisions made to avoid bottlenecks and keep production lines operational, as smallest of delays can have major repercussions further down the line. This is particularly important for manufacturers reliant on artificial intelligence (AI) and machine learning (ML). Both disciplines are data intensive, bandwidth hungry and require robust storage management processes that enable parallel processing at scale. Indeed, the value of any IoT derived data is incredibly short lived, and unless the associated storage management infrastructure can keep pace with the constantly changing data, an IoT investment can very quickly become an expensive white elephant.

So, what happens to all the IoT data? 

 The data flow in an industrial IoT (IIoT) network, in simple terms it is a three-layer process as follows: 

  • Data sources: IoT gathers data from an array of devices and/or embedded sensors and the information can either be processed locally, depending on the availability of appropriate infrastructure, the sensitivity of the data, or the nature of industry, or transported via an edge gateway to a colocation facility or the cloud for processing and handling.
  • Data storage: The data captured by the embedded technologies then needs to be appropriately stored for long-term and short-term applications. Some of the data might require immediate processing depending on the application (the operability of an industrial robot for example), whereas some might need to be securely transported and/or stored for future applications. 
  • Data analytics & applications: This layer analyses the data so useful information can be generated and acted upon to speed up operations and process control. Detailed insights into production lines/product life cycles enable slicker operations through predictive maintenance and other troubleshooting measures, thus avoiding downtimes and outages that can impact efficiency and profitability. 

Data storage is a small cog in a big IoT wheel 

Storage is just one element of the IoT data processing ecosystem. BUT it is an element that is becoming increasingly integral as insufficient storage capacity is detrimental to operability. The storage capabilities of any IoT network must assure data integrity, reliability, and safety. Moreover, they must be agile to support a range of environments, technologies, and applications, whilst facilitating seamless interconnectivity between edge gateways, other edge devices and the cloud. Substandard storage is the Achilles heel for many manufacturers, with outdated comms rooms not allowing them to harness IoT data to its full potential. Insufficient storage capacity is such an issue that, according to industry research, between 60% and 73% of machine generated data goes unanalysed.

The IoT data that powers Industry 4.0 needs to be processed as close to the source as possible for operability and safety reasons and organisations reliant on mission critical data are quickly realising that conventional colocation facilities cannot always assure the ultrahigh speed and litra low latency needed. In paradox, many on-premises facilities are not fit for purposes as far as IoT data storage management is concerned because they are unable to house the specialist IT needed.

And even if they are, there is seldom room for expansion as capacity requirements escalate. High performance computers (HPC), because of their sheer magnitude, GPU-based processing power, associated cooling technology, and high energy consumption, need specialist facilities that are fireproof, weatherproof, comprise seamless connectivity to the cloud and support dynamic power consumption.

Commissioning a bespoke facility robust enough to meet the demands of IoT data is a non-starter for many manufacturers because of the high costs involved anything between £7-£12 million per MW and lead times in excess of 18 months. What is needed is a viable means of providing centralised data centre capabilities locally without the associated expense of building a bespoke facility that assures HPC processing.

This has not been possible, thus far, however, due to financial constraints, complex project management requirements and excessive deployment times. However, the IoT data handling quandary in manufacturing is about to be transformed thanks to a disruptive approach to edge date centre infrastructures being championed by UK firm DataQube Global. 

Recognising the need for data handling at source, the company has developed a portfolio of podular data centres for internal and external usage that assure high-speed, high-performance, low latency processing needed for IoT data. Installs are possible from less than 10 watts to +100 MW and individual pods can operate independently as a mini data centre or merged in stacks, depending on the size of a manufacturing facility or the storage capacity needed. 

Without a cost effective and viable means of delivering HPC at the edge, IoT data will remain untapped regardless of the accuracy or sophistication of the associated embedded sensors. Edge data centre infrastructures must adapt to meet this changing data processing landscape.

The author is David Keegan, Group CEO of DataQube Global.

Comment on this article below or via Twitter: @IoTNow_OR @jcIoTnow


Aeris Intelligent IoT network provides System Loco with reliable connectivity for worldwide track and trace of sustainable IoT-enabled smart pallets

Posted on: July 5, 2022

Aeris, the global Internet of Things (IoT) solutions provider, announced that System Loco, a global provider of supply chain and asset tracking solutions, has selected the Aeris Intelligent IoT Network to provide next generation connectivity to support and manage the worldwide track and trace of smart pallets employed by System Loco’s customers throughout the world.

Read more

Removing IT bottlenecks: The key’s simplicit

Posted on: July 5, 2022

A report by Mulesoft found that the IT teams of just 37% of organisations completed all of their assigned projects last year. Digitalisation can gather valuable data insights and optimise processes, but progress is often held back by the large IT workload such projects create. Here, Johan Jonzon, CMO and co-founder in edge analytics for

Read more

9 IoT applications that will change everything

Posted on: September 1, 2021

Whether you are a future-minded CEO, tech-driven CEO or IT leader, you’ve come across the term IoT before. It’s often used alongside superlatives regarding how it will revolutionize the way you work, play, and live. But is it just another buzzword, or is it the as-promised technological holy grail? The truth is that Internet of

Read more

Which IoT Platform 2021? IoT Now Enterprise Buyers’ Guide

Posted on: August 30, 2021

There are several different parts in a complete IoT solution, all of which must work together to get the result needed, write IoT Now Enterprise Buyers’ Guide – Which IoT Platform 2021? authors Robin Duke-Woolley, the CEO and Bill Ingle, a senior analyst, at Beecham Research. Figure 1 shows these parts and, although not all

Read more

CAT-M1 vs NB-IoT – examining the real differences

Posted on: June 21, 2021

As industry players look to provide the next generation of IoT connectivity, two different standards have emerged under release 13 of 3GPP – CAT-M1 and NB-IoT.

Read more

IoT and home automation: What does the future hold?

Posted on: June 10, 2020

Once a dream, home automation using iot is slowly but steadily becoming a part of daily lives around the world. In fact, it is believed that the global market for smart home automation will reach $40 billion by 2020.

Read more

5 challenges still facing the Internet of Things

Posted on: June 3, 2020

The Internet of Things (IoT) has quickly become a huge part of how people live, communicate and do business. All around the world, web-enabled devices are turning our world into a more switched-on place to live.

Read more

What is IoT?

Posted on: July 7, 2019

What is IoT Data as a new oil IoT connectivity What is IoT video So what’s IoT? The phrase ‘Internet of Things’ (IoT) is officially everywhere. It constantly shows up in my Google news feed, the weekend tech supplements are waxing lyrical about it and the volume of marketing emails I receive advertising ‘smart, connected

Read more
IoT Newsletter

Join the IoT Now online community for FREE, to receive: Exclusive offers for entry to all the IoT events that matter, round the world

Free access to a huge selection of the latest IoT analyst reports and industry whitepapers

The latest IoT news, as it breaks, to your inbox