During the recent IoT Data Analytics and Visualization conference in Palo Alto, many of the presentations, discussions and questions reflected how important the topic of IoT data analytics has become to enterprises. To achieve the new IoT opportunities and services such as predictive maintenance, smart manufacturing and improved customer retail experiences, enterprises need to tap into the hidden and potential value of data from all connected things, unlocked by analytics.
To unlock this potential, IoT data analytics presents enterprises with three significant challenge areas: 1) a changing technology base, 2) issues of data privacy and security, and 3) overall business readiness and maturity of enterprises and IoT.
A changing technology base
As enterprises achieve more and more implementations of connected device infrastructures and new IoT applications, a next stage consideration within the enterprise is how to manage these increasing volumes and variety of data exhaust, generated in real-time by the devices. Not in all cases will IoT solutions generate vast amounts or types of data, allowing for limited and incremental demands on data storage and processing. Where enterprises have started to experience significant data growth, potentially spread across different data types such as structured, semi-structured and unstructured data, new storage solutions such as distributed file systems, cloud services and NoSQL may and will be required. In the context of IoT, achieving a unified view across all data sources requires what has become known as data lakes, pools or oceans, avoiding the limitations of isolated data silos.
Data ingestion (capture) and storage have had to adapt to the new scale, speed and structure of data. Other significant changes are driven by the requirements of IoT analytics. First of all, is it the analysis of data in motion or data at rest? For the latter, traditional ETL approaches remain robust and relevant. But with emerging requirements to analyse streaming data, and provide quicker and substantially more time-sensitive analysis for applications such as fraud detection, automated decision-making systems for self-driving vehicles or machine set-ups, new technologies such as Complex Event Processing, Massive Parallel Processing (MPP) capabilities and data management tools have had to be designed. Second, what are the expected outcomes from the analytics? Descriptive, predictive, prescriptive? To achieve higher levels of insights from data, advanced algorithms and machine learning are now become integral requirements for these processes. And final consideration is that of where analytics needs to take place, generating substantial requirements for fog computing and distributed analytics.
These are some of the examples which illustrate the changing base of IT technologies to meet the requirements of IoT data analytics and visualizations.
Data privacy and security
Data privacy and security are distinct topics; one relating to the ownership of the data whereas the other to the protection of that data. Yet, the two are closely related, and for IoT data, enterprises have started to recognize that security through perimeter defence approaches need to be replaced with security by design throughout the complete IoT infrastructure. In parallel with these security developments, enterprises have also started to recognize that any user data is owned by the user, and to the extent possible, enterprises should endeavour to make this as transparent and manageable as possible for data owners.
Where the discussion becomes extremely significant is when data owners are involved in the design of the security and data privacy measures of enterprises, recognizing that this is not a binary environment but one with many possible solutions, changing over time.
Business readiness and maturity
Enterprises are moving ahead with the changing technologies for IoT data including and addressing data privacy and security issues. The extent to which enterprises are able to unlock the greater opportunities from IoT data depends on the IoT readiness and maturity of the enterprise. This may be identified by the extent to which enterprises turn actionable insights from IoT data into new business processes and services.
The readiness and maturity of the enterprise includes a number of attributes such as the enterprises awareness of IoT data and analytical options, the extent to which they have an IoT organization in place and for example, to what extent the enterprise has involved data scientists and business analysts with domain expertise to design and develop algorithmic tools. It will ultimately be measured by the extent to which the enterprise has become a data-driven business.
A growing opportunity
As more and more data becomes available to enterprises not only from their deployed IoT assets but also from existing enterprise systems, customer products, and partners, enterprises are exploring the approaches and techniques to turn this data into new assets for the business. IoT data analytics and visualization will be important tools in this process.
The author of this blog is Emil Berthelsen , principal analyst, Machina Research
