As enterprises continue to amass data at exponential rates, it has become imperative to unlock the true business value of data and analytics to drive agility and get actionable insights. On a global level, says Ram Chakravarti, chief technology officer at BMC Software, organisations are rapidly adopting data-driven strategies to enhance operational agility and business continuity.
In fact, according to an Accenture survey, 88% of C-suite respondents agree that using forward-looking data sets and analytic approaches to better predict and respond to events is crucial to their success.
With trillions of data points from consumer activity and enterprise systems, data is becoming more important than ever before. For example, there were approximately 14 billion mobile devices recorded last year, and nearly 36 billion connected devices this year.
In addition, social networking sites are estimated to reach 3.96 billion users in 2022, and these figures are expected to grow as mobile device usage and mobile social networks gain traction in previously underserved markets. Furthermore, we now have access to satellite images, geophysical data, genetic data, roof top imagery, and a whole host of other data unlike ever before.
Out with the old
Although data and analytics is becoming a standard procedure for many enterprises, a significant number of transformations fail to deliver business expectations. In fact, Gartner revealed that only 22% of analytic insights will deliver business outcomes in 2022. There are several reasons for these failures, such as new data or more complex business models making simplification challenging, as well as ‘process mismatch,’ whereby traditional data management processes do not work well with innovative technologies, such as artificial intelligence.
Some organisations may find there is a lack of collaboration within the business or inadequate business involvement to drive a successful cultural shift, as well as a lack of budget for the amount of investment involved. Since data analytics teams require a combination of internal domain knowledge and deep technical knowledge, there also may be a scarce talent pool within the industry, and challenges in operationalising at scale with rapidly rising stakeholder expectations on speed, flexibility, timeliness, and customisation of new capabilities.
A recipe for success
These challenges have resulted in the emergence of DataOps, an application of agile engineering and DevOps best practices in data management. DataOps rapidly turns new insights into fully operationalised production deliverables that unlock business value from data.
There are several key components to DataOps, the first of which is extensive collaboration across the data management ecosystem between business and IT. DataOps enables pervasive automation which encompasses orchestration, provisioning, configuration, and self-service. Process standardisation and refinement is used to incorporate changes that would be more effective with artificial intelligence, as well as a pragmatic approach to data governance, security, and metadata management.
Considering DataOps is based on the application of DevOps best practices, it should come as no surprise that collaboration and automation are critical to the success of it. Success requires extensive collaboration across the data management ecosystem between data professionals, such as data owners, stewards, architects, and engineers, as well as data consumers including data scientists, business owners, power users and end users. DataOps platforms are built to serve the needs of multiple data consumers, from traditional data users, such as data warehouses, to more sophisticated analytics platforms and machine learning pipelines typically used by data scientists.
DataOps enforces the delivery discipline required to be successful in data and analytics transformations. However, this is not a magical recipe that guarantees instant success. As with any new technology capability, organisations should approach this as a multi-horizon initiative with a set of deliberate steps on their way to becoming a data-driven business.
