New market research commissioned by Starburst and Red Hat, shows that data access has become more critical for 53% of survey respondents throughout the pandemic as analytics workloads and demands increase significantly.
The survey, conducted by independent research firm, Enterprise Management Associates, found that the imperative for faster data access is about driving business outcomes, with 35% of survey respondents looking to analyse real-time changes to risk and 36% wanting to improve growth and revenue generation through more intelligent customer engagements.
More than one-third (37%) of survey respondents aren’t confident in their ability to access timely, relevant data for critical analytics and decision-making. “The State of Data and What’s Next” survey revealed that this lack of confidence stems from four main challenges currently facing data teams:
- Data is significantly distributed and this complexity isn’t going away. The survey found that half (52%) of respondents have data in five or more data storage platforms. And more enterprises are expected to follow this trend, with 56% of respondents claiming they will have data on more than five platforms in the next year. It appears distributed data is here to stay.
- Moving data across sources is riddled with challenges. Many enterprises are finding the task of building and deploying data pipelines difficult. Some of the biggest obstacles they face are combining data in motion with data at rest (32%), the excessive time it takes to address break and fix (30%), data pipeline complexity (26%) and the manual coding lift for deploying error-free data pipelines (25%).
- Developing a data pipeline is time-intensive. Building the right infrastructure for accessing data takes significant time and manpower, and enterprises can’t afford to wait. For 45% of survey respondents, it currently takes more than a day to develop a data pipeline, with 27% saying it can take anywhere from three days to two months. Not to mention making that data pipeline operational takes more time 52% of respondents said that it adds another day or more, with 24% saying it adds another week to the process.
- Every second between querying data and gaining insight counts. With the rapidly changing landscape of almost every industry due to the pandemic, today’s enterprises have significantly less time to gain insight on their data before it’s outdated. Viability of business decisions comes down to a matter of milliseconds (according to 17% of respondents), with 39% of business decisions requiring latency of one second or less.
“Data is the lifeblood of any business trying to navigate today’s digital economy. What we’re seeing in these survey results is that organisations have clear demands for faster and more comprehensive data access, but technical challenges still exist,” says Justin Borgman, CEO of Starburst. “It’s imperative that companies overcome these challenges because their customer experience, competitive advantage and growth depend on it. By empowering our customers to access all of their data, regardless of location, Starburst enables better and faster business decisions.”
The survey shows that enterprises are turning to specific practices and product capabilities to meet these challenges:
- Multi-cloud flexibility. Among survey respondents, 56% of their data is in the cloud and 44% is on-premises. The move to the cloud is quickly progressing, however, with respondents expecting to have 62% of their data in the cloud and 38% on-premises by the end of 2021. And this migration is not to one specific cloud provider: the number one criteria respondents looked for in cloud data storage was the flexibility to access data from multiple clouds (47%).
- Automation, search & cataloging. When asked, almost half (44%) of respondents identified automating IT and data operations as the most important practice to improve their organisation’s data strategy. Survey respondents also identified implementing search (32%) and cataloguing data (30%) among the most important practices, pointing toward the necessity for organisations to make finding and using the right data a quick and easy process for users across the enterprise.
- Modern analytical capabilities. Legacy data solutions are no longer meeting the needs of the modern enterprise data analysts need future-facing tools that enable the processes they are engaging in most. For 42% of respondents, that is running a single query across relational databases, file systems and object storage. Data analysts are also looking for the most support in analysing streaming data or real-time events (38%) and running a single query across structured and semi-structured data (37%).
“The opportunity for every organisation is to convert data into action,” comments Mike Piech, vice president and general manager of cloud storage and data services at Red Hat. “But the challenge we see organisations facing is a veritable tsunami of diverse data in their hybrid cloud environments, with a range of users needing access to it, including application developer and DevOps teams, data science and data engineering teams, and cloud infrastructure teams.”
“The imperative is to create a consistent foundation for storage that can handle a wider range of workload types, from structured databases and semi-structured data warehouses to unstructured data lakes. With this foundation, organisations can be better positioned to harness insights from their data, no matter where it comes from in their hybrid cloud environment.”
“It was clear even before this research that enterprise technology needs have drastically shifted over the past year,” adds John Santaferraro, research director at Enterprise Management Associates. “This research identifies that there is a chasm between data requirements and current capabilities, pinpoints what is most needed to bridge that chasm, and provides the roadmap for how technology solutions will need to evolve to meet these needs.”
To download the full report click here.
Methodology
The survey of 402 respondents in the US, UK, Canada, France, Germany, Australia and Singapore was conducted by independent research firm Enterprise Management Associates in December 2020. Respondents included IT decision-makers and data management professionals from organisations across a range of industries, including manufacturing, retail, telecommunications, finance and more.
Comment on this article below or via Twitter: @IoTNow_OR @jcIoTnow
