Two thirds of retail and e-commerce companies now scraping external data to make more profitable business decisions

Andrius Palionis of

Retail and E-Commerce companies in the UK are increasingly relying on external real-time data to inform business decisions, with 57% now scraping external public web data to gain market insights and help them make more profitable business decisions.

This is one of the key findings from the Oxylabs white paper: ‘The Growing Importance of External Data in the Retail and E-Commerce Industry,’ which reveals a huge rise in demand for external data scraping over the last twelve months.

251 senior data decision-makers including directors of engineering, CTOs, VPs of engineering, chief data officers, heads of business intelligence, and heads of data & analytics were questioned to gain insight into the use of external data and the challenges they face. Respondents were selected from a broad spectrum of UK-based retail and e-commerce businesses.

The findings reveal that businesses in the retail and e-commerce sector have a growing need for public web data, and the interest in old school sources of information is waning. Companies are becoming more interested in novel pricing strategies, shipping information, stock levels, customer reviews, and customer sentiment.

However, traditional sources, such as internal business systems data, remain relevant and useful for the retail and e-commerce sector. Based on our data, 51% of companies still employ data from traditional sources such as CRM, POS, and supply chain systems. In addition, some 38% use third-party data, market research, and statistical database aggregators, such as Nielsen and Euromonitor.

Andrius Palionis VP of enterprise solutions at, comments, “Traditional data sources are slowly losing relevance, although they will continue to be widely used. However, with external data it is now possible to plug in the gaps of knowledge left by traditional sources, allowing for greater predictive accuracy. In turn, this enables companies to significantly improve the ROI of nearly all operations.”

Gediminas Rickevicius

Online retail and e-commerce firms are now able to generate, process, and rapidly analyse data in real-time resulting in actionable business insights. Such data enables highly data-driven organisations to gain a significant proportion of growth and revenue from analytics, according to a McKinsey study.

Gediminas Rickevicius, director of strategic partnerships at, comments, ” E-commerce has experienced explosive growth in the past few years. Any data left on publicly accessible websites becomes fair game for companies looking to gain insight into market movements, consumer sentiment, etc. Therefore, it is completely natural that data-savvy companies will jump at the opportunity to improve their profitability.

To download your copy of the Oxylabs’ From ‘The Growing Importance of External Data in the Retail and E-Commerce Industry’ white paper,’ white paper, please visit Here.

Comment on this article below or via Twitter: @IoTNow_OR @jcIoTnow


9 IoT applications that will change everything

Posted on: September 1, 2021

Whether you are a future-minded CEO, tech-driven CEO or IT leader, you’ve come across the term IoT before. It’s often used alongside superlatives regarding how it will revolutionize the way you work, play, and live. But is it just another buzzword, or is it the as-promised technological holy grail? The truth is that Internet of

Read more

Which IoT Platform 2021? IoT Now Enterprise Buyers’ Guide

Posted on: August 30, 2021

There are several different parts in a complete IoT solution, all of which must work together to get the result needed, write IoT Now Enterprise Buyers’ Guide – Which IoT Platform 2021? authors Robin Duke-Woolley, the CEO and Bill Ingle, a senior analyst, at Beecham Research. Figure 1 shows these parts and, although not all

Read more

CAT-M1 vs NB-IoT – examining the real differences

Posted on: June 21, 2021

As industry players look to provide the next generation of IoT connectivity, two different standards have emerged under release 13 of 3GPP – CAT-M1 and NB-IoT.

Read more

IoT and home automation: What does the future hold?

Posted on: June 10, 2020

Once a dream, iot home automation is slowly but steadily becoming a part of daily lives around the world. In fact, it is believed that the global market for smart home automation will reach $40 billion by 2020.

Read more

Infineon and Rainforest Connection create real-time monitoring system to detect wildfires

Posted on: October 22, 2021

Munich and San Jose, California, 21 October, 2021 – Infineon Technologies AG a provider of semiconductors for mobility, energy efficiency and the IoT, announced a collaboration with Rainforest Connection (RFCx), a non-profit organisation that uses acoustic technology, Big Data and Artificial Intelligence / Machine Learning to save the rainforests and monitor biodiversity.

Read more

Infineon simplifies secure IoT device-to-cloud authentication with CIRRENT Cloud ID service

Posted on: October 21, 2021

Munich, Germany. 21 October 2021 – Infineon Technologies AG launched CIRRENT Cloud ID, a service that automates cloud certificate provisioning and IoT device-to-cloud authentication. The easy-to-use service extends the chain of trust and makes tasks easier and more secure from chip-to-cloud, while lowering companies’ total cost of ownership. Cloud ID is ideal for cloud-connected product companies

Read more