DMX Change Data Capture simplifies continuous delivery of fresh ‘Big Iron’ data

Syncsort, a global provider of data liberation, integrity and integration solutions for analytics, announced new capabilities in its mainframe data access and integration solution. This populates Hadoop data lakes with changes in mainframe data.

The new DMX Change Data Capture (DMX CDC) functionality delivers real-time data replication that enables organisations to continually keep Hadoop data in sync with changes made on the mainframe, and consistently make the most current Big Data information available in the Hadoop data lake for analytics.

“Many organisations are using our industry-leading big data integration solution, DMX-h, to quickly and efficiently populate their data lakes with enterprise-wide data, including complex data from the mainframe for a variety of use cases, such as Hadoop as a Service, Data as a Service, data archive in the Cloud, fraud detection, anti-money laundering and Customer 360,” said Tendü Yoğurtçu, chief technology officer, Syncsort.

“After populating the data lake, it is very important to keep that data fresh to power real-time analytics and accurate decisions based on up-to-date information. Our new CDC offering provides our customers with an easy-to-use, highly efficient solution for ensuring the data lake is refreshed in real-time with the incremental updates, while meeting SLAs and conserving network resources.”

Tendü Yoğurtçu

Syncsort already is in many large production deployments for accessing data from traditional enterprise systems and integrating it with Hadoop, including DataFunnel, its single-click solution for populating the data lake data from thousands of tables and automatically creating the corresponding metadata in Hadoop.

The new CDC capabilities extend this with real-time data replication, reducing the network load and providing up-to-the-minute mainframe data for analytics by:

    • Speeding and simplifying the process of synchronising mainframe data with Hadoop, on-premise or in the cloud
    • Saving time and resources using the DMX-h GUI and dynamic optimisations, which eliminates the need for coding and tuning
    • Eliminating impact on mainframe database performance by avoiding database triggers
    • Affording reliable data transfer – even during loss of mainframe to Hadoop connections or Hadoop cluster failures – picking-up where the transfer stopped without restarting the entire process
    • Saving money with virtually no use of chargeable mainframe CPU resources
    • Supporting IBM DB2 for z/OS and IBM z/OS VSAM files, with more sources to come
    • Enabling queries on most current data via rapid update of Hive table data and statistics
    • Handling all enterprise Hive file formats including text, ORC, Avro and Parquet

The combination of all the time savings and low resource utilisation with the new CDC facilities also helps organisations meet short SLAs with virtually no impact to MIPS costs.

Comment on this article below or via Twitter: @IoTNow OR @jcIoTnow

RECENT ARTICLES

Nordic-powered smart utility access cover offers anti-theft protection and detects open/close events

Posted on: August 16, 2022

Oslo, Norway – China-based Jian-IOT has launched a smart utility access cover that can detect when the cover has been opened or closed, records water level and temperature data, identifies any damage, and includes an anti-theft system that tracks the current location of the device. The ‘Integrated Intelligent Manhole Cover’ employs Nordic’s nRF52832 SoC to both act

Read more

Helbiz to offer on-demand taxi services to widen international intermodal offering

Posted on: August 16, 2022

New York, United States – Helbiz, a global provider in micro-mobility, announced the beginning of a new chapter for its intermodal mobility offering, Helbiz Taxi. The micro-mobility company aims to integrate mobility services around the world in its app to widen the intra and extra urban mobility offerings to include vehicles from additional companies with

Read more
FEATURED IoT STORIES

9 IoT applications that will change everything

Posted on: September 1, 2021

Whether you are a future-minded CEO, tech-driven CEO or IT leader, you’ve come across the term IoT before. It’s often used alongside superlatives regarding how it will revolutionize the way you work, play, and live. But is it just another buzzword, or is it the as-promised technological holy grail? The truth is that Internet of

Read more

Which IoT Platform 2021? IoT Now Enterprise Buyers’ Guide

Posted on: August 30, 2021

There are several different parts in a complete IoT solution, all of which must work together to get the result needed, write IoT Now Enterprise Buyers’ Guide – Which IoT Platform 2021? authors Robin Duke-Woolley, the CEO and Bill Ingle, a senior analyst, at Beecham Research. Figure 1 shows these parts and, although not all

Read more

CAT-M1 vs NB-IoT – examining the real differences

Posted on: June 21, 2021

As industry players look to provide the next generation of IoT connectivity, two different standards have emerged under release 13 of 3GPP – CAT-M1 and NB-IoT.

Read more

IoT and home automation: What does the future hold?

Posted on: June 10, 2020

Once a dream, home automation using iot is slowly but steadily becoming a part of daily lives around the world. In fact, it is believed that the global market for smart home automation will reach $40 billion by 2020.

Read more

5 challenges still facing the Internet of Things

Posted on: June 3, 2020

The Internet of Things (IoT) has quickly become a huge part of how people live, communicate and do business. All around the world, web-enabled devices are turning our world into a more switched-on place to live.

Read more

What is IoT?

Posted on: July 7, 2019

What is IoT Data as a new oil IoT connectivity What is IoT video So what’s IoT? The phrase ‘Internet of Things’ (IoT) is officially everywhere. It constantly shows up in my Google news feed, the weekend tech supplements are waxing lyrical about it and the volume of marketing emails I receive advertising ‘smart, connected

Read more
IoT Newsletter

Join the IoT Now online community for FREE, to receive: Exclusive offers for entry to all the IoT events that matter, round the world

Free access to a huge selection of the latest IoT analyst reports and industry whitepapers

The latest IoT news, as it breaks, to your inbox