DMX Change Data Capture simplifies continuous delivery of fresh ‘Big Iron’ data

Syncsort, a global provider of data liberation, integrity and integration solutions for analytics, announced new capabilities in its mainframe data access and integration solution. This populates Hadoop data lakes with changes in mainframe data.

The new DMX Change Data Capture (DMX CDC) functionality delivers real-time data replication that enables organisations to continually keep Hadoop data in sync with changes made on the mainframe, and consistently make the most current Big Data information available in the Hadoop data lake for analytics.

“Many organisations are using our industry-leading big data integration solution, DMX-h, to quickly and efficiently populate their data lakes with enterprise-wide data, including complex data from the mainframe for a variety of use cases, such as Hadoop as a Service, Data as a Service, data archive in the Cloud, fraud detection, anti-money laundering and Customer 360,” said Tendü Yoğurtçu, chief technology officer, Syncsort.

“After populating the data lake, it is very important to keep that data fresh to power real-time analytics and accurate decisions based on up-to-date information. Our new CDC offering provides our customers with an easy-to-use, highly efficient solution for ensuring the data lake is refreshed in real-time with the incremental updates, while meeting SLAs and conserving network resources.”

Tendü Yoğurtçu

Syncsort already is in many large production deployments for accessing data from traditional enterprise systems and integrating it with Hadoop, including DataFunnel, its single-click solution for populating the data lake data from thousands of tables and automatically creating the corresponding metadata in Hadoop.

The new CDC capabilities extend this with real-time data replication, reducing the network load and providing up-to-the-minute mainframe data for analytics by:

    • Speeding and simplifying the process of synchronising mainframe data with Hadoop, on-premise or in the cloud
    • Saving time and resources using the DMX-h GUI and dynamic optimisations, which eliminates the need for coding and tuning
    • Eliminating impact on mainframe database performance by avoiding database triggers
    • Affording reliable data transfer – even during loss of mainframe to Hadoop connections or Hadoop cluster failures – picking-up where the transfer stopped without restarting the entire process
    • Saving money with virtually no use of chargeable mainframe CPU resources
    • Supporting IBM DB2 for z/OS and IBM z/OS VSAM files, with more sources to come
    • Enabling queries on most current data via rapid update of Hive table data and statistics
    • Handling all enterprise Hive file formats including text, ORC, Avro and Parquet

The combination of all the time savings and low resource utilisation with the new CDC facilities also helps organisations meet short SLAs with virtually no impact to MIPS costs.

Comment on this article below or via Twitter: @IoTNow OR @jcIoTnow

FEATURED IoT STORIES

9 IoT applications that will change everything

Posted on: September 1, 2021

Whether you are a future-minded CEO, tech-driven CEO or IT leader, you’ve come across the term IoT before. It’s often used alongside superlatives regarding how it will revolutionize the way you work, play, and live. But is it just another buzzword, or is it the as-promised technological holy grail? The truth is that Internet of

Read more

Which IoT Platform 2021? IoT Now Enterprise Buyers’ Guide

Posted on: August 30, 2021

There are several different parts in a complete IoT solution, all of which must work together to get the result needed, write IoT Now Enterprise Buyers’ Guide – Which IoT Platform 2021? authors Robin Duke-Woolley, the CEO and Bill Ingle, a senior analyst, at Beecham Research. Figure 1 shows these parts and, although not all

Read more

CAT-M1 vs NB-IoT – examining the real differences

Posted on: June 21, 2021

As industry players look to provide the next generation of IoT connectivity, two different standards have emerged under release 13 of 3GPP – CAT-M1 and NB-IoT.

Read more

IoT and home automation: What does the future hold?

Posted on: June 10, 2020

Once a dream, iot home automation is slowly but steadily becoming a part of daily lives around the world. In fact, it is believed that the global market for smart home automation will reach $40 billion by 2020.

Read more
RECENT ARTICLES

IoT set to overtake cloud computing as primary Industry 4.0 technology, Inmarsat research reveals

Posted on: October 14, 2021

New research by Inmarsat, the provider of global mobile satellite communications, reveals that investment in the Internet of Things (IoT) is set to overtake cloud computing, next generation security, big data analytics and other digital transformation technologies in the near future.

Read more

IDTechEx looks at the setbacks and explores how to move forward

Posted on: October 14, 2021

Bill Gates backed a Belmont smart city in the Arizona desert little has happened beyond a land purchase. Authorities demand that the Colorado river’s diminishing water supply is unharmed. Arizona suffers historic water shortage. The Southwest and much of the West is suffering from an intense 22-year drought, resulting in increasingly low water levels, dry

Read more