Hazelcast Jet 0.5 – New high-level API increases developer productivity for real-time IoT, big data applications

Greg Luck, CEO of Hazelcast

Hazelcast, the open source in-memory data grid (IMDG) with tens of thousands of installed clusters and over 39 million server starts per month, announced the 0.5 release of Hazelcast Jet – an application embeddable, distributed computing platform for fast processing of big data sets.

New functionality in Hazelcast Jet 0.5 includes the Pipeline API for general purpose programming of batch and stream processing, and fault tolerance using snapshotting with the integrated Hazelcast IMDG.

The overall focus of the latest release is to increase developer productivity via an extremely simple and intuitive API. Jet is a single library with no dependencies which is therefore easily embedded and deployed, removing the need for multiple systems. Typical application use cases include online trades, sensor updates in IoT architectures, real-time fraud detection, system log events, in-store e-commerce systems and social media platforms.

The Pipeline API is the primary programming interface of Hazelcast Jet for batch and stream processing, making it more appealing to a wider Java audience. This is a major enhancement to the Hazelcast Jet low-level Core API which uses directed acyclic graphs (DAG) to model data flow – allowing detailed DAG assembly of processing jobs.

The new Pipeline API is easier to use and provides developers with tools to compose batch computations from building blocks such as filters, aggregators and joiners. The Java 8 Stream API is also available in Hazelcast Jet 0.5, a well-known and popular API in the Java community which supports functional-style operations on streams of elements. The key point is that ANY Java developer will find the new Pipeline API familiar and productive.

Greg Luck, CEO of Hazelcast, said: “Since its first release Jet has put the Fast in Fast Big Data with performance up to 15 times faster than Spark and Flink. In this release we have been working on bringing Hazelcast’s legendary programming simplicity to Jet, which we think we have now achieved with the Pipeline API. Programmers, start your Jet engines.”

Also new is fault tolerance using distributed in-memory snapshots – in Hazelcast Jet 0.5 snapshots are distributed across the cluster and held in multiple replicas to provide redundancy. Jet is now able to tolerate multiple faults such as node failure, network partition or job execution failure. Snapshots are periodically created and backed up.

If there is a node failure Jet uses the latest state snapshot and automatically restarts all jobs that contain the failed node as a job participant. No additional infrastructure, such as distributed file system or external snapshot storage, is necessary to ensure Hazelcast Jet is fault tolerant out of the box.

With this release, Jet is fully integrated with Hazelcast IMDG for sources, sinks and enrichment. It also has streaming as well as batch integrations. Jet benefits from being integrated with Hazelcast IMDG, accessing an elastic, in-memory storage capability. Using data already held in IMDG versus external sources gives a 5 times performance enhancement. We expect our millions of IMDG users to start leveraging Jet’s capabilities to process the data they already hold in IMDG.

Hazelcast Jet is an Apache 2 licensed open source project that performs parallel execution to enable data-intensive applications to operate in near real-time. Built on top of a one-record-per-time architecture (sometimes known as continuous operators), Hazelcast Jet processes incoming records as soon as possible, opposed to accumulating records into micro-batches, consequently lowering latency for applications.

Hazelcast Jet is extremely simple to program and deploy – and can be fully embedded for OEMs and Microservices – making it is easier for manufacturers to build and maintain next generation systems.

Hazelcast Jet Professional Support helps ensure the service-level agreement (SLA) operation of applications. It offers 24×7 service and support with code reviews, simulation testing, and access to patch-level fixes.

Hazelcast Jet benchmark comparisons with Apache Spark and Flink are provided here.

Comment on this article below or via Twitter: @IoTNow_OR @jcIoTnow

FEATURED IoT STORIES

9 IoT applications that will change everything

Posted on: September 1, 2021

Whether you are a future-minded CEO, tech-driven CEO or IT leader, you’ve come across the term IoT before. It’s often used alongside superlatives regarding how it will revolutionize the way you work, play, and live. But is it just another buzzword, or is it the as-promised technological holy grail? The truth is that Internet of

Read more

Which IoT Platform 2021? IoT Now Enterprise Buyers’ Guide

Posted on: August 30, 2021

There are several different parts in a complete IoT solution, all of which must work together to get the result needed, write IoT Now Enterprise Buyers’ Guide – Which IoT Platform 2021? authors Robin Duke-Woolley, the CEO and Bill Ingle, a senior analyst, at Beecham Research. Figure 1 shows these parts and, although not all

Read more

CAT-M1 vs NB-IoT – examining the real differences

Posted on: June 21, 2021

As industry players look to provide the next generation of IoT connectivity, two different standards have emerged under release 13 of 3GPP – CAT-M1 and NB-IoT.

Read more

IoT and home automation: What does the future hold?

Posted on: June 10, 2020

Once a dream, iot home automation is slowly but steadily becoming a part of daily lives around the world. In fact, it is believed that the global market for smart home automation will reach $40 billion by 2020.

Read more
RECENT ARTICLES

Infineon simplifies secure IoT device-to-cloud authentication with CIRRENT Cloud ID service

Posted on: October 21, 2021

Munich, Germany. 21 October 2021 – Infineon Technologies AG launched CIRRENT Cloud ID, a service that automates cloud certificate provisioning and IoT device-to-cloud authentication. The easy-to-use service extends the chain of trust and makes tasks easier and more secure from chip-to-cloud, while lowering companies’ total cost of ownership. Cloud ID is ideal for cloud-connected product companies

Read more

MNOs want clearer views of network performance and user experience to generate new 5G revenues

Posted on: October 21, 2021

Quebec City, Canada – While 88% of mobile network operators (MNOs) are set to deploy 5G standalone (SA) in the next two years, many are still searching for the tools that will enable these networks to generate revenues from enterprises and industry. This is according to joint research findings from Heavy Reading and EXFO Inc., the communications industry’s

Read more