Hazelcast Jet 0.5 – New high-level API increases developer productivity for real-time IoT, big data applications

Greg Luck, CEO of Hazelcast

Hazelcast, the open source in-memory data grid (IMDG) with tens of thousands of installed clusters and over 39 million server starts per month, announced the 0.5 release of Hazelcast Jet – an application embeddable, distributed computing platform for fast processing of big data sets.

New functionality in Hazelcast Jet 0.5 includes the Pipeline API for general purpose programming of batch and stream processing, and fault tolerance using snapshotting with the integrated Hazelcast IMDG.

The overall focus of the latest release is to increase developer productivity via an extremely simple and intuitive API. Jet is a single library with no dependencies which is therefore easily embedded and deployed, removing the need for multiple systems. Typical application use cases include online trades, sensor updates in IoT architectures, real-time fraud detection, system log events, in-store e-commerce systems and social media platforms.

The Pipeline API is the primary programming interface of Hazelcast Jet for batch and stream processing, making it more appealing to a wider Java audience. This is a major enhancement to the Hazelcast Jet low-level Core API which uses directed acyclic graphs (DAG) to model data flow – allowing detailed DAG assembly of processing jobs.

The new Pipeline API is easier to use and provides developers with tools to compose batch computations from building blocks such as filters, aggregators and joiners. The Java 8 Stream API is also available in Hazelcast Jet 0.5, a well-known and popular API in the Java community which supports functional-style operations on streams of elements. The key point is that ANY Java developer will find the new Pipeline API familiar and productive.

Greg Luck, CEO of Hazelcast, said: “Since its first release Jet has put the Fast in Fast Big Data with performance up to 15 times faster than Spark and Flink. In this release we have been working on bringing Hazelcast’s legendary programming simplicity to Jet, which we think we have now achieved with the Pipeline API. Programmers, start your Jet engines.”

Also new is fault tolerance using distributed in-memory snapshots – in Hazelcast Jet 0.5 snapshots are distributed across the cluster and held in multiple replicas to provide redundancy. Jet is now able to tolerate multiple faults such as node failure, network partition or job execution failure. Snapshots are periodically created and backed up.

If there is a node failure Jet uses the latest state snapshot and automatically restarts all jobs that contain the failed node as a job participant. No additional infrastructure, such as distributed file system or external snapshot storage, is necessary to ensure Hazelcast Jet is fault tolerant out of the box.

With this release, Jet is fully integrated with Hazelcast IMDG for sources, sinks and enrichment. It also has streaming as well as batch integrations. Jet benefits from being integrated with Hazelcast IMDG, accessing an elastic, in-memory storage capability. Using data already held in IMDG versus external sources gives a 5 times performance enhancement. We expect our millions of IMDG users to start leveraging Jet’s capabilities to process the data they already hold in IMDG.

Hazelcast Jet is an Apache 2 licensed open source project that performs parallel execution to enable data-intensive applications to operate in near real-time. Built on top of a one-record-per-time architecture (sometimes known as continuous operators), Hazelcast Jet processes incoming records as soon as possible, opposed to accumulating records into micro-batches, consequently lowering latency for applications.

Hazelcast Jet is extremely simple to program and deploy – and can be fully embedded for OEMs and Microservices – making it is easier for manufacturers to build and maintain next generation systems.

Hazelcast Jet Professional Support helps ensure the service-level agreement (SLA) operation of applications. It offers 24×7 service and support with code reviews, simulation testing, and access to patch-level fixes.

Hazelcast Jet benchmark comparisons with Apache Spark and Flink are provided here.

Comment on this article below or via Twitter: @IoTNow_OR @jcIoTnow

RECENT ARTICLES

Semtech enhances global connectivity with NTN support in HL78 modules

Posted on: March 29, 2024

Semtech Corporation has announced the integration of non-terrestrial network (NTN) support into its HL series LPWA modules, specifically the HL7810 and HL7812. This significant advancement showcases a leap forward in enabling uninterrupted global connectivity even amidst the most challenging conditions.

Read more

Enhance EV charging performance with cellular connectivity

Posted on: March 28, 2024

Electric vehicles (EVs) are steadily growing their market share at the expense of internal combustion engine vehicles. The growth is fuelled by several factors. Perhaps most importantly, prices for EVs have started to drop as competition in the industry is intensifying. New players and models are emerging, prompting several established EV makers to lower their

Read more
FEATURED IoT STORIES

What is IoT? A Beginner’s Guide

Posted on: April 5, 2023

What is IoT? IoT, or the Internet of Things, refers to the connection of everyday objects, or “things,” to the internet, allowing them to collect, transmit, and share data. This interconnected network of devices transforms previously “dumb” objects, such as toasters or security cameras, into smart devices that can interact with each other and their

Read more

The IoT Adoption Boom – Everything You Need to Know

Posted on: September 28, 2022

In an age when we seem to go through technology boom after technology boom, it’s hard to imagine one sticking out. However, IoT adoption, or the Internet of Things adoption, is leading the charge to dominate the next decade’s discussion around business IT. Below, we’ll discuss the current boom, what’s driving it, where it’s going,

Read more

9 IoT applications that will change everything

Posted on: September 1, 2021

Whether you are a future-minded CEO, tech-driven CEO or IT leader, you’ve come across the term IoT before. It’s often used alongside superlatives regarding how it will revolutionize the way you work, play, and live. But is it just another buzzword, or is it the as-promised technological holy grail? The truth is that Internet of

Read more

Which IoT Platform 2021? IoT Now Enterprise Buyers’ Guide

Posted on: August 30, 2021

There are several different parts in a complete IoT solution, all of which must work together to get the result needed, write IoT Now Enterprise Buyers’ Guide – Which IoT Platform 2021? authors Robin Duke-Woolley, the CEO and Bill Ingle, a senior analyst, at Beecham Research. Figure 1 shows these parts and, although not all

Read more

CAT-M1 vs NB-IoT – examining the real differences

Posted on: June 21, 2021

As industry players look to provide the next generation of IoT connectivity, two different standards have emerged under release 13 of 3GPP – CAT-M1 and NB-IoT.

Read more

IoT and home automation: What does the future hold?

Posted on: June 10, 2020

Once a dream, home automation using iot is slowly but steadily becoming a part of daily lives around the world. In fact, it is believed that the global market for smart home automation will reach $40 billion by 2020.

Read more

5 challenges still facing the Internet of Things

Posted on: June 3, 2020

The Internet of Things (IoT) has quickly become a huge part of how people live, communicate and do business. All around the world, web-enabled devices are turning our world into a more switched-on place to live.

Read more