logo dark

Hazelcast Jet Management Center Reference Manual

Version 0.7

Welcome to the Hazelcast Jet Management Center Manual. This manual includes information on how to use Hazelcast Jet Management Center.

Hazelcast Jet Management Center enables you to monitor and manage your cluster members running Hazelcast Jet. In addition to monitoring the overall state of your clusters, you can also analyze and troubleshoot your pipelines in detail and manage their lifecycle.

Summary of Contents

  1. Get Started leads you through the steps to get Hazelcast Jet Management Center up and running on your machine.

  2. Configuration explains how to configure Hazelcast Jet Management Center. You can configure licensing, the underlying Hazelcast Jet Client, security, etc.

  3. User Interface Overview explains the user interface of the Hazelcast Jet Management Center.

1. Get Started

In this section we’ll get you started using Hazelcast Jet Management Center. We’ll show you how to set up a running instance of Hazelcast Jet Management Center.

1.1. Requirements

You need Java Runtime Environment 1.8 or later for running Hazelcast Jet Management Center.

1.2. Starting the Hazelcast Jet Management Center

You can start the Hazelcast Jet Management Center from the command line.

Here are the steps.

  • Unpack the received Hazelcast Jet Management Center ZIP distribution. The ZIP contains the hazelcast-jet-management-center-0.7.jar file under the directory hazelcast-jet-management-center-0.7.

  • You can directly start hazelcast-jet-management-center-0.7.jar file from the command line. The following command will start Hazelcast Jet Management Center on port 8081(http://localhost:8081/). Refer to the configuration section to run it on the custom port.

java -jar hazelcast-jet-management-center-0.7.jar

1.3. Licensing

Hazelcast Jet Management Center can be used without a license if the cluster that you want to monitor has at most 1 members.

When starting Hazelcast Jet Management Center from the command line, a license can be provided using the system property jet.licenseKey. For example by using the command line parameter:

java -Djet.licenseKey=YOUR_LICENSE_KEY -jar {mc-jar-file}

When you try to connect to a cluster that has more than 2 members without entering a license key or if your license key is expired, the following warning message is shown at the top.

License Warning

2. Configuration

Depending on the deployment you can configure Hazelcast Jet Management Center via command line system properties by providing address, group name and password of the cluster. Then Hazelcast Jet Management Center will connect to your cluster to start monitoring.

If you’d like more control over how Hazelcast Jet Management Center should interact with your cluster then you can provide a fully-featured Hazelcast Jet Client XML Configuration file to Hazelcast Jet Management Center.

2.1. How to pass command-line system properties

To provide license key or Hazelcast Jet Client configuration file, following properties are used;

jet.licenseKey

License key for Hazelcast Jet Management Center.
Example: -Djet.licenseKey=YOUR_LICENSE_KEY

jet.clientXml

Hazelcast Jet Client XML Configuration file.
Example: -Djet.clientXml=/path/to/client/config.xml

To start Hazelcast Jet Management Center with the properties we’ve defined above, run the following:

java -Djet.licenseKey=YOUR_LICENSE_KEY -Djet.clientXml=/path/to/client/config.xml -jar hazelcast-jet-management-center-0.7.jar

2.2. How to pass Hazelcast Jet Client XML Configuration file

You can pass a hazelcast-jet-client.xml file, which should contain group and network details, to Hazelcast Jet Management Center to able to connect to the Hazelcast Jet cluster.

An example configuration, looks like the following, contains information about the Hazelcast Jet Cluster:

<hazelcast-client xsi:schemaLocation="http://www.hazelcast.com/schema/client-config hazelcast-client-config-3.10.xsd"
                  xmlns="http://www.hazelcast.com/schema/client-config"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <group>
        <name>jet</name>
        <password>jet-pass</password>
    </group>
    <network>
    <cluster-members>
      <address>10.1.1.21</address>
      <address>10.1.1.22:5703</address>
    </cluster-members>
    </network>
</hazelcast-client>

After you’ve created the Hazelcast Jet Client XML Configuration file, then it can be passed to the Hazelcast Jet Management Center at start-up time via jet.clientXml system property like following:

java -jar hazelcast-jet-management-center-0.7.jar -Djet.clientXml=/path/to/client.xml

2.2.1. Security Configuration

Please refer to the Hazelcast IMDG reference manual for how to configure Hazelcast Jet Client inside the Hazelcast Jet Management Center to use TLS/SSL while communicating with the Hazelcast Jet cluster.

3. User Interface Overview

When the application initially loaded Dashboard appears as shown below.

Dashboard

This page provides the fundamental information about the cluster and the jobs which are explained in detail in the Dashboard section.

The Application includes a menu on the left which acts as a main way of navigation between pages,

Side Menu

Below is the list of menu items with links to their explanations.

  • Jobs - List of jobs

  • Cluster Overview - List of cluster members

  • Documentation - Link to the Hazelcast Jet Documentation

  • Helpdesk - Link to the Hazelcast Jet Support Site.

The Application includes a connectivity indicator on the bottom left, which shows connectivity status between Hazelcat Jet Management Center and Hazelcast Jet Cluster.

Connectivity Indicator
  • cluster: group name of the Hazelcast Jet cluster

  • version: version of the Hazelcast Jet cluster

  • status indicator: indicates whether connected or not.

3.3. Dashboard

Main dashboard provides general overview of the state of the Hazelcast Jet cluster and jobs.

Dashboard

The metrics you can observe on this screen are listed below;

Cluster Summary

Shows a summary of the cluster by providing following metrics;

Running Jobs

Shows the list of actively running jobs in the cluster.

  • total records in: total number of records read from the sources of the jobs

  • total records out: total number of records written to the sinks of the jobs

  • name: name/id of the job

  • start time: start time of the job.

  • up time: how long the job is running.

  • records in: number of records read from the source of this job

  • records out: number of records written to the sink of this job

Completed Jobs

Shows an expandable/collapsable of list of completed jobs in the cluster with the following information;

  • name: name/id of the job

  • completion time: when the job is completed

  • duration: how long the job is run.

Failed Jobs

Shows an list of failed jobs in the cluster with the following information;

  • name: name/id of the job

  • time: time of the failure

  • description: reason of the failure or the exception.

3.4. Cluster Overview

Cluster Overview screen provides an overview of the state of the Hazelcast Jet cluster members.

Cluster Overview

The metrics you can observe on this screen are listed below;

  • ip address: ip address of the cluster member

  • member uuid: unique member identifier within the cluster

  • cpu usage: cpu load percentage reported by the JVM

  • memory usage: used and available heap memory information

You can click on any of the cluster members and see a detailed view of them.

3.5. Member Details

Member Detailed screen provides detailed and historical information about resource usages and garbage collections about the cluster members.

Member Details

The metrics you can observe on this screen are listed below;

Member Details
  • ip address: ip address of the cluster member

  • member uuid: unique member identifier within the cluster

Resource Usage

Shows resource usage information of the member by providing following metrics;

  • uptime: number of nodes in the cluster

  • cpu count: number of available cpu cores on member reported by the JVM

  • cpu usage: cpu load percentage reported by the JVM

  • heap usage: used and available heap memory information

Garbage Collection

Shows garbage collection information of the member by providing following metrics;

  • gc major: number of major (stop-the-world) garbage collections

  • gc major time: total duration of major (stop-the-world) garbage collections

  • gc minor: number of minor garbage collections

  • gc minor time: total duration of minor garbage collections

CPU Usage

Shows the cpu load for the last 5 minutes.

Memory Usage

Shows the used memory (the amount of memory used by the member) and total memory(the amount of memory currently available to the JVM) usage for the last 5 minutes.

3.6. Jobs

Jobs screen provides general overview of the jobs in the Hazelcast Jet cluster.

Jobs
Running Jobs

Shows the list of actively running jobs in the cluster.

  • total records in: total number of records read from the sources of the jobs

  • total records out: total number of records written to the sinks of the jobs

  • name: name/id of the job

  • start time: start time of the job.

  • up time: how long the job is running.

  • records in: number of records read from the source of this job

  • records out: number of records written to the sink of this job

Completed Jobs

Shows an expandable/collapsable of list of completed jobs in the cluster with the following information;

  • name: name/id of the job

  • completion time: when the job is completed

  • duration: how long the job is run.

3.7. Job Details

Job detail screen is a tool for diagnosing data flow within the job. It provides graphical visualization of the stages, ability to manage lifecycle of the job and allows you to peek into dataflow stats across the DAG. One can diagnose bottlenecks this way.

Job Details
Job Details
  • name: name/id of the job

  • up time: how long the job is running.

Records Flow

Shows the list of actively running jobs in the cluster.

  • total in: total number of records read from the source of the job

  • total out: total number of records written to the sink of the job

  • last min in: number of records read from the source of the job in the last minute

  • last min out: number of records written to the sink of the job in the last minute

Nodes
  • name: name/id of the job

Last Successful Snapshot
  • completion: latest successful snapshot completion time

  • size: the size of the snapshot

  • duration: how long it took to create a snapshot

  • mode: processing guarantee mode of the job, either None, At Least Once or Exactly Once

Job Visualization

Graphical representation of the job topology.

DAG Visualization

Vertex Details

Shows information about the selected vertex on the Job Visualization section. Vertex Details

Parallelism
  • local: number of processors running for that vertex on each member

  • global: total number of processors running for that vertex on cluster.

Incoming Records

Lists all of the incoming edges by their source vertices and shows the following metrics and totals for each of them.

  • all-time: total number of records received by this vertex

  • last min in: number of records received by this vertex in the last minute

Outgoing Records

Lists all of the outgoing edges by their target vertices and shows the following metrics and totals for each of them.

  • all-time: total number of records sent by this vertex

  • last min in: number of records sent by this vertex in the last minute

Processors

Lists all of the processors this vertex has in the cluster and shows the following metrics for each of them.

  • queue size: current size of the processor inbox queue

  • queue cap: capacity of the processor inbox queue

  • queue cap usage: queue utilization percentage

  • records in: total number of records received by this processor

  • records out: total number of records sent by this processor

Edge Details

Shows information about the selected edge on the Job Visualization section. Edge Details

Records Flow
  • total: total number of records passed through this edge

  • last min in: number of records passed through this edge in the last minute