Package com.hazelcast.jet.config

Enum Class ProcessingGuarantee

java.lang.Object
java.lang.Enum<ProcessingGuarantee>
com.hazelcast.jet.config.ProcessingGuarantee
All Implemented Interfaces:
Serializable, Comparable<ProcessingGuarantee>, Constable
public enum ProcessingGuarantee extends Enum<ProcessingGuarantee>
Defines what message processing guarantees are given under failure conditions. Specifically, if a member of the cluster leaves the cluster during job execution and the job is restarted automatically it defines the semantics of at which point in the stream the job is resumed from.

When AT_LEAST_ONCE or EXACTLY_ONCE is set, distributed snapshotting will be enabled for the job. The distributed snapshot algorithm works by sending barriers down the stream which upon receiving causes the processors to save their state as a snapshot. Snapshots are saved in memory and replicated across the cluster.

Since a processor can have multiple inputs, it must wait until the barrier is received from all inputs before taking a snapshot. The difference between AT_LEAST_ONCE and EXACTLY_ONCE is that in AT_LEAST_ONCE mode the processor can continue to process items from inputs which have already received the barrier. This will result in lower latency and higher throughput overall, with the caveat that some items may be processed twice after a restart.

Since:
Jet 3.0

  • Enum Constant Details

    • NONE

      public static final ProcessingGuarantee NONE
      No processing guarantees are given and no snapshots are taken during job execution. When a job is restarted automatically it will be as if the job is starting from scratch which can cause items to be lost or duplicated.

      This option provides the overall best throughput and latency and no storage overheads from snapshotting. However, it doesn't provide any correctness guarantees under failure.

    • AT_LEAST_ONCE

      public static final ProcessingGuarantee AT_LEAST_ONCE
      Enables at-least-once processing semantics. When a job is restarted it will be resumed from the latest available snapshot. Items which have been processed before the snapshot might be processed again after the job is resumed.

      This option requires in-memory snapshotting which will cause additional storage requirements and overhead compared to NONE. However, it provides better latency than EXACTLY_ONCE with weaker guarantees.

    • EXACTLY_ONCE

      public static final ProcessingGuarantee EXACTLY_ONCE
      Enables exactly-once processing semantics. When a job is restarted it will be resumed from the latest available snapshot. Items which have been processed before the snapshot are guaranteed not to be processed again after the job is resumed.

      This option requires in-memory snapshotting which will cause additional storage requirements and overhead compared to NONE. It provides the strongest correctness guarantee. However, latency might increase due to the aligning of barriers which are required in this processing mode.

  • Method Details

    • values

      public static ProcessingGuarantee[] values()
      Returns an array containing the constants of this enum class, in the order they are declared.
      Returns:
      an array containing the constants of this enum class, in the order they are declared

    • valueOf

      public static ProcessingGuarantee valueOf(String name)
      Returns the enum constant of this class with the specified name. The string must match exactly an identifier used to declare an enum constant in this class. (Extraneous whitespace characters are not permitted.)
      Parameters:
      name - the name of the enum constant to be returned.
      Returns:
      the enum constant with the specified name
      Throws:
      IllegalArgumentException - if this enum class has no constant with the specified name
      NullPointerException - if the argument is null