Package com.hazelcast.jet.pipeline

Class SourceBuilder.FaultTolerant<B,S>

java.lang.Object
com.hazelcast.jet.pipeline.SourceBuilder.FaultTolerant<B,S>
Type Parameters:
B - type of the builder this sub-builder was created from
S - type of the object saved to the state snapshot
Enclosing class:
SourceBuilder<C>
public final class SourceBuilder.FaultTolerant<B,S> extends Object
Represents a step in building a custom source where you add a restoreSnapshotFn(com.hazelcast.function.BiConsumerEx<? super C, ? super java.util.List<S>>) after adding a createSnapshotFn.
Since:
Jet 3.1

  • Method Details

    • restoreSnapshotFn

      @Nonnull public B restoreSnapshotFn(@Nonnull BiConsumerEx<? super C,? super List<S>> restoreSnapshotFn)
      Sets the function that restores the source's state from a snapshot.

      When Jet is restarting a job after it was interrupted (failure or other reasons), it first initializes your source as if starting a new job and then passes the snapshot object (the one it got from your createSnapshotFn) to this function. Then it starts calling fillBufferFn, which must resume emitting the stream from the same item it was about to emit when the snapshot was taken.

      If your source is not distributed, the `List` in the second argument contains exactly 1 element; it is safe to use `get(0)` on it. If your source is distributed, the list will contain objects returned by createSnapshotFn in all parallel instances. This is why restoreSnapshotFn accepts a list of snapshot objects. It should figure out which part of the snapshot data pertains to it and it can do so as explained here.

      Parameters:
      restoreSnapshotFn - the function to restore the state into the context. It must be stateless.
      Since:
      Jet 3.1