com.hazelcast.jet.pipeline

Class SourceBuilder.FaultTolerant<B,S>

java.lang.Object

com.hazelcast.jet.pipeline.SourceBuilder.FaultTolerant<B,S>

Type Parameters:

B - type of the builder this sub-builder was created from

S - type of the object saved to the state snapshot

Enclosing class:

SourceBuilder<C>

public final class SourceBuilder.FaultTolerant<B,S>
extends Object

Represents a step in building a custom source where you add a restoreSnapshotFn(com.hazelcast.function.BiConsumerEx<? super C, ? super java.util.List<S>>) after adding a createSnapshotFn.

Since:

Jet 3.1

Method Summary

Modifier and Type	Method and Description
B	restoreSnapshotFn(BiConsumerEx<? super C,? super List<S>> restoreSnapshotFn) Sets the function that restores the source's state from a snapshot.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

restoreSnapshotFn

@Nonnull
public B restoreSnapshotFn(@Nonnull
                                    BiConsumerEx<? super C,? super List<S>> restoreSnapshotFn)

Sets the function that restores the source's state from a snapshot.

When Jet is restarting a job after it was interrupted (failure or other reasons), it first initializes your source as if starting a new job and then passes the snapshot object (the one it got from your createSnapshotFn) to this function. Then it starts calling fillBufferFn, which must resume emitting the stream from the same item it was about to emit when the snapshot was taken.

If your source is not distributed, the `List` in the second argument contains exactly 1 element; it is safe to use `get(0)` on it. If your source is distributed, the list will contain objects returned by createSnapshotFn in all parallel instances. This is why restoreSnapshotFn accepts a list of snapshot objects. It should figure out which part of the snapshot data pertains to it and it can do so as explained here.

Parameters:

restoreSnapshotFn - the function to restore the state into the context. It must be stateless.

Since:

Jet 3.1