Uses of Interface
com.hazelcast.jet.core.Processor
Package
Description
Jet's Core API.
Contains static utility classes with factories of Jet processors.
Utilities for writing tests of Core API Processors.
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
This package contains various mock sources to help with pipeline testing
and development.
-
Uses of Processor in com.hazelcast.jet.core
Modifier and TypeClassDescriptionclass
Base class to implement custom processors.Modifier and TypeMethodDescriptionCollection<? extends Processor>
ProcessorMetaSupplier.ExpectNothingProcessorSupplier.get
(int count) Collection<? extends Processor>
ProcessorSupplier.get
(int count) Called afterProcessorSupplier.init(Context)
to retrieve instances ofProcessor
that will be used during the execution of the Jet job.Modifier and TypeMethodDescriptionDAG.newUniqueVertex
(String namePrefix, SupplierEx<? extends Processor> simpleSupplier) Creates a vertex from aSupplier<Processor>
and adds it to this DAG.DAG.newVertex
(String name, SupplierEx<? extends Processor> simpleSupplier) Creates a vertex from aSupplier<Processor>
and adds it to this DAG.static ProcessorMetaSupplier
ProcessorMetaSupplier.of
(int preferredLocalParallelism, SupplierEx<? extends Processor> procSupplier) Factory method that wraps the givenSupplier<Processor>
and uses it as the supplier of allProcessor
instances.static ProcessorMetaSupplier
ProcessorMetaSupplier.of
(SupplierEx<? extends Processor> procSupplier) Factory method that wraps the givenSupplier<Processor>
and uses it as the supplier of allProcessor
instances.static ProcessorSupplier
ProcessorSupplier.of
(SupplierEx<? extends Processor> processorSupplier) Returns aProcessorSupplier
which will delegate to the givenSupplier<Processor>
to create allProcessor
instances.static ProcessorMetaSupplier
ProcessorMetaSupplier.preferLocalParallelismOne
(SupplierEx<? extends Processor> procSupplier) Variant ofProcessorMetaSupplier.preferLocalParallelismOne(ProcessorSupplier)
where the suppliedSupplierEx<Processor>
will be wrapped into aProcessorSupplier
.static ProcessorMetaSupplier
ProcessorMetaSupplier.preferLocalParallelismOne
(Permission permission, SupplierEx<? extends Processor> procSupplier) Variant ofProcessorMetaSupplier.preferLocalParallelismOne(SupplierEx)
where the processor requires given permission to run.static ProcessorMetaSupplier
ProcessorMetaSupplier.preferLocalParallelismOne
(Permission permission, SupplierEx<? extends Processor> procSupplier, String connectorName) Variant ofProcessorMetaSupplier.preferLocalParallelismOne(SupplierEx)
where the processor requires given permission to run.ModifierConstructorDescriptionVertex
(String name, SupplierEx<? extends Processor> processorSupplier) Creates a vertex from aSupplier<Processor>
. -
Uses of Processor in com.hazelcast.jet.core.processor
Modifier and TypeMethodDescriptionstatic <K,
A> SupplierEx<Processor> Processors.accumulateByFrameP
(List<FunctionEx<?, ? extends K>> keyFns, List<ToLongFunctionEx<?>> timestampFns, TimestampKind timestampKind, SlidingWindowPolicy winPolicy, AggregateOperation<A, ?> aggrOp) Returns a supplier of processors for the first-stage vertex in a two-stage sliding window aggregation setup (see theclass Javadoc
for an explanation of aggregation stages).static <K,
A> SupplierEx<Processor> Processors.accumulateByFrameP
(List<FunctionEx<?, ? extends K>> keyFns, List<ToLongFunctionEx<?>> timestampFns, TimestampKind timestampKind, SlidingWindowPolicy winPolicy, AggregateOperation<A, ?> aggrOp, byte watermarkKey) Returns a supplier of processors for the first-stage vertex in a two-stage sliding window aggregation setup (see theclass Javadoc
for an explanation of aggregation stages).static <K,
A> SupplierEx<Processor> Processors.accumulateByKeyP
(List<FunctionEx<?, ? extends K>> getKeyFns, AggregateOperation<A, ?> aggrOp) Returns a supplier of processors for the first-stage vertex in a two-stage group-and-aggregate setup.static <A,
R> SupplierEx<Processor> Processors.accumulateP
(AggregateOperation<A, R> aggrOp) Returns a supplier of processors for a vertex that performs the accumulation step of the provided aggregate operation on all the items it receives.static <K,
A, R, OUT>
SupplierEx<Processor>Processors.aggregateByKeyP
(List<FunctionEx<?, ? extends K>> keyFns, AggregateOperation<A, R> aggrOp, BiFunctionEx<? super K, ? super R, OUT> mapToOutputFn) Returns a supplier of processors for a vertex that groups items by key and performs the provided aggregate operation on each group.static <A,
R> SupplierEx<Processor> Processors.aggregateP
(AggregateOperation<A, R> aggrOp) Returns a supplier of processors for a vertex that performs the provided aggregate operation on all the items it receives.static <K,
A, R, OUT>
SupplierEx<Processor>Processors.aggregateToSessionWindowP
(long sessionTimeout, long earlyResultsPeriod, List<ToLongFunctionEx<?>> timestampFns, List<FunctionEx<?, ? extends K>> keyFns, AggregateOperation<A, ? extends R> aggrOp, KeyedWindowResultFunction<? super K, ? super R, ? extends OUT> mapToOutputFn) Returns a supplier of processors for a vertex that aggregates events into session windows.static <K,
A, R, OUT>
SupplierEx<Processor>Processors.aggregateToSlidingWindowP
(List<FunctionEx<?, ? extends K>> keyFns, List<ToLongFunctionEx<?>> timestampFns, TimestampKind timestampKind, SlidingWindowPolicy winPolicy, long earlyResultsPeriod, AggregateOperation<A, ? extends R> aggrOp, KeyedWindowResultFunction<? super K, ? super R, ? extends OUT> mapToOutputFn) Returns a supplier of processors for a vertex that aggregates events into a sliding window in a single stage (see theclass Javadoc
for an explanation of aggregation stages).static <K,
A, R, OUT>
SupplierEx<Processor>Processors.aggregateToSlidingWindowP
(List<FunctionEx<?, ? extends K>> keyFns, List<ToLongFunctionEx<?>> timestampFns, TimestampKind timestampKind, SlidingWindowPolicy winPolicy, long earlyResultsPeriod, AggregateOperation<A, ? extends R> aggrOp, KeyedWindowResultFunction<? super K, ? super R, ? extends OUT> mapToOutputFn, byte windowWatermarkKey) Returns a supplier of processors for a vertex that aggregates events into a sliding window in a single stage (see theclass Javadoc
for an explanation of aggregation stages).static <K,
A, R, OUT>
SupplierEx<Processor>Processors.combineByKeyP
(AggregateOperation<A, R> aggrOp, BiFunctionEx<? super K, ? super R, OUT> mapToOutputFn) Returns a supplier of processors for the second-stage vertex in a two-stage group-and-aggregate setup.static <A,
R> SupplierEx<Processor> Processors.combineP
(AggregateOperation<A, R> aggrOp) Returns a supplier of processors for a vertex that performs the combining and finishing steps of the provided aggregate operation.static <K,
A, R, OUT>
SupplierEx<Processor>Processors.combineToSlidingWindowP
(SlidingWindowPolicy winPolicy, AggregateOperation<A, ? extends R> aggrOp, KeyedWindowResultFunction<? super K, ? super R, ? extends OUT> mapToOutputFn) Returns a supplier of processors for the second-stage vertex in a two-stage sliding window aggregation setup (see theclass Javadoc
for an explanation of aggregation stages).static <K,
A, R, OUT>
SupplierEx<Processor>Processors.combineToSlidingWindowP
(SlidingWindowPolicy winPolicy, AggregateOperation<A, ? extends R> aggrOp, KeyedWindowResultFunction<? super K, ? super R, ? extends OUT> mapToOutputFn, byte windowWatermarkKey) Returns a supplier of processors for the second-stage vertex in a two-stage sliding window aggregation setup (see theclass Javadoc
for an explanation of aggregation stages) with specifiedwindowWatermarkKey
.static <T> SupplierEx<Processor>
Processors.filterP
(PredicateEx<? super T> filterFn) Returns a supplier of processors for a vertex that emits the same items it receives, but only those that pass the given predicate.static <T,
R> SupplierEx<Processor> Processors.flatMapP
(FunctionEx<? super T, ? extends Traverser<? extends R>> flatMapFn) Returns a supplier of processors for a vertex that applies the provided item-to-traverser mapping function to each received item and emits all the items from the resulting traverser.static <T,
K, S, R> SupplierEx<Processor> Processors.flatMapStatefulP
(long ttl, FunctionEx<? super T, ? extends K> keyFn, ToLongFunctionEx<? super T> timestampFn, Supplier<? extends S> createFn, TriFunction<? super S, ? super K, ? super T, ? extends Traverser<R>> statefulFlatMapFn, TriFunction<? super S, ? super K, ? super Long, ? extends Traverser<R>> onEvictFn) Returns a supplier of processors for a vertex that performs a stateful flat-mapping of its input.static <T> SupplierEx<Processor>
Processors.insertWatermarksP
(FunctionEx<ProcessorSupplier.Context, EventTimePolicy<? super T>> eventTimePolicyProvider) Returns a supplier of processors for a vertex that insertswatermark items
into the stream.static <T> SupplierEx<Processor>
Processors.insertWatermarksP
(EventTimePolicy<? super T> eventTimePolicy) Returns a supplier of processors for a vertex that insertswatermark items
into the stream.static <T,
R> SupplierEx<Processor> Processors.mapP
(FunctionEx<? super T, ? extends R> mapFn) Returns a supplier of processors for a vertex which, for each received item, emits the result of applying the given mapping function to it.static <T,
K, S, R> SupplierEx<Processor> Processors.mapStatefulP
(long ttl, FunctionEx<? super T, ? extends K> keyFn, ToLongFunctionEx<? super T> timestampFn, Supplier<? extends S> createFn, TriFunction<? super S, ? super K, ? super T, ? extends R> statefulMapFn, TriFunction<? super S, ? super K, ? super Long, ? extends R> onEvictFn) Returns a supplier of processors for a vertex that performs a stateful mapping of its input.static SupplierEx<Processor>
Processors.noopP()
Returns a supplier of a processor that swallows all its normal input (if any), does nothing with it, forwards the watermarks, produces no output and completes immediately.static <T> SupplierEx<Processor>
DiagnosticProcessors.peekInputP
(FunctionEx<T, ? extends CharSequence> toStringFn, PredicateEx<T> shouldLogFn, SupplierEx<Processor> wrapped) Same aspeekInput(toStringFn, shouldLogFn, metaSupplier)
, but accepts aSupplierEx
of processors instead of a meta-supplier.static SupplierEx<Processor>
DiagnosticProcessors.peekInputP
(SupplierEx<Processor> wrapped) Convenience forpeekInput(toStringFn, shouldLogFn, metaSupplier)
with a pass-through filter andObject#toString
as the formatting function.static <T> SupplierEx<Processor>
DiagnosticProcessors.peekOutputP
(FunctionEx<? super T, ? extends CharSequence> toStringFn, PredicateEx<? super T> shouldLogFn, SupplierEx<Processor> wrapped) Same aspeekOutput(toStringFn, shouldLogFn, metaSupplier)
, but accepts aSupplierEx
of processors instead of a meta-supplier.static SupplierEx<Processor>
DiagnosticProcessors.peekOutputP
(SupplierEx<Processor> wrapped) Convenience forpeekOutput(toStringFn, shouldLogFn, metaSupplier
with a pass-through filter andObject#toString
as the formatting function.static <K,
V> SupplierEx<Processor> DiagnosticProcessors.peekSnapshotP
(FunctionEx<? super Map.Entry<K, V>, ? extends CharSequence> toStringFn, PredicateEx<? super Map.Entry<K, V>> shouldLogFn, SupplierEx<Processor> wrapped) Same aspeekSnapshot(toStringFn, shouldLogFn, metaSupplier)
, but accepts aSupplierEx
of processors instead of a meta-supplier.static SupplierEx<Processor>
DiagnosticProcessors.peekSnapshotP
(SupplierEx<Processor> wrapped) Convenience forpeekSnapshot(toStringFn, shouldLogFn, metaSupplier
with a pass-through filter andObject#toString
as the formatting function.static <T> SupplierEx<Processor>
Processors.sortP
(Comparator<T> comparator) Returns a supplier of processors for a vertex that sorts its input using aPriorityQueue
and emits it in thecomplete
phase.static <W,
T> SupplierEx<Processor> SinkProcessors.writeBufferedP
(FunctionEx<? super Processor.Context, ? extends W> createFn, BiConsumerEx<? super W, ? super T> onReceiveFn, ConsumerEx<? super W> flushFn) Shortcut forSinkProcessors.writeBufferedP(FunctionEx, BiConsumerEx, ConsumerEx, ConsumerEx)
with a no-opdestroyFn
.static <W,
T> SupplierEx<Processor> SinkProcessors.writeBufferedP
(FunctionEx<? super Processor.Context, ? extends W> createFn, BiConsumerEx<? super W, ? super T> onReceiveFn, ConsumerEx<? super W> flushFn, ConsumerEx<? super W> destroyFn) Returns a supplier of processors for a vertex that drains all the items from the inbox to an internal writer object and then does a flush.Modifier and TypeMethodDescriptionstatic <T> SupplierEx<Processor>
DiagnosticProcessors.peekInputP
(FunctionEx<T, ? extends CharSequence> toStringFn, PredicateEx<T> shouldLogFn, SupplierEx<Processor> wrapped) Same aspeekInput(toStringFn, shouldLogFn, metaSupplier)
, but accepts aSupplierEx
of processors instead of a meta-supplier.static SupplierEx<Processor>
DiagnosticProcessors.peekInputP
(SupplierEx<Processor> wrapped) Convenience forpeekInput(toStringFn, shouldLogFn, metaSupplier)
with a pass-through filter andObject#toString
as the formatting function.static <T> SupplierEx<Processor>
DiagnosticProcessors.peekOutputP
(FunctionEx<? super T, ? extends CharSequence> toStringFn, PredicateEx<? super T> shouldLogFn, SupplierEx<Processor> wrapped) Same aspeekOutput(toStringFn, shouldLogFn, metaSupplier)
, but accepts aSupplierEx
of processors instead of a meta-supplier.static SupplierEx<Processor>
DiagnosticProcessors.peekOutputP
(SupplierEx<Processor> wrapped) Convenience forpeekOutput(toStringFn, shouldLogFn, metaSupplier
with a pass-through filter andObject#toString
as the formatting function.static <K,
V> SupplierEx<Processor> DiagnosticProcessors.peekSnapshotP
(FunctionEx<? super Map.Entry<K, V>, ? extends CharSequence> toStringFn, PredicateEx<? super Map.Entry<K, V>> shouldLogFn, SupplierEx<Processor> wrapped) Same aspeekSnapshot(toStringFn, shouldLogFn, metaSupplier)
, but accepts aSupplierEx
of processors instead of a meta-supplier.static SupplierEx<Processor>
DiagnosticProcessors.peekSnapshotP
(SupplierEx<Processor> wrapped) Convenience forpeekSnapshot(toStringFn, shouldLogFn, metaSupplier
with a pass-through filter andObject#toString
as the formatting function. -
Uses of Processor in com.hazelcast.jet.core.test
Modifier and TypeMethodDescriptionstatic <T extends Processor>
TestSupport.TestEventTestSupport.processorAssertion
(Consumer<T> assertion) Create a test event with a processor assertion.Modifier and TypeMethodDescriptionTestSupport.supplierFrom
(ProcessorMetaSupplier supplier) Wraps the providedProcessorMetaSupplier
with aSupplier<Processor>
that returns processors obtained from it.TestSupport.supplierFrom
(ProcessorMetaSupplier supplier, ProcessorSupplier.Context context) Wraps the providedProcessorMetaSupplier
with aSupplier<Processor>
that returns processors obtained from it.TestSupport.supplierFrom
(ProcessorSupplier supplier) Wraps the providedProcessorSupplier
with aSupplier<Processor>
that returns processors obtained from it.TestSupport.supplierFrom
(ProcessorSupplier supplier, ProcessorSupplier.Context context) Wraps the providedProcessorSupplier
with aSupplier<Processor>
that returns processors obtained from it.Modifier and TypeMethodDescriptionstatic TestSupport
TestSupport.verifyProcessor
(SupplierEx<Processor> supplier) -
Uses of Processor in com.hazelcast.jet.pipeline
Modifier and TypeMethodDescriptiondefault <R> BatchStage<R>
BatchStage.customTransform
(String stageName, SupplierEx<Processor> procSupplier) default <R> BatchStage<R>
BatchStageWithKey.customTransform
(String stageName, SupplierEx<Processor> procSupplier) <R> GeneralStage<R>
GeneralStage.customTransform
(String stageName, SupplierEx<Processor> procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.<R> GeneralStage<R>
GeneralStageWithKey.customTransform
(String stageName, SupplierEx<Processor> procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.default <R> StreamStage<R>
StreamStage.customTransform
(String stageName, SupplierEx<Processor> procSupplier) default <R> StreamStage<R>
StreamStageWithKey.customTransform
(String stageName, SupplierEx<Processor> procSupplier) -
Uses of Processor in com.hazelcast.jet.pipeline.test
Modifier and TypeClassDescriptionclass
Implements theTestSources.longStream(long, long)
source.class
Implements a mock batch source with parallelism.class
Implements a mock stream source with parallelism.