Package | Description |
---|---|
com.hazelcast.jet.core |
Jet's Core API.
|
com.hazelcast.jet.core.processor |
Contains static utility classes with factories of Jet processors.
|
com.hazelcast.jet.core.test |
Utilities for writing tests of Core API Processors.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.pipeline.test |
This package contains various mock sources to help with pipeline testing
and development.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractProcessor
Base class to implement custom processors.
|
Modifier and Type | Method and Description |
---|---|
Collection<? extends Processor> |
ProcessorSupplier.get(int count)
Called after
ProcessorSupplier.init(Context) to retrieve instances of
Processor that will be used during the execution of the Jet job. |
Modifier and Type | Method and Description |
---|---|
Vertex |
DAG.newUniqueVertex(String namePrefix,
SupplierEx<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this
DAG. |
Vertex |
DAG.newVertex(String name,
SupplierEx<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this DAG. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(int preferredLocalParallelism,
SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorSupplier |
ProcessorSupplier.of(SupplierEx<? extends Processor> processorSupplier)
Returns a
ProcessorSupplier which will delegate to the given
Supplier<Processor> to create all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.preferLocalParallelismOne(Permission permission,
SupplierEx<? extends Processor> procSupplier)
Variant of
ProcessorMetaSupplier.preferLocalParallelismOne(SupplierEx) where the
processor requires given permission to run. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.preferLocalParallelismOne(SupplierEx<? extends Processor> procSupplier)
Variant of
ProcessorMetaSupplier.preferLocalParallelismOne(ProcessorSupplier) where
the supplied SupplierEx<Processor> will be
wrapped into a ProcessorSupplier . |
Constructor and Description |
---|
Vertex(String name,
SupplierEx<? extends Processor> processorSupplier)
Creates a vertex from a
Supplier<Processor> . |
Modifier and Type | Method and Description |
---|---|
static <K,A> SupplierEx<Processor> |
Processors.accumulateByFrameP(List<FunctionEx<?,? extends K>> keyFns,
List<ToLongFunctionEx<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A> SupplierEx<Processor> |
Processors.accumulateByKeyP(List<FunctionEx<?,? extends K>> getKeyFns,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> SupplierEx<Processor> |
Processors.accumulateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the
accumulation step of the provided aggregate operation on all the items
it receives.
|
static <K,A,R,OUT> |
Processors.aggregateByKeyP(List<FunctionEx<?,? extends K>> keyFns,
AggregateOperation<A,R> aggrOp,
BiFunctionEx<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <A,R> SupplierEx<Processor> |
Processors.aggregateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.aggregateToSessionWindowP(long sessionTimeout,
long earlyResultsPeriod,
List<ToLongFunctionEx<?>> timestampFns,
List<FunctionEx<?,? extends K>> keyFns,
AggregateOperation<A,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events into
session windows.
|
static <K,A,R,OUT> |
Processors.aggregateToSlidingWindowP(List<FunctionEx<?,? extends K>> keyFns,
List<ToLongFunctionEx<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
long earlyResultsPeriod,
AggregateOperation<A,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events
into a sliding window in a single stage (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A,R,OUT> |
Processors.combineByKeyP(AggregateOperation<A,R> aggrOp,
BiFunctionEx<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> SupplierEx<Processor> |
Processors.combineP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the
combining and finishing steps of the provided aggregate operation.
|
static <K,A,R,OUT> |
Processors.combineToSlidingWindowP(SlidingWindowPolicy winPolicy,
AggregateOperation<A,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T> SupplierEx<Processor> |
Processors.filterP(PredicateEx<? super T> filterFn)
Returns a supplier of processors for a vertex that emits the same items
it receives, but only those that pass the given predicate.
|
static <T,R> SupplierEx<Processor> |
Processors.flatMapP(FunctionEx<? super T,? extends Traverser<? extends R>> flatMapFn)
Returns a supplier of processors for a vertex that applies the provided
item-to-traverser mapping function to each received item and emits all
the items from the resulting traverser.
|
static <T,K,S,R> SupplierEx<Processor> |
Processors.flatMapStatefulP(long ttl,
FunctionEx<? super T,? extends K> keyFn,
ToLongFunctionEx<? super T> timestampFn,
Supplier<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> statefulFlatMapFn,
TriFunction<? super S,? super K,? super Long,? extends Traverser<R>> onEvictFn)
Returns a supplier of processors for a vertex that performs a stateful
flat-mapping of its input.
|
static <T> SupplierEx<Processor> |
Processors.insertWatermarksP(EventTimePolicy<? super T> eventTimePolicy)
Returns a supplier of processors for a vertex that inserts
watermark items into the stream. |
static <T,R> SupplierEx<Processor> |
Processors.mapP(FunctionEx<? super T,? extends R> mapFn)
Returns a supplier of processors for a vertex which, for each received
item, emits the result of applying the given mapping function to it.
|
static <T,K,S,R> SupplierEx<Processor> |
Processors.mapStatefulP(long ttl,
FunctionEx<? super T,? extends K> keyFn,
ToLongFunctionEx<? super T> timestampFn,
Supplier<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> statefulMapFn,
TriFunction<? super S,? super K,? super Long,? extends R> onEvictFn)
Returns a supplier of processors for a vertex that performs a stateful
mapping of its input.
|
static SupplierEx<Processor> |
Processors.noopP()
Returns a supplier of a processor that swallows all its normal input (if
any), does nothing with it, forwards the watermarks, produces no output
and completes immediately.
|
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(FunctionEx<T,? extends CharSequence> toStringFn,
PredicateEx<T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(SupplierEx<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(FunctionEx<? super T,? extends CharSequence> toStringFn,
PredicateEx<? super T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(SupplierEx<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(FunctionEx<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
PredicateEx<? super Map.Entry<K,V>> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(SupplierEx<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <T> SupplierEx<Processor> |
Processors.sortP(Comparator<T> comparator)
Returns a supplier of processors for a vertex that sorts its input using
a
PriorityQueue and emits it in the complete phase. |
static <W,T> SupplierEx<Processor> |
SinkProcessors.writeBufferedP(FunctionEx<? super Processor.Context,? extends W> createFn,
BiConsumerEx<? super W,? super T> onReceiveFn,
ConsumerEx<? super W> flushFn)
Shortcut for
SinkProcessors.writeBufferedP(FunctionEx, BiConsumerEx,
ConsumerEx, ConsumerEx) with a no-op destroyFn . |
static <W,T> SupplierEx<Processor> |
SinkProcessors.writeBufferedP(FunctionEx<? super Processor.Context,? extends W> createFn,
BiConsumerEx<? super W,? super T> onReceiveFn,
ConsumerEx<? super W> flushFn,
ConsumerEx<? super W> destroyFn)
Returns a supplier of processors for a vertex that drains all the items
from the inbox to an internal writer object and then does a flush.
|
Modifier and Type | Method and Description |
---|---|
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(FunctionEx<T,? extends CharSequence> toStringFn,
PredicateEx<T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(SupplierEx<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(FunctionEx<? super T,? extends CharSequence> toStringFn,
PredicateEx<? super T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(SupplierEx<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(FunctionEx<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
PredicateEx<? super Map.Entry<K,V>> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(SupplierEx<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorMetaSupplier supplier)
Wraps the provided
ProcessorMetaSupplier with a Supplier<Processor> that returns processors obtained from it. |
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorMetaSupplier supplier,
ProcessorSupplier.Context context)
Wraps the provided
ProcessorMetaSupplier with a Supplier<Processor> that returns processors obtained from it. |
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorSupplier supplier)
Wraps the provided
ProcessorSupplier with a Supplier<Processor> that returns processors obtained from it. |
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorSupplier supplier,
ProcessorSupplier.Context context)
Wraps the provided
ProcessorSupplier with a Supplier<Processor> that returns processors obtained from it. |
Modifier and Type | Method and Description |
---|---|
static TestSupport |
TestSupport.verifyProcessor(SupplierEx<Processor> supplier) |
Modifier and Type | Method and Description |
---|---|
default <R> StreamStage<R> |
StreamStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
default <R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
<R> GeneralStage<R> |
GeneralStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
<R> GeneralStage<R> |
GeneralStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
default <R> BatchStage<R> |
BatchStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
default <R> BatchStage<R> |
BatchStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
Modifier and Type | Class and Description |
---|---|
class |
LongStreamSourceP
Implements the
TestSources.longStream(long, long) source. |
class |
ParallelBatchP<T>
Implements a mock batch source with parallelism.
|
class |
ParallelStreamP<T>
Implements a mock stream source with parallelism.
|
Copyright © 2023 Hazelcast, Inc.. All rights reserved.