Package | Description |
---|---|
com.hazelcast.jet.core |
Jet's Core API.
|
com.hazelcast.jet.core.processor |
Contains static utility classes with factories of Jet processors.
|
com.hazelcast.jet.core.test |
Utilities for writing tests of Core API Processors.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractProcessor
Base class to implement custom processors.
|
Modifier and Type | Method and Description |
---|---|
Collection<? extends Processor> |
ProcessorSupplier.get(int count)
Called after
ProcessorSupplier.init(Context) to retrieve instances of
Processor that will be used during the execution of the Jet job. |
Modifier and Type | Method and Description |
---|---|
Vertex |
DAG.newVertex(String name,
DistributedSupplier<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this DAG. |
static ProcessorSupplier |
ProcessorSupplier.of(DistributedSupplier<? extends Processor> processorSupplier)
Returns a
ProcessorSupplier which will delegate to the given
Supplier<Processor> to create all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier,
int preferredLocalParallelism)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.preferLocalParallelismOne(DistributedSupplier<? extends Processor> procSupplier)
Variant of
ProcessorMetaSupplier.preferLocalParallelismOne(ProcessorSupplier) where
the supplied DistributedSupplier<Processor> will be
wrapped into a ProcessorSupplier . |
Constructor and Description |
---|
Vertex(String name,
DistributedSupplier<? extends Processor> processorSupplier)
Creates a vertex from a
Supplier<Processor> . |
Modifier and Type | Method and Description |
---|---|
static <K,A> DistributedSupplier<Processor> |
Processors.accumulateByFrameP(List<DistributedFunction<?,? extends K>> keyFns,
List<DistributedToLongFunction<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A> DistributedSupplier<Processor> |
Processors.accumulateByKeyP(List<DistributedFunction<?,? extends K>> getKeyFns,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> DistributedSupplier<Processor> |
Processors.accumulateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.aggregateByKeyP(List<DistributedFunction<?,? extends K>> keyFns,
AggregateOperation<A,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <A,R> DistributedSupplier<Processor> |
Processors.aggregateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.aggregateToSessionWindowP(long sessionTimeout,
List<DistributedToLongFunction<?>> timestampFns,
List<DistributedFunction<?,? extends K>> keyFns,
AggregateOperation<A,R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events into
session windows.
|
static <K,A,R,OUT> |
Processors.aggregateToSlidingWindowP(List<DistributedFunction<?,? extends K>> keyFns,
List<DistributedToLongFunction<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
AggregateOperation<A,R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events
into a sliding window in a single stage (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A,R,OUT> |
Processors.combineByKeyP(AggregateOperation<A,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> DistributedSupplier<Processor> |
Processors.combineP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.combineToSlidingWindowP(SlidingWindowPolicy winPolicy,
AggregateOperation<A,R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T> DistributedSupplier<Processor> |
Processors.filterP(DistributedPredicate<T> filterFn)
Returns a supplier of processors for a vertex that emits the same items
it receives, but only those that pass the given predicate.
|
static <T,R> DistributedSupplier<Processor> |
Processors.flatMapP(DistributedFunction<T,? extends Traverser<? extends R>> flatMapFn)
Returns a supplier of processors for a vertex that applies the provided
item-to-traverser mapping function to each received item and emits all
the items from the resulting traverser.
|
static <T> DistributedSupplier<Processor> |
Processors.insertWatermarksP(WatermarkGenerationParams<? super T> wmGenParams)
Returns a supplier of processors for a vertex that inserts
watermark items into the stream. |
static <T,R> DistributedSupplier<Processor> |
Processors.mapP(DistributedFunction<T,R> mapFn)
Returns a supplier of processors for a vertex which, for each received
item, emits the result of applying the given mapping function to it.
|
static DistributedSupplier<Processor> |
Processors.noopP()
Returns a supplier of processor that swallows all its input (if any) and
does nothing with it and produces no output.
|
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,? extends CharSequence> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<? super T,? extends CharSequence> toStringFn,
DistributedPredicate<? super T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
DistributedPredicate<? super Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <T,K,A,R,OUT> |
Processors.rollingAggregateP(DistributedFunction<? super T,? extends K> keyFn,
AggregateOperation1<? super T,A,? extends R> aggrOp,
DistributedTriFunction<? super T,? super K,? super R,? extends OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that performs a rolling
aggregation.
|
static <W,T> DistributedSupplier<Processor> |
SinkProcessors.writeBufferedP(DistributedFunction<? super Processor.Context,? extends W> createFn,
DistributedBiConsumer<? super W,? super T> onReceiveFn,
DistributedConsumer<? super W> flushFn)
Shortcut for
SinkProcessors.writeBufferedP(DistributedFunction,
DistributedBiConsumer, DistributedConsumer, DistributedConsumer) with
a no-op destroyFn . |
static <W,T> DistributedSupplier<Processor> |
SinkProcessors.writeBufferedP(DistributedFunction<? super Processor.Context,? extends W> createFn,
DistributedBiConsumer<? super W,? super T> onReceiveFn,
DistributedConsumer<? super W> flushFn,
DistributedConsumer<? super W> destroyFn)
Returns a supplier of processors for a vertex that drains all the items
from the inbox to an internal writer object and then does a flush.
|
Modifier and Type | Method and Description |
---|---|
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,? extends CharSequence> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<? super T,? extends CharSequence> toStringFn,
DistributedPredicate<? super T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
DistributedPredicate<? super Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorMetaSupplier supplier)
Wraps the provided
ProcessorMetaSupplier with a Supplier<Processor> that returns processors obtained from it. |
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorSupplier supplier)
Wraps the provided
ProcessorSupplier with a Supplier<Processor> that returns processors obtained from it. |
Modifier and Type | Method and Description |
---|---|
static TestSupport |
TestSupport.verifyProcessor(DistributedSupplier<Processor> supplier) |
Modifier and Type | Method and Description |
---|---|
<R> StreamStage<R> |
StreamStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier) |
<R> GeneralStage<R> |
GeneralStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
<R> BatchStage<R> |
BatchStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier) |
Copyright © 2018 Hazelcast, Inc.. All rights reserved.