Package | Description |
---|---|
com.hazelcast.jet |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.core |
Jet's Core API.
|
com.hazelcast.jet.core.processor |
Apache Kafka reader/writer support for Hazelcast Jet.
|
com.hazelcast.jet.core.test |
Utilities for writing tests of Core API Processors.
|
Modifier and Type | Method and Description |
---|---|
<R> ComputeStage<R> |
ComputeStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier)
Attaches to this stage a stage with a custom transform based on the
provided supplier of Core API
Processor s. |
Modifier and Type | Class and Description |
---|---|
class |
CloseableProcessorSupplier<E extends Processor & Closeable>
A
ProcessorSupplier which closes created processor instances
when the job is complete. |
Modifier and Type | Class and Description |
---|---|
class |
AbstractProcessor
Base class to implement custom processors.
|
Modifier and Type | Method and Description |
---|---|
Collection<? extends Processor> |
ProcessorSupplier.get(int count)
Called after
ProcessorSupplier.init(Context) to retrieve instances of
Processor that will be used during the execution of the Jet job. |
Modifier and Type | Method and Description |
---|---|
static ProcessorMetaSupplier |
ProcessorMetaSupplier.dontParallelize(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
Vertex |
DAG.newVertex(String name,
DistributedSupplier<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this DAG. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorSupplier |
ProcessorSupplier.of(DistributedSupplier<? extends Processor> processorSupplier)
Returns a
ProcessorSupplier which will delegate to the given
Supplier<Processor> to create all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier,
int preferredLocalParallelism)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
Constructor and Description |
---|
Vertex(String name,
DistributedSupplier<? extends Processor> processorSupplier)
Creates a vertex from a
Supplier<Processor> . |
Modifier and Type | Method and Description |
---|---|
static <T,K,A> DistributedSupplier<Processor> |
Processors.accumulateByFrameP(DistributedFunction<? super T,K> getKeyFn,
DistributedToLongFunction<? super T> getTimestampFn,
TimestampKind timestampKind,
WindowDefinition windowDef,
AggregateOperation1<? super T,A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T,K,A> DistributedSupplier<Processor> |
Processors.accumulateByKeyP(DistributedFunction<? super T,K> getKeyFn,
AggregateOperation1<? super T,A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <T,A,R> DistributedSupplier<Processor> |
Processors.accumulateP(AggregateOperation1<T,A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <T,K,A,R> DistributedSupplier<Processor> |
Processors.aggregateByKeyP(DistributedFunction<? super T,K> getKeyFn,
AggregateOperation1<? super T,A,R> aggrOp)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <T,A,R> DistributedSupplier<Processor> |
Processors.aggregateP(AggregateOperation1<T,A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <T,K,A,R> DistributedSupplier<Processor> |
Processors.aggregateToSessionWindowP(long sessionTimeout,
DistributedToLongFunction<? super T> getTimestampFn,
DistributedFunction<? super T,K> getKeyFn,
AggregateOperation1<? super T,A,R> aggrOp)
Returns a supplier of processors for a vertex that aggregates events into
session windows.
|
static <T,K,A,R> DistributedSupplier<Processor> |
Processors.aggregateToSlidingWindowP(DistributedFunction<? super T,K> getKeyFn,
DistributedToLongFunction<? super T> getTimestampFn,
TimestampKind timestampKind,
WindowDefinition windowDef,
AggregateOperation1<? super T,A,R> aggrOp)
Returns a supplier of processors for a vertex that aggregates events
into a sliding window in a single stage (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A> DistributedSupplier<Processor> |
Processors.coAccumulateByKeyP(List<DistributedFunction<?,? extends K>> getKeyFs,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <K,A,R> DistributedSupplier<Processor> |
Processors.coAggregateByKeyP(List<DistributedFunction<?,? extends K>> getKeyFs,
AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <A,R> DistributedSupplier<Processor> |
Processors.combineByKeyP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for the second-stage vertex in a
two-stage group-and-aggregate setup.
|
static <T,A,R> DistributedSupplier<Processor> |
Processors.combineP(AggregateOperation1<T,A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R> DistributedSupplier<Processor> |
Processors.combineToSlidingWindowP(WindowDefinition windowDef,
AggregateOperation1<?,A,R> aggrOp)
Returns a supplier of processors for the second-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T> DistributedSupplier<Processor> |
Processors.filterP(DistributedPredicate<T> predicate)
Returns a supplier of processors for a vertex that emits the same items
it receives, but only those that pass the given predicate.
|
static <T,R> DistributedSupplier<Processor> |
Processors.flatMapP(DistributedFunction<T,? extends Traverser<? extends R>> mapper)
Returns a supplier of processors for a vertex that applies the provided
item-to-traverser mapping function to each received item and emits all
the items from the resulting traverser.
|
static <T> DistributedSupplier<Processor> |
Processors.insertWatermarksP(DistributedToLongFunction<T> getTimestampF,
DistributedSupplier<WatermarkPolicy> newWmPolicyF,
WatermarkEmissionPolicy wmEmitPolicy)
Returns a supplier of processors for a vertex that inserts
watermark items into the stream. |
static <T,R> DistributedSupplier<Processor> |
Processors.mapP(DistributedFunction<T,R> mapper)
Returns a supplier of processors for a vertex which, for each received
item, emits the result of applying the given mapping function to it.
|
static DistributedSupplier<Processor> |
Processors.nonCooperativeP(DistributedSupplier<Processor> wrapped)
Decorates a
Supplier<Processor> into one that will declare
its processors non-cooperative. |
static DistributedSupplier<Processor> |
Processors.noopP()
Returns a supplier of processor that consumes all its input (if any) and
does nothing with it.
|
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<Map.Entry<K,V>,String> toStringFn,
DistributedPredicate<Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static DistributedSupplier<Processor> |
Processors.nonCooperativeP(DistributedSupplier<Processor> wrapped)
Decorates a
Supplier<Processor> into one that will declare
its processors non-cooperative. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<Map.Entry<K,V>,String> toStringFn,
DistributedPredicate<Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static java.util.function.Supplier<Processor> |
TestSupport.supplierFrom(ProcessorMetaSupplier supplier)
Wraps the provided
ProcessorMetaSupplier with a Supplier<Processor> that returns processors obtained from it. |
static java.util.function.Supplier<Processor> |
TestSupport.supplierFrom(ProcessorSupplier supplier)
Wraps the provided
ProcessorSupplier with a Supplier<Processor> that returns processors obtained from it. |
Modifier and Type | Method and Description |
---|---|
static TestSupport |
TestSupport.verifyProcessor(Processor processor) |
Modifier and Type | Method and Description |
---|---|
static TestSupport |
TestSupport.verifyProcessor(java.util.function.Supplier<Processor> supplier) |
Copyright © 2017 Hazelcast, Inc.. All Rights Reserved.