Package | Description |
---|---|
com.hazelcast.jet.aggregate |
Contains
AggregateOperation and several its variants, as well
as a builder object for the aggregate operations. |
com.hazelcast.jet.core |
Jet's Core API.
|
com.hazelcast.jet.core.processor |
Contains static utility classes with factories of Jet processors.
|
com.hazelcast.jet.core.test |
Utilities for writing tests of Core API Processors.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.stream |
java.util.stream implementation using Hazelcast Jet
|
Modifier and Type | Method and Description |
---|---|
DistributedSupplier<A> |
AggregateOperation.createFn()
A primitive that returns a new accumulator.
|
Modifier and Type | Method and Description |
---|---|
static <T,K,R,A,M extends Map<K,R>> |
AggregateOperations.groupingBy(DistributedFunction<? super T,? extends K> toKeyFn,
DistributedSupplier<M> createMapFn,
AggregateOperation1<? super T,A,R> downstream)
Returns an
AggregateOperation1 that accumulates the items into a
Map (as obtained from createMapFn ) where the key is the
result of applying toKeyFn and the value is the result of
applying the downstream aggregate operation to the items with that key. |
static <T,C extends Collection<T>> |
AggregateOperations.toCollection(DistributedSupplier<C> createCollectionFn)
Returns an aggregate operation that accumulates the items into a
Collection . |
static <T,K,U,M extends Map<K,U>> |
AggregateOperations.toMap(DistributedFunction<? super T,? extends K> toKeyFn,
DistributedFunction<? super T,? extends U> toValueFn,
DistributedBinaryOperator<U> mergeFn,
DistributedSupplier<M> createMapFn)
Returns an
AggregateOperation1 that accumulates elements
into a Map whose keys and values are the result of applying the
provided mapping functions to the input elements. |
static <A> AggregateOperationBuilder<A> |
AggregateOperation.withCreate(DistributedSupplier<A> createFn)
Returns a builder object, initialized with the supplied
create
primitive, that can be used to construct the definition of an aggregate
operation in a step-by-step manner. |
Modifier and Type | Method and Description |
---|---|
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.limitingLag(long lag)
Maintains a watermark that lags behind the top observed timestamp by the
given amount.
|
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.limitingLagAndDelay(long lag,
long maxDelayMs)
Maintains a watermark that lags behind the top observed timestamp by at
most the given amount and is additionally guaranteed to reach the
timestamp of any given event within
maxDelayMs of system time
after observing it. |
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.limitingLagAndLull(long lag,
long maxLullMs)
Maintains watermark that lags behind the top timestamp by the amount
specified with
lag . |
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.limitingTimestampAndWallClockLag(long timestampLag,
long wallClockLag)
Maintains watermark that lags behind the top timestamp by at most
timestampLag and behind wall-clock time by at most wallClockLag . |
DistributedSupplier<WatermarkPolicy> |
WatermarkGenerationParams.newWmPolicyFn()
Returns the factory of the watermark policy objects.
|
Modifier and Type | Method and Description |
---|---|
Vertex |
DAG.newVertex(String name,
DistributedSupplier<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this DAG. |
static ProcessorSupplier |
ProcessorSupplier.of(DistributedSupplier<? extends Processor> processorSupplier)
Returns a
ProcessorSupplier which will delegate to the given
Supplier<Processor> to create all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier,
int preferredLocalParallelism)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.preferLocalParallelismOne(DistributedSupplier<? extends Processor> procSupplier)
Variant of
ProcessorMetaSupplier.preferLocalParallelismOne(ProcessorSupplier) where
the supplied DistributedSupplier<Processor> will be
wrapped into a ProcessorSupplier . |
static <T> WatermarkGenerationParams<T> |
WatermarkGenerationParams.wmGenParams(DistributedToLongFunction<? super T> timestampFn,
DistributedObjLongBiFunction<? super T,?> wrapFn,
DistributedSupplier<WatermarkPolicy> wmPolicy,
WatermarkEmissionPolicy wmEmitPolicy,
long idleTimeoutMillis)
Creates and returns new watermark generation parameters.
|
static <T> WatermarkGenerationParams<T> |
WatermarkGenerationParams.wmGenParams(DistributedToLongFunction<T> timestampFn,
DistributedSupplier<WatermarkPolicy> wmPolicy,
WatermarkEmissionPolicy wmEmitPolicy,
long idleTimeoutMillis)
Creates and returns a watermark generation parameters object.
|
Constructor and Description |
---|
Vertex(String name,
DistributedSupplier<? extends Processor> processorSupplier)
Creates a vertex from a
Supplier<Processor> . |
Modifier and Type | Method and Description |
---|---|
static <K,A> DistributedSupplier<Processor> |
Processors.accumulateByFrameP(List<DistributedFunction<?,? extends K>> keyFns,
List<DistributedToLongFunction<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A> DistributedSupplier<Processor> |
Processors.accumulateByKeyP(List<DistributedFunction<?,? extends K>> getKeyFns,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> DistributedSupplier<Processor> |
Processors.accumulateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.aggregateByKeyP(List<DistributedFunction<?,? extends K>> keyFns,
AggregateOperation<A,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <A,R> DistributedSupplier<Processor> |
Processors.aggregateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.aggregateToSessionWindowP(long sessionTimeout,
List<DistributedToLongFunction<?>> timestampFns,
List<DistributedFunction<?,? extends K>> keyFns,
AggregateOperation<A,R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events into
session windows.
|
static <K,A,R,OUT> |
Processors.aggregateToSlidingWindowP(List<DistributedFunction<?,? extends K>> keyFns,
List<DistributedToLongFunction<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
AggregateOperation<A,R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events
into a sliding window in a single stage (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A,R,OUT> |
Processors.combineByKeyP(AggregateOperation<A,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> DistributedSupplier<Processor> |
Processors.combineP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.combineToSlidingWindowP(SlidingWindowPolicy winPolicy,
AggregateOperation<A,R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T> DistributedSupplier<Processor> |
Processors.filterP(DistributedPredicate<T> filterFn)
Returns a supplier of processors for a vertex that emits the same items
it receives, but only those that pass the given predicate.
|
static <T,R> DistributedSupplier<Processor> |
Processors.flatMapP(DistributedFunction<T,? extends Traverser<? extends R>> flatMapFn)
Returns a supplier of processors for a vertex that applies the provided
item-to-traverser mapping function to each received item and emits all
the items from the resulting traverser.
|
static <T> DistributedSupplier<Processor> |
Processors.insertWatermarksP(WatermarkGenerationParams<T> wmGenParams)
Returns a supplier of processors for a vertex that inserts
watermark items into the stream. |
static <T,R> DistributedSupplier<Processor> |
Processors.mapP(DistributedFunction<T,R> mapFn)
Returns a supplier of processors for a vertex which, for each received
item, emits the result of applying the given mapping function to it.
|
static DistributedSupplier<Processor> |
Processors.nonCooperativeP(DistributedSupplier<Processor> wrapped)
Decorates a
Supplier<Processor> into one that will declare
its processors non-cooperative. |
static DistributedSupplier<Processor> |
Processors.noopP()
Returns a supplier of processor that consumes all its input (if any) and
does nothing with it.
|
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,? extends CharSequence> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<? super T,? extends CharSequence> toStringFn,
DistributedPredicate<? super T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
DistributedPredicate<? super Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <W,T> DistributedSupplier<Processor> |
SinkProcessors.writeBufferedP(DistributedFunction<? super Processor.Context,? extends W> createFn,
DistributedBiConsumer<? super W,? super T> onReceiveFn,
DistributedConsumer<? super W> flushFn,
DistributedConsumer<? super W> destroyFn)
Returns a supplier of processors for a vertex that drains all the items
from the inbox to an internal writer object and then does a flush.
|
static <W,T> DistributedSupplier<Processor> |
SinkProcessors.writeBufferedP(DistributedFunction<Processor.Context,W> createFn,
DistributedBiConsumer<W,T> onReceiveFn,
DistributedConsumer<W> flushFn)
Shortcut for
SinkProcessors.writeBufferedP(DistributedFunction,
DistributedBiConsumer, DistributedConsumer, DistributedConsumer) with
a no-op destroyFn . |
Modifier and Type | Method and Description |
---|---|
static DistributedSupplier<Processor> |
Processors.nonCooperativeP(DistributedSupplier<Processor> wrapped)
Decorates a
Supplier<Processor> into one that will declare
its processors non-cooperative. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,? extends CharSequence> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<? super T,? extends CharSequence> toStringFn,
DistributedPredicate<? super T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
DistributedPredicate<? super Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static TestSupport |
TestSupport.verifyProcessor(DistributedSupplier<Processor> supplier) |
Modifier and Type | Method and Description |
---|---|
<R> StreamStage<R> |
StreamStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier) |
<R> GeneralStage<R> |
GeneralStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier)
Attaches to this stage a stage with a custom transform based on the
provided supplier of Core API
Processor s. |
<R> BatchStage<R> |
BatchStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier) |
Modifier and Type | Method and Description |
---|---|
DistributedSupplier<A> |
DistributedCollector.supplier() |
Copyright © 2018 Hazelcast, Inc.. All rights reserved.