Package | Description |
---|---|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Method and Description |
---|---|
default StreamStage<T> |
GeneralStage.addTimestamps()
Adds a timestamp to each item in the stream using the current system
time.
|
StreamStage<T> |
GeneralStage.addTimestamps(DistributedToLongFunction<? super T> timestampFn,
long allowedLag)
Adds a timestamp to each item in the stream using the supplied function
and specifies the allowed amount of disorder between them.
|
<R,OUT> StreamStage<OUT> |
StageWithKeyAndWindow.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given group-and-aggregate operation
over the window described by the window definition captured by this
object.
|
<R,OUT> StreamStage<OUT> |
StageWithWindow.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
WindowResultFunction<? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over all
the items that belong to a given window.
|
default <R> StreamStage<TimestampedItem<R>> |
StageWithWindow.aggregate(AggregateOperation1<? super T,?,R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to a given window.
|
default <R> StreamStage<TimestampedEntry<K,R>> |
StageWithKeyAndWindow.aggregate(AggregateOperation1<? super T,?,R> aggrOp)
Attaches a stage that performs the given group-and-aggregate operation.
|
default <T1,R0,R1> StreamStage<TimestampedItem<Tuple2<R0,R1>>> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given co-aggregate operation over the
items from both this stage and
stage1 you supply. |
default <T1,R0,R1,OUT> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
WindowResult2Function<? super R0,? super R1,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given co-aggregate operation over the
items from both this stage and
stage1 you supply. |
default <T1,R0,R1> StreamStage<TimestampedEntry<K,Tuple2<R0,R1>>> |
StageWithKeyAndWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,R0,R1,OUT> |
StageWithKeyAndWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
KeyedWindowResult2Function<? super K,? super R0,? super R1,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,R> StreamStage<TimestampedItem<R>> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
<T1,R,OUT> StreamStage<OUT> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,R> aggrOp,
WindowResultFunction<? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,R> StreamStage<TimestampedEntry<K,R>> |
StageWithKeyAndWindow.aggregate2(StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
<T1,R,OUT> StreamStage<OUT> |
StageWithKeyAndWindow.aggregate2(StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2,OUT> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
WindowResult3Function<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithKeyAndWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,T2,R0,R1,R2,OUT> |
StageWithKeyAndWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
KeyedWindowResult3Function<? super K,? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,T2,R> StreamStage<TimestampedItem<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R,OUT> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp,
WindowResultFunction<? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
default <T1,T2,R> StreamStage<TimestampedEntry<K,R>> |
StageWithKeyAndWindow.aggregate3(StreamStageWithKey<T1,? extends K> stage1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R,OUT> |
StageWithKeyAndWindow.aggregate3(StreamStageWithKey<T1,? extends K> stage1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from this stage as well as
stage1 and stage2 you supply. |
StreamStage<TimestampedEntry<K,ItemsByTag>> |
WindowGroupAggregateBuilder.build()
Convenience for
WindowGroupAggregateBuilder.build(KeyedWindowResultFunction) which results
in a stage that emits TimestampedEntry s. |
StreamStage<TimestampedItem<ItemsByTag>> |
WindowAggregateBuilder.build()
Convenience for
build(mapToOutputFn)
which emits TimestampedItem s as output. |
<R> StreamStage<TimestampedEntry<K,R>> |
WindowGroupAggregateBuilder1.build(AggregateOperation<?,R> aggrOp)
Convenience for
WindowGroupAggregateBuilder1.build(AggregateOperation, KeyedWindowResultFunction)
which results in a stage that emits TimestampedEntry s. |
<R,OUT> StreamStage<OUT> |
WindowGroupAggregateBuilder1.build(AggregateOperation<?,R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs a windowed
cogroup-and-aggregate of the pipeline stages registered with this builder object.
|
<R,OUT> StreamStage<OUT> |
WindowAggregateBuilder1.build(AggregateOperation<?,R> aggrOp,
WindowResultFunction<? super R,? extends OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs a windowed
co-aggregation of the pipeline stages registered with this builder
object.
|
<A,R> StreamStage<TimestampedItem<R>> |
WindowAggregateBuilder1.build(AggregateOperation<A,R> aggrOp)
Convenience for
build(aggrOp, mapToOutputFn) which emits TimestampedItem s as output. |
<R> StreamStage<R> |
StreamHashJoinBuilder.build(DistributedBiFunction<T0,ItemsByTag,R> mapToOutputFn)
Builds a new pipeline stage that performs the hash-join operation.
|
<OUT> StreamStage<OUT> |
WindowGroupAggregateBuilder.build(KeyedWindowResultFunction<? super K,ItemsByTag,OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs a windowed
cogroup-and-aggregate operation on the stages registered with this
builder object.
|
<OUT> StreamStage<OUT> |
WindowAggregateBuilder.build(WindowResultFunction<? super ItemsByTag,? extends OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs a windowed
co-aggregation of the stages registered with this builder object.
|
<R> StreamStage<R> |
StreamStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier) |
default StreamStage<TimestampedItem<T>> |
StageWithWindow.distinct()
Attaches a stage that passes through just the items that are distinct
within their window (no two items emitted for a window are equal).
|
default StreamStage<TimestampedItem<T>> |
StageWithKeyAndWindow.distinct()
Attaches a stage that passes through just the items that are distinct
within their window according to the grouping key (no two items emitted
for a window map to the same key).
|
default <R> StreamStage<R> |
StageWithWindow.distinct(WindowResultFunction<? super T,? extends R> mapToOutputFn)
Attaches a stage that passes through just the items that are distinct
within their window (no two items emitted for a window are equal).
|
default <R> StreamStage<R> |
StageWithKeyAndWindow.distinct(WindowResultFunction<? super T,? extends R> mapToOutputFn)
Attaches a stage that passes through just the items that are distinct
within their window according to the grouping key (no two items emitted
for a window map to the same key).
|
<T> StreamStage<T> |
Pipeline.drawFrom(StreamSource<? extends T> source)
Returns a pipeline stage that represents an unbounded data source (i.e., an
event stream).
|
StreamStage<T> |
StreamStage.filter(DistributedPredicate<T> filterFn) |
<C> StreamStage<T> |
StreamStage.filterUsingContext(ContextFactory<C> contextFactory,
DistributedBiPredicate<? super C,? super T> filterFn) |
<C> StreamStage<T> |
StreamStageWithKey.filterUsingContext(ContextFactory<C> contextFactory,
DistributedTriPredicate<? super C,? super K,? super T> filterFn) |
<R> StreamStage<R> |
StreamStage.flatMap(DistributedFunction<? super T,? extends Traverser<? extends R>> flatMapFn) |
<C,R> StreamStage<R> |
StreamStage.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn) |
<C,R> StreamStage<R> |
StreamStageWithKey.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn) |
<K,T1_IN,T1,R> |
StreamStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
StreamStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn) |
<R> StreamStage<R> |
StreamStage.map(DistributedFunction<? super T,? extends R> mapFn) |
<C,R> StreamStage<R> |
StreamStage.mapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends R> mapFn) |
<C,R> StreamStage<R> |
StreamStageWithKey.mapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(String mapName,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(String mapName,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(String mapName,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
StreamStage<T> |
StreamStage.merge(StreamStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
default StreamStage<T> |
StreamStage.peek() |
default StreamStage<T> |
StreamStage.peek(DistributedFunction<? super T,? extends CharSequence> toStringFn) |
StreamStage<T> |
StreamStage.peek(DistributedPredicate<? super T> shouldLogFn,
DistributedFunction<? super T,? extends CharSequence> toStringFn) |
default <R> StreamStage<Map.Entry<K,R>> |
StreamStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp) |
<R> StreamStage<R> |
StreamStage.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp) |
<R,OUT> StreamStage<OUT> |
StreamStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn) |
StreamStage<T> |
StreamStage.setLocalParallelism(int localParallelism) |
StreamStage<T> |
StreamStage.setName(String name) |
StreamStage<T> |
StageWithWindow.streamStage()
Returns the pipeline stage associated with this object.
|
Modifier and Type | Method and Description |
---|---|
<E> Tag<E> |
WindowAggregateBuilder1.add(StreamStage<E> stage)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
<T,R> Tag<R> |
WindowAggregateBuilder.add(StreamStage<T> stage,
AggregateOperation1<? super T,?,? extends R> aggrOp)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
default <T1,R0,R1> StreamStage<TimestampedItem<Tuple2<R0,R1>>> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given co-aggregate operation over the
items from both this stage and
stage1 you supply. |
default <T1,R0,R1,OUT> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
WindowResult2Function<? super R0,? super R1,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given co-aggregate operation over the
items from both this stage and
stage1 you supply. |
default <T1,R> StreamStage<TimestampedItem<R>> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
<T1,R,OUT> StreamStage<OUT> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,R> aggrOp,
WindowResultFunction<? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2,OUT> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
WindowResult3Function<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2,OUT> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
WindowResult3Function<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R> StreamStage<TimestampedItem<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
default <T1,T2,R> StreamStage<TimestampedItem<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R,OUT> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp,
WindowResultFunction<? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R,OUT> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp,
WindowResultFunction<? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
StreamStage<T> |
StreamStage.merge(StreamStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
Copyright © 2018 Hazelcast, Inc.. All rights reserved.