Package | Description |
---|---|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Method and Description |
---|---|
default <R> BatchStage<Map.Entry<K,R>> |
BatchStageWithKey.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
Attaches a stage that performs the given group-and-aggregate operation.
|
<R> BatchStage<R> |
BatchStage.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items it receives.
|
<R,OUT> BatchStage<OUT> |
BatchStageWithKey.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given group-and-aggregate operation.
|
default <T1,R0,R1> BatchStage<Tuple2<R0,R1>> |
BatchStage.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that co-aggregates the data from this and the supplied
stage by performing a separate aggregate operation on each and emitting
a
Tuple2 with their results. |
default <T1,R0,R1,OUT> |
BatchStage.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
DistributedBiFunction<? super R0,? super R1,? extends OUT> mapToOutputFn)
Attaches a stage that co-aggregates the data from this and the supplied
stage by performing a separate aggregate operation on each and then
passing both results to
mapToOutputFn , which transforms them
into the final output. |
default <T1,R0,R1> BatchStage<Map.Entry<K,Tuple2<R0,R1>>> |
BatchStageWithKey.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStageWithKey<? extends T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given cogroup-and-aggregate
transformation of the items from both this stage and
stage1
you supply. |
default <T1,R0,R1,OUT> |
BatchStageWithKey.aggregate2(AggregateOperation1<? super T,?,R0> aggrOp0,
BatchStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,R1> aggrOp1,
DistributedTriFunction<? super K,? super R0,? super R1,OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate
transformation of the items from both this stage and
stage1 you
supply. |
<T1,R> BatchStage<R> |
BatchStage.aggregate2(BatchStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items it receives from both this stage and
stage1 you supply. |
<T1,R,OUT> BatchStage<OUT> |
BatchStageWithKey.aggregate2(BatchStageWithKey<T1,? extends K> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,R> BatchStage<Map.Entry<K,R>> |
BatchStageWithKey.aggregate2(BatchStageWithKey<T1,? extends K> stage1,
AggregateOperation2<? super T,? super T1,?,R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,T2,R0,R1,R2> |
BatchStage.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that co-aggregates the data from this and the two
supplied stages by performing a separate aggregate operation on each and
emitting a
Tuple3 with their results. |
default <T1,T2,R0,R1,R2,OUT> |
BatchStage.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
DistributedTriFunction<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that co-aggregates the data from this and the two
supplied stages by performing a separate aggregate operation on each and
then passing all three results to
mapToOutputFn , which
transforms them into the final output. |
default <T1,T2,R0,R1,R2> |
BatchStageWithKey.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStageWithKey<T2,? extends K> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given cogroup-and-aggregate
transformation of the items from this stage as well as
stage1
and stage2 you supply. |
default <T1,T2,R0,R1,R2,OUT> |
BatchStageWithKey.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStageWithKey<T2,? extends K> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
DistributedQuadFunction<? super K,? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate
transformation of the items from this stage as well as
stage1
and stage2 you supply. |
<T1,T2,R> BatchStage<R> |
BatchStage.aggregate3(BatchStage<T1> stage1,
BatchStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items it receives from this stage as well as
stage1 and
stage2 you supply. |
default <T1,T2,R> BatchStage<Map.Entry<K,R>> |
BatchStageWithKey.aggregate3(BatchStageWithKey<T1,? extends K> stage1,
BatchStageWithKey<T2,? extends K> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R,OUT> |
BatchStageWithKey.aggregate3(BatchStageWithKey<T1,? extends K> stage1,
BatchStageWithKey<T2,? extends K> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from this stage as well as
stage1 and stage2 you supply. |
BatchStage<Map.Entry<K,ItemsByTag>> |
GroupAggregateBuilder.build()
Creates and returns a pipeline stage that performs the co-aggregation
of the stages registered with this builder object and emits a
Map.Entry(key, resultsByTag) for each distinct key. |
BatchStage<ItemsByTag> |
AggregateBuilder.build()
Creates and returns a pipeline stage that performs the co-aggregation
of the stages registered with this builder object.
|
<R> BatchStage<Map.Entry<K,R>> |
GroupAggregateBuilder1.build(AggregateOperation<?,R> aggrOp)
Convenience for
build(aggrOp, mapToOutputFn) which emits Map.Entry s as output. |
<R> BatchStage<R> |
AggregateBuilder1.build(AggregateOperation<?,R> aggrOp)
Creates and returns a pipeline stage that performs the co-aggregation
of pipeline stages registered with this builder object.
|
<R,OUT> BatchStage<OUT> |
GroupAggregateBuilder1.build(AggregateOperation<?,R> aggrOp,
DistributedBiFunction<? super K,? super R,OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs the
co-grouping and aggregation of pipeline stages registered with this
builder object.
|
<OUT> BatchStage<OUT> |
GroupAggregateBuilder.build(DistributedBiFunction<? super K,ItemsByTag,OUT> mapToOutputFn)
Creates and returns a pipeline stage that performs the co-aggregation
of the stages registered with this builder object.
|
<R> BatchStage<R> |
HashJoinBuilder.build(DistributedBiFunction<T0,ItemsByTag,R> mapToOutputFn)
Builds a new pipeline stage that performs the hash-join operation.
|
<R> BatchStage<R> |
AggregateBuilder.build(DistributedFunction<? super ItemsByTag,? extends R> finishFn)
Creates and returns a pipeline stage that performs the co-aggregation
of the stages registered with this builder object.
|
<R> BatchStage<R> |
BatchStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier) |
BatchStage<T> |
BatchStageWithKey.distinct()
Attaches a stage that emits just the items that are distinct according
to the grouping key (no two items which map to the same key will be on
the output).
|
default BatchStage<T> |
BatchStage.distinct()
Attaches a stage that emits just the items that are distinct according
to their definition of equality (
equals and hashCode ). |
<T> BatchStage<T> |
Pipeline.drawFrom(BatchSource<? extends T> source)
Returns a pipeline stage that represents a bounded (batch) data source.
|
BatchStage<T> |
BatchStage.filter(DistributedPredicate<T> filterFn) |
<C> BatchStage<T> |
BatchStage.filterUsingContext(ContextFactory<C> contextFactory,
DistributedBiPredicate<? super C,? super T> filterFn) |
<C> BatchStage<T> |
BatchStageWithKey.filterUsingContext(ContextFactory<C> contextFactory,
DistributedTriPredicate<? super C,? super K,? super T> filterFn) |
<R> BatchStage<R> |
BatchStage.flatMap(DistributedFunction<? super T,? extends Traverser<? extends R>> flatMapFn) |
<C,R> BatchStage<R> |
BatchStage.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn) |
<C,R> BatchStage<R> |
BatchStageWithKey.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn) |
<K,T1_IN,T1,R> |
BatchStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
BatchStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn) |
<R> BatchStage<R> |
BatchStage.map(DistributedFunction<? super T,? extends R> mapFn) |
<C,R> BatchStage<R> |
BatchStage.mapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends R> mapFn) |
<C,R> BatchStage<R> |
BatchStageWithKey.mapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends R> mapFn) |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <V,R> BatchStage<R> |
BatchStageWithKey.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingIMap(String mapName,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn) |
default <V,R> BatchStage<R> |
BatchStageWithKey.mapUsingIMap(String mapName,
DistributedBiFunction<? super T,? super V,? extends R> mapFn) |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
default <K,V,R> BatchStage<R> |
BatchStage.mapUsingReplicatedMap(String mapName,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn) |
BatchStage<T> |
BatchStage.merge(BatchStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
default BatchStage<T> |
BatchStage.peek() |
default BatchStage<T> |
BatchStage.peek(DistributedFunction<? super T,? extends CharSequence> toStringFn) |
BatchStage<T> |
BatchStage.peek(DistributedPredicate<? super T> shouldLogFn,
DistributedFunction<? super T,? extends CharSequence> toStringFn) |
default <R> BatchStage<Map.Entry<K,R>> |
BatchStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp) |
<R> BatchStage<R> |
BatchStage.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp) |
<R,OUT> BatchStage<OUT> |
BatchStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn) |
BatchStage<T> |
BatchStage.setLocalParallelism(int localParallelism) |
BatchStage<T> |
BatchStage.setName(String name) |
Modifier and Type | Method and Description |
---|---|
<T> Tag<T> |
AggregateBuilder1.add(BatchStage<T> stage)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
<T,R> Tag<R> |
AggregateBuilder.add(BatchStage<T> stage,
AggregateOperation1<? super T,?,? extends R> aggrOp)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
<K,T1_IN,T1> |
GeneralHashJoinBuilder.add(BatchStage<T1_IN> stage,
JoinClause<K,T0,T1_IN,T1> joinClause)
Adds another contributing pipeline stage to the hash-join operation.
|
default <T1,R0,R1> BatchStage<Tuple2<R0,R1>> |
BatchStage.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that co-aggregates the data from this and the supplied
stage by performing a separate aggregate operation on each and emitting
a
Tuple2 with their results. |
default <T1,R0,R1,OUT> |
BatchStage.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
DistributedBiFunction<? super R0,? super R1,? extends OUT> mapToOutputFn)
Attaches a stage that co-aggregates the data from this and the supplied
stage by performing a separate aggregate operation on each and then
passing both results to
mapToOutputFn , which transforms them
into the final output. |
<T1,R> BatchStage<R> |
BatchStage.aggregate2(BatchStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items it receives from both this stage and
stage1 you supply. |
default <T1,T2,R0,R1,R2> |
BatchStage.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that co-aggregates the data from this and the two
supplied stages by performing a separate aggregate operation on each and
emitting a
Tuple3 with their results. |
default <T1,T2,R0,R1,R2> |
BatchStage.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that co-aggregates the data from this and the two
supplied stages by performing a separate aggregate operation on each and
emitting a
Tuple3 with their results. |
default <T1,T2,R0,R1,R2,OUT> |
BatchStage.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
DistributedTriFunction<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that co-aggregates the data from this and the two
supplied stages by performing a separate aggregate operation on each and
then passing all three results to
mapToOutputFn , which
transforms them into the final output. |
default <T1,T2,R0,R1,R2,OUT> |
BatchStage.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
BatchStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
BatchStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2,
DistributedTriFunction<? super R0,? super R1,? super R2,? extends OUT> mapToOutputFn)
Attaches a stage that co-aggregates the data from this and the two
supplied stages by performing a separate aggregate operation on each and
then passing all three results to
mapToOutputFn , which
transforms them into the final output. |
<T1,T2,R> BatchStage<R> |
BatchStage.aggregate3(BatchStage<T1> stage1,
BatchStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items it receives from this stage as well as
stage1 and
stage2 you supply. |
<T1,T2,R> BatchStage<R> |
BatchStage.aggregate3(BatchStage<T1> stage1,
BatchStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items it receives from this stage as well as
stage1 and
stage2 you supply. |
<K,T1_IN,T1,R> |
StreamStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn) |
<K,T1_IN,T1,R> |
GeneralStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn)
Attaches to both this and the supplied stage a hash-joining stage and
returns it.
|
<K,T1_IN,T1,R> |
BatchStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
StreamStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
StreamStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
GeneralStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn)
Attaches to this and the two supplied stages a hash-joining stage and
returns it.
|
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
GeneralStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn)
Attaches to this and the two supplied stages a hash-joining stage and
returns it.
|
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
BatchStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
BatchStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn) |
BatchStage<T> |
BatchStage.merge(BatchStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
Copyright © 2018 Hazelcast, Inc.. All rights reserved.