Package | Description |
---|---|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Interface and Description |
---|---|
interface |
BatchStage<T>
A stage in a distributed computation
pipeline that will
observe a finite amount of data (a batch). |
interface |
StreamStage<T>
A stage in a distributed computation
pipeline that will
observe an unbounded amount of data (i.e., an event stream). |
Modifier and Type | Method and Description |
---|---|
<R> GeneralStage<R> |
GeneralStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
<R> GeneralStage<R> |
GeneralStageWithKey.customTransform(String stageName,
ProcessorSupplier procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
GeneralStage<T> |
GeneralStage.filter(DistributedPredicate<T> filterFn)
Attaches a filtering stage which applies the provided predicate function
to each input item to decide whether to pass the item to the output or
to discard it.
|
<C> GeneralStage<T> |
GeneralStage.filterUsingContext(ContextFactory<C> contextFactory,
DistributedBiPredicate<? super C,? super T> filterFn)
Attaches a filtering stage which applies the provided predicate function
to each input item to decide whether to pass the item to the output or
to discard it.
|
<C> GeneralStage<T> |
GeneralStageWithKey.filterUsingContext(ContextFactory<C> contextFactory,
DistributedTriPredicate<? super C,? super K,? super T> filterFn)
Attaches a filtering stage which applies the provided predicate function
to each input item to decide whether to pass the item to the output or
to discard it.
|
<R> GeneralStage<R> |
GeneralStage.flatMap(DistributedFunction<? super T,? extends Traverser<? extends R>> flatMapFn)
Attaches a flat-mapping stage which applies the supplied function to
each input item independently and emits all the items from the
Traverser it returns. |
<C,R> GeneralStage<R> |
GeneralStage.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends Traverser<? extends R>> flatMapFn)
Attaches a flat-mapping stage which applies the supplied function to
each input item independently and emits all items from the
Traverser it returns as the output items. |
<C,R> GeneralStage<R> |
GeneralStageWithKey.flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn)
Attaches a flat-mapping stage which applies the supplied function to
each input item independently and emits all the items from the
Traverser it returns as the output items. |
<K,T1_IN,T1,R> |
GeneralStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
DistributedBiFunction<T,T1,R> mapToOutputFn)
Attaches to both this and the supplied stage a hash-joining stage and
returns it.
|
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
GeneralStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
DistributedTriFunction<T,T1,T2,R> mapToOutputFn)
Attaches to this and the two supplied stages a hash-joining stage and
returns it.
|
<R> GeneralStage<R> |
GeneralStage.map(DistributedFunction<? super T,? extends R> mapFn)
Attaches a mapping stage which applies the supplied function to each
input item independently and emits the function's result as the output
item.
|
<C,R> GeneralStage<R> |
GeneralStage.mapUsingContext(ContextFactory<C> contextFactory,
DistributedBiFunction<? super C,? super T,? extends R> mapFn)
Attaches a mapping stage which applies the supplied function to each
input item independently and emits the function's result as the output
item.
|
<C,R> GeneralStage<R> |
GeneralStageWithKey.mapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends R> mapFn)
Attaches a mapping stage which applies the given function to each input
item independently and emits the function's result as the output item.
|
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)
Attaches a
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap . |
default <V,R> GeneralStage<R> |
GeneralStageWithKey.mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super T,? super V,? extends R> mapFn)
Attaches a
GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap . |
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingIMap(String mapName,
DistributedBiFunction<? super IMap<K,V>,? super T,? extends R> mapFn)
Attaches a
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap with the supplied name. |
default <V,R> GeneralStage<R> |
GeneralStageWithKey.mapUsingIMap(String mapName,
DistributedBiFunction<? super T,? super V,? extends R> mapFn)
Attaches a
GeneralStageWithKey.mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap with the supplied name. |
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)
Attaches a
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast ReplicatedMap . |
default <K,V,R> GeneralStage<R> |
GeneralStage.mapUsingReplicatedMap(String mapName,
DistributedBiFunction<? super ReplicatedMap<K,V>,? super T,? extends R> mapFn)
Attaches a
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedBiFunction<? super C, ? super T, ? extends R>) stage where the context is a
Hazelcast ReplicatedMap with the supplied name. |
default GeneralStage<T> |
GeneralStage.peek()
Adds a peeking layer to this compute stage which logs its output.
|
default GeneralStage<T> |
GeneralStage.peek(DistributedFunction<? super T,? extends CharSequence> toStringFn)
Adds a peeking layer to this compute stage which logs its output.
|
GeneralStage<T> |
GeneralStage.peek(DistributedPredicate<? super T> shouldLogFn,
DistributedFunction<? super T,? extends CharSequence> toStringFn)
Attaches a peeking stage which logs this stage's output and passes it
through without transformation.
|
default <R> GeneralStage<Map.Entry<K,R>> |
GeneralStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
A shortcut for:
aggregateRolling(aggrOp, Util::entry) . |
<R> GeneralStage<R> |
GeneralStage.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
Attaches a rolling aggregation stage.
|
<R,OUT> GeneralStage<OUT> |
GeneralStageWithKey.rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a rolling aggregation stage.
|
Modifier and Type | Method and Description |
---|---|
<T> SinkStage |
Pipeline.drainTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1)
Attaches the supplied sink to two pipeline stages.
|
<T> SinkStage |
Pipeline.drainTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1)
Attaches the supplied sink to two pipeline stages.
|
<T> SinkStage |
Pipeline.drainTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1,
GeneralStage<? extends T>... moreStages)
Attaches the supplied sink to two or more pipeline stages.
|
<T> SinkStage |
Pipeline.drainTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1,
GeneralStage<? extends T>... moreStages)
Attaches the supplied sink to two or more pipeline stages.
|
<T> SinkStage |
Pipeline.drainTo(Sink<? super T> sink,
GeneralStage<? extends T> stage0,
GeneralStage<? extends T> stage1,
GeneralStage<? extends T>... moreStages)
Attaches the supplied sink to two or more pipeline stages.
|
Copyright © 2018 Hazelcast, Inc.. All rights reserved.