Uses of Interface
com.hazelcast.jet.pipeline.GeneralStage
Package
Description
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
-
Uses of GeneralStage in com.hazelcast.jet.pipeline
Modifier and TypeInterfaceDescriptioninterface
BatchStage<T>
A stage in a distributed computationpipeline
that will observe a finite amount of data (a batch).interface
StreamStage<T>
A stage in a distributed computationpipeline
that will observe an unbounded amount of data (i.e., an event stream).Modifier and TypeMethodDescription<R> GeneralStage<R>
GeneralStage.customTransform
(String stageName, SupplierEx<Processor> procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.<R> GeneralStage<R>
GeneralStage.customTransform
(String stageName, ProcessorMetaSupplier procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.<R> GeneralStage<R>
GeneralStage.customTransform
(String stageName, ProcessorSupplier procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.<R> GeneralStage<R>
GeneralStageWithKey.customTransform
(String stageName, SupplierEx<Processor> procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.<R> GeneralStage<R>
GeneralStageWithKey.customTransform
(String stageName, ProcessorMetaSupplier procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.<R> GeneralStage<R>
GeneralStageWithKey.customTransform
(String stageName, ProcessorSupplier procSupplier) Attaches a stage with a custom transform based on the provided supplier of Core APIProcessor
s.GeneralStage.filter
(PredicateEx<T> filterFn) Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it.<S> GeneralStage<T>
GeneralStage.filterStateful
(SupplierEx<? extends S> createFn, BiPredicateEx<? super S, ? super T> filterFn) Attaches a stage that performs a stateful filtering operation.<S> GeneralStage<T>
GeneralStageWithKey.filterStateful
(SupplierEx<? extends S> createFn, BiPredicateEx<? super S, ? super T> filterFn) Attaches a stage that performs a stateful filtering operation.<S> GeneralStage<T>
GeneralStage.filterUsingService
(ServiceFactory<?, S> serviceFactory, BiPredicateEx<? super S, ? super T> filterFn) Attaches a filtering stage which applies the provided predicate function to each input item to decide whether to pass the item to the output or to discard it.<S> GeneralStage<T>
GeneralStageWithKey.filterUsingService
(ServiceFactory<?, S> serviceFactory, TriPredicate<? super S, ? super K, ? super T> filterFn) Deprecated.<R> GeneralStage<R>
GeneralStage.flatMap
(FunctionEx<? super T, ? extends Traverser<R>> flatMapFn) Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all the items from theTraverser
it returns.<S,
R> GeneralStage<R> GeneralStage.flatMapStateful
(SupplierEx<? extends S> createFn, BiFunctionEx<? super S, ? super T, ? extends Traverser<R>> flatMapFn) Attaches a stage that performs a stateful flat-mapping operation.<S,
R> GeneralStage<R> GeneralStageWithKey.flatMapStateful
(SupplierEx<? extends S> createFn, TriFunction<? super S, ? super K, ? super T, ? extends Traverser<R>> flatMapFn) Attaches a stage that performs a stateful flat-mapping operation.<S,
R> GeneralStage<R> GeneralStage.flatMapUsingService
(ServiceFactory<?, S> serviceFactory, BiFunctionEx<? super S, ? super T, ? extends Traverser<R>> flatMapFn) Attaches a flat-mapping stage which applies the supplied function to each input item independently and emits all items from theTraverser
it returns as the output items.<S,
R> GeneralStage<R> GeneralStageWithKey.flatMapUsingService
(ServiceFactory<?, S> serviceFactory, TriFunction<? super S, ? super K, ? super T, ? extends Traverser<R>> flatMapFn) Deprecated.Jet now has first-class support for data rebalancing, seerebalance()
andrebalance(FunctionEx)
.<K,
T1_IN, T1, R>
GeneralStage<R>GeneralStage.hashJoin
(BatchStage<T1_IN> stage1, JoinClause<K, ? super T, ? super T1_IN, ? extends T1> joinClause1, BiFunctionEx<T, T1, R> mapToOutputFn) Attaches to both this and the supplied stage a hash-joining stage and returns it.<K1,
K2, T1_IN, T2_IN, T1, T2, R>
GeneralStage<R>GeneralStage.hashJoin2
(BatchStage<T1_IN> stage1, JoinClause<K1, ? super T, ? super T1_IN, ? extends T1> joinClause1, BatchStage<T2_IN> stage2, JoinClause<K2, ? super T, ? super T2_IN, ? extends T2> joinClause2, TriFunction<T, T1, T2, R> mapToOutputFn) Attaches to this and the two supplied stages a hash-joining stage and returns it.<K,
T1_IN, T1, R>
GeneralStage<R>GeneralStage.innerHashJoin
(BatchStage<T1_IN> stage1, JoinClause<K, ? super T, ? super T1_IN, ? extends T1> joinClause1, BiFunctionEx<T, T1, R> mapToOutputFn) Attaches to both this and the supplied stage an inner hash-joining stage and returns it.<K1,
K2, T1_IN, T2_IN, T1, T2, R>
GeneralStage<R>GeneralStage.innerHashJoin2
(BatchStage<T1_IN> stage1, JoinClause<K1, ? super T, ? super T1_IN, ? extends T1> joinClause1, BatchStage<T2_IN> stage2, JoinClause<K2, ? super T, ? super T2_IN, ? extends T2> joinClause2, TriFunction<T, T1, T2, R> mapToOutputFn) Attaches to this and the two supplied stages an inner hash-joining stage and returns it.<R> GeneralStage<R>
GeneralStage.map
(FunctionEx<? super T, ? extends R> mapFn) Attaches a mapping stage which applies the given function to each input item independently and emits the function's result as the output item.<S,
R> GeneralStage<R> GeneralStage.mapStateful
(SupplierEx<? extends S> createFn, BiFunctionEx<? super S, ? super T, ? extends R> mapFn) Attaches a stage that performs a stateful mapping operation.<S,
R> GeneralStage<R> GeneralStageWithKey.mapStateful
(SupplierEx<? extends S> createFn, TriFunction<? super S, ? super K, ? super T, ? extends R> mapFn) Attaches a stage that performs a stateful mapping operation.default <K,
V, R> GeneralStage<R> GeneralStage.mapUsingIMap
(IMap<K, V> iMap, FunctionEx<? super T, ? extends K> lookupKeyFn, BiFunctionEx<? super T, ? super V, ? extends R> mapFn) Attaches a mapping stage where for each item a lookup in the suppliedIMap
is performed and the result of the lookup is merged with the item and emitted.default <K,
V, R> GeneralStage<R> GeneralStage.mapUsingIMap
(String mapName, FunctionEx<? super T, ? extends K> lookupKeyFn, BiFunctionEx<? super T, ? super V, ? extends R> mapFn) Attaches a mapping stage where for each item a lookup in theIMap
with the supplied name is performed and the result of the lookup is merged with the item and emitted.default <V,
R> GeneralStage<R> GeneralStageWithKey.mapUsingIMap
(IMap<K, V> iMap, BiFunctionEx<? super T, ? super V, ? extends R> mapFn) Attaches a mapping stage where for each item a lookup in the suppliedIMap
using the grouping key is performed and the result of the lookup is merged with the item and emitted.default <V,
R> GeneralStage<R> GeneralStageWithKey.mapUsingIMap
(String mapName, BiFunctionEx<? super T, ? super V, ? extends R> mapFn) Attaches a mapping stage where for each item a lookup in theIMap
with the supplied name using the grouping key is performed and the result of the lookup is merged with the item and emitted.default <K,
V, R> GeneralStage<R> GeneralStage.mapUsingReplicatedMap
(ReplicatedMap<K, V> replicatedMap, FunctionEx<? super T, ? extends K> lookupKeyFn, BiFunctionEx<? super T, ? super V, ? extends R> mapFn) Attaches a mapping stage where for each item a lookup in the suppliedReplicatedMap
is performed and the result of the lookup is merged with the item and emitted.default <K,
V, R> GeneralStage<R> GeneralStage.mapUsingReplicatedMap
(String mapName, FunctionEx<? super T, ? extends K> lookupKeyFn, BiFunctionEx<? super T, ? super V, ? extends R> mapFn) Attaches a mapping stage where for each item a lookup in theReplicatedMap
with the supplied name is performed and the result of the lookup is merged with the item and emitted.<S,
R> GeneralStage<R> GeneralStage.mapUsingService
(ServiceFactory<?, S> serviceFactory, BiFunctionEx<? super S, ? super T, ? extends R> mapFn) Attaches a mapping stage which applies the supplied function to each input item independently and emits the function's result as the output item.<S,
R> GeneralStage<R> GeneralStageWithKey.mapUsingService
(ServiceFactory<?, S> serviceFactory, TriFunction<? super S, ? super K, ? super T, ? extends R> mapFn) Deprecated.Jet now has first-class support for data rebalancing, seerebalance()
andrebalance(FunctionEx)
.<S,
R> GeneralStage<R> GeneralStage.mapUsingServiceAsync
(ServiceFactory<?, S> serviceFactory, int maxConcurrentOps, boolean preserveOrder, BiFunctionEx<? super S, ? super T, ? extends CompletableFuture<R>> mapAsyncFn) Asynchronous version ofmapUsingService(com.hazelcast.jet.pipeline.ServiceFactory<?, S>, com.hazelcast.function.BiFunctionEx<? super S, ? super T, ? extends R>)
: themapAsyncFn
returns aCompletableFuture<R>
instead of justR
.default <S,
R> GeneralStage<R> GeneralStage.mapUsingServiceAsync
(ServiceFactory<?, S> serviceFactory, BiFunctionEx<? super S, ? super T, ? extends CompletableFuture<R>> mapAsyncFn) Asynchronous version ofmapUsingService(com.hazelcast.jet.pipeline.ServiceFactory<?, S>, com.hazelcast.function.BiFunctionEx<? super S, ? super T, ? extends R>)
: themapAsyncFn
returns aCompletableFuture<R>
instead of justR
.<S,
R> GeneralStage<R> GeneralStageWithKey.mapUsingServiceAsync
(ServiceFactory<?, S> serviceFactory, int maxConcurrentOps, boolean preserveOrder, TriFunction<? super S, ? super K, ? super T, CompletableFuture<R>> mapAsyncFn) Deprecated.Jet now has first-class support for data rebalancing, seerebalance()
andrebalance(FunctionEx)
.default <S,
R> GeneralStage<R> GeneralStageWithKey.mapUsingServiceAsync
(ServiceFactory<?, S> serviceFactory, TriFunction<? super S, ? super K, ? super T, CompletableFuture<R>> mapAsyncFn) Deprecated.Jet now has first-class support for data rebalancing, seerebalance()
andrebalance(FunctionEx)
.<S,
R> GeneralStage<R> GeneralStage.mapUsingServiceAsyncBatched
(ServiceFactory<?, S> serviceFactory, int maxBatchSize, BiFunctionEx<? super S, ? super List<T>, ? extends CompletableFuture<List<R>>> mapAsyncFn) Batched version ofmapUsingServiceAsync(com.hazelcast.jet.pipeline.ServiceFactory<?, S>, com.hazelcast.function.BiFunctionEx<? super S, ? super T, ? extends java.util.concurrent.CompletableFuture<R>>)
:mapAsyncFn
takes a list of input items and returns aCompletableFuture<List<R>>
.<S,
R> GeneralStage<R> GeneralStageWithKey.mapUsingServiceAsyncBatched
(ServiceFactory<?, S> serviceFactory, int maxBatchSize, BiFunctionEx<? super S, ? super List<T>, ? extends CompletableFuture<List<R>>> mapAsyncFn) Deprecated.Jet now has first-class support for data rebalancing, seerebalance()
andrebalance(FunctionEx)
.<S,
R> GeneralStage<R> GeneralStageWithKey.mapUsingServiceAsyncBatched
(ServiceFactory<?, S> serviceFactory, int maxBatchSize, TriFunction<? super S, ? super List<K>, ? super List<T>, ? extends CompletableFuture<List<R>>> mapAsyncFn) Deprecated.Jet now has first-class support for data rebalancing, seerebalance()
andrebalance(FunctionEx)
.default GeneralStage<T>
GeneralStage.peek()
Adds a peeking layer to this compute stage which logs its output.default GeneralStage<T>
GeneralStage.peek
(FunctionEx<? super T, ? extends CharSequence> toStringFn) Adds a peeking layer to this compute stage which logs its output.GeneralStage.peek
(PredicateEx<? super T> shouldLogFn, FunctionEx<? super T, ? extends CharSequence> toStringFn) Attaches a peeking stage which logs this stage's output and passes it through without transformation.GeneralStage.rebalance()
Returns a new stage that applies data rebalancing to the output of this stage.<K> GeneralStage<T>
GeneralStage.rebalance
(FunctionEx<? super T, ? extends K> keyFn) Returns a new stage that applies data rebalancing to the output of this stage.default <A,
R> GeneralStage<R> GeneralStage.rollingAggregate
(AggregateOperation1<? super T, A, ? extends R> aggrOp) Attaches a rolling aggregation stage.default <A,
R> GeneralStage<Map.Entry<K, R>> GeneralStageWithKey.rollingAggregate
(AggregateOperation1<? super T, A, ? extends R> aggrOp) Attaches a rolling aggregation stage.GeneralStage.setLocalParallelism
(int localParallelism) Modifier and TypeMethodDescription<T> SinkStage
Pipeline.writeTo
(Sink<? super T> sink, GeneralStage<? extends T> stage0, GeneralStage<? extends T> stage1, GeneralStage<? extends T>... moreStages) Attaches the supplied sink to two or more pipeline stages.
rebalance()
andrebalance(FunctionEx)
.