Package | Description |
---|---|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.pipeline.test |
This package contains various mock sources to help with pipeline testing
and development.
|
com.hazelcast.jet.python |
Contributes a
PythonTransforms.mapUsingPython(com.hazelcast.jet.python.PythonServiceConfig)
transform that allows you to transform Jet pipeline data using a Python
function. |
Modifier and Type | Method and Description |
---|---|
StreamStage<T> |
GeneralStage.addTimestamps(ToLongFunctionEx<? super T> timestampFn,
long allowedLag)
Adds a timestamp to each item in the stream using the supplied function
and specifies the allowed amount of disorder between them.
|
<R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to a given window.
|
<R> StreamStage<KeyedWindowResult<K,R>> |
StageWithKeyAndWindow.aggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
Attaches a stage that performs the given group-and-aggregate operation.
|
default <T1,R0,R1> StreamStage<WindowResult<Tuple2<R0,R1>>> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given co-aggregate operations over
the items from this stage and
stage1 you supply. |
default <T1,R0,R1> StreamStage<KeyedWindowResult<K,Tuple2<R0,R1>>> |
StageWithKeyAndWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
<T1,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
<T1,R> StreamStage<KeyedWindowResult<K,R>> |
StageWithKeyAndWindow.aggregate2(StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithKeyAndWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStageWithKey<T1,? extends K> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from both this stage and
stage1 you supply. |
<T1,T2,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R> StreamStage<KeyedWindowResult<K,R>> |
StageWithKeyAndWindow.aggregate3(StreamStageWithKey<T1,? extends K> stage1,
StreamStageWithKey<T2,? extends K> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given cogroup-and-aggregate operation
over the items from this stage as well as
stage1 and stage2 you supply. |
default <R> StreamStage<R> |
StreamStage.apply(FunctionEx<? super StreamStage<T>,? extends StreamStage<R>> transformFn)
Transforms
this stage using the provided transformFn and
returns the transformed stage. |
StreamStage<WindowResult<ItemsByTag>> |
WindowAggregateBuilder.build()
Creates and returns a pipeline stage that performs a windowed
co-aggregation of the stages registered with this builder object.
|
StreamStage<KeyedWindowResult<K,ItemsByTag>> |
WindowGroupAggregateBuilder.build()
Creates and returns a pipeline stage that performs a windowed
cogroup-and-aggregate operation on the stages registered with this
builder object.
|
<R> StreamStage<KeyedWindowResult<K,R>> |
WindowGroupAggregateBuilder1.build(AggregateOperation<?,? extends R> aggrOp)
Creates and returns a pipeline stage that performs a windowed
cogroup-and-aggregate of the pipeline stages registered with this builder object.
|
<A,R> StreamStage<WindowResult<R>> |
WindowAggregateBuilder1.build(AggregateOperation<A,R> aggrOp)
Creates and returns a pipeline stage that performs a windowed
co-aggregation of the pipeline stages registered with this builder
object.
|
<R> StreamStage<R> |
StreamHashJoinBuilder.build(BiFunctionEx<T0,ItemsByTag,R> mapToOutputFn)
Builds a new pipeline stage that performs the hash-join operation.
|
<R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
<R> StreamStage<R> |
StreamStage.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
default <R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
ProcessorSupplier procSupplier) |
default <R> StreamStage<R> |
StreamStage.customTransform(String stageName,
ProcessorSupplier procSupplier) |
default <R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
default <R> StreamStage<R> |
StreamStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
default StreamStage<WindowResult<T>> |
StageWithWindow.distinct()
Attaches a stage that passes through just the items that are distinct
within their window (no two items emitted for a window are equal).
|
default StreamStage<KeyedWindowResult<K,T>> |
StageWithKeyAndWindow.distinct()
Attaches a stage that passes through just the items that are distinct
within their window according to the grouping key (no two items emitted
for a window map to the same key).
|
StreamStage<T> |
StreamStage.filter(PredicateEx<T> filterFn) |
<S> StreamStage<T> |
StreamStageWithKey.filterStateful(long ttl,
SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn)
Attaches a stage that performs a stateful filtering operation.
|
default <S> StreamStage<T> |
StreamStageWithKey.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<S> StreamStage<T> |
StreamStage.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<S> StreamStage<T> |
StreamStage.filterUsingService(ServiceFactory<?,S> serviceFactory,
BiPredicateEx<? super S,? super T> filterFn) |
<S> StreamStage<T> |
StreamStageWithKey.filterUsingService(ServiceFactory<?,S> serviceFactory,
TriPredicate<? super S,? super K,? super T> filterFn) |
<R> StreamStage<R> |
StreamStage.flatMap(FunctionEx<? super T,? extends Traverser<R>> flatMapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.flatMapStateful(long ttl,
SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn,
TriFunction<? super S,? super K,? super Long,? extends Traverser<R>> onEvictFn)
Attaches a stage that performs a stateful flat-mapping operation.
|
<S,R> StreamStage<R> |
StreamStage.flatMapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends Traverser<R>> flatMapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.flatMapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn) |
<S,R> StreamStage<R> |
StreamStage.flatMapUsingService(ServiceFactory<?,S> serviceFactory,
BiFunctionEx<? super S,? super T,? extends Traverser<R>> flatMapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.flatMapUsingService(ServiceFactory<?,S> serviceFactory,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn) |
<K,T1_IN,T1,R> |
StreamStage.hashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
BiFunctionEx<T,T1,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
StreamStage.hashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
TriFunction<T,T1,T2,R> mapToOutputFn) |
<K,T1_IN,T1,R> |
StreamStage.innerHashJoin(BatchStage<T1_IN> stage1,
JoinClause<K,? super T,? super T1_IN,? extends T1> joinClause1,
BiFunctionEx<T,T1,R> mapToOutputFn) |
<K1,K2,T1_IN,T2_IN,T1,T2,R> |
StreamStage.innerHashJoin2(BatchStage<T1_IN> stage1,
JoinClause<K1,? super T,? super T1_IN,? extends T1> joinClause1,
BatchStage<T2_IN> stage2,
JoinClause<K2,? super T,? super T2_IN,? extends T2> joinClause2,
TriFunction<T,T1,T2,R> mapToOutputFn) |
<R> StreamStage<R> |
StreamStage.map(FunctionEx<? super T,? extends R> mapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapStateful(long ttl,
SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn,
TriFunction<? super S,? super K,? super Long,? extends R> onEvictFn)
Attaches a stage that performs a stateful mapping operation.
|
<S,R> StreamStage<R> |
StreamStage.mapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends R> mapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(IMap<K,V> iMap,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(IMap<K,V> iMap,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <V,R> StreamStage<R> |
StreamStageWithKey.mapUsingIMap(String mapName,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingIMap(String mapName,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(ReplicatedMap<K,V> replicatedMap,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
default <K,V,R> StreamStage<R> |
StreamStage.mapUsingReplicatedMap(String mapName,
FunctionEx<? super T,? extends K> lookupKeyFn,
BiFunctionEx<? super T,? super V,? extends R> mapFn) |
<S,R> StreamStage<R> |
StreamStage.mapUsingService(ServiceFactory<?,S> serviceFactory,
BiFunctionEx<? super S,? super T,? extends R> mapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapUsingService(ServiceFactory<?,S> serviceFactory,
TriFunction<? super S,? super K,? super T,? extends R> mapFn) |
default <S,R> StreamStage<R> |
StreamStage.mapUsingServiceAsync(ServiceFactory<?,S> serviceFactory,
BiFunctionEx<? super S,? super T,? extends CompletableFuture<R>> mapAsyncFn) |
<S,R> StreamStage<R> |
StreamStage.mapUsingServiceAsync(ServiceFactory<?,S> serviceFactory,
int maxConcurrentOps,
boolean preserveOrder,
BiFunctionEx<? super S,? super T,? extends CompletableFuture<R>> mapAsyncFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapUsingServiceAsync(ServiceFactory<?,S> serviceFactory,
int maxConcurrentOps,
boolean preserveOrder,
TriFunction<? super S,? super K,? super T,CompletableFuture<R>> mapAsyncFn) |
default <S,R> StreamStage<R> |
StreamStageWithKey.mapUsingServiceAsync(ServiceFactory<?,S> serviceFactory,
TriFunction<? super S,? super K,? super T,CompletableFuture<R>> mapAsyncFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapUsingServiceAsyncBatched(ServiceFactory<?,S> serviceFactory,
int maxBatchSize,
BiFunctionEx<? super S,? super List<T>,? extends CompletableFuture<List<R>>> mapAsyncFn) |
<S,R> StreamStage<R> |
StreamStage.mapUsingServiceAsyncBatched(ServiceFactory<?,S> serviceFactory,
int maxBatchSize,
BiFunctionEx<? super S,? super List<T>,? extends CompletableFuture<List<R>>> mapAsyncFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapUsingServiceAsyncBatched(ServiceFactory<?,S> serviceFactory,
int maxBatchSize,
TriFunction<? super S,? super List<K>,? super List<T>,? extends CompletableFuture<List<R>>> mapAsyncFn) |
StreamStage<T> |
StreamStage.merge(StreamStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
default StreamStage<T> |
StreamStage.peek() |
default StreamStage<T> |
StreamStage.peek(FunctionEx<? super T,? extends CharSequence> toStringFn) |
StreamStage<T> |
StreamStage.peek(PredicateEx<? super T> shouldLogFn,
FunctionEx<? super T,? extends CharSequence> toStringFn) |
StreamStage<T> |
StreamStage.rebalance() |
<K> StreamStage<T> |
StreamStage.rebalance(FunctionEx<? super T,? extends K> keyFn) |
default <A,R> StreamStage<Map.Entry<K,R>> |
StreamStageWithKey.rollingAggregate(AggregateOperation1<? super T,A,? extends R> aggrOp) |
default <A,R> StreamStage<R> |
StreamStage.rollingAggregate(AggregateOperation1<? super T,A,? extends R> aggrOp) |
default <A,R> StreamStage<Map.Entry<K,R>> |
StreamStageWithKey.rollingAggregate(long ttl,
AggregateOperation1<? super T,A,? extends R> aggrOp)
Attaches a rolling aggregation stage.
|
StreamStage<T> |
StreamStage.setLocalParallelism(int localParallelism) |
StreamStage<T> |
StreamStage.setName(String name) |
StreamStage<T> |
StageWithWindow.streamStage()
Returns the pipeline stage associated with this object.
|
StreamStage<T> |
StreamSourceStage.withIngestionTimestamps()
Declares that the source will assign the time of ingestion as the event
timestamp.
|
StreamStage<T> |
StreamSourceStage.withNativeTimestamps(long allowedLag)
Declares that the stream will use the source's native timestamps.
|
StreamStage<T> |
StreamSourceStage.withoutTimestamps()
Declares that the source will not assign any timestamp to the events it
emits.
|
StreamStage<T> |
StreamSourceStage.withTimestamps(ToLongFunctionEx<? super T> timestampFn,
long allowedLag)
Declares that the source will extract timestamps from the stream items.
|
Modifier and Type | Method and Description |
---|---|
<E> Tag<E> |
WindowAggregateBuilder1.add(StreamStage<E> stage)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
<T,R> Tag<R> |
WindowAggregateBuilder.add(StreamStage<T> stage,
AggregateOperation1<? super T,?,? extends R> aggrOp)
Adds another stage that will contribute its data to the aggregate
operation to be performed.
|
default <T1,R0,R1> StreamStage<WindowResult<Tuple2<R0,R1>>> |
StageWithWindow.aggregate2(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1)
Attaches a stage that performs the given co-aggregate operations over
the items from this stage and
stage1 you supply. |
<T1,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate2(StreamStage<T1> stage1,
AggregateOperation2<? super T,? super T1,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
default <T1,T2,R0,R1,R2> |
StageWithWindow.aggregate3(AggregateOperation1<? super T,?,? extends R0> aggrOp0,
StreamStage<T1> stage1,
AggregateOperation1<? super T1,?,? extends R1> aggrOp1,
StreamStage<T2> stage2,
AggregateOperation1<? super T2,?,? extends R2> aggrOp2)
Attaches a stage that performs the given aggregate operation over all
the items that belong to the same window.
|
<T1,T2,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
<T1,T2,R> StreamStage<WindowResult<R>> |
StageWithWindow.aggregate3(StreamStage<T1> stage1,
StreamStage<T2> stage2,
AggregateOperation3<? super T,? super T1,? super T2,?,? extends R> aggrOp)
Attaches a stage that performs the given aggregate operation over the
items it receives from this stage as well as
stage1 and stage2 you supply. |
StreamStage<T> |
StreamStage.merge(StreamStage<? extends T> other)
Attaches a stage that emits all the items from this stage as well as all
the items from the supplied stage.
|
Modifier and Type | Method and Description |
---|---|
default <R> StreamStage<R> |
StreamStage.apply(FunctionEx<? super StreamStage<T>,? extends StreamStage<R>> transformFn)
Transforms
this stage using the provided transformFn and
returns the transformed stage. |
default <R> StreamStage<R> |
StreamStage.apply(FunctionEx<? super StreamStage<T>,? extends StreamStage<R>> transformFn)
Transforms
this stage using the provided transformFn and
returns the transformed stage. |
Modifier and Type | Method and Description |
---|---|
static <T> FunctionEx<StreamStage<T>,StreamStage<T>> |
Assertions.assertCollectedEventually(int timeout,
ConsumerEx<? super List<T>> assertFn)
Collects all the received items into a list and runs the
assertFn
every time a new item is received. |
static <T> FunctionEx<StreamStage<T>,StreamStage<T>> |
Assertions.assertCollectedEventually(int timeout,
ConsumerEx<? super List<T>> assertFn)
Collects all the received items into a list and runs the
assertFn
every time a new item is received. |
Modifier and Type | Method and Description |
---|---|
static <K> FunctionEx<StreamStage<String>,StreamStage<String>> |
PythonTransforms.mapUsingPython(FunctionEx<? super String,? extends K> keyFn,
PythonServiceConfig cfg)
Deprecated.
Jet now has first-class support for data rebalancing, see
GeneralStage.rebalance() and GeneralStage.rebalance(FunctionEx) . |
static <K> FunctionEx<StreamStage<String>,StreamStage<String>> |
PythonTransforms.mapUsingPython(FunctionEx<? super String,? extends K> keyFn,
PythonServiceConfig cfg)
Deprecated.
Jet now has first-class support for data rebalancing, see
GeneralStage.rebalance() and GeneralStage.rebalance(FunctionEx) . |
static FunctionEx<StreamStage<String>,StreamStage<String>> |
PythonTransforms.mapUsingPython(PythonServiceConfig cfg)
A stage-transforming method that adds a "map using Python" pipeline stage.
|
static FunctionEx<StreamStage<String>,StreamStage<String>> |
PythonTransforms.mapUsingPython(PythonServiceConfig cfg)
A stage-transforming method that adds a "map using Python" pipeline stage.
|
Copyright © 2022 Hazelcast, Inc.. All rights reserved.