T
- type of the stream itemK
- type of the grouping keypublic interface GeneralStageWithKey<T,K>
Modifier and Type | Method and Description |
---|---|
<R> GeneralStage<R> |
customTransform(String stageName,
ProcessorSupplier procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
<C> GeneralStage<T> |
filterUsingContext(ContextFactory<C> contextFactory,
DistributedTriPredicate<? super C,? super K,? super T> filterFn)
Attaches a filtering stage which applies the provided predicate function
to each input item to decide whether to pass the item to the output or
to discard it.
|
<C,R> GeneralStage<R> |
flatMapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn)
Attaches a flat-mapping stage which applies the supplied function to
each input item independently and emits all the items from the
Traverser it returns as the output items. |
DistributedFunction<? super T,? extends K> |
keyFn()
Returns the function that extracts the key from stream items.
|
<C,R> GeneralStage<R> |
mapUsingContext(ContextFactory<C> contextFactory,
DistributedTriFunction<? super C,? super K,? super T,? extends R> mapFn)
Attaches a mapping stage which applies the given function to each input
item independently and emits the function's result as the output item.
|
default <V,R> GeneralStage<R> |
mapUsingIMap(IMap<K,V> iMap,
DistributedBiFunction<? super T,? super V,? extends R> mapFn)
Attaches a
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap . |
default <V,R> GeneralStage<R> |
mapUsingIMap(String mapName,
DistributedBiFunction<? super T,? super V,? extends R> mapFn)
Attaches a
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>) stage where the context is a
Hazelcast IMap with the supplied name. |
default <R> GeneralStage<Map.Entry<K,R>> |
rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp)
A shortcut for:
aggregateRolling(aggrOp, Util::entry) . |
<R,OUT> GeneralStage<OUT> |
rollingAggregate(AggregateOperation1<? super T,?,? extends R> aggrOp,
DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Attaches a rolling aggregation stage.
|
@Nonnull DistributedFunction<? super T,? extends K> keyFn()
@Nonnull <C,R> GeneralStage<R> mapUsingContext(@Nonnull ContextFactory<C> contextFactory, @Nonnull DistributedTriFunction<? super C,? super K,? super T,? extends R> mapFn)
contextFactory
. If the
mapping result is null
, it emits nothing. Therefore this stage
can be used to implement filtering semantics as well.
Jet uses the key-extracting function
specified on this
stage for partitioning: all the items with the same key will see the
same context instance (but note that the same instance serves many keys).
One case where this is useful is fetching data from an external system
because you can use a near-cache without duplicating the cached data.
C
- type of context objectR
- the result type of the mapping functioncontextFactory
- the context factorymapFn
- a stateless mapping function@Nonnull <C> GeneralStage<T> filterUsingContext(@Nonnull ContextFactory<C> contextFactory, @Nonnull DistributedTriPredicate<? super C,? super K,? super T> filterFn)
contextFactory
.
Jet uses the key-extracting function
specified on this
stage for partitioning: all the items with the same key will see the
same context instance (but note that the same instance serves many keys).
One case where this is useful is fetching data from an external system
because you can use a near-cache without duplicating the cached data.
C
- type of context objectcontextFactory
- the context factoryfilterFn
- a stateless filter predicate function@Nonnull <C,R> GeneralStage<R> flatMapUsingContext(@Nonnull ContextFactory<C> contextFactory, @Nonnull DistributedTriFunction<? super C,? super K,? super T,? extends Traverser<? extends R>> flatMapFn)
Traverser
it returns as the output items. The traverser must
be null-terminated. The mapping function receives another
parameter, the context object, which Jet will create using the supplied
contextFactory
.
Jet uses the key-extracting function
specified on this
stage for partitioning: all the items with the same key will see the
same context instance (but note that the same instance serves many keys).
One case where this is useful is fetching data from an external system
because you can use a near-cache without duplicating the cached data.
C
- type of context objectR
- type of the output itemscontextFactory
- the context factoryflatMapFn
- a stateless flatmapping function@Nonnull default <V,R> GeneralStage<R> mapUsingIMap(@Nonnull String mapName, @Nonnull DistributedBiFunction<? super T,? super V,? extends R> mapFn)
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>)
stage where the context is a
Hazelcast IMap
with the supplied name. Jet will use the
specified key function to retrieve a value from
the map and pass it to the mapping function you supply, as the second
argument.
This stage is similar to stageWithoutKey.mapUsingIMap()
, but here Jet
knows the key and uses it to partition and distribute the input in order
to achieve data locality. The value it fetches from the IMap
is
stored on the cluster member where the processing takes place. However,
if the map doesn't use the default partitioning strategy, the data
locality will be broken.
V
- type of the value in the IMap
R
- type of the output itemmapName
- name of the IMap
mapFn
- the mapping function@Nonnull default <V,R> GeneralStage<R> mapUsingIMap(@Nonnull IMap<K,V> iMap, @Nonnull DistributedBiFunction<? super T,? super V,? extends R> mapFn)
mapUsingContext(com.hazelcast.jet.pipeline.ContextFactory<C>, com.hazelcast.jet.function.DistributedTriFunction<? super C, ? super K, ? super T, ? extends R>)
stage where the context is a
Hazelcast IMap
. It is not necessarily the map you
provide here, but a map with the same name in the Jet cluster
that executes the pipeline. Jet will use the specified key function to retrieve a value from the map and pass it to
the mapping function you supply, as the second argument.
This stage is similar to stageWithoutKey.mapUsingIMap()
, but here Jet
knows the key and uses it to partition and distribute the input in order
to achieve data locality. The value it fetches from the IMap
is
stored on the cluster member where the processing takes place. However,
if the map doesn't use the default partitioning strategy, the data
locality will be broken.
V
- type of the value in the IMap
R
- type of the output itemiMap
- the IMap
to use as the contextmapFn
- the mapping function@Nonnull <R,OUT> GeneralStage<OUT> rollingAggregate(@Nonnull AggregateOperation1<? super T,?,? extends R> aggrOp, @Nonnull DistributedBiFunction<? super K,? super R,? extends OUT> mapToOutputFn)
{2, 7, 8, -5}
, the output will be {2,
9, 17, 12}
.
This stage is fault-tolerant and saves its state to the snapshot.
NOTE: if you plan to use an aggregate operation whose
result size grows with input size (such as toList
and your data
source is unbounded, carefully consider the memory demands this implies.
The result will keep growing forever.
R
- type of the aggregate operation resultOUT
- type of the output itemaggrOp
- the aggregate operation to performmapToOutputFn
- function that transforms the key and the aggregation result into the
output item@Nonnull default <R> GeneralStage<Map.Entry<K,R>> rollingAggregate(@Nonnull AggregateOperation1<? super T,?,? extends R> aggrOp)
aggregateRolling(aggrOp, Util::entry)
.
@Nonnull <R> GeneralStage<R> customTransform(@Nonnull String stageName, @Nonnull ProcessorSupplier procSupplier)
Processor
s. The inbound edge will be distributed and
partitioned using the key function assigned to this stage.
Note that the type parameter of the returned stage is inferred from the call site and not propagated from the processor that will produce the result, so there is no actual type safety provided.
R
- the type of the output itemsstageName
- a human-readable name for the custom stageprocSupplier
- the supplier of processorsCopyright © 2018 Hazelcast, Inc.. All rights reserved.