com.hazelcast.jet

Class Processors

java.lang.Object

com.hazelcast.jet.Processors

public final class Processors
extends Object

Static utility class with factory methods for predefined processors.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Processors.NoopP A processor that does nothing.

Method Summary

Modifier and Type	Method and Description
static <T,A> ProcessorSupplier	accumulate(Distributed.Supplier<? extends A> supplier, Distributed.BiFunction<? super A,? super T,? extends A> accumulator) Convenience over accumulate(supplier, accumulator, finisher) with identity function as the finisher, which means the processor emits an item of type A.
static <T,A,R> ProcessorSupplier	accumulate(Distributed.Supplier<? extends A> supplier, Distributed.BiFunction<? super A,? super T,? extends A> accumulator, Distributed.Function<? super A,? extends R> finisher) Returns a supplier of processor with the following semantics: Calls the supplier function to obtain the initial accumulated value.
static <T,A> ProcessorSupplier	collect(Distributed.Supplier<? extends A> supplier, Distributed.BiConsumer<? super A,? super T> collector) Convenience over collect(supplier, collector, finisher) with identity function as the finisher, which means the processor emits an item of type A.
static <T,A,R> ProcessorSupplier	collect(Distributed.Supplier<? extends A> supplier, Distributed.BiConsumer<? super A,? super T> collector, Distributed.Function<? super A,? extends R> finisher) Returns a supplier of processor with the following semantics: Calls the supplier function to obtain the mutable result container.
static <T> ProcessorSupplier	countDistinct() Convenience over countDistinct(Distributed.Function) with identity function as the key extractor, which means the processor will emit the number of distinct items it has seen in the input.
static <T,K> ProcessorSupplier	countDistinct(Distributed.Function<T,K> keyExtractor) Returns a supplier of CountDistinctP processors.
static <T> ProcessorSupplier	filter(Distributed.Predicate<? super T> predicate) Returns a supplier of processor which emits the same items it receives, but only those that pass the given predicate.
static <T,R> ProcessorSupplier	flatMap(Distributed.Function<? super T,? extends Traverser<? extends R>> mapper) Returns a supplier of TransformP processor with the given item-to-traverser function.
static <T,A> ProcessorSupplier	groupAndAccumulate(Distributed.Function<? super T,?> keyExtractor, Distributed.Supplier<? extends A> supplier, Distributed.BiFunction<? super A,? super T,? extends A> accumulator) Convenience over groupAndAccumulate(keyExtractor, supplier, accumulator, finisher) with the constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<K, A>.
static <T,K,A,R> ProcessorSupplier	groupAndAccumulate(Distributed.Function<? super T,? extends K> keyExtractor, Distributed.Supplier<? extends A> supplier, Distributed.BiFunction<? super A,? super T,? extends A> accumulator, Distributed.BiFunction<? super K,? super A,? extends R> finisher) Returns a supplier of processor with the following semantics: Accepts items of type T.
static <T,A> ProcessorSupplier	groupAndAccumulate(Distributed.Supplier<? extends A> supplier, Distributed.BiFunction<? super A,? super T,? extends A> accumulator) Convenience over groupAndAccumulate(keyExtractor, supplier, accumulator, finisher) with identity function as the key extractor and constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<T, A>.
static <T,A> ProcessorSupplier	groupAndCollect(Distributed.Function<? super T,?> keyExtractor, Distributed.Supplier<? extends A> supplier, Distributed.BiConsumer<? super A,? super T> collector) Convenience over groupAndCollect(keyExtractor, supplier, collector, finisher) with the constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<K, A>.
static <T,K,A,R> ProcessorSupplier	groupAndCollect(Distributed.Function<? super T,? extends K> keyExtractor, Distributed.Supplier<? extends A> supplier, Distributed.BiConsumer<? super A,? super T> collector, Distributed.BiFunction<? super K,? super A,? extends R> finisher) Returns a supplier of processor with the following semantics: Accepts items of type T.
static <T,A> ProcessorSupplier	groupAndCollect(Distributed.Supplier<? extends A> supplier, Distributed.BiConsumer<? super A,? super T> collector) Convenience over groupAndCollect(keyExtractor, supplier, collector, finisher) with identity function as the key extractor and constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<T, A>.
static <T,R> ProcessorSupplier	map(Distributed.Function<? super T,? extends R> mapper) Returns a supplier of processor which, for each received item, emits the result of applying the given mapping function to it.
static ProcessorSupplier	nonCooperative(ProcessorSupplier wrapped) Decorates a ProcessorSupplier into one that will declare all its processors non-cooperative.
static ProcessorMetaSupplier	readList(String listName) Returns a meta-supplier of processors that emit items retrieved from an IMDG IList.
static ProcessorMetaSupplier	readList(String listName, com.hazelcast.client.config.ClientConfig clientConfig) Returns a meta-supplier of processors that emit items retrieved from an IMDG IList in a remote cluster.
static ProcessorMetaSupplier	readMap(String mapName) Returns a meta-supplier of processors that will fetch entries from the Hazelcast IMap with the specified name.
static ProcessorMetaSupplier	readMap(String mapName, com.hazelcast.client.config.ClientConfig clientConfig) Returns a meta-supplier of processors that will fetch entries from a Hazelcast IMap in a remote cluster.
static ProcessorSupplier	writeList(String listName) Returns a meta-supplier of processors that write received items to an IMDG IList.
static ProcessorSupplier	writeList(String listName, com.hazelcast.client.config.ClientConfig clientConfig) Returns a meta-supplier of processors that write received items to an IMDG IList in a remote cluster.
static ProcessorMetaSupplier	writeMap(String mapName) Returns a meta-supplier of processors that will put data into a Hazelcast IMap.
static ProcessorMetaSupplier	writeMap(String mapName, com.hazelcast.client.config.ClientConfig clientConfig) Returns a meta-supplier of processors that will put data into a Hazelcast IMap in a remote cluster.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

readMap

@Nonnull
public static ProcessorMetaSupplier readMap(@Nonnull
                                                     String mapName)

Returns a meta-supplier of processors that will fetch entries from the Hazelcast IMap with the specified name. The processors will only access data local to the member and, if localParallelism for the vertex is above one, processors will divide the labor within the member so that each one gets a subset of all local partitions to read.

The number of Hazelcast partitions should be configured to at least localParallelism * clusterSize, otherwise some processors will have no partitions assigned to them.

If the underlying map is concurrently being modified, there are no guarantees given with respect to missing or duplicate items.

readMap

@Nonnull
public static ProcessorMetaSupplier readMap(@Nonnull
                                                     String mapName,
                                                     @Nonnull
                                                     com.hazelcast.client.config.ClientConfig clientConfig)

Returns a meta-supplier of processors that will fetch entries from a Hazelcast IMap in a remote cluster.

If the underlying map is concurrently being modified, there are no guarantees given with respect to missing or duplicate items.

writeMap

@Nonnull
public static ProcessorMetaSupplier writeMap(@Nonnull
                                                      String mapName)

Returns a meta-supplier of processors that will put data into a Hazelcast IMap.

writeMap

@Nonnull
public static ProcessorMetaSupplier writeMap(@Nonnull
                                                      String mapName,
                                                      @Nonnull
                                                      com.hazelcast.client.config.ClientConfig clientConfig)

Returns a meta-supplier of processors that will put data into a Hazelcast IMap in a remote cluster.

readList

@Nonnull
public static ProcessorMetaSupplier readList(@Nonnull
                                                      String listName)

Returns a meta-supplier of processors that emit items retrieved from an IMDG IList.

readList

@Nonnull
public static ProcessorMetaSupplier readList(@Nonnull
                                                      String listName,
                                                      @Nonnull
                                                      com.hazelcast.client.config.ClientConfig clientConfig)

Returns a meta-supplier of processors that emit items retrieved from an IMDG IList in a remote cluster.

writeList

@Nonnull
public static ProcessorSupplier writeList(@Nonnull
                                                   String listName)

Returns a meta-supplier of processors that write received items to an IMDG IList.

writeList

@Nonnull
public static ProcessorSupplier writeList(@Nonnull
                                                   String listName,
                                                   @Nonnull
                                                   com.hazelcast.client.config.ClientConfig clientConfig)

Returns a meta-supplier of processors that write received items to an IMDG IList in a remote cluster.

map

@Nonnull
public static <T,R> ProcessorSupplier map(@Nonnull
                                                   Distributed.Function<? super T,? extends R> mapper)

Returns a supplier of processor which, for each received item, emits the result of applying the given mapping function to it. If the result is null, nothing will be emitted. Therefore this processor can be used to implement filtering semantics as well.

Type Parameters:

T - type of received item

R - type of emitted item

Parameters:

mapper - the mapping function

filter

@Nonnull
public static <T> ProcessorSupplier filter(@Nonnull
                                                    Distributed.Predicate<? super T> predicate)

Returns a supplier of processor which emits the same items it receives, but only those that pass the given predicate.

Type Parameters:

T - type of received item

Parameters:

predicate - the predicate to test each received item aginst

flatMap

@Nonnull
public static <T,R> ProcessorSupplier flatMap(@Nonnull
                                                       Distributed.Function<? super T,? extends Traverser<? extends R>> mapper)

Returns a supplier of TransformP processor with the given item-to-traverser function.

Type Parameters:

T - received item type

R - emitted item type

Parameters:

mapper - function that maps the received item to a traverser over output items

groupAndAccumulate

@Nonnull
public static <T,K,A,R> ProcessorSupplier groupAndAccumulate(@Nonnull
                                                                      Distributed.Function<? super T,? extends K> keyExtractor,
                                                                      @Nonnull
                                                                      Distributed.Supplier<? extends A> supplier,
                                                                      @Nonnull
                                                                      Distributed.BiFunction<? super A,? super T,? extends A> accumulator,
                                                                      @Nonnull
                                                                      Distributed.BiFunction<? super K,? super A,? extends R> finisher)

Returns a supplier of processor with the following semantics:

• Accepts items of type T.
• Applies the key extractor to each item and obtains the key of type K.
• Stores for each key the result of applying the accumulator function to the previously accumulated value and the current item. The initial accumulated value is obtained from the supplier function.
• When all the input is consumed, begins emitting the accumulated results.
• Emits items of type R obtained by applying the finisher function to each seen key and its accumulated value.

Type Parameters:

T - type of received item

K - type of key

A - type of accumulated value

R - type of emitted item

Parameters:

keyExtractor - computes the key from the entry

supplier - supplies the initial accumulated value

accumulator - accumulates the result value across all entries under the same key

finisher - transforms a key and its accumulated value into the item to emit

groupAndAccumulate

@Nonnull
public static <T,A> ProcessorSupplier groupAndAccumulate(@Nonnull
                                                                  Distributed.Function<? super T,?> keyExtractor,
                                                                  @Nonnull
                                                                  Distributed.Supplier<? extends A> supplier,
                                                                  @Nonnull
                                                                  Distributed.BiFunction<? super A,? super T,? extends A> accumulator)

Convenience over groupAndAccumulate(keyExtractor, supplier, accumulator, finisher) with the constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<K, A>. Note that K isn't a part of the method's signature since nothing in the processor depends on it. The receiving processor will in any case have to perform an unchecked cast to Entry<K, A>.

Type Parameters:

T - type of received item

A - type of accumulated value

Parameters:

keyExtractor - computes the key from the entry

supplier - supplies the initial accumulated value

accumulator - accumulates the result value across all entries under the same key

groupAndAccumulate

@Nonnull
public static <T,A> ProcessorSupplier groupAndAccumulate(@Nonnull
                                                                  Distributed.Supplier<? extends A> supplier,
                                                                  @Nonnull
                                                                  Distributed.BiFunction<? super A,? super T,? extends A> accumulator)

Convenience over groupAndAccumulate(keyExtractor, supplier, accumulator, finisher) with identity function as the key extractor and constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<T, A>.

Type Parameters:

T - type of received item

A - type of accumulated value

Parameters:

supplier - supplies the initial accumulated value

accumulator - accumulates the result value across all entries under the same key

groupAndCollect

@Nonnull
public static <T,K,A,R> ProcessorSupplier groupAndCollect(@Nonnull
                                                                   Distributed.Function<? super T,? extends K> keyExtractor,
                                                                   @Nonnull
                                                                   Distributed.Supplier<? extends A> supplier,
                                                                   @Nonnull
                                                                   Distributed.BiConsumer<? super A,? super T> collector,
                                                                   @Nonnull
                                                                   Distributed.BiFunction<? super K,? super A,? extends R> finisher)

Returns a supplier of processor with the following semantics:

• Accepts items of type T.
• Applies the key extractor to each item and obtains the key of type K.
• Stores for each seen key an accumulated value container obtained from the supplier function.
• For each received item, applies the collector function to the accumulated value container and the item.
• When all the input is consumed, begins emitting the accumulated results.
• Emits items of type R obtained by applying the finisher function to each seen key and its accumulated value.

Type Parameters:

T - type of received item

K - type of key

A - type of accumulated value

R - type of emitted item

Parameters:

keyExtractor - computes the key from the entry

supplier - supplies the mutable result container

collector - collects the results of all entries under the same key into the mutable container

finisher - transforms a key and its result container into the item to emit

groupAndCollect

@Nonnull
public static <T,A> ProcessorSupplier groupAndCollect(@Nonnull
                                                               Distributed.Function<? super T,?> keyExtractor,
                                                               @Nonnull
                                                               Distributed.Supplier<? extends A> supplier,
                                                               @Nonnull
                                                               Distributed.BiConsumer<? super A,? super T> collector)

Convenience over groupAndCollect(keyExtractor, supplier, collector, finisher) with the constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<K, A>. Note that K isn't a part of the method's signature since nothing in the processor depends on it. The receiving processor will in any case have to perform an unchecked cast to Entry<K, A>.

Type Parameters:

T - type of received item

A - type of result container

Parameters:

keyExtractor - computes the key from the entry

supplier - supplies the mutable result container

collector - collects the results of all entries under the same key into the mutable container

groupAndCollect

@Nonnull
public static <T,A> ProcessorSupplier groupAndCollect(@Nonnull
                                                               Distributed.Supplier<? extends A> supplier,
                                                               @Nonnull
                                                               Distributed.BiConsumer<? super A,? super T> collector)

Convenience over groupAndCollect(keyExtractor, supplier, collector, finisher) with identity function as the key extractor and constructor of SimpleImmutableEntry as the finisher function, which means the processor emits items of type java.util.Map.Entry<T, A>.

Type Parameters:

T - type of received item

A - type of accumulated value

Parameters:

supplier - supplies the mutable result container

collector - collects the results of all entries under the same key into the mutable container

accumulate

@Nonnull
public static <T,A,R> ProcessorSupplier accumulate(@Nonnull
                                                            Distributed.Supplier<? extends A> supplier,
                                                            @Nonnull
                                                            Distributed.BiFunction<? super A,? super T,? extends A> accumulator,
                                                            @Nonnull
                                                            Distributed.Function<? super A,? extends R> finisher)

Returns a supplier of processor with the following semantics:

• Calls the supplier function to obtain the initial accumulated value.
• Accepts items of type T.
• Stores the result of applying the accumulator function to the previously accumulated value and the current item.
• When all the input is consumed, emits the result as a single item of type R, obtained by applying the finisher function to the accumulated value.

Type Parameters:

T - type of received item

A - type of accumulated value

R - type of emitted item

Parameters:

supplier - supplies the initial accumulated value

accumulator - accumulates the result value across all the input items

finisher - transforms the accumulated value into the item to emit

accumulate

@Nonnull
public static <T,A> ProcessorSupplier accumulate(@Nonnull
                                                          Distributed.Supplier<? extends A> supplier,
                                                          @Nonnull
                                                          Distributed.BiFunction<? super A,? super T,? extends A> accumulator)

Convenience over accumulate(supplier, accumulator, finisher) with identity function as the finisher, which means the processor emits an item of type A.

Type Parameters:

T - type of received item

A - type of accumulated value

Parameters:

supplier - supplies the initial accumulated value

accumulator - accumulates the result value across all the input items

collect

@Nonnull
public static <T,A,R> ProcessorSupplier collect(@Nonnull
                                                         Distributed.Supplier<? extends A> supplier,
                                                         @Nonnull
                                                         Distributed.BiConsumer<? super A,? super T> collector,
                                                         @Nonnull
                                                         Distributed.Function<? super A,? extends R> finisher)

Returns a supplier of processor with the following semantics:

• Calls the supplier function to obtain the mutable result container.
• Accepts items of type T.
• For each received item, calls the collector function with the result container and the current item.
• When all the input is consumed, emits the result as a single item of type R, obtained by applying the finisher function to the result container.

Type Parameters:

T - type of received item

A - type of accumulated value

R - type of emitted item

Parameters:

supplier - supplies the mutable result container

collector - collects the result across all the input items into the result container

finisher - transforms the result container into the item to emit

collect

@Nonnull
public static <T,A> ProcessorSupplier collect(@Nonnull
                                                       Distributed.Supplier<? extends A> supplier,
                                                       @Nonnull
                                                       Distributed.BiConsumer<? super A,? super T> collector)

Convenience over collect(supplier, collector, finisher) with identity function as the finisher, which means the processor emits an item of type A.

Type Parameters:

T - type of received item

A - type of result container

Parameters:

supplier - supplies the mutable result container

collector - collects the result across all the input items into the result container

countDistinct

@Nonnull
public static <T,K> ProcessorSupplier countDistinct(@Nonnull
                                                             Distributed.Function<T,K> keyExtractor)

Returns a supplier of CountDistinctP processors.

Type Parameters:

T - received item type

K - key type

Parameters:

keyExtractor - the key extractor function

countDistinct

@Nonnull
public static <T> ProcessorSupplier countDistinct()

Convenience over countDistinct(Distributed.Function) with identity function as the key extractor, which means the processor will emit the number of distinct items it has seen in the input.

Type Parameters:

T - type of received item

nonCooperative

@Nonnull
public static ProcessorSupplier nonCooperative(ProcessorSupplier wrapped)

Decorates a ProcessorSupplier into one that will declare all its processors non-cooperative. The wrapped supplier must return processors that are instanceof AbstractProcessor.