com.hazelcast.jet.aggregate

Class AggregateOperations

java.lang.Object

com.hazelcast.jet.aggregate.AggregateOperations

public final class AggregateOperations
extends Object

Utility class with factory methods for several useful windowing operations.

Method Summary

Modifier and Type	Method and Description
static <T> AggregateOperation1<T,List<Object>,List<Object>>	allOf(AggregateOperation1<? super T,?,?>... operations) Returns an operation, that calculates multiple aggregations and returns their value in List<Object>.
static <T> AggregateOperation1<T,LongDoubleAccumulator,Double>	averagingDouble(DistributedToDoubleFunction<T> getDoubleValueFn) Returns an operation that calculates the arithmetic mean of double values returned by the getDoubleValueFn function.
static <T> AggregateOperation1<T,LongLongAccumulator,Double>	averagingLong(DistributedToLongFunction<T> getLongValueFn) Returns an operation that calculates the arithmetic mean of long values returned by the getLongValueFn function.
static <T> AggregateOperation1<T,LongAccumulator,Long>	counting() Returns an operation that tracks the count of items in the window.
static <T> AggregateOperation1<T,LinTrendAccumulator,Double>	linearTrend(DistributedToLongFunction<T> getXFn, DistributedToLongFunction<T> getYFn) Returns an operation that computes a linear trend on the items in the window.
static <T,U,A,R> AggregateOperation1<T,?,R>	mapping(DistributedFunction<? super T,? extends U> mapFn, AggregateOperation1<? super U,A,R> downstream) Adapts an AggregateOperation1 accepting elements of type U to one accepting elements of type T by applying a mapping function to each input element before accumulation.
static <T> AggregateOperation1<T,MutableReference<T>,T>	maxBy(DistributedComparator<? super T> comparator) Returns an operation that returns the maximum item, according the given comparator.
static <T> AggregateOperation1<T,MutableReference<T>,T>	minBy(DistributedComparator<? super T> comparator) Returns an operation that returns the minimum item, according the given comparator.
static <T,A> AggregateOperation1<T,MutableReference<A>,A>	reducing(A emptyAccValue, DistributedFunction<? super T,? extends A> toAccValueFn, DistributedBinaryOperator<A> combineAccValuesFn, DistributedBinaryOperator<A> deductAccValueFn) A reducing operation maintains an accumulated value that starts out as emptyAccValue and is being iteratively transformed by applying the combine primitive to it and each stream item's accumulated value, as returned from toAccValueFn.
static <T> AggregateOperation1<T,DoubleAccumulator,Double>	summingDouble(DistributedToDoubleFunction<T> getDoubleValueFn) Returns an operation that tracks the sum of the quantity returned by getDoubleValueFn applied to each item in the window.
static <T> AggregateOperation1<T,LongAccumulator,Long>	summingLong(DistributedToLongFunction<T> getLongValueFn) Returns an operation that tracks the sum of the quantity returned by getLongValueFn applied to each item in the window.
static <T,C extends Collection<T>> AggregateOperation1<T,C,C>	toCollection(DistributedSupplier<C> createCollectionFn) Returns an AggregateOperation1 that accumulates the input elements into a new Collection.
static <T> AggregateOperation1<T,List<T>,List<T>>	toList() Returns an AggregateOperation1 that accumulates the input elements into a new ArrayList.
static <T,K,U> AggregateOperation1<T,Map<K,U>,Map<K,U>>	toMap(DistributedFunction<? super T,? extends K> getKeyFn, DistributedFunction<? super T,? extends U> getValueFn) Returns an AggregateOperation1 that accumulates elements into a HashMap whose keys and values are the result of applying the provided mapping functions to the input elements.
static <T,K,U> AggregateOperation1<T,Map<K,U>,Map<K,U>>	toMap(DistributedFunction<? super T,? extends K> getKeyFn, DistributedFunction<? super T,? extends U> getValueFn, DistributedBinaryOperator<U> mergeFn) Returns an AggregateOperation1 that accumulates elements into a HashMap whose keys and values are the result of applying the provided mapping functions to the input elements.
static <T,K,U,M extends Map<K,U>> AggregateOperation1<T,M,M>	toMap(DistributedFunction<? super T,? extends K> getKeyFn, DistributedFunction<? super T,? extends U> getValueFn, DistributedBinaryOperator<U> mergeFn, DistributedSupplier<M> createMapFn) Returns an AggregateOperation1 that accumulates elements into a Map whose keys and values are the result of applying the provided mapping functions to the input elements.
static <T> AggregateOperation1<T,?,Set<T>>	toSet() Returns an AggregateOperation1 that accumulates the input elements into a new HashSet.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

counting

@Nonnull
public static <T> AggregateOperation1<T,LongAccumulator,Long> counting()

Returns an operation that tracks the count of items in the window.

summingLong

@Nonnull
public static <T> AggregateOperation1<T,LongAccumulator,Long> summingLong(@Nonnull
                                                                                   DistributedToLongFunction<T> getLongValueFn)

Returns an operation that tracks the sum of the quantity returned by getLongValueFn applied to each item in the window.

Type Parameters:

T - Input item type

summingDouble

@Nonnull
public static <T> AggregateOperation1<T,DoubleAccumulator,Double> summingDouble(@Nonnull
                                                                                         DistributedToDoubleFunction<T> getDoubleValueFn)

Returns an operation that tracks the sum of the quantity returned by getDoubleValueFn applied to each item in the window.

Type Parameters:

T - Input item type

minBy

@Nonnull
public static <T> AggregateOperation1<T,MutableReference<T>,T> minBy(@Nonnull
                                                                              DistributedComparator<? super T> comparator)

Returns an operation that returns the minimum item, according the given comparator.

The implementation doesn't have the deduction function . See note here.

Type Parameters:

T - Input item type

maxBy

@Nonnull
public static <T> AggregateOperation1<T,MutableReference<T>,T> maxBy(@Nonnull
                                                                              DistributedComparator<? super T> comparator)

Returns an operation that returns the maximum item, according the given comparator.

The implementation doesn't have the deduction function . See note here.

Type Parameters:

T - Input item type

averagingLong

@Nonnull
public static <T> AggregateOperation1<T,LongLongAccumulator,Double> averagingLong(@Nonnull
                                                                                           DistributedToLongFunction<T> getLongValueFn)

Returns an operation that calculates the arithmetic mean of long values returned by the getLongValueFn function.

Type Parameters:

T - Input item type

averagingDouble

@Nonnull
public static <T> AggregateOperation1<T,LongDoubleAccumulator,Double> averagingDouble(@Nonnull
                                                                                               DistributedToDoubleFunction<T> getDoubleValueFn)

Returns an operation that calculates the arithmetic mean of double values returned by the getDoubleValueFn function.

Type Parameters:

T - Input item type

linearTrend

@Nonnull
public static <T> AggregateOperation1<T,LinTrendAccumulator,Double> linearTrend(@Nonnull
                                                                                         DistributedToLongFunction<T> getXFn,
                                                                                         @Nonnull
                                                                                         DistributedToLongFunction<T> getYFn)

Returns an operation that computes a linear trend on the items in the window. The operation will produce a double-valued coefficient that approximates the rate of change of y as a function of x, where x and y are long quantities extracted from each item by the two provided functions.

allOf

@SafeVarargs
 @Nonnull
public static <T> AggregateOperation1<T,List<Object>,List<Object>> allOf(@Nonnull
                                                                                                AggregateOperation1<? super T,?,?>... operations)

Returns an operation, that calculates multiple aggregations and returns their value in List<Object>.

Useful, if you want to calculate multiple values for the same window.

Parameters:

operations - Operations to calculate.

mapping

public static <T,U,A,R> AggregateOperation1<T,?,R> mapping(@Nonnull
                                                           DistributedFunction<? super T,? extends U> mapFn,
                                                           @Nonnull
                                                           AggregateOperation1<? super U,A,R> downstream)

Adapts an AggregateOperation1 accepting elements of type U to one accepting elements of type T by applying a mapping function to each input element before accumulation.

If the mapFn maps to null, the item won't be aggregated at all. This allows the mapping to be used as a filter at the same time.

This operation is useful if we cannot precede the aggregating vertex with a map() processors, which is useful

Type Parameters:

T - the type of the input elements

U - type of elements accepted by downstream operation

A - intermediate accumulation type of the downstream operation

R - result type of operation

Parameters:

mapFn - a function to be applied to the input elements

downstream - an operation which will accept mapped values

toCollection

public static <T,C extends Collection<T>> AggregateOperation1<T,C,C> toCollection(DistributedSupplier<C> createCollectionFn)

Returns an AggregateOperation1 that accumulates the input elements into a new Collection. The Collection is created by the provided factory.

Note: due to the distributed nature of processing the order might be unspecified.

Type Parameters:

T - the type of the input elements

C - the type of the resulting Collection

Parameters:

createCollectionFn - a Supplier which returns a new, empty Collection of the appropriate type

toList

public static <T> AggregateOperation1<T,List<T>,List<T>> toList()

Returns an AggregateOperation1 that accumulates the input elements into a new ArrayList.

Type Parameters:

T - the type of the input elements

toSet

public static <T> AggregateOperation1<T,?,Set<T>> toSet()

Returns an AggregateOperation1 that accumulates the input elements into a new HashSet.

Type Parameters:

T - the type of the input elements

toMap

public static <T,K,U> AggregateOperation1<T,Map<K,U>,Map<K,U>> toMap(DistributedFunction<? super T,? extends K> getKeyFn,
                                                                     DistributedFunction<? super T,? extends U> getValueFn)

Returns an AggregateOperation1 that accumulates elements into a HashMap whose keys and values are the result of applying the provided mapping functions to the input elements.

If the mapped keys contain duplicates (according to Object.equals(Object)), an IllegalStateException is thrown when the collection operation is performed. If the mapped keys may have duplicates, use toMap(DistributedFunction, DistributedFunction, DistributedBinaryOperator) instead.

Type Parameters:

T - the type of the input elements

K - the output type of the key mapping function

U - the output type of the value mapping function

Parameters:

getKeyFn - a function to extract the key from input item

getValueFn - a function to extract value from input item

See Also:

toMap(DistributedFunction, DistributedFunction, DistributedBinaryOperator), toMap(DistributedFunction, DistributedFunction, DistributedBinaryOperator, DistributedSupplier)

toMap

public static <T,K,U> AggregateOperation1<T,Map<K,U>,Map<K,U>> toMap(DistributedFunction<? super T,? extends K> getKeyFn,
                                                                     DistributedFunction<? super T,? extends U> getValueFn,
                                                                     DistributedBinaryOperator<U> mergeFn)

Returns an AggregateOperation1 that accumulates elements into a HashMap whose keys and values are the result of applying the provided mapping functions to the input elements.

If the mapped keys contains duplicates (according to Object.equals(Object)), the value mapping function is applied to each equal element, and the results are merged using the provided merging function.

Type Parameters:

T - the type of the input elements

K - the output type of the key mapping function

U - the output type of the value mapping function

Parameters:

getKeyFn - a function to extract the key from input item

getValueFn - a function to extract value from input item

mergeFn - a merge function, used to resolve collisions between values associated with the same key, as supplied to Map.merge(Object, Object, java.util.function.BiFunction)

See Also:

toMap(DistributedFunction, DistributedFunction), toMap(DistributedFunction, DistributedFunction, DistributedBinaryOperator, DistributedSupplier)

toMap

public static <T,K,U,M extends Map<K,U>> AggregateOperation1<T,M,M> toMap(DistributedFunction<? super T,? extends K> getKeyFn,
                                                                          DistributedFunction<? super T,? extends U> getValueFn,
                                                                          DistributedBinaryOperator<U> mergeFn,
                                                                          DistributedSupplier<M> createMapFn)

Returns an AggregateOperation1 that accumulates elements into a Map whose keys and values are the result of applying the provided mapping functions to the input elements.

If the mapped keys contain duplicates (according to Object.equals(Object)), the value mapping function is applied to each equal element, and the results are merged using the provided merging function. The Map is created by a provided createMapFn function.

Type Parameters:

T - the type of the input elements

K - the output type of the key mapping function

U - the output type of the value mapping function

M - the type of the resulting Map

Parameters:

getKeyFn - a function to extract the key from input item

getValueFn - a function to extract value from input item

mergeFn - a merge function, used to resolve collisions between values associated with the same key, as supplied to Map.merge(Object, Object, java.util.function.BiFunction)

createMapFn - a function which returns a new, empty Map into which the results will be inserted

See Also:

toMap(DistributedFunction, DistributedFunction), toMap(DistributedFunction, DistributedFunction, DistributedBinaryOperator)

reducing

@Nonnull
public static <T,A> AggregateOperation1<T,MutableReference<A>,A> reducing(@Nonnull
                                                                                   A emptyAccValue,
                                                                                   @Nonnull
                                                                                   DistributedFunction<? super T,? extends A> toAccValueFn,
                                                                                   @Nonnull
                                                                                   DistributedBinaryOperator<A> combineAccValuesFn,
                                                                                   @Nullable
                                                                                   DistributedBinaryOperator<A> deductAccValueFn)

A reducing operation maintains an accumulated value that starts out as emptyAccValue and is being iteratively transformed by applying the combine primitive to it and each stream item's accumulated value, as returned from toAccValueFn. The combine must be associative because it will also be used to combine partial results, and commutative because the encounter order of items is unspecified.

The optional deduct primitive allows Jet to compute the sliding window in O(1) time. It must undo the effects of a previous combine:

     A accVal;  (has some pre-existing value)
     A itemAccVal = toAccValueFn.apply(item);
     A combined = combineAccValuesFn.apply(accVal, itemAccVal);
     A deducted = deductAccValueFn.apply(combined, itemAccVal);
     assert deducted.equals(accVal);
 

Type Parameters:

T - type of the stream item

A - type of the accumulated value

Parameters:

emptyAccValue - the reducing operation's emptyAccValue element

toAccValueFn - transforms the stream item into its accumulated value

combineAccValuesFn - combines two accumulated values into one

deductAccValueFn - deducts the right-hand accumulated value from the left-hand one (optional)