Package com.hazelcast.jet.pipeline

Interface IMapExtension.IMapGeneralStage<T>

Type Parameters:
T - type of the input item
All Known Subinterfaces:
IMapExtension.IMapBatchStage<T>, IMapExtension.IMapStreamStage<T>
Enclosing interface:
IMapExtension<T,K>
public static interface IMapExtension.IMapGeneralStage<T>
Extends Pipeline stage with convenient methods utilising IMap in addition to GeneralStage.mapUsingIMap(java.lang.String, com.hazelcast.function.FunctionEx<? super T, ? extends K>, com.hazelcast.function.BiFunctionEx<? super T, ? super V, ? extends R>) available in basic stages.
Since:
5.7

  • Method Summary

    Modifier and Type
    Method
    Description
    default <K, V, R> com.hazelcast.jet.pipeline.GeneralStage<R>
    mapUsingPutIfAbsent(com.hazelcast.map.IMap<K,V> iMap, com.hazelcast.function.FunctionEx<? super T,? extends K> lookupKeyFn, com.hazelcast.function.FunctionEx<? super T,? extends V> newValueFn, com.hazelcast.jet.function.TriFunction<? super T,? super V,? super V,? extends R> mapFn)
    Attaches a mapping stage where for each item a IMap.putIfAbsent(K, V) operation is performed in the supplied IMap.
    <K, V, R> com.hazelcast.jet.pipeline.GeneralStage<R>
    mapUsingPutIfAbsent(String mapName, com.hazelcast.function.FunctionEx<? super T,? extends K> lookupKeyFn, com.hazelcast.function.FunctionEx<? super T,? extends V> newValueFn, com.hazelcast.jet.function.TriFunction<? super T,? super V,? super V,? extends R> mapFn)
    Attaches a mapping stage where for each item a IMap.putIfAbsent(K, V) operation is performed in the IMap with the supplied name.

  • Method Details

    • mapUsingPutIfAbsent

      @Nonnull <K, V, R> com.hazelcast.jet.pipeline.GeneralStage<R> mapUsingPutIfAbsent(@Nonnull String mapName, @Nonnull com.hazelcast.function.FunctionEx<? super T,? extends K> lookupKeyFn, @Nonnull com.hazelcast.function.FunctionEx<? super T,? extends V> newValueFn, @Nonnull com.hazelcast.jet.function.TriFunction<? super T,? super V,? super V,? extends R> mapFn)
      Attaches a mapping stage where for each item a IMap.putIfAbsent(K, V) operation is performed in the IMap with the supplied name. The result of IMap.putIfAbsent(K, V) (previous value associated with the key or null) is merged with the item and emitted.

      If the result of the mapping is null, it emits nothing. Therefore, this stage can be used to implement filtering semantics as well.

      The mapping logic is equivalent to: 

      
 K key = lookupKeyFn.apply(item);
 V newValue = newValueFn.apply(item);
 V prevValue = map.putIfAbsent(key, newValue);
 return mapFn.apply(item, prevValue, newValue);

      This operation and the IMap do not participate in Jet's fault tolerance protocol. If upstream stage or source retries processing, it will be invoked again. However, keep in mind that IMap.putIfAbsent(K, V) is not idempotent: when retried it may produce a different result (previous value) than in the original invocation. The mapping function and pipeline logic must be able to deal with this.

      This method does not use transactions for IMap. All updates to IMap are immediately visible regardless of Jet's fault tolerance protocol (snapshots).

      Type Parameters:
      K - type of the key in the IMap
      V - type of the value in the IMap
      R - type of the output item
      Parameters:
      mapName - name of the IMap
      lookupKeyFn - a function which returns the IMap key. Must not return null. It must be stateless and cooperative.
      newValueFn - a function which returns the IMap value to be associated with the key. Must not return null. It must be stateless and cooperative.
      mapFn - the mapping function. It must be stateless and cooperative.
      Returns:
      the newly attached stage

    • mapUsingPutIfAbsent

      @Nonnull default <K, V, R> com.hazelcast.jet.pipeline.GeneralStage<R> mapUsingPutIfAbsent(@Nonnull com.hazelcast.map.IMap<K,V> iMap, @Nonnull com.hazelcast.function.FunctionEx<? super T,? extends K> lookupKeyFn, @Nonnull com.hazelcast.function.FunctionEx<? super T,? extends V> newValueFn, @Nonnull com.hazelcast.jet.function.TriFunction<? super T,? super V,? super V,? extends R> mapFn)
      Attaches a mapping stage where for each item a IMap.putIfAbsent(K, V) operation is performed in the supplied IMap. The result of IMap.putIfAbsent(K, V) (previous value associated with the key or null) is merged with the item and emitted.

      If the result of the mapping is null, it emits nothing. Therefore, this stage can be used to implement filtering semantics as well.

      The mapping logic is equivalent to: 

      
 K key = lookupKeyFn.apply(item);
 V newValue = newValueFn.apply(item);
 V prevValue = map.putIfAbsent(key, newValue);
 return mapFn.apply(item, prevValue, newValue);

      This operation and the IMap do not participate in Jet's fault tolerance protocol. If upstream stage or source retries processing, it will be invoked again. However, keep in mind that IMap.putIfAbsent(K, V) is not idempotent: when retried it may produce a different result (previous value) than in the original invocation. The mapping function and pipeline logic must be able to deal with this.

      This method does not use transactions for IMap. All updates to IMap are immediately visible regardless of Jet's fault tolerance protocol (snapshots).

      Type Parameters:
      K - type of the key in the IMap
      V - type of the value in the IMap
      R - type of the output item
      Parameters:
      iMap - the IMap to use
      lookupKeyFn - a function which returns the IMap key. Must not return null. It must be stateless and cooperative.
      newValueFn - a function which returns the IMap value to be associated with the key. Must not return null. It must be stateless and cooperative.
      mapFn - the mapping function. It must be stateless and cooperative.
      Returns:
      the newly attached stage