Package com.hazelcast.jet.pipeline

Class MapSinkBuilder<T,K,V>

java.lang.Object
com.hazelcast.jet.pipeline.MapSinkBuilder<T,K,V>
Type Parameters:
T - specifies type of input that will be written to sink
K - specifies key type of map sink
V - specifies value type of map sink
public class MapSinkBuilder<T,K,V> extends Object
Builder for a map that is used as sink.

Builds a local Map sink unless one of dataConnectionRef or clientConfig is provided, in which case a remote sink is built. The required parameters are:

  • mapName
  • toKeyFn - key-extracting function
  • one of toValue/updateFn - key ex

This sink provides the exactly-once guarantee thanks to idempotent updates. It means that the value with the same key is not appended, but overwritten. After the job is restarted from snapshot, duplicate items will not change the state in the target map.

Since:
5.4

  • Constructor Details

    • MapSinkBuilder

      public MapSinkBuilder(@Nonnull String mapName)
      Creates MapSinkBuilder to build a local or remote Map sink
      Parameters:
      mapName - name of the map to sink into

  • Method Details

    • dataConnectionRef

      public MapSinkBuilder<T,K,V> dataConnectionRef(DataConnectionRef dataConnectionRef)
      Sets the DataConnectionRef reference to a HazelcastDataConnection to use for remote Map sink.

      Only one of dataConnectionRef and clientConfig can be set.

      Parameters:
      dataConnectionRef - reference to a HazelcastDataConnection

    • clientConfig

      public MapSinkBuilder<T,K,V> clientConfig(ClientConfig clientConfig)
      Sets the ClientConfig with configuration for a Hazelcast client to use for remote Map sink.

      Only one of dataConnectionRef and clientConfig can be set.

      Parameters:
      clientConfig - remote Hazelcast client configuration

    • toKeyFn

      public MapSinkBuilder<T,K,V> toKeyFn(FunctionEx<? super T,? extends K> toKeyFn)
      Set the key-extracting function. The resulting value will be used as the key in the sink IMap.

      The function must be Serializable.

      Parameters:
      toKeyFn - function to extract key from incoming items

    • toValueFn

      public MapSinkBuilder<T,K,V> toValueFn(FunctionEx<? super T,? extends V> toValueFn)
      Set the function to extract a value from the incoming items. The value will be put into the IMap under key extracted using toKeyFn.

      Only one of toValueFn or updateFn can be set. Optionally mergeFn can be set together with `toValueFn`.

      The function must be Serializable.

      The given functions must be stateless and cooperative.

      Parameters:
      toValueFn - function to extract value from incoming items

    • updateFn

      public MapSinkBuilder<T,K,V> updateFn(BiFunctionEx<? super V,? super T,? extends V> updateFn)
      Set the function to update the value in Hazelcast IMap.

      For each item it receives, it applies toKeyFn to get the key and then applies updateFn to the existing value in the map and the received item to acquire the new value to associate with the key. If the new value is null, it removes the key from the map. Expressed as code, the sink performs the equivalent of the following for each item: 

       K key = toKeyFn.apply(item);
 V oldValue = map.get(key);
 V newValue = updateFn.apply(oldValue, item);
 if (newValue == null)
     map.remove(key);
 else
     map.put(key, newValue);

      Note: This operation is NOT lock-aware, it will process the entries no matter if they are locked or not. Use MapSinkEntryProcessorBuilder if you need locking.

      The function must be Serializable.

      The given functions must be stateless and cooperative.

      Parameters:
      updateFn - function that receives the existing map value and the item and returns the new map value

    • mergeFn

      public MapSinkBuilder<T,K,V> mergeFn(BinaryOperatorEx<V> mergeFn)
      Set the function to merge the existing value with new value.

      If the map already contains the key, it applies the given mergeFn to resolve the existing and the proposed value into the value to use. If the value comes out as null, it removes the key from the map. Expressed as code, the sink performs the equivalent of the following for each item: 

       K key = toKeyFn.apply(item);
 V oldValue = map.get(key);
 V newValue = toValueFn.apply(item);
 V resolved = (oldValue == null)
            ? newValue
            : mergeFn.apply(oldValue, newValue);
 if (value == null)
     map.remove(key);
 else
     map.put(key, value);
      Note: This operation is NOT lock-aware, it will process the entries no matter if they are locked or not. Use MapSinkEntryProcessorBuilder if you need locking.

      The function must be Serializable.

      The given functions must be stateless and cooperative.

      Parameters:
      mergeFn - function that merges the existing value with the value acquired from the received item

    • build

      public Sink<T> build()
      Build the sink.

      The default local parallelism for this sink is 1.

      Returns:
      the sink