com.hazelcast.jet.core.processor

Class SinkProcessors

java.lang.Object

com.hazelcast.jet.core.processor.SinkProcessors

public final class SinkProcessors
extends Object

Static utility class with factories of sink processors (the terminators of the DAG data flow). For other kinds of processors refer to the package-level documentation.

Method Summary

Modifier and Type	Method and Description
static <E,K,V> ProcessorMetaSupplier	mergeMapP(String mapName, DistributedFunction<E,K> toKeyFn, DistributedFunction<E,V> toValueFn, DistributedBinaryOperator<V> mergeFn) Returns a supplier of processors for Sinks.mapWithMerging(String, DistributedFunction, DistributedFunction, DistributedBinaryOperator).
static <E,K,V> ProcessorMetaSupplier	mergeRemoteMapP(String mapName, ClientConfig clientConfig, DistributedFunction<E,K> toKeyFn, DistributedFunction<E,V> toValueFn, DistributedBinaryOperator<V> mergeFn) Returns a supplier of processors for Sinks.remoteMapWithMerging(String, ClientConfig, DistributedFunction, DistributedFunction, DistributedBinaryOperator).
static <E,K,V> ProcessorMetaSupplier	updateMapP(String mapName, DistributedFunction<E,K> toKeyFn, DistributedBiFunction<V,E,V> updateFn) Returns a supplier of processors for Sinks.mapWithEntryProcessor(String, DistributedFunction, DistributedFunction) .
static <T,K,V> ProcessorMetaSupplier	updateMapP(String mapName, DistributedFunction<T,K> toKeyFn, DistributedFunction<T,EntryProcessor<K,V>> toEntryProcessorFn) Returns a supplier of processors for Sinks.mapWithEntryProcessor(String, DistributedFunction, DistributedFunction).
static <E,K,V> ProcessorMetaSupplier	updateRemoteMapP(String mapName, ClientConfig clientConfig, DistributedFunction<E,K> toKeyFn, DistributedBiFunction<V,E,V> updateFn) Returns a supplier of processors for Sinks.remoteMapWithUpdating(String, ClientConfig, DistributedFunction , DistributedBiFunction).
static <T,K,V> ProcessorMetaSupplier	updateRemoteMapP(String mapName, ClientConfig clientConfig, DistributedFunction<T,K> toKeyFn, DistributedFunction<T,EntryProcessor<K,V>> toEntryProcessorFn) Returns a supplier of processors for Sinks.remoteMapWithEntryProcessor(String, ClientConfig, DistributedFunction, DistributedFunction).
static <W,T> DistributedSupplier<Processor>	writeBufferedP(DistributedFunction<? super Processor.Context,? extends W> createFn, DistributedBiConsumer<? super W,? super T> onReceiveFn, DistributedConsumer<? super W> flushFn, DistributedConsumer<? super W> destroyFn) Returns a supplier of processors for a vertex that drains all the items from the inbox to an internal buffered writer object.
static <B,T> DistributedSupplier<Processor>	writeBufferedP(DistributedFunction<Processor.Context,B> newBufferFn, DistributedBiConsumer<B,T> addToBufferFn, DistributedConsumer<B> flushBufferFn) Returns a supplier of processors for a vertex that drains all the items from its inbox to an intermediate buffer and then flushes the buffer.
static ProcessorMetaSupplier	writeCacheP(String cacheName) Returns a supplier of processors for Sinks.cache(String).
static ProcessorMetaSupplier	writeFileP(String directoryName) Returns a supplier of processors for Sinks.files(String).
static <T> ProcessorMetaSupplier	writeFileP(String directoryName, DistributedFunction<T,String> toStringFn) Returns a supplier of processors for Sinks.files(String, DistributedFunction).
static <T> ProcessorMetaSupplier	writeFileP(String directoryName, DistributedFunction<T,String> toStringFn, Charset charset, boolean append) Returns a supplier of processors for Sinks.files(String, DistributedFunction, Charset, boolean).
static ProcessorMetaSupplier	writeListP(String listName) Returns a supplier of processors for Sinks.list(String).
static ProcessorMetaSupplier	writeMapP(String mapName) Returns a supplier of processors for Sinks.map(String).
static ProcessorMetaSupplier	writeRemoteCacheP(String cacheName, ClientConfig clientConfig) Returns a supplier of processors for Sinks.remoteCache(String, ClientConfig).
static ProcessorMetaSupplier	writeRemoteListP(String listName, ClientConfig clientConfig) Returns a supplier of processors for Sinks.remoteList(String, ClientConfig).
static ProcessorMetaSupplier	writeRemoteMapP(String mapName, ClientConfig clientConfig) Returns a supplier of processors for Sinks.remoteMap(String, ClientConfig).
static <T> ProcessorMetaSupplier	writeSocketP(String host, int port, DistributedFunction<T,String> toStringFn, Charset charset) Returns a supplier of processors for Sinks.socket(String, int).

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

writeMapP

@Nonnull
public static ProcessorMetaSupplier writeMapP(@Nonnull
                                                       String mapName)

Returns a supplier of processors for Sinks.map(String).

writeRemoteMapP

@Nonnull
public static ProcessorMetaSupplier writeRemoteMapP(@Nonnull
                                                             String mapName,
                                                             @Nonnull
                                                             ClientConfig clientConfig)

Returns a supplier of processors for Sinks.remoteMap(String, ClientConfig).

mergeMapP

@Nonnull
public static <E,K,V> ProcessorMetaSupplier mergeMapP(@Nonnull
                                                               String mapName,
                                                               @Nonnull
                                                               DistributedFunction<E,K> toKeyFn,
                                                               @Nonnull
                                                               DistributedFunction<E,V> toValueFn,
                                                               @Nonnull
                                                               DistributedBinaryOperator<V> mergeFn)

Returns a supplier of processors for Sinks.mapWithMerging(String, DistributedFunction, DistributedFunction, DistributedBinaryOperator).

mergeRemoteMapP

@Nonnull
public static <E,K,V> ProcessorMetaSupplier mergeRemoteMapP(@Nonnull
                                                                     String mapName,
                                                                     @Nonnull
                                                                     ClientConfig clientConfig,
                                                                     @Nonnull
                                                                     DistributedFunction<E,K> toKeyFn,
                                                                     @Nonnull
                                                                     DistributedFunction<E,V> toValueFn,
                                                                     @Nonnull
                                                                     DistributedBinaryOperator<V> mergeFn)

Returns a supplier of processors for Sinks.remoteMapWithMerging(String, ClientConfig, DistributedFunction, DistributedFunction, DistributedBinaryOperator).

updateMapP

@Nonnull
public static <E,K,V> ProcessorMetaSupplier updateMapP(@Nonnull
                                                                String mapName,
                                                                @Nonnull
                                                                DistributedFunction<E,K> toKeyFn,
                                                                @Nonnull
                                                                DistributedBiFunction<V,E,V> updateFn)

Returns a supplier of processors for Sinks.mapWithEntryProcessor(String, DistributedFunction, DistributedFunction) .

updateRemoteMapP

@Nonnull
public static <E,K,V> ProcessorMetaSupplier updateRemoteMapP(@Nonnull
                                                                      String mapName,
                                                                      @Nonnull
                                                                      ClientConfig clientConfig,
                                                                      @Nonnull
                                                                      DistributedFunction<E,K> toKeyFn,
                                                                      @Nonnull
                                                                      DistributedBiFunction<V,E,V> updateFn)

Returns a supplier of processors for Sinks.remoteMapWithUpdating(String, ClientConfig, DistributedFunction , DistributedBiFunction).

updateMapP

@Nonnull
public static <T,K,V> ProcessorMetaSupplier updateMapP(@Nonnull
                                                                String mapName,
                                                                @Nonnull
                                                                DistributedFunction<T,K> toKeyFn,
                                                                @Nonnull
                                                                DistributedFunction<T,EntryProcessor<K,V>> toEntryProcessorFn)

Returns a supplier of processors for Sinks.mapWithEntryProcessor(String, DistributedFunction, DistributedFunction).

updateRemoteMapP

@Nonnull
public static <T,K,V> ProcessorMetaSupplier updateRemoteMapP(@Nonnull
                                                                      String mapName,
                                                                      @Nonnull
                                                                      ClientConfig clientConfig,
                                                                      @Nonnull
                                                                      DistributedFunction<T,K> toKeyFn,
                                                                      @Nonnull
                                                                      DistributedFunction<T,EntryProcessor<K,V>> toEntryProcessorFn)

Returns a supplier of processors for Sinks.remoteMapWithEntryProcessor(String, ClientConfig, DistributedFunction, DistributedFunction).

writeCacheP

@Nonnull
public static ProcessorMetaSupplier writeCacheP(@Nonnull
                                                         String cacheName)

Returns a supplier of processors for Sinks.cache(String).

writeRemoteCacheP

@Nonnull
public static ProcessorMetaSupplier writeRemoteCacheP(@Nonnull
                                                               String cacheName,
                                                               @Nonnull
                                                               ClientConfig clientConfig)

Returns a supplier of processors for Sinks.remoteCache(String, ClientConfig).

writeListP

@Nonnull
public static ProcessorMetaSupplier writeListP(@Nonnull
                                                        String listName)

Returns a supplier of processors for Sinks.list(String).

writeRemoteListP

@Nonnull
public static ProcessorMetaSupplier writeRemoteListP(@Nonnull
                                                              String listName,
                                                              @Nonnull
                                                              ClientConfig clientConfig)

Returns a supplier of processors for Sinks.remoteList(String, ClientConfig).

writeSocketP

public static <T> ProcessorMetaSupplier writeSocketP(@Nonnull
                                                     String host,
                                                     int port,
                                                     @Nonnull
                                                     DistributedFunction<T,String> toStringFn,
                                                     @Nonnull
                                                     Charset charset)

Returns a supplier of processors for Sinks.socket(String, int).

writeFileP

@Nonnull
public static <T> ProcessorMetaSupplier writeFileP(@Nonnull
                                                            String directoryName,
                                                            @Nonnull
                                                            DistributedFunction<T,String> toStringFn,
                                                            @Nonnull
                                                            Charset charset,
                                                            boolean append)

Returns a supplier of processors for Sinks.files(String, DistributedFunction, Charset, boolean).

writeFileP

@Nonnull
public static <T> ProcessorMetaSupplier writeFileP(@Nonnull
                                                            String directoryName,
                                                            @Nonnull
                                                            DistributedFunction<T,String> toStringFn)

Returns a supplier of processors for Sinks.files(String, DistributedFunction).

writeFileP

@Nonnull
public static ProcessorMetaSupplier writeFileP(@Nonnull
                                                        String directoryName)

Returns a supplier of processors for Sinks.files(String).

writeBufferedP

@Nonnull
public static <B,T> DistributedSupplier<Processor> writeBufferedP(@Nonnull
                                                                           DistributedFunction<Processor.Context,B> newBufferFn,
                                                                           @Nonnull
                                                                           DistributedBiConsumer<B,T> addToBufferFn,
                                                                           @Nonnull
                                                                           DistributedConsumer<B> flushBufferFn)

Returns a supplier of processors for a vertex that drains all the items from its inbox to an intermediate buffer and then flushes the buffer. This is a useful building block to implement sinks with explicit control over buffering and flushing.

Type Parameters:

B - type of buffer

T - type of received item

Parameters:

newBufferFn - supplies the buffer. The argument to this function is the context for the given processor.

addToBufferFn - adds an item to the buffer

flushBufferFn - flushes the buffer

writeBufferedP

@Nonnull
public static <W,T> DistributedSupplier<Processor> writeBufferedP(@Nonnull
                                                                           DistributedFunction<? super Processor.Context,? extends W> createFn,
                                                                           @Nonnull
                                                                           DistributedBiConsumer<? super W,? super T> onReceiveFn,
                                                                           @Nonnull
                                                                           DistributedConsumer<? super W> flushFn,
                                                                           @Nonnull
                                                                           DistributedConsumer<? super W> destroyFn)

Returns a supplier of processors for a vertex that drains all the items from the inbox to an internal buffered writer object. As each processor completes, it will dispose of its writer by calling destroyFn.

This is a useful building block to implement sinks with explicit control over resource management, buffering and flushing.

Type Parameters:

W - type of the writer

T - type of the received item

Parameters:

createFn - supplies the buffer. The argument to this function is the context for the given processor.

onReceiveFn - function that Jet calls upon receiving each item for the sink

flushFn - function that flushes the writer

destroyFn - function that destroys the writer