public final class HdfsSinks extends Object
|Modifier and Type||Method and Description|
Returns a sink that writes to Apache Hadoop HDFS.
@Nonnull public static <E,K,V> Sink<E> hdfs(@Nonnull org.apache.hadoop.mapred.JobConf jobConf, @Nonnull FunctionEx<? super E,K> extractKeyF, @Nonnull FunctionEx<? super E,V> extractValueF)
The sink creates a number of files in the output path, identified by the
cluster member ID and the
Processor ID. Unlike MapReduce, the
data in the files is not sorted by key.
JobConf must specify an
No state is saved to snapshot for this sink. After the job is restarted, the items will likely be duplicated, providing an at-least-once guarantee.
Default local parallelism for this processor is 2 (or less if less CPUs are available).
E- stream item type
K- type of key to write to HDFS
V- type of value to write to HDFS
JobConfused for output format configuration
extractKeyF- mapper to map a key to another key
extractValueF- mapper to map a value to another value
Copyright © 2019 Hazelcast, Inc.. All rights reserved.