com.hazelcast.jet.kafka

Class KafkaSinks.Builder<E>

java.lang.Object

com.hazelcast.jet.kafka.KafkaSinks.Builder<E>

Type Parameters:

E - type of stream item

Enclosing class:

KafkaSinks

public static final class KafkaSinks.Builder<E>
extends Object

A builder for Kafka sink.

Method Summary

Modifier and Type	Method and Description
Sink<E>	build() Builds the Sink object that you pass to the GeneralStage.writeTo(Sink) method.
KafkaSinks.Builder<E>	exactlyOnce(boolean enable) Enables or disables the exactly-once behavior of the sink using two-phase commit of state snapshots.
KafkaSinks.Builder<E>	extractKeyFn(FunctionEx<? super E,?> extractKeyFn) Sets the function to extract the key from the stream items.
KafkaSinks.Builder<E>	extractValueFn(FunctionEx<? super E,?> extractValueFn) Sets the function to extract the value from the stream items.
KafkaSinks.Builder<E>	topic(String topic) Sets the topic to write the messages to, if you write all messages to a single topic.
KafkaSinks.Builder<E>	toRecordFn(FunctionEx<? super E,? extends org.apache.kafka.clients.producer.ProducerRecord<?,?>> toRecordFn) Sets the function to convert stream items into Kafka's ProducerRecord.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

topic

@Nonnull
public KafkaSinks.Builder<E> topic(String topic)

Sets the topic to write the messages to, if you write all messages to a single topic. If you write each message to a different topic, use toRecordFn(FunctionEx). If you call this method, you can't call the toRecordFn() method.

Parameters:

topic - the topic name

Returns:

this instance for fluent API

extractKeyFn

@Nonnull
public KafkaSinks.Builder<E> extractKeyFn(@Nonnull
                                                   FunctionEx<? super E,?> extractKeyFn)

Sets the function to extract the key from the stream items. You can't use this method in combination with toRecordFn(FunctionEx).

The default is to use null key.

Parameters:

extractKeyFn - a function to extract the key from the stream item

Returns:

this instance for fluent API

extractValueFn

@Nonnull
public KafkaSinks.Builder<E> extractValueFn(@Nonnull
                                                     FunctionEx<? super E,?> extractValueFn)

Sets the function to extract the value from the stream items. You can't use this method in combination with toRecordFn(FunctionEx).

The default is to use the input item directly.

Parameters:

extractValueFn - a function to extract the value from the stream item

Returns:

this instance for fluent API

toRecordFn

@Nonnull
public KafkaSinks.Builder<E> toRecordFn(@Nullable
                                                 FunctionEx<? super E,? extends org.apache.kafka.clients.producer.ProducerRecord<?,?>> toRecordFn)

Sets the function to convert stream items into Kafka's ProducerRecord. This method allows you to specify all aspects of the record (different topic for each message, partition, timestamp...). If you use this method, you can't use topic(String), extractKeyFn(FunctionEx) or extractValueFn(FunctionEx).

Parameters:

toRecordFn - a function to convert stream items into Kafka's ProducerRecord

Returns:

this instance for fluent API

exactlyOnce

@Nonnull
public KafkaSinks.Builder<E> exactlyOnce(boolean enable)

Enables or disables the exactly-once behavior of the sink using two-phase commit of state snapshots. If enabled, the processing guarantee of the job must be set to exactly-once, otherwise the sink's guarantee will match that of the job. In other words, sink's guarantee cannot be higher than job's, but can be lower to avoid the additional overhead.

See KafkaSinks.kafka(Properties) for more information.

The default value is true.

Parameters:

enable - If true, sink's guarantee will match the job guarantee. If false, sink's guarantee will be at-least-once even if job's is exactly-once

Returns:

this instance for fluent API

build

@Nonnull
public Sink<E> build()

Builds the Sink object that you pass to the GeneralStage.writeTo(Sink) method.