Package com.hazelcast.jet.kafka

Class KafkaSinks.Builder<E>

java.lang.Object
com.hazelcast.jet.kafka.KafkaSinks.Builder<E>
Type Parameters:
E - type of stream item
Enclosing class:
KafkaSinks
public static final class KafkaSinks.Builder<E> extends Object
A builder for Kafka sink.

  • Method Details

    • topic

      @Nonnull public KafkaSinks.Builder<E> topic(String topic)
      Sets the topic to write the messages to, if you write all messages to a single topic. If you write each message to a different topic, use toRecordFn(FunctionEx). If you call this method, you can't call the toRecordFn() method.
      Parameters:
      topic - the topic name
      Returns:
      this instance for fluent API

    • extractKeyFn

      @Nonnull public KafkaSinks.Builder<E> extractKeyFn(@Nonnull FunctionEx<? super E,?> extractKeyFn)
      Sets the function to extract the key from the stream items. You can't use this method in combination with toRecordFn(FunctionEx).

      The default is to use null key.

      Parameters:
      extractKeyFn - a function to extract the key from the stream item
      Returns:
      this instance for fluent API

    • extractValueFn

      @Nonnull public KafkaSinks.Builder<E> extractValueFn(@Nonnull FunctionEx<? super E,?> extractValueFn)
      Sets the function to extract the value from the stream items. You can't use this method in combination with toRecordFn(FunctionEx).

      The default is to use the input item directly.

      Parameters:
      extractValueFn - a function to extract the value from the stream item
      Returns:
      this instance for fluent API

    • toRecordFn

      @Nonnull public KafkaSinks.Builder<E> toRecordFn(@Nullable FunctionEx<? super E,? extends org.apache.kafka.clients.producer.ProducerRecord<?,?>> toRecordFn)
      Sets the function to convert stream items into Kafka's ProducerRecord. This method allows you to specify all aspects of the record (different topic for each message, partition, timestamp...). If you use this method, you can't use topic(String), extractKeyFn(FunctionEx) or extractValueFn(FunctionEx).
      Parameters:
      toRecordFn - a function to convert stream items into Kafka's ProducerRecord
      Returns:
      this instance for fluent API

    • exactlyOnce

      @Nonnull public KafkaSinks.Builder<E> exactlyOnce(boolean enable)
      Enables or disables the exactly-once behavior of the sink using two-phase commit of state snapshots. If enabled, the processing guarantee of the job must be set to exactly-once, otherwise the sink's guarantee will match that of the job. In other words, sink's guarantee cannot be higher than job's, but can be lower to avoid the additional overhead.

      See KafkaSinks.kafka(Properties) for more information.

      The default value is true.

      Parameters:
      enable - If true, sink's guarantee will match the job guarantee. If false, sink's guarantee will be at-least-once even if job's is exactly-once
      Returns:
      this instance for fluent API

    • build

      @Nonnull public Sink<E> build()
      Builds the Sink object that you pass to the GeneralStage.writeTo(Sink) method.