Package com.hazelcast.jet.pulsar

Class PulsarConsumerBuilder<M,T>

java.lang.Object

com.hazelcast.jet.pulsar.PulsarConsumerBuilder<M,T>

Type Parameters:

M - the type of the value of message read by PulsarConsumer

T - the type of the emitted item after projection.

public final class PulsarConsumerBuilder<M,T>
extends Object

See PulsarSources.pulsarConsumerBuilder(List, SupplierEx, SupplierEx, FunctionEx)}

Since:

6.0

Constructor Summary

Constructors

Constructor	Description
PulsarConsumerBuilder(SupplierEx<org.apache.pulsar.client.api.Schema<M>> schemaSupplier, FunctionEx<org.apache.pulsar.client.api.Message<M>,T> projectionFn)
PulsarConsumerBuilder(List<String> topics, SupplierEx<org.apache.pulsar.client.api.PulsarClient> connectionSupplier, SupplierEx<org.apache.pulsar.client.api.Schema<M>> schemaSupplier, FunctionEx<org.apache.pulsar.client.api.Message<M>,T> projectionFn)	All-required fields constructor of Pulsar consumer, kept for compatibility with older code from hazeclast-jet-contrib repository.

Method Summary

Modifier and Type	Method	Description
PulsarConsumerBuilder<M,T>	batchReceivePolicySupplier(SupplierEx<org.apache.pulsar.client.api.BatchReceivePolicy> batchReceivePolicySupplier)
StreamSource<T>	build()	Creates a Pulsar consumer source with the configured projection.
PulsarConsumerBuilder<M,T>	connectionSupplier(SupplierEx<org.apache.pulsar.client.api.PulsarClient> connectionSupplier)
PulsarConsumerBuilder<M,T>	consumerConfig(Map<String,Object> consumerConfig)
PulsarConsumerBuilder<M,T>	consumerCustomizer(ConsumerEx<@NonNull org.apache.pulsar.client.api.ConsumerBuilder<M>> consumerCustomizer)	Allows additional customization and invocation of other ConsumerBuilder method, that doesn't have a representative here.
PulsarConsumerBuilder<M,T>	dataConnectionRef(DataConnectionRef dataConnectionRef)
PulsarConsumerBuilder<M,T>	preferredLocalParallelism(int preferredLocalParallelism)	Preferred local parallelism for every processor created by this builder.
<NEW_T> PulsarConsumerBuilder<M,NEW_T>	projectionFn(FunctionEx<@NonNull org.apache.pulsar.client.api.Message<M>,NEW_T> projectionFn)
PulsarConsumerBuilder<M,T>	subscriberName(String subscriberName)
PulsarConsumerBuilder<M,T>	topic(String... topics)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

PulsarConsumerBuilder

public PulsarConsumerBuilder(@Nonnull
 SupplierEx<org.apache.pulsar.client.api.Schema<M>> schemaSupplier,
 @Nonnull
 FunctionEx<org.apache.pulsar.client.api.Message<M>,T> projectionFn)

PulsarConsumerBuilder

public PulsarConsumerBuilder(@Nonnull
 List<String> topics,
 @Nonnull
 SupplierEx<org.apache.pulsar.client.api.PulsarClient> connectionSupplier,
 @Nonnull
 SupplierEx<org.apache.pulsar.client.api.Schema<M>> schemaSupplier,
 @Nonnull
 FunctionEx<org.apache.pulsar.client.api.Message<M>,T> projectionFn)

All-required fields constructor of Pulsar consumer, kept for compatibility with older code from hazeclast-jet-contrib repository.

Parameters:

topics - the topics to consume, at least one is required

connectionSupplier - Pulsar client supplier

schemaSupplier - supplies the schema for consuming messages

projectionFn - converts a Pulsar message to an emitted item.

Method Details

topic

@Nonnull
public PulsarConsumerBuilder<M,T> topic(@Nonnull
 String... topics)

subscriberName

@Nonnull
public PulsarConsumerBuilder<M,T> subscriberName(@Nonnull
 String subscriberName)

connectionSupplier

@Nonnull
public PulsarConsumerBuilder<M,T> connectionSupplier(@Nonnull
 SupplierEx<org.apache.pulsar.client.api.PulsarClient> connectionSupplier)

dataConnectionRef

@Nonnull
public PulsarConsumerBuilder<M,T> dataConnectionRef(@Nonnull
 DataConnectionRef dataConnectionRef)

projectionFn

@Nonnull
public <NEW_T> PulsarConsumerBuilder<M,NEW_T> projectionFn(@Nonnull
 FunctionEx<@NonNull org.apache.pulsar.client.api.Message<M>,NEW_T> projectionFn)

consumerCustomizer

@Nonnull
public PulsarConsumerBuilder<M,T> consumerCustomizer(@Nonnull
 ConsumerEx<@NonNull org.apache.pulsar.client.api.ConsumerBuilder<M>> consumerCustomizer)

Allows additional customization and invocation of other ConsumerBuilder method, that doesn't have a representative here.

consumerConfig

@Nonnull
public PulsarConsumerBuilder<M,T> consumerConfig(@Nonnull
 Map<String,Object> consumerConfig)

Parameters:

consumerConfig - Pulsar consumer settings, subject to the same restrictions as consumerCustomizer(ConsumerEx)

batchReceivePolicySupplier

@Nonnull
public PulsarConsumerBuilder<M,T> batchReceivePolicySupplier(@Nonnull
 SupplierEx<org.apache.pulsar.client.api.BatchReceivePolicy> batchReceivePolicySupplier)

Parameters:

batchReceivePolicySupplier - supplies the batch receive policy for the consumer

preferredLocalParallelism

public PulsarConsumerBuilder<M,T> preferredLocalParallelism(int preferredLocalParallelism)

Preferred local parallelism for every processor created by this builder.

build

@Nonnull
public StreamSource<T> build()

Creates a Pulsar consumer source with the configured projection.

Each physical topic is assigned to one processor using an exclusive, durable subscription. Use a subscription name dedicated to this source. Partition changes are discovered when the job restarts.

With AT_LEAST_ONCE or EXACTLY_ONCE, the source acknowledges messages only after a successful snapshot and filters replayed messages using snapshotted positions. With NONE, messages are acknowledged after emission. The broker must retain unacknowledged messages: message TTL, backlog eviction and subscription expiration must not discard them. Non-persistent topics cannot be used with snapshotting.

Recovery uses the latest committed snapshot and the existing subscription. Restoring an older exported snapshot after newer acknowledgements is not supported. Snapshots written by the earlier consumer implementation are incompatible. End-to-end exactly-once also requires suitable downstream processors and sinks.