com.hazelcast.jet.cdc.postgres

Class PostgresCdcSources

java.lang.Object

com.hazelcast.jet.cdc.postgres.PostgresCdcSources

@EvolvingApi
public final class PostgresCdcSources
extends Object

Contains factory methods for creating change data capture sources based on PostgreSQL databases.

Since:

Jet 4.2

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PostgresCdcSources.Builder Builder for configuring a CDC source that streams change data from a PostgreSQL database to Hazelcast Jet.
static class	PostgresCdcSources.PostgresSnapshotMode Possible postgres snapshot modes, that does not require additional mandatory parameters to be set.

Method Summary

Modifier and Type	Method and Description
static PostgresCdcSources.Builder	postgres(String name) Creates a CDC source that streams change data from a PostgreSQL database to Hazelcast Jet.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

postgres

@Nonnull
public static PostgresCdcSources.Builder postgres(@Nonnull
                                                           String name)

Creates a CDC source that streams change data from a PostgreSQL database to Hazelcast Jet.

You can configure how the source will behave if the database connection breaks, by passing one of the retry strategies to setReconnectBehavior().

The default reconnect behavior is never, which treats any connection failure as an unrecoverable problem and triggers the failure of the source and the entire job.

Other behavior options, which specify that retry attempts should be made, will result in the source initiating reconnects to the database.

There is a further setting influencing reconnect behavior, specified via setShouldStateBeResetOnReconnect(). The boolean flag passed in specifies what should happen to the connector's state on reconnect, whether it should be kept or reset. If the state is kept, then database snapshotting should not be repeated and streaming the WAL should resume at the position where it left off. If the state is reset, then the source will behave as on its initial start, so will do a database snapshot and will start tailing the WAL where it syncs with the database snapshot's end.

You can also configure how often the source will send feedback about processed change record offsets to the backing database via setCommitPeriod(). The replication slots of the database will clean up their internal data structures based on this feedback. A commit period of 0 means that the source will commit offsets after every batch of change records. Also, important to note that periodic commits happen only in the case of jobs without processing guarantees. For jobs offering processing guarantees, the source will ignore this setting and commit offsets as part of the state snapshotting process. So the setting governing them will be JobConfig.setSnapshotIntervalMillis.

Parameters:

name - name of this source, needs to be unique, will be passed to the underlying Kafka Connect source

Returns:

builder that can be used to set source properties and also to construct the source once configuration is done