com.hazelcast.jet.cdc

Interface ChangeRecord

@EvolvingApi
public interface ChangeRecord

Information pertaining to a single data change event (insert, delete or update), affecting a single database record.

Each event has a key, identifying the affected record, and a value, describing the change to that record.

Most events have an operation which specifies the type of change (insert, delete or update). Events without an operation have specialized usage, for example heartbeats, and aren't supposed to affect the data model. You can observe and act upon them in a Jet CDC sink, but we discourage such usage.

All events have a timestamp specifying the moment when the change event occurred in the database. Normally this is the timestamp recorded in the database's change log, but since it has a finite size, the change stream begins with virtual events that reproduce the state of the table at the start of the change log. These events have an artificial timestamp. In principle, it should be easy to identify them because they have a separate SYNC operation instead of INSERT, however some databases emit INSERT events in both cases (a notable example is MySQL).

All events have a source-specific sequence which can be used to ensure their ordering. The sequence consists of two parts: a monotonically increasing numeric value and a source descriptor, which provides the scope of validity of the numeric value. This is needed because many CDC sources don't provide a globally valid sequence. For example, the sequence may be the offset in a write-ahead log. Then it makes sense to compare them only if they come from the same log file.

Since:

Jet 4.2

Method Summary

Modifier and Type	Method and Description
String	database() Returns the name of the database containing the record's table.
RecordPart	key() Returns the key part of the CDC event.
RecordPart	newValue() Returns the new value of the record.
RecordPart	oldValue() Returns the old value of the record.
Operation	operation() Returns the type of change this record describes (insert, delete or update).
String	schema() Returns the name of the schema containing the record's table.
long	sequenceSource() Specifies the source descriptor of the record's sequence.
long	sequenceValue() Specifies the numeric value part of the record's source sequence.
String	table() Returns the name of the table this record is part of.
long	timestamp() Specifies the moment when the change event occurred in the database.
String	toJson() Returns the raw JSON string from the CDC event underlying this ChangeRecord.
RecordPart	value() Returns the value part of the CDC event.

Method Detail

timestamp

long timestamp()

Specifies the moment when the change event occurred in the database. Normally this is the timestamp recorded in the database's change log, but since it has a finite size, the change stream begins with virtual events that reproduce the state of the table at the start of the change log. These events have an artificial timestamp. In principle, it should be easy to identify them because they have a separate SYNC operation instead of INSERT, however some databases emit INSERT events in both cases (a notable example is MySQL).

sequenceValue

long sequenceValue()

Specifies the numeric value part of the record's source sequence. As long as the source sequence doesn't change, the values will be monotonically increasing and can be used to impose ordering over the stream of records.

Since:

Jet 4.3

sequenceSource

long sequenceSource()

Specifies the source descriptor of the record's sequence. Any changes observed in its value should be interpreted as a reset in the sequence's numeric values. No ordering can be deduced for two records with different sequence sources.

Since:

Jet 4.3

operation

@Nonnull
Operation operation()

Returns the type of change this record describes (insert, delete or update). Some special events, like heartbeats, don't have an operation value.

Returns:

Operation.UNSPECIFIED if this ChangeRecord doesn't have an operation field, otherwise the appropriate Operation that matches the CDC record's operation field

database

@Nonnull
String database()

Returns the name of the database containing the record's table.

Returns:

name of the source database for the current record

schema

@Nonnull
String schema()
                throws UnsupportedOperationException

Returns the name of the schema containing the record's table. Note: not all databases have the concept of a schema (for example MySQL).

Returns:

name of the source schema for the current record

Throws:

UnsupportedOperationException

table

@Nonnull
String table()

Returns the name of the table this record is part of.

Returns:

name of the source table for the current record

key

@Nullable
RecordPart key()

Returns the key part of the CDC event. It identifies the affected record.

value

@Nonnull
RecordPart value()

Returns the value part of the CDC event. It includes fields like the timestamp, operation, and database record data.

For sync, insert and update operations the value describes the database record as it looks AFTER the event, so the latest image.

For delete operations the value describes the database record as it looked BEFORE the event, so the previous image.

newValue

@Nullable
RecordPart newValue()

Returns the new value of the record. For sync, insert and update operations the value describes the database record as it looks AFTER the event, so the latest state, for delete it returns null.

Since:

5.2

oldValue

@Nullable
RecordPart oldValue()

Returns the old value of the record. For update and delete operations the value describes the database record as it looks BEFORE the event, for sync and insert it returns null.

Since:

5.2

toJson

@Nonnull
String toJson()

Returns the raw JSON string from the CDC event underlying this ChangeRecord. You can use it if higher-level parsing (see other methods) fails for some reason (for example on some untested combination of database connector and version).