Package com.hazelcast.jet.cdc

Interface ChangeRecord

@EvolvingApi public interface ChangeRecord
Information pertaining to a single data change event (insert, delete or update), affecting a single database record.

Each event has a key, identifying the affected record, and a value, describing the change to that record.

Most events have an operation which specifies the type of change (insert, delete or update). Events without an operation have specialized usage, for example heartbeats, and aren't supposed to affect the data model. You can observe and act upon them in a Jet CDC sink, but we discourage such usage.

All events have a timestamp specifying the moment when the change event occurred in the database. Normally this is the timestamp recorded in the database's change log, but since it has a finite size, the change stream begins with virtual events that reproduce the state of the table at the start of the change log. These events have an artificial timestamp. In principle, it should be easy to identify them because they have a separate SYNC operation instead of INSERT, however some databases emit INSERT events in both cases (a notable example is MySQL).

All events have a source-specific sequence which can be used to ensure their ordering. The sequence consists of two parts: a monotonically increasing numeric value and a source descriptor, which provides the scope of validity of the numeric value. This is needed because many CDC sources don't provide a globally valid sequence. For example, the sequence may be the offset in a write-ahead log. Then it makes sense to compare them only if they come from the same log file.

Since:
Jet 4.2

  • Method Summary

    Modifier and Type
    Method
    Description
    String
    database()
    Returns the name of the database containing the record's table.
    RecordPart
    key()
    Returns the key part of the CDC event.
    RecordPart
    newValue()
    Returns the new value of the record.
    RecordPart
    oldValue()
    Returns the old value of the record.
    Operation
    operation()
    Returns the type of change this record describes (insert, delete or update).
    String
    schema()
    Returns the name of the schema containing the record's table.
    long
    sequenceSource()
    Specifies the source descriptor of the record's sequence.
    long
    sequenceValue()
    Specifies the numeric value part of the record's source sequence.
    String
    table()
    Returns the name of the table this record is part of.
    long
    timestamp()
    Specifies the moment when the change event occurred in the database.
    String
    toJson()
    Returns the raw JSON string from the CDC event underlying this ChangeRecord.
    RecordPart
    value()
    Returns the value part of the CDC event.

  • Method Details

    • timestamp

      long timestamp()
      Specifies the moment when the change event occurred in the database. Normally this is the timestamp recorded in the database's change log, but since it has a finite size, the change stream begins with virtual events that reproduce the state of the table at the start of the change log. These events have an artificial timestamp. In principle, it should be easy to identify them because they have a separate SYNC operation instead of INSERT, however some databases emit INSERT events in both cases (a notable example is MySQL).

    • sequenceValue

      long sequenceValue()
      Specifies the numeric value part of the record's source sequence. As long as the source sequence doesn't change, the values will be monotonically increasing and can be used to impose ordering over the stream of records.
      Since:
      Jet 4.3

    • sequenceSource

      long sequenceSource()
      Specifies the source descriptor of the record's sequence. Any changes observed in its value should be interpreted as a reset in the sequence's numeric values. No ordering can be deduced for two records with different sequence sources.
      Since:
      Jet 4.3

    • operation

      @Nonnull Operation operation()
      Returns the type of change this record describes (insert, delete or update). Some special events, like heartbeats, don't have an operation value.
      Returns:
      Operation.UNSPECIFIED if this ChangeRecord doesn't have an operation field, otherwise the appropriate Operation that matches the CDC record's operation field

    • database

      @Nonnull String database()
      Returns the name of the database containing the record's table.
      Returns:
      name of the source database for the current record

    • schema

      @Nonnull String schema() throws UnsupportedOperationException
      Returns the name of the schema containing the record's table. Note: not all databases have the concept of a schema (for example MySQL).
      Returns:
      name of the source schema for the current record
      Throws:
      UnsupportedOperationException

    • table

      @Nonnull String table()
      Returns the name of the table this record is part of.
      Returns:
      name of the source table for the current record

    • key

      @Nullable RecordPart key()
      Returns the key part of the CDC event. It identifies the affected record.

    • value

      @Nonnull RecordPart value()
      Returns the value part of the CDC event. It includes fields like the timestamp, operation, and database record data.

      For sync, insert and update operations the value describes the database record as it looks AFTER the event, so the latest image.

      For delete operations the value describes the database record as it looked BEFORE the event, so the previous image.

    • newValue

      @Nullable RecordPart newValue()
      Returns the new value of the record. For sync, insert and update operations the value describes the database record as it looks AFTER the event, so the latest state, for delete it returns null.
      Since:
      5.2

    • oldValue

      @Nullable RecordPart oldValue()
      Returns the old value of the record. For update and delete operations the value describes the database record as it looks BEFORE the event, for sync and insert it returns null.
      Since:
      5.2

    • toJson

      @Nonnull String toJson()
      Returns the raw JSON string from the CDC event underlying this ChangeRecord. You can use it if higher-level parsing (see other methods) fails for some reason (for example on some untested combination of database connector and version).