Package com.hazelcast.jet.core

Interface WatermarkPolicy

public interface WatermarkPolicy
This object tracks and determines the current Watermark given the event timestamps as they occur for a single input stream. Typically, the watermark will be advanced with a fixed lag behind the top observed timestamp so far.

This object is used by source processors to determine the current watermark. The processor may choose to create several of these objects to track each source partition separately and each processor will also have their own instance. The implementation does not need to be thread-safe.

Since:
Jet 3.0
See Also:

  • Method Summary

    Modifier and Type
    Method
    Description
    long
    getCurrentWatermark()
    Called to get the current watermark.
    static SupplierEx<WatermarkPolicy>
    limitingLag(long lag)
    Maintains a watermark that lags behind the top observed timestamp by the given amount.
    static SupplierEx<WatermarkPolicy>
    limitingRealTimeLag(long lag)
    Maintains a watermark that lags behind the real time by the given amount.
    void
    reportEvent(long timestamp)
    Called to report the observation of an event with the given timestamp.

  • Method Details

    • reportEvent

      void reportEvent(long timestamp)
      Called to report the observation of an event with the given timestamp. The next call to getCurrentWatermark() should reflect this.
      Parameters:
      timestamp - event's timestamp

    • getCurrentWatermark

      long getCurrentWatermark()
      Called to get the current watermark. The watermark may advance based just on the passage of time.

    • limitingLag

      @Nonnull static SupplierEx<WatermarkPolicy> limitingLag(long lag)
      Maintains a watermark that lags behind the top observed timestamp by the given amount.

      Note: if Jet stops receiving events at some point (e.g., at the end of a business day), the watermark will stop advancing and stay behind the most recent events. Jet will not output the results of aggregating these events until it starts receiving events again (e.g., at the start of the next business day).

      Parameters:
      lag - the desired difference between the top observed timestamp and the watermark

    • limitingRealTimeLag

      @Nonnull static SupplierEx<WatermarkPolicy> limitingRealTimeLag(long lag)
      Maintains a watermark that lags behind the real time by the given amount. Doesn't consider the event timestamp at all.
      Parameters:
      lag - the desired difference between the top observed timestamp and the watermark
      Since:
      Jet 4.3