T
- event typepublic final class WatermarkGenerationParams<T> extends Object implements Serializable
timestampFn
: extracts the timestamp from an event in the stream
newWmPolicyFn
: a factory of watermark policy objects.
Refer to its documentation
for explanation.
wmEmitPolicy
: a policy object
that
allows the processor to filter out redundant watermark items before
emitting them. For example, a sliding/tumbling window processor doesn't
need to observe more than one watermark item per frame.
idleTimeoutMillis
: a measure to mitigate the issue with temporary
lulls in a distributed event stream. It pertains to each partition
of a data source independently. If Jet doesn't receive any events
from a given partition for this long, it will mark it as "idle" and let
the watermark in downstream vertices advance as if the partition didn't
exist.
wrapFn
: a function that transforms a given event and its
timestamp into the item to emit from the processor. For example, the
Pipeline API uses this to wrap items into JetEvent
s as a way
to propagate the event timestamps through the pipeline regardless of
the transformation the user does on the event objects themselves.
Modifier and Type | Field and Description |
---|---|
static long |
DEFAULT_IDLE_TIMEOUT
The default idle timeout in milliseconds.
|
Modifier and Type | Method and Description |
---|---|
long |
idleTimeoutMillis()
Returns the amount of time allowed to pass without receiving any events
from a partition before marking it as "idle".
|
DistributedSupplier<? extends WatermarkPolicy> |
newWmPolicyFn()
Returns the factory of the watermark policy objects.
|
static <T> WatermarkGenerationParams<T> |
noWatermarks()
Returns watermark generation parameters that result in no watermarks
being emitted.
|
DistributedToLongFunction<? super T> |
timestampFn()
Returns the function that extracts the timestamp from the event.
|
WatermarkGenerationParams<T> |
withEmitPolicy(WatermarkEmissionPolicy emitPolicy)
Returns new instance with emit policy replaced with the given argument.
|
WatermarkEmissionPolicy |
wmEmitPolicy()
Returns the watermark emission policy, which decides how to suppress
redundant watermarks.
|
static <T> WatermarkGenerationParams<T> |
wmGenParams(DistributedToLongFunction<? super T> timestampFn,
DistributedObjLongBiFunction<? super T,?> wrapFn,
DistributedSupplier<? extends WatermarkPolicy> newWmPolicyFn,
WatermarkEmissionPolicy wmEmitPolicy,
long idleTimeoutMillis)
Creates and returns new watermark generation parameters.
|
static <T> WatermarkGenerationParams<T> |
wmGenParams(DistributedToLongFunction<? super T> timestampFn,
DistributedSupplier<? extends WatermarkPolicy> wmPolicy,
WatermarkEmissionPolicy wmEmitPolicy,
long idleTimeoutMillis)
Creates and returns a watermark generation parameters object.
|
DistributedObjLongBiFunction<? super T,?> |
wrapFn()
Returns the function that transforms the received item and its timestamp
into the emitted item.
|
public static final long DEFAULT_IDLE_TIMEOUT
public static <T> WatermarkGenerationParams<T> wmGenParams(@Nonnull DistributedToLongFunction<? super T> timestampFn, @Nonnull DistributedObjLongBiFunction<? super T,?> wrapFn, @Nonnull DistributedSupplier<? extends WatermarkPolicy> newWmPolicyFn, @Nonnull WatermarkEmissionPolicy wmEmitPolicy, long idleTimeoutMillis)
noWatermarks()
.timestampFn
- function that extracts the timestamp from the eventwrapFn
- function that transforms the received item and its timestamp into the
emitted itemnewWmPolicyFn
- factory of the watermark policy objectswmEmitPolicy
- watermark emission policy (decides how to suppress redundant watermarks)idleTimeoutMillis
- the timeout after which a partition will be marked as idle.
If <= 0, partitions will never be marked as idle.public static <T> WatermarkGenerationParams<T> wmGenParams(@Nonnull DistributedToLongFunction<? super T> timestampFn, @Nonnull DistributedSupplier<? extends WatermarkPolicy> wmPolicy, @Nonnull WatermarkEmissionPolicy wmEmitPolicy, long idleTimeoutMillis)
noWatermarks()
.timestampFn
- function that extracts the timestamp from the eventwmPolicy
- factory of the watermark policy objectswmEmitPolicy
- watermark emission policy (decides how to suppress redundant watermarks)idleTimeoutMillis
- the timeout after which a partition will be marked as idle.
If <= 0, partitions will never be marked as idle.public static <T> WatermarkGenerationParams<T> noWatermarks()
@Nonnull public DistributedToLongFunction<? super T> timestampFn()
@Nonnull public DistributedObjLongBiFunction<? super T,?> wrapFn()
@Nonnull public DistributedSupplier<? extends WatermarkPolicy> newWmPolicyFn()
@Nonnull public WatermarkEmissionPolicy wmEmitPolicy()
public long idleTimeoutMillis()
WatermarkCoalescer.IDLE_MESSAGE
to its
output edges. This signals Jet that the watermark can advance as
if the partition didn't exist.
If supply a zero or negative value, partitions will never be marked as idle.
@Nonnull public WatermarkGenerationParams<T> withEmitPolicy(WatermarkEmissionPolicy emitPolicy)
Copyright © 2018 Hazelcast, Inc.. All rights reserved.