Package | Description |
---|---|
com.hazelcast.jet |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.aggregate |
Contains
AggregateOperation and several its variants, as well
as a builder object for the aggregate operations. |
com.hazelcast.jet.core |
Jet's Core API.
|
com.hazelcast.jet.core.processor |
Apache Kafka reader/writer support for Hazelcast Jet.
|
com.hazelcast.jet.function |
Serializable variants of functional interfaces from
java.util.function . |
com.hazelcast.jet.stream |
java.util.stream implementation using Hazelcast Jet
|
Modifier and Type | Method and Description |
---|---|
<R> ComputeStage<R> |
ComputeStage.customTransform(String stageName,
DistributedSupplier<Processor> procSupplier)
Attaches to this stage a stage with a custom transform based on the
provided supplier of Core API
Processor s. |
Modifier and Type | Method and Description |
---|---|
DistributedSupplier<A> |
AggregateOperation.createFn()
A primitive that returns a new accumulator.
|
Modifier and Type | Method and Description |
---|---|
static <T,C extends Collection<T>> |
AggregateOperations.toCollection(DistributedSupplier<C> createCollectionFn)
Returns an
AggregateOperation1 that accumulates the input
elements into a new Collection . |
static <T,K,U,M extends Map<K,U>> |
AggregateOperations.toMap(DistributedFunction<? super T,? extends K> getKeyFn,
DistributedFunction<? super T,? extends U> getValueFn,
DistributedBinaryOperator<U> mergeFn,
DistributedSupplier<M> createMapFn)
Returns an
AggregateOperation1 that accumulates elements
into a Map whose keys and values are the result of applying the
provided mapping functions to the input elements. |
static <A> AggregateOperationBuilder<A> |
AggregateOperation.withCreate(DistributedSupplier<A> createFn)
Returns a builder object, initialized with the supplied
create
primitive, that can be used to construct the definition of an aggregate
operation in a step-by-step manner. |
Modifier and Type | Method and Description |
---|---|
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.limitingLagAndDelay(long lag,
long maxDelayMs)
Maintains watermark that lags behind the top observed timestamp by at
most the given amount and is additionally guaranteed to reach the
timestamp of any given event within
maxDelayMs after observing
it. |
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.limitingLagAndLull(long lag,
long maxLullMs)
Maintains watermark that lags behind the top timestamp by the amount
specified with
lag . |
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.limitingTimestampAndWallClockLag(long timestampLag,
long wallClockLag)
Maintains watermark that lags behind the top timestamp by at most
timestampLag and behind wall-clock time by at most wallClockLag . |
static DistributedSupplier<WatermarkPolicy> |
WatermarkPolicies.withFixedLag(long lag)
Maintains watermark that lags behind the top observed timestamp by the
given amount.
|
Modifier and Type | Method and Description |
---|---|
static ProcessorMetaSupplier |
ProcessorMetaSupplier.dontParallelize(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
Vertex |
DAG.newVertex(String name,
DistributedSupplier<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this DAG. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorSupplier |
ProcessorSupplier.of(DistributedSupplier<? extends Processor> processorSupplier)
Returns a
ProcessorSupplier which will delegate to the given
Supplier<Processor> to create all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier,
int preferredLocalParallelism)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
Constructor and Description |
---|
CloseableProcessorSupplier(DistributedSupplier<E> simpleSupplier) |
Vertex(String name,
DistributedSupplier<? extends Processor> processorSupplier)
Creates a vertex from a
Supplier<Processor> . |
Modifier and Type | Method and Description |
---|---|
static <T,K,A> DistributedSupplier<Processor> |
Processors.accumulateByFrameP(DistributedFunction<? super T,K> getKeyFn,
DistributedToLongFunction<? super T> getTimestampFn,
TimestampKind timestampKind,
WindowDefinition windowDef,
AggregateOperation1<? super T,A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T,K,A> DistributedSupplier<Processor> |
Processors.accumulateByKeyP(DistributedFunction<? super T,K> getKeyFn,
AggregateOperation1<? super T,A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <T,A,R> DistributedSupplier<Processor> |
Processors.accumulateP(AggregateOperation1<T,A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <T,K,A,R> DistributedSupplier<Processor> |
Processors.aggregateByKeyP(DistributedFunction<? super T,K> getKeyFn,
AggregateOperation1<? super T,A,R> aggrOp)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <T,A,R> DistributedSupplier<Processor> |
Processors.aggregateP(AggregateOperation1<T,A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <T,K,A,R> DistributedSupplier<Processor> |
Processors.aggregateToSessionWindowP(long sessionTimeout,
DistributedToLongFunction<? super T> getTimestampFn,
DistributedFunction<? super T,K> getKeyFn,
AggregateOperation1<? super T,A,R> aggrOp)
Returns a supplier of processors for a vertex that aggregates events into
session windows.
|
static <T,K,A,R> DistributedSupplier<Processor> |
Processors.aggregateToSlidingWindowP(DistributedFunction<? super T,K> getKeyFn,
DistributedToLongFunction<? super T> getTimestampFn,
TimestampKind timestampKind,
WindowDefinition windowDef,
AggregateOperation1<? super T,A,R> aggrOp)
Returns a supplier of processors for a vertex that aggregates events
into a sliding window in a single stage (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A> DistributedSupplier<Processor> |
Processors.coAccumulateByKeyP(List<DistributedFunction<?,? extends K>> getKeyFs,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <K,A,R> DistributedSupplier<Processor> |
Processors.coAggregateByKeyP(List<DistributedFunction<?,? extends K>> getKeyFs,
AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <A,R> DistributedSupplier<Processor> |
Processors.combineByKeyP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for the second-stage vertex in a
two-stage group-and-aggregate setup.
|
static <T,A,R> DistributedSupplier<Processor> |
Processors.combineP(AggregateOperation1<T,A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R> DistributedSupplier<Processor> |
Processors.combineToSlidingWindowP(WindowDefinition windowDef,
AggregateOperation1<?,A,R> aggrOp)
Returns a supplier of processors for the second-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T> DistributedSupplier<Processor> |
Processors.filterP(DistributedPredicate<T> predicate)
Returns a supplier of processors for a vertex that emits the same items
it receives, but only those that pass the given predicate.
|
static <T,R> DistributedSupplier<Processor> |
Processors.flatMapP(DistributedFunction<T,? extends Traverser<? extends R>> mapper)
Returns a supplier of processors for a vertex that applies the provided
item-to-traverser mapping function to each received item and emits all
the items from the resulting traverser.
|
static <T> DistributedSupplier<Processor> |
Processors.insertWatermarksP(DistributedToLongFunction<T> getTimestampF,
DistributedSupplier<WatermarkPolicy> newWmPolicyF,
WatermarkEmissionPolicy wmEmitPolicy)
Returns a supplier of processors for a vertex that inserts
watermark items into the stream. |
static <T,R> DistributedSupplier<Processor> |
Processors.mapP(DistributedFunction<T,R> mapper)
Returns a supplier of processors for a vertex which, for each received
item, emits the result of applying the given mapping function to it.
|
static DistributedSupplier<Processor> |
Processors.nonCooperativeP(DistributedSupplier<Processor> wrapped)
Decorates a
Supplier<Processor> into one that will declare
its processors non-cooperative. |
static DistributedSupplier<Processor> |
Processors.noopP()
Returns a supplier of processor that consumes all its input (if any) and
does nothing with it.
|
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<Map.Entry<K,V>,String> toStringFn,
DistributedPredicate<Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static <T> DistributedSupplier<Processor> |
Processors.insertWatermarksP(DistributedToLongFunction<T> getTimestampF,
DistributedSupplier<WatermarkPolicy> newWmPolicyF,
WatermarkEmissionPolicy wmEmitPolicy)
Returns a supplier of processors for a vertex that inserts
watermark items into the stream. |
static DistributedSupplier<Processor> |
Processors.nonCooperativeP(DistributedSupplier<Processor> wrapped)
Decorates a
Supplier<Processor> into one that will declare
its processors non-cooperative. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekInputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekOutputP(DistributedSupplier<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<Map.Entry<K,V>,String> toStringFn,
DistributedPredicate<Map.Entry<K,V>> shouldLogFn,
DistributedSupplier<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier) ,
but accepts a DistributedSupplier of processors instead of a
meta-supplier. |
static DistributedSupplier<Processor> |
DiagnosticProcessors.peekSnapshotP(DistributedSupplier<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
T |
DistributedOptional.orElseGet(DistributedSupplier<? extends T> other)
Return the value if present, otherwise invoke
other and return
the result of that invocation. |
<X extends Throwable> |
DistributedOptional.orElseThrow(DistributedSupplier<? extends X> exceptionSupplier)
Return the contained value, if present, otherwise throw an exception
to be created by the provided supplier.
|
Modifier and Type | Method and Description |
---|---|
DistributedSupplier<A> |
DistributedCollector.supplier()
A function that creates and returns a new mutable result container.
|
Modifier and Type | Method and Description |
---|---|
default <R> R |
DistributedStream.collect(DistributedSupplier<R> supplier,
DistributedBiConsumer<R,? super T> accumulator,
DistributedBiConsumer<R,R> combiner)
Performs a mutable
reduction operation on the elements of this stream.
|
default <R> R |
DistributedDoubleStream.collect(DistributedSupplier<R> supplier,
DistributedObjDoubleConsumer<R> accumulator,
DistributedBiConsumer<R,R> combiner)
Performs a mutable
reduction operation on the elements of this stream.
|
default <R> R |
DistributedIntStream.collect(DistributedSupplier<R> supplier,
DistributedObjIntConsumer<R> accumulator,
DistributedBiConsumer<R,R> combiner)
Performs a mutable
reduction operation on the elements of this stream.
|
default <R> R |
DistributedLongStream.collect(DistributedSupplier<R> supplier,
DistributedObjLongConsumer<R> accumulator,
DistributedBiConsumer<R,R> combiner)
Performs a mutable
reduction operation on the elements of this stream.
|
static <T,K,D,A,M extends Map<K,D>> |
DistributedCollectors.groupingBy(DistributedFunction<? super T,? extends K> classifier,
DistributedSupplier<M> mapFactory,
DistributedCollector<? super T,A,D> downstream)
Returns a
DistributedCollector implementing a cascaded "group by" operation
on input elements of type T , grouping elements according to a
classification function, and then performing a reduction operation on
the values associated with a given key using the specified downstream
DistributedCollector . |
static <T,A,R> DistributedCollector<T,A,R> |
DistributedCollector.of(DistributedSupplier<A> supplier,
DistributedBiConsumer<A,T> accumulator,
DistributedBinaryOperator<A> combiner,
DistributedFunction<A,R> finisher,
java.util.stream.Collector.Characteristics... characteristics)
Returns a new
DistributedCollector described by the given supplier ,
accumulator , combiner , and finisher functions. |
static <T,R> DistributedCollector<T,R,R> |
DistributedCollector.of(DistributedSupplier<R> supplier,
DistributedBiConsumer<R,T> accumulator,
DistributedBinaryOperator<R> combiner,
java.util.stream.Collector.Characteristics... characteristics)
Returns a new
DistributedCollector described by the given supplier ,
accumulator , and combiner functions. |
static <T,C extends Collection<T>> |
DistributedCollectors.toCollection(DistributedSupplier<C> collectionFactory)
Returns a
DistributedCollector that accumulates the input
elements into a new Collection , in encounter order. |
static <T,K,U,M extends Map<K,U>> |
DistributedCollectors.toMap(DistributedFunction<? super T,? extends K> keyMapper,
DistributedFunction<? super T,? extends U> valueMapper,
DistributedBinaryOperator<U> mergeFunction,
DistributedSupplier<M> mapSupplier)
Returns a
DistributedCollector that accumulates elements into a
Map whose keys and values are the result of applying the provided
mapping functions to the input elements. |
Copyright © 2017 Hazelcast, Inc.. All Rights Reserved.