| Package | Description |
|---|---|
| com.hazelcast.jet.aggregate |
Contains
AggregateOperation and its several variants, as well
as a builder object for the aggregate operations. |
| com.hazelcast.jet.avro |
Apache Avro file read/write support for Hazelcast Jet.
|
| com.hazelcast.jet.core |
Jet's Core API.
|
| com.hazelcast.jet.core.processor |
Contains static utility classes with factories of Jet processors.
|
| com.hazelcast.jet.core.test |
Utilities for writing tests of Core API Processors.
|
| com.hazelcast.jet.elastic |
Contains sources and sinks for Elasticsearch 7
|
| com.hazelcast.jet.grpc |
Contributes
gRPC service factories that can be to apply transformations to
a pipeline which for each input item calls to a gRPC service. |
| com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
| com.hazelcast.jet.pipeline.test |
This package contains various mock sources to help with pipeline testing
and development.
|
| com.hazelcast.jet.s3 |
AWS S3 read/write support for Hazelcast Jet.
|
| Modifier and Type | Method and Description |
|---|---|
SupplierEx<A> |
AggregateOperation.createFn()
A primitive that returns a new accumulator.
|
| Modifier and Type | Method and Description |
|---|---|
static <T,K,R,A,M extends Map<K,R>> |
AggregateOperations.groupingBy(FunctionEx<? super T,? extends K> keyFn,
SupplierEx<M> createMapFn,
AggregateOperation1<? super T,A,R> downstream)
Returns an
AggregateOperation1 that accumulates the items into a
Map (as obtained from createMapFn) where the key is the
result of applying keyFn and the value is the result of
applying the downstream aggregate operation to the items with that key. |
static <T,C extends Collection<T>> |
AggregateOperations.toCollection(SupplierEx<C> createCollectionFn)
Returns an aggregate operation that accumulates the items into a
Collection. |
static <T,K,U,M extends Map<K,U>> |
AggregateOperations.toMap(FunctionEx<? super T,? extends K> keyFn,
FunctionEx<? super T,? extends U> valueFn,
BinaryOperatorEx<U> mergeFn,
SupplierEx<M> createMapFn)
Returns an aggregate operation that accumulates elements into a
user-supplied
Map instance. |
static <A> AggregateOperationBuilder<A> |
AggregateOperation.withCreate(SupplierEx<A> createFn)
Returns a builder object, initialized with the supplied
create primitive, that can be used to construct the definition of an
aggregate operation in a step-by-step manner. |
| Modifier and Type | Method and Description |
|---|---|
static <R> Sink<R> |
AvroSinks.files(String directoryName,
org.apache.avro.Schema schema,
SupplierEx<org.apache.avro.io.DatumWriter<R>> datumWriterSupplier)
Returns a sink that that writes the items it receives to Apache Avro
files.
|
static <D> AvroSourceBuilder<D> |
AvroSources.filesBuilder(String directory,
SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier)
Returns a builder object that offers a step-by-step fluent API to build
a custom Avro file source for the Pipeline API.
|
static <D,T> ProcessorMetaSupplier |
AvroProcessors.readFilesP(String directory,
String glob,
boolean sharedFileSystem,
SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier,
BiFunctionEx<String,? super D,T> mapOutputFn)
Returns a supplier of processors for
AvroSources.filesBuilder(java.lang.String, java.lang.Class<D>). |
static <D> ProcessorMetaSupplier |
AvroProcessors.writeFilesP(String directoryName,
org.apache.avro.Schema schema,
SupplierEx<org.apache.avro.io.DatumWriter<D>> datumWriterSupplier)
Returns a supplier of processors for
AvroSinks.files(java.lang.String, org.apache.avro.Schema, com.hazelcast.function.SupplierEx<org.apache.avro.io.DatumWriter<R>>). |
| Modifier and Type | Method and Description |
|---|---|
static SupplierEx<WatermarkPolicy> |
WatermarkPolicy.limitingLag(long lag)
Maintains a watermark that lags behind the top observed timestamp by the
given amount.
|
static SupplierEx<WatermarkPolicy> |
WatermarkPolicy.limitingRealTimeLag(long lag)
Maintains a watermark that lags behind the real time by the given
amount.
|
SupplierEx<? extends WatermarkPolicy> |
EventTimePolicy.newWmPolicyFn()
Returns the factory of the watermark policy objects.
|
| Modifier and Type | Method and Description |
|---|---|
static <T> EventTimePolicy<T> |
EventTimePolicy.eventTimePolicy(ToLongFunctionEx<? super T> timestampFn,
ObjLongBiFunction<? super T,?> wrapFn,
SupplierEx<? extends WatermarkPolicy> newWmPolicyFn,
long watermarkThrottlingFrameSize,
long watermarkThrottlingFrameOffset,
long idleTimeoutMillis)
Creates and returns a new event time policy.
|
static <T> EventTimePolicy<T> |
EventTimePolicy.eventTimePolicy(ToLongFunctionEx<? super T> timestampFn,
SupplierEx<? extends WatermarkPolicy> newWmPolicyFn,
long watermarkThrottlingFrameSize,
long watermarkThrottlingFrameOffset,
long idleTimeoutMillis)
Creates and returns a new event time policy.
|
Vertex |
DAG.newUniqueVertex(String namePrefix,
SupplierEx<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this
DAG. |
Vertex |
DAG.newVertex(String name,
SupplierEx<? extends Processor> simpleSupplier)
Creates a vertex from a
Supplier<Processor> and adds it to this DAG. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(int preferredLocalParallelism,
SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorSupplier |
ProcessorSupplier.of(SupplierEx<? extends Processor> processorSupplier)
Returns a
ProcessorSupplier which will delegate to the given
Supplier<Processor> to create all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.preferLocalParallelismOne(SupplierEx<? extends Processor> procSupplier)
Variant of
ProcessorMetaSupplier.preferLocalParallelismOne(ProcessorSupplier) where
the supplied SupplierEx<Processor> will be
wrapped into a ProcessorSupplier. |
| Constructor and Description |
|---|
Vertex(String name,
SupplierEx<? extends Processor> processorSupplier)
Creates a vertex from a
Supplier<Processor>. |
| Modifier and Type | Method and Description |
|---|---|
static <K,A> SupplierEx<Processor> |
Processors.accumulateByFrameP(List<FunctionEx<?,? extends K>> keyFns,
List<ToLongFunctionEx<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A> SupplierEx<Processor> |
Processors.accumulateByKeyP(List<FunctionEx<?,? extends K>> getKeyFns,
AggregateOperation<A,?> aggrOp)
Returns a supplier of processors for the first-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> SupplierEx<Processor> |
Processors.accumulateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the
accumulation step of the provided aggregate operation on all the items
it receives.
|
static <K,A,R,OUT> |
Processors.aggregateByKeyP(List<FunctionEx<?,? extends K>> keyFns,
AggregateOperation<A,R> aggrOp,
BiFunctionEx<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that groups items by key
and performs the provided aggregate operation on each group.
|
static <A,R> SupplierEx<Processor> |
Processors.aggregateP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the provided
aggregate operation on all the items it receives.
|
static <K,A,R,OUT> |
Processors.aggregateToSessionWindowP(long sessionTimeout,
long earlyResultsPeriod,
List<ToLongFunctionEx<?>> timestampFns,
List<FunctionEx<?,? extends K>> keyFns,
AggregateOperation<A,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events into
session windows.
|
static <K,A,R,OUT> |
Processors.aggregateToSlidingWindowP(List<FunctionEx<?,? extends K>> keyFns,
List<ToLongFunctionEx<?>> timestampFns,
TimestampKind timestampKind,
SlidingWindowPolicy winPolicy,
long earlyResultsPeriod,
AggregateOperation<A,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Returns a supplier of processors for a vertex that aggregates events
into a sliding window in a single stage (see the
class Javadoc for an explanation of aggregation stages). |
static <K,A,R,OUT> |
Processors.combineByKeyP(AggregateOperation<A,R> aggrOp,
BiFunctionEx<? super K,? super R,OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage group-and-aggregate setup.
|
static <A,R> SupplierEx<Processor> |
Processors.combineP(AggregateOperation<A,R> aggrOp)
Returns a supplier of processors for a vertex that performs the
combining and finishing steps of the provided aggregate operation.
|
static <K,A,R,OUT> |
Processors.combineToSlidingWindowP(SlidingWindowPolicy winPolicy,
AggregateOperation<A,? extends R> aggrOp,
KeyedWindowResultFunction<? super K,? super R,? extends OUT> mapToOutputFn)
Returns a supplier of processors for the second-stage vertex in a
two-stage sliding window aggregation setup (see the
class Javadoc for an explanation of aggregation stages). |
static <T> SupplierEx<Processor> |
Processors.filterP(PredicateEx<? super T> filterFn)
Returns a supplier of processors for a vertex that emits the same items
it receives, but only those that pass the given predicate.
|
static <T,R> SupplierEx<Processor> |
Processors.flatMapP(FunctionEx<? super T,? extends Traverser<? extends R>> flatMapFn)
Returns a supplier of processors for a vertex that applies the provided
item-to-traverser mapping function to each received item and emits all
the items from the resulting traverser.
|
static <T,K,S,R> SupplierEx<Processor> |
Processors.flatMapStatefulP(long ttl,
FunctionEx<? super T,? extends K> keyFn,
ToLongFunctionEx<? super T> timestampFn,
Supplier<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> statefulFlatMapFn,
TriFunction<? super S,? super K,? super Long,? extends Traverser<R>> onEvictFn)
Returns a supplier of processors for a vertex that performs a stateful
flat-mapping of its input.
|
static <T> SupplierEx<Processor> |
Processors.insertWatermarksP(EventTimePolicy<? super T> eventTimePolicy)
Returns a supplier of processors for a vertex that inserts
watermark items into the stream. |
static <T,R> SupplierEx<Processor> |
Processors.mapP(FunctionEx<? super T,? extends R> mapFn)
Returns a supplier of processors for a vertex which, for each received
item, emits the result of applying the given mapping function to it.
|
static <T,K,S,R> SupplierEx<Processor> |
Processors.mapStatefulP(long ttl,
FunctionEx<? super T,? extends K> keyFn,
ToLongFunctionEx<? super T> timestampFn,
Supplier<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> statefulMapFn,
TriFunction<? super S,? super K,? super Long,? extends R> onEvictFn)
Returns a supplier of processors for a vertex that performs a stateful
mapping of its input.
|
static SupplierEx<Processor> |
Processors.noopP()
Returns a supplier of a processor that swallows all its normal input (if
any), does nothing with it, forwards the watermarks, produces no output
and completes immediately.
|
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(FunctionEx<T,? extends CharSequence> toStringFn,
PredicateEx<T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier),
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(SupplierEx<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(FunctionEx<? super T,? extends CharSequence> toStringFn,
PredicateEx<? super T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier),
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(SupplierEx<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(FunctionEx<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
PredicateEx<? super Map.Entry<K,V>> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier),
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(SupplierEx<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <T> SupplierEx<Processor> |
Processors.sortP(Comparator<T> comparator)
Returns a supplier of processors for a vertex that sorts its input using
a
PriorityQueue and emits it in the complete phase. |
static <W,T> SupplierEx<Processor> |
SinkProcessors.writeBufferedP(FunctionEx<? super Processor.Context,? extends W> createFn,
BiConsumerEx<? super W,? super T> onReceiveFn,
ConsumerEx<? super W> flushFn)
Shortcut for
SinkProcessors.writeBufferedP(FunctionEx,
BiConsumerEx, ConsumerEx, ConsumerEx) with
a no-op destroyFn. |
static <W,T> SupplierEx<Processor> |
SinkProcessors.writeBufferedP(FunctionEx<? super Processor.Context,? extends W> createFn,
BiConsumerEx<? super W,? super T> onReceiveFn,
ConsumerEx<? super W> flushFn,
ConsumerEx<? super W> destroyFn)
Returns a supplier of processors for a vertex that drains all the items
from the inbox to an internal writer object and then does a flush.
|
| Modifier and Type | Method and Description |
|---|---|
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(FunctionEx<T,? extends CharSequence> toStringFn,
PredicateEx<T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekInput(toStringFn, shouldLogFn, metaSupplier),
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekInputP(SupplierEx<Processor> wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(FunctionEx<? super T,? extends CharSequence> toStringFn,
PredicateEx<? super T> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekOutput(toStringFn, shouldLogFn, metaSupplier),
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekOutputP(SupplierEx<Processor> wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(FunctionEx<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
PredicateEx<? super Map.Entry<K,V>> shouldLogFn,
SupplierEx<Processor> wrapped)
Same as
peekSnapshot(toStringFn, shouldLogFn, metaSupplier),
but accepts a SupplierEx of processors instead of a
meta-supplier. |
static SupplierEx<Processor> |
DiagnosticProcessors.peekSnapshotP(SupplierEx<Processor> wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <T> ProcessorMetaSupplier |
SourceProcessors.readJdbcP(SupplierEx<? extends Connection> newConnectionFn,
ToResultSetFunction resultSetFn,
FunctionEx<? super ResultSet,? extends T> mapOutputFn)
Returns a supplier of processors for
Sources.jdbc(
SupplierEx, ToResultSetFunction, FunctionEx). |
static <T> ProcessorMetaSupplier |
SourceProcessors.streamJmsQueueP(SupplierEx<? extends javax.jms.Connection> newConnectionFn,
FunctionEx<? super javax.jms.Session,? extends javax.jms.MessageConsumer> consumerFn,
FunctionEx<? super javax.jms.Message,?> messageIdFn,
FunctionEx<? super javax.jms.Message,? extends T> projectionFn,
EventTimePolicy<? super T> eventTimePolicy,
ProcessingGuarantee maxGuarantee)
Returns a supplier of processors for
Sources.jmsQueueBuilder(com.hazelcast.function.SupplierEx<? extends javax.jms.ConnectionFactory>). |
static <T> ProcessorMetaSupplier |
SourceProcessors.streamJmsTopicP(SupplierEx<? extends javax.jms.Connection> newConnectionFn,
FunctionEx<? super javax.jms.Session,? extends javax.jms.MessageConsumer> consumerFn,
boolean isSharedConsumer,
FunctionEx<? super javax.jms.Message,?> messageIdFn,
FunctionEx<? super javax.jms.Message,? extends T> projectionFn,
EventTimePolicy<? super T> eventTimePolicy,
ProcessingGuarantee maxGuarantee)
Returns a supplier of processors for
Sources.jmsTopicBuilder(com.hazelcast.function.SupplierEx<? extends javax.jms.ConnectionFactory>). |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJdbcP(String updateQuery,
SupplierEx<? extends CommonDataSource> dataSourceSupplier,
BiConsumerEx<? super PreparedStatement,? super T> bindFn,
boolean exactlyOnce,
int batchLimit)
Returns a supplier of processors for
Sinks.jdbcBuilder(). |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJmsQueueP(String queueName,
boolean exactlyOnce,
SupplierEx<? extends javax.jms.Connection> newConnectionFn,
BiFunctionEx<? super javax.jms.Session,? super T,? extends javax.jms.Message> messageFn)
Returns a supplier of processors for
Sinks.jmsQueueBuilder(com.hazelcast.function.SupplierEx<javax.jms.ConnectionFactory>). |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJmsTopicP(String topicName,
boolean exactlyOnce,
SupplierEx<? extends javax.jms.Connection> newConnectionFn,
BiFunctionEx<? super javax.jms.Session,? super T,? extends javax.jms.Message> messageFn)
Returns a supplier of processors for
Sinks.jmsTopicBuilder(com.hazelcast.function.SupplierEx<javax.jms.ConnectionFactory>). |
| Modifier and Type | Method and Description |
|---|---|
static TestSupport |
TestSupport.verifyProcessor(SupplierEx<Processor> supplier) |
| Modifier and Type | Method and Description |
|---|---|
ElasticSinkBuilder<T> |
ElasticSinkBuilder.bulkRequestFn(SupplierEx<org.elasticsearch.action.bulk.BulkRequest> bulkRequestFn)
Set the supplier function for BulkRequest, defaults to new
BulkRequest() |
ElasticSourceBuilder<T> |
ElasticSourceBuilder.clientFn(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn)
Set the client supplier function
|
ElasticSinkBuilder<T> |
ElasticSinkBuilder.clientFn(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn)
Set the client supplier function
|
static BatchSource<String> |
ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn)
Creates a source which queries Elasticsearch using client obtained from
RestClientBuilder supplier function. |
static <T> BatchSource<T> |
ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn,
FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn)
Creates a source which queries Elasticsearch using client obtained from
RestClientBuilder supplier function. |
static <T> Sink<T> |
ElasticSinks.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn,
FunctionEx<? super T,? extends org.elasticsearch.action.DocWriteRequest<?>> mapToRequestFn)
Creates an Elasticsearch sink, uses a client obtained from
clientFn and maps items using given mapToRequestFn
|
static <T> BatchSource<T> |
ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn,
SupplierEx<org.elasticsearch.action.search.SearchRequest> searchRequestFn,
FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn)
Creates a source which queries Elasticsearch using client obtained from
RestHighLevelClient supplier. |
static <T> BatchSource<T> |
ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn,
SupplierEx<org.elasticsearch.action.search.SearchRequest> searchRequestFn,
FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn)
Creates a source which queries Elasticsearch using client obtained from
RestHighLevelClient supplier. |
ElasticSourceBuilder<T> |
ElasticSourceBuilder.searchRequestFn(SupplierEx<org.elasticsearch.action.search.SearchRequest> searchRequestFn)
Set the search request supplier function
|
| Modifier and Type | Method and Description |
|---|---|
static <T,R> ServiceFactory<?,? extends GrpcService<T,R>> |
GrpcServices.bidirectionalStreamingService(SupplierEx<? extends io.grpc.ManagedChannelBuilder<?>> channelFn,
FunctionEx<? super io.grpc.ManagedChannel,? extends FunctionEx<io.grpc.stub.StreamObserver<R>,io.grpc.stub.StreamObserver<T>>> callStubFn)
Creates a
ServiceFactory that calls out to a
bidrectional streaming gRPC service. |
static <T,R> ServiceFactory<?,? extends GrpcService<T,R>> |
GrpcServices.unaryService(SupplierEx<? extends io.grpc.ManagedChannelBuilder<?>> channelFn,
FunctionEx<? super io.grpc.ManagedChannel,? extends BiConsumerEx<T,io.grpc.stub.StreamObserver<R>>> callStubFn)
Creates a
ServiceFactory that calls out to a
unary gRPC service. |
| Modifier and Type | Method and Description |
|---|---|
default <R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
<R> GeneralStage<R> |
GeneralStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processors. |
default <R> BatchStage<R> |
BatchStageWithKey.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
default <R> StreamStage<R> |
StreamStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
<R> GeneralStage<R> |
GeneralStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processors. |
default <R> BatchStage<R> |
BatchStage.customTransform(String stageName,
SupplierEx<Processor> procSupplier) |
JdbcSinkBuilder<T> |
JdbcSinkBuilder.dataSourceSupplier(SupplierEx<? extends CommonDataSource> dataSourceSupplier)
Sets the supplier of
DataSource or XADataSource. |
<S> StreamStage<T> |
StreamStageWithKey.filterStateful(long ttl,
SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn)
Attaches a stage that performs a stateful filtering operation.
|
default <S> StreamStage<T> |
StreamStageWithKey.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<S> GeneralStage<T> |
GeneralStageWithKey.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn)
Attaches a stage that performs a stateful filtering operation.
|
<S> BatchStage<T> |
BatchStageWithKey.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<S> StreamStage<T> |
StreamStage.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<S> GeneralStage<T> |
GeneralStage.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn)
Attaches a stage that performs a stateful filtering operation.
|
<S> BatchStage<T> |
BatchStage.filterStateful(SupplierEx<? extends S> createFn,
BiPredicateEx<? super S,? super T> filterFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.flatMapStateful(long ttl,
SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn,
TriFunction<? super S,? super K,? super Long,? extends Traverser<R>> onEvictFn)
Attaches a stage that performs a stateful flat-mapping operation.
|
<S,R> StreamStage<R> |
StreamStage.flatMapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends Traverser<R>> flatMapFn) |
<S,R> GeneralStage<R> |
GeneralStage.flatMapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends Traverser<R>> flatMapFn)
Attaches a stage that performs a stateful flat-mapping operation.
|
<S,R> BatchStage<R> |
BatchStage.flatMapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends Traverser<R>> flatMapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.flatMapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn) |
<S,R> GeneralStage<R> |
GeneralStageWithKey.flatMapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn)
Attaches a stage that performs a stateful flat-mapping operation.
|
<S,R> BatchStage<R> |
BatchStageWithKey.flatMapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends Traverser<R>> flatMapFn) |
static <T> Sink<T> |
Sinks.jdbc(String updateQuery,
SupplierEx<? extends CommonDataSource> dataSourceSupplier,
BiConsumerEx<PreparedStatement,T> bindFn)
A shortcut for:
|
static <T> BatchSource<T> |
Sources.jdbc(SupplierEx<? extends Connection> newConnectionFn,
ToResultSetFunction resultSetFn,
FunctionEx<? super ResultSet,? extends T> createOutputFn)
Returns a source which connects to the specified database using the given
newConnectionFn, queries the database and creates a result set
using the the given resultSetFn. |
static StreamSource<javax.jms.Message> |
Sources.jmsQueue(String name,
SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier)
Shortcut equivalent to:
|
static <T> Sink<T> |
Sinks.jmsQueue(String queueName,
SupplierEx<javax.jms.ConnectionFactory> factorySupplier)
Convenience for
Sinks.jmsQueueBuilder(SupplierEx). |
static StreamSource<javax.jms.Message> |
Sources.jmsQueue(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier,
String name)
Deprecated.
|
static JmsSourceBuilder |
Sources.jmsQueueBuilder(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier)
Returns a builder object that offers a step-by-step fluent API to build
a custom JMS
StreamSource for the Pipeline API. |
static <T> JmsSinkBuilder<T> |
Sinks.jmsQueueBuilder(SupplierEx<javax.jms.ConnectionFactory> factorySupplier)
Returns a builder object that offers a step-by-step fluent API to build
a custom JMS queue sink for the Pipeline API.
|
static StreamSource<javax.jms.Message> |
Sources.jmsTopic(String name,
SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier)
Shortcut equivalent to:
|
static <T> Sink<T> |
Sinks.jmsTopic(String topicName,
SupplierEx<javax.jms.ConnectionFactory> factorySupplier)
Shortcut for:
|
static StreamSource<javax.jms.Message> |
Sources.jmsTopic(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier,
String name)
Deprecated.
|
static JmsSourceBuilder |
Sources.jmsTopicBuilder(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier)
Returns a builder object that offers a step-by-step fluent API to build
a custom JMS
StreamSource for the Pipeline API. |
static <T> JmsSinkBuilder<T> |
Sinks.jmsTopicBuilder(SupplierEx<javax.jms.ConnectionFactory> factorySupplier)
Returns a builder object that offers a step-by-step fluent API to build
a custom JMS topic sink for the Pipeline API.
|
<S,R> StreamStage<R> |
StreamStageWithKey.mapStateful(long ttl,
SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn,
TriFunction<? super S,? super K,? super Long,? extends R> onEvictFn)
Attaches a stage that performs a stateful mapping operation.
|
<S,R> StreamStage<R> |
StreamStage.mapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends R> mapFn) |
<S,R> GeneralStage<R> |
GeneralStage.mapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends R> mapFn)
Attaches a stage that performs a stateful mapping operation.
|
<S,R> BatchStage<R> |
BatchStage.mapStateful(SupplierEx<? extends S> createFn,
BiFunctionEx<? super S,? super T,? extends R> mapFn) |
<S,R> StreamStage<R> |
StreamStageWithKey.mapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn) |
<S,R> GeneralStage<R> |
GeneralStageWithKey.mapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn)
Attaches a stage that performs a stateful mapping operation.
|
<S,R> BatchStage<R> |
BatchStageWithKey.mapStateful(SupplierEx<? extends S> createFn,
TriFunction<? super S,? super K,? super T,? extends R> mapFn) |
| Modifier and Type | Method and Description |
|---|---|
static <S> AssertionSinkBuilder<S,Void> |
AssertionSinkBuilder.assertionSink(String name,
SupplierEx<? extends S> createFn)
Returns a builder object that offers a step-by-step fluent API to build
an assertion
Sink for the Pipeline API. |
| Modifier and Type | Method and Description |
|---|---|
static <T> BatchSource<T> |
S3Sources.s3(List<String> bucketNames,
String prefix,
Charset charset,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier,
BiFunctionEx<String,String,? extends T> mapFn)
Creates an AWS S3
BatchSource which lists all the objects in the
bucket-list using given prefix, reads them line by line,
transforms each line to the desired output object using given mapFn and emits them to downstream. |
static BatchSource<String> |
S3Sources.s3(List<String> bucketNames,
String prefix,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier)
Convenience for
S3Sources.s3(List, String, Charset, SupplierEx, BiFunctionEx). |
static <I,T> BatchSource<T> |
S3Sources.s3(List<String> bucketNames,
String prefix,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier,
FunctionEx<? super InputStream,? extends Stream<I>> readFileFn,
BiFunctionEx<String,? super I,? extends T> mapFn)
Creates an AWS S3
BatchSource which lists all the objects in the
bucket-list using given prefix, reads them using provided readFileFn, transforms each read item to the desired output object
using given mapFn and emits them to downstream. |
static <I,T> BatchSource<T> |
S3Sources.s3(List<String> bucketNames,
String prefix,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier,
TriFunction<? super InputStream,String,String,? extends Stream<I>> readFileFn,
BiFunctionEx<String,? super I,? extends T> mapFn)
Creates an AWS S3
BatchSource which lists all the objects in the
bucket-list using given prefix, reads them using provided readFileFn, transforms each read item to the desired output object
using given mapFn and emits them to downstream. |
static <T> Sink<? super T> |
S3Sinks.s3(String bucketName,
String prefix,
Charset charset,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier,
FunctionEx<? super T,String> toStringFn)
Creates an AWS S3
Sink which writes items to files into the
given bucket. |
static <T> Sink<? super T> |
S3Sinks.s3(String bucketName,
SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier)
Convenience for
S3Sinks.s3(String, String, Charset, SupplierEx, FunctionEx)
Uses Object.toString() to convert the items to lines. |
Copyright © 2021 Hazelcast, Inc.. All rights reserved.