Package | Description |
---|---|
com.hazelcast.jet.avro |
Apache Avro file read/write support for Hazelcast Jet.
|
com.hazelcast.jet.core |
Jet's Core API.
|
com.hazelcast.jet.core.processor |
Contains static utility classes with factories of Jet processors.
|
com.hazelcast.jet.core.test |
Utilities for writing tests of Core API Processors.
|
com.hazelcast.jet.hadoop |
Apache Hadoop read/write support for Hazelcast Jet.
|
com.hazelcast.jet.kafka |
Apache Kafka reader/writer support for Hazelcast Jet.
|
com.hazelcast.jet.pipeline |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.pipeline.file |
This package offers the
FileSourceBuilder
which allows you to construct various kinds of Pipeline sources that read from local or distributed files. |
Modifier and Type | Method and Description |
---|---|
static <D,T> ProcessorMetaSupplier |
AvroProcessors.readFilesP(String directory,
String glob,
boolean sharedFileSystem,
SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier,
BiFunctionEx<String,? super D,T> mapOutputFn)
Returns a supplier of processors for
AvroSources.filesBuilder(java.lang.String, java.lang.Class<D>) . |
static <D> ProcessorMetaSupplier |
AvroProcessors.writeFilesP(String directoryName,
org.apache.avro.Schema schema,
SupplierEx<org.apache.avro.io.DatumWriter<D>> datumWriterSupplier)
Returns a supplier of processors for
AvroSinks.files(java.lang.String, org.apache.avro.Schema, com.hazelcast.function.SupplierEx<org.apache.avro.io.DatumWriter<R>>) . |
Modifier and Type | Method and Description |
---|---|
static ProcessorMetaSupplier |
ProcessorMetaSupplier.forceTotalParallelismOne(ProcessorSupplier supplier)
Variant of
forceTotalParallelismOne(ProcessorSupplier, String) where the node
for the supplier will be chosen randomly. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.forceTotalParallelismOne(ProcessorSupplier supplier,
Address memberAddress)
Wraps the provided
ProcessorSupplier into a meta-supplier that
will only use the given ProcessorSupplier on a node with the
given Address . |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.forceTotalParallelismOne(ProcessorSupplier supplier,
String partitionKey)
Wraps the provided
ProcessorSupplier into a meta-supplier that
will only use the given ProcessorSupplier on a single node. |
ProcessorMetaSupplier |
Vertex.getMetaSupplier()
Returns this vertex's meta-supplier of processors.
|
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)
Factory method that creates a
ProcessorMetaSupplier from the
supplied function that maps a cluster member address to a ProcessorSupplier . |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(int preferredLocalParallelism,
FunctionEx<? super Address,? extends ProcessorSupplier> addressToSupplier)
Factory method that creates a
ProcessorMetaSupplier from the
supplied function that maps a cluster member address to a ProcessorSupplier . |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(int preferredLocalParallelism,
ProcessorSupplier procSupplier)
Factory method that wraps the given
ProcessorSupplier and
returns the same instance for each given Address . |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(int preferredLocalParallelism,
SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(ProcessorSupplier procSupplier)
Wraps the provided
ProcessorSupplier into a meta-supplier that
will always return it. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(SupplierEx<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.preferLocalParallelismOne(ProcessorSupplier supplier)
Wraps the provided
ProcessorSupplier into a meta-supplier that
will always return it. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.preferLocalParallelismOne(SupplierEx<? extends Processor> procSupplier)
Variant of
preferLocalParallelismOne(ProcessorSupplier) where
the supplied SupplierEx<Processor> will be
wrapped into a ProcessorSupplier . |
Modifier and Type | Method and Description |
---|---|
Vertex |
DAG.newUniqueVertex(String namePrefix,
ProcessorMetaSupplier metaSupplier)
Creates a vertex from a
ProcessorMetaSupplier and adds it to
this DAG. |
Vertex |
DAG.newVertex(String name,
ProcessorMetaSupplier metaSupplier)
Creates a vertex from a
ProcessorMetaSupplier and adds it to this DAG. |
Modifier and Type | Method and Description |
---|---|
void |
Vertex.updateMetaSupplier(UnaryOperator<ProcessorMetaSupplier> updateFn)
Applies the provided operator function to the current processor
meta-supplier and replaces it with the one it returns.
|
Constructor and Description |
---|
Vertex(String name,
ProcessorMetaSupplier metaSupplier)
Creates a vertex from a
ProcessorMetaSupplier . |
Modifier and Type | Method and Description |
---|---|
static <C,T,S> ProcessorMetaSupplier |
SourceProcessors.convenientSourceP(FunctionEx<? super Processor.Context,? extends C> createFn,
BiConsumerEx<? super C,? super SourceBuilder.SourceBuffer<T>> fillBufferFn,
FunctionEx<? super C,? extends S> createSnapshotFn,
BiConsumerEx<? super C,? super List<S>> restoreSnapshotFn,
ConsumerEx<? super C> destroyFn,
int preferredLocalParallelism,
boolean isBatch)
Returns a supplier of processors for a source that the user can create
using the
SourceBuilder . |
static <C,T,S> ProcessorMetaSupplier |
SourceProcessors.convenientTimestampedSourceP(FunctionEx<? super Processor.Context,? extends C> createFn,
BiConsumerEx<? super C,? super SourceBuilder.TimestampedSourceBuffer<T>> fillBufferFn,
EventTimePolicy<? super T> eventTimePolicy,
FunctionEx<? super C,? extends S> createSnapshotFn,
BiConsumerEx<? super C,? super List<S>> restoreSnapshotFn,
ConsumerEx<? super C> destroyFn,
int preferredLocalParallelism)
Returns a supplier of processors for a source that the user can create
using the
SourceBuilder . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.mergeMapP(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a supplier of processors for
Sinks.mapWithMerging(String, FunctionEx, FunctionEx,
BinaryOperatorEx) . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.mergeRemoteMapP(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn,
BinaryOperatorEx<V> mergeFn)
Returns a supplier of processors for
Sinks.remoteMapWithMerging(String, ClientConfig, FunctionEx,
FunctionEx, BinaryOperatorEx) . |
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(FunctionEx<T,? extends CharSequence> toStringFn,
PredicateEx<T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(ProcessorMetaSupplier wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(FunctionEx<? super T,? extends CharSequence> toStringFn,
PredicateEx<? super T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(ProcessorMetaSupplier wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(FunctionEx<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
PredicateEx<? super Map.Entry<K,V>> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(ProcessorMetaSupplier wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static ProcessorMetaSupplier |
SourceProcessors.readCacheP(String cacheName)
Returns a supplier of processors for
Sources.cache(String) . |
static <R> ProcessorMetaSupplier |
SourceProcessors.readFilesP(String directory,
Charset charset,
String glob,
boolean sharedFileSystem,
BiFunctionEx<? super String,? super String,? extends R> mapOutputFn)
Returns a supplier of processors for
Sources.filesBuilder(java.lang.String) . |
static <I> ProcessorMetaSupplier |
SourceProcessors.readFilesP(String directory,
String glob,
boolean sharedFileSystem,
boolean ignoreFileNotFound,
FunctionEx<? super Path,? extends Stream<I>> readFileFn)
Returns a supplier of processors for
FileSources.files(String)
to read local files. |
static <I> ProcessorMetaSupplier |
SourceProcessors.readFilesP(String directory,
String glob,
boolean sharedFileSystem,
FunctionEx<? super Path,? extends Stream<I>> readFileFn)
Returns a supplier of processors for
Sources.filesBuilder(java.lang.String) . |
static <T> ProcessorMetaSupplier |
SourceProcessors.readJdbcP(String connectionURL,
String query,
FunctionEx<? super ResultSet,? extends T> mapOutputFn)
Returns a supplier of processors for
Sources.jdbc(String, String, FunctionEx) . |
static <T> ProcessorMetaSupplier |
SourceProcessors.readJdbcP(SupplierEx<? extends Connection> newConnectionFn,
ToResultSetFunction resultSetFn,
FunctionEx<? super ResultSet,? extends T> mapOutputFn)
Returns a supplier of processors for
Sources.jdbc(
SupplierEx, ToResultSetFunction, FunctionEx) . |
static ProcessorMetaSupplier |
SourceProcessors.readListP(String listName)
Returns a supplier of processors for
Sources.list(String) . |
static ProcessorMetaSupplier |
SourceProcessors.readMapP(String mapName)
Returns a supplier of processors for
Sources.map(String) . |
static <T,K,V> ProcessorMetaSupplier |
SourceProcessors.readMapP(String mapName,
Predicate<K,V> predicate,
Projection<? super Map.Entry<K,V>,? extends T> projection)
Returns a supplier of processors for
Sources.map(String, Predicate, Projection) . |
static ProcessorMetaSupplier |
SourceProcessors.readRemoteListP(String listName,
ClientConfig clientConfig)
Returns a supplier of processors for
Sources.remoteList(String, ClientConfig) . |
static <K,V> ProcessorMetaSupplier |
SourceProcessors.streamCacheP(String cacheName,
JournalInitialPosition initialPos,
EventTimePolicy<? super Map.Entry<K,V>> eventTimePolicy)
Returns a supplier of processors for
Sources.cacheJournal(String, JournalInitialPosition) . |
static <T,K,V> ProcessorMetaSupplier |
SourceProcessors.streamCacheP(String cacheName,
PredicateEx<? super EventJournalCacheEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalCacheEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos,
EventTimePolicy<? super T> eventTimePolicy)
Returns a supplier of processors for
Sources.cacheJournal(String, JournalInitialPosition, FunctionEx, PredicateEx) . |
static ProcessorMetaSupplier |
SourceProcessors.streamFilesP(String watchedDirectory,
Charset charset,
String glob,
boolean sharedFileSystem,
BiFunctionEx<? super String,? super String,?> mapOutputFn)
Returns a supplier of processors for
Sources.filesBuilder(java.lang.String) . |
static <T> ProcessorMetaSupplier |
SourceProcessors.streamJmsQueueP(SupplierEx<? extends javax.jms.Connection> newConnectionFn,
FunctionEx<? super javax.jms.Session,? extends javax.jms.MessageConsumer> consumerFn,
FunctionEx<? super javax.jms.Message,?> messageIdFn,
FunctionEx<? super javax.jms.Message,? extends T> projectionFn,
EventTimePolicy<? super T> eventTimePolicy,
ProcessingGuarantee maxGuarantee)
Returns a supplier of processors for
Sources.jmsQueueBuilder(com.hazelcast.function.SupplierEx<? extends javax.jms.ConnectionFactory>) . |
static <T> ProcessorMetaSupplier |
SourceProcessors.streamJmsTopicP(SupplierEx<? extends javax.jms.Connection> newConnectionFn,
FunctionEx<? super javax.jms.Session,? extends javax.jms.MessageConsumer> consumerFn,
boolean isSharedConsumer,
FunctionEx<? super javax.jms.Message,?> messageIdFn,
FunctionEx<? super javax.jms.Message,? extends T> projectionFn,
EventTimePolicy<? super T> eventTimePolicy,
ProcessingGuarantee maxGuarantee)
Returns a supplier of processors for
Sources.jmsTopicBuilder(com.hazelcast.function.SupplierEx<? extends javax.jms.ConnectionFactory>) . |
static <K,V> ProcessorMetaSupplier |
SourceProcessors.streamMapP(String mapName,
JournalInitialPosition initialPos,
EventTimePolicy<? super Map.Entry<K,V>> eventTimePolicy)
Returns a supplier of processors for
Sources.mapJournal(String, JournalInitialPosition) )}. |
static <T,K,V> ProcessorMetaSupplier |
SourceProcessors.streamMapP(String mapName,
PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos,
EventTimePolicy<? super T> eventTimePolicy)
Returns a supplier of processors for
Sources.mapJournal(String, JournalInitialPosition, FunctionEx, PredicateEx) . |
static <K,V> ProcessorMetaSupplier |
SourceProcessors.streamRemoteCacheP(String cacheName,
ClientConfig clientConfig,
JournalInitialPosition initialPos,
EventTimePolicy<? super Map.Entry<K,V>> eventTimePolicy)
Returns a supplier of processors for
Sources.remoteCacheJournal(String, ClientConfig, JournalInitialPosition) . |
static <T,K,V> ProcessorMetaSupplier |
SourceProcessors.streamRemoteCacheP(String cacheName,
ClientConfig clientConfig,
PredicateEx<? super EventJournalCacheEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalCacheEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos,
EventTimePolicy<? super T> eventTimePolicy)
Returns a supplier of processors for
Sources.remoteCacheJournal(String, ClientConfig, JournalInitialPosition, FunctionEx, PredicateEx) . |
static <K,V> ProcessorMetaSupplier |
SourceProcessors.streamRemoteMapP(String mapName,
ClientConfig clientConfig,
JournalInitialPosition initialPos,
EventTimePolicy<? super Map.Entry<K,V>> eventTimePolicy)
Returns a supplier of processors for
Sources.remoteMapJournal(String, ClientConfig, JournalInitialPosition) . |
static <T,K,V> ProcessorMetaSupplier |
SourceProcessors.streamRemoteMapP(String mapName,
ClientConfig clientConfig,
PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn,
FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn,
JournalInitialPosition initialPos,
EventTimePolicy<? super T> eventTimePolicy)
Returns a supplier of processors for
Sources.remoteMapJournal(String, ClientConfig, JournalInitialPosition, FunctionEx, PredicateEx) . |
static ProcessorMetaSupplier |
SourceProcessors.streamSocketP(String host,
int port,
Charset charset)
Returns a supplier of processors for
Sources.socket(String, int, Charset) . |
static <T,K,V,R> ProcessorMetaSupplier |
SinkProcessors.updateMapP(int maxParallelAsyncOps,
String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends EntryProcessor<K,V,R>> toEntryProcessorFn)
Returns a supplier of processors for
Sinks.mapWithEntryProcessor(int, String, FunctionEx, FunctionEx) . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.updateMapP(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a supplier of processors for
Sinks.mapWithEntryProcessor(String, FunctionEx, FunctionEx) . |
static <T,K,V,R> ProcessorMetaSupplier |
SinkProcessors.updateMapP(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends EntryProcessor<K,V,R>> toEntryProcessorFn)
Returns a supplier of processors for
Sinks.mapWithEntryProcessor(String, FunctionEx, FunctionEx) . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.updateRemoteMapP(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
BiFunctionEx<? super V,? super T,? extends V> updateFn)
Returns a supplier of processors for
Sinks.remoteMapWithUpdating(String, ClientConfig, FunctionEx
, BiFunctionEx) . |
static <T,K,V,R> ProcessorMetaSupplier |
SinkProcessors.updateRemoteMapP(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends EntryProcessor<K,V,R>> toEntryProcessorFn)
Returns a supplier of processors for
Sinks.remoteMapWithEntryProcessor(String, ClientConfig, FunctionEx,
FunctionEx) . |
static ProcessorMetaSupplier |
SinkProcessors.writeCacheP(String cacheName)
Returns a supplier of processors for
Sinks.cache(String) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeFileP(String directoryName,
Charset charset,
String datePattern,
long maxFileSize,
boolean exactlyOnce,
FunctionEx<? super T,? extends String> toStringFn)
Returns a supplier of processors for
Sinks.filesBuilder(java.lang.String) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJdbcP(String updateQuery,
SupplierEx<? extends CommonDataSource> dataSourceSupplier,
BiConsumerEx<? super PreparedStatement,? super T> bindFn,
boolean exactlyOnce,
int batchLimit)
Returns a supplier of processors for
Sinks.jdbcBuilder() . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJmsQueueP(String queueName,
boolean exactlyOnce,
SupplierEx<? extends javax.jms.Connection> newConnectionFn,
BiFunctionEx<? super javax.jms.Session,? super T,? extends javax.jms.Message> messageFn)
Returns a supplier of processors for
Sinks.jmsQueueBuilder(com.hazelcast.function.SupplierEx<javax.jms.ConnectionFactory>) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeJmsTopicP(String topicName,
boolean exactlyOnce,
SupplierEx<? extends javax.jms.Connection> newConnectionFn,
BiFunctionEx<? super javax.jms.Session,? super T,? extends javax.jms.Message> messageFn)
Returns a supplier of processors for
Sinks.jmsTopicBuilder(com.hazelcast.function.SupplierEx<javax.jms.ConnectionFactory>) . |
static ProcessorMetaSupplier |
SinkProcessors.writeListP(String listName)
Returns a supplier of processors for
Sinks.list(String) . |
static ProcessorMetaSupplier |
DiagnosticProcessors.writeLoggerP()
|
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.writeLoggerP(FunctionEx<T,? extends CharSequence> toStringFn)
Returns a meta-supplier of processors for a sink vertex that logs all
the data items it receives.
|
static <K,V> ProcessorMetaSupplier |
SinkProcessors.writeMapP(String mapName)
Returns a supplier of processors for
Sinks.map(String) . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.writeMapP(String mapName,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn)
Returns a supplier of processors for
Sinks.map(String, FunctionEx, FunctionEx) . |
static ProcessorMetaSupplier |
SinkProcessors.writeObservableP(String name)
Returns a supplier of processors for
Sinks.observable(java.lang.String) . |
static ProcessorMetaSupplier |
SinkProcessors.writeRemoteCacheP(String cacheName,
ClientConfig clientConfig)
Returns a supplier of processors for
Sinks.remoteCache(String, ClientConfig) . |
static ProcessorMetaSupplier |
SinkProcessors.writeRemoteListP(String listName,
ClientConfig clientConfig)
Returns a supplier of processors for
Sinks.remoteList(String, ClientConfig) . |
static ProcessorMetaSupplier |
SinkProcessors.writeRemoteMapP(String mapName,
ClientConfig clientConfig)
Returns a supplier of processors for
Sinks.remoteMap(String, ClientConfig) . |
static <T,K,V> ProcessorMetaSupplier |
SinkProcessors.writeRemoteMapP(String mapName,
ClientConfig clientConfig,
FunctionEx<? super T,? extends K> toKeyFn,
FunctionEx<? super T,? extends V> toValueFn)
Returns a supplier of processors for
Sinks.remoteMap(String, ClientConfig, FunctionEx, FunctionEx) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeSocketP(String host,
int port,
FunctionEx<? super T,? extends String> toStringFn,
Charset charset)
Returns a supplier of processors for
Sinks.socket(String, int) . |
Modifier and Type | Method and Description |
---|---|
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(FunctionEx<T,? extends CharSequence> toStringFn,
PredicateEx<T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(ProcessorMetaSupplier wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(FunctionEx<? super T,? extends CharSequence> toStringFn,
PredicateEx<? super T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(ProcessorMetaSupplier wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(FunctionEx<? super Map.Entry<K,V>,? extends CharSequence> toStringFn,
PredicateEx<? super Map.Entry<K,V>> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(ProcessorMetaSupplier wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorMetaSupplier supplier)
Wraps the provided
ProcessorMetaSupplier with a Supplier<Processor> that returns processors obtained from it. |
static Supplier<Processor> |
TestSupport.supplierFrom(ProcessorMetaSupplier supplier,
ProcessorSupplier.Context context)
Wraps the provided
ProcessorMetaSupplier with a Supplier<Processor> that returns processors obtained from it. |
static TestSupport |
TestSupport.verifyProcessor(ProcessorMetaSupplier supplier) |
Modifier and Type | Method and Description |
---|---|
static <K,V,R> ProcessorMetaSupplier |
HadoopProcessors.readHadoopP(org.apache.hadoop.conf.Configuration configuration,
BiFunctionEx<K,V,R> projectionFn)
Returns a supplier of processors for
HadoopSources.inputFormat(Configuration, BiFunctionEx) . |
static <K,V,R> ProcessorMetaSupplier |
HadoopProcessors.readHadoopP(ConsumerEx<org.apache.hadoop.conf.Configuration> configureFn,
BiFunctionEx<K,V,R> projectionFn)
Returns a supplier of processors for
FileSources.files(String) . |
static <E,K,V> ProcessorMetaSupplier |
HadoopProcessors.writeHadoopP(org.apache.hadoop.conf.Configuration configuration,
FunctionEx<? super E,K> extractKeyFn,
FunctionEx<? super E,V> extractValueFn)
Returns a supplier of processors for
HadoopSinks.outputFormat(Configuration, FunctionEx, FunctionEx) . |
Modifier and Type | Method and Description |
---|---|
static <K,V,T> ProcessorMetaSupplier |
KafkaProcessors.streamKafkaP(Properties properties,
FunctionEx<? super org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,? extends T> projectionFn,
EventTimePolicy<? super T> eventTimePolicy,
String... topics)
Returns a supplier of processors for
KafkaSources.kafka(Properties, FunctionEx, String...) . |
static <T,K,V> ProcessorMetaSupplier |
KafkaProcessors.writeKafkaP(Properties properties,
FunctionEx<? super T,? extends org.apache.kafka.clients.producer.ProducerRecord<K,V>> toRecordFn,
boolean exactlyOnce)
Returns a supplier of processors for
KafkaSinks.kafka(Properties, FunctionEx) . |
static <T,K,V> ProcessorMetaSupplier |
KafkaProcessors.writeKafkaP(Properties properties,
String topic,
FunctionEx<? super T,? extends K> extractKeyFn,
FunctionEx<? super T,? extends V> extractValueFn,
boolean exactlyOnce)
Returns a supplier of processors for
KafkaSinks.kafka(Properties, String, FunctionEx, FunctionEx) . |
Modifier and Type | Method and Description |
---|---|
static <T> BatchSource<T> |
Sources.batchFromProcessor(String sourceName,
ProcessorMetaSupplier metaSupplier)
Returns a bounded (batch) source constructed directly from the given
Core API processor meta-supplier.
|
<R> StreamStage<R> |
StreamStageWithKey.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
<R> GeneralStage<R> |
GeneralStageWithKey.customTransform(String stageName,
ProcessorMetaSupplier procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
<R> BatchStage<R> |
BatchStageWithKey.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
<R> StreamStage<R> |
StreamStage.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
<R> GeneralStage<R> |
GeneralStage.customTransform(String stageName,
ProcessorMetaSupplier procSupplier)
Attaches a stage with a custom transform based on the provided supplier
of Core API
Processor s. |
<R> BatchStage<R> |
BatchStage.customTransform(String stageName,
ProcessorMetaSupplier procSupplier) |
static <T> Sink<T> |
Sinks.fromProcessor(String sinkName,
ProcessorMetaSupplier metaSupplier)
Returns a sink constructed directly from the given Core API processor
meta-supplier.
|
static <T> Sink<T> |
Sinks.fromProcessor(String sinkName,
ProcessorMetaSupplier metaSupplier,
FunctionEx<? super T,?> partitionKeyFn)
Returns a sink constructed directly from the given Core API processor
meta-supplier.
|
static <T> StreamSource<T> |
Sources.streamFromProcessor(String sourceName,
ProcessorMetaSupplier metaSupplier)
Returns an unbounded (event stream) source constructed directly from the given
Core API processor meta-supplier.
|
Modifier and Type | Method and Description |
---|---|
static <T> StreamSource<T> |
Sources.streamFromProcessorWithWatermarks(String sourceName,
boolean supportsNativeTimestamps,
FunctionEx<EventTimePolicy<? super T>,ProcessorMetaSupplier> metaSupplierFn)
Returns an unbounded (event stream) source that will use the supplied
function to create processor meta-suppliers as required by the Core API.
|
Modifier and Type | Method and Description |
---|---|
ProcessorMetaSupplier |
FileSourceBuilder.buildMetaSupplier()
Builds a
ProcessorMetaSupplier based on the current state of the
builder. |
Copyright © 2021 Hazelcast, Inc.. All rights reserved.