Package | Description |
---|---|
com.hazelcast.jet |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
com.hazelcast.jet.core |
Jet's Core API.
|
com.hazelcast.jet.core.processor |
Apache Kafka reader/writer support for Hazelcast Jet.
|
com.hazelcast.jet.core.test |
Utilities for writing tests of Core API Processors.
|
com.hazelcast.jet.stream |
java.util.stream implementation using Hazelcast Jet
|
Modifier and Type | Method and Description |
---|---|
static <E> Sink<E> |
Sinks.fromProcessor(String sinkName,
ProcessorMetaSupplier metaSupplier)
Returns a sink constructed directly from the given Core API processor
meta-supplier.
|
static <T> Source<T> |
Sources.fromProcessor(String sourceName,
ProcessorMetaSupplier metaSupplier)
Returns a source constructed directly from the given Core API processor
meta-supplier.
|
Modifier and Type | Method and Description |
---|---|
static ProcessorMetaSupplier |
ProcessorMetaSupplier.dontParallelize(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.dontParallelize(ProcessorSupplier supplier)
Wraps the provided
ProcessorSupplier into a meta-supplier that
will always return it. |
ProcessorMetaSupplier |
Vertex.getMetaSupplier()
Returns this vertex's meta-supplier of processors.
|
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedFunction<com.hazelcast.nio.Address,ProcessorSupplier> addressToSupplier)
Factory method that creates a
ProcessorMetaSupplier from the
supplied function that maps a cluster member address to a ProcessorSupplier . |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedFunction<com.hazelcast.nio.Address,ProcessorSupplier> addressToSupplier,
int preferredLocalParallelism)
Factory method that creates a
ProcessorMetaSupplier from the
supplied function that maps a cluster member address to a ProcessorSupplier . |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(DistributedSupplier<? extends Processor> procSupplier,
int preferredLocalParallelism)
Factory method that wraps the given
Supplier<Processor>
and uses it as the supplier of all Processor instances. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(ProcessorSupplier procSupplier)
Wraps the provided
ProcessorSupplier into a meta-supplier that
will always return it. |
static ProcessorMetaSupplier |
ProcessorMetaSupplier.of(ProcessorSupplier procSupplier,
int preferredLocalParallelism)
Factory method that wraps the given
ProcessorSupplier and
returns the same instance for each given Address . |
Modifier and Type | Method and Description |
---|---|
Vertex |
DAG.newVertex(String name,
ProcessorMetaSupplier metaSupplier)
Creates a vertex from a
ProcessorMetaSupplier and adds it to this DAG. |
Modifier and Type | Method and Description |
---|---|
void |
Vertex.updateMetaSupplier(java.util.function.UnaryOperator<ProcessorMetaSupplier> updateFn)
Applies the provided operator function to the current processor
meta-supplier and replaces it with the one it returns.
|
Constructor and Description |
---|
Vertex(String name,
ProcessorMetaSupplier metaSupplier)
Creates a vertex from a
ProcessorMetaSupplier . |
Modifier and Type | Method and Description |
---|---|
static ProcessorMetaSupplier |
Processors.nonCooperativeP(ProcessorMetaSupplier wrapped)
Decorates a processor meta-supplier with one that will declare all its
processors non-cooperative.
|
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(ProcessorMetaSupplier wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(ProcessorMetaSupplier wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<Map.Entry<K,V>,String> toStringFn,
DistributedPredicate<Map.Entry<K,V>> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(ProcessorMetaSupplier wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static ProcessorMetaSupplier |
SourceProcessors.readCacheP(String cacheName)
Returns a supplier of processors for
Sources.cache(String) . |
static ProcessorMetaSupplier |
SourceProcessors.readFilesP(String directory,
Charset charset,
String glob)
Returns a supplier of processors for
Sources.files(String, Charset, String) . |
static ProcessorMetaSupplier |
SourceProcessors.readListP(String listName)
Returns a supplier of processors for
Sources.list(String) . |
static ProcessorMetaSupplier |
SourceProcessors.readMapP(String mapName)
Returns a supplier of processors for
Sources.map(String) . |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.readMapP(String mapName,
com.hazelcast.query.Predicate<K,V> predicate,
DistributedFunction<Map.Entry<K,V>,T> projectionFn)
Returns a supplier of processors for
Sources.map(String, Predicate, DistributedFunction) }. |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.readMapP(String mapName,
com.hazelcast.query.Predicate<K,V> predicate,
com.hazelcast.projection.Projection<Map.Entry<K,V>,T> projectionFn)
Returns a supplier of processors for
Sources.map(String, Predicate, Projection) }. |
static ProcessorMetaSupplier |
SourceProcessors.readRemoteCacheP(String cacheName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a supplier of processors for
Sources.remoteCache(String, ClientConfig) . |
static ProcessorMetaSupplier |
SourceProcessors.readRemoteListP(String listName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a supplier of processors for
Sources.remoteList(String, ClientConfig) . |
static ProcessorMetaSupplier |
SourceProcessors.readRemoteMapP(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a supplier of processors for
Sources.remoteMap(String, ClientConfig) . |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.readRemoteMapP(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
com.hazelcast.query.Predicate<K,V> predicate,
DistributedFunction<Map.Entry<K,V>,T> projectionFn)
Returns a supplier of processors for
Sources.remoteMap(String, ClientConfig, Predicate, DistributedFunction) . |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.readRemoteMapP(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
com.hazelcast.query.Predicate<K,V> predicate,
com.hazelcast.projection.Projection<Map.Entry<K,V>,T> projection)
Returns a supplier of processors for
Sources.remoteMap(String, ClientConfig, Predicate, Projection) . |
static ProcessorMetaSupplier |
SourceProcessors.streamCacheP(String cacheName,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.cacheJournal(String, boolean) . |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.streamCacheP(String cacheName,
DistributedPredicate<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>> predicate,
DistributedFunction<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>,T> projection,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.cacheJournal(String, DistributedPredicate, DistributedFunction, boolean) . |
static ProcessorMetaSupplier |
SourceProcessors.streamFilesP(String watchedDirectory,
Charset charset,
String glob)
Returns a supplier of processors for
Sources.fileWatcher(String, Charset, String) . |
static <K,V,T> ProcessorMetaSupplier |
KafkaProcessors.streamKafkaP(Properties properties,
DistributedBiFunction<K,V,T> projectionFn,
String... topics)
Returns a supplier of processors for
KafkaSources.kafka(Properties, DistributedBiFunction, String...) . |
static ProcessorMetaSupplier |
KafkaProcessors.streamKafkaP(Properties properties,
String... topics)
Returns a supplier of processors for
KafkaSources.kafka(Properties, String...) . |
static ProcessorMetaSupplier |
SourceProcessors.streamMapP(String mapName,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.mapJournal(String, boolean) . |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.streamMapP(String mapName,
DistributedPredicate<com.hazelcast.map.journal.EventJournalMapEvent<K,V>> predicate,
DistributedFunction<com.hazelcast.map.journal.EventJournalMapEvent<K,V>,T> projection,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.mapJournal(String, DistributedPredicate, DistributedFunction, boolean) . |
static ProcessorMetaSupplier |
SourceProcessors.streamRemoteCacheP(String cacheName,
com.hazelcast.client.config.ClientConfig clientConfig,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.remoteCacheJournal(String, ClientConfig, boolean) . |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.streamRemoteCacheP(String cacheName,
com.hazelcast.client.config.ClientConfig clientConfig,
DistributedPredicate<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>> predicate,
DistributedFunction<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>,T> projection,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.remoteCacheJournal(
String, ClientConfig, DistributedPredicate, DistributedFunction, boolean
) . |
static ProcessorMetaSupplier |
SourceProcessors.streamRemoteMapP(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.remoteMapJournal(String, ClientConfig, boolean) . |
static <K,V,T> ProcessorMetaSupplier |
SourceProcessors.streamRemoteMapP(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
DistributedPredicate<com.hazelcast.map.journal.EventJournalMapEvent<K,V>> predicate,
DistributedFunction<com.hazelcast.map.journal.EventJournalMapEvent<K,V>,T> projection,
boolean startFromLatestSequence)
Returns a supplier of processors for
Sources.remoteMapJournal(
String, ClientConfig, DistributedPredicate, DistributedFunction, boolean
) . |
static ProcessorMetaSupplier |
SourceProcessors.streamSocketP(String host,
int port,
Charset charset)
Returns a supplier of processors for
Sources.socket(String, int, Charset) . |
static ProcessorMetaSupplier |
SinkProcessors.writeCacheP(String cacheName)
Returns a supplier of processors for
Sinks.cache(String) . |
static ProcessorMetaSupplier |
SinkProcessors.writeFileP(String directoryName)
Returns a supplier of processors for
Sinks.files(String) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeFileP(String directoryName,
DistributedFunction<T,String> toStringFn)
Returns a supplier of processors for
Sinks.files(String, DistributedFunction) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeFileP(String directoryName,
DistributedFunction<T,String> toStringFn,
Charset charset,
boolean append)
Returns a supplier of processors for
Sinks.files(String, DistributedFunction, Charset, boolean) . |
static <E,K,V> ProcessorMetaSupplier |
HdfsProcessors.writeHdfsP(org.apache.hadoop.mapred.JobConf jobConf,
DistributedFunction<? super E,K> extractKeyFn,
DistributedFunction<? super E,V> extractValueFn)
Returns a supplier of processors for
HdfsSinks.hdfs(JobConf, DistributedFunction, DistributedFunction) . |
static <T,K,V> ProcessorMetaSupplier |
KafkaProcessors.writeKafkaP(Properties properties,
DistributedFunction<? super T,org.apache.kafka.clients.producer.ProducerRecord<K,V>> toRecordFn)
Returns a supplier of processors for
KafkaSinks.kafka(Properties, DistributedFunction) . |
static <T,K,V> ProcessorMetaSupplier |
KafkaProcessors.writeKafkaP(Properties properties,
String topic,
DistributedFunction<? super T,K> extractKeyFn,
DistributedFunction<? super T,V> extractValueFn)
Returns a supplier of processors for
KafkaSinks.kafka(Properties, String, DistributedFunction, DistributedFunction) . |
static ProcessorMetaSupplier |
SinkProcessors.writeListP(String listName)
Returns a supplier of processors for
Sinks.list(String) . |
static ProcessorMetaSupplier |
DiagnosticProcessors.writeLoggerP()
Convenience for
DiagnosticProcessors.writeLoggerP(DistributedFunction) that uses
toString() as toStringFn . |
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.writeLoggerP(DistributedFunction<T,String> toStringFn)
Returns a meta-supplier of processors for a sink vertex that logs all
the data items it receives.
|
static ProcessorMetaSupplier |
SinkProcessors.writeMapP(String mapName)
Returns a supplier of processors for
Sinks.map(String) . |
static ProcessorMetaSupplier |
SinkProcessors.writeRemoteCacheP(String cacheName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a supplier of processors for
Sinks.remoteCache(String, ClientConfig) . |
static ProcessorMetaSupplier |
SinkProcessors.writeRemoteListP(String listName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a supplier of processors for
Sinks.remoteList(String, ClientConfig) . |
static ProcessorMetaSupplier |
SinkProcessors.writeRemoteMapP(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a supplier of processors for
Sinks.remoteMap(String, ClientConfig) . |
static <T> ProcessorMetaSupplier |
SinkProcessors.writeSocketP(String host,
int port,
DistributedFunction<T,String> toStringFn,
Charset charset)
Returns a supplier of processors for
Sinks.socket(String, int) . |
Modifier and Type | Method and Description |
---|---|
static ProcessorMetaSupplier |
Processors.nonCooperativeP(ProcessorMetaSupplier wrapped)
Decorates a processor meta-supplier with one that will declare all its
processors non-cooperative.
|
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekInputP(ProcessorMetaSupplier wrapped)
Convenience for
peekInput(toStringFn,
shouldLogFn, metaSupplier) with a pass-through filter and Object#toString as the formatting function. |
static <T> ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(DistributedFunction<T,String> toStringFn,
DistributedPredicate<T> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekOutputP(ProcessorMetaSupplier wrapped)
Convenience for
peekOutput(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
static <K,V> ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(DistributedFunction<Map.Entry<K,V>,String> toStringFn,
DistributedPredicate<Map.Entry<K,V>> shouldLogFn,
ProcessorMetaSupplier wrapped)
Returns a meta-supplier that wraps the provided one and adds a logging
layer to each processor it creates.
|
static ProcessorMetaSupplier |
DiagnosticProcessors.peekSnapshotP(ProcessorMetaSupplier wrapped)
Convenience for
peekSnapshot(toStringFn,
shouldLogFn, metaSupplier with a pass-through filter and Object#toString as the formatting function. |
Modifier and Type | Method and Description |
---|---|
static java.util.function.Supplier<Processor> |
TestSupport.supplierFrom(ProcessorMetaSupplier supplier)
Wraps the provided
ProcessorMetaSupplier with a Supplier<Processor> that returns processors obtained from it. |
static TestSupport |
TestSupport.verifyProcessor(ProcessorMetaSupplier supplier) |
Modifier and Type | Method and Description |
---|---|
static <T> DistributedStream<T> |
DistributedStream.fromSource(JetInstance instance,
ProcessorMetaSupplier metaSupplier)
Returns a distributed
Stream with given processors as its source. |
Copyright © 2017 Hazelcast, Inc.. All Rights Reserved.