Package | Description |
---|---|
com.hazelcast.jet |
The Pipeline API is Jet's high-level API to build and execute
distributed computation jobs.
|
Modifier and Type | Method and Description |
---|---|
static <K,V> Source<Map.Entry<K,V>> |
Sources.cache(String cacheName)
Returns a source that fetches entries from the Hazelcast
ICache
with the specified name and emits them as Map.Entry . |
static <K,V> Source<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>> |
Sources.cacheJournal(String cacheName,
boolean startFromLatestSequence)
Convenience for
Sources.cacheJournal(String, DistributedPredicate,
DistributedFunction, boolean) with no projection or filtering. |
static <K,V,T> Source<T> |
Sources.cacheJournal(String cacheName,
DistributedPredicate<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>> predicateFn,
DistributedFunction<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>,T> projectionFn,
boolean startFromLatestSequence)
Returns a source that will stream the
EventJournalCacheEvent
events of the Hazelcast ICache with the specified name. |
static Source<String> |
Sources.files(String directory)
Convenience for
readFiles(directory, UTF_8, "*") . |
static Source<String> |
Sources.files(String directory,
Charset charset,
String glob)
A source that emits lines from files in a directory (but not its
subdirectories.
|
static Source<String> |
Sources.fileWatcher(String watchedDirectory)
Convenience for
streamFiles(watchedDirectory, UTF_8, "*") . |
static Source<String> |
Sources.fileWatcher(String watchedDirectory,
Charset charset,
String glob)
A source that emits a stream of lines of text coming from files in
the watched directory (but not its subdirectories).
|
static <T> Source<T> |
Sources.fromProcessor(String sourceName,
ProcessorMetaSupplier metaSupplier)
Returns a source constructed directly from the given Core API processor
meta-supplier.
|
static <K,V> Source<Map.Entry<K,V>> |
HdfsSources.hdfs(org.apache.hadoop.mapred.JobConf jobConf)
Convenience for
HdfsSources.hdfs(JobConf, DistributedBiFunction)
with Map.Entry as its output type. |
static <K,V,E> Source<E> |
HdfsSources.hdfs(org.apache.hadoop.mapred.JobConf jobConf,
DistributedBiFunction<K,V,E> mapper)
Returns a source that reads records from Apache Hadoop HDFS and emits
the results of transforming each record (a key-value pair) with the
supplied mapping function.
|
static <K,V,T> Source<T> |
KafkaSources.kafka(Properties properties,
DistributedBiFunction<K,V,T> projectionFn,
String... topics)
Returns a source that consumes one or more Apache Kafka topics and emits
items from them as
Map.Entry instances. |
static <K,V> Source<Map.Entry<K,V>> |
KafkaSources.kafka(Properties properties,
String... topics)
Convenience for
KafkaSources.kafka(Properties, DistributedBiFunction,
String...) wrapping the output in Map.Entry . |
static <E> Source<E> |
Sources.list(String listName)
Returns a source that emits items retrieved from a Hazelcast
IList . |
static <K,V> Source<Map.Entry<K,V>> |
Sources.map(String mapName)
Returns a source that fetches entries from a local Hazelcast
IMap
with the specified name and emits them as Map.Entry . |
static <K,V,T> Source<T> |
Sources.map(String mapName,
com.hazelcast.query.Predicate<K,V> predicate,
DistributedFunction<Map.Entry<K,V>,T> projectionFn)
Convenience for
Sources.map(String, Predicate, Projection)
which uses a DistributedFunction as the projection function. |
static <K,V,T> Source<T> |
Sources.map(String mapName,
com.hazelcast.query.Predicate<K,V> predicate,
com.hazelcast.projection.Projection<Map.Entry<K,V>,T> projection)
Returns a source that fetches entries from a local Hazelcast
IMap with the specified name. |
static <K,V> Source<com.hazelcast.map.journal.EventJournalMapEvent<K,V>> |
Sources.mapJournal(String mapName,
boolean startFromLatestSequence)
Convenience for
Sources.mapJournal(String, DistributedPredicate,
DistributedFunction, boolean) with no projection or filtering. |
static <K,V,T> Source<T> |
Sources.mapJournal(String mapName,
DistributedPredicate<com.hazelcast.map.journal.EventJournalMapEvent<K,V>> predicateFn,
DistributedFunction<com.hazelcast.map.journal.EventJournalMapEvent<K,V>,T> projectionFn,
boolean startFromLatestSequence)
Returns a source that will stream the
EventJournalMapEvent
events of the Hazelcast IMap with the specified name. |
static <K,V> Source<Map.Entry<K,V>> |
Sources.remoteCache(String cacheName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a source that fetches entries from the Hazelcast
ICache
with the specified name in a remote cluster identified by the supplied
ClientConfig and emits them as Map.Entry . |
static <K,V> Source<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>> |
Sources.remoteCacheJournal(String cacheName,
com.hazelcast.client.config.ClientConfig clientConfig,
boolean startFromLatestSequence)
Convenience for
Sources.remoteCacheJournal(String, ClientConfig,
DistributedPredicate, DistributedFunction, boolean) with no projection
or filtering. |
static <K,V,T> Source<T> |
Sources.remoteCacheJournal(String cacheName,
com.hazelcast.client.config.ClientConfig clientConfig,
DistributedPredicate<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>> predicateFn,
DistributedFunction<com.hazelcast.cache.journal.EventJournalCacheEvent<K,V>,T> projectionFn,
boolean startFromLatestSequence)
Returns a source that will stream the
EventJournalCacheEvent
events of the Hazelcast ICache with the specified name from a
remote cluster. |
static <E> Source<E> |
Sources.remoteList(String listName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a source that emits items retrieved from a Hazelcast
IList in a remote cluster identified by the supplied ClientConfig . |
static <K,V> Source<Map.Entry<K,V>> |
Sources.remoteMap(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig)
Returns a source that fetches entries from the Hazelcast
IMap
with the specified name in a remote cluster identified by the supplied
ClientConfig and emits them as Map.Entry . |
static <K,V,T> Source<T> |
Sources.remoteMap(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
com.hazelcast.query.Predicate<K,V> predicate,
DistributedFunction<Map.Entry<K,V>,T> projectionFn)
Convenience for
Sources.remoteMap(String, ClientConfig, Predicate, Projection)
which use a DistributedFunction as the projection function. |
static <K,V,T> Source<T> |
Sources.remoteMap(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
com.hazelcast.query.Predicate<K,V> predicate,
com.hazelcast.projection.Projection<Map.Entry<K,V>,T> projection)
Returns a source that fetches entries from a remote Hazelcast
IMap with the specified name in a remote cluster identified by the
supplied ClientConfig . |
static <K,V> Source<com.hazelcast.map.journal.EventJournalMapEvent<K,V>> |
Sources.remoteMapJournal(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
boolean startFromLatestSequence)
Convenience for
Sources.remoteMapJournal(String, ClientConfig,
DistributedPredicate, DistributedFunction, boolean) with no projection
or filtering. |
static <K,V,T> Source<T> |
Sources.remoteMapJournal(String mapName,
com.hazelcast.client.config.ClientConfig clientConfig,
DistributedPredicate<com.hazelcast.map.journal.EventJournalMapEvent<K,V>> predicateFn,
DistributedFunction<com.hazelcast.map.journal.EventJournalMapEvent<K,V>,T> projectionFn,
boolean startFromLatestSequence)
Returns a source that will stream the
EventJournalMapEvent
events of the Hazelcast IMap with the specified name from a
remote cluster. |
static Source<String> |
Sources.socket(String host,
int port,
Charset charset)
Returns a source which connects to the specified socket and emits lines
of text received from it.
|
Modifier and Type | Method and Description |
---|---|
<E> ComputeStage<E> |
Pipeline.drawFrom(Source<E> source)
Returns a new pipeline stage that has no upstream stages and produces
some output for its downstream stages.
|
Copyright © 2017 Hazelcast, Inc.. All Rights Reserved.