Uses of Interface
com.hazelcast.jet.pipeline.StreamSource

Packages that use StreamSource

Package	Description
com.hazelcast.jet.cdc	Contains source/sink connectors that deal with Change Data Capture (CDC) events from various databases as well as a generic connector for Debezium CDC sources.
com.hazelcast.jet.cdc.mysql	Contains connectors for change data capture events from MySQL databases.
com.hazelcast.jet.cdc.postgres	Contains connectors for change data capture events from PostgreSQL databases.
com.hazelcast.jet.kafka	Apache Kafka reader/writer support for Hazelcast Jet.
com.hazelcast.jet.kafka.connect	Contains a generic Kafka Connect source provides ability to plug any Kafka Connect source for data ingestion to Jet pipelines.
com.hazelcast.jet.kinesis	Amazon Kinesis Data Streams producer/consumer support for Hazelcast Jet.
com.hazelcast.jet.mongodb	Contains sources and sinks for MongoDB.
com.hazelcast.jet.pipeline	The Pipeline API is Jet's high-level API to build and execute distributed computation jobs.
com.hazelcast.jet.pipeline.test	This package contains various mock sources to help with pipeline testing and development.

Uses of StreamSource in com.hazelcast.jet.cdc

Methods in com.hazelcast.jet.cdc that return StreamSource

Modifier and Type	Method and Description
StreamSource<T>	DebeziumCdcSources.Builder.build() Returns the CDC source based on the properties set.

Uses of StreamSource in com.hazelcast.jet.cdc.mysql

Methods in com.hazelcast.jet.cdc.mysql that return StreamSource

Modifier and Type	Method and Description
StreamSource<ChangeRecord>	MySqlCdcSources.Builder.build() Returns the source based on the properties set so far.

Uses of StreamSource in com.hazelcast.jet.cdc.postgres

Methods in com.hazelcast.jet.cdc.postgres that return StreamSource

Modifier and Type	Method and Description
StreamSource<ChangeRecord>	PostgresCdcSources.Builder.build() Returns the source based on the properties set so far.

Uses of StreamSource in com.hazelcast.jet.kafka

Methods in com.hazelcast.jet.kafka that return StreamSource

Modifier and Type	Method and Description
static <K,V,T> StreamSource<T>	KafkaSources.kafka(DataConnectionRef dataConnectionRef, FunctionEx<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,T> projectionFn, String... topics) Returns a source that consumes one or more Apache Kafka topics and emits items from them as Map.Entry instances.
static <K,V> StreamSource<Map.Entry<K,V>>	KafkaSources.kafka(DataConnectionRef dataConnectionRef, String... topics) Convenience for KafkaSources.kafka(DataConnectionRef, FunctionEx, String...) wrapping the output in Map.Entry.
static <K,V,T> StreamSource<T>	KafkaSources.kafka(Properties properties, FunctionEx<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,T> projectionFn, String... topics) Returns a source that consumes one or more Apache Kafka topics and emits items from them as Map.Entry instances.
static <K,V,T> StreamSource<T>	KafkaSources.kafka(Properties properties, FunctionEx<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,T> projectionFn, TopicsConfig topicsConfig) Returns a source that consumes one or more Apache Kafka topics and emits items from them as Map.Entry instances.
static <K,V> StreamSource<Map.Entry<K,V>>	KafkaSources.kafka(Properties properties, String... topics) Convenience for KafkaSources.kafka(Properties, FunctionEx, String...) wrapping the output in Map.Entry.

Uses of StreamSource in com.hazelcast.jet.kafka.connect

Methods in com.hazelcast.jet.kafka.connect that return StreamSource

Modifier and Type	Method and Description
static StreamSource<org.apache.kafka.connect.source.SourceRecord>	KafkaConnectSources.connect(Properties properties) A generic Kafka Connect source provides ability to plug any Kafka Connect source for data ingestion to Jet pipelines.
static <T> StreamSource<T>	KafkaConnectSources.connect(Properties properties, FunctionEx<org.apache.kafka.connect.source.SourceRecord,T> projectionFn) A generic Kafka Connect source provides ability to plug any Kafka Connect source for data ingestion to Jet pipelines.

Uses of StreamSource in com.hazelcast.jet.kinesis

Methods in com.hazelcast.jet.kinesis that return StreamSource

Modifier and Type	Method and Description
StreamSource<T>	KinesisSources.Builder.build() Constructs the source based on the options provided so far.

Uses of StreamSource in com.hazelcast.jet.mongodb

Methods in com.hazelcast.jet.mongodb that return StreamSource

Modifier and Type	Method and Description
StreamSource<T>	MongoSourceBuilder.Stream.build() Creates and returns the MongoDB StreamSource which watches the given collection.
static StreamSource<? extends org.bson.Document>	MongoSources.stream(String connectionString, String database, String collection, org.bson.Document filter, org.bson.Document projection) Returns a MongoDB stream source which watches the changes on the collection.

Uses of StreamSource in com.hazelcast.jet.pipeline

Methods in com.hazelcast.jet.pipeline that return StreamSource

Modifier and Type	Method and Description
StreamSource<javax.jms.Message>	JmsSourceBuilder.build() Convenience for JmsSourceBuilder.build(FunctionEx).
StreamSource<T>	SourceBuilder.Stream.build() Builds and returns the unbounded stream source.
StreamSource<T>	SourceBuilder.TimestampedStream.build() Builds and returns the timestamped stream source.
<T> StreamSource<T>	JmsSourceBuilder.build(FunctionEx<? super javax.jms.Message,? extends T> projectionFn) Creates and returns the JMS StreamSource with the supplied components and the projection function projectionFn.
StreamSource<String>	FileSourceBuilder.buildWatcher() Convenience for FileSourceBuilder.buildWatcher(BiFunctionEx).
<T> StreamSource<T>	FileSourceBuilder.buildWatcher(BiFunctionEx<String,String,? extends T> mapOutputFn) Builds a source that emits a stream of lines of text coming from files in the watched directory (but not its subdirectories).
static <K,V> StreamSource<Map.Entry<K,V>>	Sources.cacheJournal(String cacheName, JournalInitialPosition initialPos) Convenience for Sources.cacheJournal(String, JournalInitialPosition, FunctionEx, PredicateEx) which will pass only CREATED and UPDATED events and will project the event's key and new value into a Map.Entry.
static <T,K,V> StreamSource<T>	Sources.cacheJournal(String cacheName, JournalInitialPosition initialPos, FunctionEx<? super EventJournalCacheEvent<K,V>,? extends T> projectionFn, PredicateEx<? super EventJournalCacheEvent<K,V>> predicateFn) Returns a source that will stream the EventJournalCacheEvent events of a Hazelcast ICache with the specified name.
static StreamSource<String>	Sources.fileWatcher(String watchedDirectory) A source to stream lines added to files in a directory.
static StreamSource<javax.jms.Message>	Sources.jmsQueue(String name, SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier) Shortcut equivalent to:
static StreamSource<javax.jms.Message>	Sources.jmsQueue(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier, String name) Deprecated. see Sources.jmsQueue(String, SupplierEx).
static StreamSource<javax.jms.Message>	Sources.jmsTopic(String name, SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier) Shortcut equivalent to:
static StreamSource<javax.jms.Message>	Sources.jmsTopic(SupplierEx<? extends javax.jms.ConnectionFactory> factorySupplier, String name) Deprecated. see Sources.jmsTopic(String, SupplierEx).
static StreamSource<Map<String,Object>>	Sources.jsonWatcher(String watchedDirectory) Convenience for Sources.jsonWatcher(String, Class) which converts each line appended to the Map representation of the JSON string.
static <T> StreamSource<T>	Sources.jsonWatcher(String watchedDirectory, Class<T> type) A source to stream lines added to files in a directory.
static <K,V> StreamSource<Map.Entry<K,V>>	Sources.mapJournal(IMap<? extends K,? extends V> map, JournalInitialPosition initialPos) Convenience for Sources.mapJournal(IMap, JournalInitialPosition, FunctionEx, PredicateEx) which will pass only ADDED and UPDATED events and will project the event's key and new value into a Map.Entry.
static <T,K,V> StreamSource<T>	Sources.mapJournal(IMap<? extends K,? extends V> map, JournalInitialPosition initialPos, FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn, PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn) Returns a source that will stream EventJournalMapEvents of the given Hazelcast IMap.
static <K,V> StreamSource<Map.Entry<K,V>>	Sources.mapJournal(String mapName, JournalInitialPosition initialPos) Convenience for Sources.mapJournal(String, JournalInitialPosition, FunctionEx, PredicateEx) which will pass only ADDED and UPDATED events and will project the event's key and new value into a Map.Entry.
static <T,K,V> StreamSource<T>	Sources.mapJournal(String mapName, JournalInitialPosition initialPos, FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn, PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn) Returns a source that will stream EventJournalMapEvents of the Hazelcast IMap with the specified name.
static <K,V> StreamSource<Map.Entry<K,V>>	Sources.remoteCacheJournal(String cacheName, ClientConfig clientConfig, JournalInitialPosition initialPos) Convenience for Sources.remoteCacheJournal(String, ClientConfig, JournalInitialPosition, FunctionEx, PredicateEx) which will pass only CREATED and UPDATED events and will project the event's key and new value into a Map.Entry.
static <T,K,V> StreamSource<T>	Sources.remoteCacheJournal(String cacheName, ClientConfig clientConfig, JournalInitialPosition initialPos, FunctionEx<? super EventJournalCacheEvent<K,V>,? extends T> projectionFn, PredicateEx<? super EventJournalCacheEvent<K,V>> predicateFn) Returns a source that will stream the EventJournalCacheEvent events of the Hazelcast ICache with the specified name from a remote cluster.
static <K,V> StreamSource<Map.Entry<K,V>>	Sources.remoteMapJournal(String mapName, ClientConfig clientConfig, JournalInitialPosition initialPos) Convenience for Sources.remoteMapJournal(String, ClientConfig, JournalInitialPosition, FunctionEx, PredicateEx) which will pass only ADDED and UPDATED events and will project the event's key and new value into a Map.Entry.
static <T,K,V> StreamSource<T>	Sources.remoteMapJournal(String mapName, ClientConfig clientConfig, JournalInitialPosition initialPos, FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn, PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn) Returns a source that will stream the EventJournalMapEvent events of the Hazelcast IMap with the specified name from a remote cluster.
static <K,V> StreamSource<Map.Entry<K,V>>	Sources.remoteMapJournal(String mapName, DataConnectionRef dataConnectionRef, JournalInitialPosition initialPos) Convenience for Sources.remoteMapJournal(String, DataConnectionRef, JournalInitialPosition, FunctionEx, PredicateEx) which will pass only ADDED and UPDATED events and will project the event's key and new value into a Map.Entry.
static <T,K,V> StreamSource<T>	Sources.remoteMapJournal(String mapName, DataConnectionRef dataConnectionRef, JournalInitialPosition initialPos, FunctionEx<? super EventJournalMapEvent<K,V>,? extends T> projectionFn, PredicateEx<? super EventJournalMapEvent<K,V>> predicateFn) The same as the Sources.remoteMapJournal(String, ClientConfig, JournalInitialPosition, FunctionEx, PredicateEx) method.
StreamSource<T>	StreamSource.setPartitionIdleTimeout(long timeoutMillis) Sets a timeout after which idle partitions will be excluded from watermark coalescing.
static StreamSource<String>	Sources.socket(String host, int port) Convenience for socket(host, port, charset) with UTF-8 as the charset.
static StreamSource<String>	Sources.socket(String host, int port, Charset charset) Returns a source which connects to the specified socket and emits lines of text received from it.
static <T> StreamSource<T>	Sources.streamFromProcessor(String sourceName, ProcessorMetaSupplier metaSupplier) Returns an unbounded (event stream) source constructed directly from the given Core API processor meta-supplier.
static <T> StreamSource<T>	Sources.streamFromProcessorWithWatermarks(String sourceName, boolean supportsNativeTimestamps, FunctionEx<EventTimePolicy<? super T>,ProcessorMetaSupplier> metaSupplierFn) Returns an unbounded (event stream) source that will use the supplied function to create processor meta-suppliers as required by the Core API.

Methods in com.hazelcast.jet.pipeline with parameters of type StreamSource

Modifier and Type	Method and Description
<T> StreamSourceStage<T>	Pipeline.readFrom(StreamSource<? extends T> source) Returns a pipeline stage that represents an unbounded data source (i.e., an event stream).

Uses of StreamSource in com.hazelcast.jet.pipeline.test

Methods in com.hazelcast.jet.pipeline.test that return StreamSource

Modifier and Type	Method and Description
static StreamSource<SimpleEvent>	TestSources.itemStream(int itemsPerSecond) Returns a streaming source that generates events of type SimpleEvent at the specified rate.
static <T> StreamSource<T>	TestSources.itemStream(int itemsPerSecond, GeneratorFunction<? extends T> generatorFn) Returns a streaming source that generates events created by the generatorFn at the specified rate.
static StreamSource<Long>	TestSources.longStream(long eventsPerSecond, long initialDelayMillis) Returns a StreamSource that emits an ever-increasing sequence of Long numbers with native timestamps that are exactly the same amount of time apart, as specified by the supplied eventsPerSecond parameter.