Uses of Interface
com.hazelcast.jet.pipeline.BatchSource

Packages that use BatchSource

Package	Description
com.hazelcast.jet.avro	Apache Avro file read/write support for Hazelcast Jet.
com.hazelcast.jet.elastic	Contains sources and sinks for Elasticsearch 7
com.hazelcast.jet.hadoop	Apache Hadoop read/write support for Hazelcast Jet.
com.hazelcast.jet.mongodb	Contains sources and sinks for MongoDB.
com.hazelcast.jet.pipeline	The Pipeline API is Jet's high-level API to build and execute distributed computation jobs.
com.hazelcast.jet.pipeline.file	This package offers the FileSourceBuilder which allows you to construct various kinds of Pipeline sources that read from local or distributed files.
com.hazelcast.jet.pipeline.test	This package contains various mock sources to help with pipeline testing and development.
com.hazelcast.jet.s3	AWS S3 read/write support for Hazelcast Jet.

Uses of BatchSource in com.hazelcast.jet.avro

Methods in com.hazelcast.jet.avro that return BatchSource

Modifier and Type	Method	Description
BatchSource<D>	AvroSourceBuilder.build()	Convenience for AvroSourceBuilder.build(BiFunctionEx).
<T> BatchSource<T>	AvroSourceBuilder.build(BiFunctionEx<String,? super D,T> mapOutputFn)	Builds a custom Avro file BatchSource with supplied components and the output function mapOutputFn.
static <D> BatchSource<D>	AvroSources.files(String directory, BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)	Convenience for AvroSources.filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.
static <D> BatchSource<D>	AvroSources.files(String directory, Class<D> datumClass)	Convenience for AvroSources.filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass.

Uses of BatchSource in com.hazelcast.jet.elastic

Methods in com.hazelcast.jet.elastic that return BatchSource

Modifier and Type	Method	Description
BatchSource<T>	ElasticSourceBuilder.build()	Build Elasticsearch BatchSource with supplied parameters
static BatchSource<String>	ElasticSources.elastic()	Creates a source which queries local instance of Elasticsearch for all documents
static <T> BatchSource<T>	ElasticSources.elastic(FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn)	Creates a source which queries local instance of Elasticsearch for all documents.
static BatchSource<String>	ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn)	Creates a source which queries Elasticsearch using client obtained from RestClientBuilder supplier function.
static <T> BatchSource<T>	ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn, FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn)	Creates a source which queries Elasticsearch using client obtained from RestClientBuilder supplier function.
static <T> BatchSource<T>	ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn, SupplierEx<org.elasticsearch.action.search.SearchRequest> searchRequestFn, FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn)	Creates a source which queries Elasticsearch using client obtained from RestHighLevelClient supplier.

Uses of BatchSource in com.hazelcast.jet.hadoop

Methods in com.hazelcast.jet.hadoop that return BatchSource

Modifier and Type	Method	Description
static <K, V, E> BatchSource<E>	HadoopSources.inputFormat(ConsumerEx<org.apache.hadoop.conf.Configuration> configureFn, BiFunctionEx<K,V,E> projectionFn)	Returns a source that reads records from Apache Hadoop HDFS and emits the results of transforming each record (a key-value pair) with the supplied projection function.
static <K, V> BatchSource<Map.Entry<K,V>>	HadoopSources.inputFormat(org.apache.hadoop.conf.Configuration jobConf)	Convenience for HadoopSources.inputFormat(Configuration, BiFunctionEx) with Map.Entry as its output type.
static <K, V, E> BatchSource<E>	HadoopSources.inputFormat(org.apache.hadoop.conf.Configuration configuration, BiFunctionEx<K,V,E> projectionFn)	Returns a source that reads records from Apache Hadoop HDFS and emits the results of transforming each record (a key-value pair) with the supplied projection function.

Uses of BatchSource in com.hazelcast.jet.mongodb

Methods in com.hazelcast.jet.mongodb that return BatchSource

Modifier and Type	Method	Description
static BatchSource<org.bson.Document>	MongoSources.batch(DataConnectionRef dataConnectionRef, String database, String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection)	Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
static BatchSource<org.bson.Document>	MongoSources.batch(String connectionString, String database, String collection, org.bson.conversions.Bson filter, org.bson.conversions.Bson projection)	Returns a MongoDB batch source which queries the collection using given filter and applies the given projection on the documents.
BatchSource<T>	MongoSourceBuilder.Batch.build()	Creates and returns the MongoDB BatchSource.

Uses of BatchSource in com.hazelcast.jet.pipeline

Methods in com.hazelcast.jet.pipeline that return BatchSource

Modifier and Type	Method	Description
static <T> BatchSource<T>	Sources.batchFromProcessor(String sourceName, ProcessorMetaSupplier metaSupplier)	Returns a bounded (batch) source constructed directly from the given Core API processor meta-supplier.
BatchSource<String>	FileSourceBuilder.build()	Deprecated. Use FileSources.files(java.lang.String).
<T> BatchSource<T>	FileSourceBuilder.build(BiFunctionEx<String,String,? extends T> mapOutputFn)	Deprecated. Use FileSources.files(java.lang.String).
<T> BatchSource<T>	FileSourceBuilder.build(FunctionEx<? super Path,? extends Stream<T>> readFileFn)	Deprecated. Use FileSources.files(java.lang.String).
<N> BatchSource<N>	RemoteMapSourceBuilder.build()	Build the source using the parameters set in this builder.
BatchSource<T>	SourceBuilder.Batch.build()	Builds and returns the batch source.
static <K, V> BatchSource<Map.Entry<K,V>>	Sources.cache(String cacheName)	Returns a source that fetches entries from a Hazelcast ICache with the given name and emits them as Map.Entry.
static BatchSource<String>	Sources.files(String directory)	A source to read all files in a directory in a batch way.
static <T> BatchSource<T>	Sources.jdbc(SupplierEx<? extends Connection> newConnectionFn, ToResultSetFunction resultSetFn, FunctionEx<? super ResultSet,? extends T> createOutputFn)	Returns a source which connects to the specified database using the given newConnectionFn, queries the database and creates a result set using the the given resultSetFn.
static <T> BatchSource<T>	Sources.jdbc(DataConnectionRef dataConnectionRef, ToResultSetFunction resultSetFn, FunctionEx<? super ResultSet,? extends T> createOutputFn)	Returns a source which connects to the specified database using the given dataConnectionRef, queries the database and creates a result set using the given resultSetFn.
static <T> BatchSource<T>	Sources.jdbc(String connectionURL, String query, FunctionEx<? super ResultSet,? extends T> createOutputFn)	Convenience for Sources.jdbc(SupplierEx, ToResultSetFunction, FunctionEx).
static <T> BatchSource<T>	Sources.jdbc(String connectionURL, String query, Properties properties, FunctionEx<? super ResultSet,? extends T> createOutputFn)	Same as @{link Sources.jdbc(String, String, FunctionEx)}
static BatchSource<Map<String,Object>>	Sources.json(String directory)	Convenience for Sources.json(String, Class) which converts each JSON string to a Map.
static <T> BatchSource<T>	Sources.json(String directory, Class<T> type)	A source to read all files in a directory in a batch way.
static <T> BatchSource<T>	Sources.list(IList<? extends T> list)	Returns a source that emits items retrieved from a Hazelcast IList.
static <T> BatchSource<T>	Sources.list(String listName)	Returns a source that emits items retrieved from a Hazelcast IList.
static <K, V> BatchSource<Map.Entry<K,V>>	Sources.map(IMap<? extends K,? extends V> map)	Returns a source that fetches entries from the given Hazelcast IMap and emits them as Map.Entry.
static <T, K, V> BatchSource<T>	Sources.map(IMap<? extends K,? extends V> map, Predicate<K,V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection)	Returns a source that fetches entries from the given Hazelcast IMap.
static <K, V> BatchSource<Map.Entry<K,V>>	Sources.map(String mapName)	Returns a source that fetches entries from a local Hazelcast IMap with the specified name and emits them as Map.Entry.
static <T, K, V> BatchSource<T>	Sources.map(String mapName, Predicate<K,V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection)	Returns a source that fetches entries from a local Hazelcast IMap with the specified name.
static <K, V> BatchSource<Map.Entry<K,V>>	Sources.remoteCache(String cacheName, ClientConfig clientConfig)	Returns a source that fetches entries from the Hazelcast ICache with the specified name in a remote cluster identified by the supplied ClientConfig and emits them as Map.Entry.
static <T> BatchSource<T>	Sources.remoteList(String listName, ClientConfig clientConfig)	Returns a source that emits items retrieved from a Hazelcast IList in a remote cluster identified by the supplied ClientConfig.
static <K, V> BatchSource<Map.Entry<K,V>>	Sources.remoteMap(String mapName, ClientConfig clientConfig)	Returns a source that fetches entries from the Hazelcast IMap with the specified name in a remote cluster identified by the supplied ClientConfig and emits them as Map.Entry.
static <T, K, V> BatchSource<T>	Sources.remoteMap(String mapName, ClientConfig clientConfig, Predicate<K,V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection)	Returns a source that fetches entries from a remote Hazelcast IMap with the specified name in a remote cluster identified by the supplied ClientConfig.
static <K, V> BatchSource<Map.Entry<K,V>>	Sources.remoteMap(String mapName, DataConnectionRef dataConnectionRef)	The same as the Sources.remoteMap(String, ClientConfig, Predicate, Projection) method.
static <T, K, V> BatchSource<T>	Sources.remoteMap(String mapName, DataConnectionRef dataConnectionRef, Predicate<K,V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection)	The same as the Sources.remoteMap(String, ClientConfig, Predicate, Projection) method.

Methods in com.hazelcast.jet.pipeline with parameters of type BatchSource

Modifier and Type	Method	Description
<T> BatchStage<T>	Pipeline.readFrom(BatchSource<? extends T> source)	Returns a pipeline stage that represents a bounded (batch) data source.

Uses of BatchSource in com.hazelcast.jet.pipeline.file

Methods in com.hazelcast.jet.pipeline.file that return BatchSource

Modifier and Type	Method	Description
BatchSource<T>	FileSourceBuilder.build()	Builds a BatchSource based on the current state of the builder.

Uses of BatchSource in com.hazelcast.jet.pipeline.test

Methods in com.hazelcast.jet.pipeline.test that return BatchSource

Modifier and Type	Method	Description
static <T> BatchSource<T>	TestSources.items(Iterable<? extends T> items)	Returns a batch source which iterates through the supplied iterable and then terminates.
static <T> BatchSource<T>	TestSources.items(T... items)	Returns a batch source which iterates through the supplied items and then terminates.
static <T> BatchSource<T>	TestSources.itemsDistributed(Iterable<? extends T> items)	Returns a batch source which iterates through the supplied iterable and then terminates.
static <T> BatchSource<T>	TestSources.itemsDistributed(T... items)	Returns a batch source which iterates through the supplied items and then terminates.

Uses of BatchSource in com.hazelcast.jet.s3

Methods in com.hazelcast.jet.s3 that return BatchSource

Modifier and Type	Method	Description
static BatchSource<String>	S3Sources.s3(List<String> bucketNames, String prefix, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier)	Convenience for S3Sources.s3(List, String, Charset, SupplierEx, BiFunctionEx).
static <I, T> BatchSource<T>	S3Sources.s3(List<String> bucketNames, String prefix, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, FunctionEx<? super InputStream,? extends Stream<I>> readFileFn, BiFunctionEx<String,? super I,? extends T> mapFn)	Creates an AWS S3 BatchSource which lists all the objects in the bucket-list using given prefix, reads them using provided readFileFn, transforms each read item to the desired output object using given mapFn and emits them to downstream.
static <I, T> BatchSource<T>	S3Sources.s3(List<String> bucketNames, String prefix, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, TriFunction<? super InputStream,String,String,? extends Stream<I>> readFileFn, BiFunctionEx<String,? super I,? extends T> mapFn)	Creates an AWS S3 BatchSource which lists all the objects in the bucket-list using given prefix, reads them using provided readFileFn, transforms each read item to the desired output object using given mapFn and emits them to downstream.
static <T> BatchSource<T>	S3Sources.s3(List<String> bucketNames, String prefix, Charset charset, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, BiFunctionEx<String,String,? extends T> mapFn)	Creates an AWS S3 BatchSource which lists all the objects in the bucket-list using given prefix, reads them line by line, transforms each line to the desired output object using given mapFn and emits them to downstream.