Uses of Interface
com.hazelcast.jet.pipeline.BatchSource

Packages that use BatchSource

Package	Description
com.hazelcast.jet.avro	Apache Avro file read/write support for Hazelcast Jet.
com.hazelcast.jet.elastic	Contains sources and sinks for Elasticsearch 7
com.hazelcast.jet.hadoop	Apache Hadoop read/write support for Hazelcast Jet.
com.hazelcast.jet.pipeline	The Pipeline API is Jet's high-level API to build and execute distributed computation jobs.
com.hazelcast.jet.pipeline.file	This package offers the FileSourceBuilder which allows you to construct various kinds of Pipeline sources that read from local or distributed files.
com.hazelcast.jet.pipeline.test	This package contains various mock sources to help with pipeline testing and development.
com.hazelcast.jet.s3	AWS S3 read/write support for Hazelcast Jet.

Uses of BatchSource in com.hazelcast.jet.avro

Methods in com.hazelcast.jet.avro that return BatchSource

Modifier and Type	Method and Description
BatchSource<D>	AvroSourceBuilder.build() Convenience for AvroSourceBuilder.build(BiFunctionEx).
<T> BatchSource<T>	AvroSourceBuilder.build(BiFunctionEx<String,? super D,T> mapOutputFn) Builds a custom Avro file BatchSource with supplied components and the output function mapOutputFn.
static <D> BatchSource<D>	AvroSources.files(String directory, BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn) Convenience for AvroSources.filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.
static <D> BatchSource<D>	AvroSources.files(String directory, Class<D> datumClass) Convenience for AvroSources.filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass.

Uses of BatchSource in com.hazelcast.jet.elastic

Methods in com.hazelcast.jet.elastic that return BatchSource

Modifier and Type	Method and Description
BatchSource<T>	ElasticSourceBuilder.build() Build Elasticsearch BatchSource with supplied parameters
static BatchSource<String>	ElasticSources.elastic() Creates a source which queries local instance of Elasticsearch for all documents
static <T> BatchSource<T>	ElasticSources.elastic(FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn) Creates a source which queries local instance of Elasticsearch for all documents.
static BatchSource<String>	ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn) Creates a source which queries Elasticsearch using client obtained from RestClientBuilder supplier function.
static <T> BatchSource<T>	ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn, FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn) Creates a source which queries Elasticsearch using client obtained from RestClientBuilder supplier function.
static <T> BatchSource<T>	ElasticSources.elastic(SupplierEx<org.elasticsearch.client.RestClientBuilder> clientFn, SupplierEx<org.elasticsearch.action.search.SearchRequest> searchRequestFn, FunctionEx<? super org.elasticsearch.search.SearchHit,T> mapToItemFn) Creates a source which queries Elasticsearch using client obtained from RestHighLevelClient supplier.

Uses of BatchSource in com.hazelcast.jet.hadoop

Methods in com.hazelcast.jet.hadoop that return BatchSource

Modifier and Type	Method and Description
static <K,V> BatchSource<Map.Entry<K,V>>	HadoopSources.inputFormat(org.apache.hadoop.conf.Configuration jobConf) Convenience for HadoopSources.inputFormat(Configuration, BiFunctionEx) with Map.Entry as its output type.
static <K,V,E> BatchSource<E>	HadoopSources.inputFormat(org.apache.hadoop.conf.Configuration configuration, BiFunctionEx<K,V,E> projectionFn) Returns a source that reads records from Apache Hadoop HDFS and emits the results of transforming each record (a key-value pair) with the supplied projection function.
static <K,V,E> BatchSource<E>	HadoopSources.inputFormat(ConsumerEx<org.apache.hadoop.conf.Configuration> configureFn, BiFunctionEx<K,V,E> projectionFn) Returns a source that reads records from Apache Hadoop HDFS and emits the results of transforming each record (a key-value pair) with the supplied projection function.

Uses of BatchSource in com.hazelcast.jet.pipeline

Methods in com.hazelcast.jet.pipeline that return BatchSource

Modifier and Type	Method and Description
static <T> BatchSource<T>	Sources.batchFromProcessor(String sourceName, ProcessorMetaSupplier metaSupplier) Returns a bounded (batch) source constructed directly from the given Core API processor meta-supplier.
BatchSource<String>	FileSourceBuilder.build() Deprecated. Use FileSources.files(java.lang.String). Will be removed in Jet 5.0.
BatchSource<T>	SourceBuilder.Batch.build() Builds and returns the batch source.
<T> BatchSource<T>	FileSourceBuilder.build(BiFunctionEx<String,String,? extends T> mapOutputFn) Deprecated. Use FileSources.files(java.lang.String). Will be removed in Jet 5.0.
<T> BatchSource<T>	FileSourceBuilder.build(FunctionEx<? super Path,? extends Stream<T>> readFileFn) Deprecated. Use FileSources.files(java.lang.String). Will be removed in Jet 5.0.
static <K,V> BatchSource<Map.Entry<K,V>>	Sources.cache(String cacheName) Returns a source that fetches entries from a Hazelcast ICache with the given name and emits them as Map.Entry.
static BatchSource<String>	Sources.files(String directory) A source to read all files in a directory in a batch way.
static <T> BatchSource<T>	Sources.jdbc(String connectionURL, String query, FunctionEx<? super ResultSet,? extends T> createOutputFn) Convenience for Sources.jdbc(SupplierEx, ToResultSetFunction, FunctionEx).
static <T> BatchSource<T>	Sources.jdbc(SupplierEx<? extends Connection> newConnectionFn, ToResultSetFunction resultSetFn, FunctionEx<? super ResultSet,? extends T> createOutputFn) Returns a source which connects to the specified database using the given newConnectionFn, queries the database and creates a result set using the the given resultSetFn.
static BatchSource<Map<String,Object>>	Sources.json(String directory) Convenience for Sources.json(String, Class) which converts each JSON string to a Map.
static <T> BatchSource<T>	Sources.json(String directory, Class<T> type) A source to read all files in a directory in a batch way.
static <T> BatchSource<T>	Sources.list(IList<? extends T> list) Returns a source that emits items retrieved from a Hazelcast IList.
static <T> BatchSource<T>	Sources.list(String listName) Returns a source that emits items retrieved from a Hazelcast IList.
static <K,V> BatchSource<Map.Entry<K,V>>	Sources.map(IMap<? extends K,? extends V> map) Returns a source that fetches entries from the given Hazelcast IMap and emits them as Map.Entry.
static <T,K,V> BatchSource<T>	Sources.map(IMap<? extends K,? extends V> map, Predicate<K,V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection) Returns a source that fetches entries from the given Hazelcast IMap.
static <K,V> BatchSource<Map.Entry<K,V>>	Sources.map(String mapName) Returns a source that fetches entries from a local Hazelcast IMap with the specified name and emits them as Map.Entry.
static <T,K,V> BatchSource<T>	Sources.map(String mapName, Predicate<K,V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection) Returns a source that fetches entries from a local Hazelcast IMap with the specified name.
static <K,V> BatchSource<Map.Entry<K,V>>	Sources.remoteCache(String cacheName, ClientConfig clientConfig) Returns a source that fetches entries from the Hazelcast ICache with the specified name in a remote cluster identified by the supplied ClientConfig and emits them as Map.Entry.
static <T> BatchSource<T>	Sources.remoteList(String listName, ClientConfig clientConfig) Returns a source that emits items retrieved from a Hazelcast IList in a remote cluster identified by the supplied ClientConfig.
static <K,V> BatchSource<Map.Entry<K,V>>	Sources.remoteMap(String mapName, ClientConfig clientConfig) Returns a source that fetches entries from the Hazelcast IMap with the specified name in a remote cluster identified by the supplied ClientConfig and emits them as Map.Entry.
static <T,K,V> BatchSource<T>	Sources.remoteMap(String mapName, ClientConfig clientConfig, Predicate<K,V> predicate, Projection<? super Map.Entry<K,V>,? extends T> projection) Returns a source that fetches entries from a remote Hazelcast IMap with the specified name in a remote cluster identified by the supplied ClientConfig.

Methods in com.hazelcast.jet.pipeline with parameters of type BatchSource

Modifier and Type	Method and Description
<T> BatchStage<T>	Pipeline.readFrom(BatchSource<? extends T> source) Returns a pipeline stage that represents a bounded (batch) data source.

Uses of BatchSource in com.hazelcast.jet.pipeline.file

Methods in com.hazelcast.jet.pipeline.file that return BatchSource

Modifier and Type	Method and Description
BatchSource<T>	FileSourceBuilder.build() Builds a BatchSource based on the current state of the builder.

Uses of BatchSource in com.hazelcast.jet.pipeline.test

Methods in com.hazelcast.jet.pipeline.test that return BatchSource

Modifier and Type	Method and Description
static <T> BatchSource<T>	TestSources.items(Iterable<? extends T> items) Returns a batch source which iterates through the supplied iterable and then terminates.
static <T> BatchSource<T>	TestSources.items(T... items) Returns a batch source which iterates through the supplied items and then terminates.
static <T> BatchSource<T>	TestSources.itemsDistributed(Iterable<? extends T> items) Returns a batch source which iterates through the supplied iterable and then terminates.
static <T> BatchSource<T>	TestSources.itemsDistributed(T... items) Returns a batch source which iterates through the supplied items and then terminates.

Uses of BatchSource in com.hazelcast.jet.s3

Methods in com.hazelcast.jet.s3 that return BatchSource

Modifier and Type	Method and Description
static <T> BatchSource<T>	S3Sources.s3(List<String> bucketNames, String prefix, Charset charset, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, BiFunctionEx<String,String,? extends T> mapFn) Creates an AWS S3 BatchSource which lists all the objects in the bucket-list using given prefix, reads them line by line, transforms each line to the desired output object using given mapFn and emits them to downstream.
static BatchSource<String>	S3Sources.s3(List<String> bucketNames, String prefix, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier) Convenience for S3Sources.s3(List, String, Charset, SupplierEx, BiFunctionEx).
static <I,T> BatchSource<T>	S3Sources.s3(List<String> bucketNames, String prefix, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, FunctionEx<? super InputStream,? extends Stream<I>> readFileFn, BiFunctionEx<String,? super I,? extends T> mapFn) Creates an AWS S3 BatchSource which lists all the objects in the bucket-list using given prefix, reads them using provided readFileFn, transforms each read item to the desired output object using given mapFn and emits them to downstream.
static <I,T> BatchSource<T>	S3Sources.s3(List<String> bucketNames, String prefix, SupplierEx<? extends software.amazon.awssdk.services.s3.S3Client> clientSupplier, TriFunction<? super InputStream,String,String,? extends Stream<I>> readFileFn, BiFunctionEx<String,? super I,? extends T> mapFn) Creates an AWS S3 BatchSource which lists all the objects in the bucket-list using given prefix, reads them using provided readFileFn, transforms each read item to the desired output object using given mapFn and emits them to downstream.