Package com.hazelcast.jet.avro
Class AvroSourceBuilder<D>
java.lang.Object
com.hazelcast.jet.avro.AvroSourceBuilder<D>
- Type Parameters:
D
- the type of the datum read bydatumReaderSupplier
Builder for an Avro file source that reads records from Avro files in a
directory (but not its subdirectories) and emits objects returned by
mapOutputFn
.- Since:
- Jet 3.0
-
Method Summary
Modifier and TypeMethodDescriptionbuild()
Convenience forbuild(BiFunctionEx)
.<T> BatchSource<T>
build
(BiFunctionEx<String, ? super D, T> mapOutputFn) Builds a custom Avro fileBatchSource
with supplied components and the output functionmapOutputFn
.Sets the globbing mask, seegetPathMatcher()
.sharedFileSystem
(boolean sharedFileSystem) Sets whether files are in a shared storage visible to all members.
-
Method Details
-
glob
Sets the globbing mask, seegetPathMatcher()
. The default value is"*"
, which means all files. -
build
Builds a custom Avro fileBatchSource
with supplied components and the output functionmapOutputFn
.The source does not save any state to the snapshot. If the job is restarted, it will re-emit all entries.
Any
IOException
will cause the job to fail. The files must not change while being read; if they do, the behavior is unspecified.The default local parallelism for this processor is 4 (or available CPU count if it is less than 4).
- Type Parameters:
T
- the type of the items the source emits- Parameters:
mapOutputFn
- the function which creates output object from each record. Gets the filename and record read bydatumReader
as parameters
-
build
Convenience forbuild(BiFunctionEx)
. Builds a source that emits the records as read bydatumReader
, without any transformation.
-