Package com.hazelcast.jet.avro

Class AvroSources

java.lang.Object
com.hazelcast.jet.avro.AvroSources
public final class AvroSources extends Object
Contains factory methods for Apache Avro sources.
Since:
Jet 3.0

  • Method Details

    • filesBuilder

      @Nonnull public static <D> AvroSourceBuilder<D> filesBuilder(@Nonnull String directory, @Nonnull Class<D> recordClass)
      Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).
      Type Parameters:
      D - the type of the datum
      Parameters:
      directory - parent directory of the files
      recordClass - the class to read

    • filesBuilder

      @Nonnull public static <D> AvroSourceBuilder<D> filesBuilder(@Nonnull String directory, @Nonnull SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier)
      Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).
      Type Parameters:
      D - the type of the datum
      Parameters:
      directory - parent directory of the files
      datumReaderSupplier - the supplier of datum reader which reads records from the files

    • files

      @Nonnull public static <D> BatchSource<D> files(@Nonnull String directory, @Nonnull Class<D> datumClass)
      Convenience for filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass. If datumClass implements SpecificRecord, SpecificDatumReader is used to read the records, ReflectDatumReader is used otherwise.

    • files

      @Nonnull public static <D> BatchSource<D> files(@Nonnull String directory, @Nonnull BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)
      Convenience for filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.