Package com.hazelcast.jet.avro

Class AvroSources

  • public final class AvroSources
extends java.lang.Object
    Contains factory methods for Apache Avro sources.
    Since:
    Jet 3.0

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static <D> BatchSource<D> files​(java.lang.String directory, BiFunctionEx<java.lang.String,​org.apache.avro.generic.GenericRecord,​D> mapOutputFn)
      Convenience for filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.
      static <D> BatchSource<D> files​(java.lang.String directory, java.lang.Class<D> datumClass)
      Convenience for filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass.
      static <D> AvroSourceBuilder<D> filesBuilder​(java.lang.String directory, SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier)
      Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API.
      static <D> AvroSourceBuilder<D> filesBuilder​(java.lang.String directory, java.lang.Class<D> recordClass)
      Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • filesBuilder

        @Nonnull
public static <D> AvroSourceBuilder<D> filesBuilder​(@Nonnull
                                                    java.lang.String directory,
                                                    @Nonnull
                                                    java.lang.Class<D> recordClass)
        Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).
        Type Parameters:
        D - the type of the datum
        Parameters:
        directory - parent directory of the files
        recordClass - the class to read

      • filesBuilder

        @Nonnull
public static <D> AvroSourceBuilder<D> filesBuilder​(@Nonnull
                                                    java.lang.String directory,
                                                    @Nonnull
                                                    SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier)
        Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).
        Type Parameters:
        D - the type of the datum
        Parameters:
        directory - parent directory of the files
        datumReaderSupplier - the supplier of datum reader which reads records from the files

      • files

        @Nonnull
public static <D> BatchSource<D> files​(@Nonnull
                                       java.lang.String directory,
                                       @Nonnull
                                       java.lang.Class<D> datumClass)
        Convenience for filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass. If datumClass implements SpecificRecord, SpecificDatumReader is used to read the records, ReflectDatumReader is used otherwise.

      • files

        @Nonnull
public static <D> BatchSource<D> files​(@Nonnull
                                       java.lang.String directory,
                                       @Nonnull
                                       BiFunctionEx<java.lang.String,​org.apache.avro.generic.GenericRecord,​D> mapOutputFn)
        Convenience for filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.