com.hazelcast.jet.avro

Class AvroSources

java.lang.Object

com.hazelcast.jet.avro.AvroSources

public final class AvroSources
extends Object

Contains factory methods for Apache Avro sources.

Since:

Jet 3.0

Method Summary

Modifier and Type	Method and Description
static <D> BatchSource<D>	files(String directory, BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn) Convenience for filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.
static <D> BatchSource<D>	files(String directory, Class<D> datumClass) Convenience for filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass.
static <D> AvroSourceBuilder<D>	filesBuilder(String directory, Class<D> recordClass) Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API.
static <D> AvroSourceBuilder<D>	filesBuilder(String directory, SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier) Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

filesBuilder

@Nonnull
public static <D> AvroSourceBuilder<D> filesBuilder(@Nonnull
                                                             String directory,
                                                             @Nonnull
                                                             Class<D> recordClass)

Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).

Type Parameters:

D - the type of the datum

Parameters:

directory - parent directory of the files

recordClass - the class to read

filesBuilder

@Nonnull
public static <D> AvroSourceBuilder<D> filesBuilder(@Nonnull
                                                             String directory,
                                                             @Nonnull
                                                             SupplierEx<? extends org.apache.avro.io.DatumReader<D>> datumReaderSupplier)

Returns a builder object that offers a step-by-step fluent API to build a custom Avro file source for the Pipeline API. The source reads records from Apache Avro files in a directory (but not its subdirectories).

Type Parameters:

D - the type of the datum

Parameters:

directory - parent directory of the files

datumReaderSupplier - the supplier of datum reader which reads records from the files

files

@Nonnull
public static <D> BatchSource<D> files(@Nonnull
                                                String directory,
                                                @Nonnull
                                                Class<D> datumClass)

Convenience for filesBuilder(String, Class) which reads all the files in the supplied directory as specific records using supplied datumClass. If datumClass implements SpecificRecord, SpecificDatumReader is used to read the records, ReflectDatumReader is used otherwise.

files

@Nonnull
public static <D> BatchSource<D> files(@Nonnull
                                                String directory,
                                                @Nonnull
                                                BiFunctionEx<String,org.apache.avro.generic.GenericRecord,D> mapOutputFn)

Convenience for filesBuilder(String, SupplierEx) which reads all the files in the supplied directory as generic records and emits the results of transforming each generic record with the supplied mapping function.