public final class AvroSinks extends Object
Modifier and Type | Method and Description |
---|---|
static Sink<org.apache.avro.generic.IndexedRecord> |
files(String directoryName,
DistributedSupplier<org.apache.avro.Schema> schemaSupplier)
Convenience for
files(String, DistributedSupplier,
DistributedSupplier) which uses GenericDatumWriter . |
static <R> Sink<R> |
files(String directoryName,
DistributedSupplier<org.apache.avro.Schema> schemaSupplier,
Class<R> recordClass)
Convenience for
files(String, DistributedSupplier,
DistributedSupplier) which uses either SpecificDatumWriter or
ReflectDatumWriter depending on the supplied recordClass . |
static <R> Sink<R> |
files(String directoryName,
DistributedSupplier<org.apache.avro.Schema> schemaSupplier,
DistributedSupplier<org.apache.avro.io.DatumWriter<R>> datumWriterSupplier)
Returns a sink that that writes the items it receives to Apache Avro
files.
|
@Nonnull public static <R> Sink<R> files(@Nonnull String directoryName, @Nonnull DistributedSupplier<org.apache.avro.Schema> schemaSupplier, @Nonnull DistributedSupplier<org.apache.avro.io.DatumWriter<R>> datumWriterSupplier)
The sink creates a DataFileWriter
for each processor using the
supplied datumWriterSupplier
with the given Schema
.
No state is saved to snapshot for this sink. After the job is restarted, the items will be missing since files will be overwritten.
The default local parallelism for this sink is 1.
R
- the type of the recorddirectoryName
- directory to create the files in. Will be created
if it doesn't exist. Must be the same on all members.schemaSupplier
- the record schema supplierdatumWriterSupplier
- the record writer supplier@Nonnull public static <R> Sink<R> files(@Nonnull String directoryName, @Nonnull DistributedSupplier<org.apache.avro.Schema> schemaSupplier, @Nonnull Class<R> recordClass)
files(String, DistributedSupplier,
DistributedSupplier)
which uses either SpecificDatumWriter
or
ReflectDatumWriter
depending on the supplied recordClass
.@Nonnull public static Sink<org.apache.avro.generic.IndexedRecord> files(@Nonnull String directoryName, @Nonnull DistributedSupplier<org.apache.avro.Schema> schemaSupplier)
files(String, DistributedSupplier,
DistributedSupplier)
which uses GenericDatumWriter
.Copyright © 2018 Hazelcast, Inc.. All rights reserved.