com.hazelcast.jet.pipeline

Class FileSinkBuilder<T>

java.lang.Object

com.hazelcast.jet.pipeline.FileSinkBuilder<T>

Type Parameters:

T - type of the items the sink accepts

public final class FileSinkBuilder<T>
extends Object

See Sinks.filesBuilder(java.lang.String).

Since:

Jet 3.0

Field Summary

Fields

Modifier and Type	Field and Description
static long	DISABLE_ROLLING A value to pass to rollByFileSize(long) if you want to disable rolling by file size.
static String	TEMP_FILE_SUFFIX A suffix added to file names until they are committed.

Method Summary

Modifier and Type	Method and Description
Sink<T>	build() Creates and returns the file Sink with the supplied components.
FileSinkBuilder<T>	charset(Charset charset) Sets the character set used to encode the files.
FileSinkBuilder<T>	exactlyOnce(boolean enable) Enables or disables the exactly-once behavior of the sink using two-phase commit of state snapshots.
FileSinkBuilder<T>	rollByDate(String datePattern) Sets a date pattern that will be included in the file name.
FileSinkBuilder<T>	rollByFileSize(long maxFileSize) Enables rolling by file size.
FileSinkBuilder<T>	toStringFn(FunctionEx<? super T,String> toStringFn) Sets the function which converts the item to its string representation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TEMP_FILE_SUFFIX

public static final String TEMP_FILE_SUFFIX

A suffix added to file names until they are committed. Files ending with this suffix should be ignored when processing. See Sinks.filesBuilder(java.lang.String) for more information.

See Also:

Constant Field Values

DISABLE_ROLLING

public static final long DISABLE_ROLLING

A value to pass to rollByFileSize(long) if you want to disable rolling by file size.

See Also:

Constant Field Values

Method Detail

toStringFn

@Nonnull
public FileSinkBuilder<T> toStringFn(@Nonnull
                                              FunctionEx<? super T,String> toStringFn)

Sets the function which converts the item to its string representation. Each item is followed with a platform-specific line separator. Default value is Object.toString().

The given function must be stateless and cooperative.

charset

@Nonnull
public FileSinkBuilder<T> charset(@Nonnull
                                           Charset charset)

Sets the character set used to encode the files. Default value is StandardCharsets.UTF_8.

rollByDate

@Nonnull
public FileSinkBuilder<T> rollByDate(@Nullable
                                              String datePattern)

Sets a date pattern that will be included in the file name. Each time the formatted current time changes a new file will be started. For example, if the datePattern is yyyy-MM-dd, a new file will be started every day.

The rolling is based on system time, not on event time. By default no rolling by date is done. If the system clock goes back, the outcome is unspecified and possibly corrupt.

Since:

Jet 4.0

rollByFileSize

@Nonnull
public FileSinkBuilder<T> rollByFileSize(long maxFileSize)

Enables rolling by file size. If the size after writing a batch of items exceeds the limit, a new file will be started. From this follows that the file will typically be larger than the given maximum.

To disable rolling after certain size, pass DISABLE_ROLLING. This is the default value.

Since:

Jet 4.0

exactlyOnce

@Nonnull
public FileSinkBuilder<T> exactlyOnce(boolean enable)

Enables or disables the exactly-once behavior of the sink using two-phase commit of state snapshots. If enabled, the processing guarantee of the job must be set to exactly-once, otherwise the sink's guarantee will match that of the job. In other words, sink's guarantee cannot be higher than job's, but can be lower to avoid the additional overhead.

See Sinks.filesBuilder(String) for more information.

The default value is true.

Parameters:

enable - If true, sink's guarantee will match the job guarantee. If false, sink's guarantee will be at-least-once even if job's is exactly-once

Returns:

this instance for fluent API

Since:

Jet 4.0

build

@Nonnull
public Sink<T> build()

Creates and returns the file Sink with the supplied components.