Package com.hazelcast.jet.pipeline

Class FileSinkBuilder<T>

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static long DISABLE_ROLLING
      A value to pass to rollByFileSize(long) if you want to disable rolling by file size.
      static java.lang.String TEMP_FILE_SUFFIX
      A suffix added to file names until they are committed.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      Sink<T> build()
      Creates and returns the file Sink with the supplied components.
      FileSinkBuilder<T> charset​(java.nio.charset.Charset charset)
      Sets the character set used to encode the files.
      FileSinkBuilder<T> exactlyOnce​(boolean enable)
      Enables or disables the exactly-once behavior of the sink using two-phase commit of state snapshots.
      FileSinkBuilder<T> rollByDate​(java.lang.String datePattern)
      Sets a date pattern that will be included in the file name.
      FileSinkBuilder<T> rollByFileSize​(long maxFileSize)
      Enables rolling by file size.
      FileSinkBuilder<T> toStringFn​(FunctionEx<? super T,​java.lang.String> toStringFn)
      Sets the function which converts the item to its string representation.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • toStringFn

        @Nonnull
public FileSinkBuilder<T> toStringFn​(@Nonnull
                                     FunctionEx<? super T,​java.lang.String> toStringFn)
        Sets the function which converts the item to its string representation. Each item is followed with a platform-specific line separator. Default value is Object.toString().

        The given function must be stateless and cooperative.

      • charset

        @Nonnull
public FileSinkBuilder<T> charset​(@Nonnull
                                  java.nio.charset.Charset charset)
        Sets the character set used to encode the files. Default value is StandardCharsets.UTF_8.

      • rollByDate

        @Nonnull
public FileSinkBuilder<T> rollByDate​(@Nullable
                                     java.lang.String datePattern)
        Sets a date pattern that will be included in the file name. Each time the formatted current time changes a new file will be started. For example, if the datePattern is yyyy-MM-dd, a new file will be started every day.

        The rolling is based on system time, not on event time. By default no rolling by date is done. If the system clock goes back, the outcome is unspecified and possibly corrupt.

        Since:
        Jet 4.0

      • rollByFileSize

        @Nonnull
public FileSinkBuilder<T> rollByFileSize​(long maxFileSize)
        Enables rolling by file size. If the size after writing a batch of items exceeds the limit, a new file will be started. From this follows that the file will typically be larger than the given maximum.

        To disable rolling after certain size, pass DISABLE_ROLLING. This is the default value.

        Since:
        Jet 4.0

      • exactlyOnce

        @Nonnull
public FileSinkBuilder<T> exactlyOnce​(boolean enable)
        Enables or disables the exactly-once behavior of the sink using two-phase commit of state snapshots. If enabled, the processing guarantee of the job must be set to exactly-once, otherwise the sink's guarantee will match that of the job. In other words, sink's guarantee cannot be higher than job's, but can be lower to avoid the additional overhead.

        See Sinks.filesBuilder(String) for more information.

        The default value is true.

        Parameters:
        enable - If true, sink's guarantee will match the job guarantee. If false, sink's guarantee will be at-least-once even if job's is exactly-once
        Returns:
        this instance for fluent API
        Since:
        Jet 4.0

      • build

        @Nonnull
public Sink<T> build()
        Creates and returns the file Sink with the supplied components.