Package com.hazelcast.jet.pipeline.test

Class TestSources

  • @EvolvingApi
public final class TestSources
extends java.lang.Object
    Contains factory methods for various mock sources which can be used for pipeline testing and development.
    Since:
    Jet 3.2

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static <T> BatchSource<T> items​(java.lang.Iterable<? extends T> items)
      Returns a batch source which iterates through the supplied iterable and then terminates.
      static <T> BatchSource<T> items​(T... items)
      Returns a batch source which iterates through the supplied items and then terminates.
      static <T> BatchSource<T> itemsDistributed​(java.lang.Iterable<? extends T> items)
      Returns a batch source which iterates through the supplied iterable and then terminates.
      static <T> BatchSource<T> itemsDistributed​(T... items)
      Returns a batch source which iterates through the supplied items and then terminates.
      static StreamSource<SimpleEvent> itemStream​(int itemsPerSecond)
      Returns a streaming source that generates events of type SimpleEvent at the specified rate.
      static <T> StreamSource<T> itemStream​(int itemsPerSecond, GeneratorFunction<? extends T> generatorFn)
      Returns a streaming source that generates events created by the generatorFn at the specified rate.
      static StreamSource<java.lang.Long> longStream​(long eventsPerSecond, long initialDelayMillis)
      Returns a StreamSource that emits an ever-increasing sequence of Long numbers with native timestamps that are exactly the same amount of time apart, as specified by the supplied eventsPerSecond parameter.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • items

        @Nonnull
public static <T> BatchSource<T> items​(@Nonnull
                                       java.lang.Iterable<? extends T> items)
        Returns a batch source which iterates through the supplied iterable and then terminates. The source is non-distributed.
        Since:
        Jet 3.2

      • items

        @Nonnull
public static <T> BatchSource<T> items​(@Nonnull
                                       T... items)
        Returns a batch source which iterates through the supplied items and then terminates. The source is non-distributed.
        Since:
        Jet 3.2

      • itemsDistributed

        @Nonnull
public static <T> BatchSource<T> itemsDistributed​(@Nonnull
                                                  java.lang.Iterable<? extends T> items)
        Returns a batch source which iterates through the supplied iterable and then terminates. The source is distributed - a slice of the items is emitted on each member with local parallelism of 1.
        Since:
        Jet 4.4

      • itemsDistributed

        @Nonnull
public static <T> BatchSource<T> itemsDistributed​(@Nonnull
                                                  T... items)
        Returns a batch source which iterates through the supplied items and then terminates. The source is distributed - a slice of the items is emitted on each member with local parallelism of 1.
        Since:
        Jet 4.4

      • itemStream

        @EvolvingApi
@Nonnull
public static StreamSource<SimpleEvent> itemStream​(int itemsPerSecond)
        Returns a streaming source that generates events of type SimpleEvent at the specified rate.

        The source supports native timestamps. The timestamp is the current system time at the moment they are generated. The source is not distributed and all the items are generated on the same node. This source is not fault-tolerant. The sequence will be reset once a job is restarted.

        Note: There is no absolute guarantee that the actual rate of emitted items will match the supplied value. It is ensured that no emitted event's timestamp will be in the future.

        Parameters:
        itemsPerSecond - how many items should be emitted each second
        Since:
        Jet 3.2

      • itemStream

        @EvolvingApi
@Nonnull
public static <T> StreamSource<T> itemStream​(int itemsPerSecond,
                                             @Nonnull
                                             GeneratorFunction<? extends T> generatorFn)
        Returns a streaming source that generates events created by the generatorFn at the specified rate.

        The source supports native timestamps. The timestamp is the current system time at the moment they are generated. The source is not distributed and all the items are generated on the same node. This source is not fault-tolerant. The sequence will be reset once a job is restarted.

        Note: There is no absolute guarantee that the actual rate of emitted items will match the supplied value. It is ensured that no emitted event's timestamp will be in the future.

        Parameters:
        itemsPerSecond - how many items should be emitted each second
        generatorFn - a function which takes the timestamp and the sequence of the generated item and maps it to the desired type
        Since:
        Jet 3.2

      • longStream

        @Nonnull
public static StreamSource<java.lang.Long> longStream​(long eventsPerSecond,
                                                      long initialDelayMillis)
        Returns a StreamSource that emits an ever-increasing sequence of Long numbers with native timestamps that are exactly the same amount of time apart, as specified by the supplied eventsPerSecond parameter. The source is distributed and suitable for high-throughput performance testing. It emits the events at the maximum possible speed, constrained by the invariant that it will never emit an event whose timestamp is in the future.

        The emission of events is distributed across the parallel processors in a round-robin fashion: processor 0 emits the first event, processor 1 the second one, and so on. There is no coordination that would prevent processor 1 from emitting its event before processor 0, though, so this only applies to the event timestamps.

        Use the initialDelayMillis parameter to give enough time to the Jet cluster to initialize the job on the whole cluster before the time of the first event arrives, so that there is no initial flood of events from the past. The point of reference is the moment at which the coordinator node creates the job's execution plan, before sending it out to the rest of the cluster.

        This source is not fault-tolerant. The sequence will be reset once a job is restarted.

        Note: A clock skew between any two cluster members may result in an artificial increase of latency.

        Parameters:
        eventsPerSecond - the desired event rate
        initialDelayMillis - initial delay in milliseconds before emitting values
        Since:
        Jet 4.3