Package com.hazelcast.jet.pipeline.test

Class TestSources

java.lang.Object
com.hazelcast.jet.pipeline.test.TestSources
@EvolvingApi public final class TestSources extends Object
Contains factory methods for various mock sources which can be used for pipeline testing and development.
Since:
Jet 3.2

  • Method Details

    • items

      @Nonnull public static <T> BatchSource<T> items(@Nonnull Iterable<? extends T> items)
      Returns a batch source which iterates through the supplied iterable and then terminates. The source is non-distributed.
      Since:
      Jet 3.2

    • items

      @Nonnull public static <T> BatchSource<T> items(@Nonnull T... items)
      Returns a batch source which iterates through the supplied items and then terminates. The source is non-distributed.
      Since:
      Jet 3.2

    • itemsDistributed

      @Nonnull public static <T> BatchSource<T> itemsDistributed(@Nonnull Iterable<? extends T> items)
      Returns a batch source which iterates through the supplied iterable and then terminates. The source is distributed - a slice of the items is emitted on each member with local parallelism of 1.
      Since:
      Jet 4.4

    • itemsDistributed

      @Nonnull public static <T> BatchSource<T> itemsDistributed(@Nonnull T... items)
      Returns a batch source which iterates through the supplied items and then terminates. The source is distributed - a slice of the items is emitted on each member with local parallelism of 1.
      Since:
      Jet 4.4

    • itemStream

      @EvolvingApi @Nonnull public static StreamSource<SimpleEvent> itemStream(int itemsPerSecond)
      Returns a streaming source that generates events of type SimpleEvent at the specified rate.

      The source supports native timestamps. The timestamp is the current system time at the moment they are generated. The source is not distributed and all the items are generated on the same node. This source is not fault-tolerant. The sequence will be reset once a job is restarted.

      Note: There is no absolute guarantee that the actual rate of emitted items will match the supplied value. It is ensured that no emitted event's timestamp will be in the future.

      Parameters:
      itemsPerSecond - how many items should be emitted each second
      Since:
      Jet 3.2

    • itemStream

      @EvolvingApi @Nonnull public static <T> StreamSource<T> itemStream(int itemsPerSecond, @Nonnull GeneratorFunction<? extends T> generatorFn)
      Returns a streaming source that generates events created by the generatorFn at the specified rate.

      The source supports native timestamps. The timestamp is the current system time at the moment they are generated. The source is not distributed and all the items are generated on the same node. This source is not fault-tolerant. The sequence will be reset once a job is restarted.

      Note: There is no absolute guarantee that the actual rate of emitted items will match the supplied value. It is ensured that no emitted event's timestamp will be in the future.

      Parameters:
      itemsPerSecond - how many items should be emitted each second
      generatorFn - a function which takes the timestamp and the sequence of the generated item and maps it to the desired type
      Since:
      Jet 3.2

    • longStream

      @Nonnull public static StreamSource<Long> longStream(long eventsPerSecond, long initialDelayMillis)
      Returns a StreamSource that emits an ever-increasing sequence of Long numbers with native timestamps that are exactly the same amount of time apart, as specified by the supplied eventsPerSecond parameter. The source is distributed and suitable for high-throughput performance testing. It emits the events at the maximum possible speed, constrained by the invariant that it will never emit an event whose timestamp is in the future.

      The emission of events is distributed across the parallel processors in a round-robin fashion: processor 0 emits the first event, processor 1 the second one, and so on. There is no coordination that would prevent processor 1 from emitting its event before processor 0, though, so this only applies to the event timestamps.

      Use the initialDelayMillis parameter to give enough time to the Jet cluster to initialize the job on the whole cluster before the time of the first event arrives, so that there is no initial flood of events from the past. The point of reference is the moment at which the coordinator node creates the job's execution plan, before sending it out to the rest of the cluster.

      This source is not fault-tolerant. The sequence will be reset once a job is restarted.

      Note: A clock skew between any two cluster members may result in an artificial increase of latency.

      Parameters:
      eventsPerSecond - the desired event rate
      initialDelayMillis - initial delay in milliseconds before emitting values
      Since:
      Jet 4.3