Package com.hazelcast.jet.core

Class Vertex

java.lang.Object
com.hazelcast.jet.core.Vertex
All Implemented Interfaces:
DataSerializable, IdentifiedDataSerializable
public class Vertex extends Object implements IdentifiedDataSerializable
Represents a unit of data processing in a Jet computation job. Conceptually, a vertex receives data items over its inbound edges and pushes data items to its outbound edges. Practically, a single vertex is represented by a set of instances of Processor. The localParallelism property determines the number of processor instances running on each cluster member.

Each processor is assigned a set of partition IDs it is responsible for. When an inbound edge is partitioned, the processor will receive only those data items whose partition ID it is responsible for. For data traveling over a partitioned edge which is also distributed, the whole cluster contains a single unique processor instance responsible for any given partition ID. For non-distributed edges, the processor is unique only within a member and each member has its own processor for any given partition ID. Finally, there is a guarantee of collation across all the partitioned edges impinging on a vertex: within each member, all the data with a given partition ID is received by the same processor.

A vertex is uniquely identified in a DAG by its name.

Since:
Jet 3.0

  • Field Details

  • Constructor Details

    • Vertex

      public Vertex(@Nonnull String name, @Nonnull SupplierEx<? extends Processor> processorSupplier)
      Creates a vertex from a Supplier<Processor>.

      This is useful for vertices where all the Processor instances will be instantiated the same way.

      NOTE: this constructor should not be abused with a stateful implementation which produces a different processor each time. In such a case the full ProcessorSupplier type should be implemented.

      Parameters:
      name - the unique name of the vertex. This name identifies the vertex in the snapshot
      processorSupplier - the simple, parameterless supplier of Processor instances

    • Vertex

      public Vertex(@Nonnull String name, @Nonnull ProcessorSupplier processorSupplier)
      Creates a vertex from a ProcessorSupplier.
      Parameters:
      name - the unique name of the vertex. This name identifies the vertex in the snapshot
      processorSupplier - the supplier of Processor instances which will be used on all members

    • Vertex

      public Vertex(@Nonnull String name, @Nonnull ProcessorMetaSupplier metaSupplier)
      Creates a vertex from a ProcessorMetaSupplier.
      Parameters:
      name - the unique name of the vertex. This name identifies the vertex in the snapshot
      metaSupplier - the meta-supplier of ProcessorSuppliers for each member

  • Method Details

    • checkLocalParallelism

      public static int checkLocalParallelism(int parallelism)
      Says whether the given integer is valid as the value of localParallelism.

    • determineLocalParallelism

      public int determineLocalParallelism(int defaultParallelism)
      Determines the local parallelism value for the vertex by looking at its local parallelism and meta supplier's preferred local parallelism.

      If none of them is set, returns the provided default parallelism

    • localParallelism

      @Nonnull public Vertex localParallelism(int localParallelism)
      Sets the number of processors corresponding to this vertex that will be created on each member.

      If the value is -1, Jet will determine the vertex's local parallelism during job initialization from the global default and processor meta-supplier's preferred value.

    • getLocalParallelism

      public int getLocalParallelism()
      Returns the number of processors corresponding to this vertex that will be created on each member. A value of -1 means that this property is not set; in that case the default configured on the Jet instance will be used.

    • getName

      @Nonnull public String getName()
      Returns the name of this vertex.

    • getMetaSupplier

      @Nonnull public ProcessorMetaSupplier getMetaSupplier()
      Returns this vertex's meta-supplier of processors.

    • updateMetaSupplier

      public void updateMetaSupplier(@Nonnull UnaryOperator<ProcessorMetaSupplier> updateFn)
      Applies the provided operator function to the current processor meta-supplier and replaces it with the one it returns. Typically used to decorate the existing meta-supplier.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • writeData

      public void writeData(@Nonnull ObjectDataOutput out) throws IOException
      Description copied from interface: DataSerializable
      Writes object fields to output stream
      Specified by:
      writeData in interface DataSerializable
      Parameters:
      out - output
      Throws:
      IOException - if an I/O error occurs. In particular, an IOException may be thrown if the output stream has been closed.

    • readData

      public void readData(@Nonnull ObjectDataInput in) throws IOException
      Description copied from interface: DataSerializable
      Reads fields from the input stream
      Specified by:
      readData in interface DataSerializable
      Parameters:
      in - input
      Throws:
      IOException - if an I/O error occurs. In particular, an IOException may be thrown if the input stream has been closed.

    • getFactoryId

      public int getFactoryId()
      Description copied from interface: IdentifiedDataSerializable
      Returns DataSerializableFactory factory ID for this class.
      Specified by:
      getFactoryId in interface IdentifiedDataSerializable
      Returns:
      factory ID

    • getClassId

      public int getClassId()
      Description copied from interface: IdentifiedDataSerializable
      Returns type identifier for this class. It should be unique per DataSerializableFactory.
      Specified by:
      getClassId in interface IdentifiedDataSerializable
      Returns:
      type ID