Package com.hazelcast.jet.config

Class EdgeConfig

java.lang.Object
com.hazelcast.jet.config.EdgeConfig
All Implemented Interfaces:
DataSerializable, IdentifiedDataSerializable
public class EdgeConfig extends Object implements IdentifiedDataSerializable
A configuration object for a DAG Edge that holds fine-tuning parameters that influence its performance characteristics.
Since:
Jet 3.0

  • Field Details

  • Constructor Details

    • EdgeConfig

      public EdgeConfig()

  • Method Details

    • setQueueSize

      public EdgeConfig setQueueSize(int queueSize)
      Sets the capacity of processor-to-processor concurrent queues. The value is rounded upwards to the next power of 2.

      When data needs to travel between two processors on the same cluster member, Jet sends it over a concurrent single-producer, single-consumer (SPSC) queue of fixed capacity.

      Since there are several processors executing the logic of each vertex, and since the queues are SPSC, there will be senderParallelism * receiverParallelism queues representing the edge on each member. The edge capacity should strike a balance between performance and memory usage. The default of 1024 is a good size for simple DAGs and moderate parallelism.

      Returns:
      this instance for fluent API

    • getQueueSize

      public int getQueueSize()
      Returns the size of the SPSC queues used to implement this edge. The real queue size will be rounded upwards to the next power of 2.

    • setReceiveWindowMultiplier

      public EdgeConfig setReceiveWindowMultiplier(int receiveWindowMultiplier)
      Sets the scaling factor used by the adaptive receive window sizing function.

      For each distributed edge the receiving member regularly sends flow-control ("ack") packets to its sender which prevent it from sending too much data and overflowing the buffers. The sender is allowed to send the data one receive window further than the last acknowledged byte and the receive window is sized in proportion to the rate of processing at the receiver.

      Ack packets are sent in regular intervals and the receive window multiplier sets the factor of the linear relationship between the amount of data processed within one such interval and the size of the receive window.

      To put it another way, let us define an ackworth as the amount of data processed between two consecutive ack packets. The receive window multiplier determines the number of ackworths the sender can be ahead of the last acked byte.

      The default value is 3. This setting has no effect on a non-distributed edge.

      Returns:
      this instance for fluent API

    • getReceiveWindowMultiplier

      public int getReceiveWindowMultiplier()
      Returns:
      the receive window multiplier

    • setPacketSizeLimit

      public EdgeConfig setPacketSizeLimit(int packetSizeLimit)
      For a distributed edge, data is sent to a remote member via Hazelcast network packets. Each packet is dedicated to the data of a single edge, but may contain any number of data items. This setting limits the size of the packet in bytes. Packets should be large enough to drown out any fixed overheads, but small enough to allow good interleaving with other packets.

      Note that a single item cannot straddle packets, therefore the maximum packet size can exceed the value configured here by the size of a single data item.

      The default value is 16384. This setting has no effect on a non-distributed edge.

      Returns:
      this instance for fluent API

    • getPacketSizeLimit

      public int getPacketSizeLimit()
      Returns the limit on the network packet size, in bytes

    • getFactoryId

      public int getFactoryId()
      Description copied from interface: IdentifiedDataSerializable
      Returns DataSerializableFactory factory ID for this class.
      Specified by:
      getFactoryId in interface IdentifiedDataSerializable
      Returns:
      factory ID

    • getClassId

      public int getClassId()
      Description copied from interface: IdentifiedDataSerializable
      Returns type identifier for this class. It should be unique per DataSerializableFactory.
      Specified by:
      getClassId in interface IdentifiedDataSerializable
      Returns:
      type ID

    • writeData

      public void writeData(ObjectDataOutput out) throws IOException
      Description copied from interface: DataSerializable
      Writes object fields to output stream
      Specified by:
      writeData in interface DataSerializable
      Parameters:
      out - output
      Throws:
      IOException - if an I/O error occurs. In particular, an IOException may be thrown if the output stream has been closed.

    • readData

      public void readData(ObjectDataInput in) throws IOException
      Description copied from interface: DataSerializable
      Reads fields from the input stream
      Specified by:
      readData in interface DataSerializable
      Parameters:
      in - input
      Throws:
      IOException - if an I/O error occurs. In particular, an IOException may be thrown if the input stream has been closed.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object