com.hazelcast.jet.config

Class EdgeConfig

java.lang.Object

com.hazelcast.jet.config.EdgeConfig

All Implemented Interfaces:

Serializable

public class EdgeConfig
extends Object
implements Serializable

A configuration object for a DAG Edge that holds fine-tuning parameters that influence its performance characteristics.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_PACKET_SIZE_LIMIT The default packet size limit.
static int	DEFAULT_QUEUE_SIZE The default size of the concurrent queues connecting processors within a cluster member.
static int	DEFAULT_RECEIVE_WINDOW_MULTIPLIER The default receive window multiplier.

Constructor Summary

Constructors

Constructor and Description
EdgeConfig()

Method Summary

Modifier and Type	Method and Description
int	getPacketSizeLimit() Returns the limit on the network packet size, in bytes
int	getQueueSize() Returns the size of the SPSC queues used to implement this edge.
int	getReceiveWindowMultiplier()
EdgeConfig	setPacketSizeLimit(int packetSizeLimit) For a distributed edge, data is sent to a remote member via Hazelcast network packets.
EdgeConfig	setQueueSize(int queueSize) Sets the capacity of processor-to-processor concurrent queues.
EdgeConfig	setReceiveWindowMultiplier(int receiveWindowMultiplier) Sets the scaling factor used by the adaptive receive window sizing function.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_QUEUE_SIZE

public static final int DEFAULT_QUEUE_SIZE

The default size of the concurrent queues connecting processors within a cluster member.

See Also:

Constant Field Values

DEFAULT_RECEIVE_WINDOW_MULTIPLIER

public static final int DEFAULT_RECEIVE_WINDOW_MULTIPLIER

The default receive window multiplier.

See Also:

Constant Field Values

DEFAULT_PACKET_SIZE_LIMIT

public static final int DEFAULT_PACKET_SIZE_LIMIT

The default packet size limit.

See Also:

Constant Field Values

Constructor Detail

EdgeConfig

public EdgeConfig()

Method Detail

setQueueSize

public EdgeConfig setQueueSize(int queueSize)

Sets the capacity of processor-to-processor concurrent queues.

When data needs to travel between two processors on the same cluster member, it is sent over a concurrent single-producer, single-consumer (SPSC) queue of fixed capacity.

Since there are several processors executing the logic of each vertex, and since the queues are SPSC, there will be senderParallelism * receiverParallelism queues representing the edge on each member. Care should be taken to strike a balance between performance and memory usage. The default of 1024 is a good size for simple DAGs and moderate parallelism, but the optimum can be detemined only by experiment.

getQueueSize

public int getQueueSize()

Returns the size of the SPSC queues used to implement this edge.

setReceiveWindowMultiplier

public EdgeConfig setReceiveWindowMultiplier(int receiveWindowMultiplier)

Sets the scaling factor used by the adaptive receive window sizing function.

For each distributed edge the receiving member regularly sends flow-control ("ack") packets to its sender which prevent it from sending too much data and overflowing the buffers. The sender is allowed to send the data one receive window further than the last acknowledged byte and the receive window is sized in proportion to the rate of processing at the receiver.

Ack packets are sent in regular intervals and the receive window multiplier sets the factor of the linear relationship between the amount of data processed within one such interval and the size of the receive window.

To put it another way, let us define an ackworth as the amount of data processed between two consecutive ack packets. The receive window multiplier determines the number of ackworths the sender can be ahead of the last acked byte.

The default value is 3. This setting has no effect on a non-distributed edge.

getReceiveWindowMultiplier

public int getReceiveWindowMultiplier()

Returns:

the receive window multiplier

setPacketSizeLimit

public EdgeConfig setPacketSizeLimit(int packetSizeLimit)

For a distributed edge, data is sent to a remote member via Hazelcast network packets. Each packet is dedicated to the data of a single edge, but may contain any number of data items. This setting limits the size of the packet in bytes. Packets should be large enough to drown out any fixed overheads, but small enough to allow good interleaving with other packets.

Note that a single item cannot straddle packets, therefore the maximum packet size can exceed the value configured here by the size of a single data item.

The default value is 16384. This setting has no effect on a non-distributed edge.

getPacketSizeLimit

public int getPacketSizeLimit()

Returns the limit on the network packet size, in bytes