com.hazelcast.jet.config

Class EdgeConfig

java.lang.Object

com.hazelcast.jet.config.EdgeConfig

All Implemented Interfaces:

DataSerializable, IdentifiedDataSerializable

public class EdgeConfig
extends Object
implements IdentifiedDataSerializable

A configuration object for a DAG Edge that holds fine-tuning parameters that influence its performance characteristics.

Since:

Jet 3.0

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_PACKET_SIZE_LIMIT The default packet size limit.
static int	DEFAULT_QUEUE_SIZE The default size of the concurrent queues connecting processors within a cluster member.
static int	DEFAULT_RECEIVE_WINDOW_MULTIPLIER The default receive window multiplier.

Constructor Summary

Constructors

Constructor and Description
EdgeConfig()

Method Summary

Modifier and Type	Method and Description
boolean	equals(Object o)
int	getClassId() Returns type identifier for this class.
int	getFactoryId() Returns DataSerializableFactory factory ID for this class.
int	getPacketSizeLimit() Returns the limit on the network packet size, in bytes
int	getQueueSize() Returns the size of the SPSC queues used to implement this edge.
int	getReceiveWindowMultiplier()
int	hashCode()
void	readData(ObjectDataInput in) Reads fields from the input stream
EdgeConfig	setPacketSizeLimit(int packetSizeLimit) For a distributed edge, data is sent to a remote member via Hazelcast network packets.
EdgeConfig	setQueueSize(int queueSize) Sets the capacity of processor-to-processor concurrent queues.
EdgeConfig	setReceiveWindowMultiplier(int receiveWindowMultiplier) Sets the scaling factor used by the adaptive receive window sizing function.
String	toString()
void	writeData(ObjectDataOutput out) Writes object fields to output stream

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_QUEUE_SIZE

public static final int DEFAULT_QUEUE_SIZE

The default size of the concurrent queues connecting processors within a cluster member.

See Also:

Constant Field Values

DEFAULT_RECEIVE_WINDOW_MULTIPLIER

public static final int DEFAULT_RECEIVE_WINDOW_MULTIPLIER

The default receive window multiplier.

See Also:

Constant Field Values

DEFAULT_PACKET_SIZE_LIMIT

public static final int DEFAULT_PACKET_SIZE_LIMIT

The default packet size limit.

See Also:

Constant Field Values

Constructor Detail

EdgeConfig

public EdgeConfig()

Method Detail

setQueueSize

public EdgeConfig setQueueSize(int queueSize)

Sets the capacity of processor-to-processor concurrent queues. The value is rounded upwards to the next power of 2.

When data needs to travel between two processors on the same cluster member, Jet sends it over a concurrent single-producer, single-consumer (SPSC) queue of fixed capacity.

Since there are several processors executing the logic of each vertex, and since the queues are SPSC, there will be senderParallelism * receiverParallelism queues representing the edge on each member. The edge capacity should strike a balance between performance and memory usage. The default of 1024 is a good size for simple DAGs and moderate parallelism.

Returns:

this instance for fluent API

getQueueSize

public int getQueueSize()

Returns the size of the SPSC queues used to implement this edge. The real queue size will be rounded upwards to the next power of 2.

setReceiveWindowMultiplier

public EdgeConfig setReceiveWindowMultiplier(int receiveWindowMultiplier)

Sets the scaling factor used by the adaptive receive window sizing function.

For each distributed edge the receiving member regularly sends flow-control ("ack") packets to its sender which prevent it from sending too much data and overflowing the buffers. The sender is allowed to send the data one receive window further than the last acknowledged byte and the receive window is sized in proportion to the rate of processing at the receiver.

Ack packets are sent in regular intervals and the receive window multiplier sets the factor of the linear relationship between the amount of data processed within one such interval and the size of the receive window.

To put it another way, let us define an ackworth as the amount of data processed between two consecutive ack packets. The receive window multiplier determines the number of ackworths the sender can be ahead of the last acked byte.

The default value is 3. This setting has no effect on a non-distributed edge.

Returns:

this instance for fluent API

getReceiveWindowMultiplier

public int getReceiveWindowMultiplier()

Returns:

the receive window multiplier

setPacketSizeLimit

public EdgeConfig setPacketSizeLimit(int packetSizeLimit)

For a distributed edge, data is sent to a remote member via Hazelcast network packets. Each packet is dedicated to the data of a single edge, but may contain any number of data items. This setting limits the size of the packet in bytes. Packets should be large enough to drown out any fixed overheads, but small enough to allow good interleaving with other packets.

Note that a single item cannot straddle packets, therefore the maximum packet size can exceed the value configured here by the size of a single data item.

The default value is 16384. This setting has no effect on a non-distributed edge.

Returns:

this instance for fluent API

getPacketSizeLimit

public int getPacketSizeLimit()

Returns the limit on the network packet size, in bytes

getFactoryId

public int getFactoryId()

Description copied from interface: IdentifiedDataSerializable

Returns DataSerializableFactory factory ID for this class.

Specified by:

getFactoryId in interface IdentifiedDataSerializable

Returns:

factory ID

getClassId

public int getClassId()

Description copied from interface: IdentifiedDataSerializable

Returns type identifier for this class. It should be unique per DataSerializableFactory.

Specified by:

getClassId in interface IdentifiedDataSerializable

Returns:

type ID

writeData

public void writeData(ObjectDataOutput out)
               throws IOException

Description copied from interface: DataSerializable

Writes object fields to output stream

Specified by:

writeData in interface DataSerializable

Parameters:

out - output

Throws:

IOException - if an I/O error occurs. In particular, an IOException may be thrown if the output stream has been closed.

readData

public void readData(ObjectDataInput in)
              throws IOException

Description copied from interface: DataSerializable

Reads fields from the input stream

Specified by:

readData in interface DataSerializable

Parameters:

in - input

Throws:

IOException - if an I/O error occurs. In particular, an IOException may be thrown if the input stream has been closed.

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

toString

public String toString()

Overrides:

toString in class Object