Class JetConfig
- Since:
- Jet 3.0
-
Field Summary
Modifier and TypeFieldDescriptionstatic final int
The default value of thebackup-count
static final int
The default value of theflow-control period
. -
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionboolean
int
Returns thenumber of backups
used for job metadata and snapshots.int
Returns the number of cooperative execution threads.Returns the default DAG edge configuration.int
Returns theflow-control period
in milliseconds.Deprecated.long
Returns the maximum number of records that can be accumulated by any singleProcessor
instance.long
Returns the scale-up delay, seesetScaleUpDelayMillis(long)
.int
hashCode()
boolean
Returns if Jet is enabledboolean
Returns if lossless restart is enabled, seesetLosslessRestartEnabled(boolean)
.boolean
Returns if uploading resources when submitting the job enabledsetBackupCount
(int newBackupCount) Sets the number of backups that Jet will maintain for the job metadata and snapshots.setCooperativeThreadCount
(int size) Sets the number of threads each cluster member will use to execute Jet jobs.setDefaultEdgeConfig
(EdgeConfig defaultEdgeConfig) Sets the configuration object that specifies the defaults to use for a DAG edge configuration.setEnabled
(boolean enabled) Sets if Jet is enabledsetFlowControlPeriodMs
(int flowControlPeriodMs) While executing a Jet job there is the issue of regulating the rate at which one member of the cluster sends data to another member.setInstanceConfig
(InstanceConfig instanceConfig) Deprecated.The fields fromInstanceConfig
class were moved toJetConfig
class.setLosslessRestartEnabled
(boolean enabled) Sets whether lossless job restart is enabled for the node.setMaxProcessorAccumulatedRecords
(long maxProcessorAccumulatedRecords) Sets the maximum number of records that can be accumulated by any singleProcessor
instance.setResourceUploadEnabled
(boolean resourceUploadEnabled) Sets if uploading resources when submitting the job enabledsetScaleUpDelayMillis
(long millis) Sets the delay after which auto-scaled jobs will restart if a new member is added to the cluster.toString()
-
Field Details
-
DEFAULT_FLOW_CONTROL_PERIOD_MS
public static final int DEFAULT_FLOW_CONTROL_PERIOD_MSThe default value of theflow-control period
.- See Also:
-
DEFAULT_BACKUP_COUNT
public static final int DEFAULT_BACKUP_COUNTThe default value of thebackup-count
- See Also:
-
-
Constructor Details
-
JetConfig
public JetConfig()Creates a new, emptyJetConfig
with the default configuration. Doesn't consider any configuration XML files.
-
-
Method Details
-
setCooperativeThreadCount
Sets the number of threads each cluster member will use to execute Jet jobs. This refers only to threads executing cooperative processors; each blocking processor is assigned its own thread.- Since:
- 5.0
-
getCooperativeThreadCount
public int getCooperativeThreadCount()Returns the number of cooperative execution threads.- Since:
- 5.0
-
setFlowControlPeriodMs
While executing a Jet job there is the issue of regulating the rate at which one member of the cluster sends data to another member. The receiver will regularly report to each sender how much more data it is allowed to send over a given DAG edge. This method sets the length (in milliseconds) of the interval between flow-control ("ack") packets.- Since:
- 5.0
-
getFlowControlPeriodMs
public int getFlowControlPeriodMs()Returns theflow-control period
in milliseconds.- Since:
- 5.0
-
setBackupCount
Sets the number of backups that Jet will maintain for the job metadata and snapshots. Each backup is on another cluster member; all backup write operations must complete before the overall write operation can complete. The maximum allowed number of backups is 6 and the default is 1.For example, if you set the backup count to 2, Jet will replicate all the job metadata and snapshot data to two other members. If one or two members of the cluster fail, Jet can recover the data from the snapshot and continue executing the job on the remaining members without loss.
- Since:
- 5.0
-
getBackupCount
public int getBackupCount()Returns thenumber of backups
used for job metadata and snapshots.- Since:
- 5.0
-
setScaleUpDelayMillis
Sets the delay after which auto-scaled jobs will restart if a new member is added to the cluster. The default is 10 seconds. Has no effect on jobs with auto scaling disabled.- Parameters:
millis
- the delay, in milliseconds- Returns:
- this instance for fluent API
- Since:
- 5.0
-
getScaleUpDelayMillis
public long getScaleUpDelayMillis()Returns the scale-up delay, seesetScaleUpDelayMillis(long)
.- Since:
- 5.0
-
setLosslessRestartEnabled
Sets whether lossless job restart is enabled for the node. With lossless restart you can restart the whole cluster without losing the jobs and their state. The feature is implemented on top of the Hot Restart feature of Hazelcast IMDG which persists the data to disk.If enabled, you have to also configure Hot Restart:
JetConfig jetConfig = new JetConfig(); jetConfig.getInstanceConfig().setLosslessRestartEnabled(true); jetConfig.getHazelcastConfig().getHotRestartPersistenceConfig() .setEnabled(true) .setBaseDir(new File("/mnt/hot-restart")) .setParallelism(2);
Note: the snapshots exported using
Job.exportSnapshot(java.lang.String)
will also have Hot Restart storage enabled.Feature is disabled by default. If you enable this option in open-source Hazelcast Jet, the member will fail to start, you need Jet Enterprise to run it and obtain a license from Hazelcast.
- Since:
- 5.0
-
isLosslessRestartEnabled
public boolean isLosslessRestartEnabled()Returns if lossless restart is enabled, seesetLosslessRestartEnabled(boolean)
.- Since:
- 5.0
-
setMaxProcessorAccumulatedRecords
Sets the maximum number of records that can be accumulated by any singleProcessor
instance.Operations like grouping, sorting or joining require certain amount of records to be accumulated before they can proceed. You can set this option to reduce the probability of
OutOfMemoryError
.This option applies to each
Processor
instance separately, hence the effective limit of records accumulated by each cluster member is influenced by the vertex'slocalParallelism
and the number of jobs in the cluster.Currently,
maxProcessorAccumulatedRecords
limits:- number of items sorted by the sort operation
- number of distinct keys accumulated by aggregation operations
- number of keys in all incomplete frames in window aggregation
- number of entries in the hash-join lookup tables
- number of buffered rows in the stream-to-stream SQL join
- number of entries in stateful transforms
- number of distinct items in distinct operation
Note: the limit does not apply to streaming aggregations.
The default value is
Long.MAX_VALUE
.- Since:
- 5.0
-
getMaxProcessorAccumulatedRecords
public long getMaxProcessorAccumulatedRecords()Returns the maximum number of records that can be accumulated by any singleProcessor
instance.- Since:
- 5.0
-
isEnabled
public boolean isEnabled()Returns if Jet is enabled -
setEnabled
Sets if Jet is enabled -
isResourceUploadEnabled
public boolean isResourceUploadEnabled()Returns if uploading resources when submitting the job enabled -
setResourceUploadEnabled
Sets if uploading resources when submitting the job enabled -
getInstanceConfig
Deprecated.The fields fromInstanceConfig
class were moved toJetConfig
class. Get the fields directly fromJetConfig
.Returns the Jet instance config. -
setInstanceConfig
Deprecated.The fields fromInstanceConfig
class were moved toJetConfig
class. Set the fields directly onJetConfig
.Sets the Jet instance config. -
getDefaultEdgeConfig
Returns the default DAG edge configuration. -
setDefaultEdgeConfig
Sets the configuration object that specifies the defaults to use for a DAG edge configuration. -
equals
-
hashCode
public int hashCode() -
toString
-
InstanceConfig
class were moved toJetConfig
class.