You can configure Hazelcast Jet either programmatically or declaratively (XML).
Programmatic configuration is the simplest way to configure Jet. For example, the following will configure Jet to use only two threads for cooperative execution:
JetConfig config = new JetConfig(); config.getInstanceConfig().setCooperativeThreadCount(2); JetInstance jet = Jet.newJetInstance(config);
Any XML configuration files that might be present will be ignored when programmatic configuration is used.
It is also possible to configure Jet through XML files when a
JetInstance is created without any explicit
JetConfig file. Jet will
look for a configuration file in the following order:
- Check the system property
hazelcast.jet.config. If the value is set, and starts with
classpath:, then it will be treated as a classpath resource. Otherwise, it will be treated as a file reference.
- Check for the presence of
hazelcast-jet.xmlin the working directory.
- Check for the presence of
hazelcast-jet.xmlin the classpath.
- If all the above checks fail, then the default XML configuration will be loaded.
An example configuration looks like the following:
<hazelcast-jet xsi:schemaLocation="http://www.hazelcast.com/schema/jet-config hazelcast-jet-config-0.3.xsd" xmlns="http://www.hazelcast.com/schema/jet-config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <instance> <!-- number of threads to use for DAG execution --> <cooperative-thread-count>8</cooperative-thread-count> <!-- frequency of flow control packets, in milliseconds --> <flow-control-period>100</flow-control-period> <!-- working directory to use for placing temporary files --> <temp-dir>/var/tmp/jet</temp-dir> </instance> <properties> <property name="custom.property">custom property</property> </properties> <edge-defaults> <!-- number of available slots for each concurrent queue between two vertices --> <queue-size>1024</queue-size> <!-- number of slots in each outbox's bucket --> <outbox-capacity>2048</outbox-capacity> <!-- maximum packet size in bytes, only applies to distributed edges --> <packet-size-limit>16384</packet-size-limit> <!-- target receive window size multiplier, only applies to distributed edges --> <receive-window-multiplier>3</receive-window-multiplier> </edge-defaults> <!-- custom properties which can be read within a ProcessorSupplier --> </hazelcast-jet>
The following table lists the configuration elements for Hazelcast Jet:
|Cooperative Thread Count||Maximum number of cooperative threads to be used for execution of jobs.||
|Temp Directory||Directory where temporary files will be placed, such as JAR files submitted by clients.||Jet will create a temp directory, which will be deleted on exit.|
|Flow Control Period||While executing a Jet job there is the issue of regulating the rate at which one member of the cluster sends data to another member. The receiver will regularly report to each sender how much more data it is allowed to send over a given DAG edge. This option sets the length (in milliseconds) of the interval between flow-control packets.||100ms|
|Edge Defaults||The default values to be used for all edges.||Please see the section on Tuning Edges.|
Configuring Underlying Hazelcast Instance
Each Jet member or client, will have a respective underlying Hazelcast member or client. Please refer to the Hazelcast Reference Manual for specific configuration options for Hazelcast IMDG.
The underlying Hazelcast IMDG member can be configured as follows:
JetConfig jetConfig = new JetConfig(); jetConfig.getHazelcastConfig().getGroupConfig().setName("test"); JetInstance jet = Jet.newJetInstance(jetConfig);
The underlying Hazelcast IMDG client can be configured as follows:
ClientConfig clientConfig = new ClientConfig(); clientConfig.getGroupConfig().setName("test"); JetInstance jet = Jet.newJetClient(clientConfig);
The underlying Hazelcast IMDG configuration can also be updated declaratively. Please refer to the Hazelcast Reference Manual for information on how to do this.