Class JobConfig
- java.lang.Object
-
- com.hazelcast.jet.config.JobConfig
-
- All Implemented Interfaces:
DataSerializable
,IdentifiedDataSerializable
public class JobConfig extends java.lang.Object implements IdentifiedDataSerializable
Contains the configuration specific to one Hazelcast Jet job.Serialized form of this class should not be changed due to backward compatibility for
JobResult
.- Since:
- Jet 3.0
- See Also:
DeltaJobConfig
-
-
Constructor Summary
Constructors Constructor Description JobConfig()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description JobConfig
addClass(java.lang.Class... classes)
Adds the given classes and recursively all their nested (inner & anonymous) classes to the Jet job's classpath.JobConfig
addClasspathResource(java.io.File file)
Adds a file that will be available as a resource on the Jet job's classpath.JobConfig
addClasspathResource(java.io.File file, java.lang.String id)
Adds a file that will be available as a resource on the Jet job's classpath.JobConfig
addClasspathResource(java.lang.String path)
Adds a file that will be available as a resource on the Jet job's classpath.JobConfig
addClasspathResource(java.lang.String path, java.lang.String id)
Adds a file that will be available as a resource on the Jet job's classpath.JobConfig
addClasspathResource(java.net.URL url)
Adds a resource that will be available on the Jet job's classpath.JobConfig
addClasspathResource(java.net.URL url, java.lang.String id)
Adds a resource that will be available on the Jet job's classpath.JobConfig
addCustomClasspath(java.lang.String name, java.lang.String path)
Adds custom classpath element to a stage with the given name.JobConfig
addCustomClasspaths(java.lang.String name, java.util.List<java.lang.String> paths)
Adds custom classpath elements to a stage with the given name.JobConfig
addJar(java.io.File file)
Adds a JAR whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code.JobConfig
addJar(java.lang.String path)
Adds a JAR whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code.JobConfig
addJar(java.net.URL url)
Adds a JAR whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code.JobConfig
addJarsInZip(java.io.File file)
Adds a ZIP file with JARs whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code.JobConfig
addJarsInZip(java.lang.String path)
Adds a ZIP file with JARs whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code.JobConfig
addJarsInZip(java.net.URL url)
Adds a ZIP file with JARs whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code.JobConfig
addPackage(java.lang.String... packages)
Adds recursively all the classes and resources in given packages to the Jet job's classpath.JobConfig
attachAll(java.util.Map<java.lang.String,java.io.File> idToFile)
Attaches all the files/directories in the supplied map, as if by callingattachDirectory(dir, id)
for every entry that resolves to a directory andattachFile(file, id)
for every entry that resolves to a regular file.JobConfig
attachDirectory(java.io.File file)
Adds the supplied directory to the list of files that will be available to the job while it's executing in the Jet cluster.JobConfig
attachDirectory(java.io.File file, java.lang.String id)
Adds the supplied directory to the list of files that will be available to the job while it's executing in the Jet cluster.JobConfig
attachDirectory(java.lang.String path)
Adds the directory identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster.JobConfig
attachDirectory(java.lang.String path, java.lang.String id)
Adds the directory identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster.JobConfig
attachDirectory(java.net.URL url)
Adds the directory identified by the supplied URL to the list of directories that will be available to the job while it's executing in the Jet cluster.JobConfig
attachDirectory(java.net.URL url, java.lang.String id)
Adds the directory identified by the supplied URL to the list of directories that will be available to the job while it's executing in the Jet cluster.JobConfig
attachFile(java.io.File file)
Adds the supplied file to the list of resources that will be available to the job while it's executing in the Jet cluster.JobConfig
attachFile(java.io.File file, java.lang.String id)
Adds the supplied file to the list of files that will be available to the job while it's executing in the Jet cluster.JobConfig
attachFile(java.lang.String path)
Adds the file identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster.JobConfig
attachFile(java.lang.String path, java.lang.String id)
Adds the file identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster.JobConfig
attachFile(java.net.URL url)
Adds the file identified by the supplied URL as a resource that will be available to the job while it's executing in the Jet cluster.JobConfig
attachFile(java.net.URL url, java.lang.String id)
Adds the file identified by the supplied URL to the list of resources that will be available to the job while it's executing in the Jet cluster.boolean
equals(java.lang.Object o)
<T> T
getArgument(java.lang.String key)
Returns the value to which the specified key is mapped, or null if there is no mapping for the key.int
getClassId()
Returns type identifier for this class.JobClassLoaderFactory
getClassLoaderFactory()
Returns the configuredJobClassLoaderFactory
.java.util.Map<java.lang.String,java.util.List<java.lang.String>>
getCustomClassPaths()
Returns configured custom classpath elements, SeeaddCustomClasspath(String, String)
andaddCustomClasspaths(String, List)
int
getFactoryId()
Returns DataSerializableFactory factory ID for this class.java.lang.String
getInitialSnapshotName()
Returns the configured initial snapshot name ornull
if no initial snapshot is configured.long
getMaxProcessorAccumulatedRecords()
Returns the maximum number of records that can be accumulated by any singleProcessor
instance in the context of the job.java.lang.String
getName()
Returns the name of the job ornull
if no name was given.ProcessingGuarantee
getProcessingGuarantee()
Returns the configuredprocessing guarantee
.java.util.Map<java.lang.String,ResourceConfig>
getResourceConfigs()
Returns all the registered resource configurations.java.util.Map<java.lang.String,java.lang.String>
getSerializerConfigs()
Returns all the registered serializer configurations.long
getSnapshotIntervalMillis()
Returns the configuredsnapshot interval
.long
getTimeoutMillis()
Returns maximum execution time for the job in milliseconds.int
hashCode()
boolean
isAutoScaling()
Returns whether auto-scaling is enabled, seesetAutoScaling(boolean)
.boolean
isMetricsEnabled()
Returns if metrics collection is enabled for the job.boolean
isSplitBrainProtectionEnabled()
Tells whethersplit brain protection
is enabled.boolean
isStoreMetricsAfterJobCompletion()
Returns whether metrics should be stored in the cluster after the job completes successfully.boolean
isSuspendOnFailure()
Returns whether the job will be suspended on failure, seesetSuspendOnFailure(boolean)
.void
lock()
Used to prevent further mutations the config after submitting it with a job execution.void
readData(ObjectDataInput in)
Reads fields from the input stream<T,S extends StreamSerializer<T>>
JobConfigregisterSerializer(java.lang.Class<T> clazz, java.lang.Class<S> serializerClass)
Registers the given serializer for the given class for the scope of the job.JobConfig
setArgument(java.lang.String key, java.lang.Object value)
Associates the specified value with the specified key.JobConfig
setAutoScaling(boolean enabled)
Sets whether Jet will scale the job up or down when a member is added or removed from the cluster.JobConfig
setClassLoaderFactory(JobClassLoaderFactory classLoaderFactory)
Sets a customJobClassLoaderFactory
that will be used to load job classes and resources on Jet members.JobConfig
setInitialSnapshotName(java.lang.String initialSnapshotName)
Sets the exported state snapshot name to restore the initial job state from.JobConfig
setMaxProcessorAccumulatedRecords(long maxProcessorAccumulatedRecords)
Sets the maximum number of records that can be accumulated by any singleProcessor
instance in the context of the job.JobConfig
setMetricsEnabled(boolean enabled)
Sets whether metrics collection should be enabled for the job.JobConfig
setName(java.lang.String name)
Sets the name of the job.JobConfig
setProcessingGuarantee(ProcessingGuarantee processingGuarantee)
Set theprocessing guarantee
for the job.JobConfig
setSnapshotIntervalMillis(long snapshotInterval)
Sets the snapshot interval in milliseconds — the interval between the completion of the previous snapshot and the start of a new one.JobConfig
setSplitBrainProtection(boolean isEnabled)
Configures the split brain protection feature.JobConfig
setStoreMetricsAfterJobCompletion(boolean storeMetricsAfterJobCompletion)
Sets whether metrics should be stored in the cluster after the job completes.JobConfig
setSuspendOnFailure(boolean suspendOnFailure)
Sets what happens if the job execution fails: If enabled, the job will be suspended.JobConfig
setTimeoutMillis(long timeoutMillis)
Sets the maximum execution time for the job in milliseconds.java.lang.String
toString()
void
writeData(ObjectDataOutput out)
Writes object fields to output stream
-
-
-
Method Detail
-
getName
@Nullable public java.lang.String getName()
Returns the name of the job ornull
if no name was given.
-
setName
@Nonnull public JobConfig setName(@Nullable java.lang.String name)
Sets the name of the job. There can be at most one active job in the cluster with particular name, however, the name can be reused after the previous job with that name completed or failed. SeeJetService.newJobIfAbsent(com.hazelcast.jet.core.DAG, com.hazelcast.jet.config.JobConfig)
. An active job is a job that is running, suspended or waiting to be run.The job name is printed in logs and is visible in Management Center.
The default value is
null
. Must be set tonull
for light jobs.- Returns:
this
instance for fluent API- Since:
- Jet 3.0
-
isSplitBrainProtectionEnabled
public boolean isSplitBrainProtectionEnabled()
Tells whethersplit brain protection
is enabled.
-
setSplitBrainProtection
@Nonnull public JobConfig setSplitBrainProtection(boolean isEnabled)
Configures the split brain protection feature. When enabled, Jet will restart the job after a topology change only if the cluster quorum is satisfied. The quorum value iscluster size at job submission time / 2 + 1
.The job can be restarted only if the size of the cluster after restart is at least the quorum value. Only one of the clusters formed due to a split-brain condition can satisfy the quorum. For example, if at the time of job submission the cluster size was 5 and a network partition causes two clusters with sizes 3 and 2 to form, the job will restart only on the cluster with size 3.
Adding new nodes to the cluster after starting the job may defeat this mechanism. For instance, if there are 5 members at submission time (i.e., the quorum value is 3) and later a new node joins, a split into two clusters of size 3 will allow the job to be restarted on both sides.
Split-brain protection is disabled by default.
If auto scaling is disabled and you manually
Job.resume()
the job, the job won't start executing until the quorum is met, but will remain in the resumed state.Ignored for light jobs.
- Returns:
this
instance for fluent API
-
isAutoScaling
public boolean isAutoScaling()
Returns whether auto-scaling is enabled, seesetAutoScaling(boolean)
.
-
setAutoScaling
public JobConfig setAutoScaling(boolean enabled)
Sets whether Jet will scale the job up or down when a member is added or removed from the cluster. Enabled by default. Ignored for light jobs.+--------------------------+-----------------------+----------------+ | Auto scaling | Member added | Member removed | +--------------------------+-----------------------+----------------+ | Enabled | restart (after delay) | restart | | Disabled - snapshots on | no action | suspend | | Disabled - snapshots off | no action | fail | +--------------------------+-----------------------+----------------+
- Returns:
this
instance for fluent API- See Also:
Configuring the scale-up delay
,Enabling/disabling snapshots
-
isSuspendOnFailure
public boolean isSuspendOnFailure()
Returns whether the job will be suspended on failure, seesetSuspendOnFailure(boolean)
.- Since:
- Jet 4.3
-
setSuspendOnFailure
public JobConfig setSuspendOnFailure(boolean suspendOnFailure)
Sets what happens if the job execution fails:- If enabled, the job will be suspended. It can later be resumed or upgraded and the computation state will be preserved.
- If disabled, the job will be terminated. The state snapshots will be deleted.
By default, it's disabled. Ignored for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.3
-
getProcessingGuarantee
@Nonnull public ProcessingGuarantee getProcessingGuarantee()
Returns the configuredprocessing guarantee
.
-
setProcessingGuarantee
@Nonnull public JobConfig setProcessingGuarantee(@Nonnull ProcessingGuarantee processingGuarantee)
Set theprocessing guarantee
for the job. When the processing guarantee is set to at-least-once or exactly-once, the snapshot interval can be configured viasetSnapshotIntervalMillis(long)
, otherwise it will default to 10 seconds.The default value is
ProcessingGuarantee.NONE
. Must be set toNONE
for light jobs.- Returns:
this
instance for fluent API
-
getSnapshotIntervalMillis
public long getSnapshotIntervalMillis()
Returns the configuredsnapshot interval
.
-
setSnapshotIntervalMillis
@Nonnull public JobConfig setSnapshotIntervalMillis(long snapshotInterval)
Sets the snapshot interval in milliseconds — the interval between the completion of the previous snapshot and the start of a new one. Must be set to a positive value. This setting is only relevant with at-least-once or exactly-once processing guarantees.Default value is set to 10 seconds.
- Returns:
this
instance for fluent API
-
addClass
@Nonnull public JobConfig addClass(@Nonnull java.lang.Class... classes)
Adds the given classes and recursively all their nested (inner & anonymous) classes to the Jet job's classpath. They will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. (An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.)See also
addJar(java.net.URL)
andaddClasspathResource(java.net.URL)
.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addPackage
@Nonnull public JobConfig addPackage(@Nonnull java.lang.String... packages)
Adds recursively all the classes and resources in given packages to the Jet job's classpath. They will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. (An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.)See also
addJar(java.net.URL)
andaddClasspathResource(java.net.URL)
.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.1
-
addJar
@Nonnull public JobConfig addJar(@Nonnull java.net.URL url)
Adds a JAR whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.This variant identifies the JAR with a URL, which must contain at least one path segment. The last path segment ("filename") will be used as the resource ID, so two JARs with the same filename will be in conflict.
Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addJar
@Nonnull public JobConfig addJar(@Nonnull java.io.File file)
Adds a JAR whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.This variant identifies the JAR with a
File
. The filename part of the path will be used as the resource ID, so two JARs with the same filename will be in conflict.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addJar
@Nonnull public JobConfig addJar(@Nonnull java.lang.String path)
Adds a JAR whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.This variant identifies the JAR with a path string. The filename part will be used as the resource ID, so two JARs with the same filename will be in conflict.
Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addJarsInZip
@Nonnull public JobConfig addJarsInZip(@Nonnull java.net.URL url)
Adds a ZIP file with JARs whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. (An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.)This variant identifies the ZIP file with a URL, which must contain at least one path segment. The last path segment ("filename") will be used as the resource ID, so two ZIPs with the same filename will be in conflict.
The ZIP file should contain only JARs. Any other files will be ignored.
Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
addJarsInZip
@Nonnull public JobConfig addJarsInZip(@Nonnull java.io.File file)
Adds a ZIP file with JARs whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. (An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.)This variant identifies the ZIP file with a
File
. The filename part will be used as the resource ID, so two ZIPs with the same filename will be in conflict.The ZIP file should contain only JARs. Any other files will be ignored.
Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
addJarsInZip
@Nonnull public JobConfig addJarsInZip(@Nonnull java.lang.String path)
Adds a ZIP file with JARs whose contents will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. (An important example is theIMap
data source, which can instantiate only the classes from the Jet instance's classpath.)This variant identifies the ZIP file with a path string. The filename part will be used as the resource ID, so two ZIPs with the same filename will be in conflict.
The ZIP file should contain only JARs. Any other files will be ignored.
Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
addClasspathResource
@Nonnull public JobConfig addClasspathResource(@Nonnull java.net.URL url)
Adds a resource that will be available on the Jet job's classpath. All the code attached to the underlying pipeline or DAG will have access to it.This variant identifies the resource with a URL, which must contain at least one path segment. The last path segment ("filename") will be used as the resource ID, so two resources with the same filename will be in conflict.
Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addClasspathResource
@Nonnull public JobConfig addClasspathResource(@Nonnull java.net.URL url, @Nonnull java.lang.String id)
Adds a resource that will be available on the Jet job's classpath. All the code attached to the underlying pipeline or DAG will have access to it. The suppliedid
becomes the path under which the resource is available from the class loader.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addClasspathResource
@Nonnull public JobConfig addClasspathResource(@Nonnull java.io.File file)
Adds a file that will be available as a resource on the Jet job's classpath. All the code attached to the underlying pipeline or DAG will have access to it. The file will reside in the root of the classpath, under its own filename. This means that two files with the same filename will be in conflict.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addClasspathResource
@Nonnull public JobConfig addClasspathResource(@Nonnull java.io.File file, @Nonnull java.lang.String id)
Adds a file that will be available as a resource on the Jet job's classpath. All the code attached to the underlying pipeline or DAG will have access to it. The suppliedid
becomes the path under which the resource is available from the class loader.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addClasspathResource
@Nonnull public JobConfig addClasspathResource(@Nonnull java.lang.String path)
Adds a file that will be available as a resource on the Jet job's classpath. All the code attached to the underlying pipeline or DAG will have access to it. It will reside in the root of the classpath, under its own filename. This means that two files with the same filename will be in conflict.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addClasspathResource
@Nonnull public JobConfig addClasspathResource(@Nonnull java.lang.String path, @Nonnull java.lang.String id)
Adds a file that will be available as a resource on the Jet job's classpath. All the code attached to the underlying pipeline or DAG will have access to it. The suppliedid
becomes the path under which the resource is available from the class loader.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
addCustomClasspath
@Nonnull @Beta public JobConfig addCustomClasspath(@Nonnull java.lang.String name, @Nonnull java.lang.String path)
Adds custom classpath element to a stage with the given name.BatchSource<String> source = ... JobConfig config = new JobConfig(); config.addCustomClasspath(source.name(), "hazelcast-client-3.12.13.jar");
- Parameters:
name
- name of the stage, must be unique for the whole pipeline (the stage name can be set viaStage.setName(String)
)path
- path to the jar relative to the `ext` directory- Returns:
this
instance for fluent API
-
addCustomClasspaths
@Nonnull @Beta public JobConfig addCustomClasspaths(@Nonnull java.lang.String name, @Nonnull java.util.List<java.lang.String> paths)
Adds custom classpath elements to a stage with the given name.BatchSource<String> source = ... JobConfig config = new JobConfig(); config.addCustomClasspaths(source.name(), jarList);
- Parameters:
name
- name of the stage, must be unique for the whole pipeline (the stage name can be set viaStage.setName(String)
)paths
- paths to the jar relative to the `ext` directory- Returns:
this
instance for fluent API
-
attachFile
@Nonnull public JobConfig attachFile(@Nonnull java.net.URL url)
Adds the file identified by the supplied URL as a resource that will be available to the job while it's executing in the Jet cluster. The resource's filename (the last path segment in the URL) becomes its ID, so two resources with the same filename will be in conflict.To retrieve the file from within the Jet job, call
ctx.attachedFile(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. The file will have the same name as the one supplied here, but it will be in a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachFile
@Nonnull public JobConfig attachFile(@Nonnull java.net.URL url, @Nonnull java.lang.String id)
Adds the file identified by the supplied URL to the list of resources that will be available to the job while it's executing in the Jet cluster. The file will be registered under the supplied ID.To retrieve the file from within the Jet job, call
ctx.attachedFile(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. The file will have the same name as the one supplied here, but it will be in a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachFile
@Nonnull public JobConfig attachFile(@Nonnull java.io.File file)
Adds the supplied file to the list of resources that will be available to the job while it's executing in the Jet cluster. The filename becomes the ID of the file, so two files with the same name will be in conflict.To retrieve the file from within the Jet job, call
ctx.attachedFile(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. The file will have the same name as the one supplied here, but it will be in a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachFile
@Nonnull public JobConfig attachFile(@Nonnull java.io.File file, @Nonnull java.lang.String id)
Adds the supplied file to the list of files that will be available to the job while it's executing in the Jet cluster. The file will be registered under the supplied ID.To retrieve the file from within the Jet job, call
ctx.attachedFile(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. The file will have the same name as the one supplied here, but it will be in a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachFile
@Nonnull public JobConfig attachFile(@Nonnull java.lang.String path)
Adds the file identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster. The filename becomes the ID of the file, so two files with the same name will be in conflict.To retrieve the file from within the Jet job, call
ctx.attachedFile(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. The file will have the same name as the one supplied here, but it will be in a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachFile
@Nonnull public JobConfig attachFile(@Nonnull java.lang.String path, @Nonnull java.lang.String id)
Adds the file identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster. The file will be registered under the supplied ID.To retrieve the file from within the Jet job, call
ctx.attachedFile(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. The file will have the same name as the one supplied here, but it will be in a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachDirectory
@Nonnull public JobConfig attachDirectory(@Nonnull java.net.URL url)
Adds the directory identified by the supplied URL to the list of directories that will be available to the job while it's executing in the Jet cluster. Directory name (the last path segment in the URL) becomes its ID, so two directories with the same name will be in conflict. Hidden files are ignored.To retrieve the directory from within the Jet job, call
ctx.attachedDirectory(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. It will be a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachDirectory
@Nonnull public JobConfig attachDirectory(@Nonnull java.net.URL url, @Nonnull java.lang.String id)
Adds the directory identified by the supplied URL to the list of directories that will be available to the job while it's executing in the Jet cluster. The directory will be registered under the supplied ID. Hidden files are ignored.To retrieve the directory from within the Jet job, call
ctx.attachedDirectory(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. It will be a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachDirectory
@Nonnull public JobConfig attachDirectory(@Nonnull java.lang.String path)
Adds the directory identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster. The directory name (the last path segment) becomes its ID, so two directories with the same name will be in conflict. Hidden files are ignored.To retrieve the directory from within the Jet job, call
ctx.attachedDirectory(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. It will be a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachDirectory
@Nonnull public JobConfig attachDirectory(@Nonnull java.lang.String path, @Nonnull java.lang.String id)
Adds the directory identified by the supplied pathname to the list of files that will be available to the job while it's executing in the Jet cluster. The directory will be registered under the supplied ID. Hidden files are ignored.To retrieve the directory from within the Jet job, call
ctx.attachedDirectory(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. It will be a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachDirectory
@Nonnull public JobConfig attachDirectory(@Nonnull java.io.File file)
Adds the supplied directory to the list of files that will be available to the job while it's executing in the Jet cluster. The directory name (the last path segment) becomes its ID, so two directories with the same name will be in conflict. Hidden files are ignored.To retrieve the directory from within the Jet job, call
ctx.attachedDirectory(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. It will be a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachDirectory
@Nonnull public JobConfig attachDirectory(@Nonnull java.io.File file, @Nonnull java.lang.String id)
Adds the supplied directory to the list of files that will be available to the job while it's executing in the Jet cluster. The directory will be registered under the supplied ID. Hidden files are ignored.To retrieve the directory from within the Jet job, call
ctx.attachedDirectory(id)
, wherectx
is theProcessorSupplier
context available, for example, toServiceFactory.createContextFn()
. It will be a temporary directory on the Jet server.Cannot be used for light jobs.
- Returns:
this
instance for fluent API- Since:
- Jet 4.0
-
attachAll
@Nonnull public JobConfig attachAll(@Nonnull java.util.Map<java.lang.String,java.io.File> idToFile)
Attaches all the files/directories in the supplied map, as if by callingattachDirectory(dir, id)
for every entry that resolves to a directory andattachFile(file, id)
for every entry that resolves to a regular file.Cannot be used for light jobs.
- Returns:
this
instance for fluent API
-
getResourceConfigs
@Nonnull @PrivateApi public java.util.Map<java.lang.String,ResourceConfig> getResourceConfigs()
Returns all the registered resource configurations.
-
getCustomClassPaths
public java.util.Map<java.lang.String,java.util.List<java.lang.String>> getCustomClassPaths()
Returns configured custom classpath elements, SeeaddCustomClasspath(String, String)
andaddCustomClasspaths(String, List)
-
registerSerializer
@Nonnull @EvolvingApi public <T,S extends StreamSerializer<T>> JobConfig registerSerializer(@Nonnull java.lang.Class<T> clazz, @Nonnull java.lang.Class<S> serializerClass)
Registers the given serializer for the given class for the scope of the job. It will be accessible to all the code attached to the underlying pipeline or DAG, but not to any other code. There are several serializer types you can register, see the light jobs.- Parameters:
clazz
- class to register serializer forserializerClass
- class of the serializer to be registered- Returns:
this
instance for fluent API- Since:
- Jet 4.1
-
getSerializerConfigs
@Nonnull @PrivateApi public java.util.Map<java.lang.String,java.lang.String> getSerializerConfigs()
Returns all the registered serializer configurations. This is a private API.
-
setArgument
@Nonnull public JobConfig setArgument(java.lang.String key, java.lang.Object value)
Associates the specified value with the specified key. The mapping will be available to the job while it's executing in the Jet cluster.- Parameters:
key
- key with which the specified value is to be associatedvalue
- value to be associated with the specified key- Returns:
this
instance for fluent API- Since:
- 5.0
-
getArgument
@Nullable public <T> T getArgument(java.lang.String key)
Returns the value to which the specified key is mapped, or null if there is no mapping for the key.- Parameters:
key
- the key whose associated value is to be returned- Since:
- 5.0
-
setClassLoaderFactory
@Nonnull public JobConfig setClassLoaderFactory(@Nullable JobClassLoaderFactory classLoaderFactory)
Sets a customJobClassLoaderFactory
that will be used to load job classes and resources on Jet members. Not supported for light jobs- Returns:
this
instance for fluent API
-
getClassLoaderFactory
@Nullable public JobClassLoaderFactory getClassLoaderFactory()
Returns the configuredJobClassLoaderFactory
.
-
getInitialSnapshotName
@Nullable public java.lang.String getInitialSnapshotName()
Returns the configured initial snapshot name ornull
if no initial snapshot is configured.
-
setInitialSnapshotName
@Nonnull public JobConfig setInitialSnapshotName(@Nullable java.lang.String initialSnapshotName)
Sets the exported state snapshot name to restore the initial job state from. This state will be used for initial state and also for the case when the execution restarts before it produces first snapshot.The job will use the state even if processing guarantee is set to
ProcessingGuarantee.NONE
.Cannot be used for light jobs.
- Parameters:
initialSnapshotName
- the snapshot name given toJob.exportSnapshot(String)
- Returns:
this
instance for fluent API- Since:
- Jet 3.0
-
isMetricsEnabled
public boolean isMetricsEnabled()
Returns if metrics collection is enabled for the job.- Since:
- Jet 3.2
-
setMetricsEnabled
@Nonnull public JobConfig setMetricsEnabled(boolean enabled)
Sets whether metrics collection should be enabled for the job. NeedsBaseMetricsConfig.isEnabled()
to be on in order to function.Metrics for running jobs can be queried using
Job.getMetrics()
It's enabled by default. Ignored for light jobs.- Since:
- Jet 3.2
-
isStoreMetricsAfterJobCompletion
public boolean isStoreMetricsAfterJobCompletion()
Returns whether metrics should be stored in the cluster after the job completes successfully. Needs bothBaseMetricsConfig.isEnabled()
andisMetricsEnabled()
to be on in order to function.If enabled, metrics can be retrieved by calling
Job.getMetrics()
.It's disabled by default.
- Since:
- Jet 3.2
-
setStoreMetricsAfterJobCompletion
public JobConfig setStoreMetricsAfterJobCompletion(boolean storeMetricsAfterJobCompletion)
Sets whether metrics should be stored in the cluster after the job completes. If enabled, metrics can be retrieved for the configured job after it has completed successfully and is no longer running by callingJob.getMetrics()
.If disabled, once the configured job stops running
Job.getMetrics()
will always return empty metrics for it, regardless of the settings forglobal metrics collection
orper job metrics collection
.It's disabled by default. Ignored for light jobs.
- Since:
- Jet 3.2
-
getMaxProcessorAccumulatedRecords
public long getMaxProcessorAccumulatedRecords()
Returns the maximum number of records that can be accumulated by any singleProcessor
instance in the context of the job.- Since:
- 5.0
-
setMaxProcessorAccumulatedRecords
public JobConfig setMaxProcessorAccumulatedRecords(long maxProcessorAccumulatedRecords)
Sets the maximum number of records that can be accumulated by any singleProcessor
instance in the context of the job.For more info see
JetConfig.setMaxProcessorAccumulatedRecords(long)
.If set, it has precedence over
JetConfig
's one.The default value is
-1
- in that caseJetConfig
's value is used.- Since:
- 5.0
-
getTimeoutMillis
public long getTimeoutMillis()
Returns maximum execution time for the job in milliseconds.- Since:
- 5.0
-
setTimeoutMillis
public JobConfig setTimeoutMillis(long timeoutMillis)
Sets the maximum execution time for the job in milliseconds. If the execution time (counted from the time job is submitted), exceeds this value, the job is forcefully cancelled. The default value is0
, which denotes no time limit on the execution of the job.- Since:
- 5.0
-
getFactoryId
public int getFactoryId()
Description copied from interface:IdentifiedDataSerializable
Returns DataSerializableFactory factory ID for this class.- Specified by:
getFactoryId
in interfaceIdentifiedDataSerializable
- Returns:
- factory ID
-
getClassId
public int getClassId()
Description copied from interface:IdentifiedDataSerializable
Returns type identifier for this class. It should be unique per DataSerializableFactory.- Specified by:
getClassId
in interfaceIdentifiedDataSerializable
- Returns:
- type ID
-
writeData
public void writeData(ObjectDataOutput out) throws java.io.IOException
Description copied from interface:DataSerializable
Writes object fields to output stream- Specified by:
writeData
in interfaceDataSerializable
- Parameters:
out
- output- Throws:
java.io.IOException
- if an I/O error occurs. In particular, anIOException
may be thrown if the output stream has been closed.
-
readData
public void readData(ObjectDataInput in) throws java.io.IOException
Description copied from interface:DataSerializable
Reads fields from the input stream- Specified by:
readData
in interfaceDataSerializable
- Parameters:
in
- input- Throws:
java.io.IOException
- if an I/O error occurs. In particular, anIOException
may be thrown if the input stream has been closed.
-
equals
public boolean equals(java.lang.Object o)
- Overrides:
equals
in classjava.lang.Object
-
hashCode
public int hashCode()
- Overrides:
hashCode
in classjava.lang.Object
-
toString
public java.lang.String toString()
- Overrides:
toString
in classjava.lang.Object
-
lock
@PrivateApi public void lock()
Used to prevent further mutations the config after submitting it with a job execution.It's not a public API, can be removed in the future.
-
-