com.hazelcast.jet.core

Class ProcessorMetaSupplier.SpecificMemberPms

java.lang.Object

com.hazelcast.jet.core.ProcessorMetaSupplier.SpecificMemberPms

All Implemented Interfaces:

ProcessorMetaSupplier, DataSerializable, IdentifiedDataSerializable, Serializable

Enclosing interface:

ProcessorMetaSupplier

public static class ProcessorMetaSupplier.SpecificMemberPms
extends Object
implements ProcessorMetaSupplier, IdentifiedDataSerializable

A meta-supplier that will only use the given ProcessorSupplier on a node with given Address.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface com.hazelcast.jet.core.ProcessorMetaSupplier

ProcessorMetaSupplier.Context, ProcessorMetaSupplier.ExpectNothingProcessorSupplier, ProcessorMetaSupplier.SpecificMemberPms

Method Summary

Modifier and Type	Method and Description
boolean	closeIsCooperative() Returns true if the ProcessorMetaSupplier.close(Throwable) method of this instance is cooperative.
Function<? super Address,? extends ProcessorSupplier>	get(List<Address> addresses) Called to create a mapping from member Address to the ProcessorSupplier that will be sent to that member.
int	getClassId() Returns type identifier for this class.
int	getFactoryId() Returns DataSerializableFactory factory ID for this class.
void	init(ProcessorMetaSupplier.Context context) Called on the cluster member that receives the job request.
boolean	initIsCooperative() Returns true if the ProcessorMetaSupplier.init(Context) method of this instance is cooperative.
boolean	isReusable() Returns true if the same instance can be reused in different job executions or in different vertices.
int	preferredLocalParallelism() Returns the local parallelism the vertex should be configured with.
void	readData(ObjectDataInput in) Reads fields from the input stream
void	writeData(ObjectDataOutput out) Writes object fields to output stream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.hazelcast.jet.core.ProcessorMetaSupplier

close, forceTotalParallelismOne, forceTotalParallelismOne, forceTotalParallelismOne, forceTotalParallelismOne, forceTotalParallelismOne, getRequiredPermission, getTags, of, of, of, of, of, of, of, of, preferLocalParallelismOne, preferLocalParallelismOne, preferLocalParallelismOne, preferLocalParallelismOne

Method Detail

init

public void init(@Nonnull
                 ProcessorMetaSupplier.Context context)
          throws Exception

Description copied from interface: ProcessorMetaSupplier

Called on the cluster member that receives the job request. Gives access to the Hazelcast instance's services and provides the parallelism parameters determined from the cluster size.

Specified by:

init in interface ProcessorMetaSupplier

Throws:

Exception

See Also:

ProcessorMetaSupplier.isReusable()

get

@Nonnull
public Function<? super Address,? extends ProcessorSupplier> get(@Nonnull
                                                                          List<Address> addresses)

Description copied from interface: ProcessorMetaSupplier

Called to create a mapping from member Address to the ProcessorSupplier that will be sent to that member. Jet calls this method with a list of all cluster members' addresses and the returned function must be a mapping that returns a non-null value for each given address.

The method will be called once per job execution on the job's coordinator member. init() will have already been called.

Specified by:

get in interface ProcessorMetaSupplier

See Also:

ProcessorMetaSupplier.isReusable()

preferredLocalParallelism

public int preferredLocalParallelism()

Description copied from interface: ProcessorMetaSupplier

Returns the local parallelism the vertex should be configured with. The default implementation returns Vertex.LOCAL_PARALLELISM_USE_DEFAULT.

Specified by:

preferredLocalParallelism in interface ProcessorMetaSupplier

isReusable

public boolean isReusable()

Description copied from interface: ProcessorMetaSupplier

Returns true if the same instance can be reused in different job executions or in different vertices. In that case, ProcessorMetaSupplier.init(com.hazelcast.jet.core.ProcessorMetaSupplier.Context), ProcessorMetaSupplier.get(java.util.List<com.hazelcast.cluster.Address>) and ProcessorMetaSupplier.close(java.lang.Throwable) methods must be thread-safe and obey additional conditions defined below.

When a job is submitted from a client, the job definition is serialized, so the job coordinator will receive a different copy of processor meta-suppliers even if they are used multiple times within the same DAG, or across different DAGs, or submitted through different jobs. While this serialization mechanism ensures that processor meta-supplier instances do not share any internal state, it is unnecessary —and avoided— for jobs consisting of reusable meta-suppliers and submitted from the job coordinator —which is always the case for member-originated light jobs.

Non-reusable meta-suppliers (default) have a simple order of method executions: ProcessorMetaSupplier.init(com.hazelcast.jet.core.ProcessorMetaSupplier.Context) (once), ProcessorMetaSupplier.get(java.util.List<com.hazelcast.cluster.Address>) (at most once after ProcessorMetaSupplier.init(com.hazelcast.jet.core.ProcessorMetaSupplier.Context), ProcessorMetaSupplier.close(java.lang.Throwable) (last).

Reusable meta-suppliers differ because the meta supplier instance may be shared and reused. That is why:

1. ProcessorMetaSupplier.init(com.hazelcast.jet.core.ProcessorMetaSupplier.Context) can be invoked multiple times (also after ProcessorMetaSupplier.close(java.lang.Throwable)).
2. ProcessorMetaSupplier.get(java.util.List<com.hazelcast.cluster.Address>) can be invoked multiple times, but each ProcessorMetaSupplier.get(java.util.List<com.hazelcast.cluster.Address>) invocation will be preceded by ProcessorMetaSupplier.init(com.hazelcast.jet.core.ProcessorMetaSupplier.Context) invocation for given job execution.
3. ProcessorMetaSupplier.close(java.lang.Throwable) can be invoked multiple times with or without preceding invocations of the other methods.
4. Meta-supplier method invocation sequences for different concurrent job executions may be interleaved.

It is recommended that reusable meta-supplier does not have any mutable state that is changed by any of the methods. It is, however, allowed to initialize some thread-safe constant fields in ProcessorMetaSupplier.init(com.hazelcast.jet.core.ProcessorMetaSupplier.Context) method, for example, save some constant data from ProcessorMetaSupplier.Context for further usage. Note, however, that cluster topology may change and should not be stored.

Specified by:

isReusable in interface ProcessorMetaSupplier

initIsCooperative

public boolean initIsCooperative()

Description copied from interface: ProcessorMetaSupplier

Returns true if the ProcessorMetaSupplier.init(Context) method of this instance is cooperative. If it's not, the call to the init() method is off-loaded to another thread.

Specified by:

initIsCooperative in interface ProcessorMetaSupplier

closeIsCooperative

public boolean closeIsCooperative()

Description copied from interface: ProcessorMetaSupplier

Returns true if the ProcessorMetaSupplier.close(Throwable) method of this instance is cooperative. If it's not, the call to the close() method is off-loaded to another thread.

Specified by:

closeIsCooperative in interface ProcessorMetaSupplier

writeData

public void writeData(ObjectDataOutput out)
               throws IOException

Description copied from interface: DataSerializable

Writes object fields to output stream

Specified by:

writeData in interface DataSerializable

Parameters:

out - output

Throws:

IOException - if an I/O error occurs. In particular, an IOException may be thrown if the output stream has been closed.

readData

public void readData(ObjectDataInput in)
              throws IOException

Description copied from interface: DataSerializable

Reads fields from the input stream

Specified by:

readData in interface DataSerializable

Parameters:

in - input

Throws:

IOException - if an I/O error occurs. In particular, an IOException may be thrown if the input stream has been closed.

getFactoryId

public int getFactoryId()

Description copied from interface: IdentifiedDataSerializable

Returns DataSerializableFactory factory ID for this class.

Specified by:

getFactoryId in interface IdentifiedDataSerializable

Returns:

factory ID

getClassId

public int getClassId()

Description copied from interface: IdentifiedDataSerializable

Returns type identifier for this class. It should be unique per DataSerializableFactory.

Specified by:

getClassId in interface IdentifiedDataSerializable

Returns:

type ID