com.hazelcast.mapreduce

Interface Job<KeyIn,ValueIn>

Type Parameters:

KeyIn - type of key used as input key type

ValueIn - type of value used as input value type

All Known Implementing Classes:

AbstractJob, KeyValueJob

@Beta
public interface Job<KeyIn,ValueIn>

This interface describes a mapreduce Job that is build by JobTracker.newJob(KeyValueSource).
It is used to execute mappings and calculations on the different cluster nodes and reduce or collate these mapped values to results.

Implementations returned by the JobTracker are fully threadsafe and can be used concurrently and multiple times once the configuration is finished.

Caution: The generic types of Jobs change depending on the used methods which can make it necessary to use different assignment variables when used over multiple source lines.

An example on how to use it:

 HazelcastInstance hz = getHazelcastInstance();
 IMap<Integer, Integer> map = (...) hz.getMap( "default" );
 JobTracker tracker = hz.getJobTracker( "default" );
 Job<Integer, Integer> job = tracker.newJob( KeyValueSource.fromMap( map ) );
 ICompletableFuture<Map<String, Integer>> future = job
      .mapper( buildMapper() ).reducer( buildReducer() ).submit();
 Map<String, Integer> results = future.get();
 

Since:

3.2

Method Summary

Methods

Modifier and Type	Method and Description
Job<KeyIn,ValueIn>	chunkSize(int chunkSize) Defines the number of elements per chunk.
Job<KeyIn,ValueIn>	keyPredicate(KeyPredicate<KeyIn> predicate) Defines the KeyPredicate implementation to preselect keys the MapReduce task will be executed on.
<KeyOut,ValueOut> MappingJob<KeyIn,KeyOut,ValueOut>	mapper(Mapper<KeyIn,ValueIn,KeyOut,ValueOut> mapper) Defines the mapper for this task.
Job<KeyIn,ValueIn>	onKeys(Iterable<KeyIn> keys) Defines keys to execute the mapper and a possibly defined reducer against.
Job<KeyIn,ValueIn>	onKeys(KeyIn... keys) Defines keys to execute the mapper and a possibly defined reducer against.
Job<KeyIn,ValueIn>	topologyChangedStrategy(TopologyChangedStrategy topologyChangedStrategy) Defines the strategy to handle topology changes while executing the map reduce job.

Method Detail

onKeys

Job<KeyIn,ValueIn> onKeys(Iterable<KeyIn> keys)

Defines keys to execute the mapper and a possibly defined reducer against. If keys are known before submitting the task setting them can improve execution speed.

Parameters:

keys - keys to be executed against

Returns:

instance of this Job with generics changed on usage

onKeys

Job<KeyIn,ValueIn> onKeys(KeyIn... keys)

Defines keys to execute the mapper and a possibly defined reducer against. If keys are known before submitting the task setting them can improve execution speed.

Parameters:

keys - keys to be executed against

Returns:

instance of this Job with generics changed on usage

chunkSize

Job<KeyIn,ValueIn> chunkSize(int chunkSize)

Defines the number of elements per chunk. Whenever the chunk size is reached and a ReducerFactory is defined the chunk will be send to the nodes that is responsible for the emitted keys.
Please note, that chunks are deactivated when no ReducerFactory is defined

Parameters:

chunkSize - the number of elements per chunk

Returns:

instance of this Job with generics changed on usage

topologyChangedStrategy

Job<KeyIn,ValueIn> topologyChangedStrategy(TopologyChangedStrategy topologyChangedStrategy)

Defines the strategy to handle topology changes while executing the map reduce job. For further information see TopologyChangedStrategy.

Parameters:

topologyChangedStrategy - strategy to use

Returns:

instance of this Job with generics changed on usage

keyPredicate

Job<KeyIn,ValueIn> keyPredicate(KeyPredicate<KeyIn> predicate)

Defines the KeyPredicate implementation to preselect keys the MapReduce task will be executed on. Preselecting keys can speed up the job massively.
This method can be used in conjunction with onKeys(Iterable) or onKeys(Object...) to define a range of known and evaluated keys.

Parameters:

predicate - predicate implementation to be used to evaluate keys

Returns:

instance of this Job with generics changed on usage

mapper

<KeyOut,ValueOut> MappingJob<KeyIn,KeyOut,ValueOut> mapper(Mapper<KeyIn,ValueIn,KeyOut,ValueOut> mapper)

Defines the mapper for this task. This method is not idempotent and can be callable only one time. Further calls result in an IllegalStateException to be thrown telling you to not change the internal state.

Type Parameters:

KeyOut - type of the emitted key

ValueOut - type of the emitted value

Parameters:

mapper - tasks mapper

Returns:

instance of this Job with generics changed on usage