com.hazelcast.mapreduce.impl

Class AbstractJob<KeyIn,ValueIn>

java.lang.Object

com.hazelcast.mapreduce.impl.AbstractJob<KeyIn,ValueIn>

Type Parameters:

KeyIn - type of the input key

ValueIn - type of the input value

All Implemented Interfaces:

Job<KeyIn,ValueIn>

Direct Known Subclasses:

KeyValueJob

public abstract class AbstractJob<KeyIn,ValueIn>
extends Object
implements Job<KeyIn,ValueIn>

Base class for all map reduce job implementations

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	AbstractJob.MappingJobImpl<EntryKey,Key,Value> This class is just used to comply to the public DSL style API
protected class	AbstractJob.ReducingJobImpl<EntryKey,Key,Value> This class is just used to comply to the public DSL style API
protected class	AbstractJob.ReducingSubmittableJobImpl<EntryKey,Key,Value> This class is just used to comply to the public DSL style API

Field Summary

Fields

Modifier and Type	Field and Description
protected int	chunkSize
protected CombinerFactory<?,?,?>	combinerFactory
protected JobTracker	jobTracker
protected Collection<KeyIn>	keys
protected KeyValueSource<KeyIn,ValueIn>	keyValueSource
protected Mapper<KeyIn,ValueIn,?,?>	mapper
protected String	name
protected KeyPredicate<KeyIn>	predicate
protected ReducerFactory<?,?,?>	reducerFactory
protected TopologyChangedStrategy	topologyChangedStrategy

Constructor Summary

Constructors

Constructor and Description
AbstractJob(String name, JobTracker jobTracker, KeyValueSource<KeyIn,ValueIn> keyValueSource)

Method Summary

Methods

Modifier and Type	Method and Description
Job<KeyIn,ValueIn>	chunkSize(int chunkSize) Defines the number of elements per chunk.
protected abstract <T> JobCompletableFuture<T>	invoke(Collator collator)
Job<KeyIn,ValueIn>	keyPredicate(KeyPredicate<KeyIn> predicate) Defines the KeyPredicate implementation to preselect keys the MapReduce task will be executed on.
<KeyOut,ValueOut> MappingJob<KeyIn,KeyOut,ValueOut>	mapper(Mapper<KeyIn,ValueIn,KeyOut,ValueOut> mapper) Defines the mapper for this task.
Job<KeyIn,ValueIn>	onKeys(Iterable<KeyIn> keys) Defines keys to execute the mapper and a possibly defined reducer against.
Job<KeyIn,ValueIn>	onKeys(KeyIn... keys) Defines keys to execute the mapper and a possibly defined reducer against.
protected void	prepareKeyPredicate()
protected <T> JobCompletableFuture<T>	submit(Collator collator)
Job<KeyIn,ValueIn>	topologyChangedStrategy(TopologyChangedStrategy topologyChangedStrategy) Defines the strategy to handle topology changes while executing the map reduce job.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

name

protected final String name

jobTracker

protected final JobTracker jobTracker

keyValueSource

protected final KeyValueSource<KeyIn,ValueIn> keyValueSource

mapper

protected Mapper<KeyIn,ValueIn,?,?> mapper

combinerFactory

protected CombinerFactory<?,?,?> combinerFactory

reducerFactory

protected ReducerFactory<?,?,?> reducerFactory

keys

protected Collection<KeyIn> keys

predicate

protected KeyPredicate<KeyIn> predicate

chunkSize

protected int chunkSize

topologyChangedStrategy

protected TopologyChangedStrategy topologyChangedStrategy

Constructor Detail

AbstractJob

public AbstractJob(String name,
           JobTracker jobTracker,
           KeyValueSource<KeyIn,ValueIn> keyValueSource)

Method Detail

mapper

public <KeyOut,ValueOut> MappingJob<KeyIn,KeyOut,ValueOut> mapper(Mapper<KeyIn,ValueIn,KeyOut,ValueOut> mapper)

Description copied from interface: Job

Defines the mapper for this task. This method is not idempotent and can be callable only one time. Further calls result in an IllegalStateException to be thrown telling you to not change the internal state.

Specified by:

mapper in interface Job<KeyIn,ValueIn>

Type Parameters:

KeyOut - type of the emitted key

ValueOut - type of the emitted value

Parameters:

mapper - tasks mapper

Returns:

instance of this Job with generics changed on usage

onKeys

public Job<KeyIn,ValueIn> onKeys(Iterable<KeyIn> keys)

Description copied from interface: Job

Defines keys to execute the mapper and a possibly defined reducer against. If keys are known before submitting the task, setting them can improve execution speed.

Specified by:

onKeys in interface Job<KeyIn,ValueIn>

Parameters:

keys - keys to be executed against

Returns:

instance of this Job with generics changed on usage

onKeys

public Job<KeyIn,ValueIn> onKeys(KeyIn... keys)

Description copied from interface: Job

Defines keys to execute the mapper and a possibly defined reducer against. If keys are known before submitting the task, setting them can improve execution speed.

Specified by:

onKeys in interface Job<KeyIn,ValueIn>

Parameters:

keys - keys to be executed against

Returns:

instance of this Job with generics changed on usage

keyPredicate

public Job<KeyIn,ValueIn> keyPredicate(KeyPredicate<KeyIn> predicate)

Description copied from interface: Job

Defines the KeyPredicate implementation to preselect keys the MapReduce task will be executed on. Preselecting keys can speed up the job massively.
This method can be used in conjunction with Job.onKeys(Iterable) or Job.onKeys(Object...) to define a range of known and evaluated keys.

Specified by:

keyPredicate in interface Job<KeyIn,ValueIn>

Parameters:

predicate - predicate implementation to be used to evaluate keys

Returns:

instance of this Job with generics changed on usage

chunkSize

public Job<KeyIn,ValueIn> chunkSize(int chunkSize)

Description copied from interface: Job

Defines the number of elements per chunk. Whenever the chunk size is reached and a ReducerFactory is defined, the chunk will be sent to the nodes that is responsible for the emitted keys.
Please note, that chunks are deactivated when no ReducerFactory is defined

Specified by:

chunkSize in interface Job<KeyIn,ValueIn>

Parameters:

chunkSize - the number of elements per chunk

Returns:

instance of this Job with generics changed on usage

topologyChangedStrategy

public Job<KeyIn,ValueIn> topologyChangedStrategy(TopologyChangedStrategy topologyChangedStrategy)

Description copied from interface: Job

Defines the strategy to handle topology changes while executing the map reduce job. For further information see TopologyChangedStrategy.

Specified by:

topologyChangedStrategy in interface Job<KeyIn,ValueIn>

Parameters:

topologyChangedStrategy - strategy to handle topology changes while executing the map reduce job

Returns:

instance of this Job with generics changed on usage

submit

protected <T> JobCompletableFuture<T> submit(Collator collator)

invoke

protected abstract <T> JobCompletableFuture<T> invoke(Collator collator)

prepareKeyPredicate

protected void prepareKeyPredicate()