ValueIn
- value type of the incoming valuesValueOut
- value type of the reduced values@Beta public abstract class Combiner<ValueIn,ValueOut> extends Object
The abstract Combiner class is used to build combiners for the Job
.
Those Combiners are distributed inside of the cluster and are running alongside
the Mapper
implementations in the same node.
Combiners are called in a threadsafe way so internal locking is not required.
Combiners are normally used to build intermediate results on the mapping nodes to
lower the traffic overhead between the different nodes before the reducing phase.
Combiners need to be capable of combining data in multiple chunks to create a more
streaming like internal behavior.
A simple Combiner implementation in combination with a Reducer
could look
like this avg-function implementation.
public class AvgCombiner implements Combiner<Integer, Tuple<Long, Long>> { private long count; private long amount; public void combine(Integer value) { count++; amount += value; } public Tuple<Long, Long> finalizeChunk() { Tuple<Long, Long> tuple = new Tuple<>( count, amount ); count = 0; amount = 0; return tuple; } } public class SumReducer implements Reducer<Tuple<Long, Long>, Integer> { private long count; private long amount; public void reduce( Tuple<Long, Long> value ) { count += value.getFirst(); amount += value.getSecond(); } public Integer finalizeReduce() { return amount / count; } }
Constructor and Description |
---|
Combiner() |
Modifier and Type | Method and Description |
---|---|
void |
beginCombine()
This method is called before the first value is submitted to this Combiner instance.
|
abstract void |
combine(ValueIn value)
This method is called to supply values to be combined into an intermediate result chunk.
The combine method might be called multiple times so the combined chunk needs to be held internally in a member state of the Combiner. After this method is called you need to reset() the internal state to prepare for combining of
the next chunk. |
abstract ValueOut |
finalizeChunk()
Creates a chunk of
ValueOut to be sent to the Reducer for the according
key. |
void |
finalizeCombine()
This method is called after the mapping phase on the local node is over.
|
void |
reset()
This method is always called after a chunk of data is retrieved.
|
public void beginCombine()
public abstract void combine(ValueIn value)
reset()
the internal state to prepare for combining of
the next chunk.value
- value to be reduced (combined into an intermediate result chunk)public abstract ValueOut finalizeChunk()
ValueOut
to be sent to the Reducer
for the according
key.public void reset()
finalizeChunk()
, as with the last version of the API.public void finalizeCombine()
Copyright © 2016 Hazelcast, Inc.. All Rights Reserved.