com.hazelcast.mapreduce
Class Combiner<KeyIn,ValueIn,ValueOut>

java.lang.Object
  com.hazelcast.mapreduce.Combiner<KeyIn,ValueIn,ValueOut>

Type Parameters:

KeyIn - key type of the resulting keys

ValueIn - value type of the incoming values

ValueOut - value type of the reduced values

@Beta
public abstract class Combiner<KeyIn,ValueIn,ValueOut>
extends Object

The abstract Combiner class is used to build combiners for the Job.
Those Combiners are distributed inside of the cluster and are running alongside the Mapper implementations in the same node.
Combiners are called in a threadsafe way so internal locking is not required.

Combiners are normally used to build intermediate results on the mapping nodes to lower the traffic overhead between the different nodes before the reducing phase.
Combiners need to be capable of combining data in multiple chunks to create a more streaming like internal behavior.

A simple Combiner implementation in combination with a Reducer could look like that avg-function implementation:

 public class AvgCombiner implements Combiner<String, Integer, Tuple<Long, Long>>
 {
   private long count;
   private long amount;
   public void combine(String key, Integer value)
   {
     count++;
     amount += value;
   }

   public Tuple<Long, Long> finalizeChunk()
   {
     Tuple<Long, Long> tuple = new Tuple<>(count, amount);
     count = 0;
     amount = 0;
     return tuple;
   }
 }

 public class SumReducer implements Reducer<String, Tuple<Long, Long>, Integer>
 {
   private long count;
   private long amount;
   public void reduce( String key, Tuple<Long, Long> value )
   {
     count += value.getFirst();
     amount += value.getSecond();
   }

   public Integer finalizeReduce()
   {
     return amount / count;
   }
 }
 

Since:

3.2

Constructor Summary

Combiner()

Method Summary

void	beginCombine() This method is called before the first value is submitted to this Combiner instance.
abstract void	combine(KeyIn key, ValueIn value) This method is called to supply values to be combined into an intermediate result chunk. The combine method might be called multiple times so the combined chunk needs to be hold internally in a member state of the Combiner. After this method is called you need to reset the internal state to prepare combining of the next chunk.
abstract ValueOut	finalizeChunk() Creates a chunk of ValueOut to be send to the Reducer for the according key.
void	finalizeCombine() This method is called after mapping phase is over.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Combiner

public Combiner()

Method Detail

beginCombine

public void beginCombine()

This method is called before the first value is submitted to this Combiner instance. It can be used to setup any internal needed state before starting to combining the actual values.
The method is called only one time and is not called again before starting a new chunk.

combine

public abstract void combine(KeyIn key,
                             ValueIn value)

This method is called to supply values to be combined into an intermediate result chunk.
The combine method might be called multiple times so the combined chunk needs to be hold internally in a member state of the Combiner.
After this method is called you need to reset the internal state to prepare combining of the next chunk.

Parameters:

key - key of the mapped values

value - value to be reduced

finalizeChunk

public abstract ValueOut finalizeChunk()

Creates a chunk of ValueOut to be send to the Reducer for the according key.

Returns:

chunk of intermediate data

finalizeCombine

public void finalizeCombine()

This method is called after mapping phase is over. It is intended to be overridden to cleanup internal state and free possible resources.