public interface Processor
Processor
on each cluster member to do the work of a given vertex. The
vertex's localParallelism
property controls the number of
processors per member.
The processor is a single-threaded processing unit that performs the computation needed to transform zero or more input data streams into zero or more output streams. Each input/output stream corresponds to an edge on the vertex. The correspondence between a stream and an edge is established via the edge's ordinal.
The special case of zero input streams applies to a source vertex, which gets its data from the environment. The special case of zero output streams applies to a sink vertex, which pushes its data to the environment.
The processor accepts input from instances of Inbox
and pushes
its output to an instance of Outbox
.
See the isCooperative()
for important restrictions to how the
processor should work.
Modifier and Type | Interface and Description |
---|---|
static interface |
Processor.Context
Context passed to the processor in the
init() call. |
Modifier and Type | Method and Description |
---|---|
default boolean |
complete()
Called after all the inbound edges' streams are exhausted.
|
default boolean |
completeEdge(int ordinal)
Called after the edge input with the supplied
ordinal is
exhausted. |
default boolean |
finishSnapshotRestore()
Called after a job was restarted from a snapshot and the processor
has consumed all the snapshot data.
|
default void |
init(Outbox outbox,
Processor.Context context)
Initializes this processor with the outbox that the processing methods
must use to deposit their output items.
|
default boolean |
isCooperative()
Tells whether this processor is able to participate in cooperative
multithreading.
|
default void |
process(int ordinal,
Inbox inbox)
Called with a batch of items retrieved from an inbound edge's stream.
|
default void |
restoreFromSnapshot(Inbox inbox)
Called when a batch of items is received during the "restore from
snapshot" operation.
|
default boolean |
saveToSnapshot()
Stores its snapshotted state by adding items to the outbox's
snapshot bucket . |
default boolean |
tryProcess()
Called when there is no pending data in the inbox.
|
default boolean isCooperative()
A cooperative processor should also not attempt any blocking operations,
such as I/O operations, waiting for locks/semaphores or sleep
operations. Violations to this rule will manifest themselves as less
than 100% CPU usage under maximum load. The processor should also return
as soon as an item is rejected by the outbox (that is when the offer()
method returns false
).
If this processor declares itself cooperative, it will share a thread with other cooperative processors. Otherwise it will run in a dedicated Java thread.
Jet prefers cooperative processors because they result in greater overall throughput. A processor should be non-cooperative only if it involves blocking operations, which would cause all other processors on the same shared thread to starve.
Processor instances on single vertex are allowed to return different value, but single processor instance must return constant value.
The default implementation returns true
.
default void init(@Nonnull Outbox outbox, @Nonnull Processor.Context context)
process(int, Inbox)
and complete()
).
The default implementation does nothing.
default void process(int ordinal, @Nonnull Inbox inbox)
If the method returns with items still present in the inbox, it will be called again before proceeding to call any other methods. There is at least one item in the inbox when this method is called.
The default implementation does nothing.
ordinal
- ordinal of the inbound edgeinbox
- the inbox containing the pending itemsdefault boolean tryProcess()
false
,
it will be called again before proceeding to call any other method.default boolean completeEdge(int ordinal)
ordinal
is
exhausted. If it returns false
, it will be called again before
proceeding to call any other method.true
if the processor is now done completing the edge,
false
otherwise.default boolean complete()
false
, it will be invoked again until it returns true
.
After this method is called, no other processing methods will be called on
this processor, except for saveToSnapshot()
.
Non-cooperative processors are required to return from this method from time to time to give the system a chance to check for snapshot requests and job cancellation. The time the processor spends in this method affects the latency of snapshots and job cancellations.
true
if the completing step is now done, false
otherwise.default boolean saveToSnapshot()
snapshot bucket
. If it returns
false
, it will be called again before proceeding to call any
other method.
This method will only be called after a call to process()
returns and the inbox is empty. After all the input is
exhausted, it may also be called between complete()
calls. Once
complete()
returns true
, this method won't be called
anymore.
The default implementation takes no action and returns true
.
default void restoreFromSnapshot(@Nonnull Inbox inbox)
Map.Entry
. May emit items to the outbox.
If it returns with items still present in the inbox, it will be called again before proceeding to call any other methods. It is never called with an empty inbox.
The default implementation throws an exception.
default boolean finishSnapshotRestore()
If it returns false
, it will be called again before proceeding
to call any other methods.
The default implementation takes no action and returns true
.
Copyright © 2017 Hazelcast, Inc.. All Rights Reserved.