Class DAG
- All Implemented Interfaces:
DataSerializable
,IdentifiedDataSerializable
,Iterable<Vertex>
vertex
represents a unit of data processing and an
edge
represents the path along which the data travels to
the next vertex.
The work of a single vertex is parallelized and distributed, so that there are
several instances of the Processor
type on each member
corresponding to it. Whenever possible, each instance should be
tasked with only a slice of the total data and a partitioning strategy
can be employed to ensure that the data sent to each vertex is collated
by a partitioning key.
There are three basic kinds of vertices:
- source with just outbound edges;
- processor with both inbound and outbound edges;
- sink with just inbound edges.
Note that {@link #iterator()) must be invoked at least once in order to validate the DAG and check against cycles.
- Since:
- Jet 3.0
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionAdds an edge to this DAG.Returns an iterator over the DAG's edges in unspecified order.int
Returns type identifier for this class.int
Returns DataSerializableFactory factory ID for this class.getInboundEdges
(String vertexName) Returns the inbound edges connected to the vertex with the given name.getOutboundEdges
(String vertexName) Returns the outbound edges connected to the vertex with the given name.Returns the vertex with the given name, ornull
if there is no vertex with that name.iterator()
Validates the DAG and returns an iterator over the DAG's vertices in topological order.void
lock()
Used to prevent further mutations to the DAG after submitting it for execution.newUniqueVertex
(String namePrefix, SupplierEx<? extends Processor> simpleSupplier) Creates a vertex from aSupplier<Processor>
and adds it to this DAG.newUniqueVertex
(String namePrefix, ProcessorMetaSupplier metaSupplier) Creates a vertex from aProcessorMetaSupplier
and adds it to this DAG.newUniqueVertex
(String namePrefix, ProcessorSupplier processorSupplier) Creates a vertex from aProcessorSupplier
and adds it to this DAG.newVertex
(String name, SupplierEx<? extends Processor> simpleSupplier) Creates a vertex from aSupplier<Processor>
and adds it to this DAG.newVertex
(String name, ProcessorMetaSupplier metaSupplier) Creates a vertex from aProcessorMetaSupplier
and adds it to this DAG.newVertex
(String name, ProcessorSupplier processorSupplier) Creates a vertex from aProcessorSupplier
and adds it to this DAG.void
Reads fields from the input streamReturns a DOT format (graphviz) representation of the DAG.toDotString
(int defaultLocalParallelism, int defaultQueueSize) Returns a DOT format (graphviz) representation of the DAG, annotates the vertices using supplied default parallelism value, and the edges using supplied default queue size value.com.hazelcast.internal.json.JsonObject
toJson
(int defaultLocalParallelism) Returns a JSON representation of the DAG.toString()
toString
(int defaultLocalParallelism) Returns a string representation of the DAG.Adds a vertex to this DAG.vertices()
Returns a copy of the DAG's vertices.void
Writes object fields to output streamMethods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface java.lang.Iterable
forEach, spliterator
-
Constructor Details
-
DAG
public DAG()
-
-
Method Details
-
newVertex
@Nonnull public Vertex newVertex(@Nonnull String name, @Nonnull SupplierEx<? extends Processor> simpleSupplier) Creates a vertex from aSupplier<Processor>
and adds it to this DAG.- Parameters:
name
- the unique name of the vertexsimpleSupplier
- the simple, parameterless supplier ofProcessor
instances- See Also:
-
newUniqueVertex
@Nonnull public Vertex newUniqueVertex(@Nonnull String namePrefix, @Nonnull SupplierEx<? extends Processor> simpleSupplier) Creates a vertex from aSupplier<Processor>
and adds it to this DAG. The vertex will be given a unique name created from thenamePrefix
.- Parameters:
namePrefix
- the prefix for unique name of the vertexsimpleSupplier
- the simple, parameterless supplier ofProcessor
instances- Since:
- Jet 4.4
- See Also:
-
newVertex
@Nonnull public Vertex newVertex(@Nonnull String name, @Nonnull ProcessorSupplier processorSupplier) Creates a vertex from aProcessorSupplier
and adds it to this DAG.- Parameters:
name
- the unique name of the vertexprocessorSupplier
- the supplier ofProcessor
instances which will be used on all members- See Also:
-
newUniqueVertex
@Nonnull public Vertex newUniqueVertex(@Nonnull String namePrefix, @Nonnull ProcessorSupplier processorSupplier) Creates a vertex from aProcessorSupplier
and adds it to this DAG. The vertex will be given a unique name created from thenamePrefix
.- Parameters:
namePrefix
- the prefix for unique name of the vertexprocessorSupplier
- the supplier ofProcessor
instances which will be used on all members- Since:
- Jet 4.4
- See Also:
-
newVertex
Creates a vertex from aProcessorMetaSupplier
and adds it to this DAG.- Parameters:
name
- the unique name of the vertexmetaSupplier
- the meta-supplier ofProcessorSupplier
s for each member- See Also:
-
newUniqueVertex
@Nonnull public Vertex newUniqueVertex(@Nonnull String namePrefix, @Nonnull ProcessorMetaSupplier metaSupplier) Creates a vertex from aProcessorMetaSupplier
and adds it to this DAG. The vertex will be given a unique name created from thenamePrefix
.- Parameters:
namePrefix
- the prefix for unique name of the vertexmetaSupplier
- the meta-supplier ofProcessorSupplier
s for each member- Since:
- Jet 4.4
- See Also:
-
vertex
Adds a vertex to this DAG. The vertex name must be unique. -
edge
Adds an edge to this DAG. The vertices it connects must already be present in the DAG. It is an error to connect an edge to a vertex at the same ordinal as another existing edge. However, inbound and outbound ordinals are independent, so there can be two edges at the same ordinal, one inbound and one outbound.Jet supports multigraphs, that is you can add two edges between the same two vertices. However, they have to have different ordinals.
-
getInboundEdges
Returns the inbound edges connected to the vertex with the given name. -
getOutboundEdges
Returns the outbound edges connected to the vertex with the given name. -
getVertex
Returns the vertex with the given name, ornull
if there is no vertex with that name. -
vertices
Returns a copy of the DAG's vertices. Adding a vertex to or removing a vertex from the returnedSet
will not be reflected in the DAG, and vice-versa. -
iterator
Validates the DAG and returns an iterator over the DAG's vertices in topological order.Note that this method must be invoked at least once in order to validate the DAG and check against cycles.
-
edgeIterator
Returns an iterator over the DAG's edges in unspecified order.- Since:
- 5.4
-
toString
-
toString
Returns a string representation of the DAG.- Parameters:
defaultLocalParallelism
- the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplier
-
toJson
@Nonnull public com.hazelcast.internal.json.JsonObject toJson(int defaultLocalParallelism) Returns a JSON representation of the DAG.Note: the exact structure of the JSON is unspecified.
- Parameters:
defaultLocalParallelism
- the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplier
-
toDotString
Returns a DOT format (graphviz) representation of the DAG. -
toDotString
Returns a DOT format (graphviz) representation of the DAG, annotates the vertices using supplied default parallelism value, and the edges using supplied default queue size value.- Parameters:
defaultLocalParallelism
- the local parallelism that will be shown if neither overridden on the vertex nor the preferred parallelism is defined by meta-supplierdefaultQueueSize
- the queue size that will be shown if not overridden on the edge
-
writeData
Description copied from interface:DataSerializable
Writes object fields to output stream- Specified by:
writeData
in interfaceDataSerializable
- Parameters:
out
- output- Throws:
IOException
- if an I/O error occurs. In particular, anIOException
may be thrown if the output stream has been closed.
-
readData
Description copied from interface:DataSerializable
Reads fields from the input stream- Specified by:
readData
in interfaceDataSerializable
- Parameters:
in
- input- Throws:
IOException
- if an I/O error occurs. In particular, anIOException
may be thrown if the input stream has been closed.
-
getFactoryId
public int getFactoryId()Description copied from interface:IdentifiedDataSerializable
Returns DataSerializableFactory factory ID for this class.- Specified by:
getFactoryId
in interfaceIdentifiedDataSerializable
- Returns:
- factory ID
-
getClassId
public int getClassId()Description copied from interface:IdentifiedDataSerializable
Returns type identifier for this class. It should be unique per DataSerializableFactory.- Specified by:
getClassId
in interfaceIdentifiedDataSerializable
- Returns:
- type ID
-
lock
Used to prevent further mutations to the DAG after submitting it for execution.It's not a public API, can be removed in the future.
-